URL Encoding in curl — --data-urlencode, -G, and Query String Tips

curl doesn’t automatically encode URLs you pass in — you must encode them manually or use curl’s built-in encoding options. Using the wrong approach causes malformed requests.

Encode and decode URL components with the URL Encoder.

Problem: spaces and special characters in URLs

# ❌ Wrong: space in URL breaks the request
curl "https://api.example.com/search?q=hello world"
# curl: (3) URL using bad/illegal format or missing URL

# ✅ Manually encode:
curl "https://api.example.com/search?q=hello%20world"

# ✅ Or use curl's --get and --data-urlencode for query params:
curl --get --data-urlencode "q=hello world" "https://api.example.com/search"
# Automatically encodes the value

—data-urlencode for GET query parameters

# Single parameter:
curl --get --data-urlencode "query=hello world & things" https://api.example.com/search

# Multiple parameters:
curl --get \
  --data-urlencode "q=hello world" \
  --data-urlencode "lang=en" \
  --data-urlencode "page=1" \
  https://api.example.com/search
# GET https://api.example.com/search?q=hello+world&lang=en&page=1

# Short form: -G and -d:
curl -G -d "q=hello world" -d "page=1" https://api.example.com/search

—data-urlencode for POST requests

# Encode POST form data:
curl -X POST \
  --data-urlencode "email=alice@example.com" \
  --data-urlencode "name=Alice Johnson" \
  --data-urlencode "message=Hello! This is a message with special chars: &, =, +" \
  https://api.example.com/contact

# Formats:
# --data-urlencode "name=value"    → URL-encodes value, uses "name" as key
# --data-urlencode "=value"        → URL-encodes entire string (key=value)
# --data-urlencode "name@filename" → URL-encodes content of file
# --data-urlencode "@filename"     → URL-encodes entire file content

Encode URLs in bash scripts

# Using Python to URL-encode a string (most portable):
urlencode() {
  python3 -c "import urllib.parse; print(urllib.parse.quote('$1'))"
}

search_term="hello world & things"
encoded=$(urlencode "$search_term")
curl "https://api.example.com/search?q=${encoded}"

# Using jq to build JSON with special characters (alternative to URL encoding):
curl -X POST https://api.example.com/search \
  -H "Content-Type: application/json" \
  -d "$(jq -n --arg q "hello world & things" '{"q": $q}')"

# Using sed for basic encoding (limited — avoid for production):
urlencode_basic() {
  echo "$1" | sed 's/ /%20/g; s/&/%26/g; s/=/%3D/g; s/+/%2B/g'
}

# Using xxd/printf for full encoding (more complete):
urlencode_full() {
  local string="${1}"
  local length="${#string}"
  local encoded=""
  local pos char ord
  for (( pos=0 ; pos<length ; pos++ )); do
    char="${string:$pos:1}"
    case "$char" in
      [-_.~a-zA-Z0-9]) encoded+="$char" ;;
      *) printf -v ord '%d' "'$char"
         printf -v enc '%%%02X' "$ord"
         encoded+="$enc" ;;
    esac
  done
  echo "$encoded"
}

Common curl URL encoding scenarios

# API key with special characters in header:
curl -H "Authorization: Bearer my+token/with=special&chars" https://api.example.com/data
# Don't encode headers — they use different rules

# Path segment with spaces:
curl "https://api.example.com/files/$(python3 -c "import urllib.parse; print(urllib.parse.quote('my file name.pdf'))")"
# → https://api.example.com/files/my%20file%20name.pdf

# JSON POST (no URL encoding needed — use Content-Type: application/json):
curl -X POST https://api.example.com/users \
  -H "Content-Type: application/json" \
  -d '{"name": "Alice & Bob", "email": "alice+bob@example.com"}'
# JSON handles special characters without URL encoding

# Webhook with URL-encoded payload:
curl -X POST https://api.example.com/webhook \
  --data-urlencode "payload={\"event\": \"click\", \"user\": \"alice\"}"

Double encoding pitfall

# ❌ Double encoding: spaces become %2520 (%25 = %, so %25 + 20 = %2520)
ENCODED="hello%20world"
curl "https://api.example.com/search?q=${ENCODED}"
# Server receives: q=hello%20world (correct — not double-encoded by curl in URL)

# But with --data-urlencode on already-encoded string:
curl --get --data-urlencode "q=hello%20world" https://api.example.com
# Encodes the % → q=hello%2520world (double-encoded!)
# Only pass RAW (unencoded) values to --data-urlencode

Test URL encoding with curl verbose

curl -v --get --data-urlencode "q=hello world" https://httpbin.org/get 2>&1 | grep ">"
# > GET /get?q=hello+world HTTP/2
# + is equivalent to %20 in query strings (application/x-www-form-urlencoded)

Related tools

• URL Encoder — encode/decode URL components
• URL Query String Guide — build query strings
• URL Encoding JavaScript — encode in JavaScript