URL Encoding Troubleshooting: Fix 15+ Common Issues Like a Pro

1. Issue #1: Double Encoding (%25%20 instead of %20)

Double encoding happens when you encode an already encoded string. The percent sign (%) itself gets encoded to %25, causing cascading encoding.

Original: "hello world"
Step 1 (encode): "hello%20world"
Step 2 (encode again): "hello%2520world" ← WRONG!

Result when decoded once: "hello%20world" (still encoded!)
Result when decoded twice: "hello world"

// Detect double encoding
function fixDoubleEncoding(str) {
  let decoded = str;
  let prev = "";
  while (decoded !== prev && decoded.includes("%")) {
    prev = decoded;
    decoded = decodeURIComponent(decoded);
  }
  return decoded;
}

// Then re-encode ONCE
const clean = fixDoubleEncoding(input);
const encoded = encodeURIComponent(clean);

2. Issue #2: Mixed Encoding Standards (+ vs %20)

URL: /search?q=hello+world&filter=category%20books

Space in q: encoded as + (application/x-www-form-urlencoded style)
Space in filter: encoded as %20 (RFC 3986 style)
→ Mixed standards cause parsing issues!

// Always use RFC 3986 (%20) for consistency
function encodeParam(value) {
  return encodeURIComponent(value)
    .replace(/!/g, '%21')
    .replace(/'/g, '%27')
    .replace(/(/g, '%28')
    .replace(/)/g, '%29')
    .replace(/*/g, '%2A')
    .replace(/~/g, '%7E');
}

// For application/x-www-form-urlencoded (form data)
function encodeFormData(value) {
  return encodeURIComponent(value).replace(/%20/g, '+');
}

3. Issue #3: Unicode/Multibyte Character Corruption

Non-ASCII characters (like emojis, Chinese, Hindi, Arabic) need special handling. They must be UTF-8 encoded first, then percent-encoded.

4. Issue #4: Invalid Percent Sequences

Input: "hello %2 world"
Problem: %2 is incomplete (needs 2 hex digits)
Input: "hello %2G world"
Problem: 'G' is not a valid hex digit (only 0-9, A-F)

// Option 1: Strict validation
function isValidPercentEncoding(str) {
  return !/%[^0-9A-Fa-f]/.test(str) && !/%[0-9A-Fa-f][^0-9A-Fa-f]/.test(str);
}

// Option 2: Safe decode with error handling
function safeDecode(str) {
  try {
    return decodeURIComponent(str);
  } catch (e) {
    console.warn("Invalid percent encoding:", str);
    // Fallback: replace invalid sequences
    return str.replace(/%[^0-9A-Fa-f]/g, '%25$&');
  }
}

5. Issue #5: Encoding Function Mismatch

// Using encodeURI() for query parameters
const url = "https://api.com/search?q=" + encodeURI(userInput);
// If userInput = "hello&world", encodeURI leaves & unchanged
// Result: URL breaks!

// Use encodeURIComponent() for parameters
const url = "https://api.com/search?q=" + encodeURIComponent(userInput);
// "hello&world" → "hello%26world"

// Use encodeURI() only for full URLs
const fullUrl = encodeURI("https://example.com/path with spaces");
// Preserves protocol and domain structure

6. Issue #6: Missing Encoding (Raw Special Chars in URL)

// DANGER: This URL will break!
https://api.com/search?q=hello&world&filter=price>100

// The & character is interpreted as parameter separator
// The > character breaks the URL structure

// Actual parsed parameters:
// q = "hello"
// world = "" (empty parameter!)
// filter = "price"

const params = {
  q: "hello&world",
  filter: "price>100"
};

const queryString = Object.entries(params)
  .map(([k, v]) => encodeURIComponent(k) + "=" + encodeURIComponent(v))
  .join('&');

const url = "https://api.com/search?" + queryString;
// Result: https://api.com/search?q=hello%26world&filter=price%3E100
// Now safe!

7. Issue #7: Over-Encoding Safe Characters

While technically allowed, encoding unreserved characters (A-Z, a-z, 0-9, -, ., _, ~) is unnecessary and reduces readability.

// This works but is ugly
const overEncoded = "hello%20world";
const superEncoded = "%68%65%6C%6C%6F%20%77%6F%72%6C%64";

// Both decode to "hello world"
// But superEncoded is 3x larger and unreadable!

// Only encode what needs encoding
function smartEncode(str) {
  // Only encode reserved characters
  return encodeURIComponent(str)
    // But don't over-encode what's already safe
    .replace(/%2D/g, '-')
    .replace(/%2E/g, '.')
    .replace(/%5F/g, '_')
    .replace(/%7E/g, '~');
}

8. Issue #8: Truncated/Incomplete URLs

Special characters like # (fragment) or ? (query start) can cause URLs to be truncated if not properly handled.

// Wrong: No encoding of # in value
const url = "https://api.com/search?query=chapter#1";
// Browser interprets #1 as fragment identifier
// Server only sees: "https://api.com/search?query=chapter"

// Correct: Encode # as %23
const url = "https://api.com/search?query=chapter%231";
// Entire value reaches server

9. Issue #9: URL Length Exceeded

10. Issue #10: Security Filter Bypasses

// Original XSS payload that gets blocked
<script>alert('XSS')</script>

// Encoded version that bypasses naive filters
%3Cscript%3Ealert('XSS')%3C/script%3E

// Double encoded version (bypasses single decode checks)
%253Cscript%253Ealert('XSS')%253C/script%253E

// Always canonicalize before validation
function canonicalize(input) {
  let canonical = input;
  let prev = "";
  // Decode until no more percent signs
  while (canonical !== prev && canonical.includes("%")) {
    prev = canonical;
    canonical = decodeURIComponent(canonical);
  }
  return canonical;
}

// Then validate the canonicalized string
const safe = canonicalize(userInput);
if (isXSSPayload(safe)) {
  reject();
}

11. Issue #11: Browser-Specific Behaviors

// JavaScript behavior varies:
decodeURIComponent("%"); // Throws URIError in Chrome, Firefox

// URL hash/fragment handling:
window.location.hash = "hello world";
// Chrome: space becomes %20
// Firefox: space becomes %20
// Safari: space becomes %20 (consistent now)

12. Diagnostic Checklist

13. Automated Fixes & Tools

function autoFixUrl(url) {
  let fixed = url;
  
  // Fix 1: Convert + to %20 in query (for RFC 3986 compliance)
  fixed = fixed.replace(/\+/g, '%20');
  
  // Fix 2: Fix double encoding
  while (fixed.includes('%25')) {
    fixed = fixed.replace(/%25([0-9A-Fa-f]{2})/g, '%$1');
  }
  
  // Fix 3: Remove over-encoding of safe chars
  fixed = fixed
    .replace(/%2D/g, '-')
    .replace(/%2E/g, '.')
    .replace(/%5F/g, '_')
    .replace(/%7E/g, '~');
  
  // Fix 4: Uppercase hex digits (consistency)
  fixed = fixed.replace(/%[0-9a-f]{2}/g, match => match.toUpperCase());
  
  return fixed;
}

14. Prevention Strategies