Debugging a cURL Request That Works in curl But Not in Python
It is almost never "the API hates requests." It is almost always one of five things curl does that your code didn't — all of them visible in the converted output on this site, if you know which line to read.
1. User-Agent
curl sends User-Agent: curl/8.x. Some CDNs prefer it and reject generic script UAs out of hand — you're getting a 403 from the edge, not the app. Fix: set a real UA header in code and test again. If the code works with a UA header, that was it.
2. Header case and ordering
HTTP headers are case-insensitive in the spec; badly-written APIs are not. curl's copy-paste sends Content-Type; your requests.post(json=...) sends it automatically, and your hand-rolled dict added content-type — and their middleware checks the exact casing of the one they read. Match the copy from devtools verbatim, once, and normalize later.
3. The body encoding you didn't choose
curl -d '{"a":1}' sends bytes. requests.post(url, data={"a":1}) form-encodes the dict: you sent a=1, the API expected JSON, you got a 400 "invalid json". The curl equivalent of json= is data=json.dumps(...) plus a Content-Type header. This mismatch is the single most common cause of the complaint in this page's title. Our multipart upload example covers the same trap from the file side.
4. TLS/redirect handling
curl -L follows redirects and preserves the method; requests follows but converts 301/302 POST to GET. If your POST becomes a GET at a redirect hop, the body is gone by the time it lands — the failure looks like "my payload disappeared."
5. Timeouts and connection reuse
curl reuses nothing and times out fast. Your code might reuse a keep-alive connection a load balancer already killed (that RemoteDisconnected you keep seeing), or sit on a 130s connect timeout while a CI watchdog blames the API. Explicit timeouts on every call; retry only idempotent verbs.
The debugging move: paste your curl into the converter, read the diff between what it generated and what you actually wrote, and ship the difference. Nine times out of ten that diff is the bug.
FAQ
requests gives 403 but curl gives 200. What first? Set User-Agent to what curl sends (check with curl -v). Cloudflare and Akamai commonly gate on UA alone.
Should I retry on RemoteDisconnected? Only idempotent methods (GET/PUT/HEAD) and only with a fresh connection — disable connection reuse or set a pool timeout shorter than your LB's idle timeout.
Why did my POST body vanish after a redirect? Your HTTP client downgraded POST→GET on 301/302 per RFC behavior. Point the client at the final URL directly, or configure 307/308-preserving redirect handling.