Try 500 MB of US mobile proxy data free for 30 days.Start free trial
October 21, 202614 min read

Proxy Error Troubleshooting: Find the Failing Layer Fast

Isometric request chain with one marked failure point

Most proxy errors are solved by working out which hop in the request chain failed, not by changing settings at random. A request travels from your code to your network, through the proxy gateway, and on to the destination site. The error you see tells you where something was reported, and the cause is often one hop further along. This guide shows how to isolate that hop with curl, then walks through the errors engineers hit most: 407 Proxy Authentication Required, 502 Bad Gateway, 504 Gateway Timeout, failed CONNECT tunnels and environment variable traps.

TL;DR

  • 407 means the proxy rejected your credentials. Send a valid Proxy-Authorization header, or check the username, password and endpoint your client is really using.
  • 502 and 504 usually come from further down the chain. The proxy is reporting what it saw from the destination, but first rule out your VPN, DNS and firewall.
  • Reproduce the failure in curl before touching your code. If curl works and your application does not, the bug is in your client configuration.
  • Change one variable at a time. Credentials, targeting and request behavior altered together make a simple fault impossible to isolate.
  • For HTTPS, read the CONNECT step separately. A failure there points to a different cause than a failure on the request that follows.

Table of Contents

Start by Drawing the Request Chain

Before you read any log, write the chain down. For a typical scraper it looks like this:

  1. Your code and its HTTP library
  2. Your machine or server network: DNS, firewall, VPN, environment variables
  3. The proxy gateway, where authentication and routing happen
  4. The destination website

Each hop can fail on its own, and each reports failure differently. A refused TCP connection means you never reached the gateway. A 407 means you reached it and it said no. A 502 means the gateway reached something upstream and did not like the answer. Treat the error message as a symptom that tells you which hop to test next.

Then bisect. Test the innermost hop that you control, confirm it works, and move outward one hop at a time. The first hop that fails is where you dig. This is slower for the first two minutes and much faster over the whole afternoon.

Isolate the layer with curl

curl removes your application code from the picture. Set the proxy explicitly and ask for verbose output:

curl -v -x http://USER:[email protected]:PORT https://httpbin.org/ip

Read the output in order. DNS resolution of the proxy host comes first, then the TCP connection, then the proxy's response to the CONNECT request, then the TLS handshake with the destination. Whichever line fails first names your layer. If curl succeeds, run the same request from your application and compare the credentials, protocol and host it uses. We keep a set of copy-paste curl proxy commands for debugging DNS for this exact step.

Curl diagnostic stages from DNS through TLS

A few curl behaviors are worth knowing. A non-zero exit code is a diagnostic, not noise: curl exits with code 56 when a CONNECT through the proxy fails, which tells you the tunnel was refused before any request reached the destination. Use -i to see raw response headers, since a 407 from a proxy and a 401 from the destination are easy to confuse in library error messages.

Fixing 407 Proxy Authentication Required

This is the error readers search for most, and it has the clearest meaning. According to MDN's reference on the 407 status, the response means the request lacks valid credentials for the proxy server sitting between client and destination. The fix specified there is to send a Proxy-Authorization header with valid credentials in a follow-up request. With the Basic scheme, that is the username and password joined by a colon and base64 encoded.

Note what that means. The 407 is about the proxy, not the website you are scraping. Your target never saw the request.

A checklist for 407 errors

Work through these in order, stopping at the first one that applies:

  1. Typos and whitespace. Copy credentials from your dashboard into a plain text editor and look for trailing spaces or line breaks. A pasted newline in an environment variable is a classic.
  2. Special characters in the password. If a password contains @, :, / or #, a proxy URL like http://user:p@ss@host is parsed wrongly. Percent-encode the characters in the URL, or pass credentials separately through your library's proxy auth option.
  3. Wrong credential pair. Teams with several credential sets sometimes mix a username from one project with a password from another. Test each pair in curl.
  4. Library not sending credentials on the tunnel. Some clients attach proxy credentials to the request but not to the CONNECT step used for HTTPS. The tunnel is rejected, and you get a 407 only on HTTPS URLs.
  5. Authentication scheme mismatch. Read the Proxy-Authenticate response header with curl -i. It names the scheme the proxy expects, and your client must offer it.
  6. Credentials stripped by a redirect or middleware. A corporate proxy or a framework layer in front of yours may swallow the header.

If you work in a particular stack, the details differ. Our guides cover HTTP proxy authentication with curl and the RFCs, 407s in C# HttpClient with SocketsHttpHandler and 407s and rotation in Java with Apache HttpClient.

407 on HTTPS but not HTTP

This pattern is a strong hint. As MDN explains for the CONNECT method, the client asks the proxy to open a tunnel to the destination and then forwards data in both directions. A 407 during that phase means the tunnel itself was rejected for missing or invalid proxy credentials. Plain HTTP requests carry the header on the request itself, so a library that handles only that case works for HTTP and fails for HTTPS. Switch the library's proxy authentication option on, or test with curl to prove the credentials are fine.

Fixing 502 Bad Gateway and 504 Gateway Timeout

These two look alike and mean different things. Per MDN's 502 reference, a 502 means a server acting as a gateway or proxy received an invalid response from the upstream server. A 504 means the gateway received no response from upstream within its time budget.

SymptomWhat it usually meansFirst thing to test
407Proxy rejected credentialsSame credentials in curl with -i
502Invalid or failed answer from upstreamFetch the same URL from another network or a different exit location
504No answer from upstream in timeTry a lighter page on the same site, then raise your client timeout
Connection refused or timeout to gatewayYou never reached the proxyHost, port, firewall, VPN
curl exit 56 on HTTPSCONNECT tunnel failedRead the status line of the CONNECT response

The proxy is often the messenger

A proxy reports what it experienced. If the destination site is down, slow or rejecting the connection, the gateway tells you with a 502 or 504 even though the gateway is healthy. That is why changing your proxy credentials in response to a 504 achieves nothing.

The reverse also happens. MDN notes that 5xx errors can come from the client side of things too: if the service works for other users but not for your setup, look at your local networking, VPNs, DNS configuration and firewall rules before blaming the infrastructure.

A short triage for gateway errors

  • Does the failure repeat on one URL or on every URL? One URL points to the destination. Every URL points to your path to the gateway or the gateway itself.
  • Does it repeat from a second machine or network? If not, your local environment is the suspect: a VPN client, a security product inspecting traffic, or a stale DNS cache.
  • Does a retry succeed? Intermittent 502s and 504s on busy sites are normal. Retry with exponential backoff and a cap, rather than a tight loop that adds load.
  • Is your timeout realistic? A client timeout shorter than the proxy's upstream budget makes you give up early and misread slow pages as errors. Mobile networks can add latency, so set separate connect and read timeouts.

If you hit this client-side category often, such as one developer's laptop failing while CI succeeds, it is worth reading up on how spoofed or mismatched IP addresses show up in networking checks when you compare what each environment is presenting to the network.

When the Tunnel Itself Fails

HTTPS through a proxy depends on CONNECT, so tunnel failures deserve their own step. In the CONNECT exchange the 407, 502 and 504 codes split cleanly:

  • 407 on CONNECT: authentication to the proxy failed. Fix credentials or how your client sends them.
  • 502 or 504 on CONNECT: authentication passed, and the proxy could not connect to the target. Think DNS resolution or connectivity between the gateway and the destination.
  • Other 4xx on CONNECT: the proxy refused the target host or port by policy.

That last case surprises people. According to MDN, an open CONNECT method can be abused to relay unwanted traffic, so well-run proxy services restrict which ports tunnels may reach. If you point a tunnel at an unusual port and it is refused, that is policy, not a fault. Use standard ports for the destination, or ask your provider what is permitted.

The same logic applies on your side. If you run your own egress or office gateway, restricting what unmanaged tunnels can reach is good hygiene, and an IT team sprint on remote access security is one example of how organizations approach limiting unauthorized traffic.

Environment Variables That Quietly Break Proxies

Sometimes there is no proxy error at all, because the request never used the proxy, or used one you forgot about. Check these first on any server or CI runner:

  • HTTP_PROXY, HTTPS_PROXY and ALL_PROXY can be set globally, and different libraries read different ones. A stray value inherited from a container image sends traffic to the wrong gateway.
  • Lowercase and uppercase variants are not always treated the same. Print both with env | grep -i proxy.
  • NO_PROXY is the common trap. Per the curl manual, it matches on the host name as provided and does not resolve names first, so a rule for an IP will not catch a request by host name, and it is not a port filter. List the exact hostnames you want to bypass.

When results differ between your terminal and your application, dump the proxy environment inside the application process itself. Running in a different shell, service account or container is the usual reason.

Rotating, Sticky and Rate-Related Failures

Not every failure is a connection fault. If requests succeed for a while and then start failing, think about the destination's view of your traffic rather than the proxy path.

  • Session continuity. A multi-step flow such as login, search and checkout needs the same exit IP throughout. If it breaks on step three, use a sticky session rather than rotating on every request.
  • Request pacing. Many sites answer fast, repeated requests with throttling, typically a 429 Too Many Requests response. Slow down, add jitter and respect any retry delay the site sends.
  • Fingerprint mismatch. An IP that looks like a phone connection paired with a bare script user agent can look inconsistent. Align headers with the traffic you are presenting.

If you are judging a provider by a single percentage, be careful: a 95% to 99% success rate can mislead unless you know how failures are counted, and uptime checks need three to five samples rather than one. To test rotation and speed yourself, see our roundup of proxy checker tools.

Protocol and DNS Mistakes

Two configuration choices cause a surprising share of errors.

Scheme in the proxy URL. The scheme describes how you talk to the proxy, not the destination. An http:// proxy URL can carry HTTPS traffic through CONNECT. Using socks5:// against an endpoint that speaks HTTP, or the reverse, produces handshake failures that look like timeouts. Match the scheme to the port your provider documents. Our comparison of HTTP and HTTPS proxy decision rules covers the practical differences.

Where DNS resolves. With SOCKS5, whether your machine or the proxy resolves the destination name depends on the client setting. Resolving locally can leak lookups and return different addresses than the proxy would see. In curl, socks5h:// sends name resolution through the proxy. We describe five tests for DNS leaks and for picking SOCKS5 or HTTP.

Our Perspective: Debug the Chain, Not the Symptom

We run a mobile proxy network on real US carrier IPs, and the support questions we see follow a pattern. The majority of first reports say the proxy is down. When we ask for one curl command with verbose output, the answer is usually something else: a password with a character that needed encoding, an HTTPS request whose library skipped credentials on the CONNECT step, or a proxy environment variable left over from another project.

Our Perspective: Debug the Chain, Not the Symptom — overview diagram

Our advice is to build the habit of reproducing the failure in curl first, before anything else. It takes a minute, and it splits the problem cleanly into your code versus your path to the gateway versus the destination. We also recommend keeping credentials per project and per team member in the dashboard, so a 407 can be traced to one credential set without rotating everything. Use sticky sessions when a workflow needs continuity, rotating sessions when you want a fresh IP per request, and change one setting per test.

Also be honest about what a proxy cannot fix. Real mobile IPs make requests look like ordinary carrier traffic and reduce blocking, but a destination that is down, slow or rate-limiting you is still down, slow or rate-limiting you. Pace your requests and read the status codes.

— Jon

Test Against Real Mobile IPs

If you want to check your setup against a real carrier connection, you can try MaskLabs with a free trial that includes a limited amount of data. We offer real US mobile carrier IPs in 46 cities, sticky and rotating sessions, HTTP, HTTPS and SOCKS5 support, and self-serve credentials, so you can run the curl tests above without a sales call. Plans start at $30 per month for Starter, with Basic at $100 per month, Advanced at $500 per month and Scale at $1,000 per month. Billing is metered per byte, and prepaid data add-ons are available. See the plans and pricing or start from the MaskLabs homepage. For teams comparing features first, eight proxy API features to compare is a good checklist.

A Repeatable Debugging Routine

Keep this sequence next to your terminal:

  1. Write the chain: code, local network, gateway, destination.
  2. Run curl with -v and -x using the exact credentials and host your application uses.
  3. Read the first failing line: DNS, TCP, proxy status, TLS, or destination status.
  4. Check the proxy environment: env | grep -i proxy, plus any library-level proxy settings.
  5. Test from a second network or machine to separate local from remote causes.
  6. Change one variable, retest, and record the result.
  7. Only then contact support, with the curl output, timestamps, the exit location you targeted and the exact status code.

That last step matters. A support request that includes the curl transcript can be answered in one reply, since it already shows which hop failed.

FAQ

What does 407 Proxy Authentication Required mean?

It means the proxy server refused your request because it did not receive valid credentials. The destination site never saw the request. Send a correct Proxy-Authorization header, or supply the proxy username and password through your client's proxy auth setting, and test the same pair in curl.

Why do I get a 407 only on HTTPS sites?

HTTPS goes through a CONNECT tunnel, and your client must authenticate that step too. Some libraries attach credentials to plain requests but not to the tunnel request. Enable the library's proxy authentication option for tunnels, or confirm with curl -v -x that the same credentials succeed.

Is a 502 or 504 error my proxy's fault?

Usually not directly. Both codes mean the gateway had trouble with the upstream server, either an invalid response or no response in time. Check whether the destination is reachable from another network, rule out your VPN, DNS and firewall, then retry with backoff and a longer read timeout.

Should a proxy be on or off?

On only for the traffic you want routed through it. If you are not using a proxy, clear the proxy settings and environment variables so requests go direct. If you are, make sure every library in your stack reads the same configuration.

Why does NO_PROXY not exclude my domain?

NO_PROXY compares the host name exactly as you gave it, without resolving it, and it does not filter by port. List each hostname you want to bypass explicitly, and verify the result with curl -v to see whether the proxy was used.

Sources

Recommended