Proxy Concurrency Limits: How They Work and How to Size Yours

A proxy concurrency limit is a cap on how many connections or sessions can be open at the same time. It is enforced at three places: your own workers, the proxy gateway, and the target website. Whichever you hit first sets your real ceiling. To size a job, work out the number of requests you need per second, multiply by the average time each request takes, and start below that number. Then raise it in small steps while you watch errors and latency.
This guide is for developers, data scientists and analysts who run scrapers, price monitors, ad verification or geo-targeted research through a proxy. It explains what the limits are, how to estimate the number you need, and how to raise it without triggering blocks. At Masklabs we run US mobile proxies on real carrier IPs, so we also cover how mobile traffic changes the picture.
TL;DR
- Concurrency is the number of requests in flight at once. A rate limit counts requests over time, and a bandwidth cap counts bytes. They are different limits and fail in different ways.
- Limits can sit at your machine, at the proxy gateway and at the target site. The first one reached governs, and each shows its own error signature.
- Estimate concurrency with a simple rule: target requests per second multiplied by average response time in seconds. Add a margin, then test upward in steps.
- Rising 429 responses, connection resets and socket timeouts are the signs you have gone past a limit. Back off first, then tune.
- On mobile proxies, concurrency is shaped more by the target's per-IP tolerance and your session design than by raw thread count.
Table of Contents
- What a concurrency limit actually is
- Where limits are imposed
- How concurrency differs across proxy types
- How to estimate the concurrency you need
- Recognizing that you have hit a limit
- How to raise concurrency without getting throttled
- Common mistakes
- Our perspective
- Run it on real mobile IPs
- FAQ
- Sources
What a concurrency limit actually is
Concurrency means the number of operations in progress at the same moment. For a proxy, that is open connections or active sessions, not the total number of requests you send in a day. A scraper that sends 10 requests per second, each taking 2 seconds to complete, has about 20 requests in flight at any instant. Those 20 are its concurrency.
Three neighboring limits are easy to confuse with it:
| Limit | What it counts | Typical symptom when exceeded |
|---|---|---|
| Concurrency limit | Simultaneous open connections or sessions | Queued requests, timeouts, connection resets |
| Rate limit | Requests per second, minute or hour | HTTP 429 responses |
| Bandwidth or data cap | Bytes transferred | Throttling or a stopped account |
| IP limit | Number of distinct IPs or ports you can use | Refused new sessions or credentials |
The distinction matters when you debug. The HTTP standard's 429 status means the client sent too many requests in a given amount of time, and a Retry-After header may tell you how long to wait. That is a rate limit. A flood of simultaneous connections can trigger it, but so can a single fast client with only one connection. If your errors are resets and timeouts rather than 429s, you are probably looking at a concurrency ceiling somewhere along the path.
Where limits are imposed
Think of three layers between your code and the page you want.
Your own workers
Every open connection uses memory, a file descriptor and some kernel bookkeeping, and TLS handshakes add CPU cost. USENIX's write-up on operating a large reverse proxy notes that each connection consumes memory, file descriptors and kernel resources, and that reusing connections saves handshake work while long-lived idle connections still hold resources. The same article cautions that adding CPU cores does not scale throughput in a straight line, because cache behavior and uneven socket distribution can leave one core saturated while overall CPU looks free.
In practice, a scraper that opens thousands of sockets from one process can fail with "too many open files" before the proxy or the target ever objects. Check your operating system's file descriptor limit and your HTTP client's pool settings first. They are free to change and are the most common hidden ceiling.
The proxy gateway
Proxy infrastructure enforces limits in several scopes. Documentation for enterprise proxy software describes limits that can be global, per client IP or per client connection policy, and the first limit reached is the one that applies. The same logic holds for most gateways: an account-wide cap, a per-credential cap and sometimes a per-IP cap can all exist at once, and the smallest wins. For a deeper look at how these scopes interact in API design, this walkthrough of API rate limiting design and enforcement scopes covers global, per-IP and client-policy enforcement in detail.
Capacity inside a proxy also depends on its configuration. NevisProxy's scaling documentation lists processes and threads, CPU and memory, backend response times, expected session lifetime, file descriptors and session-store memory as factors, and names settings such as maximum clients, keep-alive and connection pooling as controls over parallel requests. The same page documents default behavior in which a local session store may stop creating new sessions at 90% memory use. The lesson for you: a provider's gateway has real resource limits too, and long-lived sessions cost more than short ones.
The target website
The destination site is usually the strictest layer. It rarely publishes a concurrency number. Instead it watches how many requests arrive from one IP, one session or one fingerprint, and it responds with 429s, CAPTCHAs, soft blocks or silently altered content. MDN notes that rate limiting is typically based on the client's IP but can also be tied to users or authenticated applications. This is why total proxy concurrency and per-IP concurrency are two separate questions.
How concurrency differs across proxy types
The type of IP changes what the target tolerates and how your sessions behave.
- Datacenter IPs come from hosting ranges. They are fast and cheap, but sites often treat whole ranges with suspicion, so a high number of parallel requests from them is more likely to be throttled or blocked.
- Residential IPs belong to home connections. They look more like ordinary users, but each IP is expected to behave like one household.
- Mobile carrier IPs are assigned by mobile networks, and many real users share the same addresses. Targets tend to be cautious about blocking them outright. Our own approach is built on this: Masklabs uses real US carrier IPs rather than datacenter ranges, so requests look like ordinary mobile traffic and are less likely to be blocked or throttled.
Whatever the type, one rule holds: a single IP should look like one plausible user. Ten parallel requests to the same product page from one address is not plausible. Ten requests spread over ten addresses is. That is why concurrency planning is really planning how many IPs you use at once and how many requests each one carries.
Sticky versus rotating sessions
Sticky sessions keep the same IP for a period, and rotating sessions change the IP per request or on a schedule. They affect concurrency in different ways.
- Rotating spreads parallel requests across many IPs. It is the natural fit for wide, shallow jobs such as price checks across thousands of product pages. Each request is independent, so you can run more in parallel without any one IP looking busy.
- Sticky holds one IP for a login, a cart or a paginated flow. Concurrency here means concurrent sessions, each pinned to its own IP and each running requests one after another. Increasing parallelism means opening more sessions, not firing more requests down one.
We offer both on Masklabs, and our session management tips for stable proxy workflows explain how to choose a sticky duration that matches a task, for example pinning one IP for about 3 minutes for a short multi-step flow.
How to estimate the concurrency you need
Start from the output you want, not from a thread count. The steps below work for scraping, geo-targeted research and price monitoring alike.
Step 1: Define the throughput target
Write down the number of pages you need and the time window. If you need 600,000 product pages checked in 6 hours, that is 21,600 seconds, so you need roughly 28 pages per second.
Step 2: Measure average response time
Run a small sample through the proxy and record how long a full response takes, including the proxy hop. Mobile routes are often slower than datacenter ones, so measure on the same type you will use. Say it averages 1.5 seconds.
Step 3: Apply the basic relationship
Concurrency is approximately throughput multiplied by response time. In the example, 28 pages per second times 1.5 seconds gives about 42 requests in flight. That is your starting estimate. If responses slow to 3 seconds, the same throughput needs 84, which is why slow targets force higher concurrency, and why it is worth checking whether the slowness is on your side.

Step 4: Add the per-IP constraint
Decide how many requests one IP may carry at once. For sensitive targets, one at a time per IP is a safe default. With 42 requests in flight and one per IP, you need 42 distinct IPs active at the same moment. With rotating sessions that is handled for you. With sticky sessions it means 42 concurrent sessions.
Step 5: Add margin and check each layer
Add roughly 20 to 30 percent headroom for retries and slow responses, which is a planning habit rather than a rule. Then compare the final number to each layer: your file descriptor and pool limits, any account or credential limit the proxy plan states, and what the target tolerates. The lowest number is your ceiling.
A worked example
A retail analytics team compares prices for the same 400 products across 20 US cities. That is 8,000 page loads per run. They want a run to finish in 10 minutes, so about 13 pages per second. With a 2-second average response, they need about 26 in flight. Because the pages are city-specific, they use city-level targeting so each page is requested from an IP in the right market. They run 26 parallel workers with rotating sessions and keep each IP to one request at a time. If errors stay low, they raise the target step by step. If they climb, they stop and back off.

Recognizing that you have hit a limit
Different ceilings look different, and reading the signature saves hours.
- HTTP 429 responses. The target or gateway says you are sending too fast. Check for a Retry-After header and honor it.
- Connection resets and socket timeouts. Often a sign of queued or refused connections at the gateway or a saturated local machine. One engineering glossary describes how opening far more sessions than a gateway allows leaves the excess requests stalled in a queue, producing timeouts or resets.
- "Too many open files" (EMFILE). Your own process ran out of file descriptors.
- Latency creeping upward while success rate holds. You are near a limit and queues are forming. This is the best moment to stop raising concurrency.
- 200 responses with wrong content. Soft blocks: the target returns a page, but not the real one. This is why a headline success rate can mislead, as we explain in why a 95% to 99% proxy success rate can mislead.
Log enough to tell these apart. Capture status code, error type, latency, the IP or session used and the target. Our guide to proxy usage logging and what to capture lists the fields that make this analysis possible.
How to raise concurrency without getting throttled
- Raise in steps. Increase by 10 to 20 percent, run for a few minutes, compare error rate and p95 latency, then decide. Do not jump from 50 to 500.
- Use an adaptive limiter. Cap in-flight requests with a semaphore or worker pool in your code, and let the cap drop automatically when 429s or latency rise, then recover slowly. This is the same idea as TCP congestion control.
- Honor Retry-After and use exponential backoff with jitter. Retrying immediately at full speed turns a small limit into a ban. Our post on four proxy failover strategies that stop wasted retries covers how to retry without burning data.
- Reuse connections where it helps. Keep-alive and pooling cut handshake cost, but cap idle connections so they do not hold descriptors and memory.
- Spread load across IPs and time. Distribute requests across a larger pool and add small random delays. For managing that pool, see proxy pool management for scraping teams.
- Separate targets. Give each target its own concurrency budget. A tolerant site should not be starved by a strict one, and a strict one should not drag the rest down.
- Monitor with repeated samples. A single test run hides variance. Use several, as described in proxy uptime monitoring with 3 to 5 samples.
If you work in Java, our copy-ready Java proxy scraping configs show connection pool settings that prevent pool throttling and 407 authentication errors. Protocol choice also affects overhead: see our HTTP vs HTTPS proxy decision rules and the explanation of how the HTTPS CONNECT tunnel works.
Common mistakes
- Treating thread count as concurrency. Ten threads that each open five connections are 50 connections.
- Raising concurrency to fix slowness. If the target is slow because you are already near its limit, more parallelism makes it worse.
- Ignoring per-IP behavior. High total concurrency is fine when it is spread out. High concurrency on one IP is what gets flagged.
- Retrying on every error. Retries multiply load exactly when a limit is being hit.
- Sizing only the proxy side. The target's tolerance and your own machine's limits are as real as the gateway's.
Our perspective
We see concurrency as a design decision, not a dial to turn up until something breaks. The number that matters is how many distinct, believable users your job looks like to the target at any moment.
That is why we built Masklabs on real US mobile carrier IPs with city-level targeting, and why we offer both sticky and rotating sessions. Rotating spreads wide jobs across many addresses. Sticky keeps a multi-step flow on one address for as long as it needs. You choose per job.
Billing is metered per byte against a monthly data pool, so what you pay for is the data you move. That makes it practical to test at low concurrency, read the results in the usage dashboard, and raise parallelism only when the numbers support it.
Our advice to every team: measure, step up, log everything, and keep one IP to one plausible user.
Jon
Run it on real mobile IPs
If your jobs need authentic US carrier IPs, city-level targeting, rotating or sticky sessions and HTTP, HTTPS or SOCKS5 access, Masklabs is built for that. Self-serve credential management and a usage dashboard let you start without a sales call or a long-term commitment. Plans are tiered by monthly data pool: Starter is $30.00 per month, Basic $100.00, Advanced $500.00 and Scale $1,000.00, with prepaid data add-ons billed per GB and a free trial with limited data. See US mobile proxies on real carrier IPs for details.

FAQ
What is a proxy concurrency limit?
It is the maximum number of connections or sessions that can be open at the same time through a proxy. It can be set globally, per account or credential, or per IP, and the first limit you reach applies. It is separate from a rate limit, which counts requests over time.
How is concurrency different from a rate limit?
Concurrency counts how many requests are in flight right now. A rate limit counts how many you send per second or minute. A slow target can keep concurrency high at a low request rate, and a fast one can hit a rate limit with very few open connections.
How many concurrent requests do I need?
Multiply the requests per second you need by the average response time in seconds. Thirty pages per second at 2 seconds each means about 60 in flight. Add some headroom, then test upward in small steps while watching error rate and latency.
Why do I get connection resets when I add more threads?
Resets and timeouts usually mean connections are queuing or being refused, either at the gateway, at the target or because your machine ran out of file descriptors. Lower concurrency, check your open-file limit and pool settings, and raise it again gradually.
Do sticky sessions need more concurrency than rotating ones?
Not necessarily, but they scale differently. With sticky sessions you add parallelism by opening more sessions, each on its own IP. With rotating sessions you spread parallel requests across the pool automatically.
Sources
- 429 Too Many Requests, MDN Web Docs
- Setting resource limits, PingDirectory 11.0 documentation
- Scaling, Nevis documentation
- Design Tradeoffs at the Edge, USENIX
- API Rate Limiting Design: Algorithms, Distribution, Client Handling