Try 25 MB of US mobile proxy data free for 30 days.Start free trial
November 13, 202611 min read

Ruby Net::HTTP Proxy: Setup, HTTPS, Auth and Fixes

Isometric proxy route and secure tunnel title card

To send a request through a proxy in Ruby, pass the proxy host, port, username and password after the destination host and port: Net::HTTP.new(host, port, proxy_host, proxy_port, proxy_user, proxy_pass). Or set the http_proxy environment variable and let Net::HTTP pick it up. The constructor is the safer choice for scrapers and workers, because it behaves the same on every machine. Below you get the full setup, HTTPS, authentication, opt-outs and the errors you will actually hit. We also cover what to point the client at once you need real mobile carrier IPs.

TL;DR

  • Net::HTTP reads the http_proxy environment variable by default. Explicit constructor arguments override it, and an explicit nil turns discovery off.
  • Proxy arguments go after the target host and port: Net::HTTP.new(host, port, p_addr, p_port, p_user, p_pass).
  • For HTTPS targets, set http.use_ssl = true. A conventional proxy opens a CONNECT tunnel, and TLS protects the traffic to the target.
  • A 407 means the proxy wants credentials. Check the URL, the credentials and any percent-encoding first, then set timeouts.
  • For scraping and geo-targeted work, we built our mobile proxies for exactly this Ruby setup: real US carrier IPs in 46 cities with HTTP and HTTPS support.

Table of Contents

How Net::HTTP decides whether to use a proxy

Before writing any code, it helps to know that Net::HTTP has three modes. Ruby's documentation states that Net::HTTP.new and Net::HTTP.start default their proxy address to :ENV. In that mode, if http_proxy holds a valid URI with a scheme, Ruby derives the proxy host, port and optional credentials from it. If the variable is absent, the connection is direct (Ruby 4.1 Net::HTTP documentation).

The three modes are:

ModeWhat you passResult
Environment discoveryNothing (default :ENV)Uses http_proxy if set, direct otherwise
Explicit proxyHost, port, user, passwordUses exactly that proxy, ignoring the environment
Disablednil as the proxy addressAlways direct

The Ruby source shows that proxy behavior is constructor state. :ENV enables discovery, nil disables it and explicit values override it (lib/net/http.rb, Ruby 3.3). Automatic detection was added in a feature request on the Ruby tracker, and explicit configuration stayed supported alongside it (Feature #6546).

This matters in practice. A script that works on your laptop, where a shell profile exports http_proxy, can take a different route on a CI runner or a worker container. If you want one predictable route, stop relying on the environment.

Set a proxy directly in the constructor

This is the pattern to use for production scrapers. The proxy values come right after the destination host and port:

require 'net/http'
require 'uri'

uri = URI('https://example.com/')

http = Net::HTTP.new(
  uri.host, uri.port,
  'proxy.example.com', 8080,
  'PROXY_USER', 'PROXY_PASS'
)
http.use_ssl = (uri.scheme == 'https')
http.open_timeout = 10
http.read_timeout = 20

response = http.get(uri.request_uri)
puts response.code

The signature is documented as Net::HTTP.new(host, port, p_addr, p_port, p_user, p_pass) (Ruby 4.1 documentation). Two details trip people up:

  • Net::HTTP.new only builds the object. It does not open a TCP connection or an HTTP session, so a wrong proxy address shows up on the first request, not on this line.
  • The first pair of arguments is the target, not the proxy. Swapping them is the most common first-time mistake.

The block form with Net::HTTP.start

Net::HTTP.start accepts the same proxy parameters and opens the connection for you. With a block, it closes the connection when the block ends (Net::HTTP.start reference):

Net::HTTP.start(
  uri.host, uri.port,
  'proxy.example.com', 8080,
  'PROXY_USER', 'PROXY_PASS',
  use_ssl: uri.scheme == 'https',
  open_timeout: 10,
  read_timeout: 20
) do |http|
  res = http.get(uri.request_uri)
  puts res.code
end

The block form is the right default for scripts that make several requests to the same host, because the session is reused and always cleaned up. If you call start without a block, you must call finish yourself when you are done, per the same reference. Forgetting it leaks sockets in long-running workers.

Use the http_proxy environment variable

If you would rather configure the proxy outside the code, export the variable with a full URI, scheme included:

export http_proxy='http://PROXY_USER:[email protected]:8080'
ruby fetch.rb

Then Net::HTTP.get(URI('http://example.com/')) or Net::HTTP.new(host, port) will route through it with no further arguments. Two cautions:

  1. Percent-encode special characters in the username and password, such as @, : or /. An unencoded @ in a password splits the URI in the wrong place and produces an authentication failure that looks like a bad credential.
  2. Environment-derived proxies ignore some constructor options. The documentation for proxy_use_ssl says it is ignored for proxies taken from the environment (Net::HTTP.new reference).

The environment approach suits a quick script or a container where operations owns networking. For code you ship to a team, the constructor is easier to review and test.

Disable or override the automatic proxy

To guarantee a direct connection, pass nil explicitly as the proxy address:

http = Net::HTTP.new(uri.host, uri.port, nil)

The documentation says this disables environment discovery. To bypass the proxy only for certain destinations, it offers a p_no_proxy argument (Ruby 4.1 documentation).

The trailing nil is easy to drop by accident. The Ruby-Doc Learn guide on building a reliable fetcher warns that leaving it off allows environment proxy discovery, which can send a laptop and a worker process down different network routes (Ruby-Doc Learn). If your tests pass locally and fail in CI with timeouts, check this first.

HTTPS through a proxy

For an HTTPS target, create the client with the proxy and set http.use_ssl = true, as in the examples above (Ruby documentation).

Under the hood, a conventional HTTP proxy handles HTTPS with a CONNECT tunnel. The proxy opens a raw TCP path to the target, and TLS then protects the connection between your Ruby process and the target site (Ruby-Doc Learn). The proxy sees the destination host and port, but not the request path or the body.

A separate question is TLS to the proxy itself. Ruby exposes a proxy_use_ssl option for that, which applies when the destination also uses SSL/TLS (Net::HTTP.new reference). Most setups do not need it: a plain HTTP proxy endpoint carrying a CONNECT tunnel is the standard arrangement. The net-http issue tracker also records a reported problem with HTTPS-proxy support involving nested TLS sockets (ruby/net-http issue 212), so keep your test requests small when you first try that mode.

If you want to see what a CONNECT exchange looks like on the wire, our walkthrough of how the HTTPS proxy CONNECT tunnel works shows how to read it and debug it. For the choice between proxy types, see our HTTP vs HTTPS proxy decision rules.

HTTPS through a proxy

Authenticate to the proxy

Supply the proxy username and password as the fifth and sixth arguments, p_user and p_pass, in either Net::HTTP.new or Net::HTTP.start (Ruby documentation). That is all Net::HTTP needs for basic proxy authentication.

When the proxy rejects the credentials, you get status 407. Ruby maps it to Net::HTTPProxyAuthenticationRequired in the response class hierarchy (Ruby Net module reference). You can handle it explicitly:

response = http.get(uri.request_uri)

case response
when Net::HTTPSuccess
  puts response.body.bytesize
when Net::HTTPProxyAuthenticationRequired
  warn 'Proxy rejected the credentials (407)'
else
  warn "Unexpected status: #{response.code}"
end

Note that a 407 comes from the proxy, while a 401 comes from the target site. They need different fixes, and mixing them up wastes hours. Our guide to fixing 407 errors and proxy authentication walks through the exchange with curl so you can reproduce it outside Ruby.

Troubleshooting checklist

Work from the outside in. Most Net::HTTP proxy failures sit in one of a few categories: an incorrect proxy address or port, wrong credentials, a no_proxy match, or TLS settings (DeepWiki's net-http proxy overview). The Ruby-Doc Learn reliability guide adds practical habits: validate the proxy URL, percent-decode credentials before handing them to Net::HTTP, set open, read and write timeouts, disable retries when you do not want them, and keep TLS peer and hostname verification on (Ruby-Doc Learn).

SymptomLikely layerFirst thing to try
Errno::ECONNREFUSED or timeout on connectProxy address or portConfirm host and port, raise open_timeout briefly
407 responseProxy authenticationRecheck user and password, decode any percent-encoding
Works locally, fails on serverEnvironment discoveryPass an explicit proxy or a trailing nil
OpenSSL::SSL::SSLErrorTLS to the targetConfirm use_ssl = true and the right port (443)
Hangs after connectRead timeout unsetSet read_timeout and write_timeout
Bypass not honoredno_proxy matchingCheck the host against your no_proxy list

A 407 behind a proxy is not unique to Ruby. A Rails issue reports the same response from a tool running behind a proxy (rails/importmap-rails issue 177), which is a reminder that the proxy layer is often the culprit and not your code.

When the cause is unclear, find which layer fails: DNS, TCP, the proxy handshake, TLS or the target. Our proxy error troubleshooting guide is organized that way. If your requests succeed but the target treats them oddly, the cause may be headers the proxy adds or strips, covered in HTTP proxy headers.

A production-ready wrapper

Putting the pieces together, this small helper builds a client from environment-provided values but passes them explicitly, so there is no hidden discovery:

require 'net/http'
require 'uri'

def proxied_get(url)
  uri = URI(url)
  host = ENV.fetch('PROXY_HOST')
  port = Integer(ENV.fetch('PROXY_PORT'))
  user = ENV.fetch('PROXY_USER')
  pass = ENV.fetch('PROXY_PASS')

  Net::HTTP.start(
    uri.host, uri.port, host, port, user, pass,
    use_ssl: uri.scheme == 'https',
    open_timeout: 10,
    read_timeout: 30
  ) do |http|
    http.get(uri.request_uri, 'User-Agent' => 'my-collector/1.0')
  end
end

res = proxied_get('https://example.com/')
puts res.code

Credentials live in separate variables, so there is nothing to percent-encode. Every value is explicit, so the route is the same on every machine. Timeouts are bounded, so a dead proxy fails in seconds, not minutes. To keep a long job from hammering one connection, wrap the call in your own retry logic with a small, capped number of attempts.

Which proxy to point Net::HTTP at

Net::HTTP does not care what sits behind the proxy address. That is where results diverge. For data collection, price monitoring, ad verification or local search checks, the type of IP decides whether your requests look like ordinary traffic. Datacenter ranges are the ones most often blocked or throttled. Real carrier addresses are not.

That is the reasoning behind our service. MaskLabs provides mobile proxies on real US carrier IPs, so your Ruby requests arrive from genuine mobile networks and not from datacenter ranges. The Ruby side stays exactly as shown above: a host, a port and credentials.

Perspective

Most Ruby proxy problems we see discussed come down to one thing: a developer treats the proxy as a string in an environment variable and never decides where it is configured. Make the proxy part of your client's constructor, give it timeouts, and the failures become diagnosable.

The second decision is the IP itself. A scraper that sends clean, correctly configured requests from a datacenter range can still be blocked. We built MaskLabs around real US mobile carrier IPs, with city-level targeting, sticky and rotating sessions, and HTTP and HTTPS support, so the traffic your Net::HTTP code sends looks like the traffic it is meant to resemble.

Setup should not need a sales call. You manage credentials yourself, watch usage in the dashboard, and pay by data used, with no long-term commitment.

Jon

Get real mobile IPs behind your Ruby client

Once your Net::HTTP code is explicit about its proxy, the next question is capacity. MaskLabs plans are tiered by monthly data pool: Starter is $30.00 per month, Basic $100.00, Advanced $500.00 and Scale $1000.00, with prepaid data add-ons billed per GB and a free trial with limited data. Billing is metered per byte, and credentials can be managed for a whole team, which suits scraping and price-monitoring jobs that run from several Ruby workers.

Masklabs

You can compare the tiers on the MaskLabs pricing page, or see the US mobile proxies on real carrier IPs with targeting across 46 US cities. Because Net::HTTP only needs a host, port and credentials, switching an existing script over is a matter of changing those values.

Related setups in other languages

If part of your stack is not Ruby, the same ideas carry over: explicit proxy configuration, bounded timeouts and clear handling of 407. We have written them up for Python with HTTPX, Java and Apache HttpClient, C# HttpClient, Rust with reqwest and PHP cURL. Keeping one convention across services makes mixed-language data pipelines far easier to debug.

FAQ

Does Net::HTTP use a proxy automatically?

Yes, if the http_proxy environment variable holds a valid URI with a scheme. The default proxy address is :ENV, which triggers that lookup. If the variable is absent, the connection is direct, and passing nil as the proxy address turns the lookup off.

How do I pass proxy credentials in Ruby?

Use the fifth and sixth arguments of Net::HTTP.new or Net::HTTP.start: the proxy username and password. If you use the http_proxy URI instead, percent-encode any special characters in the credentials. Passing them as separate arguments avoids that step.

Why do I get a 407 from my Ruby proxy request?

A 407 means the proxy requires authentication and did not accept what it received. Check the username, password, host and port, and make sure encoded credentials were decoded before you passed them in. Ruby represents it as Net::HTTPProxyAuthenticationRequired, so you can handle it separately from other errors.

Can I use a proxy for HTTPS sites with Net::HTTP?

Yes. Create the client with the proxy arguments and set use_ssl = true. With a conventional proxy, the request goes through a CONNECT tunnel and TLS protects the connection to the target site.

How do I bypass the proxy for one request?

Create that client with nil as the proxy address, for example Net::HTTP.new(host, port, nil). That guarantees a direct connection regardless of environment variables. For a list of hosts that should skip the proxy, use the p_no_proxy argument.

Sources

Recommended