Python Requests Proxy: Setup, Authentication, and Rotation (2026)

Python Requests supports HTTP, HTTPS, and SOCKS proxies through the proxies argument. Put a proxy URL under the key matching the destination scheme, set an explicit timeout, and keep proxy credentials outside source control. Requests does not rotate proxy IPs itself: choose endpoints in your application or use a provider gateway whose documented product includes rotation.

This guide covers per-request and session configuration, environment variables, HTTP Basic authentication, SOCKS DNS behavior, HTTPS certificates, prepared requests, endpoint rotation, failure handling, and operational trade-offs.

1. Install Requests and choose your proxy type

For ordinary HTTP or HTTPS proxies, install Requests:

python -m pip install requests

For SOCKS4 or SOCKS5 URLs, install the optional extra:

python -m pip install 'requests[socks]'

A proxy URL must include a scheme and host. Common forms are:

Proxy URL	Use
`http://proxy.example:3128`	HTTP proxy. It can proxy both HTTP and HTTPS destinations.
`http://user:password@proxy.example:3128`	HTTP proxy with Basic credentials.
`socks5://proxy.example:1080`	SOCKS5; DNS is resolved by the client.
`socks5h://proxy.example:1080`	SOCKS5; DNS is resolved by the proxy.

The dictionary keys describe the destination URL scheme, while the values describe the proxy connection. Therefore an HTTPS destination commonly uses an http:// proxy URL in the https entry.

2. Send one request through an HTTP proxy

The most explicit configuration is per request. This follows the Requests proxy documentation.

A Requests call can send traffic through an authenticated HTTP or SOCKS proxy.

import requests

proxy_url = "http://proxy.example:3128"
proxies = {
    "http": proxy_url,
    "https": proxy_url,
}

response = requests.get(
    "https://example.com",
    proxies=proxies,
    timeout=(5, 20),  # connect timeout, then read timeout
)
response.raise_for_status()
print(response.status_code)
print(response.text[:200])

Always use a timeout. Requests has no default timeout, so a stalled proxy or origin can otherwise leave a worker waiting indefinitely. A tuple separates connection time from response reading time; choose values that fit your workload.

3. Authenticate to the proxy

For a proxy using HTTP Basic authentication, place the username and password in the proxy URL:

import os
import requests

proxy_user = os.environ["PROXY_USER"]
proxy_password = os.environ["PROXY_PASSWORD"]
proxy_host = os.environ.get("PROXY_HOST", "proxy.example")
proxy_port = os.environ.get("PROXY_PORT", "3128")

proxy_url = f"http://{proxy_user}:{proxy_password}@{proxy_host}:{proxy_port}"
proxies = {"http": proxy_url, "https": proxy_url}

response = requests.get(
    "https://example.com",
    proxies=proxies,
    timeout=(5, 20),
)
response.raise_for_status()

Use credentials supplied by the proxy operator. Do not commit them to a repository, paste them into a shared configuration file, or print the complete proxy URL in logs. Prefer a secret manager or injected runtime environment. If a password contains reserved URL characters such as @, :, /, or #, URL-encode the username and password before constructing the URL.

Proxy authentication is different from destination authentication. The auth= argument authenticates to the origin server; credentials embedded in the proxy URL authenticate to the proxy. Requests documents that credentials in the proxy URL take precedence over a manually supplied Proxy-Authorization header.

4. Use a Session for repeated requests

A Session reuses connections and can hold default proxy settings:

import requests

session = requests.Session()
session.proxies.update({
    "http": "http://proxy.example:3128",
    "https": "http://proxy.example:3128",
})

for url in ["https://example.com", "https://www.python.org"]:
    response = session.get(url, timeout=(5, 20))
    response.raise_for_status()
    print(url, response.status_code)

Session defaults are convenient, but they can surprise you when environment proxy variables are present. Requests recognizes http_proxy, https_proxy, no_proxy, and all_proxy, along with uppercase variants. Environment values may override values stored in session.proxies. When a particular request must use a known proxy, pass proxies=... directly on that request.

Inspect the process environment when debugging:

import os

for name in (
    "http_proxy", "https_proxy", "no_proxy", "all_proxy",
    "HTTP_PROXY", "HTTPS_PROXY", "NO_PROXY", "ALL_PROXY",
):
    value = os.environ.get(name)
    if value:
        print(name, "is set")  # do not print secrets

5. Understand `no_proxy` and prepared requests

no_proxy (or NO_PROXY) lists hosts that should bypass the proxy. A corporate runtime may set it globally, causing one hostname to connect directly while another uses the proxy. Check both lowercase and uppercase variables, and remove or adjust them in the process environment when your application owns proxy policy.

Advanced code that prepares a Request directly does not automatically apply environment settings. If environment behavior is intentional, prepare through a session and merge environment settings before sending:

import requests

session = requests.Session()
request = requests.Request("GET", "https://example.com")
prepared = session.prepare_request(request)
environment = session.merge_environment_settings(
    prepared.url,
    proxies={},
    stream=None,
    verify=None,
    cert=None,
)
response = session.send(prepared, timeout=(5, 20), **environment)
response.raise_for_status()

If you need deterministic behavior, pass the proxy mapping explicitly and set certificate options deliberately instead of relying on inherited shell configuration.

6. Use SOCKS5 and choose where DNS is resolved

After installing requests[socks], use a SOCKS URL exactly as you would an HTTP proxy:

import requests

proxies = {
    "http": "socks5h://user:password@proxy.example:1080",
    "https": "socks5h://user:password@proxy.example:1080",
}

response = requests.get(
    "https://example.com",
    proxies=proxies,
    timeout=(5, 20),
)
response.raise_for_status()

The distinction between socks5 and socks5h is DNS location. With socks5, your machine resolves the hostname before opening the SOCKS connection. With socks5h, the proxy resolves the hostname. Use the latter when the proxy should receive the DNS lookup, or when local DNS cannot resolve the target.

7. Keep HTTPS certificate verification enabled

Requests verifies HTTPS certificates by default, as described in its SSL verification documentation. An HTTPS destination can still require your client to trust a proxy’s root certificate, depending on how the proxy handles TLS.

import requests

response = requests.get(
    "https://example.com",
    proxies={
        "http": "http://proxy.example:3128",
        "https": "http://proxy.example:3128",
    },
    verify="/etc/ssl/certs/company-proxy-ca.pem",
    timeout=(5, 20),
)
response.raise_for_status()

You can also set REQUESTS_CA_BUNDLE or CURL_CA_BUNDLE to a trusted CA bundle. Do not use verify=False as a routine fix: it accepts untrusted or mismatched certificates and permits man-in-the-middle attacks. Reserve it for controlled testing where the risk is understood.

8. Rotate proxies in application code

Requests accepts a proxy but has no built-in IP rotation. Application-managed rotation means selecting a different endpoint before each request. A provider-managed gateway keeps one URL in your code while the provider selects exit IPs according to its product settings.

Application-managed rotation needs endpoint selection, failure tracking, and cooldowns.

Round-robin rotation

from itertools import cycle
import requests

proxy_urls = [
    "http://user:pass@proxy-a.example:3128",
    "http://user:pass@proxy-b.example:3128",
    "http://user:pass@proxy-c.example:3128",
]

if not proxy_urls:
    raise ValueError("At least one proxy is required")

proxy_cycle = cycle(proxy_urls)

with requests.Session() as session:
    for url in [
        "https://example.com/one",
        "https://example.com/two",
        "https://example.com/three",
    ]:
        proxy = next(proxy_cycle)
        response = session.get(
            url,
            proxies={"http": proxy, "https": proxy},
            timeout=(5, 20),
        )
        response.raise_for_status()
        print(url, response.status_code)

Random selection with retryable proxy failures

import random
import time
import requests

proxy_urls = [
    "http://user:pass@proxy-a.example:3128",
    "http://user:pass@proxy-b.example:3128",
]


def get_with_proxy_failover(url, attempts=3):
    if not proxy_urls:
        raise ValueError("proxy_urls cannot be empty")

    last_error = None
    for attempt in range(attempts):
        proxy = random.choice(proxy_urls)
        try:
            response = requests.get(
                url,
                proxies={"http": proxy, "https": proxy},
                timeout=(5, 20),
            )
            response.raise_for_status()
            return response
        except (requests.exceptions.ProxyError,
                requests.exceptions.ConnectTimeout,
                requests.exceptions.ReadTimeout,
                requests.exceptions.ConnectionError) as error:
            last_error = error
            if attempt + 1 < attempts:
                time.sleep(2 ** attempt)

    raise RuntimeError("All proxy attempts failed") from last_error

response = get_with_proxy_failover("https://example.com")
print(response.status_code)

Round-robin is predictable; random choice can select the same endpoint repeatedly. Neither method proves that an endpoint is healthy. Track failures, temporarily remove unhealthy endpoints, and restore them after a cooldown. Avoid retrying non-idempotent operations unless you know whether the origin received the request; a timeout does not prove that it did not.

Provider-managed rotation

Some proxy products expose one gateway URL and rotate upstream addresses, while others provide dedicated endpoints and require your code to select them. Verify the provider’s current rotation cadence, sticky-session controls, geography, authentication rules, and account limits. A sticky session may be necessary when a site expects a consistent IP across several requests. Rotation does not guarantee access, reliability, or compliance with a destination’s terms and rate limits.

9. A production-ready request wrapper

import logging
import os
from urllib.parse import quote
import requests

log = logging.getLogger(__name__)


def proxy_url():
    user = quote(os.environ["PROXY_USER"], safe="")
    password = quote(os.environ["PROXY_PASSWORD"], safe="")
    host = os.environ["PROXY_HOST"]
    port = os.environ.get("PROXY_PORT", "3128")
    return f"http://{user}:{password}@{host}:{port}"


def fetch(url):
    proxy = proxy_url()
    try:
        response = requests.get(
            url,
            proxies={"http": proxy, "https": proxy},
            timeout=(5, 30),
        )
        response.raise_for_status()
        return response
    except requests.exceptions.ProxyError:
        log.warning("proxy connection or authentication failed")
        raise
    except requests.exceptions.Timeout:
        log.warning("request timed out; the origin may still have received it")
        raise


if __name__ == "__main__":
    result = fetch("https://example.com")
    print(result.status_code, len(result.content))

Log the endpoint identifier rather than credentials. Record latency, exception type, response status, and retry count. For concurrent workers, give each task a clear proxy-selection policy and avoid sharing mutable rotation state without synchronization.

10. Troubleshooting common errors

Symptom	Likely cause	Fix
`ProxyError` or “407 Proxy Authentication Required”	Wrong host, port, scheme, or credentials.	Test the endpoint independently, URL-encode credentials, and confirm the proxy operator’s authentication method.
Requests appears to ignore `session.proxies`	Environment variables override session values.	Inspect proxy variables and pass `proxies=` on the individual request.
One hostname bypasses the proxy	`no_proxy` or `NO_PROXY` matches it.	Remove the matching entry or define an explicit per-request mapping.
“Missing dependencies for SOCKS support”	The optional extra is not installed.	Run `python -m pip install 'requests[socks]'` in the active environment.
DNS failure with SOCKS	Local DNS cannot resolve the hostname, or DNS location is wrong.	Try `socks5h://` so the proxy resolves the hostname.
SSL certificate verify failure	The proxy or destination uses a CA your process does not trust.	Install the correct CA bundle and set `verify`, `REQUESTS_CA_BUNDLE`, or `CURL_CA_BUNDLE`.
Requests hang indefinitely	No timeout was supplied.	Set connect and read timeouts, then handle `Timeout`.
Only some requests fail after rotation	An endpoint is unhealthy, rate-limited, or has different geography.	Track failures per endpoint, use cooldowns, and verify provider behavior before retrying.
Origin returns a block page	The destination rejects the proxy, traffic pattern, or credentials.	Respect the destination’s rules and rate limits; rotation is not a guaranteed bypass.

11. Performance, reliability, and cost considerations

Connection reuse: A Session can reuse connections to reduce setup overhead. Do not assume reuse across different proxy endpoints.
Timeout budgets: Set a total request budget. Connect, TLS negotiation, proxy authentication, and origin reading all consume it.
Retries: Retry transient proxy and connection errors with bounded exponential backoff. Do not blindly retry writes or every HTTP status.
Pool health: Keep endpoint failure counts and cooldown timestamps. A larger list is not automatically a healthier list.
DNS and geography: SOCKS DNS mode and provider exit location can change which address is reached. Test the behavior required by your application.
Credential safety: Environment variables are better than hard-coded secrets, but secret managers and redacted logs are stronger controls.
Provider pricing: Proxy cost depends on the provider’s product, traffic, geography, and session settings. Confirm current terms before estimating a per-request cost.

12. Or skip the browser setup

If your actual goal is to capture pages as images or PDFs, you do not need to maintain a browser, proxy pool, cookie handling, popup removal, or screenshot timing logic. ScreenshotNeo is a website screenshot API and MCP server. Its clean capture flow accepts cookie and consent banners, removes more than 60 known consent platforms plus newsletter popups and chat widgets, and lets you turn each step off.

Only clean shots are billed. Bot checks or CAPTCHAs, blank pages, timeouts, failed loads, and cache hits cost nothing, and each response reports the result with X-Page-Verdict and X-Billed headers. The MCP server exposes take_screenshot, get_page_info, and capture_pdf for Claude, Cursor, and other MCP clients.

One GET request returns PNG, JPEG, WebP, or PDF. See the ScreenshotNeo API documentation for all options.

cURL

curl -G "https://api.screenshotneo.com/v1/shot" -d access_key=YOUR_API_KEY --data-urlencode url=https://stripe.com -o shot.webp

Python

import requests
r = requests.get("https://api.screenshotneo.com/v1/shot", params={"access_key": "YOUR_API_KEY", "url": "https://stripe.com"}, timeout=90)
open("shot.webp", "wb").write(r.content)

Node.js

const q = new URLSearchParams({ access_key: 'YOUR_API_KEY', url: 'https://stripe.com' }); const res = await fetch(`https://api.screenshotneo.com/v1/shot?${q}`);

ScreenshotNeo also supports full-page capture with lazy images loaded, CSS-selector element capture, dark mode, device presets and custom viewports, retina scale, PDF paper sizes and page ranges, custom CSS and JavaScript, clicks, selector or network-idle waits, blocked requests and resource types, headers, cookies, user agents, Authorization, timezone, geolocation, transparent backgrounds, resizing, chosen cache TTLs, signed links, asynchronous jobs with signed webhooks, bulk capture for up to 100 URLs, a usage API, and an OpenAPI specification. Its parameter names match those used by other screenshot APIs to make switching easier.

There is a free plan with 1,000 screenshots per month and no card. Paid plans start at $5 for 3,000 shots; every feature is available on every plan. Create your free ScreenshotNeo account.

13. FAQ

How do I use a proxy with Python Requests?

Pass a mapping such as {"http": "http://proxy.example:3128", "https": "http://proxy.example:3128"} to requests.get(..., proxies=proxies). Include a scheme and set a timeout.

Can I use `auth=` for proxy credentials?

No. auth= is for the destination. Put proxy credentials in the proxy URL or use the proxy authentication method documented by its operator.

Does Requests rotate proxies automatically?

No. Select endpoints in your code, or configure a provider gateway that documents rotation or sticky sessions.

Should I use `socks5` or `socks5h`?

Use socks5 when the client should resolve DNS. Use socks5h when the proxy should resolve DNS.

Why does a timeout not prove the request failed?

The connection can time out while the origin is still processing or has already received the request. Treat retries of non-idempotent operations carefully.

Where can I find the current Requests behavior?

Check the version-specific Requests documentation and your proxy provider’s current product settings before deployment.