Rotating proxies in Python, in about ten lines

Rotation is simpler than most tutorials make it look. With a rotating endpoint you do not manage a list of IPs at all. You point at one host and the provider hands you a different IP per request.

Every snippet below uses HOST, PORT, USERNAME and PASSWORD as placeholders. There is no single ProxyPanda gateway address to copy from a blog post: the dashboard shows the host, port and credentials on each service's page, so take them from there.

The whole thing

import requests

PROXY = "http://USERNAME:PASSWORD@HOST:PORT"
proxies = {"http": PROXY, "https": PROXY}

for url in urls:
    r = requests.get(url, proxies=proxies, timeout=30)
    print(r.status_code, len(r.content))

That is it. With a residential service set to Randomize IP in the dashboard, each request goes out of a different IP.

To see the rotation with your own eyes, ask an IP echo service a few times:

for _ in range(3):
    print(requests.get("https://httpbin.org/ip", proxies=proxies, timeout=30).json())

Three calls, three different origin addresses.

Keeping the same IP for a few requests

Sometimes you need the IP to stay put: a login, then the pages behind it. Switch the residential service to Sticky IP in the dashboard and set how long the session should hold. The dashboard builds the credential with a session ID in it; copy the whole string as it is shown.

STICKY = "http://USERNAME:STICKY_PASSWORD@HOST:PORT"
sticky = {"http": STICKY, "https": STICKY}

Same session ID, same exit IP, for as long as the network can hold it within the lifetime you chose. Generate a new session in the dashboard when you want a fresh one. An exit IP belongs to somebody's home connection, so a sticky session can end early when that connection drops; write the login step so it can run again.

The two mistakes

Mistake one: reusing a Session and expecting rotation.

s = requests.Session()
s.proxies = proxies

Session keeps connections alive. For an HTTPS target, a kept-alive connection is the same tunnel through the proxy, so it is the same exit IP. Rotation cannot happen, because you never made a new connection. If you want a new IP per request, either do not use a session, or send Connection: close.

This is the single most common "your rotation is broken" report, and it is nearly always this.

Mistake two: not handling the retry.

Rotation means some requests will land on an IP the target already dislikes. That is normal and it is why rotation works: you retry and get a different one.

from urllib3.util.retry import Retry
from requests.adapters import HTTPAdapter

retry = Retry(total=3, backoff_factor=0.4, status_forcelist=[429, 502, 503])
s = requests.Session()
s.mount("https://", HTTPAdapter(max_retries=retry))

On our per-GB meter, a 429 or a 5xx from the site is a response like any other, so each retry costs the bytes of that response. They are usually small, a fraction of a real page, but a retry loop without backoff collects a lot of them, so keep the backoff_factor. Timeouts, resets and errors from our own gateway bill zero. Providers draw that line in different places, so check yours. Retrying without burning bandwidth goes further into which failures are worth a second attempt.

httpx, if you prefer async

import asyncio
import httpx

PROXY = "http://USERNAME:PASSWORD@HOST:PORT"

async def main(urls):
    async with httpx.AsyncClient(proxy=PROXY, timeout=30) as client:
        return await asyncio.gather(*(client.get(u) for u in urls))

results = asyncio.run(main(["https://httpbin.org/ip"] * 3))

Note the same caveat: AsyncClient pools connections. For a fresh IP each time, create the client per request, or use sticky sessions deliberately and switch sessions yourself.

Rotating your own list of static IPs

ISP and datacenter proxies work the other way round. You buy a set of fixed IPs, the dashboard lists each one with its port and credentials, and rotation is your job. It is one more line:

from itertools import cycle

pool = cycle([
    "http://USERNAME:PASSWORD@HOST_1:PORT_1",
    "http://USERNAME:PASSWORD@HOST_2:PORT_2",
    "http://USERNAME:PASSWORD@HOST_3:PORT_3",
])

for url in urls:
    proxy = next(pool)
    r = requests.get(url, proxies={"http": proxy, "https": proxy}, timeout=30)

Which product

If your target accepts datacenter IPs, a few of them cycled like this is usually the cheapest way to spread requests out, and you can pick the country they sit in. If you get captchas or empty pages, move to residential, which rotates for you but is billed by the gigabyte and does not let you choose the country. Do not start at residential because a tutorial told you to: most jobs do not need it. Current rates are on the pricing page.

Stuck on any of this? Paste it in Discord, someone will spot it in a minute.