Proxies for web scraping

Rotation on every request

Choose “Rotating (new IP per request)”: a single endpoint (port 823 for HTTP, 824 for SOCKS5) gives a new IP on every connection. Nothing needs to be switched in your script.

Datacenter or residential

Start with datacenter proxies: they're the fastest and cheapest. If a website returns captchas or 403 errors, switch to residential ones: they look like regular visitors.

Saving traffic

Traffic is billed by volume, not by the number of requests. Turn off loading of images, fonts and scripts you don't need, and use the website's API if it has one.

Error control

Your dashboard shows a log of requests and errors by site. If a site keeps returning errors, add it to the blocked list so you don't waste traffic.

How websites spot a scraper

Website protection rarely looks at just one signal. Most often it counts requests per minute from one IP, checks whether the address belongs to a datacenter or is already flagged as a proxy, and compares request headers with a normal browser. A script that fires requests without pauses, with the same User-Agent and no cookies, stands out even from a clean address.

So proxies solve only part of the problem: they spread the load and give you an address in the right country. The rest — pace, headers, retries — you set up in your script. No proxy provider can promise that a given site will never show a captcha, and neither can we.

Setup in five steps

1. Create a plan of the right type: datacenter for open sites, residential for sites that reject datacenter addresses.
2. Pick a country in the dashboard generator. The country is included in the price, while precise geo (city, region, ZIP, carrier) counts ×2. Turn it on only where the site's response really depends on the city.
3. Choose the IP mode. Port 823 (HTTP) or 824 (SOCKS5) gives a new address on every request. Ports 10000–20000 give a fixed IP on each port, held for 1 to 120 minutes (30 by default).
4. Download the list of lines in the format you need (.txt), for example username:password@gw.prokso.com:823 or host:port:username:password. If your script runs on a server with a static address, you can add it to the IP whitelist (up to 5 addresses per plan) and skip the username and password.
5. Connect it in code using our guides: Python requests, Scrapy, Playwright. For no-code data collection there is a guide for Octoparse. Before the run, check the line with the Proxy Checker.

Rotation or a fixed IP

Catalogs, product cards and open pages that need no login call for rotation on every request. Each page comes from a different address, and no single IP collects a suspicious number of hits. More about the modes is on the rotating proxies page.

A fixed IP is needed when the site ties actions into one session: logging in, a cart, pagination with a token, filters stored in cookies. Take a separate port from the 10000–20000 range for each thread and set the session duration with room to spare for the whole session. A residential address can change earlier if the device goes offline. The port then gets a new IP in the same country, so your script should be able to log in again.

There is also a third option without separate ports. In advanced mode you can add a session number to the username: on ports 823 and 824, requests with the same number then go through one IP for about 30 minutes.

Pace, 403 and 429 errors

Rotation is no substitute for a sensible pace. A few rules that save both traffic and time:

• 429 Too Many Requests means the site wants you to slow down. Use a pause that grows with each attempt (for example 2, 4, 8 seconds) and reduce the number of threads on that domain.
• 403 or a captcha means the address or the request itself looks suspicious. Retry through a new IP once or twice. If the error persists, check your headers and cookies or move that site to residential proxies.
• Cap your retries. Endless attempts to load a page that will never answer burn traffic. After 2–3 failures, move the URL to a separate queue and deal with it later.
• Spread threads across domains. A plan handles up to 1,000 concurrent connections, but dozens of threads on one small site is a bad idea for you and for the site.
• The anonymity filter in the pool settings drops IPs that sites have already flagged as proxy or VPN. It helps on sensitive sites where the share of 403s is noticeably high.

Some sites are closed to all customers: government resources, banks and payment systems. The list is on the blocked sites page.

How much traffic scraping takes

Traffic is counted in both directions, but most of it is the site's response. So the key question is what exactly you download: only the HTML, or the full page with images and scripts through a browser.

These numbers are rough: page weight varies a lot from site to site. The most accurate way is to run a small batch of 100–200 pages and check the usage in your dashboard. To stay within budget, set a traffic limit on the plan (per hour, day or week) that stops it or sends a warning. Datacenter proxies cost $0.80 per GB, residential $1.59. The price is the same for any volume; details are on the pricing page.

Which countries to choose

For global sites the country usually doesn't matter, and you can take any of the 195. But many stores and services show different prices, languages and product ranges depending on the visitor's country. Then take IPs from the country whose data you need: the USA for American marketplaces, Germany and the Netherlands for European ones, the United Kingdom for British ones, Poland and Ukraine for local ones. Mobile proxies are rarely needed for scraping, unless a site shows different content to users on mobile networks.

And an honest note about the pool: it's shared, so another customer may have used an address before you. That's why some IPs can hit a captcha right away on strict sites. This is normal, and a retry through a new address usually solves it. UDP isn't supported, but HTTP scraping doesn't need it.

Best proxies for web scraping: which type to pick

The best proxies for web scraping are the ones that get through on your target site at the lowest cost. Start with datacenter IPs: at $0.80 per GB they are the cheapest scraping proxies and enough for most public sites. If the site blocks datacenter ranges, point the same scraper at residential IPs ($1.59 per GB) — only the connection string changes. As a scraping proxy service we bill per GB with no subscription, so traffic waits on the plan until your scraper uses it.