Proxies for price monitoring

A new IP for every request

Choose “Rotating (new IP per request)” mode: a single endpoint (port 823 for HTTP, 824 for SOCKS5) gives a new IP on every connection. Your script just goes through one proxy, and the address switching happens on our side. See the Python requests guide for a code example.

Datacenter for volume, residential for protected sites

Most online stores serve pages to datacenter addresses without trouble. They're the fastest and cheapest, so start with them. If a particular site answers with a captcha or a 403, switch only that site to residential and keep the rest on datacenter.

Regional prices

Some stores show different prices and availability depending on the city. If you need the price for Kyiv or Dnipro specifically, turn on precise geo. Traffic with it counts ×2, so use it only for those sites.

A gentle load

Rotation doesn't replace good manners. Check robots.txt, pause between requests to the same site, don't point hundreds of threads at one store, and collect prices only as often as you really need. If the site has an open feed or API, take the data from there.

What it costs

You pay per gigabyte, not per request. Download only the HTML, without images, fonts and scripts: a product page then weighs tens of kilobytes, and thousands of checks fit into a small volume. Purchased traffic never expires.

How stores tell a price monitor from a shopper

Online stores and marketplaces protect their catalogs from bulk collection the same way other sites do. They count requests from one IP, notice datacenter addresses, and check headers, cookies and whether the client runs JavaScript. Large marketplaces are stricter than small stores, and that's where captchas show up most often.

Proxies spread requests across many addresses and give you IPs in the right country, but they don't make a script invisible. You set the pace, headers and error handling. Nobody can guarantee that a store will never limit access.

A working setup in five steps

1. Split your list of sites into two groups: those that serve pages without trouble, and those that answer with a captcha or a 403. Run the first group on datacenter proxies and the second on residential ones.
2. Create a separate plan for each group or each client. A plan has its own username, password, balance and traffic limit, so costs are easy to track separately.
3. Pick the store's country in the generator and the “Rotating (new IP per request)” mode: port 823 for HTTP or 824 for SOCKS5. More about rotation is on the rotating proxies page.
4. Connect your scraper using the guides for Scrapy, or Playwright if the price is loaded by a script. If you don't write code, Octoparse will do.
5. Check the line with the Proxy Checker and run a small batch of 50–100 products to see the real traffic usage.

Prices that depend on the city and the cart

Some stores show a price only after you choose a city or delivery method, and keep that choice in cookies. Rotation on every request gets in the way here: the next page opens from a new IP without the saved city. Take a fixed IP on ports 10000–20000 and hold it for as long as one store crawl lasts. Usually 10–30 minutes is enough.

If the store detects the city by IP, you need precise geo with a city or carrier. It counts ×2, so turn it on only for those sites. A residential address can change before the set time if the device goes offline. The port then gets a new IP in the same country, and the scraper has to pick the city again.

Errors and clean data

In price monitoring a bad answer is worse than no answer: a wrong price ends up in reports and decisions. A few rules:

• Check the page before saving. A 200 code doesn't mean success: the store may return a captcha page or a placeholder instead of the price. Look for the SKU, price and currency on the page, and retry if they're missing.
• 429 means slow down. Use a pause that grows with each attempt and reduce the number of threads on that store.
• 403 or a captcha: retry through a new IP once or twice. If that doesn't help, move the store to the residential group or turn on the anonymity filter in the pool settings.
• Cap retries at 2–3 attempts and leave failed products for the next run. That way traffic isn't spent on pages that aren't answering right now.
• Flag sharp jumps. If a price has changed several times over, check it again before acting on it.

How much traffic monitoring takes

The gap between the rows shows the main point: the download format saves more than the proxy type. If the store returns the price in HTML or through a call to its own API, don't open the page in a browser. Datacenter proxies cost $0.80 per GB, residential $1.59. Set a traffic limit on the plan and turn on email or Telegram alerts for when traffic runs low, so an overnight run doesn't stop unnoticed.

The figures in the table are approximate: a product page weighs tens of kilobytes on one site and hundreds on another. Run a small batch and check the exact usage in your dashboard. Purchased traffic has no expiry date, so traffic for several months ahead won't go to waste.

Crawl schedule

Not every product needs checking equally often. Prices on best-sellers and sale categories change daily, while rare accessories change once every few weeks. Split the catalog into groups: frequent checks for top products and direct competitors, weekly checks for the rest. This cuts the number of requests, and so the traffic, several times over without missing important changes.

Spread each run over time. A thousand pages of one store are better crawled over an hour with a few threads than in five minutes with all of them at once: the store won't notice a spike, and you'll get fewer captchas. Competitors' new products are easier to find through the sitemap or an open feed than through a full daily crawl of the catalog.

If the store's server supports conditional requests with If-Modified-Since or ETag headers, an unchanged page comes back as an almost empty response. On large catalogs this is a noticeable saving.

Countries for monitoring

The IP should be from the country whose prices you collect: stores often show a different currency or range, or redirect to another language version. For the Ukrainian market take Ukraine; for neighboring and European markets, Poland, Germany and France. Importers and sellers on international marketplaces will find Turkey and the USA useful.

The pool is shared, so some addresses may get a captcha right away on strict marketplaces. That's expected and solved by retrying through a new IP. Mobile proxies are rarely needed for price monitoring, only if a store shows different prices to mobile networks.