429 Too Many Requests Error

What the error means

The 429 code is sent by the site, not by the proxy. It is a rate limit: the server counts requests from one address, account or key and starts refusing after a threshold.

The response often has a Retry-After header — the number of seconds after which you can retry. The limit is usually temporary and lifts on its own if you reduce the load.

Causes

• Too many requests from one address. A sticky IP or a long session quickly hits the site's limit.
• The limit is tied to an account or a key. Then changing the address will not help: the counter runs per account.
• Requests go without pauses. A steady stream with equal intervals looks automated.
• Parallel threads hit one page. Dozens of simultaneous connections to one URL of the site trigger the limit fastest.

How to fix it

1. Lower the rate: add pauses between requests and make them uneven.
2. Respect Retry-After: do not retry the request before the stated time.
3. Turn on rotation: port 823 (HTTP) or 824 (SOCKS5) gives a new address for every request. See the rotating proxies page for details.
4. Reduce the number of parallel threads to one site.
5. If the limit is per account, spread the work over time: changing the address does not lift such a limit.
6. Save the data you have already received so that you do not request the same thing again.

How to check that it works

Send one request after a pause of a few minutes. If it goes through but 429 comes back at the previous pace, you have found the site's threshold; keep your pace below it.