HTTP headers

Along with the page address, a browser sends a dozen headers: User-Agent, Accept-Language with the language, Referer with the page you came from, Cookie with cookies.

Websites check whether the headers are consistent with each other and with the address. If the IP is from Poland but the browser language is different and the time zone is from another continent, the site sees a contradiction.

A regular proxy does not change headers: your program sends them. But some proxies add their own, such as Via or X-Forwarded-For, and thereby give themselves away. More on this in the transparent proxy article.

Through a tunnel to HTTPS sites, headers travel encrypted, and the proxy cannot see them.

How to see the headers your program sends

In a browser, open DevTools, the Network tab, click a request and look at the Request Headers section. In the console, run curl with the -v flag: lines starting with > are what curl sent, lines with < are what came back.

Libraries have their own defaults. Python requests sends User-Agent: python-requests/2.x, curl sends curl/8.x or similar, depending on the version. Such values tell a site right away that the request is automated, and the site may limit it more strictly.

Which headers come from you and which from the proxy

The Proxy-Authorization header carries the plan username and password to the proxy. For HTTPS sites it goes only in the CONNECT request to the proxy, so the site does not receive it. The proxy checker shows which proxy headers reach a site, and the X-Forwarded-For article explains IPs in headers.

Consistency between headers

Chromium-based browsers send Sec-CH-UA headers with the browser name and version alongside User-Agent. If they name different versions, the request looks contradictory. Over HTTP/2 header names are always sent in lower case, so do not be surprised to see user-agent instead of User-Agent in logs. Accept-Language should match the language you expect to see: the server often uses it to choose which version of the page to return.