Web scraping
Web scraping is the automated collection of public data from web pages: prices, descriptions, stock, reviews.
A program opens pages from a list, finds the needed fields on them and saves them to a spreadsheet or database. This is how competitor prices, catalogs, search rankings and public listings are collected.
The main obstacle is the rate limit: a site will quickly notice thousands of requests from one address. So requests are spread across many addresses using rotation.
You may collect only public data, with regard for the site's rules and the law: personal data and content behind an account login are a separate matter.
Where to start and which proxy type to choose is described in the web scraping and data collection use case.
FAQ
Is web scraping legal?
Collecting public data is acceptable in most cases, but there are limits: copyright, personal data, the rules of the specific site. If in doubt, it is worth consulting a lawyer.
Which proxy do I need for web scraping?
Start with datacenter proxies: they are fast and inexpensive. If the site responds with CAPTCHAs or refusals, switch to residential ones.
Related terms
Residential $1.59, mobile $2.50, datacenter $0.80 per GB
Sign up in a minute, get access right after payment.