What is web scraping?

Web scraping at, its core, is collecting publicly available information from the internet for a specific purpose. It has taken form under many different names, such as following:

• Spiders
• Crawlers
• Bots

Although the name may carry a negative connotation, the practice of web scraping has been around since the beginning of the internet and has grown into various technologies and techniques. In fact, some companies have built their entire business on web scraping!