Price comparison

Another known use case is to find specific products or services sold through various websites and track their prices. You would be able to see who sells the item, who has the lowest price, or when it is most likely to be in stock. You might even be interested in similar products from different sources. Having a web scraper periodically visit websites to monitor these products and services would be easily solve this problem. This is very similar to tracking prices for flights, hotels, and rental cars as well.

Sites like camelcamelcamel (https://camelcamelcamel.com/) build their business model around such a case. According to their blog post explaining how their system works, they actively collect pricing information from multiple retailers every half hour to every few hours, covering millions of products. This allows users to view pricing differences across multiple platforms, as well as get notified if the price of an item drops. 

This type of web scraper requires very careful parsing of the web pages to extract only the content that is relevant. In later chapters, you will learn how to extract information from HTML pages in order to collect this information.