Youtube Webscraping Using Selenium
1. What is Web scraping?
Web scraping refers to the extraction of data from a website. This information is collected and then exported into a format that is more useful for the user. Web Scraping also called web data mining or web harvesting, is the process of constructing an agent which can extract, parse, download and organize useful information from the web automatically.
2. Why Web Scraping?
In general, we may get data from a database or data file and other sources. But what if we need a large amount of data that is available online? One way to get such kind of data is to manually search (clicking away in a web browser) and save (copy-pasting into a spreadsheet or file) the required data. This method is quite tedious and time-consuming. In various fields Web Scraping is used: Scrape product details (price, images, rating, reviews, etc.) from retailer/manufacturer/eCommerce websites (Ex: Amazon, eBay, AliExpress, Alibaba, etc.) to show on own websites, to provide price comparisons, to perform a price watch on competing sellers, etc. Scrape property details and agent contact details from real estate websites. Scrape contact details of businesses as well as individuals from yellow pages websites. Scrape people profiles from social networks like Facebook, LinkedIn, etc. for tracking online reputation. Scrape hospital/clinic websites to build a catalog of health physicians including their contact details. Scrape reviews of products and places/hotels/restaurants. Crawl forums and communities to extract data from poets and authors. Scrape articles from various article/PR websites to feature in own website. Scrape data from various Government websites, most of which do not provide an easy way to download the data which they display.
