Language: Python
Editor: Atom
Repository: https://github.com/zuggyg/webscraping
Purpose:
The purpose of this project was to learn simple web scraping and develop scripts I can use for different contexts.
In this project I started by webscraping a simple website and aimed to make more sophisticated scripts as I moved from website to website with increasing requirement demands. With all the scripts the harvested information is saved into a CSV file.
The first script I made was for a Aldi’s Special Buys Page, that is a static website. The second script I made was for Coles’ search results that provides dynamic results and multiple pages. The third script I made was for Allhomes’ search results that has data that includes commas but also may or may not have complete data for each object.
Skills:
– Creating an easily adaptable script
– Developing to increasingly challenging requirements
– Creating a web scraping script
– Understanding website architecture HTML/CSS/PHP
– Writing data in CSV Files
Basics Practiced:
– Loops
– Arrays
– Functions
– Try/Exception Blocks
– Reading File Inputs and Save File Outputs
– Using open source libraries
Library Sources
Selenium: https://selenium-python.readthedocs.io/installation.html
Chrome Webdriver: https://sites.google.com/a/chromium.org/chromedriver/downloads