The aim of this project was to scrape data that is essential during the COVID pandemic. To accomplish this we targeted the prices of products essential during COVID (Masks, Sanitizer, etc) and culture jobs that are being lost. We then wanted to provide the data and an interface to access it.
These are the main tools we used during this project:
- Requests: Python library for http requests.
- Lxml: Python library for parsing html.
- Selenium Webdriver: Python library for scripting a web browser. It is useful for bypassing anti-botting measures.
- AWS Lambda and S3: Cloud services that we used for running scrapers and storing data. S3 was also used for hosting the static websites.
Creating during CodeLabs 2020
By: Nicholas Contreras, Daniel Katz, and Joseph Ngo
Mentor: Quentin Geddes
