Create a daily report that tracks the Daily Deal activity by specified markets. We have a set list of roughly 20 URLs to be crawled.
We'd like to have this written in Python and use SQLite as the persistent datasource. Please use the BeautifulSoup library ([url removed, login to view]) for the screen scraping technology.
Report will follow a set Excel template which includes Date, Region, Deal Title, Retail Price, Sale Price, and Number Sold
This report must be run every day. "Number Sold" must be collected after 11:30pm local time. Other data can be collected at any time during the day.
If a source isn't able to be crawled, delay 5 minutes and try again. If it fails 3 times, it will send an automated email.
[url removed, login to view] requirements notice: As originally posted, this bid request does not have complete details. Should a dispute arise and this project go into arbitration "as is", the contract's vagueness might cause it to be interpreted against you, even though you were acting in good-faith. So for your protection, if you are interested in this project, please work-out and document the requirements onsite.