I have a webpage that I need scraped. I need to do it every hour. The site requires log in and captcha so my thought is I will log in with chrome or IE or FF and the cookie saves forever.. Once I log in, I could then run what you make to goto 1 url. On that url, is a list of titles, if you click the title, there is a tab on that page that has some extended info I need but its between " " so should be easy to grab that info.
Once the script runs through all 50 items on the page, it should then goto a webform and submit the data it collected in csv type format.
So, I need:
title,sitename(this is hardcoded),url(this is the url of the title),extended_info
The script you make would just paste that info and hit submit, I will do the backend part of that webform to add it to my database.
It should goto the url (we can hardcode that into the script), do its scraping, paste it to webform, then sleep for 60 minutes and start over.