I need a smart .NET screen scraper program in VB.NET (either windows or web) that will screenscrape other sites (mainly newspaper sites) gathering information.
Here are the details:
I provide it URLs from my database such as [url removed, login to view]
[url removed, login to view]
I need it to go a gather each section and at minimum put the data in a dataset so I can insert it into my SQL Server.
I need it flexable so it can change the URL incrementally, so in the url was /2003/October/19/biz/ I could set variables to change to URL automatically.
Also need to be able to tell it a specific tag to start and end with, along with start and end sections of the page.
Let me know if you are interested or already have something made similar. I'll explain what I am trying to do if you place a bid.
Need relatively quickly, including source.
1) Complete and fully-functional working program(s) in executable form as well as complete source code of all work done.
2) Installation package that will install the software (in ready-to-run condition) on the platform(s) specified in this bid request.
3) Exclusive and complete copyrights to all work purchased. (No GPL, 3rd party components, etc. unless all copyright ramifications are explained AND AGREED TO by the buyer on the site).
VB.NET (and ASP.NET if desired)