Scraper can use whatever method you choose as long as data is complete and entered into database correctly.
Typical scraper PATH:
[url removed, login to view] GOTO ->
http://realtytimes.com/rtmcrstate/"STATE"/"CITY"/ GOTO -> First "Reported By" section, Follow the "MORE" link, follow the "View My Newsletter" link, which will take you to this page: (example)
[url removed, login to view]
If you can capture the usenames you can scrape the data from their newsletters.
Append the Username onto [url removed, login to view] as in [url removed, login to view]
The data to be scraped includes:
Example: [url removed, login to view]
Script should check for and avoid duplicate records.
I already Have a Primary database of agents. The information captured here must comply with the structure of the parent database without losing integrety of the information. Information scraped here should suppliment the primary database.
The script must be in PHP file format that can be run as a cron job on a regular basis.
This type of script is the basis for most of our business. Therefore, the coder who strives to understand our goals and can write and implement these types of scripts consistently will go on to handle all such projects. (Writing and maintaining dozens of these scripts).
If that person also had the skills to manage our databases, a permanent contract and/or employment in the US would not be out of the question.