I would like to crawl and index sites related to a list of keywords/keyphrases, linked to a list of all provinces, cities and natural parks/areas in my country. I do have the complete list of countries, cities and natural parks in my country.
I’ve created (still adding a few terms) a list of keywords/keyphrases that will be added to the name of province/city/natural park in my country, so that we search Google for the best sites related to those terms, index them, and crawl each of those results in search of a contact email or emails.
The idea for this project would be creating a crawler application that will create its own database with this data:
1. Search Google for best sites under a group of keywords/keyphrases related to my country’s provinces, cities and natural parks. For instance: ‘cityX townhall’ ‘provinceY tourist office’, etc, for each of the provinces, cities, etc. The tool would go one by one on Google’s search results for each keyphrase, index it, save google PR of site, crawl the best sites in search for emails on these sites, save those emails and references to the site where it was taken from.
2. It should perform a search on Google for individual names of cities and provinces of my country and do the same we did with point 1.
3. It should perform a search on Google for web sites that contain the names of the all cities, provinces and natural parks in our country. It should search for .com, .es, .net, .org, having the name of the city/province/natural park. When found, add URL to the database, crawl it in search for emails, save emails and references, save PR of site.
4. Same as point 3 but with best domain names that contain name of city, for instance: [url removed, login to view], etc.
5. I’ll provide you with a list of keywords/keyphrases that will be linked to the name of province/city so that you can perform the searches and crawl results.
6. The tool should be able to be set up to add more keywords and keyphrases so that we can perform other searches in the future if we want to.
Please, send me a PM with references to previous works.
Looking forward to your bids and comments. Project would be started immediatly after been accepted.