Web Crawler / Spider

Description: Seeking a web crawler developer to develop a small website crawler for one auto classifieds site.

The web crawler/spider will have the following characteristics :

1. Primary purpose is to crawl this specific auto sales website for all auto listings Private and Trade email addresses and store the listing email addresses into an excel spreadsheet.


a) Crawler – A crawler which crawls the specified site and indexes autos listings email addresses. The crawler must be set up to crawl all the following AUTO listings makes and models: Cars 4x4s, Bikes, commercials, Plant and Leisure. Search, Private and Trade and by Region.

3. ADMIN CONSOLE - The software must have a management console enabling the following functions :

a) Automated Scheduling of crawler for target site [hourly, daily, weekly, bi-weekly, etc] for the crawler to run.

b) Reporting of crawl progress, results, log.

c) Exception handling – providing details of items not crawled.

d) Duplicate email address handling IS paramount, to delete duplicate listings email addresses.

e) Deletion handling to recognise that previously crawled listings/email addresses are no longer listed on the target site and to handle these accordingly by ignoring these listings into an inactive or archive table separate to the main listings

f) Backup functions to enable all of the email database to be backed up.

g) The ability to easily search for and edit and remove email address records.



- Previous experience of crawler / spider development and examples/references are essential.

- This project is Price & Quality Sensitive

Habilidades: ASP, Programação C, Linux, PHP, XML

Veja mais: web site development price, website commercials, web search remove, web develop php, web development description, web developer search, web developer price table, web developer price, web developer characteristics, web crawler developer, web-crawler, web commercials, spreadsheet on web, spreadsheet on the web, software web php developer, seeking address, search web developer, g php developer, e - trade, e trade, develop website auto search, development management web, developer console, commercials commercials, characteristics of web developer

Acerca do Empregador:
( 5 comentários ) Dublin, Ireland

ID do Projeto: #170593

13 freelancers estão ofertando em média $202 para esse trabalho


Please, check PM

$111 USD in 4 dias
(164 Comentários)

Hello! Talk to me if you have an appropriate budget... For better orientation of our past experience on Search Engines and Crawlers, please see our Feedback Page... All projects have been developed with excellent resul Mais

$300 USD in 15 dias
(39 Comentários)

we are an experts in making crawlers. please check pmb for our portfolio.

$300 USD in 15 dias
(36 Comentários)

hi. give me the url for a demo listing.

$200 USD in 3 dias
(80 Comentários)

Please see PMB

$200 USD in 4 dias
(38 Comentários)

Hi, i have developed such crawler (Japan Auctions Cars) and can complete your project in 5 days for only $100. See PM.

$100 USD in 5 dias
(125 Comentários)

See private message.

$240 USD in 7 dias
(47 Comentários)

Hi, I have the perfect solution. Please see PM.

$150 USD in 5 dias
(8 Comentários)

Please see PMB

$125 USD in 2 dias
(5 Comentários)

Will deliver a quality script, will crawl these autos and retrieve what you ask for

$250 USD in 5 dias
(1 Comentário)

I need specification

$200 USD in 15 dias
(3 Comentários)

Here is a link to my CV: [login to view URL] My hour rate is $30/h

$300 USD in 15 dias
(0 Comentários)

I am currently looking into your project. I would be happy to provide a sample. Would you please upload an example so that I may provide you with a sample? Please feel free to ask any questions. Thank you.

$150 USD in 7 dias
(0 Comentários)