Em Andamento

Custom Automation Script Needed

I need an script or application that can scan 12 million web pages that I specify and look for a specific string in the page html source and identity which pages contain this string and which pages do not.

## Deliverables

I have 12 text files with 1,000,000 urls in each file (12 million urls total), I need a script or application that can visit each of the 12 million urls and look for a specific word in the page html source and provide me with two list, one list that contains the urls which found the word and another list which contains the urls that do NOT contain the word I specify. Out of 12 million maybe 8 million will contain the keyword and 4 million will not. I have a strong windows 2008 server with 100mbps unlimited connection OR we can do this with PHP and a cron job on a linux server which I can obtain. PLEASE tell me how you plan to build this application. I would also like for the script to go fast, with at least 200 threads, remember its just loading the html source, no images or scripts on the page. It should also be able to auto-resume if it crashes or stops in the middle without having to start from 1 again. PLEASE ANSWER THESE QUESTIONS IN YOUR BID OR I WILL IGNORE IT COMPLETELY: 1) How fast can you get me a fully functional program that will not freeze up and have to be constantly restarted? 2) Will you use Windows Server 2008 compatible application or using PHP/MySql? 3) How will your script handle errors such as page timeouts, unable to load url, 404 errors, 500 error....etc? 4) Approx how long do you estimate to scan all 12 million pages? I need this completed ASAP so please keep this in mind. 1) All deliverables will be considered "work made for hire" under U.S. Copyright law. Employer will receive exclusive and complete copyrights to all work purchased. (No GPL, GNU, 3rd party components, etc. unless all copyright ramifications are explained AND AGREED TO by the employer on the site per the worker's Worker Legal Agreement). 2) Complete and fully-functional working program(s) in executable form as well as complete source code of all work done. 3) Deliverables must be in ready-to-run condition, as follows (depending on the nature of the deliverables): a) For web sites or other server-side deliverables intended to only ever exist in one place in the Employer's environment--Deliverables must be installed by the Worker in ready-to-run condition in the Employer's environment. b) For all others including desktop software or software the employer intends to distribute: A software installation package that will install the software in ready-to-run condition on the platform(s) specified in this project.

## Platform

Windows Server 2008 OR PHP/MySql

Habilidades: Programação C, PHP

Ver mais: two page resume, script resume, resume two pages, resume one page or two, programming resume, programming job sites, mind programming, html programming hire, how to do a job resume, how long should resume be, how long should a resume be, how do you get linux, hire the script, hire freeze, do you need a software to start programming, Custom Software Programming, can a resume be 2 pages, automation programming, how to answer job application questions, how to get a job in programming, windows automation, script automation, load 500 url, custom scripts, automation project

Acerca do Empregador:
( 241 comentários ) Riverside, United States

ID do Projeto: #2982005

Premiar a:

SerhiiA

See private message.

$85 USD em 5 dias
(79 Avaliações)
5.2

15 freelancers estão ofertando em média $238 para este trabalho

andreioniga

See private message.

$244.8 USD in 5 dias
(41 Comentários)
6.8
christiangnoth

See private message.

$425 USD in 5 dias
(20 Comentários)
5.1
jkormyk

See private message.

$233.75 USD in 5 dias
(23 Comentários)
4.9
desirablecoder

See private message.

$102 USD in 5 dias
(13 Comentários)
4.5
simona2002

See private message.

$242.25 USD in 5 dias
(20 Comentários)
4.0
alex89vw

See private message.

$93.5 USD in 5 dias
(19 Comentários)
3.9
mohamedsharaf

See private message.

$148.75 USD in 5 dias
(18 Comentários)
3.4
vw7677496vw

See private message.

$85 USD in 5 dias
(14 Comentários)
2.6
fbagim

See private message.

$102 USD in 5 dias
(6 Comentários)
1.5
linexteam

See private message.

$102 USD in 5 dias
(1 Comentário)
0.0
romaniancodervw

See private message.

$110.5 USD in 5 dias
(0 Comentários)
0.0
hmachine83

See private message.

$170 USD in 5 dias
(0 Comentários)
0.0
sundewsl

See private message.

$1275 USD in 5 dias
(0 Comentários)
0.0
path2727

See private message.

$153 USD in 5 dias
(0 Comentários)
0.0