Edit MultiThreaded Ruby Script for Downloading Webpages

I want to make a script in Ruby which reads from a file of URLs and downloads all the text from all the web pages, and puts them into a big file in a folder called results. I want this to be multithreaded supporting at least 100 threads. I have some code, such as the code to read the URL text document, but need help with the multithreading and parsing the text from the web pages. Both HTTPS and HTTP sites must be supported, and this will need to be able to support thousands or million of URLs.

Habilidades: Ruby on Rails, Ruby, Captura de dados na web, HTTP

Veja mais: webpages database script, curl script downloading file, php script downloading pdf link, read webpages python script, downloading files php script, make script auto backup, make script automatically dump db python, make script autovoting, make script block usb port, make script check web page updates, make script defrag, make script fill data day, make script fmylife, make script friends opens passwords, make script mafia type games, make script metatrader, make script prompt rasdial password, make script router, make script scrape

Acerca do Empregador:
( 0 comentários ) Orange, United States

ID do Projeto: #30949866

3 freelancers estão ofertando em média $25 nesse trabalho


Python developer I have read your job description and I am pretty sure that I can complete every bit of your requirements. Further details and cost can be discussed in chat

$30 USD em 1 dia
(3 Comentários)

Hi there. I read your job description carefully and would like to work with you. I am very familiar with Ruby/HTTP and have rich exp of 10+years. have done many project with Ruby like your project. So if you hire me,I Mais

$20 USD in 3 dias
(0 Comentários)

Earlier I was working on the same kind of work to download files from US Govt website using multitreading. It can be done in Ruby or python. Please let me know your opinion.Thanks.

$25 USD in 3 dias
(0 Comentários)