Stagecoach Software Pty Ltd is developing a website to contain statistical profiles of a particular industry in Australia. Our stack is Ruby on Rails 4.1.2 and MySQL 5.5. We run a 3 server configuration on Engine Yard (Gentoo) with dedicated instances for our db, web app and utilities.
Our database receives about 30,000 new rows per day through scraping operations. We have about 22 scrapers that run at least once per day on cron jobs on our utility instance. We want to convert the scrapers from cron jobs to run in a rails queue to make better use of our system resources.
Bidders should state what rails processes they would use or recommend to achieve this objective and why.
There is a steady pipeline of projects available to the freelancer who completes this job successfully and promptly.
10 freelancers estão ofertando em média $598 para esse trabalho
Hello. I hope scrapers written in Ruby? I plan use Sidekiq (together Redis) for to achieve goals. Sidekiq is a mature solution that we used a lot of time in their projects.