I need a web scraper written for the following url:
[login to view URL]
All pages will need to be retrieved not just page one.
The data on this site changes and the number of pages will vary, however, we need to scrape data from all available pages.
The output should be a pipe (|) delimited file with the following column mappings:
origin_city --> data is located in the "Pick Up City" column
origin_state --> data is located in the "Pick Up State" column
ship_date --> data is located in the "Pickup Date" column, changed to the YYYY-MM-DD format
destination_city --> data is located in the "Drop Off City" column
destination_state --> data is located in the "Drop Off State" column
receive_date --> leave blank
trailer_type --> data is located in the "Equip Type" column
load_size --> add text "Full" to the column
weight --> data is located in the "Weight" column
length --> leave blank
width --> leave blank
height --> leave blank
trip_miles --> leave blank
pay_rate --> data is located in the "Rate" column
contact_phone --> leave blank
contact_name --> leave blank
tarp_required --> leave blank
comment --> leave blank
load_number --> data located in the "Comment" column; data located in the "Loads" column, add text "Loads=" before data; data located in the "Tarp" column, add text "Tarp=" before data
commodity --> leave blank
The first line of the output should contain all of the column headers.
Any field that contain no data should be left blank.
Please do not use words like "null" or "blank" in blank columns.
Below is a sample output of the first 5 columns using sample data:
The deliverable will be a Perl .pl file that must run on
Ubuntu Linux and must use Modern::Perl. The Perl .pl file
should be called '[login to view URL]' and the output file should be
called '[login to view URL]'
It will be scheduled in cron to run unattended every 15 minutes.
We suggest WWW::Mechanize but you are free to use other Perl libraries.
Please specific what language/OS/tools you will be using in your bid.
Also, please include the word "raccoon" in your bid so I know that
you read this description.
I can provide you a Perl scraping script (using WWW::Mechanize, HTML::TreeBuilder::LibXML etc) that will output into a pipe delimited file. Thanks. Roman
15 freelancers estão ofertando em média $156 para esse trabalho
Hi, The task is very doable. I understood the requirements clearly and I can start right away. I can also submit the data directly in Excel if you prefer that way. Hope to hear from you soon!
I'm Perfect at Scraping Big data from Websites to Spreadsheet or to some Database. I'm Good at Perl, Linux, Web Scraping. , Please Send Me a message so that we can discuss more about this project.