You need to build a scraper for a website that searches for flights. Date and airport code need to be filed as input. the scraper needs to check if there are any flights available and what the fastest connection is in terms of changes. This scraper needs to be checked with me before moving to step 2.
You will be given a set of around 3500 airport codes and you need to check all combinations. You may need to use proxies (use proxybonanza for example) to not get blocked and multicurl to fetch many curls at once. If there are flights results for an airport combination you need to save that combination in a .txt file. 1 txt file for flights with 0 stopover, 1 txt for 1 stopover and 1 txt file for more than 1 stopover. The idea is that you pick the flight with the shortest duration for an airport combination and save that airport combination in the .txt file as follow: AMS-BCN. When finished you need to break the .txt file in separate files if they become larger than 50 mb.
Important: the script must be written in php and you need to work on this to finish it as fast as possible and not accept any other work before this task is finished.
41 freelancers are bidding on average $197 for this job
Hi There, I am a software engineer with skill of PHP, Python, Software Architecture, Web Scraping, Java, Asp.net, C#... I would like to complete your project as your requrement