I need a crawler reading a website, extracting data I need, and saving these data to a mysql database, according to tables and rules I specify.
The crawler must run on my php hosting server, whose configuration I will specify (any further clarification must be asked by bidder, no use if the app runs only on another server, I need it to work on mine, as well as db mysql).
The crawler must run at all times, auto-restarting and keeping the process up, (a second triggering server is available, if needed). Data must be read continuously, at quite a fast pace (once per minute or so), compared with saved data, and if new data are present they must be appended to database under a format we will agree.
This is an ongoing project: I have other sites I need to extract data from, each will be a paid module, so the total earning of coder can get relevant, as time goes by.
Pls try to figure out all info you need before starting, and to ask all possible questions. My site is running on magento script.
For this first project, I have a sample file I will show you for guidance.