Scrape ecommerce web sites information
R$750-2250 BRL
Pago na entrega
ATTENTION!! ONLY CONTACT ME IF YOU REALLY HAD READ THE SCOPE!!
Development of service that obtains and compares values of products on the web.
Optimized service that allows to schedule and to make periodic searches in a determined quantity of sites of interest (initial scope: [login to view URL], [login to view URL], [login to view URL], [login to view URL], [login to view URL] , [login to view URL], [login to view URL], [login to view URL]). Performance is fundamental, considering that the websites have more than 15 thousand products each and I have the goal of obtaining all the data in less than 1 hour. The search service should be planned considering the possibility of including discount coupons when they are available for each store / category.
The service should also contemplate the possibility of searching for this information by the store's affiliates and / or service when available.
The service should be customizable with regard to the frequency of daily searches and sites involved in each search.
The data obtained must be kept in a database, where the site to which it refers, category, product, value, time of consultation, product image, discount coupon used (if any) and the service used The consultation, for the purpose of having a history.
Once products with lower values are persisted, they should be made available on a web page. (There will be more complex filters and a publisher staying for another phase of the project.
I do not need at this time a very far-fetched site (can be even developed in wordpress) because at this stage I am more concerned with evaluating the flow.
Take into consideration the need for variation of source ip and other policies necessary to avoid blocking access by the sites where the price searches will be performed.
This is the first phase of the project, there are at least two more phases with scope to be detailed after delivery of the current phase.
I also need the necessary infrastructure design, estimation of costs involved and suggestion of the hosting company.
The delivery of source code and database creation script are inherent in hiring.
Requirements:
- I'm looking for a committed professional.
- Preference for similar development experience and / or have ready routines that can be customized to meet my needs
- Payments according to deliveries made
- Profit sharing of the project can be negotiable as part or total of the payment as long as the resource is interested in following the project evolving the solution.
- Interesting experience using web crowler, at first I understand to be the best technology for the project but I am open to other options.
When contact me give me the answer for these questions:
1 - Did you read the scope?
2 - Did you understand the project?
3 - Do you have expirience with web crawlers?
4 - Which programming language would you use for the project?
5 - Do you work with ruby on rails?
6 - About performance, what is the estimated time to get all product information from each site? Taking into account that each site has more than 15 thousand products.
7 - On the need for variation of the source ip and other policies to avoid blocking access by sites where as price surveys are carried out, how do you want to solve the problem?
ID do Projeto: #13267411
