A bot/spider to crawl the web for flash and non-mobile optimized websites

We'd like a spider/bot to crawl the web for old sites that need an update.

So we'd like the spider to crawl and filter sites according to some criteria:

- 100% flash sites

- Sites that don't have a mobile-friendly version (maybe looking for viewport media queries in css? )

- Sites that have an old date on the footer , like © 2014 (or lower, to be configurable)

IT should output the results to a spreadsheet.

So the results should look like:

[url removed, login to view]


[url removed, login to view]

No mobile version

[url removed, login to view]

older than XXXX

Some important things:

- Each of the three options should toggle on/off

- This should be cascading, meaning first it checks if the site is flash.

If it is, it adds the domain to the database and continue crawling. If not flash, it runs the second rule, if has a mobile version. And so forth.

- Limit the results according to a domain filter (example: only .[url removed, login to view] domains, for targeting australian sites only)

In your proposals please specify which language are you going to use, how the interface is going to be and provide as much information as possible. Similar projects that you've done in the past are welcome as well.



Habilidades: Extração de Dados, PHP, Shell Script, Arquitetura de software

Veja mais: spreadsheet on the web, site web flash, proposals for projects, example of a proposals, flash web based game bot, flash web browser mobile, non flash web template, non flash web animation, non flash web banners, web bot flash website, software write mq4, differences flash web site non flash website, software write chip epson, useful software write book, professional web crawler spider bot, software write web specs, free software write user guide, software write edid, free software write company profile, software write websites idea, software write book images, software write books, software write protection, free software write book, falsh non flash web site

Acerca do Empregador:
( 1 comentário ) São Paulo, Brazil

ID do Projeto: #10832617

10 freelancers estão ofertando em média $248 para esse trabalho


Hi I work towards providing reliable, relevant and robust IT solutions at most competitive prices to my customers. I ensure 100% customer satisfaction so lets start Thanks

$237 USD in 10 dias
(322 Comentários)

Hi, This is susan We could build a website. Payment could be released after you are completely satisfied with our output. our recent online store wordpress site in woocommerce. [login to view URL] https://w Mais

$266 USD in 10 dias
(55 Comentários)

Olá, Meu nome é Rui Pimentel, mais conhecido recentemente pelo meu envolvimento em projetos de crawling a sites com netflix, nike, supremenewyork e tantos outros trending neste momento. Eu li completamente a descriçã Mais

$147 USD in 3 dias
(35 Comentários)

Hi, I have read your specification and can definitely help you with your requirements. Please open a line of communication to discuss the project so that i can make it as per your vision and I am ready to start Mais

$155 USD in 10 dias
(48 Comentários)
$155 USD in 3 dias
(18 Comentários)

I went through the description and found the project interesting. I can make a spider for you to run checks on websites (three as per your specs: flash, mobile support and old footer date) and output you a spreadsheet Mais

$250 USD in 7 dias
(5 Comentários)

I am expert in python for Web scraping , API integration and lexical analysis. Also a good hand in web application designing using dijango and flask. I am a Oracle Certified professional (OCP) with experience in Orac Mais

$245 USD in 7 dias
(10 Comentários)

Hey. I got acquainted with you provided Info. The project is very interested. I understand perfectly the essence of the task. There is a lot of work experience, have experience in hozhih projects. I would be happy to i Mais

$555 USD in 14 dias
(7 Comentários)

Python programmator since 2012. I used to create a lot of applications about Web Scraping/Crawling with BeautifulSoup/Selenium/PhantomJS/lxml I love Computer&Hack and I'd like to share my passion helping someone with c Mais

$250 USD in 5 dias
(8 Comentários)
$155 USD in 3 dias
(6 Comentários)

Please see my reviews

$250 USD in 5 dias
(3 Comentários)

Hi, I'm a java developer with big experience in web and console application development. Can make this bot using java that will parse pages for that info.

$122 USD in 4 dias
(2 Comentários)