Website Webcrawler App

PROJECT OVERVIEW -- Website Webcrawler

Website Webcrawler receives as input --

a. the url of a website

b. a list of keywords separated by commas

upon this input, the app crawls only those links WITHIN the website, and returns specific intelligence on that website. The output intelligence includes --

a. All Email Addresses that has the website as the domain name. So, if the website is [url removed, login to view], it would return only email addresses with [url removed, login to view] as the domain root.

b. All Webpages the has KEYWORD METADATA, URL LINKS, or the CONTENT that matches one or more of the keywords provided. For example, if one of the keywords were "Search," you would flag any webpage that had the word "Search" in their metadata, in its url, or within the page content within that website.

c. A list of All Forms and the Action= post page for that form.


Upon initiation, the application --

a. Using the parameters set in a config file to connect to a Remote or Local Database. The config file would have the following parameters (I prefer xml but name / value also works) --

DBName: xxy

DBHost: ip address

DBUser: xxx

DBPassword: yyy

DBTrustedConnection: False

IF DBTrustedConnection is True, it connects locally to DBName, if False, it connects to DBName using DBHost, DBUser, and DBPassword

b. In a CONTINUOUS LOOP manner, the application would call a Stored Procedure (GetWebsite) in SqlServer Database with NO PARAMETERS. The continuous loops ends once NO RESULTS are returned.

c. The results from the SP are returned in a 1 Row SELECT STATEMENT / dataset with the following columns--

i. WebsiteID

ii. URL

iii. Keywords

d. You would process this result set according to the logic in the above OVERVIEW SECTION and return the results in PostWebsite SP.

e. The PostWebsite SP would have the following input parameters --

i. WebsiteID

ii. URL

iii. EmailList -- XML FORMAT


<email emailID="" />

<email emailID="" />


iv. Formlist -- xml formatted list of forms, such as


<form webpage="" action=""/>

<form .../>


v. KeywordList -- xml formatted webpage list of keywords matched, such as


<keyword name="search" matchTypeID="1" webpage="abc/search.aspx"/>

<keyword .../>


MatchTypeID=1 (keyword was found in Metadata keywords)

MatchTypeID=2 (keyword was found in URL)

MatchTypeID=3 (keyword was found in Webpage Content)

f. Upon completion of PostWebsite, application would call GetWebsite for the next set of Website info.

That is all folks!! Simple, right?? :-)


Habilidades: Captura de dados na web

Veja mais: web scraping process, value website, true results, scraping web content, scraping email addresses from the web, root info, matches list, list matches, found app, formatted website, email database web scraping, app works, action website, webcrawler app, www.webcrawler.com, scraping a website, webcrawler search, ABC, website to app, website scraping, website from app, Website and app, Webcrawler, web intelligence, technical website

Acerca do Empregador:
( 77 comentários ) Beverly Hills, United States

ID do Projeto: #561458

14 freelancers estão ofertando em média $335 para esse trabalho


We can help in your project, please check PMB to see our related experience.

$250 USD in 5 dias
(32 Comentários)

looking forward to work on this project

$300 USD in 6 dias
(26 Comentários)

Hi sir, Let me do it NOW! Thanks, Kimi.

$360 USD in 8 dias
(64 Comentários)

Dear Customer! I am very experienced with this kind of job. Please see PMB for examples of my previous work in this field. Very interestEd in your project. Ready to start immediately and finish as soon as possible. Mais

$350 USD in 3 dias
(20 Comentários)

Kindly check PM for more details.

$325 USD in 5 dias
(6 Comentários)

Hi, Please check your PM. THakns

$250 USD in 2 dias
(7 Comentários)

I did similar tasks, and have all sufficient knowledge to implement you task.

$250 USD in 4 dias
(4 Comentários)

Hi, i am having more than 3 years experience in software and web development. i can do this job easily. it will be better to chat on any of IM, so please go to my GAF profile and find my web link and contact details. Mais

$400 USD in 4 dias
(5 Comentários)

Kindly check PM once.

$250 USD in 4 dias
(2 Comentários)

Greetings. I have strong experience building and designing web crawlers. PMB for more info.

$300 USD in 7 dias
(0 Comentários)

Hello. We have a lot experience in such kind of projects. We are ready to start at any time. Thanks.

$650 USD in 7 dias
(0 Comentários)

Hi,i am able to do this job. Thanks!

$300 USD in 4 dias
(1 Comentário)

Respected sir, I possess 5 years of experience in ASP.NET, C#.NET, SQL Server, AJAX, Programming, Analysis and Developing. If I can provide you with any further information on my background and qualifications, pl Mais

$400 USD in 15 dias
(0 Comentários)

please check PMB.

$300 USD in 5 dias
(0 Comentários)