Message Board Crawler


Using Visual C# (preferably not C++), write a windows program that will monitor a number of Yahoo discussion boards and download new messages as they are posted.

The user should be able enter and maintain a list of URLs of the boards to be monitored, and configure how often (e.g. every 60 seconds) the program checks each board. The program should show a status line of how many messages have been downloaded.

When the program detects a new message, it saves it to a database (SQL Server or mySQL). It is not enough to simply download the html; the program must parse the message and be smart enough to separate out all relevant fields in a post such as DATE, PostNumber, Alias, MessageBody and insert them into their respective columns in the database. A unique subject id must be assigned to each set of messages.

Part#2. Create a simple website to view the stored messages using ColdFusion, ASP, or PHP. It should display all of the messages which have been posted to each subject since the last time the page was viewed. e.g. "Here are the 5 subjects that have been posted to since the last time you visited." User can drill down on a subject to view new messages for that subject. Again, messages are retrieved from the SQL database...the user never needs to go to Yahoo to view retrieved messages.

Three formats that should be initially supported are

(1) finance forums such as

[url removed, login to view]

(2) Public groups such as

[url removed, login to view]


(3) news story boards such as

[url removed, login to view]

SUpporting multiple formats like this is not trivial!!! The parsing algorithm should be smart enough to adapt to slight changes in formats. A substantial amount of thought should be given to making it as generic as possible so that the program does not need to be re-written whenever Yahoo changes their interface.

This is a deceptively difficult project!

## Deliverables

1) Complete and fully-functional working program(s) in executable form as well as complete source code of all work done.

2) Installation package that will install the software (in ready-to-run condition) on the platform(s) specified in this bid request.

3) Exclusive and complete copyrights to all work purchased. (No GPL, 3rd party components, etc. unless all copyright ramifications are explained AND AGREED TO by the buyer on the site).

## Platform

Windows XP, all parsing source code in Visual C#. Web scripting may be done in ColdFusion, ASP, or PHP. Database can be mySQL or SQL Server.

Habilidades: ASP, PHP

Veja mais: message crawler, working of web crawler, website story boards, website story board, web site story board, story subject, story board website, story board web, smart board installation, simple story boards, mysql database download for windows, message interface, line algorithm, it groups yahoo, how to write news story, how to write a simple algorithm, how does algorithm work, how can l create website, generic components, download mysql database for windows, c++ parse html 5, code for making a website using html code, 3 story software, smart story, how to create a web crawler

Acerca do Empregador:
( 45 comentários ) United States

ID do Projeto: #3011228

Concedido a:


See private message.

$850 USD em 90 dias
(444 Comentários)

2 freelancers estão ofertando em média $786 para esse trabalho


See private message.

$722.5 USD in 90 dias
(7 Comentários)