Encerrado

Hadoop jobserver

We need a jobserver where we can dispatch jobs to a hadoop cluster. It is approximately the same we try to achieve that is discussed at: [url removed, login to view]@[url removed, login to view] We would like to target amazon mapreduce, see [url removed, login to view] and the project should include integration with this service.

The jobserver should be a standard war file that is deployable in any standard servlet container. Preferable it should be written using struts/hibernate/mysql/jquery/ext/guice or other similar open source technologies

We should be able to administrate jobs through an ui of the server that is accessed through a browser.

The first job that should be implemented on the server is for processing html pages fetched from real estate brokers. The job should run through a set of stored pages, group(reduce in hadoop terms) them according to broker/saleid/fetcheddate and dispatch them group wise to a component that we will supply, which will handle further processing.

Also the job should handle submit of htmlpages through an http based interface. For each htmlpage the following attributes should be stored:

* htmlcontent(copy of the page that is fetched, could be zipped, cleaned to minimize storage need, it will amount to between 50-150 kb pr entity)

* url

* fetchtime

* broker

* saleId(unique id for the broker)

* isparsed

After the job has processed an htmlpage the isparsed attribute should be modified accordingly and the htmlpage should be store if possible. The processing of a htmlpage group can fail and this should be stored in a database/htable/log and should be available for later retrieval by the ui.

Habilidades: Administração de Bancos de Dados, Engenharia, Java, Linux, MySQL, PHP, Arquitetura de software, Teste de Software, SQL

Ver mais: the container store jobs, the container store, struts.org, struts org, struts jobs, real estate jobs, pr jobs, php struts, jquery jobs, jobs for real estate, jobs at target, jobs at amazon, first copy technologies, dispatch jobs, container store jobs, container store, amazon jobs, amazon com jobs, 7 11 jobs, jobserver mysql, www.amazon.com jobs, mapreduce java, mail processing jobs, jobs pr, jobs in real estate

Acerca do Empregador:
( 93 comentários ) Denmark

ID do Projeto: #2986913

7 freelancers estão ofertando em média $959 para este trabalho

cygnusinnovation

See private message.

$722.5 USD in 21 dias
(4 Comentários)
5.7
infocular

See private message.

$1530 USD in 21 dias
(36 Comentários)
5.8
YoctoPetaBorg

See private message.

$425 USD in 21 dias
(26 Comentários)
4.8
diaconsultancyvw

See private message.

$1700 USD in 21 dias
(13 Comentários)
3.2
alienwebdevvw

See private message.

$425 USD in 21 dias
(6 Comentários)
2.9
melhorinfo

See private message.

$892.5 USD in 21 dias
(5 Comentários)
0.0
softwarepat

See private message.

$1020 USD in 21 dias
(7 Comentários)
0.0