Find Jobs
Hire Freelancers

Show me how to set up and run a simplified streaming pipeline on AWS - Spark, Kafka, Cassandra ($50-$100)

$50-100 USD

Em Andamento
Publicado há aproximadamente 5 anos

$50-100 USD

Pago na entrega
Beginning Big Data developer here who knows a little AWS, Hadoop, Spark, Python and Java looking for someone who can take me through the process of setting up a Spark + Kafka + Cassandra (or maybe MongoDB) streaming pipeline on AWS with multiple nodes (say, around 5 nodes, with EMR or EC2 or whatever makes the most sense). This is for getting myself started with setting up and running a semi--semi-realistic streaming pipeline to play around with for my self-education purposes, not for production, so a very basic "hello world" type setup doing the basics is what I am looking for. In other words, Kafka plus Spark streaming to process the input and put it into db, without anything too fancy in addition besides some trivial analytics or computations. You can make up your own problem to solve or re-use something you've already done! Success would be giving me reproducible instructions and very basic code (Java, Python or Scala) which I can take and successfully run myself. I don't have any hard requirements for Kafka and Spark redundancy, but it would be good to know what to do if I want to scale this out more. Pipeline automation like Airflow or NiFi would be a nice bonus but not necessary. Try to give me something that should be simple enough for a software engineer with very little background info to do without requiring more than 2 or 3 emails to get clarifications. Or maybe I'm asking too much for a beginner to take on? If so, let me know something else similar but more reasonable! But it must be a multi-node streaming config ... I can probably do anything simpler on my own. I'm hoping the cost I'm offering will cover the AWS time setting this up and verifying it, but I can be flexible. (Just running the pipeline for a few minutes and verifying it works is good enough). If the consensus is that the fee is too low then I will certainly accept something higher. If we get into debugging the setup, I will pay any additional AWS fees necessary you may incur. Let me know what you think you do!
ID do Projeto: 19082932

Sobre o projeto

7 propostas
Projeto remoto
Ativo há 5 anos

Quer ganhar algum dinheiro?

Benefícios de ofertar no Freelancer

Defina seu orçamento e seu prazo
Seja pago pelo seu trabalho
Descreva sua proposta
É grátis para se inscrever e fazer ofertas em trabalhos
Concedido a:
Avatar do Usuário
Hi Sir, I have 5 years + experience in spark kafka hadoop yarn and in other big data technologies. I have done setups for the many enterprises. I can assure you the work and learning as well.
$110 USD em 10 dias
0,0 (0 avaliações)
0,0
0,0
7 freelancers estão ofertando em média $177 USD for esse trabalho
Avatar do Usuário
Hi there, I have read your project description and i'm confident i can do this project for you perfectly.I still have a few questions. please leave a message on my chat so we can discuss the budget and deadline of the project. Thanks . .
$85 USD em 3 dias
4,8 (25 avaliações)
6,2
6,2
Avatar do Usuário
do kindly share the requirements in a zip file. you can reach out to me over chat to discuss further
$166 USD em 5 dias
4,8 (26 avaliações)
4,5
4,5
Avatar do Usuário
Hi, Coincidentally I worked on a similar streaming pipeline for a project that had TBs of data flowing through multiple asynchronous jobs. Would love to talk more about it and also discuss more about your requirements. Thanks, Sajid
$111 USD em 2 dias
5,0 (2 avaliações)
1,6
1,6
Avatar do Usuário
I am working as a data engineer and my current project is creating real time recommendation pipeline with the same technologies you want to learn.
$130 USD em 8 dias
0,0 (0 avaliações)
0,0
0,0
Avatar do Usuário
i am a hadoop bigdata cloud expert..
$83 USD em 10 dias
0,0 (0 avaliações)
0,0
0,0

Sobre o cliente

Bandeira do(a) UNITED STATES
Scottsdale, United States
0,0
0
Método de pagamento verificado
Membro desde mar. 29, 2019

Verificação do Cliente

Obrigado! Te enviamos um link por e-mail para que você possa reivindicar seu crédito gratuito.
Algo deu errado ao enviar seu e-mail. Por favor, tente novamente.
Usuários Registrados Total de Trabalhos Publicados
Freelancer ® is a registered Trademark of Freelancer Technology Pty Limited (ACN 142 189 759)
Copyright © 2024 Freelancer Technology Pty Limited (ACN 142 189 759)
Carregando pré-visualização
Permissão concedida para Geolocalização.
Sua sessão expirou e você foi desconectado. Por favor, faça login novamente.