Create an async multiprocessing data pipeline application which caters to the following requirements(python) -- 2
₹600-1500 INR
Fechado
Publicado há mais de 3 anos
₹600-1500 INR
Pago na entrega
1.) Perform CRUD operations in MongoDB or Postgres
2.) The data pipeline application should be able to store the data output generated at each stage of the multi-stage data pipeline.
3.) The various stages of this pipeline are -
1.) Data Ingestion
The application should read JSON/CSV data from a specific folder and save it into MongoDB or Postgres. CRUD capabilities should be provided as well.
2.) Data Transformation
Provide an API function through which a user can query the data, perform basic transformation like trimming, lowercase, formatting, etc and save the data back into the DB as a new record with relevant checks.
3.) data availablity as a query
Provide an API function through which a user can query the data set.
4.) Use either FASTAPI or Flask for the API services
5.) sample dataset attached