ETL Jobs

I’m standing up a brand-new business-intelligence stack in Snowflake and need a reusable migration framework built with dbt and solid SQL. The framework has to pull from two kinds of sources—existing relational databases and the steady stream of CSV/Excel flat files our teams export—then land, cleanse, validate, and automate each load all the way into analytics-ready tables. Here’s what I’m after: • A dbt project structured for easy extension, complete with models, seeds, macros, and tests that enforce data-quality rules at every stage. • Parameterised SQL and Snowflake tasks/procedures that orchestrate fully automated loads, from raw landing zones through staging to final marts. • Built-in monitoring and reporting—leveraging dbt tes...

I have an SSIS package that pulls data from an Oracle database through an connection and loads it downstream for ETL processing. The transfer works, but network-level performance is far below what I need. Here is what I’m looking for: • Diagnose the current ADO.NET-to-Oracle connection, identify any network, buffer, or packet-size bottlenecks, and benchmark the baseline throughput. • Tune the SSIS data flow (buffers, rows per batch, commit size, async settings, etc.) and, if necessary, adjust the Oracle driver or provider configuration. • Provide an updated package or detailed change list so I can reproduce the performance gains in other environments. • Produce a concise report summarizing findings, before-and-after metrics, and next-step recommendations....

Job Description We are looking for an experienced Azure Data Engineer / Data Integration Specialist to design and implement a robust, scalable data pipeline that pulls data from the Blackbaud API and loads it into an Azure SQL Database using Azure Data Factory (ADF). The goal is to have a fully automated, secure, and monitored ETL pipeline that runs on a scheduled basis and supports future scaling. Project Scope 1. Data Ingestion Connect to Blackbaud REST APIs (OAuth authentication) Handle pagination, rate limits, and API throttling Extract multiple endpoints (e.g., constituents, gifts, transactions, etc.) 2. Data Transformation Clean, normalize, and structure raw API JSON Handle nulls, schema drift, and data type conversions Add audit fields (load date, source system, batch id) 3. Data Lo...

Background & Goal We're a medium size ecommerce retailer running on Netsuite for our ERP and Big Commerce for our front end website. We're looking for a data engineer to help build a data warehouse for Netsuite, Big Commerce, and potentially some other data platforms to help us analyse our data faster and more accurately. We wish to engage a third party data Engineer to build a central Data Warehouse in Google BigQuery (or other similar service). We want to analyse data in a data visualisation tool such as Power BI. We are also interested in other capability that we will be able to use the data warehousing for in the future (better exception reporting, better customer analytics, new sales & marketing opportunities, etc) We have a capable internal IT/Analytics team, howe...

REQUISITO EXCLUYENTE: Debes hablar español de forma fluida. Por favor, no apliques si no cumples este requisito, ya que la comunicación constante con el equipo es vital. Buscamos un experto freelance en Power BI y n8n para una colaboración a largo plazo. Tenemos múltiples proyectos de automatización en cola, pero comenzaremos con un Dashboard de Productividad operativo. Buscamos un perfil técnico que domine la integración de datos (ETL) y la visualización de alto nivel. El Primer Proyecto: Dashboard de Productividad El objetivo es añadir una sección a un reporte existente para calcular la productividad por operario. Fuentes de datos: Odoo 17 Enterprise: Extracción vía API de ausencias (vacaciones, baja...

Project Title: Automate Content Extraction from Website to Spreadsheet Using Project Description: I need a freelancer to set up an automated workflow using (formerly Integromat) to extract and aggregate content from my website, including blogs, podcasts, YouTube videos, books, and articles. The goal is to populate a structured spreadsheet (Excel or Google Sheets) with this data, making it easy for AI tools to analyze themes, trends, summaries, etc. Background on the Process: This is essentially a data extraction or content aggregation task (also known as web scraping or ETL: Extract, Transform, Load). It involves systematically collecting unstructured content from my site and organizing it into a spreadsheet. Each entry should include metadata like title, URL, summaries, tags, and more. ...

I want to add three solid, portfolio-ready papers to my personal body of work and I need an expert writer-researcher to produce them for me. Each paper must be: • 8 pages (≈ 5 000 words) • fully formatted in IEEE style and delivered as clean, compilable LaTeX • accompanied by the final PDF Scope for the three papers 1. AWS services and cloud-native architecture—cover core building blocks such as VPC, IAM, S3, Lambda, and how they interoperate in production-grade designs. 2. Data Engineering methodologies—drill into ETL processes and modern data-pipeline patterns on AWS (Glue, Step Functions, Kinesis, etc.) with diagrams and code snippets where helpful. 3. AI/ML algorithms and applications—tie in SageMaker, feature engineering, and model de...

I need a production-ready search stack that starts with an ETL flow pulling exclusively from our internal PostgreSQL databases. The pipeline must ingest and transform 38 000+ B2B category records and 5 000–10 000 company profiles, then run cleaning, vectorization, and enrichment steps so every record is categorized and stored in a pgvector-enabled schema. Once the data is in place, a separate microservice should expose a REST API that supports hybrid search: dense vectors (OpenAI text-embedding-3-small) combined with BM25 and blended with RRF scoring. Results have to work equally well in Hungarian and English; huspacy, spaCy, and Open AI are the preferred tools for language handling and any fallback generation. I expect the codebase in Python 3.10+, organised as two deployable unit...