
Open
Posted
•
Ends in 16 hours
We are seeking a highly skilled Data Engineer to join our onsite team. In this role, you will design, build, and optimize complex, highly scalable big data pipelines and infrastructure on AWS to support advanced regulatory analytics, surveillance systems, and market research. Key Responsibilities Pipeline Development: Design, develop, and maintain high-performance ETL/ELT data pipelines capable of ingesting and transforming massive volumes of structured and unstructured financial data. Big Data Processing: Utilize PySpark, Apache Spark, and AWS EMR to process billions of records efficiently, ensuring low latency and high reliability. Cloud Architecture: Architect and optimize solutions within FINRA’s AWS data lake and data warehousing environment (S3, Athena, Glue, Snowflake, Presto/Trino). Data Quality & Governance: Implement rigorous data quality checks, validation frameworks, and observability pipelines to ensure data lineage, consistency, and strict regulatory compliance. Collaboration: Partner cross-functionally with data scientists, economists, software engineers, and regulatory stakeholders to translate complex business requirements into robust technical solutions. CI/CD & DevOps: Deploy scalable infrastructure using CI/CD pipelines (Jenkins, Bitbucket, Git) and infrastructure-as-code (Terraform, CloudFormation). Performance Optimization: Troubleshoot production data pipelines, tune PySpark/SQL queries for optimal compute performance, and manage AWS resource costs. Required Qualifications Experience: 5+ years of hands-on, dedicated Data Engineering experience in a production environment. Programming Languages: Expert-level proficiency in Python (specifically for data engineering) and advanced SQL. Big Data Ecosystem: Deep, hands-on experience with Apache Spark (PySpark) and distributed computing architecture. Cloud Platforms: Strong implementation experience within the AWS ecosystem, specifically with EMR, S3, Glue, Athena, IAM, and Step Functions. Data Architecture: Proven experience working with data lakes, modern data warehousing (e.g., Snowflake, Redshift), and data formats (Parquet, Apache Iceberg, Delta Lake). Education: Bachelor’s or Master’s degree in Computer Science, Information Technology, Engineering, or a closely related discipline. Work Authorization: Eligible to work in US Preferred Qualifications Prior domain experience in financial services, capital markets, payroll, or regulatory environments. Experience with streaming data technologies such as Apache Kafka or Amazon Kinesis. AWS Certifications (e.g., AWS Certified Data Analytics, AWS Certified Solutions Architect). Familiarity with AI-assisted developer tooling (GitHub Copilot, Amazon Q) and integrating AI into engineering workflows. Excellent communication skills with the ability to explain complex technical concepts to non-technical financial stakeholders. Analytics SME JD offshore- We are looking for an experienced Adobe Analytics and Adobe Launch SME to lead analytics implementation, tag management, data collection, and reporting across web and digital platforms. The ideal candidate will have strong hands-on experience with Adobe Analytics, Adobe Experience Platform Data Collection Tags—formerly Adobe Launch—and modern web analytics practices. Key Responsibilities Design and implement Adobe Analytics tracking for websites and digital experiences. Configure Adobe Launch properties, rules, data elements, extensions, and publishing workflows. Develop and maintain solution design reference documents, data layers, event mappings, eVars, props, and success events. Implement tracking using Adobe Web SDK, AppMeasurement, and Experience Cloud ID Service. Troubleshoot data collection, tag-firing, attribution, and reporting issues using browser debugging tools. Validate implementations through QA, network-call testing, and data reconciliation. Build Adobe Analytics reports, segments, calculated metrics, dashboards, and Analysis Workspace projects. Work with marketing, product, development, privacy, and business teams to translate measurement requirements into technical solutions. Support consent-management integrations and ensure analytics tags comply with privacy requirements. Establish tagging standards, governance, documentation, and release-management processes. Provide technical guidance and mentoring to analytics developers and business users. Required Qualifications 6+ years of experience in digital analytics, including strong hands-on Adobe Analytics experience. 4+ years of experience implementing and managing Adobe Launch or Adobe Experience Platform Data Collection Tags. Strong knowledge of JavaScript, HTML, browser developer tools, cookies, local storage, and network requests. Experience designing data layers and enterprise analytics implementation frameworks. Strong understanding of eVars, props, events, processing rules, classifications, marketing channels, and attribution. Experience with Adobe Web SDK, Experience Cloud ID Service, and Adobe Debugger or Assurance. Ability to troubleshoot complex implementation and data-quality issues. Strong communication, documentation, and stakeholder-management skills. Preferred Qualifications Adobe Analytics Business Practitioner or Developer certification. Experience with Adobe Experience Platform, Customer Journey Analytics, Adobe Target, AEM, and OneTrust. Knowledge of server-side tagging, APIs, mobile-app analytics, and customer journey measurement. Experience working in Agile delivery environments
Project ID: 40742799
25 proposals
Open for bidding
Remote project
Active 5 days ago
Set your budget and timeframe
Get paid for your work
Outline your proposal
It's free to sign up and bid on jobs
25 freelancers are bidding on average $111 USD/hour for this job

Hi, I’m a Principal-level Software Engineer with 21+ years of hands-on experience building scalable backend, cloud, data, and distributed systems. I personally lead architecture and critical implementation, with a small senior engineering team available when additional execution capacity is needed. For this project, my experience aligns well with AWS-based data engineering, including: • Scalable ETL/data pipelines and distributed processing • Apache Spark, Kafka, and event-driven architectures • AWS cloud architecture, S3, IAM, data processing, and automation • Python, SQL, Java, and high-performance backend systems • Data platforms, performance optimization, monitoring, and production troubleshooting • CI/CD, infrastructure automation, modernization, and reliability engineering I’m comfortable working with complex datasets and translating business requirements into reliable, scalable production systems. I focus on architecture, performance, maintainability, and clear communication. I’d be happy to discuss the existing architecture and how I can contribute to the project.
$125 USD in 40 days
5.8
5.8

Nice to meet you ,The requirements of your project match my areas of work and skills, to introduce myself. My name is Anthony Muñoz and i am the lead engineer for DS Pro IT agency. I have worked for over 10 years as a Full-Stack and software development engineer and have successfully done multiple jobs. It will be a pleasure to work together to make your project. Feel free to discuss about the project with me, greetings.
$109 USD in 40 days
5.8
5.8

I understand you want an experienced data engineer to build and tune large, reliable AWS data pipelines that process huge volumes of financial data quickly, keep it accurate and compliant, and turn complex business needs into solid, cost-efficient solutions your analysts and regulators can depend on. I have experience on building large-scale ETL pipelines with Python, PySpark, AWS EMR, Glue, S3, Athena and Snowflake, using Parquet and Iceberg data lakes, Terraform and CI/CD, with strong data quality checks and query tuning to keep pipelines fast, stable and low cost. I can start work right away and contact me to discuss more details. Thanks. Smit
$100 USD in 40 days
4.9
4.9

Hi, This role interests me because it sits at the intersection of analytics, implementation, debugging, and stakeholder communication rather than being limited to dashboard creation. I’m a Software Architect with 18+ years of experience across web platforms, JavaScript, APIs, analytics, data pipelines, cloud systems, and enterprise integrations. I’m comfortable working with complex implementations where tracking accuracy, governance, and data quality matter as much as the reporting layer. My relevant strengths include: • JavaScript, browser/network debugging, cookies and local storage • Data-layer design and event mapping • Analytics implementation and validation • Tagging strategy and governance • API and server-side integrations • Data quality checks and reconciliation • Dashboard/reporting architecture • Privacy-aware tracking and consent integration • Technical documentation and stakeholder communication I can work closely with product, marketing, development, and privacy teams to translate measurement requirements into reliable technical implementations and troubleshoot issues down to the network-call level. I also bring architecture and team-lead experience, so I’m comfortable establishing standards, reviewing implementations, mentoring developers, and improving long-term maintainability. Regards, Farrukh Naveed Anjum Software Architect | Analytics, Data & Integration
$100 USD in 40 days
3.9
3.9

Hi, I am a professional web developer and I can do this project "Data engineer", I have 5 years of experience in web development. I have done many projects like this. I can do this job for you. I can start right now. Please contact me. Thanks
$100 USD in 2 days
4.1
4.1

Hi, We can support your data engineering and analytics requirements with scalable data pipelines, cloud data platforms and analytics implementation support. For the Data Engineering scope, we can work on Python and SQL based ETL/ELT pipelines, large-scale data processing, AWS services, data lakes/warehouses, data quality, governance, observability, CI/CD and infrastructure automation. We can also support performance tuning, troubleshooting and cost optimization for production workloads. For the Adobe Analytics scope, we can support web analytics implementation, tag management, data layers, event mappings, tracking validation, debugging, reporting and documentation. We can also work with JavaScript, HTML, browser debugging, consent requirements and analytics QA workflows. Our data engineering and analytics capabilities cover data processing, cloud solutions, analytics and API based integrations. We can share relevant project details upon your response. A few questions: 1. Is this one combined requirement or two separate resources? 2. Which role needs to be filled first? 3. Is the engagement strictly onsite, or is offshore/remote delivery possible? 4. For Adobe Analytics, which implementation is currently used: AppMeasurement, Web SDK or both? 5. What is the expected engagement duration? Let’s get connected to discuss the required team structure and scope. Note: The final cost may vary based on the confirmed scope. Best Regards, Agnidipa Team Qloron
$125 USD in 40 days
3.0
3.0

Hi, I have experience developing data engineering pipelines, analytics platforms, data processing systems, and web analytics solutions using Python, SQL, AWS, and modern analytics tools. The most important part of this type of project is keeping large-scale data pipelines reliable while maintaining data quality, lineage, and accurate analytics tracking. In similar projects, I faced issues such as slow processing when handling large datasets, inconsistent data between source systems and analytics reports, and tracking events not firing correctly across different web flows. To solve these issues, I optimized SQL and distributed processing logic so large datasets could be processed more efficiently. For analytics problems, I used structured data layers and validated network requests and event mappings against the expected reporting data. As a result, the pipelines and analytics implementations became more reliable, data quality improved, and reporting was more consistent and easier to troubleshoot. I can help you with this kind of experience and contribute to both the data engineering and Adobe Analytics requirements. Looking forward to discussing your vision with you. Best regards, Michel
$100 USD in 40 days
2.7
2.7

Drawing on my 5+ years of dedicated experience as a Data Engineer, I am more than equipped to tackle the complexities of your project. Proficient in Python and advanced SQL, I have a deep understanding of ETL/ELT processes and the AWS ecosystem. My work has demanded intimate familiarity with platforms such as EMR, S3, Glue, Athena, IAM, Step Functions and Snowflake - skills that would be instrumental in optimizing your AWS data lake and ensuring compliance. Furthermore, my domain knowledge aligns astutely with your project requirements. Having worked extensively in the financial services sector in a similar capacity, I am familiar with regulatory environments and understand the nuances of payroll systems. As an additional asset let me add that we even specialize in AI solutions and integrating AI into engineering workflows so if you ever need to further augment your systems down the line we have it covered there as well. In sum, I offer not just solid technical abilities but also an alignment with your specific objectives that can greatly benefit your project's success. Consider TechWagger for diligent execution and comprehensive value addition throughout.
$100 USD in 40 days
2.9
2.9

Hi, I’m an Engineer with 5+ years of experience developing connected systems, industrial data acquisition, automation and IoT solutions. I’m interested in supporting your data engineering projects, particularly where large-scale data processing connects with real-world systems and analytics. My relevant experience includes: - Python-based data processing and automation - Industrial IoT and sensor data pipelines - ESP32/STM32 systems generating high-volume telemetry - MQTT, REST APIs and system integration - SQL/database-oriented data handling - Data validation, monitoring and troubleshooting - Edge-to-cloud architectures - Siemens PLC / OPC UA data integration - Linux-based edge systems and Docker - Git-based development and deployment workflows I have worked on systems involving sensor telemetry, alarms, environmental data, industrial equipment and PLC data, including architectures where data is collected at the edge and transferred to higher-level applications for analysis. For an AWS/Spark-heavy production environment, I would first align with your existing architecture and coding standards, then work within the established S3/Glue/EMR/Athena/Snowflake ecosystem. I’m particularly interested in long-term offshore collaboration and can work effectively with distributed engineering teams and technical stakeholders. I’d be happy to discuss the current pipeline architecture, data volumes, processing requirements and expected role responsibilities. Best regards,
$125 USD in 40 days
0.0
0.0

The main point I’d clarify before proposing the right resource is that the description appears to combine two highly specialised roles: senior AWS/PySpark data engineering and Adobe Analytics/AEP implementation. I can help coordinate the appropriate specialist profile around the required AWS, big-data and enterprise analytics capabilities rather than treating this as a generic full-stack role. I’d also want to verify the onsite and US work-authorisation requirements before confirming availability. Are you looking for one engineer who covers both JDs, or are these intended to be two separate positions? Regards, Ilyaas
$100 USD in 40 days
0.0
0.0

I’m a Python/AI Engineer with 5+ years of production experience, including building scalable data and ML systems with AWS, Python, SQL, Docker, and CI/CD. I can contribute to high-volume data pipelines, ETL/ELT, cloud infrastructure, and performance optimization. I’m available to start immediately. My portfolio: https://www.freelancer.com/u/muzammilf0 Best regards, Muzammil Fareed
$125 USD in 40 days
0.0
0.0

Hyderabad, India
Member since Dec 29, 2025
$100-150 USD / hour
$20-60 USD / hour
$100-150 USD / hour
₹12500-37500 INR
$100-150 USD / hour
₹12500-37500 INR
$3000-5000 USD
$25-50 USD / hour
$30-250 USD
₹12500-37500 INR
$30-250 USD
₹1500-12500 INR
$250-750 USD
₹1500-12500 INR
$30-250 USD
£750-1500 GBP
$250-750 AUD
$250-750 AUD
₹1500-12500 INR
$30-250 USD
€250-750 EUR
$10-30 USD
₹1500-12500 INR