Senior Data Engineer

About the Position

We are looking for a Senior Data Engineer to develop and optimize data pipelines within a large scale data migration program in a capital markets environment. This is a hands on, developer oriented role for someone with a strong software engineering background who has moved into Data Engineering and cloud data migration on AWS. You will design and implement robust, scalable, production grade data pipelines, ensuring data quality, consistency, reconciliation, and traceability across multiple systems.

About the Project

The program is a multi phase data migration initiative replacing legacy capital markets systems with a modern platform ecosystem. It includes critical datasets across custody, clearing and settlement, derivatives processing, and CCP operations. The project spans 18 months and follows an incremental approach, with a strong emphasis on data integrity, reconciliation, and traceability.

Responsibilities

• Design and implement scalable ETL/ELT pipelines using AWS Glue, Python, and dbt.
• Take a highly hands on role, spending most of your time coding and reviewing code.
• Integrate data from multiple Oracle and PostgreSQL legacy systems into AWS.
• Write and optimize complex SQL and Java/Python data transformation logic.
• Define and enforce coding standards, testing practices, code reviews, CI/CD processes, and deployment procedures.
• Guide and mentor mid level engineers through technical direction and code reviews.
• Ensure data quality, consistency, integrity, reconciliation, and traceability.
• Collaborate with data architects on target data models and transformation strategies.
• Optimize pipelines for performance, scalability, and cost efficiency across AWS Glue, Athena, and Redshift.
• Implement error handling, logging, monitoring, and observability.
• Collaborate with data analysts and business stakeholders to ensure migrated data meets business requirements.
• Troubleshoot complex data and code related issues and drive root cause analysis.
• Contribute to data platform architecture and document technical solutions.

Requirements

• Advanced Java skills with strong, production grade development experience.
• Advanced SQL skills with Oracle and PostgreSQL, along with strong database knowledge.
• Advanced Python skills for data processing and transformation.
• Strong hands on experience with dbt or equivalent transformation frameworks.
• Experience with Maven for build and dependency management.
• Experience with Git and modern version control workflows.
• Proven experience building production grade ETL/ELT pipelines at scale.
• Strong experience with data migration to AWS Cloud, combined with a solid software development background.
• Solid understanding of data modeling and data warehouse architectures.
• Experience implementing CI/CD practices in data environments.
• Strong problem solving skills and the ability to design scalable, maintainable solutions.
• Fluency in Spanish, both written and spoken.

Nice to Have

• Experience with Spring or Spring Boot.
• Experience with AWS Data Engineering services, including Glue, Athena, and Redshift.
• Experience with Bash scripting.
• Experience with Docker or Podman.
• Experience with API design and integration.
• Experience in capital markets, including custody, clearing, settlement, derivatives, or CCP.
• Experience working on large scale data migration or transformation programs.
• Experience with Oracle legacy systems.
• Understanding of data governance, data lineage, and metadata management.
• Experience with observability and monitoring solutions.

Technologies

Java, Python, SQL (Oracle and PostgreSQL), AWS (Glue, Athena, Redshift), dbt, Maven, Git