02 sep
|
DataArt
|
Monterrey
Project overview: The program is a multi phase data migration initiative replacing legacy capital markets systems with a modern platform ecosystem. It includes critical datasets across custody, clearing and settlement, derivatives processing, and CCP operations. The project spans 18 months and follows an incremental approach, with a strong emphasis on data integrity, reconciliation, and traceability.
- Position overview: We are looking for a Senior Data Engineer to develop and optimize data pipelines within a large scale data migration program in a capital markets environment. This is a hands on, developer oriented role for someone with a strong software engineering background who has moved into Data Engineering and cloud data migration on AWS.
- You will design and implement robust, scalable, production grade data pipelines, ensuring data quality, consistency, reconciliation, and traceability across multiple systems.
Technology stack: Java, Python, SQL (Oracle and PostgreSQL), AWS (Glue, Athena, Redshift), dbt, Maven, Git
- Responsibilities: Design and implement scalable ETL/ELT pipelines using AWS Glue, Python, and dbt.
- Take a highly hands on role, spending most of your time coding and reviewing code.
- Integrate data from multiple Oracle and PostgreSQL legacy systems into AWS.
- Write and optimize complex SQL and Java/Python data transformation logic.
- Define and enforce coding standards, testing practices, code reviews, CI/CD processes, and deployment procedures.
- Guide and mentor mid level engineers through technical direction and code reviews.
- Ensure data quality, consistency, integrity, reconciliation, and traceability.
- Collaborate with data architects on target data models and transformation strategies.
- Optimize pipelines for performance, scalability, and cost efficiency across AWS Glue, Athena, and Redshift.
- Implement error handling, logging, monitoring, and observability.
- Collaborate with data analysts and business stakeholders to ensure migrated data meets business requirements.
- Troubleshoot complex data and code related issues and drive root cause analysis.
- Contribute to data platform architecture and document technical solutions.
- Requirements: Advanced Java skills with strong, production grade development experience.
- Advanced SQL skills with Oracle and PostgreSQL, along with strong database knowledge.
- Advanced Python skills for data processing and transformation.
- Strong hands on experience with dbt or equivalent transformation frameworks.
- Experience with Maven for build and dependency management.
- Experience with Git and modern version control workflows.
- Proven experience building production grade ETL/ELT pipelines at scale.
- Strong experience with data migration to AWS Cloud, combined with a solid software development background.
- Solid understanding of data modeling and data warehouse architectures.
- Experience implementing CI/CD practices in data environments.
- Strong problem solving skills and the ability to design scalable, maintainable solutions.
- Fluency in Spanish, both written and spoken.
- Nice to have: Experience with Spring or Spring Boot.
- Experience with AWS Data Engineering services, including Glue, Athena, and Redshift.
- Experience with Bash scripting.
- Experience with Docker or Podman.
- Experience with API design and integration.
- Experience in capital markets, including custody, clearing, settlement, derivatives, or CCP.
- Experience working on large scale data migration or transformation programs.
- Experience with Oracle legacy systems.
- Understanding of data governance, data lineage, and metadata management.
- Experience with observability and monitoring solutions.
📌 Senior Data Engineer (Monterrey)
🏢 DataArt
📍 Monterrey