24 oct
|
Coforge
|
México
We are seeking a highly skilled Senior Data Engineer to design, develop, and optimize large-scale data platforms and pipelines supporting business-critical analytics and operational workloads. The idóneo candidate will possess strong expertise in AWS,
Apache Spark, Apache Flink, and modern data engineering practices while leading the development of scalable, reliable, and high-performance data solutions. The role requires experience across both legacy enterprise implementations and modern cloud-native architectures.
Key Responsibilities
- Design, develop, and maintain scalable batch and streaming data pipelines using AWS EMR, Apache Spark, and Apache Flink.
- Build and support ETL/ELT processes integrating data from Teradata, SQL Server, and other enterprise data sources.
- Develop robust and scalable data processing applications using Python, PySpark, SQL, and Java where required.
- Implement and maintain workflow orchestration and scheduling using Apache Airflow.
- Design and optimize data ingestion, transformation, and processing frameworks for large-scale datasets.
- Support both real-time and batch processing workloads across AWS data platforms.
- Optimize data processing jobs for performance, scalability, reliability, and cost efficiency.
- Monitor, troubleshoot, and resolve production issues across data pipelines and platform services.
- Utilize Splunk and other monitoring tools to support operational excellence and incident management.
- Collaborate with Data Architects, Data Engineers, Analytics teams,
and business stakeholders to deliver enterprise data solutions.
- Perform data validation, reconciliation, and data quality assurance activities.
- Implement best practices related to coding standards, testing, governance, security, and operational support.
- Support release management, deployment activities, documentation, and continuous improvement initiatives.
- Mentor junior engineers and provide technical guidance across data engineering projects.
Must Have Skills
AWS Platform
- AWS EMR, EC2
- AWS Data Engineering Services
Big Data & Processing Frameworks
- Apache Spark & Apache Flink
Programming & Development
- Python, PySpark, SQL
Orchestration
- Apache Airflow
Enterprise Data Sources
- Teradata, SQL Server
Good to Have Skills
Big Data Ecosystem
- Hadoop Stack
Programming Languages
- JavaReporting & Analytics Exposure
- Tableau, Looker, Alteryx, Trino Query Books
Qualifications
- Bachelor's degree in Computer Science, Information Technology, Engineering, or a related discipline.
- 7 to 10 years of experience in Data Engineering and large-scale data platform development.
- Strong hands-on experience with AWS EMR, Apache Spark, Apache Flink, Python/PySpark, SQL, and Apache Airflow.
- Experience designing and supporting enterprise-grade ETL/ELT solutions.
- Proven ability to optimize data pipelines and resolve complex production issues.
- Strong analytical, problem-solving, communication, and stakeholder management skills.
📌 Senior Data Engineer AWS (México)
🏢 Coforge
📍 México