24 oct
|
Coforge
|
México
We are seeking a highly skilled Senior Data Engineer to design, develop, and optimize large-scale data platforms and pipelines supporting business-critical analytics and operational workloads. The idóneo candidate will possess strong expertise in AWS, Apache Spark, Apache Flink, and modern data engineering practices while leading the development of scalable, reliable, and high-performance data solutions. The role requires experience across both legacy enterprise implementations and modern cloud-native architectures.
Key Responsibilities
• Design, develop, and maintain scalable batch and streaming data pipelines using AWS EMR, Apache Spark, and Apache Flink.
• Build and support ETL/ELT processes integrating data from Teradata, SQL Server, and other enterprise data sources.
• Develop robust and scalable data processing applications using Python, PySpark, SQL, and Java where required.
• Implement and maintain workflow orchestration and scheduling using Apache Airflow.
• Design and optimize data ingestion, transformation, and processing frameworks for large-scale datasets.
• Support both real-time and batch processing workloads across AWS data platforms.
• Optimize data processing jobs for performance, scalability, reliability, and cost efficiency.
• Monitor, troubleshoot, and resolve production issues across data pipelines and platform services.
• Utilize Splunk and other monitoring tools to support operational excellence and incident management.
• Collaborate with Data Architects, Data Engineers, Analytics teams,
and business stakeholders to deliver enterprise data solutions.
• Perform data validation, reconciliation, and data quality assurance activities.
• Implement best practices related to coding standards, testing, governance, security, and operational support.
• Support release management, deployment activities, documentation, and continuous improvement initiatives.
• Mentor junior engineers and provide technical guidance across data engineering projects. Must Have Skills AWS Platform
• AWS EMR, EC2
• AWS Data Engineering Services Big Data & Processing Frameworks
• Apache Spark & Apache Flink Programming & Development
• Python, PySpark, SQL Orchestration
• Apache Airflow Enterprise Data Sources
• Teradata, SQL Server Good to Have Skills Big Data Ecosystem
• Hadoop Stack Programming Languages
• JavaReporting & Analytics Exposure
• Tableau, Looker, Alteryx, Trino Query Books Qualifications
• Bachelor's degree in Computer Science, Information Technology, Engineering, or a related discipline.
• 7 to 10 years of experience in Data Engineering and large-scale data platform development.
• Strong hands-on experience with AWS EMR, Apache Spark, Apache Flink, Python/PySpark, SQL, and Apache Airflow.
• Experience designing and supporting enterprise-grade ETL/ELT solutions.
• Proven ability to optimize data pipelines and resolve complex production issues.
• Strong analytical, problem-solving, communication, and stakeholder management skills.
📌 Senior Data Engineer AWS (México)
🏢 Coforge
📍 México