Overview

We are seeking a highly skilled Senior Data Engineer with hands-on experience in Databricks, Unity Catalog, and Azure Data Factory (ADF) to support a large-scale enterprise data modernization initiative. The idóneo candidate will be responsible for designing, developing, and maintaining scalable data pipelines, enabling secure data governance, and supporting cloud-based analytics solutions in Azure.

This role will collaborate closely with architects, business stakeholders, and cross-functional engineering teams to build reliable, high-performance data platforms and accelerate data-driven decision-making.

Responsibilities

Your Impact

- Design, develop, and optimize ETL/ELT pipelines using Azure Data Factory and Databricks.
- Build and maintain scalable data ingestion frameworks for structured and unstructured data sources.
- Develop data transformation processes using PySpark, Spark SQL, and Databricks notebooks.
- Implement and manage Unity Catalog for enterprise data governance, security, and access controls.
- Create and optimize Delta Lake architectures for analytics and reporting workloads.
- Monitor pipeline performance and troubleshoot production issues.
- Collaborate with data architects to implement enterprise data models and best practices.
- Ensure data quality, integrity, security, and compliance standards are met.
- Support CI/CD deployment processes and DevOps practices for data engineering solutions.




- Work closely with business and analytics teams to understand requirements and deliver scalable solutions.

Requirements

Your Skills & Experience

- 4-7+ years of experience in Data Engineering or Big Data development.
- Strong hands-on experience with Azure Databricks.
- Proficiency in PySpark, Spark SQL, and distributed data processing.
- Experience implementing and managing Unity Catalog.
- Strong experience with Azure Data Factory (ADF).
- Experience working with Delta Lake and modern Lakehouse architectures.
- Solid understanding of data modeling, ETL/ELT, and data warehousing concepts.
- Experience with Azure cloud services and ecosystem.
- Proficiency with Git, version control, and CI/CD processes.
- Strong analytical and problem-solving skills.
- Excellent communication and stakeholder management abilities.

Preferred Qualifications

- Experience with Azure Synapse Analytics.
- Familiarity with Collibra or other data governance platforms.
- Experience with Infrastructure as Code (Terraform, ARM Templates).
- Databricks Certification (Data Engineer Associate/Professional).
- Microsoft Azure Data Engineer Associate Certification.

Nice to Have

- Experience supporting enterprise-scale data migration projects.
- Knowledge of healthcare, retail, or consumer products data environments.
- Exposure to real-time data processing and streaming technologies.

📌 Senior Data Engineer (Databricks PySpark ADF Unity Catalog) (Ciudad de México)
🏢 Mps Group
📍 Ciudad de México