We are seeking a Senior Databricks Engineer to lead the design and implementation of a modern Databricks Lakehouse architecture. This role is focused on migrating legacy data solutions into scalable, governed, and performance-optimized pipelines using Databricks, Py Spark, and Delta Lake.
The engineer will play a key role in defining and building Bronze, Silver, and Gold data layers, enabling both batch and near real-time processing, and improving the reliability and usability of enterprise data platforms. This position will work closely with data engineers, architects, and stakeholders while providing technical leadership and guidance to more junior team members.
Key Responsibilities
Architecture & Engineering Architect and implement end-to-end data solutions within the Databricks Lakehouse environment
Design and build ingestion, transformation, and serving pipelines using Py Spark, Spark SQL, and Delta Lake Establish and enforce medallion architecture practices across Bronze, Silver, and Gold layer Migration & Modernizatio n Lead the migration of legacy Azure SQL, SSIS, and ADF pipelines into Databricks-native workflows
Convert existing T-SQL and ETL logic into scalable Spark-based processing
Support modernisation of data platforms with a focus on governance, scalability, and maintainability Streaming, Governance & Optimization Implement Auto Loader, Delta Lake Change Data Feed (CDF), and streaming/batch ingestion patterns
Define and enforce governance using Unity Catalog, including RBAC and data lineage




Optimise performance through Photon, partitioning, Z-Order indexing, and cluster tuning Validation & Technical Leadership Guide validation and reconciliation strategies between legacy SQL outputs and Delta-based pipelines
Perform code reviews and promote engineering best practices
Mentor mid-level and junior engineers across design, development, and troubleshooting activities Required Experience & Qualifications Bachelor’s degree in Computer Science, Engineering, Information Systems, or equivalent experience 6+ years of experience in data engineering, with 3+ years of hands-on Databricks experience Strong expertise in Py Spark, Spark SQL, and Delta Lake Proven experience migrating legacy data platforms such as SQL Server, SSIS, and ADF Strong understanding of ETL/ELT design, data modeling, and pipeline architecture
Hands-on experience with the Azure ecosystem, including ADLS, ADF, and related services
Strong knowledge of performance tuning techniques for distributed data processing
Experience leading technical implementation efforts and supporting junior engineers Preferred Qualifications Experience with Unity Catalog, Lakehouse Federation, or CDC frameworks Databricks certifications (Associate or Professional level)
Exposure to large-scale regulated datasets such as healthcare, claims, or financial data Experience with both batch and streaming data pipelines
Familiarity with enterprise-grade data governance and lineage practices #J-18808-Ljbffr

📌 Databricks Engineer (Ciudad de México)
🏢 Apex Systems
📍 Ciudad de México