We are building a next-generation analytics platform and need a Lead Data Engineer to shape scalable ingestion, transformation, and governed data products on Databricks. You will combine hands-on delivery with technical leadership and architecture to enable self-service analytics and AI-assisted engineering. Responsibilities Design scalable data ingestion and transformation frameworks on Databricks Define and enforce Databricks best practices for medallion layers and operational excellence Build batch and streaming pipelines using Apache Spark and Py Spark Deliver end-to-end data products from ingestion through governed consumption layers Develop trusted analytical datasets and semantic models for reporting and self-service analytics Implement data quality, lineage, monitoring, and observability standards Lead proof-of-concepts and evaluate emerging capabilities in the Databricks ecosystem Mentor engineers and drive reusable frameworks adopted across teams Requirements5+ years of data engineering experience with production-grade platforms Strong Databricks experience in enterprise production environments Lead-level technical leadership skills to drive standards and mentor engineers End-to-end delivery experience across ingestion, transformation, semantic layers,



and reporting Advanced Apache Spark and Py Spark expertise for scalable pipeline development Advanced SQL skills for transformation, modeling, and performance tuning Upper-Intermediate English proficiency (B2) for leading technical discussions Nice to have Data architecture experience for enterprise lakehouse patterns Data solution architecture skills to define scalable platform designs Scala proficiency for Spark development and optimization We offer International projects with top brands Work with global teams of highly skilled, diverse peers Healthcare benefits Employee financial programs Paid time off and sick leave Upskilling, reskilling and certification courses Unlimited access to the Linked In Learning library and 22,000+ courses Integral career opportunities Volunteer and community involvement opportunities EPAM Employee Groups Award-winning culture recognized by Glassdoor, Newsweek and Linked In EPAM is an Equal Opportunity Employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, national origin, age, sexual orientation, gender identity or expression, disability, protected veteran status, or any other characteristic protected by applicable law.
#J-18808-Ljbffr

📌 Lead data engineer (databricks & spark) (Ciudad de México)
🏢 Epam Systems
📍 Ciudad de México