We are seeking a skilled Databricks Engineer with strong experience in Azure Databricks and Python (PySpark) to design, build, and optimize scalable data processing pipelines.
The idóneo candidate will work closely with data engineers, analysts, and stakeholders to deliver high-performance data solutions on the Azure cloud platform.
Key Responsibilities
Design, build, and maintain data pipelines and ETL processes using Databricks and Apache Spark.
Optimize data workflows for performance, scalability, and cost efficiency.
Implement data Lakehouse architecture and manage data ingestion from multiple sources.
Collaborate with data scientists and analysts to enable advanced analytics and machine learning workloads.
Ensure data quality, governance, and security across all data assets.
Monitor and troubleshoot Databricks clusters, jobs, and workflows.
Integrate Databricks with cloud services (AWS, Azure, or GCP) and other enterprise systems.
Document processes, standards, and best practices for data engineering.




Required Skills & Qualifications
Bachelor’s degree in Computer Science, Data Engineering, or related field.
3+ years of experience in data engineering or big data technologies.
Hands-on experience with Databricks, Apache Spark, and PySpark.
Strong knowledge of SQL, Python, and data modeling principles.
Experience with cloud platforms (AWS, Azure, or GCP) and their data services.
Familiarity with Delta Lake, Lakehouse architecture, and data governance.
Understanding of CI/CD pipelines and DevOps practices for data workflows.
Excellent problem-solving and communication skills.
Nice to Have
Experience with Delta Lake and Lakehouse architecture.
Knowledge of Azure Data Factory (ADF) for orchestration.
Experience with SQL and Spark SQL.
Azure certifications related to data engineering.

#J-18808-Ljbffr

📌 Azure Databricks Data Engineer | Lakehouse & ETL Expert (Ciudad de México)
🏢 Pyramid Consulting
📍 Ciudad de México