About

- The Databricks SME on Azure is responsible for the design, implementation, and management of Databricks environments on the Azure platform.
- This role requires deep expertise in Databricks and extensive experience with Azure cloud services.
- The SME will work closely with data engineering, analytics, and IT teams to ensure the efficient deployment and management of Databricks solutions, driving data-driven decision-making across the organization.
- The SME will collaborate with cross-functional teams along with other HR Integration SMEs and architects to provide technical expertise and ensure the efficient deployment and management of cloud resources.

Core Responsibilities:

Design and Implementation:

- Architect and deploy scalable Databricks solutions on Azure.
- Develop and implement data pipelines, ETL processes, and machine learning workflows using Databricks and Azure services.

Performance and Optimization:

- Monitor Databricks performance and optimize configurations for efficiency and cost-effectiveness.
- Implement best practices for data storage, processing, and analysis to ensure optimal performance.

Security and Compliance:

- Ensure Databricks environments comply with security standards and regulatory requirements.
- Implement and manage data encryption, access controls, and other security measures.

Maintenance and Support:

- Perform regular maintenance tasks, including updates, patches, and backups.
- Provide advanced technical support for Databricks-related issues and act as the escalation point for complex problems.

Collaboration and Consultation:

- Collaborate with data engineers, data scientists, and analysts to design and optimize data workflows.
- Work with IT and DevOps teams to integrate Databricks with other Azure services and tools.

Training and Documentation:





- Develop and deliver training sessions for internal teams on Databricks best practices and new features.
- Document architectures, configurations, and procedures for knowledge sharing and future reference.

Competencies:

- Strong analytical and problem-solving skills.
- Strong technical documentation capabilities.
- Agile framework and delivery lifecycle management.
- Team based development capabilities and ability to manage personal deliverables.
- Experience with multi-cloud environments and hybrid cloud strategies.
- Familiarity with DevOps practices and tools such as Azure DevOps, Jenkins, or GitHub Actions.
- Databricks Certified Associate Developer or equivalent certification.
- Relevant Azure certifications, such as Azure Data Engineer Associate or Azure Solutions Architect

Skills:

- In-depth knowledge of Databricks architecture, Spark, and Delta Lake.
- Proficiency in managing and optimizing Databricks clusters and jobs.
- Experience with deploying and managing Databricks on Azure, including integration with Azure Data Lake, Azure SQL Database, and other Azure services.
- Strong programming skills in languages such as Python, SQL, and Scala.
- Familiarity with data warehousing concepts and ETL processes.
- Strong understanding of Azure infrastructure and services, such as Azure Virtual Machines, Azure Storage, and Azure Networking.
- Azure Database and data platform
- DevOps Tooling: Jenkins, Nexus, GitHub, JIRA
- Database Services, including Azure SQL and Cosmos DB
- DevOps Practices with Azure DevOps
- Monitoring, Diagnostics, and Analytics with Azure Monitor

Additional Skills:

- Azure Data Factory and Data Pipeline Orchestration
- Azure Synapse Analytics and Data Warehousing
- Data Modeling and Database Design
- Power BI and Data Visualization
- Machine Learning and AI Integration with Azure ML

📌 Databricks Azure SME (Ciudad de México)
🏢 Pyramid Consulting
📍 Ciudad de México