overview

we are seeking a highly skilled senior data engineer with hands-on experience in databricks, unity catalog, and azure data factory (adf) to support a large-scale enterprise data modernization initiative. The adecuado candidate will be responsible for designing, developing, and maintaining scalable data pipelines, enabling secure data governance, and supporting cloud-based analytics solutions in azure.

this role will collaborate closely with architects, business stakeholders, and cross-functional engineering teams to build reliable, high-performance data platforms and accelerate data-driven decision-making.

responsibilities

your impact

- design, develop, and optimize etl/elt pipelines using azure data factory and databricks.
- build and maintain scalable data ingestion frameworks for structured and unstructured data sources.
- develop data transformation processes using pyspark, spark sql, and databricks notebooks.
- implement and manage unity catalog for enterprise data governance, security, and access controls.
- create and optimize delta lake architectures for analytics and reporting workloads.
- monitor pipeline performance and troubleshoot production issues.
- collaborate with data architects to implement enterprise data models and best practices.
- ensure data quality, integrity, security, and compliance standards are met.
- support ci/cd deployment processes and devops practices for data engineering solutions.




- work closely with business and analytics teams to understand requirements and deliver scalable solutions.

requirements

your skills & experience

- 4-7+ years of experience in data engineering or big data development.
- strong hands-on experience with azure databricks.
- proficiency in pyspark, spark sql, and distributed data processing.
- experience implementing and managing unity catalog.
- strong experience with azure data factory (adf).
- experience working with delta lake and modern lakehouse architectures.
- solid understanding of data modeling, etl/elt, and data warehousing concepts.
- experience with azure cloud services and ecosystem.
- proficiency with git, version control, and ci/cd processes.
- strong analytical and problem-solving skills.
- excellent communication and stakeholder management abilities.

preferred qualifications

- experience with azure synapse analytics.
- familiarity with collibra or other data governance platforms.
- experience with infrastructure as code (terraform, arm templates).
- databricks certification (data engineer associate/professional).
- microsoft azure data engineer associate certification.

nice to have

- experience supporting enterprise-scale data migration projects.
- knowledge of healthcare, retail, or consumer products data environments.
- exposure to real-time data processing and streaming technologies.

#j-18808-ljbffr

📌 Senior data engineer (databricks + pyspark + adf + unity catalog) (Xico)
🏢 Mps Group
📍 Xico