09 nov
|
Valce Talent Solutions
|
México
09 nov
Valce Talent Solutions
México
Remote position in México
Data Engineer
Role Overview The Data Engineer is responsible for designing, building, and maintaining scalable, cloud�native data pipelines and data infrastructure that support analytics, reporting, business intelligence, and real-time data processing.
Key Responsibilities
- Design, develop, and maintain scalable ETL/ELT pipelines for batch and real-time data processing.
- Build and optimize data models, Delta Tables, and Lakehouse architectures to support analytics and reporting.
- Develop and integrate RESTful APIs and data services to facilitate seamless data exchange across enterprise systems.
- Implement real-time and high-frequency data ingestion frameworks using streaming technologies and event-driven architectures.
- Design and manage cloud-native data solutions leveraging Azure services including
Azure Data Factory, Azure Databricks, ADLS, Event Hubs, and Synapse Analytics.
- Develop and optimize Databricks Spark applications for large-scale data transformation and processing.
- Ensure data quality, governance, security, and compliance across data platforms.
- Collaborate with data scientists, analysts, application teams, and business stakeholders to deliver scalable data solutions.
- Troubleshoot, monitor, and optimize pipeline performance and data platform reliability.
- Support DataOps and CI/CD practices for data pipeline deployment and automation.
Required Skills & Qualifications
- Strong proficiency in SQL and relational databases such as Oracle, SQL Server, and
MySQL.
- Strong programming skills in Python, PySpark, PL/SQL, Java, or Scala.
- Hands-on experience with Azure Cloud technologies:
o Azure Data Factory (ADF)
o Azure Databricks o Azure Data Lake Storage (ADLS)
o Azure Synapse Analytics o Azure Event Hubs o Azure Functions o Azure API Management o Azure DevOps
- Experience with Databricks Lakehouse architecture, Delta Lake, and Delta Tables.
- Expertise in API development, API integration, RESTful services, and microservices architecture.
- Experience processing high-volume and high-frequency data with low-latency requirements.
- Strong knowledge of real-time data ingestion and streaming technologies such as
Kafka, Azure Event Hubs, or Kinesis.
- Experience with Spark, Hadoop, and distributed data processing frameworks.
- Hands-on experience with OpenShift, Kubernetes, Docker, and containerized deployments.
- Experience with workflow orchestration tools such as Apache Airflow and Azure
Data Factory.
- Understanding of data governance, data security, and compliance best practices.
Preferred Qualifications
- Experience with Delta Live Tables (DLT), Auto Loader, and Change Data Capture
(CDC).
- Knowledge of DataOps, CI/CD, and Infrastructure as Code (IaC).
- Familiarity with event-driven architectures and real-time analytics platforms.
- Azure Data Engineer (DP-203) and Databricks certifications mandatory skills:
- Python
- Azure
- SQL
REMOTE
ADVANCED ENGLISH
📌 Azure data engineer (México)
🏢 Valce Talent Solutions
📍 México