04 ago
|
Programming.Com
|
México
04 ago
Programming.Com
México
Kindly review the Job requirement and let me know your interest.
Role: Data Engineer
Location: Remote (Requires occasional travel) Responsibilities:
· Build and operate robust data pipelines for ingestion, cleaning, and transformation using Databricks, Airflow, or Dagster.
· Develop efficient ETL/ELT workflows in Python and SQL to support both batch and streaming workloads.
· Collaborate with ML and AI teams to deliver high-quality datasets for training, evaluation, and production features.
· Model and maintain structured data assets (Delta, Parquet, Iceberg) for reliability, versioning, and lineage tracking.
· Implement orchestration and monitoring — schedule jobs, track dependencies, and automate recovery from failures.
· Ensure data quality and compliance through validation frameworks, schema enforcement, and audit logging.
· Contribute to data platform evolution — evaluate tools, standardize best practices, and improve developer experience.
· Support performance and cost optimization across compute, storage, and orchestration systems.
Qualifications
· 3–6 years of experience as a Data Engineer or ETL Developer in a production environment.
· Proficiency in Python and SQL; strong familiarity with Databricks, Spark, or equivalent big-data frameworks.
· Experience with workflow orchestration tools such as Airflow, Dagster, Luigi or Prefect.
· Deep understanding of data modeling, data warehousing, and distributed data processing.
· Knowledge of modern data lakehouse architectures (Delta, Parquet, Iceberg).
· Familiarity with CI/CD, GitHub Actions, and data pipeline testing frameworks.
· Comfort working in a cross-functional environment with ML, product, and analytics teams.
📌 ETL Developer (México)
🏢 Programming.Com
📍 México