Overview We are looking for a Senior Data Engineer who will help in implementing data analytics products using on-prem and cloud data platforms.Position Responsibilities:Work with cutting-edge big data platforms (e.G., Databricks, Apache Spark) at large scale, pushing the boundaries of data processing and model enablement.Build and maintain robust ETL/ELT pipelines for ingestion, transformation, and aggregation of large-scale datasets on Hadoop and enterprise data platforms.Develop high-performance data processing jobs using PySpark/Spark, Python on data platforms such as cloudera and databricks.Optimize pipeline performance and cost through partitioning, file formats, compute tuning, and efficient query patternsContribute to CI/CD for data workflows (testing, code reviews, deployment automation),



promoting engineering best practices and maintainable codebases.Partner with Product Managers to develop a deep understanding of users and use cases and apply that knowledge to scoping and building new modules and featuresIdeal Candidate Qualifications:Strong hands-on experience in data engineering building production-grade pipelines on big data platforms (Hadoop ecosystem and cloud data platforms - databricks).
High proficiency in using Python, Spark, Hadoop platforms & tools (Hive, Impala, Airflow, NiFi), SQL to build Big Data products.Hands-on experience with cloud data platforms such as databricks, Client (databricks preferred)Experience with orchestration/integration tools such as Apache Airflow, Apache NiFi, or Talend.Working knowledge of DevOps/CI-CD practices: version control (Git), automated testing, release pipelines, and observability.Strong problem-solving skills with the ability to debug complex data issues and communicate clearly with technical and non-technical stakeholders.Experience developing Java based applications is an added advantage.
#J-*****-Ljbffr

📌 Data Engineer (Jalisco)
🏢 Pyramid Consulting
📍 Jalisco