About This RoleWe're looking for a Senior Data Engineer to design, build, and operate the batch and streaming data infrastructure powering a large-scale e-commerce/retail data platform. You'll work across the full pipeline lifecycle — from ingestion through warehousing to delivery — with real ownership over the systems you build, including on-call.

Responsibilities

- Design, build and operate batch and streaming data pipelines in Python and PySpark, orchestrated with Apache Airflow
- Model and warehouse data in BigQuery — partitioning, clustering and query design that keep both latency and spend under control
- Build real-time ingestion on Apache Kafka and Spark Streaming for storefront, order and inventory events
- Run and tune Spark workloads on Dataproc: cluster sizing, job profiling, shuffle and skew problems, cost per run
- Write and optimize complex SQL - multi-source joins, window functions, incremental logic - across relational and NoSQL stores
- Integrate internal and third-party RESTful APIs as pipeline sources and sinks, handling pagination, retries, rate limits and schema drift
- Own the delivery path: GitLab CI pipelines, automated tests on data logic, code review and repeatable deployments
- Instrument pipelines with monitoring, alerting and data-quality checks; share on-call for the systems you build
- Partner with product, analytics and backend engineers to turn ambiguous requirements into schemas and contracts people can rely on
- Raise the bar through documentation, design reviews and mentorship of mid-level engineers

Requirements




- 5–8 years building production data pipelines, with strong Python engineering fundamentals
- Hands-on Apache Airflow: authoring DAGs, dependency and backfill strategy, operating scheduled workloads
- Depth in PySpark, Spark SQL and Spark Streaming, including performance tuning of real jobs
- Working Scala proficiency for Spark and JVM-based data work
- Expert-level SQL — you can write, read and optimize genuinely complex queries
- Solid experience across relational and NoSQL databases, with clear judgment about when each fits
- Production experience with Apache Kafka for event streaming
- Google Cloud Platform, specifically BigQuery and Dataproc
- Experience working with RESTful APIs inside data pipelines
- CI/CD with GitLab CI — branching strategy, pipeline configuration, automated deployment
- Clear written and verbal communication in a distributed, client-facing team

Nice to Have
- Backend development experience with Java and Spring Boot — building or consuming the services your pipelines depend on
- Infrastructure as code (Terraform) and containerized workloads on GKE or Cloud Run
- Adjacent GCP services: Pub/Sub, Dataflow, Cloud Composer
- dbt or a comparable transformation and testing framework
- Large-scale e-commerce, marketplace or omnichannel retail data domains

Benefits
- Competitive salary
- Remote work opportunity
- Comfortable work in your local time zone
- Adaptable work schedule
- Professional growth and development
- Multicultural working environment

📌 Senior Data Engineer — GCP (México)
🏢 Jaxel
📍 México