29 oct
|
Pentasia
|
México
the role
you'll own the data lake — the system of record for millions of financial events a day across million of active users. You decide how that data lands, is stored, retained, and governed on s3 + snowflake/databricks, so analytics, finance, and regulators all see accurate, reconciled data with zero drift from source.
what you'll be doing
- own the lakehouse architecture: bronze/silver/gold layers, iceberg/delta tables, schema evolution.
- land operational data via cdc streaming (kafka, debezium), handling late and duplicate events.
- design data layout for speed and cost: partitioning, compaction, file sizing, query performance on trino/athena/snowflake.
- own retention and archival: storage tiering, regulatory retention, immutability, gdpr deletion.
- guarantee correctness: freshness slas, drift detection, reconciliation against the source wallet and ledger systems.
- own governance: catalog and lineage, row/column access control, pii masking, encryption, audit trails.
- monitor ingestion health, data anomalies, and cloud storage/compute spend.
what we're looking for
- 3+ years in data engineering, with real ownership of a large-scale data lake or lakehouse.
- cloud object storage (s3/gcs) plus snowflake, databricks, or bigquery in production.
- hands-on with an open table format (iceberg, delta, or hudi).
- deep understanding of partitioning, file layout, and query optimization at terabyte-plus scale.
- cdc and streaming ingestion experience (kafka + debezium or equivalent).
- strong sql and data modeling; enough relational grounding to reason about the oltp systems you capture from — critical for financial ledgers.
- data lake governance: catalogs, lineage, access control, pii, retention.
- python and an orchestrator (airflow/dagster) as tools — this is an architecture and ownership role, not heads-down coding.
- nice to have: fintech, igaming, or another regulated, audit-heavy environment.
#j-18808-ljbffr
📌 Data engineer (México)
🏢 Pentasia
📍 México