06 nov
|
Arkstem
|
México
Purpose of the role.
Bring an approved design to a deployed and validated bundle,
operating the multi-agent and answering for the quality of what goes into dev. It is not
the one who writes all the code; it is the one who answers for it.
Summary of the profile sought
A data engineer with his or her own technical judgment, who is comfortable reviewing
and validating code that he or she did not write, debugging in production environments,
and deciding when to query rather than assume. You will work with AI-assisted
generation tools as a normal part of the workflow.
Experience Required
1. 2. Python and SQL in production environments.
2. 3. Hands-on experience with Apache Spark
3. 5. Experience with version control and pull request flow — code review as a
4. regular practice, not occasional.
Main Responsibilities
1. 1. Agent Operation and Auditing: Interact with the multi-agent system, subject the
2. plan.md to scrutiny and thoroughly review the generated code bundle (detection
3. of bottlenecks such as data skew, excessive shuffles or mistyping).
4. 2. Parity and Reconciliation Testing:
Design and execute data parity validations
5. that certify that the output in Databricks is 100% square (volume, values, and
6. schemas) against the source legacy load.
7. 3. Exception Management and Scalability: Resolve minor code warnings
8. autonomously and diligently escalates NEEDS-HUMAN events, pattern changes,
9. or semantic inconsistencies to the Architect.
10. 4. Code Quality and Security: Correct findings of SAST tools (e.g. Fluid Attacks),
11. dependency vulnerabilities and non-conformities before opening the PR.
12. 5. Delivery Cycle (Git Flow): Manage feat/TICKET-slug branches, ensure strict
13. compliance with local pre-checks and lead the resolution of REJECTED tickets or
14. defects raised in QA/UAT.
15. 6. Stability in Transition: Sustain operation, rapid anomaly correction and pipeline
16. telemetry during the post-deployment hypercare window until formal handover to
17. support.
Desirable: Alteryx/SSIS Basic Read, Pytest, Code Security, Advanced Git.
#J-18808-Ljbffr
📌 Data Engineer (México)
🏢 Arkstem
📍 México