Advanced Proficiency in Python concepts like Code Structures, Modules, Packages, Class, SubClass, Inheritance, Multi-Threading and Functional Programming.
Experience in developing reusable Python packages for internal or public usage
Ability to write automating ETL processes and scheduling jobs like Airflow DAG.
Ability to track job pipeline runs to reprocess error records
Ability to orchestration different pipelines to run in sequence or parallel.
Troubleshoot data pipeline errors and fix issues
Export or Import data to/from various formats like CSV, JSON, XML etc preferably from S3 or other cloud storage.
Experience in using AI IDE tool
SQL:
Advanced SQL skills, including complex joins, CTE's and subqueries
Experience in optimizing SQL queries for performance and optimization in data warehouse technologies preferably Snowflake
Testing and documentation:
Proficiency in Python unit,
integration and system test.
Proficiency in implementing DBT tests for data validation and quality checks
Code Generation:
Experience in generating code using configurations using python and jinja templates
Version control:
Experience in GitHub, including implementing CI/CD process from scratch
AWS Expertise:
Data Storage solutions:
In depth understanding of AWS S3 for data storage, ECS, IAM including best practices for organization and security
Cloud Security:
Knowledge of AWS security best practices, including IAM roles, encryption standard, secure coding guidelines, DBT profiles access configurations and more.
Data Integration (nice to have):
Experience with AWS lambda for serverless data processing tasks
Workflow Orchestration (nice to have):
Proficiency in using Apache Airflow on AWS to design ,schedule and monitor complex data flows
Ability to integrate Airflow with AWS services and DBT models such as
📌 AWS Data Eng (México)
🏢 Valce Talent Solutions
📍 México
Postulate a este anuncio
Muestra tus habilidades a la empresa, rellenar el formulario y deja un toque personal en la carta, ayudará el reclutador en la elección del candidato.