Remote

Advanced english

Only México

Must have below:

Python and Data Pipelines:

- Advanced Proficiency in Python concepts like Code Structures, Modules, Packages, Class, SubClass, Inheritance, Multi-Threading and Functional Programming.
- Experience in developing reusable Python packages for internal or public usage
- Ability to write automating ETL processes and scheduling jobs like Airflow DAG.
- Ability to track job pipeline runs to reprocess error records
- Ability to orchestration different pipelines to run in sequence or parallel.
- Troubleshoot data pipeline errors and fix issues
- Export or Import data to/from various formats like CSV, JSON, XML etc preferably from S3 or other cloud storage.
- Experience in using AI IDE tool

SQL:

- Advanced SQL skills, including complex joins, CTE's and subqueries
- Experience in optimizing SQL queries for performance and optimization in data warehouse technologies preferably Snowflake

Testing and documentation:

- Proficiency in Python unit, integration and system test.
- Proficiency in implementing DBT tests for data validation and quality checks

Code Generation:

- Experience in generating code using configurations using python and jinja templates

Version control:

- Experience in GitHub, including implementing CI/CD process from scratch

AWS Expertise:

- Data Storage solutions:




- In depth understanding of AWS S3 for data storage, ECS, IAM including best practices for organization and security

- Cloud Security:

- Knowledge of AWS security best practices, including IAM roles, encryption standard, secure coding guidelines, DBT profiles access configurations and more.

Data Integration (nice to have):

- Experience with AWS lambda for serverless data processing tasks

Workflow Orchestration (nice to have):

- Proficiency in using Apache Airflow on AWS to design ,schedule and monitor complex data flows
- Ability to integrate Airflow with AWS services and DBT models such as triggering a DBT model or EMR or reading from s3 writing to redshift or snowflake. Experience in API integrations to Salesforce, SFTP, SharePoint, One Drive and others.

DBT Core/ Cloud Proficiency (nice to have):

- Experience in creating complex DBT models including full refresh, incremental models, snapshots, and documentation.
- Ability to write and maintain DBT macros for reusable code
- Experience in creating custom DBT macros using jinja and Python allowing for reusable components within DBT models

Monitoring and Logging (nice to have):

- Familiarity with AWS cloud watch, Data Dog for monitoring the pipelines and setting up alerts for workflow failures

#J-18808-Ljbffr

📌 AWS Data Eng (México)
🏢 Turtle Trax
📍 México