Python and Data Pipelines:
- Advanced Proficiency in Python concepts like Code Structures, Modules, Packages, Class, SubClass, Inheritance, Multi-Threading and Functional Programming.
- Experience in developing reusable Python packages for internal or public usage
- Ability to write automating ETL processes and scheduling jobs like Airflow DAG.
- Ability to track job pipeline runs to reprocess error records
- Ability to orchestration different pipelines to run in sequence or parallel.
- Troubleshoot data pipeline errors and fix issues
- Export or Import data to/from various formats like CSV, JSON, XML etc preferably from S3 or other cloud storage.
- Experience in using AI IDE tool
SQL:
- Advanced SQL skills, including complex joins, CTE's and subqueries
- Experience in optimizing SQL queries for performance and optimization in data warehouse technologies preferably Snowflake
Testing and documentation:
- Proficiency in Python unit, integration and system test.
- Proficiency in implementing DBT tests for data validation and quality checks
Code Generation:
- Experience in generating code using configurations using python and jinja templates
Version control:
- Experience in GitHub, including implementing CI/CD process from scratch
AWS Expertise:
- Data Storage solutions:
- In depth understanding of AWS S3 for data storage, ECS, IAM including best practices for organization and security
- Cloud Security:
- Knowledge of AWS security best practices, including IAM roles, encryption standard, secure coding guidelines, DBT profiles access configurations and more.
Data Integration (nice to have):
- Experience with AWS lambda for serverless data processing tasks
Workflow Orchestration (nice to have):
- Proficiency in using Apache Airflow on AWS to design ,schedule and monitor complex data flows
- Ability to integrate Airflow with AWS services and DBT models such as triggering a DBT model or EMR or reading from s3 w
📌 AWS Data Eng (Centro)
🏢 Turtle Trax
📍 Centro
Postulate a este anuncio
Muestra tus habilidades a la empresa, rellenar el formulario y deja un toque personal en la carta, ayudará el reclutador en la elección del candidato.