10 ago
|
DataArt
|
Monterrey
Client: Our client empowers partners to break away from the one-size-fits-all approach to payments, offering flexibility and innovation.
We pride ourselves on being creative problem solvers, helping visionaries push boundaries and explore new possibilities.
- Responsibilities: Design, implement, and manage data pipelines on GCP using tools such as Dataflow, BigQuery, Pub/Sub, Cloud Storage, and Cloud Composer
- Develop and maintain CI/CD pipelines for data workflows using tools like Cloud Build, Jenkins, or similar
- Automate the monitoring, alerting, and optimization of data processes to ensure efficient and reliable data flows
- Ensure data quality and integrity through monitoring, validation checks, and data auditing processes
- Collaborate with Data Engineers, Analysts, and Business teams to ensure data solutions align with business goals
- Optimize performance and cost-efficiency of data infrastructure by tuning storage and query performance on BigQuery and Cloud Storage
- Implement robust data security practices, including access management, encryption, and compliance with privacy regulations
- Create and maintain documentation for data processes,
including technical specs, workflows, and best practices
- Use Terraform or Deployment Manager to automate the provisioning and scaling of data infrastructure on GCP
- Requirements: Bachelor's degree in Computer Science, Data Engineering, or related fields
- 3+ years of experience in a DataOps, Data Engineering, or similar role
- Strong experience with Google Cloud Platform services like BigQuery, Dataflow, Pub/Sub, Cloud Composer, and GCS
- Proficiency in SQL and experience in managing big data environments and data lakes
- Hands-on experience with CI/CD pipelines for data processing and data automation
- Experience with infrastructure-as-code tools such as Terraform, Deployment Manager, or equivalent
- Solid understanding of data governance principles, including data lineage, security, and compliance
- Familiarity with containerization (Docker, Kubernetes) and orchestration tools
- Excellent problem-solving and debugging skills
- Strong communication skills and the ability to work cross-functionally with various teams
📌 Data Engineer With Expertise In Data Ops (Monterrey)
🏢 DataArt
📍 Monterrey