08 nov
|
Coforge
|
México
We are looking for a Senior Databricks Engineer with strong hands-on experience designing, developing, and optimizing data pipelines using Databricks and PySpark. The adecuado candidate should have a solid understanding of modern data architectures,
particularly the Medallion Architecture, as well as experience supporting machine learning workflows and model inferencing.
The candidate should also be familiar with AI-powered development tools, including
Agentic IDEs, to accelerate and improve data engineering workflows. Experience in financial services, particularly Credit Decisioning, Risk Management, or Loan Servicing and
Collections, is highly desirable.
Key Responsibilities
- Design, build, and maintain scalable data pipelines using Databricks and PySpark.
- Develop and optimize ETL/ELT processes to support large-scale data processing.
- Implement and maintain data solutions following the Medallion Architecture (Bronze, Silver, and Gold layers).
- Leverage Agentic IDEs and AI-powered development tools to accelerate pipeline development and improve engineering productivity.
- Support machine learning workflows, including model deployment, inference, and integration with data pipelines.
- Work with MLflow to manage and support machine learning lifecycle activities.
- Ensure data quality, reliability, performance,
and scalability across data pipelines.
- Collaborate with data scientists, architects, and business stakeholders to translate business requirements into technical solutions.
Required Qualifications
- Strong hands-on experience with Databricks.
- Advanced programming skills in Python and PySpark.
- Proven experience building and maintaining complex data pipelines.• Strong understanding of Medallion Architecture and modern data engineering practices.
- Experience working with MLflow and understanding machine learning workflows, particularly model inferencing.
- Experience using Agentic IDEs or AI-assisted development environments for software and data engineering.
- Strong SQL skills and experience working with large datasets.
- Ability to troubleshoot, optimize, and improve data pipeline performance.
Preferred Qualifications
- Experience in financial services, banking, or fintech environments.
- Domain knowledge in one or more of the following:
- Credit Decisioning
- Risk Management / Credit Risk
- Loan Servicing
- Collections
• Understanding of how data pipelines support credit risk assessment, lending decisions, and financial operations.
- Experience working with production machine learning environments and model integration.
📌 Senior Databricks Engineer (México)
🏢 Coforge
📍 México