10 oct
|
DigitalT3
|
México
Location:
100% Remote – Mexico, India or Sri Lanka
Experience:
5+ years of experience in Platform Engineering, Cloud Engineering, DevOps, Data Infrastructure, or related engineering roles.
Role Overview
We are seeking an experienced
Databricks Platform Engineer
to design, build, automate, and operate our enterprise Databricks Lakehouse platform on AWS. This role bridges
Cloud Platform Engineering, Data Infrastructure, and Data Engineering
, with a strong focus on infrastructure automation, security, governance, and standardization.
The idóneo candidate will use
Terraform and Infrastructure as Code (IaC)
to deliver scalable and secure Databricks environments while establishing enterprise data governance through
Databricks Unity Catalog
.
Key Responsibilities
Infrastructure as Code & Platform Automation
Design, build, and maintain reusable
Terraform modules
for Databricks infrastructure across Development, Test, and Production environments.
Automate provisioning and lifecycle management of
Databricks workspaces, compute resources, SQL warehouses, networking, security configurations, and storage integrations
.
Manage IaC for AWS components including
VPCs, security groups, IAM roles, private connectivity, and storage endpoints
.
Develop and maintain
CI/CD pipelines
using GitHub Actions, GitLab CI, or similar platforms.
Implement deployment automation using
Databricks Asset Bundles, Databricks APIs, Terraform, and related automation frameworks
.
Establish reusable platform patterns and self-service "golden paths" for Data Engineering, Data Science, and Analytics teams.
Unity Catalog & Data Governance
Design and implement the enterprise
Unity Catalog architecture
, including catalogs,
schemas, volumes, storage credentials, and external locations.
Define governance standards that provide appropriate workload and team isolation while enabling secure data sharing and discoverability.
Implement and manage
Role-Based Access Control (RBAC)
and object-level permissions based on least-privilege principles.
Integrate Databricks access controls with enterprise identity and access management platforms.
Implement appropriate
row-level and column-level security
controls.
Establish data governance capabilities including
lineage, auditing, classification, tagging, and access controls
.
Automate governance policies through Terraform, APIs, and other platform automation mechanisms.
Platform Operations & Security
Design and maintain
compute and cluster policies
to ensure standardized and secure platform usage.
Manage platform configuration, upgrades, capacity, and operational support.
Partner with AWS Cloud and Security teams to implement
IAM controls, encryption, KMS integration, network
security, and vulnerability remediation
.
Troubleshoot Databricks platform, infrastructure, networking, access, and configuration issues.
Develop operational standards, architectural documentation, troubleshooting procedures, and platform runbooks.
Required Skills & Qualifications
Experience:
5+ years of experience in Platform Engineering, Cloud Engineering, DevOps, Data Infrastructure, or related engineering roles.
Databricks:
Strong hands-on experience administering enterprise Databricks environments, including workspaces, compute, SQL warehouses, cluster policies, jobs, and platform security.
Unity Catalog:
Practical experience designing and implementing Unity Catalog, including catalogs, schemas, external locations, storage credentials, permissions, and governance controls.
Terraform:
Advanced experience developing reusable Terraform modules and managing Terraform state, providers, and multi-environment deployment patterns.
AWS:
Strong hands-on experience with
IAM, S3, VPC networking, security groups, KMS, and PrivateLink/private connectivity
.
CI/CD:
Experience developing automated deployment pipelines using
GitHub Actions, GitLab CI, Jenkins, or equivalent platforms
.
Automation:
Experience with
Databricks APIs, Databricks Asset Bundles, CLI tools, and infrastructure automation frameworks
.
Programming:
Proficiency in
Python and SQL
for automation, validation, and troubleshooting.
Security:
Strong understanding of cloud security, identity management, least-privilege access, encryption, and secrets management.
Preferred / Nice-to-Have Skills
Experience with additional
AWS data services
, such as DynamoDB, Redshift, Glue, Athena, and Kinesis.
Familiarity with
Azure data services
, such as Synapse Analytics, Data Factory, ADLS, Cosmos DB, and Microsoft Fabric.
Exposure to modern
cloud data, streaming, analytics, and AI/ML services
across AWS and Azure.
#J-*****-Ljbffr
📌 Databricks Platform Engineer (Mexico/India - Remote) (México)
🏢 DigitalT3
📍 México