- Infrastructure & Networking
- Design secure, scalable AWS environments using Multi-Account architectures.
- Configure advanced AWS networking including VPCs, Transit Gateways, and Route53.
- Establish secure connectivity via AWS Direct Connect and VPNs.
- Implement strict Network Access Control Lists (NACLs) and Security Groups.
- Migration of on-premises systems to AWS cloud services.
- Perform regular updates and upgrades to AWS resources and services Data Platform Engineering
- Deploy and manage Databricks workspaces across multiple AWS environments.
- Optimize Databricks clusters for performance, scalability, and cost.
- Integrate Databricks with AWS data stores like S3, RDS.
- Implement Unity Catalog for robust data governance and access control.
- Automation & CI/CD
- Write clean, reusable, and modular infrastructure code using Terraform.
- Manage infrastructure state files securely in remote backends.
- Design and maintain CI/CD pipelines using Jenkins for automated testing.
- Automate the deployment of both AWS resources and Databricks assets.
- Security & Operations
- Implement IAM roles and policies following the principle of least privilege.
- Monitor system health and performance using AWS CloudWatch and Prometheus.
- Set up automated alerts for infrastructure anomalies and cost spikes.
- Troubleshoot complex networking, deployment, and data pipeline failures.
- Required Technical Skills
- Cloud Platform: Advanced expertise in Amazon Web Services (AWS).
- Data Tools: Hands-on administration of Databricks (Workspaces, Clusters, Unity Catalog).
- Infrastructure as Code: Expert-level proficiency with Terraform.
- CI/CD: Deep experience building and managing Jenkins pipelines.
- Networking: Strong mastery of TCP/IP, DNS, VPC peering, and routing tables.
- Scripting: Proficiency in Python, Bash