**Job Summary**
**Responsibilities**
- Develop and optimize data workflows using Spark in Scala to ensure efficient data processing and analysis.
- Implement and manage Databricks Unity Catalog Admin to maintain data governance and security.
- Utilize Databricks CLI for seamless integration and automation of data tasks.
- Design and schedule workflows with Apache Airflow to ensure timely data pipeline execution.
- Manage data storage and retrieval using Amazon S3 and Amazon Redshift for robust data solutions.
- Write and maintain Python scripts to support data processing and analysis tasks.
- Leverage Databricks SQL for querying and analyzing large datasets effectively.
- Implement Databricks Delta Lake to ensure data reliability and consistency.
- Create and manage Databricks Workflows to automate and streamline data processes.
- Collaborate with cross-functional teams to deliver high-quality data solutions that meet business needs.
- Ensure data integrity and quality through rigorous testing and validation processes.
- Contribute to the continuous improvement of data practices and technologies within the organization.
**Qualifications**
- Possess strong expertise in Spark in Scala and Databricks Unity Catalog Admin.
- Demonstrate proficiency in using Databricks CLI and Apache Airflow.
- Have experience with Amazon S3 and Amazon Redshift for data management.
- Show advanced skills in Python and Databricks SQL.
- Be knowledgeable in Databricks Delta Lake and Databricks Workflows.
- Exhibit strong problem-solving skills and attention to detail.
- Be able to work effectively in a hybrid work model with day shifts.