agileengine, llc. is seeking a senior data engineer to design and operate batch and streaming pipelines on databricks using pyspark and delta lake. The role focuses on migrating legacy workloads to a governed lakehouse and implementing medallion architecture for analytics and ai use cases.
you will work with a strong emphasis on sql, python, unity catalog governance, and modern data quality practices, collaborating across devops and analytics teams.