Senior Data Engineer - Databricks

Must Have Skills

3+ years’ of hands-on experience in delivering data solutions
Experience with Databricks implementations - developing data pipelines using PySpark
Experience in Databricks Workspaces,Notebooks, Delta Lake, APIs
Experience in Python, Scala
Experience in advanced SQL and Query optimization 
Experience in implementing data integration projects using ETL 
Experience building and deploying cloud-based solutions at scale by ingesting data from sources like DB2

Good To Have Skills

Experience with AWS services - S3,Redshift,Secrets Manager
Experience in implementing data integration projects using ETL (preferably Qlik Replicate & QlikCompose)
Experience in using Airflow or Step Functions for orchestration
Exposure to IaC tools like Terraform and to CI/CD tools
Prior experience in migrating (on-prem to cloud) and processing large amounts of data
Experience in implementing data lake and data warehouse on cloud
Experience in implementing the industry's best practices