Databricks consultant

Job Description:

• Design, code, test, document, and maintain high-quality and scalable data pipelines/solutions in cloud .

• Ingest and transform data using variety of technologies from variety of sources (API’s, streaming, Files, Databases ….).

• Develop reusable patterns and encourage innovation that will increase team’s velocity.

• Design and develop applications in an agile environment, deploy using CI/CD.

• Participate with prototypes as well as design and code reviews, own or assist with incident and problem management.

• The ideal candidate will be a self-starter who can learn things quickly, who is enthusiastic and actively engaged.

Required 3+ years of experience with the following technologies:

o Databricks, Python, Spark, pyspark, SQL, Azure Data factory.

Required 5+ years of experience with the following technologies:

o Design and Implementation of Datawarehouse/Datalake [Databricks/snowflake].

o Data architecture, Data modelling.

Required 2+ years of experience with the following technologies:

o Github actions/Jenkins or similar CICD tool, Cloud CICD, GitHub.

o Scheduling tools like Airflow / Tivoli Work scheduler.

o NoSQL and relational databases.

Good to have :

o Apache Kafka.

o Data ingestions from variety of API’s.

• o Working in Agile/Scrum environment