Data Engineer

Skills :

1. Skills Must have

a. Databricks

b. Python

c. SQL

d. Spark

e. Jenkins

f. Airflow

g. Bitbucket

2. Skills Nice to have

a. Python

b. Understanding of any ETL tool( Azure data factory, Informatica etc )

Roles & Responsibilities:

 Has more than 5 Years of experience in developing data lakes, and datamarts on the Databricks

platform.

 Develop and maintain ETL processes using Databricks notebooks.

 Implement and optimize data pipelines for data transformation and integration.

 Proven skill sets in AWS Data Lake services such as - AWS Glue, IAM, and skills in Spark,

Python, and SQL.

 Write complex SQL queries for data retrieval, manipulation, and analysis.

 Perform database performance tuning and optimization.

 Good understanding of developing data warehouses, data marts, etc.

 Has a good understanding of system architectures, and design patterns and should be able to

design and develop applications using these principles.

 Good collaboration and communication skills

 Excellent problem-solving skills to be able to structure the right analytical solutions. Strong sense

of teamwork, ownership, and accountability

 Analytical and conceptual thinking

 Ability to work in a fast-paced environment with tight schedules.

 Good presentation skills with the ability to convey complex ideas to peers and management.

Key Responsibilities:

 Design of data solutions on Databricks including delta lake, data warehouse, data marts and

other data solutions to support the analytics needs of the organization.

 Apply best practices during design in data modeling (logical, physical) and ETL pipelines

(streaming and batch) using cloud-based services.

 Interact with stakeholders regarding data landscape understanding, conducting discovery

exercises, developing proof of concepts, and demonstrating it to stakeholders.

 Code Review, candidate should have knowledge on bitbucket merge