Functional Responsibilities:
Designing and implementing Data pipelines. Experience profiling data sets and developing scripts to improve data quality.
Collaborating with cross functional teams to identify and solve complex data problems.
Developing and maintaining ETL processes.
Monitoring and optimizing data pipelines for performance and reliability.
Ensuring data quality and integrity.
Working with large datasets and designing scalable data solutions.
Experience and Qualifications:
Demonstrate background experience and knowledge base in Data Engineering and functional responsibilities described. Bachelor's degree in computer science,
5+ years of experience as a Data Engineer, Strong SQL skills, Strong Python skills and optimizing data pipelines for performance and reliability, Ensuring data quality and integrity,
Working with large datasets and designing scalable data solutions, Strong Python skills and experience with Python data libraries, e.g., Pandas, PySpark (Desired)
Experience working with Azure Synapse Analytics (Desired)
Experience with ETL tools such as Azure Data Factory or Apache NiFi. (Desired)
Experience with Big data technologies such as Parquet, Hadoop, or Spark.
Bachelor's degree in Computer Science