Azure Data Engineer

We are seeking a highly skilled Advanced Data Engineer with a strong background in Databricks and a passion for data engineering. The ideal candidate will have extensive experience in designing and implementing data solutions using various technologies, including PySpark, Azure Data Factory, and Informatica Data Cloud Management. This role requires a proactive individual who can work independently and collaboratively in a fast paced environment.

Responsibilities:

• Design, develop, and maintain scalable data pipelines using Databricks and PySpark.
• Implement data integration solutions using Azure Data Factory and manage data storage in Azure Storage Accounts.
• Utilize Informatica Data Cloud Management for data governance and quality assurance.
• Work with SQL, NoSQL, DB2, and JDBC to manage and query data effectively.
• Automate data processes and reporting using Microsoft Excel and Notepad++.
• Collaborate with cross functional teams to define data requirements and deliver solutions that meet business needs.
• Utilize version control systems such as GitHub and CI/CD tools like GitHub Actions and Jenkins for code management and deployment.
• Participate in Agile project management practices using JIRA and Confluence.
• Document processes and maintain knowledge bases in OneDrive.

Mandatory Skills:

• Proven experience with Databricks as a primary data processing tool.
• Strong proficiency in PySpark for data transformation and analysis.
• Hands on experience with Azure Data Factory and Azure Storage Accounts.
• Solid understanding of SQL and NoSQL databases, including DB2.
• Familiarity with data integration tools, specifically Informatica Data Cloud Management.

Preferred Skills:

• Experience with automation in Microsoft Excel.
• Knowledge of messaging systems such as MQ.
• Familiarity with testing and quality assurance tools like QMetry.
• Experience in using JIRA, Confluence, and OneDrive for project management and documentation.

Qualifications:

• Bachelor's degree in Computer Science, Information Technology, or a related field.
• 5 7 years of experience in data engineering or a related role.
• Strong analytical and problem solving skills.
• Excellent communication and collaboration abilities.
• Ability to work independently and manage multiple tasks effectively