Databricks Engineer/Lead

• Implementing and maintaining Azure Databricks:
• Responsible for the implementation and maintenance of Azure Databricks.
• Data Processing: The candidate will be responsible for processing raw data, cleaning it, and transforming it into a more usable format.
• The candidate will work closely with data scientists and other stakeholders to ensure that the data is accessible and useful.
• Continuous Improvement: The candidate will be expected to continuously improve and optimize the system, ensuring its efficiency and effectiveness.

Expertise You'll Bring:

• Proficiency in Azure: The candidate must have a strong understanding and hands-on experience with Azure, Microsoft’s cloud computing platform.
• Expertise in Azure Databricks: The candidate should have a deep understanding of Azure Databricks, an Apache Spark-based analytics platform.
• Knowledge of Databricks: The candidate should have a solid understanding of Databricks, a unified data analytics platform.
• A bachelor’s degree in computer science, Information Technology, Data Science, or a related field is required.
• Skills Experience with Python or Scala: Knowledge of these programming languages can be beneficial for data processing tasks.
• Familiarity with Spark: Experience with Apache Spark, a fast and general engine for large-scale data processing, is a plus.
• Knowledge of SQL: Understanding of SQL for managing data held in a relational database management system.
• Experience with Hadoop: Familiarity with the Hadoop software library, a framework that allows for the distributed processing of large data sets.
• Understanding of ETL processes: Knowledge of Extract, Transform, Load (ETL) processes can be beneficial.
• Familiarity with Machine Learning: Experience with machine learning algorithms and concepts