Description

Key Responsibilities

  • Design and implement robust, scalable, and secure data architectures using Databricks, Spark, Delta Lake, and cloud-native tools.
  • Collaborate with data engineers, analysts, and business stakeholders to define data models, pipelines, and governance strategies.
  • Develop and maintain data lakehouses, ensuring optimal performance and cost-efficiency.
  • Define best practices for data ingestion, transformation, and storage using Databricks notebooks, jobs, and workflows.
  • Architect solutions for real-time and batch data processing.
  • Ensure data quality, lineage, and compliance with internal and external standards.
  • Lead migration efforts from legacy systems to modern cloud-based data platforms.
  • Mentor junior team members and evangelize data architecture principles across the organization.

Required Skills & Qualifications

  • 12+ years of experience in data architecture, with 5+ years hands-on in Databricks.
  • Strong Experience in Snowflake
  • Experience in cloud platforms AWS, especially AWS Databricks.
  • Strong proficiency in Apache Spark, Delta Lake, and PySpark.
  • Experience with data modelling, ETL/ELT pipelines, and data warehousing.
  • Familiarity with CI/CD, DevOps, and Infrastructure as Code (Terraform, ARM templates).
  • Knowledge of data governance, security, and compliance frameworks.
  • Excellent communication and stakeholder management skills.

Preferred Qualifications

  • Databricks Certified Data Engineer or Architect.
  • Experience with MLflow, Unity Catalog, and Lakehouse architecture.
  • Background in machine learning, AI, or advanced analytics.
  • Experience with tools like Apache Airflow, dbt, or Power BI/Tableau

Education

Any Gradute