Senior Data Engineer

Job Description: Requirement: Senior Data Engineer Experience: 8+ years Location: Bengaluru, Gurugram, Hyderabad Work Mode: Work from home  Job Description: Solution Architect (Databricks) Responsibilities: ? Enforce adherence to architectural standards, global product-specific guidelines, and usability design standards. ? Proactively guide engineering methodologies, standards, and leading practices. ? Identify, communicate, and mitigate risks, assumptions, issues, and decisions throughout the full lifecycle. ? Provide guidelines for best practices and repeatable methodologies in Cloud Data Engineering, including: ? Data Storage ? ETL (Extract, Transform, Load) ? Data Integration & Migration ? Data Warehousing Data Governance Required Skills: Experience: ? 9+ years in Data Engineering project development/design, handling large volumes of data. ? At least 5+ years using cloud data engineering services (preferably on AWS). ? 5+ years as a Big Data architect/solution architect on Big Data platforms. ? 3+ years of proven experience as an architect/solution architect on the Databricks platform. Project Design & Implementation: ? Designed and implemented at least 2-3 end-to-end projects in Databricks. ? Expertise in E2E architecture of unified data platforms covering data lifecycle aspects: Data Ingestion, Transformation, Serving, and Consumption. ? Experience in composable architecture to fully leverage Databricks capabilities. ? Implemented and configured Databricks environments (clusters, notebooks, libraries) for optimal performance and resource utilization. Technical Skills: ? Designed and developed scalable batch and streaming data pipelines, data lake architectures, and data warehousing solutions on the Databricks platform using Spark and Delta Lake. ? Knowledgeable in the Databricks Lakehouse concept and its enterprise implementation. ? Strong understanding of data warehousing, governance, and security standards related to Databricks. ? Skilled in cluster optimization, integration with various cloud services, and performance optimization to improve efficiency and reduce costs. ? Proficient in writing unit and integration tests, and setting best practices for Databricks CI/CD. ? Experience in integrating data from various sources (structured, semi-structured, and unstructured) into Databricks for processing and analysis. Qualifications: ? Proven experience as an Architect/Solution Architect on Databricks. ? Hands-on experience with AWS Databricks platform for data processing, warehousing, and analytics solutions. ? Strong background in cloud data engineering, ETL, data integration, and data governance. ? Looking forward to receiving applications from candidates who meet these criteria and are excited to join our team.