Lead Databricks Data Architect

Job Description

Roles & Responsibilities

• Architect and develop end-to-end data solutions using Azure Databricks, Azure Data Factory, and Azure SQL DB, focusing on high-performance, scalable, and secure data architectures.
• Lead the design and development of data ingestion pipelines utilizing Delta Live Tables (DLT) and Structured Streaming to handle both batch and real-time data ingestion requirements.
• Solution Design experience: Collaborate with stakeholders to understand business requirements and design scalable and optimized data bricks Data and Data Warehousing solutions on Azure
• Strong experience in driving initiatives towards organization vision and expertise on handling large programs
• Work experience with ADO, Agile Method, Sprint Planning and  
• Strong experience in Translating business needs into long-term data architecture solutions
• Optimize the performance of Spark jobs and Databricks workflows using advanced Spark internals, performance tuning, and job optimization techniques.
• Develop and maintain metadata-driven data ingestion frameworks to improve automation, scalability, and data pipeline management.
• Collaborate with cross-functional teams, including data scientists, analysts, and business stakeholders, to define data requirements and deliver solutions that align with business goals.
• Implement and enforce data quality checks, governance policies, and data security measures to ensure the integrity, availability, and confidentiality of data.
• Continuously monitor, optimize, and enhance the performance of cloud-based data infrastructure, while ensuring cost efficiency.
• WebForms, ASP.NET MVC, Core, Blazor.
• Frameworks 4.8.2 and Core 6, 8
• Webservices and APIs
• Must have experience with upgrades of .NET Framework to Cloud
• Must have experience with Nota Fiscal integration in a related sector
• Microsoft SQL Server (DDL, DML, Stored Procedures)
• Experience working with Azure DevOps Tools (Git repository, pipelines and managing workitems)
• Analyze and document defects and work with development team for defect resolution
• Lean-agile/SCRUM environment
• Windows Server troubleshooting (IIS, Event Viewer, Peformance Monitor)
• Azure services (authentication, key vault, etc)