ETL Data Architect

Must Have Skills:
• ETL pipelines Architecture using AWS
• Exploratory Data Analysis
• Spark

Nice to Have Skills:

Detailed Job Description:
As a Senior ETL Architect, you will partner closely with the Client’s Network and Telecom Team to design and drive the end-to-end ETL/data-integration and prediction solution for a network-anomaly detections and diagnosis. You will translate client requirements into a Statement of Work, Architect Data pipelines, to handle ~25 TB/month Log data in S3 (~2 TB/day ETL workloads), and lead a global delivery team consisting of Onsite and Offshore members. You will ensure best practices in predictive modeling, anomaly detection, and exploratory data analysis, leveraging both existing tools (Splunk, BigPanda) and AWS services.

Key Responsibilities:
• Collaborate with client stakeholders to define Scope and Functional Requirements to build the Solution Architecture
• Define Scalable ETL pipelines Architecture using AWS (S3, Glue, EMR, Kinesis, Lambda) for Data ingestion and for Processing large-scale Network and Firewall log data
• Lead design of Data Models and Storage Strategies optimized for Anomaly Detection and Analytics
• Oversee Predictive-Modeling and Patterns Analysis, and define Real-time Alerting approaches
• Guide Exploratory Data Analysis (EDA) and POC development using Python, Spark and (optionally) Scala
• Integrate and extend Splunk/BigPanda or AWS ML services (SageMaker) for prediction workflows
• Define coding standards, governance and CI/CD practices for the onshore/offshore development teams
• Mentor and review deliverables from offshore DEs to ensure quality, performance and security compliance

Required Qualifications:
• Bachelor’s / Master’s in Computer Sciences Engineering, or other related technical fields
• 10+ years in Data Engineering/Data Architecture, with hands-on ETL and Pipeline Design
• Proven track record architecting large-scale AWS data solutions (S3, Glue, EMR, Kinesis, Lambda)
• Deep expertise in Python and Spark; Scala experience is a plus
• Experience integrating Splunk, BigPanda or equivalent monitoring/alerting platforms
• Strong communication skills and experience leading distributed teams in Agile environments

Preferred Qualifications:
• AWS Certified Solutions Architect (Associate or Professional)
• Prior experience in Firewall Analytics or Telecom Domain
• Familiarity with Syslog/log analytics, Network Telemetry and anomaly-detection frameworks