Senior Data Architect

Skills : spark, HDFS, hive, presto, pyspark, SQL, AWS cloud

This position is for a Principal Software Engineer who is passionate about developing high performance data platforms. To be successful in this role, you need effective communication and collaboration skills, a passion for learning and trying new things, and a strong foundational understanding of programming and computer science.

As a member of our team, you will apply your knowledge and skills to help us deliver scalable, performant, maintainable, and testable software.

Responsibilities:

• Work with a distributed team of engineers across multiple products building software collaboratively.
• Work cross-team to build consensus on approach for delivering projects
• Collaborate with business partners to understand and refine requirements
• Eliminate ambiguity in projects and communicate direction to engineers to help team members work in parallel
• Ramp up quickly on existing software to deliver incremental, integrated solutions in a complex environment
• Build high-performance, stable, scalable systems to be deployed in an enterprise setting
• Lead high-level architecture discussions and planning sessions
• Participate in the code review process by assessing pull requests
• Support systems and services during production incidents as part of the on-call rotation
• Author and recommend technical proposals and root cause analyses
• Provide mentoring and advice for other specialists
• Establish engineering practices and standards within the team to drive quality and excellence

Qualifications:

• Bachelor's degree in computer science, Information Systems, Software, Electrical or Electronics Engineering, or comparable field of study, and/or equivalent work experience.
• Minimum of 15+ years of related big data engineering experience modeling and developing large data pipelines and platforms
• Hands-on experience with distributed systems such as Spark, Hadoop (HDFS, Hive, Presto, PySpark) to query and process data
• Expert in Python and SQL skills processing big datasets
• A strong grasp of computer science fundamentals (data structures, algorithms, databases, etc.)
• Experience with at least one major MPP or cloud database technology (Snowflake, Redshift, Big Query, Databricks)
• Expert in Data Modeling techniques and Data Warehousing standard methodologies and practices
• You are a problem solver with strong attention to detail and excellent analytical and communication skills
• In-depth experience with Cloud technologies like AWS (S3, ECS, EMR, EC2, Lambda, etc.)
• Solid experience with data integration toolsets (i.e Airflow), CI/CD
• Experience using source control systems and CI/CD pipelines.

Preferred Qualifications:

• Proficient with Scrum and Agile methodologies
• Experience directly managing and mentoring a team