Description

Job Description
DataProc Oozie, AirFlow, SQL GCP Data Engineering, Cloud Composer, Big Data Spark, PySpark

 

Design and develop data-ingestion frameworks, real-time processing solutions, and data processing and transformation frameworks leveraging open source tools and data processing frameworks. Hands-on on technologies such as Kafka, Apache Spark (SQL, Scala, Java), Python, Hadoop Platform, Hive, airflow Experience in GCP Cloud Composer, Big Query, DataProc Offer system support as part of a support rotation with other team members. Operationalize open source data-analytic tools for enterprise use. Ensure data governance policies are followed by implementing or validating data lineage, quality checks, and data classification. Understand and follow the company development lifecycle to develop, deploy and deliver the solutions. Minimum Qualifications: • Bachelor's degree in Computer Science, CIS, or related field • 5-7 years of IT experience in software engineering or related field • Experience on project(s) involving the implementation of software development life cycles (SDLC)

External Skills And Expertise

Design and develop data-ingestion frameworks, real-time processing solutions, and data processing and transformation frameworks leveraging open source tools and data processing frameworks. Hands-on on technologies such as Kafka, Apache Spark (SQL, Scala, Java), Python, Hadoop Platform, Hive, airflow Experience in GCP Cloud Composer, Big Query, DataProc Offer system support as part of a support rotation with other team members. Operationalize open source data-analytic tools for enterprise use. Ensure data governance policies are followed by implementing or validating data lineage, quality checks, and data classification. Understand and follow the company development lifecycle to develop, deploy and deliver the solutions. Minimum Qualifications: • Bachelor's degree in Computer Science, CIS, or related field • 5-7 years of IT experience in software engineering or related field • Experience on project(s) involving the implementation of software development life cycles (SDLC)

Skills
dataproc oozie
airflow
sql gcp data engineering
cloud composer
big data spark
pyspark

Key Skills
Education

Any Graduate

Back To Jobs