Skill
Must Have:
Kafka: Candidate should know how to handle JSN Data.
PySpark(70% of the project is based on it)
Spark Scala(30% of the project is based on it.)
Python Programming.
Scheduling tool like Airflow.
Good Knowledge in SQL & Hive. Majorly Hive.
Understanding in below skills:
CICD- Understanding on how does it work.
Data Visualization: PowerBI / Superset.
Good to have cloud GCP.
Experience:
5+ Overall & 3+ relevant.
8 overall & 6 relevant: immediate joiner (6-8 Yrs Exp) Big Data Spark Scala with Kafka
SQL and PL/SQL
Python
Pyspark
Kafka
AWS
Azure (DF, DB, ADSL, function app)
Power BI
Bachelor's degree in Computer Science