Kafka Cloudera Spark Airflow Hadoop

Azure Data Factory Azure Synapse Analytics Azure  Databricks Synapse Pipelines Synapse Spark

ADF Synapse Analytics ADLS Azure Devops Databricks

Perl Python C C++ Java

PySpark ADF ADLS Delta tables SparkSQL

Spark streaming Kafka AWS Python Data Engineering

Java Spark Git AWS

Unify statistics data analysis Spark SQL/python adoop

Aws Python Pyspark Scala Pytest

Hadoop Hive Spark Sqoop Sql

Python PySpark Cloud (AWS) Agile deployments

Apache Spark Spark SQL PySpark MongoDB DynamoDB

Spark Hadoop MR Kafka Java Scala

AWS AWS Glue Spark Lambda Data Modelling

IICS PySpark Python ETL