Data Engineer - GCP

Key Skills: GCP Data Services & Storage, Big Data & Processing Frameworks, Monitoring & Optimization, Project Management & Agile, ETL & Data Pipelines.

Roles & Responsibilities:

• Build and manage ETL pipelines using Google Cloud Dataflow.
• Design and implement data storage solutions using BigQuery and Google Cloud Storage.
• Develop and optimize data pipelines to handle large-scale data ingestion, transformation, and storage.
• Work closely with data scientists and analysts to ensure data quality, accuracy, and availability.
• Integrate Cloud Pub/Sub for real-time data streaming.
• Manage and monitor data flow processes and troubleshoot issues as they arise.
• Ensure the security of data through proper encryption, access control, and compliance with data privacy regulations.
• Collaborate with cross-functional teams to meet business requirements and optimize data workflows.

Skills Required:

• Google Cloud Storage (GCS) - Experience in handling structured and unstructured data, lifecycle management, and access control.
• Google Cloud Dataflow (Apache Beam) - Experience in building scalable ETL/ELT pipelines for batch and streaming data processing.
• Google Dataprep - Understanding of data preparation, cleansing, and transformation for analytics.
• IAM & Role-Based Access Control (RBAC) - Knowledge of access management and security best practices for GCP services.
• Google Cloud Operations Suite (Stackdriver) - Experience in monitoring, logging, and alerting for data workflows.

Educations: Graduate with a relevant field