Collaborate closely with cross-functional teams including Data Scientists, Analysts, and Software Engineers to understand data requirements and translate them into efficient solutions using AWS Glue and AWS services.
Develop and maintain ETL processes using AWS Glue to facilitate seamless and reliable data extraction, transformation, and loading.
Implement robust data security measures and access controls in alignment with company policies and industry best practices.
Monitor, troubleshoot, and enhance data pipelines, identifying and resolving performance bottlenecks, data quality issues, and other challenges.
Stay up-to-date with the latest advancements in AWS Glue and AWS services, and advocate for their effective utilization within the organization
Experience/Minimum Requirements
Proven experience as a Data Engineer, with a strong emphasis on AWS Glue and AWS services.
In-depth understanding of architecture, performance optimization techniques, and best practices.
Proficiency in SQL and experience with database design principles.
Hands-on expertise in designing, building, and maintaining complex ETL pipelines using and AWS Glue.
Familiarity with data warehousing concepts and methodologies.
Competence in cloud computing and AWS services, with a focus on data-related services such as S3, Redshift, and Lambda.
Proficiency in scripting and programming languages such as Python, Java, or similar
Any Gradute