Description:
Job Summary: The AWS Data Engineer is responsible for developing and maintaining the organizations data pipelines and systems on AWS. This role involves building data lakes, streaming pipelines, and batch processing systems that support analytics and business intelligence efforts.Key Responsibilities: Build and maintain scalable data pipelines using AWS Glue, EMR, Lambda, and Step Functions. Develop and manage data lakes using S3 and Lake Formation. Use Redshift and Athena for querying and analytical workloads. Create ETL/ELT jobs and manage workflow orchestration. Collaborate with data scientists, analysts, and other stakeholders. Ensure data quality, governance, and compliance. Monitor and optimize system performance and costs. Integrate with third-party APIs and internal systems. Skills & Experience: 3+ years experience working with AWS cloud services. Experience with streaming data using Kinesis or Kafka. Strong SQL and Python skills. Understanding of data governance and security in cloud environments. Familiar with Infrastructure as Code (IaC) tools like CloudFormation or Terraform. Strong proficiency in: AWS Glue S3 Redshift Lambda Step Functions
10 Apr 2025;
from:
gumtree.co.za