Description:
Join a leading company pioneering the power of Artificial Intelligence (AI) to deliver actionable insights and drive transformative change for businesses worldwide. With a vision to provide next-generation insights by harnessing data through high-performance, interoperable, and simplified solutions, they are seeking an AWS Data Engineer to join their fast-paced, dynamic team.
Responsibilities
Design, build, and operationalize large-scale enterprise data solutions and applications. Re-architect and migrate on-premise data warehouses to AWS cloud platforms using native AWS services and Kafka CC. Build production-grade data pipelines from ingestion to consumption using Java, PySpark, Scala, and Kafka CC. Implement data engineering, ingestion, and curation functions on AWS cloud platforms using custom programming or AWS native tools. Assess existing data platforms and define optimal migration paths to AWS cloud. Develop and maintain analytical data infrastructure for large-scale datasets and high-performance computing. Collaborate with other tech teams to extract, transform, and load data from diverse sources using SQL, AWS big data technologies, and Kafka CC. Build and support real-time data pipelines using AWS services, including Glue, Lambda, Step Functions, PySpark, Athena, and Kafka CC. Stay updated on emerging big data and visualization technologies to enhance capabilities and efficiency.Qualifications and experience:
Bachelor''s Degree in Computer Science, Information Technology, or other relevant fields 5 8 years experience Has experience in any of the following AWS Athena and Glue Pyspark, DynamoDB, Redshift, Lambda, Step Functions and Kafka CC Proficient in AWS Redshift, S3, Glue, Athena, PySpark, Step Functions, Glue Workflows, Kafka CC, Delta.io Knowledge of software engineering best practices across the development lifecycle Experience with big data tools is a must: Delta.io, PySpark, Kafka, etc. Experience with AWS cloud services: EC2, EMR, RDS, Redshift. The Reference Number for this position is NG59270 which is a Contract position based in Braamfontein, Johannesburg offering a salary of R900k to R1.1mil cost to company per annum
14 Feb 2025;
from:
gumtree.co.za