Description:
Key Responsibilities:
Designing and implementing scalable processes to manage large data sets, ensuring quality and consistency across sources. This involves developing and maintaining data pipelines, creating optimal extraction patterns, and aligning cloud strategy with data architecture. As the Data Engineer, you will also engineer data formats for analytics, develop APIs for applications, and collaborate with stakeholders to deliver solutions while driving process improvements, enhancing scalability, and supporting testing and deployment of new features.
Minimum Requirements:
At least 4 years of relevant data engineering experience Bachelor's degree in computer science, business informatics, mathematics, statistics, engineering, or related field Strong understanding of data structures, algorithms, and software design Experience with structured and unstructured data, data stores, RDBMS, and data warehouses Proficiency in cloud tools (e.g., Redshift, Athena, Glue, Lambda) Proficiency in programming languages such as Python, Scala, Java, or C Experience with Apache Spark and cloud services (AWS, Azure, GCP) Knowledge of version control systems like Git and SVN Familiarity with DevOps practices, including CI/CD and Infrastructure as Code Exposure to AWS platform environments such as CodePipeline, CodeBuild, CodeDeploy Experience with big data, ETL, and data management processes Experience in API development and cloud architecture Strong analytical and problem-solving abilities Ability to work collaboratively in an agile environment
Additional Skills & Attributes needed:
Ability to work effectively with cross-functional teams Strong communication skills to articulate complex data concepts to non-technical stakeholders Continuous learning mindset and commitment to professional development Innovative thinking and flexibility to adapt to changing business needs Integrity, accountability, and strong work ethic Enthusiasm for experimenting with new technologies and methodologies
09 Apr 2025;
from:
gumtree.co.za