About the position
Role Purpose
We are seeking a Data Engineer to design, build, and optimise scalable data platforms and pipelines that support analytics, reporting, AI, and business intelligence. You will transform complex, high-volume data into trusted, analytics-ready datasets while ensuring data quality, governance, security, and performance. Working closely with cross-functional teams, you will deliver modern cloud-based data solutions that enable clients to make data-driven decisions.
Key Responsibilities
- Design, develop, and maintain scalable ETL/ELT pipelines for batch and real-time data processing.
- Build and optimise data solutions using Azure, Microsoft Fabric, Databricks, Azure Data Factory, Synapse, Python, PySpark, and SQL.
- Develop and support enterprise data warehouses, data lakes, and cloud-based data platforms.
- Implement DataOps, CI/CD, automation, monitoring, and data quality best practices.
- Ensure data governance, lineage, metadata management, and compliance with security and POPIA requirements.
- Collaborate with Architects, Data Scientists, Business Analysts, and client stakeholders to deliver scalable data solutions.
- Provide production support, troubleshoot complex data issues, and continuously improve platform performance.
- Mentor junior team members and contribute to technical best practices.
Minimum Requirements
- Qualifications
- Bachelor's Degree in Computer Science, Information Technology, Engineering, Data Science, or a related field.
- Experience
- 4-6 years' experience in Data Engineering, Business Intelligence, or Data Platform development.
- Experience working with cloud data platforms and Agile delivery environments.
Technical Skills
Experience with most of the following:
- Microsoft Azure
- Azure Data Factory (ADF)
- Microsoft Fabric
- Azure Databricks
- Azure Synapse
- Python PySpark
- SQL Apache Spark
- Ab Initio
- Data Warehousing Data Lakes
- CI/CD, DataOps Workflow Orchestration
- Kafka, Azure Event Hub, or similar streaming technologies
- Git and Infrastructure as Code
Key Competencies
- Strong analytical and problem-solving skills
- Excellent communication and stakeholder management
- Client-focused with a collaborative approach
- Strong attention to detail and commitment to quality
- Passion for innovation and continuous learning
Desired Skills:
- Systems Analysis
- Complex Problem Solving
- Programming/configuration
- Critical Thinking
- Time Management
