Data Engineer (Python)

About the position

• Design, develop, and maintain scalable data pipelines to ingest, process, and store structured and unstructured data from multiple sources.


• Develop ETL/ELT processes to transform raw data into clean, analysable formats for reporting, analytics and dashboards.


• Work with cloud data platforms


• Ensure data security and compliance


• Integrate and consolidate data from multiple sources, including SSE platforms, enterprise systems, and external APIs.


• Optimise data pipelines for performance, reliability, and scalability.


• Collaborate with other teams to ensure timely and accurate data delivery.


• Perform data validation and quality checks, identifying anomalies, gaps or inconsistencies and resolving them proactively.


• Automate data ingestion, transformation, and integration.


• Support analytics, machine learning, and business initiatives where necessary.


• Continuously evaluate and recommend improvements in data architecture, cloud integrations, and security practices.


• Troubleshooting and resolving data-related issues across the environments, ensuring minimal downtime or disruption.


• Own the development, maintenance, and operational support of a cloud-based AI platform, ensuring scalability, reliability, and performance for AI workloads.


• Translate business requirements into data-driven solutions.


• DevOps

Minimum Requirements:

• Qualification in Computer Science, Information Technology, Data/Software Engineering, Data Science or equivalent.


• Cloud Computing or Analytics is a plus.


• Data Science / Machine Learning


• Data Engineering


• MLOps


• Python


• CI/CD


• IaC


• Terraform


• APIs


• Any additional responsibilities assigned in the Agile Working Model (AWM) Charter


• Knowledge of cloud data platforms


• Experience with big data frameworks


• Competency in streaming technologies


• Understanding data warehousing and Lake house concepts


• Experience with Infrastructure-as-code and DevOps practices


• Understanding of data security, governance, and compliance.


• Experience in implementing and monitoring solutions

Desired Skills:

• Data Engineering
• ETL
• Python