Cloud Data Engineer at Datonomy Solutions

About the position

Role Purpose
The Cloud Data Engineer is responsible for designing, developing and maintaining enterprise-grade cloud data solutions that enable advanced analytics, reporting and business intelligence across the bank.

The role focuses on delivering scalable, secure and high-quality data products on Microsoft Azure by building robust ETL/ELT pipelines, integrating multiple data sources through APIs, and ensuring reliable data availability for downstream consumers. The successful candidate will provide technical leadership in cloud data engineering while promoting engineering best practices, automation and modern data architecture.

Key Responsibilities
Cloud Data Engineering

• Design, develop and maintain scalable cloud-native data solutions on Microsoft Azure.
• Build enterprise-grade data products for analytics, reporting and AI initiatives.
• Develop reliable ETL/ELT data pipelines to ingest, transform and publish data.
• Optimise data processing for performance, scalability and cost efficiency.
• Implement reusable engineering patterns and data engineering standards.

Azure Data Platform

• Develop solutions using Azure Data Factory (ADF).
• Build data processing pipelines using Azure Databricks.
• Develop solutions on Azure Synapse Analytics.
• Work with Azure Data Lake Storage (ADLS Gen2).
• Integrate Azure SQL Database, SQL Server and other enterprise data sources.
• Support cloud migration and modernisation initiatives.

Data Integration

• Design and implement API integrations.
• Consume REST APIs and enterprise services.
• Integrate structured and semi-structured data sources.
• Develop secure and scalable ingestion frameworks.
• Automate data ingestion from multiple internal and external platforms.

Data Quality & Governance

• Ensure data accuracy, consistency and completeness.
• Implement monitoring and alerting for pipeline failures.
• Perform root cause analysis of data quality issues.
• Support metadata management and data governance initiatives.
• Ensure compliance with security and regulatory requirements.

Advanced Analytics Enablement

• Prepare clean, trusted datasets for Data Scientists and BI teams.
• Optimise datasets for reporting and advanced analytics.
• Support machine learning and AI workloads.
• Enable self-service analytics capabilities.

Cloud Engineering Best Practices

• Participate in solution design and architecture discussions.
• Implement CI/CD pipelines for data engineering deployments.
• Use Infrastructure as Code where appropriate.
• Follow DevOps and Agile delivery methodologies.
• Perform code reviews and mentor junior engineers.

Minimum Requirements
Qualifications

• Degree in Computer Science, Information Technology, Engineering or related field.

Experience

• 5–8+ years Data Engineering experience.
• 3+ years Microsoft Azure Data Platform experience.
• Experience building enterprise cloud data solutions.
• Experience working within Agile delivery teams.
• Banking or Financial Services experience advantageous.

Technical Skills
Microsoft Azure

• Azure Data Factory (ADF)
• Azure Databricks
• Azure Synapse Analytics
• Azure Data Lake Storage Gen2 (ADLS)
• Azure SQL Database
• Azure Storage
• Azure Key Vault
• Azure Functions (advantageous)
• Azure Event Hub (advantageous)

Programming

• SQL (Advanced)
• Python
• PySpark
• Spark
• JSON
• YAML

Data Engineering

• ETL / ELT
• Data Warehousing
• Data Modelling
• Data Pipelines
• Batch Processing
• Streaming Data (advantageous)
• API Integration
• REST APIs

DevOps

• Azure DevOps
• Git
• CI/CD Pipelines
• Automation
• Monitoring

Advantageous Skills

• Microsoft Fabric
• Power BI
• Delta Lake
• Lakehouse Architecture
• Data Governance
• Purview
• Terraform
• Docker
• Kubernetes

Desired Skills:

• Azure Data Factory
• ADF
• Azure Databricks