About the position
Our client is seeking a highly skilled Microsoft Fabric Data Engineer to design, build, and operationalize modern data solutions using the Microsoft Fabric ecosystem. This is an exciting opportunity to work on cutting-edge Data & AI initiatives, enabling advanced analytics, business intelligence, and AI-driven solutions within a dynamic enterprise environment.
As a key member of the Data & AI practice, you will be responsible for developing scalable data platforms, robust ingestion pipelines, modern Lakehouse architectures, and high-performance analytics solutions while mentoring team members and driving data engineering best practices.
Key Responsibilities
Microsoft Fabric Data Engineering
- Design, develop, and maintain Microsoft Fabric solutions including:
- Lakehouse
- Warehouse
- Data Engineering workloads
- Data integration components
- Build and manage end-to-end data pipelines using Fabric Data Factory capabilities.
- Develop data ingestion, orchestration, and transformation processes.
- Implement transformation logic using:
- PySpark
- SQL
- Fabric-native technologies
- Design and maintain Medallion Architectures (Bronze, Silver, Gold layers).
- Lead migration initiatives from on-premises and cloud-based SQL environments into Microsoft Fabric.
Data Modelling & Analytics
- Design and implement dimensional data models and star schemas.
- Develop scalable analytics-ready datasets and semantic models.
- Optimize Lakehouse and Warehouse performance for reporting and analytics workloads.
- Enable downstream consumption through Power BI, AI, machine learning, and self-service analytics solutions.
Governance, Security & Quality
- Implement data quality frameworks, reconciliation checks, and monitoring processes.
- Ensure compliance with governance, security, and audit requirements.
- Manage data access controls, classification standards, retention policies, and lineage documentation.
- Support enterprise security models using Entra ID and role-based access controls (RBAC).
DevOps & Operational Excellence
- Apply modern engineering practices including:
- Git Version Control
- CI/CD Pipelines
- Automated Testing
- Release Management
- Environment Promotion
- Establish logging, alerting, monitoring, and observability for production pipelines.
- Troubleshoot and resolve production issues while driving continuous improvement initiatives.
- Optimise performance, scalability, reliability, and cost management within the Fabric ecosystem.
Leadership & Collaboration
- Mentor Data Engineers and Analysts.
- Contribute to solution design and technical leadership within the Data & AI practice.
- Engage with stakeholders to gather requirements and deliver fit-for-purpose solutions.
- Promote engineering excellence and knowledge sharing across delivery teams.
Required Skills & Experience
Strong hands-on experience with Microsoft Fabric
Data Engineering experience using:
- PySpark
- SQL
- Data Factory
- Lakehouse Architecture
Experience building and maintaining:
- Data Pipelines
- ETL/ELT Solutions
- Data Warehouses
- Data Lakes
Strong understanding of:
- Dimensional Modelling
- Star Schema Design
- Data Governance
- Data Quality Management
Experience integrating Microsoft Fabric with:
- Power BI
- Azure Data Services
- Enterprise Data Platforms
Knowledge of:
- Git
- CI/CD
- DevOps Practices
- Monitoring & Observability
Strong stakeholder engagement and problem-solving skills
Advantageous Experience
- Microsoft Fabric Certifications
- Azure Data Engineering Certifications
- Experience with AI/ML data platforms
- Experience migrating legacy SQL environments to Microsoft Fabric
- Experience in enterprise healthcare, financial services, or large corporate environments
Desired Skills:
- Microsoft Fabric
- Data Pipelines ETL/ELT
- Solutions Data Warehouse
- Pyspark
