About the position
The Data Engineer will be responsible for designing, building, and maintaining the data architecture and infrastructure necessary for effective data processing and analysis. This role involves collaborating with data analysts, and other stakeholders to ensure that data is accessible, reliable, and optimised for use in various applications. The candidate will be required to deliver to all stages of the data engineering process – data ingestion, transformation, data modelling and data warehousing, and build self-service data products.
Key responsibilities:
- Work closely with end-users and Data Analysts to understand the business and their data requirements.
- Carry out ad hoc data analysis and transform data using Databricks, MS Fabric, or Synapse Analytics.
- Building meta-data driven data ingestion patterns using Azure Data Factory and Databricks, or MS Fabric, or Synapse
- Build and maintain the on-premises SQL Data Warehouse.
- Build and maintain business focused data products.
- Build and maintain Azure Analysis Services cubes.
- Work with Architecture and Engineering teams to deliver projects and ensure that supporting code and infrastructure follows best practices outlined by these teams.
- Collaborate with Data Engineering Lead, Data Domain Lead and Architect to drive group engineering context within the team continuously.
- Develop and promote exceptional engineering documentation and practices.
- Actively engage with the business units engineering community to promote knowledge sharing.
- Collaborate with squad leader, Head of, engineers, and architects to ensure cohesive strategies and practices within the team.
Requirements - Proven engineering expertise (7+ years)
- Excellent data analysis and exploration using T-SQL
- Strong SQL programming (stored procedures, functions)
- Extensive experience with SQL Server and SSIS
- Knowledge and experience of data warehouse modelling methodologies, in particular Star Schema and Snowflake Schema
- Experience in Azure – one or more of the following: Data Factory, Databricks, MS Fabric, Synapse Analytics, ADLS Gen2.
- Experience in building robust and performant ETL processes.
- Experience in using source control, CI/CD pipelines, and Azure DevOps.
- Build ‘infrastructure-as-code' deployment pipelines.
- Familiarity with modern engineering practices and data store design, in particular Lakehouse and Medallion architecture.
- Familiarity with the Data as a Product approach
- Excellent analytical and problem-solving skills, with the ability to think critically and strategically.
- Strong communication and interpersonal skills, with the ability to engage and influence stakeholders at all levels.
Desired Skills:
- T-SQL
- SQL
- Azure
- Data Factory
- Databricks
- MS Fabric
- CI/CD
Desired Qualification Level:
About The Employer:
