About the position
Our client is seeking a Senior Cloud Data Engineer to join their Engineering team at the company. In this role, you will be a key technical contributor responsible for building, optimizing, and maintaining high-performance data processing engines.
You will apply your deep expertise in Spark Best Practices to develop modular, portable PySpark applications, ensuring the companies platform remains provider-agnostic while operating within modern environments like Microsoft Fabric.
As an experienced member of the Data Engineering team, you will focus on refactoring legacy logic into scalable Python-centric solutions, optimizing the companies’ data lakehouse architecture, and contributing to the overall technical excellence of their data platform.
Required Skills and Experience
- Senior-Level Spark: 6+ years of experience with Spark/PySpark, with the ability to diagnose performance bottlenecks via the Spark UI and optimize complex DAGs.
- Advanced Python: Strong proficiency in production-grade Python, with experience building reusable libraries and implementing automated testing.
- SQL Proficiency: Solid T-SQL skills to accurately interpret and migrate existing logic into the new Python-centric environment.
- Azure Synapse Analytics & Dedicated SQL Pools: Solid experience working within Azure Synapse, specifically with Dedicate SQL Pools and Data Factory working with complex pipelines
- Data Lakehouse Expertise: Hands-on experience with Delta Lake and Parquet, specifically in high-volume environments.
- Agnostic Mindset: Experience working with open-source standards and containerization (Docker) to ensure data workloads can be ported across different cloud infrastructures.
- Collaborative Engineering: Proven track record of working within a team to successfully deliver and modernize large-scale data workloads.
- Ensure high levels of security, compliance, and data governance across data engineering processes.
Collaboration & Leadership
- Work closely with data scientists, analysts, and business stakeholders to understand data requirements and develop solutions that meet business objectives.
- Lead initiatives to enhance data engineering capabilities, introduce new technologies, and drive best practices.
- Mentor junior engineers, conduct code reviews, and contribute to building a culture of technical excellence.
- Communicate effectively with technical and non-technical stakeholders, translating complex data concepts into actionable insights.
Required Qualifications
- Bachelor’s degree in computer science, Information Systems, Engineering, or a closely related field.
Why you’ll love working for the company
They believe in taking care of their team and creating an environment where you can thrive. As part of their company, you’ll enjoy:
- Flexible Working Arrangements: Whether you’re a night owl or an early bird, the company offers hybrid and remote options to suit your lifestyle
- Comprehensive Benefits: From a wellness program to home office reimbursements and continuous learning opportunities, the company has got you covered.
- Team Culture: Fun team-building activities, regular socials, and a supportive, inclusive culture that values transparency, accountability, and work-life balance.
- Performance Incentives: Competitive salaries, ESOP, and recognition for your hard work.
Desired Skills:
- Senior Data Engineer
- Spark/PySpark
- Advanced Python
- SQL Proficiency
- Azure Synapse
- Data Lakehouse
- Docker
