Lead Data Engineer
Oct 30, 2024Data Analytics Permanent Australia, Full-Time
We need a Databricks/Azure wizard. We are building a large scale Data & Analytics platform on Azure / Databricks. We are looking for a Data Engineering Technical Lead to drive this initiative. This role involves leading the development of scalable, secure data solutions, mentoring a skilled team, and supporting the organization’s digital transformation.
Banking / Financial Services Sector: You will be responsible for all things design, architecture, pipelines, governance & implementation of greenfield warehouse. Working closely data architects, engineers, delivery leads & program manager to turn high-level architecture into actionable plans. Guide juniors and other key business units on best practises. There is no mess left behind to clean up on this one! It greenfield, all systems go & you will be the pilot.
Key Responsibilities
- Lead the creation of data pipelines, ETL processes, and data models on Azure Databricks, ensuring alignment with business needs and industry standards.
- Ensure best practices in data engineering, including quality, performance, security, and version control.
- Work with data architects and IT teams to integrate solutions and communicate insights to stakeholders.
- Guide and support the data engineering team, championing best practices and new approaches.
- Promote innovation in processes, tools, and technologies.
- Uphold data governance, security, and quality standards throughout the data lifecycle.
- Working in a delivery team of 8-10 Engineers in a wider data / technology function of 30-35.
Requirements
- 10+ years in Business Intelligence domains; Data / Software / Business Intelligence Engineering/Development
- Ability to take ownership of projects and previous experience in doing the same is essential
- Exposure across Data Architecture, Modeling & Pipeline development
- Strong background in ETL, Azure Cloud, Databricks, and DevOps.
- Strong experience in Python, SQL, PySpark & exposure in CI/CD
- Experienced in Power BI is nice to have as you will use infrequently
- Finance, Banking or FinTech experience highly regarded
This is a full greenfield project and a great opportunity to stamp your legacy in the Databricks world.
This position requires Full Australian work rights; Citizenship or Permanent Residency. Only applicants with the same will be considered.
Apply Online