Lead Data Engineer

Oct 30, 2024

Data Analytics Permanent Australia, Full-Time

We need a Databricks/Azure wizard. We are building a large scale Data & Analytics platform on Azure / Databricks. We are looking for a Data Engineering Technical Lead to drive this initiative. This role involves leading the development of scalable, secure data solutions, mentoring a skilled team, and supporting the organization’s digital transformation.

Banking / Financial Services Sector: You will be responsible for all things design, architecture, pipelines, governance & implementation of greenfield warehouse. Working closely data architects, engineers, delivery leads & program manager to turn high-level architecture into actionable plans. Guide juniors and other key business units on best practises. There is no mess left behind to clean up on this one! It greenfield, all systems go & you will be the pilot.

Key Responsibilities

  • Lead the creation of data pipelines, ETL processes, and data models on Azure Databricks, ensuring alignment with business needs and industry standards.
  • Ensure best practices in data engineering, including quality, performance, security, and version control.
  • Work with data architects and IT teams to integrate solutions and communicate insights to stakeholders.
  • Guide and support the data engineering team, championing best practices and new approaches.
  • Promote innovation in processes, tools, and technologies.
  • Uphold data governance, security, and quality standards throughout the data lifecycle.
  • Working in a delivery team of 8-10 Engineers in a wider data / technology function of 30-35.

Requirements

  • 10+ years in Business Intelligence domains; Data / Software / Business Intelligence Engineering/Development
  • Ability to take ownership of projects and previous experience in doing the same is essential
  • Exposure across Data Architecture, Modeling & Pipeline development
  • Strong background in ETL, Azure Cloud, Databricks, and DevOps.
  • Strong experience in Python, SQL, PySpark & exposure in CI/CD
  • Experienced in Power BI is nice to have as you will use infrequently
  • Finance, Banking or FinTech experience highly regarded

This is a full greenfield project and a great opportunity to stamp your legacy in the Databricks world.

This position requires Full Australian work rights; Citizenship or Permanent Residency. Only applicants with the same will be considered.

Apply Online

Latest Jobs

View All

Data Engineer

Full-Time

  • Be apart of a high performing team building a greenfield Databricks Warehouse
  • Drive results and add to the design and implementation
  • Supported by a strong team
Read More

Lead Data Engineer

Full-Time

  • Greenfield Azure/Databricks Data Warehouse Build - From Scratch
  • Own everything from design, architecture, pipelines, governance & implementation
  • Career defining project
Read More

SAP Alliance and Marketing Lead

Sydney, New South Wales Full-Time

  • Flexible work + brilliant team
  • Work with key contacts across Australia to build pipeline
  • Dynamic role where marketing and sales collide
Read More

SAP Project Administrator

Melbourne, Victoria Full-Time

  • Jr Project Administration role with great career path
  • Well recognised SAP HCM brand
  • Work with a great leadership team in Melbourne
Read More