Data Engineering Lead:
Job Description:
- We are looking for a Data Tech Lead to join our Data Engineering practice, delivering modern data platforms for enterprise and regulated clients.
- This is a hands-on leadership role for a technically strong engineer who can lead design and delivery, act as a trusted technical authority, and mentor engineers while maintaining strong engineering discipline in real-world environments.
- This is a delivery-focused role for an experienced engineer and Tech Lead who enjoys solving complex data challenges, takes ownership of outcomes, and applies strong engineering discipline in real-world environments.
Responsibilities:
- Leading end-to-end delivery of data engineering workstreams.
- Shaping solution approach.
- Setting technical direction.
- Ensuring outcomes are delivered to a high standard.
- Designing and building scalable batch and streaming pipelines.
- Operating production systems.
- Providing technical governance and oversight: design reviews, implementation guidance, risk/issue escalation, and delivery assurance.
- Building strong stakeholder relationships.
- Operating as a credible client-facing technical lead.
- Coaching and mentoring engineers.
- Supporting learning plans.
- Helping develop capability standards and accelerators.
- Partnering with business and technology stakeholders to build and enhance reliable, scalable data platforms.
- Developing high-throughput, low-latency data processing systems.
- Working within the Databricks and Apache Spark ecosystem to support large-scale data workloads.
- Building and maintaining cloud-native data solutions on AWS.
- Using dbt for data transformation, modelling, testing, and documentation.
- Orchestrating data workflows using Airflow or similar orchestration tools.
- Writing clean, efficient, and well-tested code following software engineering best practices.
- Collaborating closely with product, analytics, and platform teams.
- Supporting production systems by troubleshooting issues and implementing continuous improvements.
- Ensuring solutions are production-ready: secure, tested, observable, and cost-efficient.
- Optimising data performance, reliability, and data quality in live environments.
- Recent hands-on Databricks experience.
- Strong use of Python and SQL.
- Experience making design or architecture decisions.
- Direct stakeholder/customer engagement.
- Exposure to complex enterprise data projects.
Qualifications:
- 8-10 years’ experience in hands-on data engineering, including technical leadership on production platforms.
- Experience providing technical governance (design reviews, standards, delivery assurance) and supporting delivery at pace.
- Comfortable contributing to estimation and commercial discussions and ensuring solutions are feasible and deliverable.
- Proven ability to mentor and help develop engineers through coaching, feedback, and structured development support.
- Strong hands-on coding skills in Python, Java, or similar languages, with strong SQL capability.
- Proven experience building and operating batch and streaming pipelines in production.
- Experience with AWS, Databricks / Spark, and modern data architectures.
- Hands-on experience using dbt for transformations and Airflow (or equivalent) for orchestration.
- Solid understanding of data modelling, data quality, and production data systems.
- Experience working with relational databases such as Postgres or SQL Server.
- Familiarity with CI/CD pipelines, testing frameworks, and monitoring / observability tooling.
Nice to Have:
- Experience enabling ML, AI, or GenAI workloads through strong data foundations.
- Exposure to modern BI and analytics tooling.
- Experience working in regulated or security-conscious environments.
- Consulting or client-facing delivery experience.
