Data Engineer (Databricks)

JOB SUMMARY

We are seeking an experienced Data Engineer with strong hands-on Databricks expertise to support the design, development, and optimization of enterprise data solutions. The successful candidate will have deep experience building and maintaining ELT pipelines, working with large-scale data platforms, and applying modern data architecture principles. This role requires alignment to UK business hours and on-site presence at our delivery center.

JOB RESPONSIBILITIES

• Design, develop, and maintain ELT pipelines using Databricks and related technologies.
• Build and optimize large-scale data processing workflows using PySpark and SQL.
• Develop and manage data models to support analytics, reporting, and business intelligence requirements.
• Implement and support data Lakehouse architectures and data warehousing solutions.
• Collaborate with data analysts, architects, and business stakeholders to understand and deliver on data requirements.
• Ensure data quality, integrity, and governance across all data platforms.
• Monitor and troubleshoot data workflows, pipeline performance, and platform issues.
• Contribute to continuous improvement of data engineering standards and practices.
• Support and work during UK business hours as required.

REQUIRED TECHNICAL SKILLSET

• Strong hands-on experience with Databricks (Delta Lake, Databricks Workflows, Unity Catalog).
• Proven experience building and supporting ELT pipelines at scale.
• Strong proficiency in PySpark and SQL.
• Experience with data modeling techniques and best practices.
• Good understanding of data Lakehouse architecture and data warehousing concepts.
• Experience with version control tools (e.g., Git).
• Familiarity with cloud platforms (Azure preferred; AWS or GCP considered).

PREFERRED EXPERIENCE

• Experience with orchestration tools such as Apache Airflow or Azure Data Factory.
• Exposure to data governance frameworks and tools (e.g., Unity Catalog, Microsoft Purview).
• Experience in enterprise consulting or regulated industry environments.
• Experience working with globally distributed teams across different time zones.

CERTIFICATIONS PREFERRED

• Databricks Certified Associate Developer for Apache Spark.
• Databricks Certified Data Engineer Associate or Professional.
• Microsoft Certified: Azure Data Engineer Associate (DP-203) or equivalent.
• Additional data or cloud certifications are a plus.