Location – Hybrid Leeds of London 3 days
Drive a major cloud data modernisation programme, migrating legacy Databricks pipelines and workloads to a modern AWS infrastructure. Working closely with Architects and stakeholders, you will build automated, large-scale data solutions to secure and transform critical healthcare data assets. Requirement: (SC Clearance or eligibility is mandatory).
Key Responsibilities
Data Engineering & Migration
Support migration of data pipelines, notebooks, and data processing workloads between cloud environments.
Analyse existing data engineering solutions and identify migration requirements.
Develop and maintain data processing pipelines using modern cloud data platform technologies.
Support testing, validation, and reconciliation of migrated datasets.
Work with structured and semi-structured data stored in cloud environments.
Troubleshoot and resolve data processing issues across development, test, and production environments.
Cloud Data Platform Engineering
Support the operation and enhancement of cloud-based data platforms.
Assist with onboarding datasets, users, and workloads onto target platforms.
Collaborate with platform and architecture teams to implement scalable data solutions.
Support platform optimisation, monitoring, and operational readiness activities.
Infrastructure as Code
Develop and maintain Terraform code supporting data platform deployments.
Support deployment of data platform components, jobs, permissions, and environment configurations.
Assist with automation of deployment and release processes.
Maintain infrastructure consistency across environments.
Platform Modernisation
Support cloud migration initiatives and platform transformation programmes.
Contribute to platform standards, engineering practices, and reusable deployment patterns.
Produce technical documentation and implementation guides.
Essential Skills
Data Engineering
* Python
* PySpark
* SQL
* Data Pipeline Development
* ETL / ELT Design
* Data Modelling fundamentals
* Data Quality and Validation
Cloud Platforms
* AWS
* S3
* IAM
* Cloud-based Data Platforms
Infrastructure as Code
* Terraform
* Infrastructure Automation
* Git
Engineering Practices
* CI/CD concepts
* Agile Delivery
* DevOps principles
Desirable Skills
* Databricks
* Palantir Foundry
* Foundry Pipeline Builder
* Ontology concepts
* Delta Lake
* Healthcare or NHS data experience
* Secure Data Environment (SDE) experience
* Cloud migration programmes
Experience
5+ years of Data Engineering experience.
Experience delivering cloud-based data platforms.
Experience supporting migration of data workloads between environments.
Experience working with Infrastructure as Code using Terraform.
Experience working within regulated or enterprise-scale environments.
Role Senior Data Platform Engineer – Cloud Platform Migration
Type – Hybrid Leeds or London – 3 days a week £70,000 to £80,000 ++ depending on experience
Candidates must be SC Clearance or have SC eligibility is mandatory
SC eligibility - Security Check (SC) clearance in the UK, candidates cannot apply independently; the process must be sponsored by an employer or government department. You must be aa UK citizen or have the right to work in the UK and have resided in the UK for at least 5 years
Salary description
£70000.00 - £80000.00 per year