Senior Data Engineer

Senior Data Engineer

Location: Bristol

Contract: Outside IR35

Rate: £68 per hour

Security Clearance: DV Cleared

Working Pattern: 5 Days On Site

Nationality Requirement: Sole British Nationals Only

THE ROLE

We are recruiting on behalf of a highly secure engineering and technology programme seeking an experienced Senior Data Engineer to support the design, development and delivery of secure, scalable and high-performance data solutions within a regulated environment.

This role will focus heavily on the Elastic Stack and Apache NiFi, supporting enterprise-scale data ingestion, transformation and analytics capabilities across complex secure systems.

The successful candidate will play a key role in architecting resilient data platforms, optimising real-time and batch processing pipelines, and ensuring compliance with strict governance and security standards.

KEY RESPONSIBILITIES

Data Architecture & Engineering

- Design and implement scalable, resilient data pipeline architectures using Elastic Stack and Apache NiFi

- Develop secure frameworks for data ingestion, transformation and integration

- Build and maintain batch and real-time data processing pipelines

- Ensure high standards of data quality, integrity and traceability across all systems

Elastic Stack & Platform Management

- Configure, optimise and maintain Elasticsearch clusters for performance and resilience

- Develop and maintain Logstash pipelines for efficient data processing

- Create dashboards and visualisations within Kibana to support operational and strategic reporting

- Monitor and manage large-scale data flows and platform performance in real time

- Carry out troubleshooting, root cause analysis and performance tuning activities

Security, Governance & Resilience

- Work closely with security and governance teams to ensure compliance with stringent security standards

- Implement secure data handling processes including encryption, RBAC and audit logging

- Support clustering, replication and failover strategies to ensure system availability

- Develop backup, disaster recovery and data retention strategies

Documentation & Collaboration

- Produce technical documentation covering architecture, workflows and security controls

- Support knowledge transfer and operational handovers to internal teams

- Collaborate with engineering, infrastructure and DevOps teams across the programme

WHAT WE'RE LOOKING FOR

- Strong hands-on experience with:

Elasticsearch
Logstash
Kibana
Apache NiFi
- Proven experience designing and implementing enterprise-scale data pipelines

- Background working within secure or regulated environments such as defence, government or finance

- Strong understanding of:

Data ingestion and processing frameworks
Real-time and batch processing architectures
Data modelling for analytics
Distributed system troubleshooting and optimisation
- Knowledge of:

Encryption and secure data handling
RBAC and audit controls
DevOps and CI/CD principles
Git/version control systems
- Active DV Clearance essential

- Sole British National essential

DESIRABLE EXPERIENCE

- AWS, Azure or GCP within secure environments

- Docker and Kubernetes

- Kafka, Spark or additional data engineering technologies

WHY APPLY?

This is an opportunity to work on highly secure, technically advanced programmes within a complex and mission-critical environment. You'll be joining a team delivering enterprise-scale data solutions where security, resilience and performance are central to operational success.

INDWIN

Our clients and their customers come from diverse backgrounds and so do we. We hire our people from various walks of life, each of whom make our company stronger with their talent, uniqueness, and expertise. This is what makes our company special; if you want to help us grow and take this ethos to our clients, then we cannot wait to collaborate with you!

The UK has now left the European Union. Any EU, EEA or Swiss citizens living in the UK that wish to remain in the UK post Brexit need to apply to the EU Settlement Scheme. Although the closing date for applications was 30th Jun 2021, if you have not yet applied but believe that you would qualify under the EU Settlement Scheme, the Home Office have confirmed that they will consider late applications. For further information please see (url removed) />
Many Thanks