Duration: 6 Months (Possibility for extension)
Location: London/Hybrid (3 days per week on site)
Rate: A highly competitive Umbrella Day Rate is available for suitable candidates
Role Profile
We are seeking a highly skilled Senior Data Engineer with strong expertise in Python and Databricks to design, build, and optimise scalable data pipelines and platforms. You will play a key role in enabling data-driven decision-making by delivering reliable, efficient, and high-quality data solutions.
Key Responsibilities
Design, develop, and maintain scalable data pipelines using Python and Databricks
Build and optimise ETL/ELT workflows leveraging Databricks (Spark, Delta Lake)
Develop robust data models and data architectures on Lakehouse platforms
Implement and manage data workflows within the Databricks ecosystem
Ensure high data quality, governance, and reliability across pipelines
Optimise performance of large-scale distributed data processing
Collaborate with data scientists and analysts to support analytics and ML
Implement monitoring, logging, and alerting for data systems
Mentor junior engineers and drive engineering best practices
Skills & Experience
Extensive experience in data engineering
Strong programming expertise in Python
Hands-on experience with Databricks (workflows, notebooks, jobs, Delta Lake)
Strong experience with Apache Spark / PySpark
Experience building scalable ETL pipelines
Proficiency in SQL and relational databases
Experience with cloud platforms (AWS / Azure / GCP)
Experience with modern data warehousing solutions (e.g., Snowflake, BigQuery, Redshift)
Familiarity with orchestration tools (e.g., Airflow, Databricks Workflows)
Strong understanding of data modelling and lakehouse architecture
Databricks certifications (e.g., Databricks Certified Data Engineer)
Experience with streaming data pipelines (Kafka, Structured Streaming)
Experience with CI/CD and DevOps practices
Knowledge of Delta Lake features (time travel, CDC, optimisation)
Experience with containerisation (Docker, Kubernetes)
Exposure to ML pipelines on Databricks
Candidates will need to show evidence of the above in their CV in order to be considered.
If you feel you have the skills and experience and want to hear more about this role 'apply now' to declare your interest in this opportunity with our client. Your application will be observed by our dedicated team.
We will respond to all successful applicants ASAP however, please be advised that we will always look to contact you further from this time should we need further applicants or if other opportunities arise relevant to your skillset.
Pontoon is an employment consultancy. We put expertise, energy, and enthusiasm into improving everyone's chance of being part of the workplace. We respect and appreciate people of all ethnicities, generations, religious beliefs, sexual orientations, gender identities, and more. We do this by showcasing their talents, skills, and unique experience in an inclusive environment that helps them thrive.
As part of our standard hiring process to manage risk, please note background screening checks will be conducted on all hires before commencing employment.
We use generative AI tools to support our candidate screening process. This helps us ensure a fair, consistent, and efficient experience for all applicants. Rest assured, all final decisions are made by our hiring team, and your application will be reviewed with care and attention
Salary description
£600.00 - £700.00 per day