- Design, develop, and maintain scalable and reliable data processing solutions using Azure Databricks.
- Build and manage robust batch and streaming data pipelines within Databricks environments.
- Develop and optimize data transformation and processing solutions using Python, PySpark, and SQL.
- Design and optimize data models to support scalable processing, performance, and reliability.
- Manage multiple parallel data processing workflows and shared data sources efficiently.
- Implement and maintain CI/CD pipelines using Azure DevOps and YAML-based configurations.
- Apply Infrastructure as Code (IaC) using ARM/Bicep for deployment and infrastructure automation.
- Monitor, troubleshoot, and optimize data processing workloads and Databricks environments.
- Collaborate with cross-functional engineering and business teams to deliver reliable and maintainable data solutions.
- Contribute to Agile development practices and continuously improve engineering standards, system stability, and performance.
- Strong hands-on experience with Azure Databricks as a core data engineering platform.
- Strong proficiency in Python, PySpark, and SQL.
- Hands‑on experience developing batch and streaming data pipelines.
- Experience with data modeling, transformation, and optimization within Databricks environments.
- Good understanding of Azure cloud services relevant to data engineering.
- Experience with Azure DevOps, CI/CD, and YAML‑based pipeline configurations.
- Hands‑on experience with Infrastructure as Code, particularly ARM/Bicep.
- Experience working in Agile engineering and delivery environments.
- Understanding of modern cloud‑based data architectures and end‑to‑end data engineering solutions.
- Strong communication, collaboration, troubleshooting, and problem‑solving skills.
- Build scalable, high‑performance, and reliable data processing solutions using Azure Databricks.
- Develop efficient PySpark and SQL‑based data transformations.
- Design and manage complex batch and streaming workloads.
- Optimize data pipelines, processing performance, and resource utilization.
- Troubleshoot complex data engineering issues and improve system reliability.
- Implement automated deployment and infrastructure management practices.
- Make pragmatic technical decisions while maintaining scalability and maintainability.
- Work effectively with engineering, architecture, and business stakeholders.
- Drive continuous improvement and maintain high standards of code and solution quality.
- Opportunity to work on enterprise‑scale Azure and Databricks data engineering initiatives.
- Exposure to modern cloud‑based data platforms and engineering practices.
- A collaborative Agile environment focused on technical excellence and innovation.
- Opportunities to work with advanced data processing, pipeline engineering, and cloud technologies.
- Continuous learning and opportunities for technical and professional growth.
- A culture focused on quality, ownership, scalability, and sustainable engineering solutions.
Salarisomschrijving
€90000 - €130000 monthly