- Design, develop, and maintain scalable data platforms and data pipelines on Microsoft Azure.
- Build robust batch and real-time data processing solutions using Azure Databricks, PySpark, and Structured Streaming.
- Develop and optimize data ingestion frameworks for integrating data from multiple structured and unstructured data sources.
- Design and implement modern data lake, data warehouse, and serverless architectures using Azure-native services.
- Collaborate with business stakeholders, architects, and engineering teams to understand data integration, migration, and analytics requirements.
- Develop and maintain data transformation, cleansing, and preprocessing workflows to support enterprise reporting and analytics.
- Monitor and support Azure DevOps pipelines, Databricks workloads, and production data processing environments.
- Implement best practices for data governance, security, performance optimization, and operational excellence.
- Participate in troubleshooting, root cause analysis, and resolution of data platform and pipeline issues.
- Contribute to architecture discussions and recommend scalable, cost-effective cloud-based data solutions.
- Support continuous improvement initiatives related to monitoring, logging, automation, and platform reliability.
- 6–8 years of experience in Data Engineering, Data Platform Development, or Cloud Data Solutions.
- Strong hands-on expertise in Python for data engineering and data processing applications.
- Extensive experience with SQL and NoSQL database technologies.
- Proven experience building batch and streaming data pipelines using Azure Databricks and PySpark.
- Strong knowledge of Azure services including Azure Data Lake Storage (ADLS), Azure Databricks, Event Hub, Azure SQL Data Warehouse, Azure Functions, and Serverless Architecture.
- Good understanding of Infrastructure as Code concepts, particularly Terraform.
- Experience designing and implementing enterprise-scale data lake and data warehouse solutions.
- Knowledge of Azure DevOps, CI/CD processes, and cloud deployment methodologies.
- Strong analytical and problem-solving capabilities with the ability to work independently.
- Excellent communication and stakeholder management skills.
- Design end-to-end data engineering solutions from high-level architectural requirements.
- Build scalable and efficient data ingestion, transformation, and processing pipelines.
- Work with diverse data sources and support enterprise data migration initiatives.
- Optimize data platform performance, reliability, and operational efficiency.
- Collaborate with architects, business users, and development teams to deliver high-quality data solutions.
- Troubleshoot complex data engineering challenges and implement sustainable solutions.
- Adapt quickly to evolving Azure technologies, monitoring tools, and cloud services.
- Ensure data quality, security governance, and compliance standards are maintained.
- Opportunity to work on modern cloud-native data engineering and analytics initiatives.
- Exposure to large-scale Azure data platforms, streaming architectures, and enterprise data ecosystems.
- Collaborative environment with experienced cloud, data, and engineering professionals.
- Challenging projects involving advanced data processing, cloud transformation, and innovation.
- Opportunities for technical leadership, professional development, and continuous learning.
- A culture focused on innovation, knowledge sharing, and engi neering excellence.
Salarisomschrijving
€60000 - €80000 monthly