Data Engineer

Intro

Come and support Company.info in the biggest transformation in the company’s history by strengthening our data engineering capabilities. We are seeking a highly skilled and motivated Senior Data Engineer to join our team. You will be responsible for designing, building, and maintaining our data pipelines and work closely with other teams to ensure that our data is clean, accurate, and readily available to support our client's business decisions.

Functieomschrijving

This is what you will do…

We are working on a multi-year big program where we completely redefine the way we build (data) products, including our current architecture and organisation. We do this using a domain-driven architecture, with the goal of improving and aligning all our data processing and technical foundation. Part of successfully transforming the organization is to focus more on boosting our data engineering capabilities.

You will work on ingesting, processing, and integrating data products for a specific domain. Some of the tasks that you will be involved with include:

• Design and implement data modelling architecture.
• Build production grade data pipelines and different processing paradigms (streaming, batch, and graph).
• Ingest data from a wide variety of sources with different sizes and formats.
• Use automated deployment tools and templates to deploy the solutions! (CI/CD)
• Work with Spark Structured Streaming, Delta Live Tables, and Change Data Capture to refresh data products that require low latency!

Our stack is mainly composed of the following tools: Python, Databricks, PySpark, AWS S3, EventBridge, and Lambdas.

Culture / the team …

You will be working together in a passionate team that continues to strive for being the first/best in the market, modern management, no hierarchy, and an informal atmosphere. Our Product & Tech department consists of 40 people. We are all passionate about our products, technology, and data.

Currently, we are transitioning towards working in domain-based teams, with skill & expertise-based chapters. We are looking for an experienced data engineer to strengthen our domain teams. In domain teams, we aim to have a combination of data engineers, back-end engineers, and a flexible pool of data scientists and analysts.

Currently, the data engineering chapter consists of five data engineers, who work together in one of the domain teams. We expect the data engineering chapter to grow to 6-8 people in the next 2 years. In terms of management style, the organization is flat, relaxed, and clear. The atmosphere is open, pleasant, social, and fun. In this role, you will have the opportunity to specialise in specific knowledge areas and become a technical lead for this domain. So, lots of opportunities for personal and technical development and growth!

This is what you’ll get from us…

We offer you a dynamic environment, internationally oriented with innovative, challenging, and diverse projects, using cutting-edge technology and directly benefiting our clients and our business managers at the same time.

• € 70.000 - € 78.000 euro per year including 8% holiday allowance, based on 40 hours per week.
• Profit sharing.
• Great premium-free pension scheme.
• Flexible working hours and working in a hybrid way.
• We will make sure you have all the software and hardware (your MacBook of choice) required to perform at your highest level.
• Because we work with the latest technologies, we offer the opportunity to keep on developing yourself by taking courses, attending (and presenting at) conferences, e.g., free full access to Databricks Academy.
• Very modern working environment, the office is new and specially furnished to the wishes of Company.info. The office is centrally located in Amsterdam and easily accessible by public transport.

Functie-eisen

You already have this knowledge and experience…

• You have at least 5+ years of relevant working experience in the IT or data sector.
• Extensive experience in production-grade data ingestion, processing, and modelling.
• Experience with cloud environments (preferably AWS).
• Preferably you have a Bachelor or Master in Computer Science, Software engineering, AI, or a related field.

Key technical skills and requirements

• Proficiency with Spark (preferably Databricks platform).
• Extensive experience with software development (Python).
• Experience with shipping production-grade software with CI/CD.
• Experience with AWS services (Lambda, Eventbridge), data stores (MySQL, PostgreSQL, Elasticsearch), and infrastructure as code (Terraform, CloudFormation).

Competencies

• Strong sense of ownership and great problem-solving skills.
• Drive to learn our data domains and develop innovative solutions.
• Like to coach more junior members and achieve team goals.
• Great communication skills (written and spoken), in English.

At Company.info, we recognize the value of diverse backgrounds, ideas, and experiences, and strive to create a work environment where everyone feels accepted and appreciated. We encourage applicants from all backgrounds to be part of our team, as we believe this contributes to a dynamic and inspiring work culture.