As Takeaway.com, we use a lot of data sources to generate new insights and sales leads. You, as part of the Data Engineering Team, are responsible for supporting the existing platform as well as creating new components and building POCs, and for all steps that are required to ensure stable runs in the production environment. You need to be able to perform large scale data mining, verify data integrity as well as to transform it to various formats requested by stakeholders in order to support their operational work, using the best tools for the job.
- Collect and transform unstructured data from different sources to structured output, be it columnar data store, flat or Parquet/ORC files, NoSQL, or streams.
- Apply known “best practice” / “lessons learned” knowledge to create reliable, multiple times a day or continuously running agents, possibly aware of model changes and other possible occurring issues
- Create reusable, maintainable, scalable (vertical, horizontal) integrations/services using a cutting-edge hybrid cloud infrastructure
- Collaborate excellently with the team and stakeholders in agile iterative processes
- You have an extensive and proven professional experience (above +4 years) handling big data pipelines, data warehouses, or other (preferably distributed) data stores.
- You have worked with distributed data warehouses like Redshift/Hive/Cassandra etc or with database management systems like Postgres/Oracle/MySQL etc.
- You are comfortable with transformation tools and workflow management systems (eg Airflow/Luigi)
- Experience in the cloud is preferred (AWS/GCP/Azure) and it would be good if you have worked with Spark/Hadoop
- You are familiar with parsing structured and unstructured data and creation of ETL pipelines for data warehousing
- You have knowledge about how to model data and processes, how to test data, how to implement proper logging, and how to troubleshoot issues swiftly.
- You are experienced in SQL, Bash or another shell scripting, and one or several programming languages - preferably Python/Scala
- You do not have to wait for a DevOps engineer to hand you servers and would rather be able to deploy services on the cloud using Continuous Integration / Continuous Deployment
- You like clean code and ideas and are able to strike a nice balance between speed of development and documentation plus testing.
- You are proficient in written and spoken English
What we deliver
- A challenging position within globally operating fast-growing, a venturous tech company with a strong market position in growing markets;
- Being part of an awesome young, professional and international team;
- Custom workstations/laptops;
- Fresh fruits every day;
- Internal as well as external training and conferences;
- An office location within the heart of Amsterdam (5 min. from Central Station);
- Table tennis, FIFA 2019, Friday drinks, daily fresh fruit and great events such as a snow trip.
Er is geen minimale opleiding vereist
Voor deze functie is er geen minimale werkervaring vereist