Responsibilities;
* Providing data sources that help build the AI/ML user experience
* Build and maintain scalable data pipelines for BigQuery using Cloud Composer and Airflow.
* Provide observability and monitoring using Monte Carlo and Looker as well as operational tools such as NewRelic and Splunk, driving reliability, data quality, data quality and robustness.
* Scope and plan work together with product engineering, data science and the business; support other teams for all tasks related to data tracking, collection, ingestion, quality monitoring and alerting.
Experience
* 5+ years data analysis experience developing applications including Scala/Java and/or Python.
* Experience managing batching and/or streaming data pipelines and infra structure using technologies such as GCP, Dataflow/Apache Beam, Dataproc, Spark/Flink, dbt;
* Working knowledge of SQL and shell scripting;
* Experience with Docker/k8s and “infrastructure as a code” tools such as Terraform, Salt or similar.
* Confident at articulating complex technical content to peers and partners across the organization via Technical Design Documents.
This is a great opportunity to further your experience across technologies such as Cloud Composer, Airflow, Dataflow, Spark, Terraform & Google Cloud Platform.
Benefits
* Hybrid working
* Long contract
* Award winning work environment
Salary description
£500.00 - £500.00 per day