Spotify Open-Sources Terraform Module for Kubeflow ML Pipelines


Spotify has open-sourced their Terraform module for running machine-learning pipeline software Kubeflow on Google Kubernetes Engine (GKE). By switching their in-house ML platform to Kubeflow, Spotify engineers have achieved faster time to production and are producing 7x more experiments than on the previous platform.

In a recent blog post, Spotify’s product manager Josh Baer and ML engineer Samuel Ngahane described Spotify’s “Paved Road” for machine learning: “an opinionated set of products and configurations to deploy an end-to-end machine learning solution using our recommended infrastructure.”