Serverless machine learning in production: manage models, not servers

2:00pm - 2:25pm on Saturday, October 6 in Madison

William Cox

Audience Level:

Intermediate

Watch:

https://youtu.be/IXqkDClqyuc

Overview

Putting machine learning models into production is difficult - especially for a team that isn’t cross-function and doesn’t have strong ops or software engineering support. We used severless technology to deploy a ML system that let data scientists rapidly put models they trained into production

Description

Putting machine learning models into production is difficult. It’s especially hard to do for a team that isn’t cross-function and doesn’t have strong ops or software engineering support. At Distil we used severless technology to deploy a Python-based machine learning system that let data scientists rapidly put models they trained into production for detecting automated threats on websites. This talk will discuss how we rapidly built this pipeline, what we learned and what we’d do differently in the future. We make hundreds of predictions per second and stream the results to hundreds of machines all without having to manage any machines or hardware.