Making Sense of Flask APIs A Billion Requests at a Time
10:15am - 10:45am on Friday, October 5 in MadisonEvan Morikawa
- Audience Level:
- Intermediate
- Slides:
- http://evanmorikawa.com/talks/pygotham-2018-10
- Watch:
- https://youtu.be/KgqbDquYIR8
Overview
Learn how to scale Flask APIs and make sense of the data. The answer is a mix of team, tools, and process. I’ll demo logging, visualization, and alerting techniques that should leave you feeling confident your APIs are performing, and assured you can respond quickly if something goes wrong.
Description
You’ve got a Flask, Django, or other API. How are people using it? Is anything going wrong? How does it perform? These questions become increasingly hard to answer once manually tailing your log files becomes impractical.
This talk will describe how to effectively answer these questions. I’ll demonstrate a suite of tools we use to quickly visualize insights across dozens of API endpoints and billions of requests. I’ll show logging techniques to more consistently store, organize, and process the right data to make sense of it all. I’ll talk about the processes we’ve put in place to routinely revisit these questions and quickly alert when things go awry. Finally, despite all the pretty graphs I’ll show you, I’ll discuss how at the end of the day it is the team and cultural aspects that keep any API in line and growing.
These stories, best practices, and advice come from my experience building and scaling the Nylas APIs. Our APIs allow developers to GET, POST and PUT email threads, calendar events, contacts, and any other item you’d find in an Exchange, GSuite, or IMAP mailbox. Hundreds of thousands of mailboxes later, and over a hundred million requests a day, making sense of what’s going on in the system at any given time present challenges I’m excited to share.