Episode 11: Data Science: The Great Stagnation

Vanishing Gradients

Hugo speaks with Mark Saroufim, an Applied AI Engineer at Meta who works on PyTorch where his team’s main focus is making it as easy as possible for people to deploy PyTorch in production outside Meta.

Mark first came on our radar with an essay he wrote called Machine Learning: the Great Stagnation, which was concerned with the stagnation in machine learning in academic research and in which he stated

Machine learning researchers can now engage in risk-free, high-income, high-prestige work. They are today’s Medieval Catholic priests.

This is just the tip of the icebergs of Mark’s critical and often sociological eye and one of the reasons I was excited to speak with him.

In this conversation, we talk about the importance of open source software in modern data science and machine learning and how Mark thinks about making it as easy to use as possible. We also talk about risk assessments in considering whether to adopt open source or not, the supreme importance of good documentation, and what we can learn from the world of video game development when thinking about open source.

We then dive into the rise of the machine learning cult leader persona, in the context of examples such as Hugging Face and the community they’ve built. We discuss the role of marketing in open source tooling, along with for profit data science and ML tooling, how it can impact you as an end user, and how much of data science can be considered differing forms of live action role playing and simulation.

We also talk about developer marketing and content for data professionals and how we see some of the largest names in ML researchers being those that have gigantic Twitter followers, such as Andrei Karpathy. This is part of a broader trend in society about the skills that are required to capture significant mind share these days.

If that’s not enough, we jump into how machine learning ideally allows businesses to build sustainable and defensible moats, by which we mean the ability to maintain competitive advantages over competitors to retain market share.

In between this interview and its release, PyTorch joined the Linux Foundation, which is something we’ll need to get Mark back to discuss sometime.

Links

• The Myth of Objective Tech Screens
• Machine Learning: The Great Stagnation
• Fear the Boom and Bust: Keynes vs. Hayek - The Original Economics Rap Battle!
• History and the Security of Property by Nick Szabo
• Mark on YouTube
• Mark's Substack
• Mark's Discord