Blog Discover Login
Podcast Insider Logo

Why experts writing AI evals is creating the fastest-growing companies in history | Brendan Foody (CEO of Mercor)

by Lenny Rachitsky

Lenny's Podcast: Product | Career | Growth

Share: Copied!

Notable Quotes

"If the model is the product, then the eval is the product requirement document."
"Models are only as good as their evals."
"You can just do things; take the initiative to build."
Podcast Insider Logo

Get episode summaries just like this for all your favourite podcasts in your inbox every day!

Get More Insights

Episode Summary

In this episode, Brendan Foody, the CEO and co-founder of Mercore, shares insights into his company's extraordinary growth and the future of AI labor. Mercore, helping AI labs hire experts to improve model capabilities, achieved a revenue run rate of $400 million in just 16 months, making it the fastest growing company in history. Brendan discusses the increasing significance of evals (evaluations) in defining model capabilities, noting that they serve as a product requirement document for AI models, enabling labs to measure their success efficiently.

He explains that while many focus on AI's potential for job displacement, it’s important to highlight the new job categories being created in the AI sector. The discussion includes how companies can leverage technology to enhance productivity and encourages young professionals to adapt by honing skills that will be relevant in the AI-era. Throughout the conversation, Brendan emphasizes the importance of customer obsession and assembling a high-caliber team to drive success.

Drawing from his experience, Brendan shares the necessity of finding extraordinary professionals who can contribute significantly to model training and evaluation. He highlights that the future of work will be closely tied to how we measure AI success and that everyone should be pushing for innovation in this space. He brings a refreshing perspective on the future of AI, downplaying fears of imminent superintelligence while remaining optimistic about the transformative power of AI in improving various sectors, including education and healthcare.

Unlock the full summary

Enter your email to read the complete summary, key takeaways and more.

Email

Episode Summary

In this episode, Brendan Foody, the CEO and co-founder of Mercore, shares insights into his company's extraordinary growth and the future of AI labor. Mercore, helping AI labs hire experts to improve model capabilities, achieved a revenue run rate of $400 million in just 16 months, making it the fastest growing company in history. Brendan discusses the increasing significance of evals (evaluations) in defining model capabilities, noting that they serve as a product requirement document for AI models, enabling labs to measure their success efficiently.

He explains that while many focus on AI's potential for job displacement, it’s important to highlight the new job categories being created in the AI sector. The discussion includes how companies can leverage technology to enhance productivity and encourages young professionals to adapt by honing skills that will be relevant in the AI-era. Throughout the conversation, Brendan emphasizes the importance of customer obsession and assembling a high-caliber team to drive success.

Drawing from his experience, Brendan shares the necessity of finding extraordinary professionals who can contribute significantly to model training and evaluation. He highlights that the future of work will be closely tied to how we measure AI success and that everyone should be pushing for innovation in this space. He brings a refreshing perspective on the future of AI, downplaying fears of imminent superintelligence while remaining optimistic about the transformative power of AI in improving various sectors, including education and healthcare.

Key Takeaways

  • The swift rise of Mercore reflects a broader trend in AI where better evaluation processes (evals) are essential for improving model capabilities.
  • AI is creating new job categories, and professionals should adapt by learning to leverage AI technologies effectively.
  • Customer obsession and a commitment to high standards have been crucial for Mercore's rapid success.

Found an issue with this summary?

Log in to Report Issue

Built for solopreneurs, makers, and business owners who don't have time to waste.