Presenters
Source
Token Maximizing: The New (and Bizarre) Metric in Big Tech 🤯
Ever feel like you’re drowning in metrics? Well, buckle up, because the tech world has a new, and frankly, wild one: token maximizing. Gergely Orosz, the brilliant mind behind The Pragmatic Engineer, dropped some serious insights on this phenomenon, and it’s a story that’s both hilarious and a little bit terrifying.
What Exactly is Token Maximizing? 🤔
Imagine a world where your job performance isn’t just about shipping code or solving problems, but about how many AI-generated “tokens” you produce. That’s essentially token maximizing. It’s a trend that’s been bubbling up in big tech companies, and Gergely’s DMs have been blowing up with stories from engineers at places like Meta and Microsoft.
The core idea? AI tools are churning out text, code, and summaries, and these companies are trying to measure engineer productivity through the output of these AI tools.
The “Why” Behind the Weirdness 🤷♀️
So, why are companies even looking at this?
- Measuring AI Usage: Leadership wants to see engineers using AI tools. This stems from a fear of falling behind, especially after seeing companies like Anthropic report significant code generation with AI.
- Performance Evaluation (Kind Of): At companies like Meta, token output is one data point among many in performance reviews. While not the sole factor, it can be “weaponized.” A low token count might signal a lack of effort, while a high one could be spun as innovation.
- Leaderboards and Peer Pressure: Some companies, like Meta (before they scrapped it after an article surfaced) and Microsoft, have had leaderboards. This creates intense pressure to appear productive, even if it means generating “junk” for the sake of the number.
- Minimum Spend Targets: At Salesforce, there’s a minimum AI token spend target per month ($175!), leading engineers to “token max” early in the month just to hit the quota.
The “How” of Token Maximizing 🤖
Engineers, being the ingenious problem-solvers they are, have found some… creative ways to boost their token counts:
- Summarizing Documentation: Instead of reading it, ask the AI to summarize, even if the answers aren’t great.
- Running Autonomous Agents: Some engineers are reportedly running autonomous agents to generate code or text, purely to inflate their metrics.
- Asking Redundant Questions: Posing questions to AI that you already know the answer to, just to get more output.
Gergely likens this to the old days of measuring productivity by lines of code – a metric we all know was flawed and easily gamed.
The Dark Side: Challenges and Tradeoffs 📉
This whole token maximizing trend isn’t just a quirky anecdote; it comes with significant downsides:
- Goodhart’s Law in Action: As Gergely points out, “whatever gets measured gets gamed.” The pursuit of token counts directly leads to the “abuse” of the system.
- Wasted Resources: Running autonomous agents to create junk code is a clear waste of computing resources and money.
- Erosion of Genuine Productivity: Instead of focusing on real impact, engineers are incentivized to chase a vanity metric.
- Weaponization of Data: Performance metrics, even AI-related ones, can be used against individuals.
- Cultural Weirdness: What started as a fun exploration of AI has devolved into a “culturally weird thing” in many companies.
The Cost of Not Adopting AI? 💸
On the flip side, there’s a genuine fear of being left behind. Gergely recalls a CTO dinner where engineers were skeptical of AI, while others, like the Dutch National Bank, were actively using it to understand and regulate the technology. The pressure to adopt AI, even if imperfectly, is immense.
Brian Armstrong, CEO of Coinbase, famously sent an email demanding AI tool adoption, and later fired an engineer who didn’t comply. This highlights the extreme measures some companies are taking to push AI adoption.
Is Token Maximizing Still “Worth It”? ⚖️
This is the million-dollar question. While the individual engineer might see some productivity gains with AI tools, the impact on teams and overall organizational velocity is still a big question mark.
- The Meter Study: A study showed participants felt 20% more productive with AI, but their actual demonstrated results showed a 20% decrease in productivity on average. (Though there was one outlier who was significantly more productive).
- Empowering Non-Coders: Gergely’s theory is that the real win might be in enabling non-technical collaborators to use AI for coding tasks. This unlocks a new level of “serverless developers” and shifts the focus from individual developer pull request productivity to broader organizational efficiency.
- The Learning Curve: Gergely emphasizes that getting good at AI takes time. There’s no manual, and understanding the theory doesn’t directly translate to better tool usage. It requires continuous learning and adaptation.
- Leaving Priors Behind: The key is to have an open mind, leave old assumptions behind, and embrace new workflows.
The Evolving Role of the Software Engineer 👨💻➡️🤖
AI isn’t just a new tool; it’s fundamentally reshaping the software engineer’s role.
- Broader Skillsets: Startups have long demanded engineers wear multiple hats (DevOps, testing, even product). AI is accelerating this trend across the industry.
- Smaller, More Capable Teams: Companies are seeing “two-pizza teams” become “one-pizza teams,” thanks to the increased efficiency AI tools can provide.
- From Coder to Orchestrator: The idea that engineers are now just “managing engineering agents” is a simplification. While orchestration is key, it’s not the same as traditional people management. It’s more akin to a tech lead role, amplified by AI. The feedback loops are faster, and the ability to control and amplify one’s capabilities is immense.
The Rise of Internal AI Infrastructure 🏗️
Beyond individual tools, big tech is going all-in on building their own AI infrastructure.
- Custom Solutions: Companies like Uber are rebuilding their entire AI infrastructure, integrating custom coding agents, MCP gateways, and risk-based code review systems.
- Why the Investment?
- Low-Risk AI Mastery: It’s a safe way to get hands-on with AI without immediately shipping unproven features.
- Handling Large Codebases: Custom solutions can better handle code that exceeds current context window limitations.
- Funding Magnet: Projects with “AI” in their name are easier to get funded.
- The “MCP Gateway” Trend: Gergely jokingly asks, “If you’re at a large company and you’re not already building an MCP gateway, what are you even doing?” This highlights the shared focus on building foundational AI capabilities.
Shopify’s Strategic Bet 🚀
Shopify’s early adoption of GitHub Copilot, even with its initial churn and expense, is a prime example of a strategic trade-off. They invested heavily to gain a competitive edge, understanding that being months ahead in AI capabilities is worth the cost for their business. This is about innovation recruitment – attracting top talent by offering cutting-edge tools.
The Pragmatic Engineer Story: From Layoffs to Leader 📈
Gergely’s own journey with The Pragmatic Engineer is a testament to finding product-market fit. Born out of the COVID-era layoffs at Uber, what started as an experiment in in-depth software engineering content quickly gained traction.
- Product-Market Fit: A confident Twitter post before publishing anything led to 100 pre-paid subscriptions.
- Focus on Quality: For two years, Gergely focused on delivering just two high-quality articles per week, a strategy that proved incredibly successful.
- Scaling the Business: Recognizing the need for sustainability, he grew the team and launched a podcast, turning The Pragmatic Engineer into a leading paid technology newsletter.
The success of The Pragmatic Engineer, coupled with the ongoing evolution of AI in the workplace, paints a picture of a rapidly changing tech landscape. While the “token maximizing” phenomenon might seem absurd, it’s a symptom of larger forces at play: the drive for AI adoption, the pressure to innovate, and the constant redefinition of what it means to be a software engineer. It’s a wild time to be in tech, and Gergely Orosz is giving us the inside scoop! ✨