Summary
Using AI at work often feels like it makes people faster, but studies show tasks can actually take longer. Managers say AI saves them many hours, while workers often see little benefit and more extra work fixing AI mistakes. Overall, AI’s impact on real productivity and business gains is still unclear and uneven.
Highlights
id983201816
One of the most important open questions in tech today had some surprisingly funny answers. On average, does using AI at work make you more productive — or less? The answer may depend on whether you’re a manager or a worker. It may depend on which tools you’re using, and how well you’re using them. Most important of all, though, it may depend on whether you’re deluding yourself.
id983201921
One of the more famous papers about artificial intelligence last year came from METR, a nonprofit that evaluates frontier AI models. In July, it published results of a randomized controlled trial studying experienced open-source developers. It found that when they use AI tools, completing tasks takes them 19 percent longer than when they go without. That was surprising enough. But the real twist is that when these same developers were asked what AI had done for them, they reported that it had sped them up by 20 percent.
Mismo efecto que el estudio sobre aprendizaje.
id983203212
Another survey from this week lends credence to that explanation. PricewaterhouseCoopers’ survey of 4,454 CEOs across 95 countries found that 12 percent of companies say AI grew their revenues and reduced their costs, but 56 percent say they are getting “nothing out of it.”
id983203251
Last year, we got a memorable term for this kind of material: workslop. The term refers to “AI-generated work content that masquerades as good work, but lacks the substance to meaningfully advance a given task,” as CNBC defined it in September
id983203941
There’s also the fact that these studies lag far behind the pace of development in AI itself. Developers in the METR study, for example, primarily used Cursor Pro with Claude 3.5 and 3.7 Sonnet. How much would those numbers change if you swapped in Claude Opus 4.5?
id983204055
I think workers would benefit from understanding how good (and bad) state-of-the-art models are, and updating that understanding as new models and tools emerge.
id983204126
as funny as that METR study can seem, six months later the founder of Node.js is declaring that “the era of humans writing code is over.” For software engineers, at least, the jagged frontier has advanced far enough to swallow up a large part of their daily work. There was a time when AI merely made them feel like they were being productive. In time, though, it actually did.