Carl Benedikt Frey, the Dieter Schwarz Associate Professor of AI and Work at the Oxford Internet Institute and Oxford Martin Citi Fellow at the Oxford Martin School, both at the University of Oxford, is the author of a new book, How Progress Ends: Technology, Innovation and The Fate Of Nations, which has been shortlisted for the Financial Times and Schroders Business Book of the Year Award. His previous book, The Technology Trap, was selected a Financial Times Best Books of the Year in 2019.
After studying economics, history and management at Lund University, Frey completed his PhD at the Max Planck Institute for Innovation and Competition in 2011. He subsequently joined the Oxford Martin School where he founded the program on the Future of Work with support from Citigroup. In 2019, Frey joined the World Economic Forum’s Global Future Council on the New Economic Agenda, as well as the Bretton Woods Committee and in 2020, he became a member of the Global Partnership on Artificial Intelligence (GPAI) – a multi-stakeholder initiative to guide the responsible development and use of AI, hosted by the OECD. Frey has served as an advisor and consultant to international organizations, think tanks, government and business, including the G20, the OECD, the European Commission, the United Nations, and several Fortune 500 companies.
He recently spoke to The Innovator about his new book and why some societies flourish and others fail in the wake of rapid technological change.
Q: What is the focus of The End of Progress?
CBF: The basic idea of the book is that there is no end of history, no recipe that allows you to be done institutionally. You always need to adjust as technology changes. Technological growth not only depends on inventions, but also on the political and institutional ecosystems that allow them to flourish. History shows that progress stalls when institutions built for one stage of development persist into another. Static factors—geography, culture, or even the broad character of institutions—can’t explain why periods of rapid growth are so often followed by stagnation or collapse. Consider the Soviet Union. In the 1950s, its economy expanded quickly. Yet its geography at the time of collapse was unchanged. Its culture didn’t shift dramatically either—if anything, it became marginally more open as radio and television pierced the Iron Curtain and exposed people to Western ideas. Nor did the USSR ever enjoy secure private property rights, whether during the boom or the bust, despite Gorbachev’s late-stage reforms. What does change is how well institutions fit the tasks they are meant to perform. Institutions that are effective for mobilizing resources and catching up can become liabilities once the growth model must pivot toward innovation, flexibility, and decentralization. In this sense, the Soviet growth spurt and subsequent collapse are best understood through the lens that different institutions are good for different things—and that failing to adapt them across stages of development eventually brings progress to an end. To catch up technologically the Soviet Union could take advantage of Ford Motor Co.’s open door policy and send delegations to Detroit to learn about mass production and transplant this approach. The Soviet Union was able to mobilize resources and scale production of all sorts of vehicles. And the central authorities could then benchmark performance across different factories to make sure managers and workers didn’t slack. Then in the 1970s mass production hit a wall, accelerated by the oil price shocks which significantly increased the cost of energy. Something new was needed for growth and the new thing was the computer. This, however, made monitoring production harder and the Soviet Union didn’t make any strides because it didn’t have pockets that permitted exploration. If you were an engineer, you could go to the Red Army and ask for funding. If they declined, you had two or three other options but if they also declined your technology or idea would die with you. Things were quite different in the US. In 1999, Bessemer Venture declined to invest in Google. This may now seem like a bad decision but Google was not a safe bet at the time. AltaVista and Yahoo dominated the market for search. However. Google didn’t die because others stepped in and invested. That is the virtue of decentralization. It allows for many kinds of experiments, and more technological trajectories to be explored. The lesson is sustaining innovation requires openness, decentralization, and competition, even at the cost of short-term inefficiency.
Q: How are today’s industrial policies around AI and quantum computing helping/hurting the U.S., China and Europe?
CBF: As a general rule industrial policy can work when an objective is clear. The Manhattan Project, for example. succeeded in developing a nuclear bomb. However, when technology is not mature and there is a lot of tech turbulence it is much harder. Europe did well in catching up to the U.S. with the formation of Airbus as a competitor to Boeing. It mobilized resources in concerted way almost unheard of since then. Some people argue we need an Airbus for AI but when technology or a domain is very fast moving it is harder to pick winners and difficult to have a clear objective when the target is constantly moving. You could, for example, end up throwing a lot of resources into large language models [LLMs] only to find that the future is small language models. Nvidia is pivoting towards that. And maybe the future of AI isn’t LLMs. I don’t know. But AI can’t be solved just by throwing more data and compute at the problem. We scaled LLMs 10,000X without any significant improvement on key reasoning tests, like ARC-AGI. We then got some improvements through innovations like chain-of-thought-reasoning, but there is still some way to go and need new approaches to get better performance. Most people know about how AlphaGo, [a computer program that plays the board game Go developed by the London-based DeepMind Technologies, an acquired subsidiary of Google] first beat a human professional player in 2016. It was a huge milestone. What most people don’t know is that in 2023 human amateurs with laptops best AlphGo by exposing it to new concepts and positions. So even when AI reaches superhuman capabilities, there is still an open question how AI will perform when circumstances change. No firm could be run by AI at the moment. And because the technology is still in flux, it is difficult for governments to mobilize resources around an industrial policy to meet an objective that is changing by the month.
Q: How well is Europe positioned in the race for supremacy in these new technologies?
CBF: I think Europe has a more fundamental question to answer. Why is it that we don’t have any of these large technology companies to begin with? Europe was very successful in catching up in manufacturing goods. But we haven’t managed to do the same in digital. Why is that? A key reason is that the single market for services is much less harmonized. In fact, if you take all the regulatory barriers to service trade in Europe they amount to a 110% tariff. Such fragmentation means that the returns to scaling are much lower in Europe than in the U.S. or China which have large harmonized domestic markets. And in addition there are much higher compliance costs. If you look at well-meaning regulations like GDPR and the EU AI Act, they create barriers to entry for startups whereas large companies have managed to offset compliance costs by capturing a larger share of the market.
Europe also has a dynamism problem. In Britain, Germany, and France, the five largest companies are on average 116, 120, and 152 years old, respectively. They sit at the center of dense networks of banks and industrial conglomerates. Entry barriers, complex regulations, and rigid labor laws make it hard for new firms to form, pivot, and scale. More flexibility is needed if Europe wants more high-growth public companies.
Another issue is that Europe spends heavily on public R&D but gets less in return. Large, multi-party consortia, encouraged by programs like Horizon 2020, often diffuse accountability and pull in different directions; smaller teams are likelier to deliver breakthroughs. European universities, meanwhile, frequently lack the autonomy to compete globally for talent, and much of the best ends up in the United States. That could shift if the U.S. continues to self-sabotage—cutting research budgets and weakening its own institutions—but Europe shouldn’t count on it.
Q: What about China?
CBF:The U.S. export controls on semiconductors have forced China to become more dynamic and embrace opensource models. This shows how China can be dynamic when needed. DeepSeek was allowed to fly under the radar when it met national goals. At the same time there has been a clear drift from a laser focus on growth to other priorities around security and national strength and common purpose. Private and foreign invested companies are most innovative and productive in China too, but they have less interest in pursuing goals other than market share and profit, so China is again becoming more reliant on state-owned enterprises which have been a drag on growth. Indeed, business dynamism and productivity are in decline in China too since the 2000s.
Q: The U.S. is in a prime position in AI, dominating the chip market and LLMs. Do you think it will maintain its lead?
CBF: The U.S. learned the wrong lessons from its history. In the 1980s, there were concerns then in the U.S. with regards to Japan, and now today with China. Toyota’s operational prowess did help dethrone GM and Ford and hasten Detroit’s decline. But Japan’s model was built for diffusion and process perfection, not frontier, paradigm-shifting innovation. The barriers to market entry in Japan were very high whereas the US was more dynamic. IBM was forced to unbundle hardware and software and AT&T was broken up through antitrust action. Consequently, when Arpanet [The Advanced Research Projects Agency Network, which developed the technologies that became the technical foundation of the Internet] was released to the world, the boardroom of AT&T could not bottleneck it if saw no immediate commercial benefits. As a result, the Internet could develop organizationally through its users. That’s the key reason the computer revolution occurred in the U.S.
Right now, the U.S. is embracing protectionism and even worse, crony capitalism. It has not just added tariff barriers it has also created a complex system of exemptions which means larger politically connected firms are benefiting from that. It also means that the managers and CEOs that tech firms are likely to hire will be experienced in dealing with Washington D.C. rather than engineering and innovation. Unfortunately, rather than doubling down on competition and trying to make it easier to enter the market, we are seeing consolidation with large firms buying up startups, sometimes just to shut them down. It is important to point out that every firm from Apple and Microsoft to Google and Amazon had IPOs in the early days of the computing and Internet revolution but since the 2000s WhatsApp, Instagram and YouTube have been acquired rather than become companies in their own right.
Moreover, the U.S.’s integration with China was a response to being out competed in manufacturing by the Japanese. It outsourced production to China to undercut the Japanese cost advantage but now the U.S. is too dependent on China. It needs to continue to diversify and embrace global competition, not bring production home.
America’s greatest economic strength has always been its capacity for industrial renewal, enabling new companies to emerge, innovate and grow but such dynamism is not guaranteed. History shows that when competition yields to cronyism, technological leadership slips.
Q: What can large corporates learn from your book?
CBF: Like governments, firms constantly struggle to balance centralization and decentralization. The computer and Internet revolutions laid this bare. U.S. and Scandinavian companies tended to outperform because they pushed decision-making lower in the hierarchy, letting small teams move fast. It’s also rational that large incumbents protect cash cows. Google helped invent Transformer architectures, yet it wasn’t the first mover in generative AI—doing so risked cannibalizing its search and ads core. The broader problem is classic: it’s hard for established firms to exploit existing models while exploring new ones at the same time. One practical workaround is externalization of exploration: take strategic stakes in promising startups, partner, or spin out ventures. That way incumbents can share in upside from frontier experimentation without dismantling the structures that sustain their current businesses.
Q: What would you like readers to take away from your book?
The key message is that progress cannot be taken for granted. If progress was inevitable, it would not have taken humanity 200,000 years to have an industrial revolution. If it was inevitable, most places around the world would be rich and prosperous today. Progress is constant work in progress.
This article is content that would normally only be available to subscribers. Become a subscriber to see what you have been missing.
