In 2018 France’s President Emmanuel Macron launched what could be considered a kind of digital call to arms: Find European models for development of technologies like AI that encompass the Continent’s values.
“You have a U.S. model, and the U.S. is not regulated,” he said. “It was driven and it is driven by private players…The U.S. model is not sustainable because there is no political accountability.,” said Macron. By the same measure, he warned that China is in a strong position to triumph in critical markets like artificial intelligence thanks to the overt role government takes in setting policy and investment priorities. But the risk if Chinese companies win is that they will impose user rules that create even more surveillance of private data and limits on speech. “How to reconcile a model for the common good: That is the challenge of our generation,” said Macron.
That challenge is more urgent than ever at a moment in time when generative AI large language models (LLMs) are going mainstream while the companies behind them simultaneously downsize their responsible AI teams. This week more than 1800 leaders from the tech industry, including Elon Musk and Apple Co-founder Steve Wozniak, expressed their worry and called for a six month moratorium on the development of the technology due to profound risks to society and humanity. With billions of dollars at stake few believe that call will be heeded.
Since 2017, an estimated 73% of AI foundation models have come from the U.S., where development is mainly driven by large technology companies, and 15% from China. As uptake of these LLM models takes off Europe risks becoming increasingly dependent on foreign AI models, potentially hampering the competitiveness of the entire European economy.
But it is not just economics at stake. Generative AI is expected to have an outsized impact on society. “To me it is very important that Europe has an offer – not just a law – but an offer of technical AI products that embody European values in them,” says Francoise Soulie, a member of the EU’s High Level Expert Group on AI, who has over 40 years of experience working with neural networks, machine learning, social network analysis and Big Data in academia and in industry.
Startups such as France’s LightOn and Germany’s Aleph Alpha are trying to do just that with alternative LLMs that safeguard data. So is Open GPT-X, an initiative by ten German organizations from business, science and media that is developing the European answer to GPT-3, that includes the Center for Information Services and High Performance Computing (ZIH) at TU Dresden. The German Federal Ministry for Economic Affairs and Climate Action is funding the Open GPT-X project within the Gaia-X funding inititative with around €15 million. Under the leadership of the Fraunhofer Institutes for Intelligent Analysis and Information Systems (IAIS) and for Integrated Circuits (IIS), the OpenGPT-X project goal is aiming to develop a large AI language model for Europe that offers data protection as well as European language diversity.
The hope is that not just Europe’s tech industry, but the general population might be inspired by the idea of using AI to protect data privacy and the common good, ensuring Europe’s sovereignty and creating a future that Europeans desire. “AI might serve as the brick that creates a unified market,” says LightOn Founder and CEO Laurent Daudet.
The opportunity is there. But industry observers say it is unclear if in a time of exponential change Europe can create better LLMs with the speed and scale that is necessary.
The European Union is home to some of the world’s best universities and coders and has long emphasized the deep math that underpins AI. But progress is being hampered by emphasis on separate national AI strategies, slow decision-making, too little investment in AI by Europe’s private sector and bureaucratic procedures in the public sector. The 2021 introduction of a coordinated plan on AI by the European Commission has done little to change that, say industry observers. “Governments in Europe are good at making bold speeches, but this only makes things worse because it gives the false impression that we are on the right path and doesn’t create a sense of urgency,” says Andre Loesekrug-Pietri, founder of JEDI, a European foundation formed outside of government that helps fund fundamental research “moonshot” projects.
JEDI, which stands for Joint European Disruptive Initiative, has a methodology similar to the one of The Defense Advanced Research Projects Agency (DARPA), an arm of the U.S. Department of Defense which is credited with helping create a precursor of the Internet known as Arpanet and other innovations.
JEDI’s contends that the EU’s huge research projects, which eat up billions in funding, are launched too slowly, spread in too many directions and are not sufficiently purpose driven. “Money is not the problem,” says Loesekrug-Pietri. “The EU has spent $240 billion on R&D over the past 35 years. Where are the global tech champions? There are none.”
Europe has been obsessed for the last 15 years with disrupting Google and “its attempts with copycat efforts like Quaero have failed miserably,” he says. By investing in generative AI and being the first one out of the gate with large language models Microsoft – which trailed way behind Google in search – suddenly has a shot at taking a big chunk of Google ‘s core business. “This gives Europe hope, he says. “It’s a huge confirmation of why it is necessary to focus on disruptive innovation. If you do something that is different and better, you can reshuffle the cards.”
Europe may have already lost the chance to capture the consumer market with LLMs, says Soulie. But there is an opportunity to offer LLM models that protect data sovereignty to both large corporates and to small and medium-sized companies which represent around 90% of all firms globally, are responsible for roughly 70% of employment and, by some estimates, contribute up to 70% of global GDP, she says.
“For Europe the challenge is simple: how can we obtain a solution we can fine tune without giving away our data? Then we can protect our companies and use that technology to offer services on top of that,” she says.
Today the choice is embracing emerging LLM models in Europe or working with a U.S. based LLM initiative that serves as alternatives to those created by Big Tech companies.
Unlike the LLMs offered by U.S. big tech companies Alelph Alpha and LightOn both have a strong focus on data sovereignty and security. But Aleph Alpha has raised only $31.1 million and LightOn even less while OpenAI, which is now owned by Microsoft, has raised $11 billion. Open GPT-X depends on a limited amount of funding from the German government while The Large European AI Models (LEAM:AI) initiative, launched by the German AI Association, is trying to raise a few hundred million of euros from government and private sources.
Another option is Hugging Face, a startup launched in France by three Frenchmen that moved to the U.S. and has since partnered with Amazon Web Services. Hugging Face has become the central hub for machine learning, with more than 100,000 free and accessible machine learning models downloaded more than one million times daily by researchers, data scientists, and machine learning engineers.With its 176 billion parameters, BLOOM is able to generate text in 46 natural languages and 13 programming languages. For almost all of them, such as Spanish, French and Arabic, BLOOM will be the first language model with over 100 billion parameters created, the culmination of a year of work involving over 1000 researchers from over 70 countries and more than 250 institutions. The BLOOM model was trained on the Jean Zay supercomputer in the south of Paris, France thanks to a grant worth an estimated €3 million from French research agencies CNRS and GENCI.
Researchers can now download, run and study BLOOM to investigate the performance and behavior of recently developed large language models down to their deepest internal operations. More generally, any individual or institution who agrees to the terms of the model’s Responsible AI License can use and build upon the model on a local machine or on a cloud provider since it’s embedded in the Hugging Face ecosystem,
While BLOOM is an alternative to LLM models developed by Microsoft and Google and the French government has invested money into it, it is not European and the data it processes is uploaded into a cloud operated by AWS, an American company.
Europe needs a solution that will scale and “we do not want Chinese or American solutions where the data goes way.,” says Soulie. “The data needs to stay here.”
Loesekrug-Pietri says JEDI can serve as “the OpenAI of disruptive innovation.” Since data security is an issue for many companies JEDI has identified this as a key goal for an upcoming challenge on AI. The objective is to hone European expertise in federated learning, a decentralized approach to training machine learning models that doesn’t require an exchange of data from client devices to global servers. Instead, the raw data on edge devices is used to train the model locally, increasing data.
Beyond data privacy there are other ways that Europe can differentiate itself on LLMs, he says. One of them is in energy usage. Today running large language models consumes huge amounts of energy which is why another upcoming JEDI challenge will focus on attracting the best and the brightest scientists and technologists to work on making LLMs an order of magnitude greener. JEDI is also aiming to launch a third challenge focused on making AI decision-making more transparent.
Loesekrug-Pietri and others also believe that Europe’s language diversity can be turned into a strength.
JEDI’s network of 43 hubs throughout Europe envisions using the ability of LLM models to do automatic translations to create a European agora akin to a central public space in ancient Greek city-states. “LLMs could make the dream of a European common public space come true, bridging today’s differences in style and language. If everybody could sit around the table speaking in their mother tongue and be able to understand each other imagine the creativity,” says Loesekrug-Pietri. “It’s a call for action for foundations and policymakers that want to make history,” he says.
Andreas Liebl, founder of Germany’s appliedAI, Europe’s largest initiative on the application of trusted AI technology, says the advantages would go far beyond politics. “If you could, for example, make a movie in Croatia and it could be instantly streamed and translated into all European languages, it would have a huge impact on the whole cultural area and drastically increase the sense of a European identity,” he says.
Germany’s appliedAI was established in 2017 as a division of UnternehmerTUM Munich and spun out into a joint venture with Innovation Park Artificial Intelligence (IPAI) Heilbronn in 2022. At the Munich and Heilbronn sites, around 80 employees pursue the goal of advancing Europe’s industry to compete in the age of AI.
“I don’t think we will ever get to a single market in Europe for AI, but we could at least have a more unified market,” says Liebl “There is too much national pride, this will never change, but the idea of uniting forces to do something for a better future could work.”
Worries That EU Regulation Will Kill Innovation
Articulating an inspiring vision might help Europe with another issue that is holding it back: loss of talent. American Big Tech companies are hoovering up Europe’s best AI experts, says Soulie. The brain drain could get worse once Europe adopts the EU AI Act.
While few in Europe agree with the U.S.’s laissez-faire approach to regulating AI, there is much apprehension about the application of the proposed new law, which is expected to be adopted shortly. There are no tools or standards in place, meaning that compliance with the EU AI Act will “be simply not doable,” says Liebl. The attitude of the regulators is that once AI is in place three to four years from now the tools will be there “but if we wait until that point in time European companies will lose,” he says. Rather than taking that chance they will simply move to the U.S. where they can develop LLM models without such constraints, he says.
Loesekrug-Pietri agrees that the upcoming EU AI Act has the potential to cripple innovation. “Our ability to shape our own future – or have it shaped by others with different value systems – depends on us and on the wisdom of the policymakers at the EU Parliament,” he says.
“Startups can’t wait two years for a standard to be settled, says Liebl. He is telling EU regulators that they need to build a regulatory accelerator model that will “help companies that want to comply to do so at as little cost as possible and as fast as possible.”
The Need For Speed
It is not just regulations that need to be accelerated. Liebl says he is skeptical that models such as The Large European AI Models (LEAM:AI) initiative can succeed because they are asking for relatively small upfront investment and are focused on one point in time. “It is not about one time, it is about maintaining speed of development and the funding that is necessary longer term,” he says.
European LLMs will need to find a way to keep pace with U.S. tech behemoths that are pouring tens of billions of dollars into LLMs that they update every few weeks and end up giving away for free or at very low cost.
To progress Europe will need to agree on its North Star for AI, says Liebl. “We need a vision with targets that we can work towards,” he says. “If we only talk about things we don’t want we might miss out on the things we could achieve.”
This article is content that would normally only be available to subscribers. Sign up for a four-week free trial to see what you have been missing.