When the state no longer knows: how should Europe respond to the power of AI companies?

Artificial intelligence is transforming one of the most sensitive functions of any state: ensuring security. Today, the state's ability to produce knowledge about threats, understand the intentions of its adversaries and anticipate events is being radically transformed. OpenAI, Anthropic, Google, Microsoft, Amazon and Palantir sell tools to governments, armed forces and intelligence services. While this outsourcing is not new, what has changed is that these companies are beginning to take on parts of the intelligence cycle itself. Their tools can now be used to collect indicators, process vast amounts of information, identify suspicious behaviour, establish links between people and organisations, attribute operations and even implement countermeasures.

"OpenAI, Anthropic, Google, Microsoft, Amazon and Palantir are beginning to take on parts of the intelligence cycle itself"

AI companies are not going to formally replace intelligence services. The main reason is simply that they do not have the same legal powers. Nor do they possess their networks of human sources or their institutional legitimacy. What OpenAI and the other companies can do, however, is increasingly concentrate in private hands an ever more important capability: turning vast amounts of information into useful knowledge —that is, processing and interpreting it in order to produce answers and inform decisions—. Europe must prevent dependence on these technologies from ultimately limiting its political autonomy as well.

Claude as an operational environment

According to a recent Anthropic threat intelligence report, actors linked to governments and organisations in China, Russia, Iran, Mali and other countries have used Claude for surveillance, espionage, propaganda, military software development and dual-use biological research. In some cases, AI is reported to have played a role in carrying out and coordinating substantial parts of cyber operations.

Anthropic says it detected operations linked to China and Iran that used Claude to monitor or profile dissidents, journalists, activists and other public figures. In Mali, it documents the case of a consultant who used Claude to develop a national surveillance platform for the state intelligence service. AI did not determine these governments' objectives, but it did greatly reduce the technical expertise, personnel, time and resources needed to put their plans into practice. As a result, surveillance capabilities once reserved for relatively sophisticated intelligence services may now become available to far more modest state apparatuses.

"Surveillance capabilities once reserved for relatively sophisticated intelligence services may now become available to far more modest state apparatuses"

Anthropic also attributes to a group using methods consistent with those of Midnight Blizzard, which is linked to Russia, a campaign targeting Ukrainian government, military and diplomatic entities. Claude was reportedly used for tasks such as preparing phishing attacks, compromising hotel Wi-Fi connections, attempting to take over WhatsApp accounts and automatically modifying malicious code when it was detected by security systems.

There are also cases involving propaganda. For example, Russian state media are reported to have used the model to produce content presented as independent journalism, including fabricated claims about electoral processes. Queries relating to firearms, missiles, armed drones and explosives were also detected from China, Russia and Yemen.

Of all the cases published, the most serious concerns biological weapons. Anthropic says it blocked accounts associated with researchers attempting to use Claude for work that could facilitate the development of this type of weapon. One operation allegedly sought assistance in drafting a research proposal on mutations of the chikungunya virus that could increase its harmfulness. The project's alleged connection to a military institution heightened the company's concern, although Anthropic acknowledges that it could not determine whether the ultimate purpose was to develop a weapon.

Although these are serious matters, some caution is required. These are assessments made by Anthropic on the basis of activity observed on its platform and do not in themselves amount to judicial findings or official attributions. Even with that caveat, one fact should not be overlooked: a private company has been able to observe state operations, reconstruct how they worked, attribute them geographically, identify possible perpetrators, remove them from its platform and, above all, decide what information to share with governments. Anthropic has protected Claude, its product, but in doing so it has also acted as a private counter-intelligence service.

The privatisation of the intelligence cycle

What do intelligence services actually do today? From MI6 to Spain's CNI or the CIA, they carry out a range of fundamental tasks at the request of governments. They are responsible for setting priorities, gathering information, processing data, developing hypotheses, assessing threats —where they exist— and ultimately delivering their conclusions to the relevant decision-makers.

A significant part of this chain can already be placed in the hands of technology companies. Amazon, Microsoft and Google provide the cloud infrastructure on which vast amounts of information are stored and processed, while Nvidia supplies much of the required computing capacity. From there, companies such as Palantir can integrate and cross-reference different databases, while models developed by OpenAI, Anthropic or Google DeepMind can translate, summarise, connect and analyse all that information.

"The state may continue to make the decisions while becoming increasingly dependent on tools it does not control to obtain the information on which those decisions are based"

The state may continue to make the decisions while becoming increasingly dependent on tools it does not control to obtain the information on which those decisions are based. A report may bear the signature of a civil servant, for example, even though some of the sources used, the relationships identified between them or the hypotheses put forward may come from systems whose operation cannot be fully examined.

The cases described by Anthropic raise another problem. When an operation takes place within a private platform, the company may hold information about how it unfolded that the state itself may later need. Governments and intelligence services then become dependent on the provider to access those records. In addition, the company itself may decide whether access to the tool is maintained or suspended.

Europe does not need autarky, but its own capabilities

Europe's response should not be to dispense with US providers. That would be unrealistic and probably counterproductive. The United States retains a considerable advantage in foundation models, advanced computing and cloud services, and Europe will continue to need to work with its companies, including in security-related areas.

The problem arises when that cooperation creates a dependency that becomes difficult to escape. Europe should be able to use the best technology available without becoming tied to a single provider across every stage, from data collection to political decision-making.

"Europe should be able to use the best technology available without becoming tied to a single provider across every stage, from data collection to political decision-making"

The first step would be to determine which parts of that process should be considered critical infrastructure. An assistant that summarises administrative documents does not pose the same problem as a model used to analyse intercepted communications, identify people who may be placed under surveillance or locate potential military targets.

Member States could begin with a common classification of systems used in security, defence and intelligence. The more directly a tool is involved in identifying individuals, developing strategic hypotheses or informing decisions concerning the use of force, the greater the degree of public oversight over its operation should be. There should also be stronger requirements regarding the traceability of its outputs and the authorities' ability to continue operating without relying exclusively on the provider.

A European procurement doctrine

Much of this debate will ultimately be settled through public procurement. Technological dependence does not always become apparent when a tool is purchased, but after years of using it. Once an organisation stores its data, adapts its procedures and trains its staff around a single platform, switching to another one can become prohibitively costly. At that point, the very dependency created over the course of the contract makes renewal more likely.

"Technological dependence does not always become apparent when a tool is purchased, but after years of using it"

Europe therefore needs specific criteria for procuring artificial intelligence systems intended for sensitive uses. Agreements should make it possible to transfer data to another platform, connect the system to technologies supplied by other providers and retain a record of automated decisions. They should also specify from the outset how the service can be discontinued. A contractual right to change provider is of little value if doing so later requires rebuilding the organisation's entire technological architecture.

There is another particularly important question for intelligence services: what happens to the information generated through the use of the platform itself. Contracts should specify where data are processed, who can access them, which logs the company retains and under which jurisdiction they are stored. The sensitive information does not lie solely in the documents uploaded to the system. It may also lie in the questions analysts ask. Knowing which countries, individuals, organisations or threats account for most of those queries can reveal a government's priorities.

For the most sensitive uses, it may be necessary to go further and require certain models to operate on infrastructure controlled by the authorities themselves, in isolated environments and with mechanisms that allow the service to continue if the relationship with the company breaks down. Updates should not depend entirely on teams based outside Europe either. State ownership of the data does not solve the problem if the state has lost control of the tools required to interpret them.

A shared European capability

It is difficult for a single European state to develop and maintain all this infrastructure on its own. Unity is necessary, and part of the response will have to be organised at EU level, although that does not mean immediately creating a European intelligence service. A first step could be to share technical capabilities among national agencies.

The Union could create a centre dedicated to evaluating models used in areas related to national security and connect it with the European Union Agency for Cybersecurity (ENISA), the EU Satellite Centre and the EU Intelligence and Situation Centre (EU INTCEN). Before a model is introduced into a sensitive environment, this body could examine its vulnerabilities and test how it behaves. It could also review updates and pool warnings identified by different Member States.

"Europe needs the means to verify those attributions independently rather than relying solely on the company's interpretation"

If a US company detects an operation taking place within its platform, it may acquire relevant information about what happened before European authorities do. Europe needs the means to verify those attributions independently rather than relying solely on the company's interpretation. To do so, it should be able to combine indicators provided by the supplier with public and classified information and reach its own assessment as to whether an operation is linked to a state.

The same scrutiny should also apply to European products. Strategic autonomy loses its meaning if a system is assumed to be secure simply because it was developed within the Union. Companies such as Mistral, Aleph Alpha or Helsing may benefit from European investment and contracts, but their tools should be subject to controls comparable to those imposed on their US competitors.

Preserving human judgement

One of the advantages of artificial intelligence is its ability to process volumes of information that no human team could ever review in full. That same scale, however, can make it more difficult to verify the results. The formal presence of a human being at the end of the process does not mean that person can exercise meaningful control over it. Likewise, if a system generates thousands of alerts or proposes hundreds of targets, the person responsible may end up validating outputs without enough time or information to reconstruct how they were produced.

"The formal presence of a human being at the end of the process does not mean that person can exercise meaningful control over it"

An assessment produced with the assistance of AI should therefore make it possible to distinguish between data derived from verified sources and conclusions generated by the model. It should also indicate the degree of uncertainty attached to those inferences. In particularly serious decisions, such as designating a person as a threat, authorising an operation or recommending the use of force, a model's output should never be sufficient on its own.

It is also important to retain teams capable of working without these tools. Dependence does not arise only when a particular technology is unavailable. It also emerges when an organisation no longer knows how to do its job without it. An intelligence service that loses its capacity for independent analysis will become more vulnerable both to errors in the system and to manipulation or disruption of access.

Sovereignty over the capacity to know

The secret nature of these activities should not preclude all forms of democratic oversight. Parliaments do not need access to the operational details of every mission, but they should know which companies are involved in producing intelligence, what tasks they perform and what controls apply to them.

A company may detect an operation taking place on its platform, close an account or establish which uses of its technology it will permit. Those decisions are part of managing its service. It is quite another matter to determine what constitutes a threat or where the national interest lies, decisions whose legitimacy belongs to public institutions. Likewise, state secrecy should not become a pretext for introducing opaque tools without some form of independent assessment.

Europe can use US models while also developing its own alternatives. The two are compatible provided that it retains sufficient control over the data, over how the systems operate and, above all, over the decisions made on the basis of their outputs. The room to do so, however, will shrink as the technologies being adopted today become increasingly difficult to replace.

"States may procure technology, but they should not lose the ability to understand the information they use, form their own judgement and take responsibility for the decisions they make"

States have always relied on tools developed outside government, but never before has so much work been delegated as it is today. They may procure technology, but they should not lose the ability to understand the information they use, form their own judgement and take responsibility for the decisions they make. If a state no longer understands how the intelligence on which it bases its decisions is produced, it may formally retain its authority while, in practice, having lost an important part of it.