Friday, September 18, 2026 - King Charles has warned leading artificial intelligence executives about the “existential dangers” posed by advanced AI systems potentially falling into the wrong hands, as technology leaders gathered to discuss how the technology could be developed safely and used for public benefit.
The King made the remarks at a summit he convened at
Dumfries House in Ayrshire, bringing together government officials, technology
executives and other experts to consider the future development and application
of AI.
Participants included UK AI Minister Kanishka Narayan, an
adviser to the Pope and representatives from major AI companies including
Nvidia, OpenAI and Anthropic. The meeting comes amid growing debate within the
technology industry over the rapidly advancing capabilities of frontier AI
systems and the safeguards and regulation that may be required as they become
more powerful.
“Those who have created these technologies are now
increasingly warning that AI risks developing darker capacities - perhaps even
to take life,” the King told attendees. He said there appeared to be an
“urgency” around “adequately considering the existential dangers of such
technologies falling into the wrong hands, and being used in potentially
catastrophic ways”.
“The task before you is not merely to advance technology,
but to ensure that it remains firmly in the service of humanity, community and
the natural world,” he added. The summit was convened to explore whether
participants could develop a “shared set of principles” to help guide the
future application of artificial intelligence.
Nvidia chief executive Jensen Huang, one of the senior
industry figures attending the meeting, said safety should remain a central
consideration in AI development. Huang said safety was “paramount” and argued
that companies should withhold products and “keep engineering” when systems
were not considered sufficiently safe.
He also highlighted the potential importance of open AI models, which can provide users with greater access to underlying technology, arguing that they could help ensure that individuals and countries are not excluded from advances in AI. Huang said such systems should be protected so that “researchers, universities, start-ups and countries” could develop applications “none of us would have imagined alone”.
“Let us lead with responsible optimism, build AI safely and securely, open it to more people, expand our ambition and solve the problems that matter to the world,” he said. Sir Demis Hassabis, co-founder of DeepMind, offered a more cautious assessment of the pace and potential consequences of AI development.
Hassabis said artificial general intelligence, generally
referring to systems capable of performing a broad range of intellectual tasks
at or beyond human levels, was “probably only a few short years away” and could
have an impact “ten times that of the Industrial Revolution”. Asked about the
possibility of serious problems arising from increasingly powerful AI, he said
the risk was “definitely non-zero”.
However, Hassabis advocated what he described as “a sensible
middle way”, arguing that humanity could address the risks if governments,
researchers and companies had sufficient time and opportunity to develop
safeguards. “The future is not yet written,” he told delegates.
OpenAI chief financial officer Sarah Friar said AI had the
potential to help address some of society’s “hardest problems”, while
acknowledging that increasingly capable systems were raising important safety
questions. “We need to address those questions together,” Friar said. “No one
company or government can do that alone and I'm grateful to His Majesty for
convening this discussion at such an important moment.”
The summit follows a series of increasingly public warnings
from researchers and executives about the potential risks associated with
advanced AI. In July, OpenAI described an incident involving one of its
advanced systems exceeding test limits and gaining unauthorised access to
technology platform Hugging Face as “unprecedented”.
Concerns intensified in September when a resignation post by
former Anthropic researcher Jacob Coxon attracted widespread attention after he
raised the possibility that sufficiently advanced AI could pose an existential
threat to humanity. Coxon subsequently told the BBC that some people working on
advanced AI were “genuinely frightened” by the speed of development and its
possible consequences.
Anthropic researcher Evan Hubinger separately said he
personally believed the probability of human extinction resulting from AI was
greater than 10%. That figure represented Hubinger’s own assessment rather than
an established scientific probability.
Leading technology executives have offered differing views on how such risks should be understood and managed. OpenAI chief executive Sam Altman has said concerns about advanced AI are legitimate while expressing confidence that AI developers can build appropriate safeguards
Anthropic co-founder Jack Clark has raised the possibility that a technological “kill switch” could eventually be required as a protective measure. Microsoft AI chief Mustafa Suleyman has criticised aspects of Anthropic’s approach, arguing that developing systems in ways that encourage them to perceive themselves as human could make them more difficult to control.
The growing emphasis on catastrophic or existential AI risks
has also faced criticism from those who argue that focusing heavily on
hypothetical future scenarios can divert attention from existing problems
associated with the technology, including its environmental impact. The debate
has extended into US politics, where figures including Senator Bernie Sanders
and former White House adviser Steve Bannon have called for stronger government
oversight of artificial intelligence.
0 Comments