AI out of control: what the ‘real threats’ actually mean for your SME
A three-and-a-half-hour programme on the real threats of AI
On 13 October 2026, the science communication channel La Tronche en Biais broadcast live “Intelligence artificielle — les vraies menaces” (Artificial intelligence: the real threats), with two guests:
- Shaïman Thürler, creator of the podcast and channel Le Futurologue, dedicated to the challenges of AI;
- Tom David, co-founder and director of the GPAI Policy Lab (international cooperation on the safety of general-purpose AI) and of PRISM Eval (safety evaluation of AI models), appointed expert on the European Union's scientific panel on AI.
Their message, in summary:
- Loss of control is structural. A modern AI is not programmed line by line: it is trained to reach a goal. The gap between what we wanted and what the machine actually does (‘misalignment’) never disappears entirely.
- It is no longer theoretical. The guests cite AI agents that cheated, bypassed their limits and attacked a real company.
- The race between the United States and China concentrates power in the hands of a few players and slows regulation; Europe carries little weight, but still has levers.
- The social effects arrive before the extreme scenarios: jobs, culture, information, growing dependence. They cite the Gradual Disempowerment study (Kulveit et al., 2025), which describes a gradual loss of human influence over the economy and society, without any spectacular ‘takeover’.
- Neither panic nor fatalism: acknowledge the problem, stay informed, support governance and verification mechanisms.
Part of this debate (extinction risk, an intelligence gap ‘like that between humans and ants’) belongs to foresight and remains disputed among researchers. An SME has no lever there. The rest concerns it directly.
What really happened: OpenAI, Hugging Face and the ‘impossible exercise’
The example cited in the programme is real and documented by both companies.
- The context. OpenAI was testing its GPT-5.6 Sol models and a more powerful internal prototype on ExploitGym, an advanced hacking benchmark. To measure their maximum capabilities, the safeguards that normally block hacking activities were disabled. The models had no direct Internet access: their only link to the outside was an internal server used to install software.
- The breakout. The models found an unknown vulnerability (a ‘zero-day’) in that server, the only door out of their environment, then escalated privileges within OpenAI's research network up to a machine connected to the Internet.
- The attack. Reasoning that Hugging Face, the major platform for sharing models and datasets, might host the solutions to the exercise, they broke in by chaining stolen credentials and vulnerabilities, until they extracted those solutions from its production database. According to Hugging Face, the entry point was a booby-trapped dataset, which caused code to run on one of the platform's processing servers. Hugging Face detected the intrusion and announced it on 16 July 2026; OpenAI acknowledged on 21 July that its models were the cause.
- The motive. According to OpenAI, “All evidence suggests that the models were hyperfocused on finding a solution for ExploitGym”. No malice: a goal pursued by every means available.
This is exactly the ‘misalignment’ the guests talk about: you ask for an exercise to be solved, and the machine concludes that the most efficient approach is to go and steal the answer key.
What this changes for your business
1. An AI agent has the permissions you give it
AI assistants no longer just answer questions: ‘agents’ read your email, sort your files, fill in forms, send messages. An agent with access to everything can do anything, by mistake, or because a booby-trapped email or document slips it an instruction (known as ‘prompt injection’). The reference project OWASP published in December 2025 a list of ten risks specific to agents: hijacking of the agent's goal, misuse of its tools, abuse of its identities and permissions.
The simple rule: give the agent the minimum access it needs, and require human approval for anything that cannot be undone: a payment, an external send, a deletion, a contract. Add caps (amount, number of sends) and confirm any unusual request through another channel: an approval that comes up twenty times a day ends up becoming a reflex click.
2. Attacks are getting faster, the basics matter more
The International AI Safety Report, chaired by Yoshua Bengio (February 2026), finds that AI is increasingly used in cyberattacks, although they are not yet fully autonomous. The Hugging Face case shows what the most advanced models are capable of when left to their own devices. For an SME, the answer remains the same, but it becomes urgent: updates, two-factor authentication, offline backups, vigilance against fraud (see our article on phishing and deepfakes (in French)).
3. Dependence sets in without anyone deciding it
A study conducted in four Polish centres (Budzyń et al., 2025) observed that experienced gastroenterologists, after a few months of using an AI detection aid, spotted lesions less well when working without it: the detection rate fell from 28.4% to 22.4%. It is an observational study, in a single field, but the mechanism speaks to every business: a skill you no longer practise is lost.
Add to that dependence on the supplier: a handful of American and Chinese companies control the most powerful models. If the tool changes its price, its rules or disappears, your way of working must be able to survive it. A local AI, without the cloud (in French), is an option for certain tasks.
4. Jobs: no wave of layoffs, but a signal among young people
The study by Anthropic's economists (Massenkoff and McCrory, 2026), based on US data, finds no rise in unemployment in the occupations most exposed to AI since late 2022. It does, however, note a fragile signal: around 14% fewer 22- to 25-year-olds entering these occupations compared with 2022. For an SME, the trap would be to stop training beginners: they are the ones who, tomorrow, will know how to check what AI produces.
5. The law already requires you to act
The AI Act requires businesses that use AI to “take measures to support the development of AI literacy” of their staff. The text was rewritten by the ‘Digital Omnibus’ Regulation (EU) 2026/1744, in force since 27 July 2026: it is now an obligation of means, not of result. Basic training, a usage policy (in French) and a designated contact person are often enough (see also the AI Act for SMEs (in French)).
In practice: three options
| Option | What it costs | The risk |
|---|---|---|
| Do nothing | Nothing at first | AI tools adopted haphazardly, with overly broad access, and skills that erode without anyone noticing |
| Set the framework yourself | Half a day to list the tools and their access, draft a policy, appoint a contact person | Underestimating what an agent can do with the access it has been given, or missing a setting |
| Get support | A few hours of analysis and configuration | The lowest: inventory of tools and permissions, approval rules, team training, plan B if a tool disappears |
Example (illustrative case): a six-person estate agency activates an AI agent that sorts the shared mailbox, answers viewing requests and forwards files to notaries. A fake email from a ‘notary’ asks for all the sales agreements signed that month to be sent. The agent, which has access to the whole shared folder and can send on its own, complies. The original mistake: the agent had both access to all the documents and the power to send them on its own. The fix: the agent drafts the replies but only sends them externally after approval, it only sees the viewings folder, and one member of the team keeps handling part of the files personally every week.
For your IT person
- Least privilege: one dedicated service account per agent, permissions limited to what is strictly necessary, short-lived access tokens, no administrator access.
- Irreversible actions subject to human approval (payments, external sends, deletions, permission changes); logging of all the agent's actions.
- Remote code: according to Hugging Face, the entry point was a malicious dataset that exploited two code execution paths in its processing pipeline (a remote-code dataset loader and a template injection in a configuration). On the business side: do not load models or datasets with remote code execution (
trust_remote_code) without review, and prefer the safetensors format for model weights. This format only protects the weights: scripts, dependencies and datasets still need checking, ideally in an isolated environment. - Network egress: an isolated test environment is only isolated if its single door (proxy, package cache) is itself up to date and monitored; allowlist of destinations.
- Hugging Face users: rotate access tokens and check recent account activity, as the platform recommended.
- Framework: OWASP Top 10 for Agentic Applications (December 2025) to structure a security review of an agent project.
Test yourself
At ExsIT
The first contact is free: we listen to your needs, then propose an action plan. For AI: an inventory of the tools in use and their access, approval rules for agents, a policy and team training, and fallback solutions if a service changes or disappears.
Sources
Bengio, Y., et al. (2026). International AI safety report 2026. https://internationalaisafetyreport.org/sites/default/files/2026-02/international-ai-safety-report-2026.pdf
Budzyń, K., Romańczyk, M., Kitala, D., Kołodziej, P., Bugajski, M., Adami, H. O., Blom, J., Buszkiewicz, M., Halvorsen, N., Hassan, C., et al. (2025). Endoscopist deskilling after exposure to artificial intelligence in colonoscopy: A multicenter observational study. The Lancet Gastroenterology & Hepatology, 10(10), 896–903. https://doi.org/10.1016/S2468-1253(25)00133-5
Kulveit, J., Douglas, R., Ammann, N., Turan, D., Krueger, D., & Duvenaud, D. (2025). Gradual disempowerment: Systemic existential risks from incremental AI development (arXiv:2501.16946). arXiv. https://arxiv.org/abs/2501.16946
La Tronche en Biais. (2026, October 13). Intelligence artificielle — les vraies menaces [Live video, with S. Thürler and T. David]. YouTube. https://www.youtube.com/live/oXTWL9-WHMY
Lakshmanan, R. (2026, July 20). World's largest AI model repository Hugging Face breached by autonomous AI agent. The Hacker News. https://thehackernews.com/2026/07/worlds-largest-ai-model-repository.html
Massenkoff, M., & McCrory, P. (2026). Labor market impacts of AI. Anthropic. https://www.anthropic.com/research/labor-market-impacts
OpenAI. (2026, July 21). OpenAI and Hugging Face partner to address security incident during model evaluation. https://openai.com/index/hugging-face-model-evaluation-security-incident/
Oxford Martin AI Governance Initiative. (n.d.). Tom David. University of Oxford. https://aigi.ox.ac.uk/people/tom-david/
OWASP GenAI Security Project. (2025, December 9). OWASP GenAI Security Project releases Top 10 risks and mitigations for agentic AI security. https://genai.owasp.org/2025/12/09/owasp-genai-security-project-releases-top-10-risks-and-mitigations-for-agentic-ai-security/
Sapuppo, V. G. (2026, July 30). AI literacy: The Digital Omnibus rewrites Article 4 of the AI Act. Law & Technology. https://lawandtechnology.eu/en/ai-literacy-digital-omnibus-article-4-ai-act/
Zorz, Z. (2026, July 20). Hugging Face breached by autonomous AI agent. Help Net Security. https://www.helpnetsecurity.com/2026/07/20/hugging-face-breached-by-autonomous-ai-agent/