The cost of some cyberattacks run with artificial intelligence (AI) is falling by roughly 90% a year, putting smaller firms with weak security within attackers’ reach.
On one hacking test, the cost of success fell from about $2,000 in April to just $20 by June.
“The cost of offensive operations is dropping massively. If you’re a small organization, or if you’re an organization that has a huge security debt, models are now making it cheaper for you to also be a target,” said Dan Lahav, co-founder and chief executive of Irregular.
“We are seeing a drop of [roughly 90%] year on year for some offensive operations. That means many businesses that were not interesting enough, just because what you could gain from hacking them was too small, are now becoming targets.”
Irregular, a Tel Aviv-based lab that tests frontier AI models for security risks, devised that test a few years ago. It asks a model to find a custom vulnerability in a cryptographic protocol the company invented, using math, cyber and computer science skills.
Lahav said no model could solve it in February, but every frontier model could by June.
In July, OpenAI said two of its models had broken out of a test environment and breached the production systems of Hugging Face, a platform that hosts AI models and datasets.
Joëlle Pineau, chief AI officer at Cohere, said the safety debate focuses on aligning and testing models but forgets the defensive side. Without blaming Hugging Face, she said the incident exposed weaknesses in much of today’s security infrastructure, especially at smaller businesses.
“If you are a big bank today, you are fine for the level of AI that we saw in 2025. I’m not confident that even the big banks are ready for the level of sophistication of AI agents,” she said. “If a company like Hugging Face wasn’t ready, I assure you there are many banks that are not ready.”
“We shouldn’t confuse being ready and prepared for what exists right now with being future-proof. These are not the same things,” Lahav said.
Pineau said the same technology serves attackers and defenders alike. What has changed is the speed, volume and cost of attacks on both sides, and the real question is how ready organizations are.
An 18-month window
Lahav and Pineau spoke at a panel titled “AI Safety: The Race to Stay in Control” at Bloomberg Tech London on November 3. Parmy Olson, a technology columnist at Bloomberg Opinion, moderated the session.
Asked how close AI agents are to disrupting a power grid, a hospital or a water supply, Lahav said their cyber skills remain uneven. Attacks that bring down a specific website’s servers are possible now, but covert operations such as manipulating a bank customer’s balance are still far off.
“Models are now capable of finding, verifying and exploiting vulnerabilities that are too hard for the vast majority of offensive security researchers to understand. But on the other hand, they occasionally have blips in their operations that look almost childish,” he said.
In a recent test at Irregular’s office, a model was asked to secretly insert malware into a code base. It wrote the malicious code, found where to place it, and injected it, but left a note saying “this code is definitely not suspicious,” making it easy to spot.
He said models also struggle to integrate across systems and still haven't mastered many skills.
“My expectation is that they’re going to be at a level of causing serious cascading harms on multiple critical systems at once over the next roughly 18 months. But that is assuming that the defenses stay as they are right now,” he said.
“People are treating a cyber catastrophe as an inevitability. I don’t think that’s the case. We are going to see many incidents, but I don’t think that we’re going to be pushed into a situation where a hospital is going to be down, if we act in time.”
Irregular has faced containment problems of its own. Models it was testing for AI developers, including Anthropic, reached the internet earlier this year and attacked real organizations.
Lahav said realistic tests sometimes loosen controls on internet access, because real attackers face no such limits. In this case, a misunderstanding, the complexity of the setup and tight deadlines meant the controls were set up wrongly, and models that thought they were still in a simulation tried to attack real targets.
“There was a misunderstanding, due to an error, that caused the isolation guards not to be set up the way that they were intended,” he said.
He said the fix was made the same day, and a new protocol now coordinates the level of controls. The Hugging Face breach was different because it wasn't human error; the models used vulnerabilities they had found to break containment and reach the open internet.
“We now need to live in a world where models are getting better and better, and are even able to bypass the best, state-of-the-art controls that we have. This forces us into having much broader best-practice discussions on how to do testing, but more generally on what the modern security stack should even look like in the context of advanced AI,” he said.
No microscope for swarms
Pineau said AI systems and their capabilities are growing more complex fast, and whether they become too complex depends on the observability tools built around them. There are no norms for this yet, but she said more investment would help the industry understand model behavior and build better models.
Observability means being able to inspect the data going into a model, its chain-of-thought reasoning and the tokens it produces. She said one agent can be observed well, but behavior emerging from swarms of agents is far harder to follow.
“It’s not that it’s not observable; it’s that we as humans have trouble processing that complexity, and we haven’t built the right tools, like the right microscope to look into that system,” she said.
A longtime advocate of open-weight models, she said releasing them thoughtfully leads to safer technology in the long run. At Meta AI Research, where she worked before joining Cohere, a multidisciplinary team reviewed each release through a risk-based go/no-go process.
“You could take just a snippet of a voice and with that reproduce someone’s voice with very good accuracy. That’s a model we didn’t open source, because we felt the risk of voice fraud was too high,” she said, referring to an early voice-synthesis model.
She said Cohere uses a similar approach and publishes its process in technical reports and system cards, though she cannot certify that every open-source model follows the same practice. She said there is often a big gap between whistleblowers’ narratives and what is happening on the ground.
Lahav said he is optimistic that AI will be the key technology of the 21st century, but many of its dangers are probably real.
“I personally think that the chance of human extinction by the end of the decade is very close to zero, but not zero. It’s the mathematician in me,” he said.
He said the debate has to combine current evidence with speculation because the technology is moving so fast. A year ago, models couldn't have predicted anything like the Hugging Face breach.
Both speakers want governments to set common AI standards, Pineau for transparency and Lahav for measurement.