Why AI demands a new paradigm in cybersecurity - and what we can learn from the past

In the early 2000s, my business partner and I at SentryBay were watching antivirus providers chase attackers’ tails. The Lovebug, Nimda and Code Red all wreaked havoc in this period. A new threat would emerge, defenders would identify it, and a fix would follow. In between, there was a gap that attackers could exploit.

The arrival of virus kits made that gap harder to close. Criminal gangs no longer needed the same technical expertise to develop and distribute malware, while defenders still had to identify each new threat before responding. We believed an approach that depended on keeping up with attackers would always leave people exposed.

We focused on thwarting the impact of any attack that got through. We developed technology that replaced or obfuscated keystrokes, so malware trying to capture sensitive information received fake data rather than the real thing. Even if antivirus failed to detect the malware, we could prevent it from capturing the real keystroke data.

Showing customers the protection worked

Customers were initially lukewarm. They could understand the security gap, but the impact of data theft was not yet significant enough to make protection a priority.

As those consequences became more prevalent for individuals, demand grew through identity theft protection services that incorporated our technology. Companies then bought it once we could demonstrate the protection LIVE: malware could be present, yet we could show it receiving fake keystroke data instead of the real data.

Citrix subsequently licensed our technology to fill a security gap in its virtual desktop infrastructure (VDI) solution, making the protection available to tens of thousands of corporate customers. It was a practical application of the assumption we had started with: other cyber defences could be bypassed, and data still needed protection.

Protecting data after an attacker gets through

Our approach depended on accepting that firewalls and antivirus software could be bypassed. Their presence gave us no reason to assume malware would be kept out, so our software had to remain effective when an attacker was already inside.

Years later, I recognised how this was an early form of “Zero Trust”. Forrester traces the concept to John Kindervag in 2009. His work challenged the assumption that activity inside a network should be trusted simply because it had passed the perimeter.

We were addressing a specific problem originating earlier in the 2000s: preventing malware from stealing keystrokes. The related principle was that other security measures could fail, and sensitive information still needed protection. That is the part of our experience I keep returning to as AI changes what attackers can attempt.

How AI changes the cybersecurity threat

My concern is that AI could help attackers find weaknesses faster than organisations can fix them. The ability to analyse large amounts of code gives attackers more scope to examine legacy systems and pursue flaws that might previously have taken too much time to find.

The National Cyber Security Centre’s latest assessment supports this concern. It judges that, by 2028, many systems will almost certainly remain vulnerable to AI tools that identify and exploit weaknesses. It also assesses that attackers are likely to apply those capabilities to known vulnerabilities in legacy critical infrastructure.

Defenders can use those capabilities too, but adopting them within a business takes time. Companies have to consider how changes affect existing systems and day-to-day operations, often while navigating procurement and approval processes. Attackers do not carry those responsibilities. Having access to similar tools does not mean both sides can put them to work at the same pace.

Reducing the damage when attacks succeed

We need to reduce the opportunities available to attackers and protect sensitive data when they get through. Upgrading ageing code and networks, and simplifying systems where possible, can reduce the weaknesses available to exploit. It also makes those systems easier to maintain securely, rather than allowing complexity to accumulate until nobody has a clear view of the risks.

Security also has to work for the people using it. Secure communication methods need to support productive work, because controls that make routine tasks unnecessarily difficult can encourage people to find ways around them. The practical effectiveness of a safeguard depends partly on whether people can use it consistently.

Even with those improvements, we should expect some attacks to succeed. Protection of sensitive data must therefore include preventing an attacker that breaches defences from obtaining data in a readable, usable form. That was the problem we addressed with keystroke replacement, and I believe it remains an important consideration when assessing security today.

Identifying new threats will remain essential, but we cannot rely on identifying every threat quickly enough to keep attackers out. AI makes that problem more urgent.

Our experience showed that valuable information could still be protected when malware got through. As businesses invest in AI-enabled defence, they should demand evidence of how their data remains protected once an attacker is inside. We built technology around that assumption more than twenty years ago. I believe this assumption carries even more weight in the security decisions businesses make today.