International Security Journal hears exclusively from Derek Whigham, Non-Executive Director and Strategic Advisor of Acumen Cyber.
Artificial intelligence has transformed from a concept in research into a ubiquitous system integral to business as usual at a breakneck pace.
Generative AI has been implemented throughout organisations for software development, customer interaction, analytics and operational efficiency in the past two years alone.
Though many businesses are still struggling to figure out how to manage and govern these tools, the technology is already advancing.
The next step isn’t only to create content, but also to build systems that can independently make and act on decisions.
This change represents a huge transformation in the way that our digital world runs.
It leaves an organisation with a whole new class of risks it isn’t prepared for.
On the one hand, security architecture cannot keep up with the pace of AI capability.
According to Gartner, more than 80% of enterprises will have used generative AI APIs or models by 2026, yet fewer than 30% currently have formal governance frameworks for AI systems.
Meanwhile, global cybercrime costs are projected to exceed £9.5 trillion annually with attackers increasingly using AI to accelerate their operations.
That is to say, the industry is building increasingly powerful automation on top of systems already grappling with some foundational security weaknesses.
Organisations that deploy intelligent systems without tackling identity governance, data management and operational oversight first are not building smarter infrastructures.
They are automating risk.
The move from assistance to action
Generative AI produces outputs, text, code, recommendations, but humans remain in control of what happens next.
Autonomous systems remove that pause.
They are designed to act.
They are designed to act: approving transactions, isolating systems, responding to threats or interacting directly with customers, often without human intervention.
This poses some important questions:
- Who is held accountable when a system makes the wrong decision?
- How do organisations hold control when decisions are rendered faster than people can respond?
- And how do you verify the behaviours of systems always adapting to change?
These matters are not alien.
Autonomous decision making is being implemented across many industry lines, such as financial services, logistics and cybersecurity.
The difference between proposing an action or actually taking one is small, but has far-reaching implications for risk.
AI amplifies existing weaknesses
We generally want AI to solve security issues with better detection and response.
Indeed, it can speed up the process, but it doesn’t solve the root of the problems that most of these environments have found.
Identity management, access control and data governance are things many organisations continue to struggle with.
These are not new issues, but their significance increases significantly when automation is involved.
Machine identities already exist in many organisations at a higher number than human users.
Applications, services and automated systems interact with the infrastructure using these identities.
They also create a way for attackers to get into automated processes should they not be secured well.
One of the challenges to handle all of this is data. AI systems use a tremendous amount of information to operate effectively.
If that data is not well handled or corrupted, the outputs of the system will turn out to be unreliable.
Once the errors in one area arise, in an automated environment they’re easily repeated at scale.
Automation makes both speed and impact greater.
An error which could possibly be slight in a manual process can soon become a major problem if the decisions are made automatically.
Where failures will most likely happen first
The first major issues connected with AI are hardly futuristic in nature. They will look similar but they will do this faster and on a greater scale.
Fraud is one obvious area. Attackers already employ AI to produce credible phishing campaigns and evade identity checks.
As companies embed automated decision making into lending, onboarding or the like, those systems become appealing targets.
Systems that are running without sufficient oversight may present another risk.
An automated process making a wrong decision may still keep making that mistake before its problems are identified.
There is also a human factor.
People trust automated systems, even with warning signs. In circumstances dealing with high pressure, this can lead to decisions that are accepted without appropriate challenge.
Not all of these failures are going to be the result of advanced technology.
They will be caused by governance, oversight and accountability gaps.
Preparing for what comes next
Artificial intelligence will be pivotal in increasing productivity and enhancing cyber defence.
The solution isn’t to slow the adoption of AI, but to take the initial steps with the right infrastructure.
Identity security has to be a top priority.
Automation brings with it a growing number of non-human identities interacting with systems.
In order to do this they have to be managed, monitored and controlled properly.
Data governance also has a vital role to play.
That is why organisations need a sense of where data comes from, how it is used and whether it can be trusted.
Oversight must also evolve.
Automated systems should work within very specific boundaries, facilitated by monitoring and auditing procedures, so organisations can tell what decisions are taken and why.
Security organisations also need to shift their tactics.
Threat modelling and testing need to extend to AI systems themselves, considering the ways in which they could be manipulated or misused.
Security must keep pace
AI is entering an environment where it’ll be making decisions about how organisations work.
This isn’t just a new tool, but an institutional shift in decision-making.
Technology transitions have long seen innovation outstrip security in past tech cycles with many changes in technology leading to safety issues.
Cloud computing and mobile platforms are good examples of this.
The transition to autonomous systems presents an opportunity to take a different approach.
For business and technology leaders, the focus should be on building resilience alongside capability.
The success of AI will not be defined by how advanced it becomes, but by how well organisations are able to control and secure it.