In an era of AI-powered impersonation, resilience depends on something more fundamental, writes Patrice Bouexel, General Manager (Europe), SIS ID.
For years, organisations have invested heavily in helping employees identify suspicious activity.
Staff are trained to recognise phishing emails, challenge unusual requests and remain alert to the warning signs of fraud.
These programmes have become a core component of modern cybersecurity strategies and, for good reason, human error continues to play a role in many security incidents. The problem is that many of these programmes are built around an assumption that is becoming increasingly difficult to defend.
They assume people can reliably distinguish legitimate communications from fraudulent ones.
In today’s threat landscape, that is no longer a certainty.
The impact of AI
AI is changing the economics of deception.
Criminals can now generate professional emails in seconds, replicate writing styles and produce communications that closely resemble genuine business correspondence.
Voice cloning technology can imitate the speech patterns of senior executives, while deepfake technology continues to become more convincing and accessible.
As a result, many of the signals employees have traditionally relied upon are becoming less useful.
A familiar voice no longer guarantees authenticity, well-written email no longer proves legitimacy and even video evidence can no longer be accepted at face value.
The challenge facing organisations is not that employees are becoming less vigilant – it is that deception is becoming harder to detect.
This matters because many organisations continue to place significant responsibility on employees to identify fraudulent requests before damage occurs.
Security awareness training remains important, but awareness alone was never designed to withstand a world where fraudulent communications can be generated at scale and tailored to individual targets with remarkable accuracy.
The new era of scams
Historically, many scams failed because attackers lacked credibility.
Poor grammar, suspicious email addresses and generic messages often exposed their intentions.
Employees who followed basic security guidance could identify many threats before any harm was done and the effectiveness of awareness training was reinforced by the fact that attackers frequently made mistakes.
Today, those mistakes are becoming less common.
Generative AI has dramatically lowered the barrier to creating convincing content.
Criminals no longer need strong language skills, detailed knowledge of a target organisation or significant technical expertise to create a believable attack.
Increasingly, they can rely on AI tools to do much of the work for them.
For security leaders, this creates an uncomfortable question: if fraudulent communications become almost indistinguishable from genuine ones, how much protection can awareness training realistically provide?
Organisations cannot simply train employees indefinitely and expect better outcomes.
There are limits to what human judgement can achieve when the information being assessed becomes increasingly difficult to verify.
The challenge
The challenge extends beyond phishing emails.
Modern attacks often target business processes rather than technical vulnerabilities.
A request to update supplier details, approve a transaction or share sensitive information may appear completely routine.
The attacker does not necessarily need to compromise a system if they can convince someone to willingly take the action on their behalf.
This is why AI-powered deception should be viewed as more than a fraud problem.
It is a resilience challenge.
When organisations depend heavily on employees spotting suspicious behaviour, they are relying on a control that becomes weaker as deception becomes more sophisticated.
Security professionals often talk about reducing attack surfaces, and traditionally, that conversation has focused on networks, applications and endpoints.
Yet one of the largest attack surfaces inside any organisation remains trust itself.
Every process that relies on an employee making a judgement about whether a request is genuine creates an opportunity for manipulation.
That does not mean organisations should abandon awareness training, but that employees must remain an important line of defence, and security education continues to play a valuable role.
Awareness should not be viewed as the primary safeguard against increasingly sophisticated impersonation attacks. Instead, organisations should focus on reducing their dependence on trust-based decision making.
The most resilient controls are often those that rely on independent verification rather than subjective judgement.
A convincing email may be difficult to identify as fraudulent, and verifying critical information through trusted and separate channels is considerably harder for an attacker to overcome.
This distinction is becoming increasingly important.
Identifying legitimate communication
Many organisations continue to evaluate whether a communication appears legitimate.
A more effective approach is to ask whether the information contained within that communication can be independently verified.
The difference may seem subtle, but it fundamentally changes how organisations manage risk.
AI will continue to improve.
Deepfakes will become more realistic, voice cloning more accessible and fraudulent communications more persuasive.
Security teams should assume this trend will continue rather than hoping it slows.
The question is not whether attackers will gain access to better tools, but how organisations adapt to a world where deception becomes increasingly difficult to recognise.
That means moving away from a mindset centred on detecting suspicious messages towards one focused on verifying critical actions.
Regardless of how convincing a request appears, changes to supplier bank details, payment instructions, account information or access rights should be validated through an independent and trusted process.
The objective is no longer simply to decide whether a communication looks genuine.
It is to ensure that high-risk decisions are supported by controls that do not rely solely on human judgement.
For years, security strategies have encouraged employees to spot the scam, and this remains an important objective.
But in an era of AI-powered impersonation, resilience depends on something more fundamental.
Trust can no longer be based on appearance alone, it must be reinforced through verification, segregation of duties and independent validation.
The organisations that will be best prepared are those that build processes capable of remaining secure even when nobody spots the scam at all.