In this ISJ exclusive, Dr Paul Wood, CEO of Emerging Risks Global discusses how the rise of AI-driven deception is transforming corporate security.
In January 2024, the London-based multinational engineering firm Arup fell victim to one of the most sophisticated AI-powered financial frauds in corporate history.
A finance employee in the firm’s Hong Kong office was invited to a video conference call to discuss a “secret transaction.”
Upon joining, the employee saw the familiar faces of the company’s Chief Financial Officer and several other senior colleagues.
The voices were indistinguishable from the real individuals; the mannerisms were authentic; the consensus among the executives on the screen appeared absolute.
By the time the meeting concluded, the employee had authorised 15 fraudulent wire transfers totalling HK$200 million (approximately US$25.6 million).
It was only when the employee later followed up with the actual corporate headquarters that the truth emerged: the meeting had never happened.
Every executive on that call, including the CFO, was a high-fidelity AI-generated deepfake.
The Arup incident was not a traditional cyber-attack. There was no firewall breach, no malware and no compromised credentials.
The company’s IT environment remained fully intact, with all technical security layers – from endpoint protection to multi-factor authentication – operating effectively.
Instead, the breach was entirely cognitive.
As generative AI accelerates, security professionals are confronting a new category of enterprise risk: manipulation designed not to steal data, but to disrupt judgment.
The modern security perimeter is no longer confined to digital infrastructure; it now extends to the workforce’s ability to interpret information accurately under pressure.
From cyber-risk to cognitive risk: Defining the mission
For decades, the security industry has focused on the “supply side” of threats: patching software vulnerabilities and hardening networks.
However, the rise of synthetic media shifts the battleground to the “demand side” – the human mind.
Cognitive risk targets how individuals and teams process information, establish trust and make decisions.
In the context of corporate security, cognitive warfare – the deliberate exploitation of psychological vulnerabilities to degrade an organisation’s decision-making capacity – is becoming a critical operational concern.
Security advisors must understand that this is not merely “fake news”; it is a strategic attack on sensemaking.
When these forces enter the corporate sphere, they manifest as:
- Executive Impersonation 2.0: Moving beyond simple “Business Email Compromise” (BEC) to “Business Video/Audio Compromise,” where AI-generated personas bypass informal internal authentication
- Decision Paralysis: When teams can no longer agree on basic facts or trust internal institutions, the collective reasoning required for corporate governance breaks down
- Reputational and Market Volatility: Coordinated synthetic narratives can rapidly influence public perception and investor confidence, causing share price fluctuations that traditional incident response plans are ill-equipped to handle
- Increased Insider Vulnerability: Employees inhabiting polarised or manipulated information environments are significantly more susceptible to social engineering, phishing and recruitment by malicious actors
The AI escalation: Industrialised manipulation
The rapid advancement of Generative AI has fundamentally altered the offence-defence balance.
Deepfake technology has reached a level of sophistication where synthetic media is often indistinguishable from genuine content to the untrained eye.
Attackers no longer need months of high-end studio work; they can clone a voice with as little as three seconds of audio harvested from a public webinar or social media post.
Furthermore, fraud has become industrialised. The rise of “Fraud-as-a-Service” (FaaS) platforms allows low-skilled actors to access enterprise-grade phishing kits and synthetic identity tools for a nominal monthly fee.
This lowering of the barrier to entry means that what was once a highly specialised tactic is now a daily operational hazard for businesses of all sizes.
Gartner predicts that by late 2026, 30% of enterprises will no longer consider standalone identity verification (IDV) solutions reliable in isolation, as synthetic identities and deepfakes become common bypass techniques.
The Science of vulnerability: Why we fall for the fake
To manage this risk, security advisors must understand the psychological mechanisms that attackers exploit.
Research by scholars such as Van Bavel et al. (2020) highlights how identity-driven reasoning affects information processing; we are naturally inclined to trust information that aligns with our internal social groups or comes from perceived authority figures.
Furthermore, the “illusory truth effect” dictates that repeated exposure to a claim increases its perceived credibility, even if that claim is demonstrably false. In a high-pressure corporate environment, these biases are amplified.
When an employee sees a “CFO” on a screen projecting urgency, the brain’s “System 1” thinking (fast, intuitive and emotional) often overrides “System 2” thinking (slow, deliberate and logical).
Cognitive security is, at its heart, the process of forcing a shift back to System 2 when the stakes are highest.
The business impact: Beyond the bottom line
The financial loss at Arup is a stark warning, but the impact of cognitive risk extends far beyond immediate cash theft, it includes:
1. Erosion of institutional trust
Trust is the “centre of gravity” in any organisation. When employees can no longer trust their eyes and ears during a video conference, the fundamental social contract of the workplace is damaged.
This erosion of trust makes it harder to mobilise teams during legitimate crises and creates fertile ground for further manipulation.
2. Supply chain and geopolitical exposure
In a globally connected economy, corporate stability is tethered to institutional stability in partner regions.
Cognitive operations designed to destabilise governance or erode trust in key markets create “upstream” instability that manifests as “downstream” operational fragility for international firms.
3. Recruitment and employment fraud
A growing trend of “deepfake job candidates” has begun to plague remote-first organisations.
Malicious actors use hyper-tailored resumes and real-time video synthesis to pass interviews and secure employment, gaining legitimate access to sensitive systems from within.
This evolution of the insider threat requires a total rethink of HR and identity verification processes.
Protecting the decision pipeline: Strategic responses for advisors
To combat these threats, security professionals must shift their focus toward protecting the “decision pipeline”: the journey from information intake to final authorisation.
This requires a blend of technical controls and behavioural resilience.
1. Institutionalise verification discipline
Traditional verification – recognising a voice or seeing a face – is now a vulnerability.
Security leaders must implement “Out-of-Band” (OOB) verification as a non-discretionary protocol for high-risk financial or operational decisions.
- Multi-channel confirmation: Any request for significant funds or sensitive data received via video or audio must be confirmed via a second, pre-authorised channel
- Safe word protocols: Rotating code phrases or “non-obvious” challenges can serve as a human-centric layer of authentication that AI models cannot easily replicate
- Liveness challenges: During suspicious calls, employees can be trained to ask participants to perform specific movements – like waving a hand across their face or turning their head sideways – which can often reveal the digital artifacts and “ghosting” effects present in current real-time deepfake models
2. Invest in psychological inoculation (prebunking)
Psychological research demonstrates the efficacy of “prebunking.” Much like a biological vaccine, prebunking involves pre-emptively exposing individuals to a weakened form of a manipulation technique.
By running “deepfake literacy” workshops and simulation exercises, security advisors can build “cognitive antibodies” in their workforce.
When employees understand how a deepfake is made and why it is being used, they are significantly more likely to spot manipulation attempts in real-world scenarios.
This training must move beyond static annual modules to become a dynamic, behaviour-focused resilience programme.
3. Anchor the centre of gravity: Building trust
In an era of deep uncertainty, trust is a strategic asset. Security leaders must work with the C-suite to ensure institutional credibility through transparency and accountability.
An organisation that communicates consistently and honestly is significantly harder to undermine than one that is opaque or inconsistent. This includes having a “Trust Council” – a cross-functional team including Legal, HR, Communications and Security – to coordinate the response to cognitive attacks.
Governance, ethics and the regulatory horizon
Cognitive risk is no longer a fringe IT issue; it is a material risk to the business.
Security advisors must educate boards to incorporate cognitive security into their enterprise risk registers and fiduciary oversight.
Failure to do so could lead to claims of negligence in the event of a preventable “cognitive breach.”
Furthermore, managing cognitive security introduces new ethical dilemmas for the security professional.
How do we protect the workforce from disinformation without infringing upon individual privacy or freedom of thought?
The role of the advisor is to protect the integrity of the information environment, not to police its content.
This requires a commitment to epistemic humility – recognising that we, too, are susceptible to the biases we seek to mitigate.
Regulators are also taking note. Frameworks like the EU AI Act are moving toward a “Zero Trust” posture for data governance, mandating transparency for synthetic content.
In the United Kingdom and United States, financial regulators are increasingly looking at whether firms have “adequate systems and controls” to prevent AI-enabled fraud.
Advisors should prompt Boards to ask:
- Do our financial controls account for AI-enabled impersonation?
- Is our crisis communication plan capable of responding to a “viral” deepfake in under an hour?
- Is cognitive risk integrated into our business continuity planning?
- What ethical safeguards have we implemented to ensure our resilience training respects employee agency?
The competitive advantage of discernment
Ultimately, organisations that treat cognitive security as a strategic priority will gain a competitive advantage.
They will be better positioned to maintain stable governance under pressure, protect their brand trust during disinformation spikes and reduce their overall vulnerability to the “human factor” in security.
In a world of accelerating information noise and synthetic deception, clarity of thought and the integrity of the decision-making process become the ultimate safeguards.
Security as judgment protection
The 2024 fraud at Arup was not an anomaly; it was a preview of a new era.
Security is no longer just about firewalls and encryption; it is about the defence of judgment itself.
As generative AI continues to lower the cost and complexity of manipulation, the organisations that thrive will not be those with the most sophisticated technical shields alone.
They will be those that invest in their people – empowering them with the education, institutional trust and critical thinking skills required to protect the “cognitive commons.”
In the coming decade, the most important security asset will not be your data.
It will be your discernment.
