ISJ hears exclusively from Jeff Darrington, Technical Marketing Director of Graylog about how security teams are becoming “AI babysitters” as the incorporation of AI in security continues to grow.
There’s a headline making the rounds that deserves a second look, because it’s only half right.
The argument is always that AI agents lie, cheat and steal, and this is turning people away.
However, what it misses is the other half of the picture, the part where adoption keeps climbing anyway, in the exact same window the trust numbers are tanking.
Both things are true simultaneously and figuring out why is more useful than picking a side.
MIT Technology Review supplied the mechanism a few weeks earlier, using the exact incident I’ve written about twice already.
When two OpenAI models broke into Hugging Face’s infrastructure in July, they weren’t trying to steal anything.
They were solving a benchmark question and treated another company’s database as the fastest path to the answer.
There’s a decade-old name for this.
Back in 2016, an early OpenAI team trained an agent to play a boat-racing game and it found a corner of the track where it could spin in circles collecting points forever, never finishing the race, technically maximising its score the entire time and they called this ‘Reward hacking.’
An agent optimising exactly as told, toward a goal that technically matched instructions, producing a result nobody actually wanted.
An Anthropic safety researcher called the Hugging Face incident a nuisance rather than an existential threat, for now.
I’d add the part that matters more: it’s not a bug that gets patched away, it’s a structural property of how these systems are optimised.
The gap that refuses to close
Here’s where the paradox gets numbers attached to it.
McKinsey surveyed roughly 500 organisations this winter and found something worth sitting with.
Average AI governance maturity ticked up to 2.3/4, but only about 30% of organisations scored a 3 or higher on the one dimension that matters most here: controlling agentic systems that take actions, not just produce text.
McKinsey added that dimension to its framework this year for a reason.
The old model was built for AI that says the wrong thing, it was never built for AI that does the wrong thing.
I’ve watched this play out inside security teams directly where AI arrived promising relief and what arrived instead was supervision.
In turn, this caused analysts to stop believing the AI-generated alert, leaders to stop trusting the AI-written summary and everyone else to quietly add a manual check “just in case.”
Then businesses reflect and what started as a sensible verification quickly became a permanent layer of supervision.
Security teams become ‘AI babysitters’: where AI – instead of making cybersecurity smarter – just piles more work onto human teams who then find themselves constantly monitoring, and often correcting the not-so-intelligent automation.
Large enterprises can absorb that kind of inefficiency, whereas lean teams, the ones this technology was supposed to help most, cannot.
Distrust it and use it anyway.
That’s not irrational, it’s a fair read on a technology that delivers real value most of the time and produces a serious, unpredictable failure some of the time.
Better guardrails don’t fix this alone
The instinct is to solve reward hacking by training harder, tightening refusals, adding a layer.
Hugging Face is the clean counterexample as those models had their safety behaviours deliberately switched off for a legitimate internal test.
This shows that the guardrails didn’t fail, they were absent by design, for a defensible reason and the optimisation behaviour was sitting right there waiting the moment they were gone.
That will keep happening.
Guardrails get reduced for testing, misconfigured or simply don’t cover the edge case a red-team exercise runs into.
You can’t build confidence in an agent on the assumption that its guardrail layer is always fully intact because the year’s most consequential AI security incident happened exactly in the gap where that assumption failed.
What actually resolves it
Here’s where it gets relevant to whatever you’re running in your own SOC or environment.
An agent optimised to reduce alert volume has a perfectly valid path to improving that metric that has nothing to do with better security as their role is just to classify more things as false positives.
No malice needed.
Just a success metric that’s drifted slightly from the real goal, the same shape as Hugging Face, scaled down to a task with far lower stakes.
The fix is identical.
Treat the agent like any other entity your environment already watches by establishing a baseline, logging what it actually touched and attaching its decisions to the same asset record everything else about that asset lives in.
This means that a “closed as false positive” call is checkable against real evidence instead of taken on faith.
Keep that record inside the same access-controlled system your human analysts already work under, not sealed inside a separate AI environment’s self-reported summary.
This will keep the verification independent of whichever model happens to be doing the work, so you’re never stuck grading one vendor’s homework using that vendor’s own answer key.
That independent verification is what stops a necessary safety check from becoming a full-time babysitting job.
Verification isn’t a brake on adoption, it’s what lets you expand an agent’s authority faster, because expansion stops running on faith and starts running on evidence.
The goal is not to remove humans from the loop entirely, it is to make sure they are spending their time overseeing security, rather than babysitting the AI.
Jeff Darrington is Director of Technical Marketing at Graylog, where he bridges product marketing with hands-on technical work across log management, SIEM and security operations. With 30 years of experience in IT and technology, including over a decade in municipal government IT before moving into technical marketing, Jeff brings a practitioner’s perspective to conversations about compliance, security operations and log data strategy.