← All field notes
insider riskgovernancejust culture

The biggest insider risk is a good admin having a bad day

The insider threat that fires most often is not a spy or a thief - it is a trusted administrator making a high-blast-radius mistake. Here is why administrative error is the dominant insider risk, and why disciplining it makes your next outage worse, not less likely.

Ask people to picture an insider threat and they picture malice: the disgruntled employee copying files on the way out, the recruited spy. That picture is not wrong, but it is rare. The insider event that actually recurs in real operations is duller and far more common - a trusted administrator with full authority doing exactly what their access allows, to the wrong target, at the worst possible moment. A destructive change applied to production instead of staging. A security control disabled during maintenance and never re-enabled. A bulk operation run against the wrong scope. Right ticket, wrong profile.

The mental model to carry into insider-risk planning is uncomfortable for most programs, because most programs are built around catching bad actors: your largest insider exposure is not a person deciding to hurt you. It is a good person, doing their job, on a bad day, with enough privilege to turn a small slip into an outage. If your insider program only looks for intent, it is aimed at the smaller problem.

Why is administrative error the dominant insider risk?

Because a privileged mistake inherits all the privilege, and there is nothing anomalous about the actor to catch. When a malicious outsider acts, there is a stolen credential, an exploit, an unusual login, a chain of tells. When a trusted administrator makes a mistake, the identity is correct, the access is authorized, the action is one they are allowed to take - the only anomaly is in the target or the timing, which most detection is not watching for. The event is invisible to everything tuned to find intruders.

And the blast radius is set by the privilege, not the intent. An administrator who can rebuild a domain can also, with one wrong parameter, break it. The reach that makes them effective is the same reach that makes their error catastrophic. This is why “just be more careful” is not a control and never has been - it asks humans to be perfect precisely where the system has decided perfection is the only safeguard. Careful people still have bad days; the question is what the system does when they do.

Why does disciplining the mistake make the next one worse?

Because it changes what people do with information, and the information you most need is the early, embarrassing kind. The reflex when a serious mistake causes visible damage is to find who did it and make an example of them. It feels like accountability and it satisfies the room. It is also the single most reliable way to guarantee the next incident is hidden until it is large.

Here is the mechanism. Engineers see a colleague disciplined for an honest error. The rational response is not to become more careful - it is to stop volunteering that anything went wrong. The near-miss that would have prompted a guardrail goes unreported. The small misconfiguration that someone notices at 2pm gets quietly worked around instead of flagged, and surfaces at 2am as an outage. You have not reduced errors; you have blinded yourself to them and pushed their discovery later, when they are more expensive. Every high-reliability field that has confronted this - aviation, medicine, site reliability engineering - reached the same conclusion: punishing honest error destroys the reporting that keeps systems safe.

There is a real distinction to hold here, and it is not “never hold anyone accountable.” It is that honest mistakes and genuine negligence or malice are different categories with different responses. Blameless applies to the competent person acting in good faith. It does not mean consequence-free for someone who was reckless or hostile. Conflating the two is how organizations either punish everyone or excuse everyone, and both are wrong.

What is the right regime for administrative error?

Three moves, in order, and none of them is punishment: contain, learn, then fix the system.

First, contain the blast radius. Reverse the destructive change, re-enable the disabled control, restore the affected service. This is pure incident response and it comes before anything about the person - the goal is to make the damage stop and shrink, using rollback and recovery, not to assign fault while the service is still down.

Second, run a blameless post-incident review. The question the review asks is the whole point: how did the system allow a routine, authorized action to become a disaster? Not who to blame, but what let one wrong parameter reach production, why the staging and production paths looked identical, why a destructive operation had no confirmation. The error is treated as evidence, and the person who made it is usually the single best source of understanding about the gap.

Third, install the systemic guardrail the review reveals. The confirmation step for destructive actions. Enforced separation between staging and production profiles so the wrong target is not one keystroke away. Four-eyes or dual control on the highest-blast-radius operations. Staged rollout so a bad change hits a canary, not the fleet. Least privilege so a routine task does not carry catastrophic reach. Break-glass access that works but is audited. Automated rollback so recovery is fast and boring. The person is a signal that the system had a gap; the fix is to close the gap so the next good person on a bad day cannot fall through it.

How do you detect an admin error when the actor is authorized?

Not with the tools that hunt intruders - the identity, the access, and the action are all legitimate, so the only anomaly is in what changed and whether a safeguard went dark. Detection has to shift from “who is acting” to “what was done, and did it match intent.”

That means integrity monitoring on the change plane, not behavioral analytics on a person. Capture every privileged change with before-and-after state, so a destructive change to production stands out against the baseline of normal work. Watch for configuration drift: a control that was on and is now off is a signal regardless of who turned it off or why, which is how you catch the safety mechanism disabled during maintenance and never re-enabled. Check scope against intent: a bulk operation whose reach exceeds the change request, or a change applied to a production profile when the ticket named staging, is exactly the fat-finger you want to catch in minutes rather than at the next outage.

Two properties make this detection actionable. It has to be attributed - tied to a person, a time, and a source - so a responder is reasoning from facts, and it has to render in plain language, not a raw diff or a bitmask, so that whoever sees the alert can understand “the staging-only control was disabled in production” and act without decoding it. That is the difference between an alert a junior admin can act on and one that sits unread. The same change-capture that surfaces the error is what later makes containment a rollback instead of a rebuild, which is why detection and recovery for administrative error are really the same investment.

How does governed identity change the outcome?

It converts an administrative error from a silent time bomb into a contained, understood, reversible event. The difference between an organization that survives admin error gracefully and one that is quietly wrecked by it is almost entirely about whether the privileged plane is governed.

Governed means three properties are true at the moment of the mistake. Every privileged action is attributed - to a person, a time, and a source - so the review has facts instead of guesses and the change can be traced and undone. Every change is reversible, so containment is a rollback rather than a rebuild. And every disabled control is alarmed, so a safety mechanism that gets turned off during maintenance does not stay off silently for weeks. When those hold, the fat-finger is annoying and recoverable. When they do not, you are relying on your best people never having a bad day, which is not a strategy - it is a hope.

This is also why administrative error belongs in the insider-risk conversation and not filed away as an ops problem. It is an insider event by definition: a trusted person, legitimate access, real harm. Treating it as a governance and systems question - rather than a discipline question - is what separates programs that get safer over time from programs that get quieter over time while the underlying risk grows.

The through-line is simple and worth stating plainly: your most dangerous insider is usually not an attacker, it is authority meeting a bad day, and the way you respond either builds a system that absorbs that or a culture that hides it.

Frequently asked questions

Is administrative error really a bigger risk than malicious insiders?

In frequency, yes for most organizations. Malicious insiders are real but rare; trusted administrators making high-blast-radius mistakes happen constantly, and because the actor is authorized there is no exploit or stolen credential to catch - only the damage. The impact of a single privileged mistake can rival a deliberate attack because it inherits the same privilege.

Why is disciplining the person who made the mistake the wrong move?

Because it optimizes for the wrong thing. Punishing an honest mistake does not make people more careful; it makes them quieter. Engineers stop reporting near-misses and early errors, so the organization loses its cheapest, earliest warning, and the next mistake is discovered late and large. You trade one correctable event for a culture of concealment.

What is a blameless post-incident review?

A review that asks how the system allowed a routine action to become a disaster, not who to blame. It assumes competent people acting in good faith and treats the error as evidence of a missing guardrail - a confirmation step, staging enforcement, dual control - so the fix is a system change that makes the same mistake impossible or harmless, not a reprimand.

Does blameless mean no accountability?

No. Blameless is about how you treat honest mistakes, not genuine negligence or malice. Accountability shifts from punishing the individual to fixing the system and documenting the change. The person is still involved - often as the person who best understands the gap and helps close it - but the outcome is a guardrail, not a scapegoat.

What systemic guardrails actually prevent high-blast-radius admin errors?

Change management with staged rollout, four-eyes or dual control on destructive actions, least privilege so a routine task does not carry catastrophic reach, clear separation between production and staging profiles, break-glass access that is audited, and automated rollback. Governed identity underpins all of them by making every privileged action attributable and reversible.