--- title: "From Influence to Evidence: Teaching the SOC Not to Make the Same Mistake Twice" id: "124770" type: "post" slug: "from-influence-to-evidence-teaching-the-soc-not-to-make-the-same-mistake-twice" published_at: "2026-09-03T05:43:42+00:00" modified_at: "2026-09-03T06:56:43+00:00" url: "https://stellarcyber.ai/from-influence-to-evidence-teaching-the-soc-not-to-make-the-same-mistake-twice/" markdown_url: "https://stellarcyber.ai/from-influence-to-evidence-teaching-the-soc-not-to-make-the-same-mistake-twice.md" excerpt: "A year ago I made an argument. This year, we measured it. Last summer I wrote about the Pyramid of Influence — the idea that in a Human-Augmented SOC, not all analyst feedback is equal. Clicking “False Positive” is Tier..." taxonomy_category: - "AI-driven security" - "Artificial Intelligence" - "Machine Learning" - "Security Operations Center (SOC)" - "SOC" - "SOCless" --- *A year ago I made an argument. This year, we measured it.* Last summer I wrote about the[Pyramid of Influence](https://stellarcyber.ai/from-pyramid-of-pain-to-pyramid-of-influence-rethinking-the-analysts-role-in-the-human-augmented-soc/) — the idea that in a Human-Augmented SOC, **not all analyst feedback is equal.** Clicking “False Positive” is Tier 1. It decorates the interface. But *“FP because powershell.exe is patch automation on this host”* — that’s Tier 4. That’s not tagging, it’s **teaching.** I ended that post with a line I believed but couldn’t yet prove: The SOC gets smarter by learning from its best teacher: the analyst who knows when to nudge, when to override, and when to **teach the system not to make the same mistake twice.** Bullish on the vision. Realist about what gets us there. So this year we did the unglamorous part: we built the mechanism that carries that Tier-4 feedback forward — **institutional memory** — and then we ran the experiment to see if it actually works. It does. Here’s the receipt. --- ## The mechanism: feedback that travels forward Memory is the piece that turns a one-time correction into a lasting change. When an analyst overrides a verdict with a real reason, the platform keeps that reasoning and applies it to the next matching alert. Not as a hard rule that fires blindly — as **influence.** The AI weighs it; the analyst still decides. A nudge on the wheel, not a hand off the road. The promise is simple: **teach the system once, and it shouldn’t repeat the mistake.** The question was always whether that holds up when you measure it. --- ## The test We didn’t want a demo. We wanted a number we’d trust ourselves. So we took **150 real alerts from a production environment.** To decide what the *correct* verdict should be — without leaning on any single system’s bias — we had several strong, independent AI reviewers judge each alert blind, then anchored those judgments with a human security expert’s labels. That gave us a reliable reference for every alert. Then we found the alerts where the original production verdict **disagreed** with that reference — real examples of the kind of miss an analyst catches and corrects. For each one, we did exactly what your team does every day: captured the correction as institutional knowledge. And we re-ran triage two ways — **without memory** and **with memory** — comparing each result back to the reference verdict. --- ## The result On those corrected alerts, the share of decisions that matched the correct verdict went from **about 13% without memory to about 52% with memory.** Roughly **four times more often right**, once the system had learned from the feedback. In plain terms: **when we taught the system about a mistake, it was far more likely to get it right the next time it saw a similar alert.** The Tier-4 feedback from that pyramid isn’t decorative. It measurably moves outcomes. *Realist’s footnote — because I promised to be one: these figures come from internal A/B testing on a controlled set of corrected alerts, not a production guarantee. We’re still validating how far the effect carries across broader, everyday traffic. But the core mechanism — feedback in, better verdicts out — is no longer a claim. It’s a measurement.* --- ## Why this matters in your SOC - **The noise you already dismissed stops coming back.** Fewer repeat false positives means attention goes to what's genuinely new. - **Your senior analysts' knowledge outlives them.** Their understanding of your environment becomes durable, governed knowledge — not tribal memory that walks out the door at turnover. - **It compounds.** The longer your team works with the system, the more it reflects your environment specifically. That's a moat that grows with use. - **MSSPs get it per customer.**Knowledge is scoped to a single tenant with strict isolation — every customer's context sharpens their own triage, and never crosses over. And because every learned item traces back to a human decision, none of this is a black box. You can always see what informed a verdict — and overrule it. Autonomy that respects your input enough to let it guide the machine. ### **Feedback is still fuel — now with a gauge** A year ago, “feedback is fuel” was a conviction. Now it’s a number. And we’re not done: the next step is letting analysts teach the system **up front** — describing their environment proactively instead of one alert at a time — so the machine walks in already knowing your patch servers, your guest segments, your service accounts. The SOC doesn’t get smarter by itself. It gets smarter by learning from the analyst who knows when to nudge, when to override, and when to teach it not to make the same mistake twice. Last year I argued that. This year I can show it. *See the Stellar Cyber Human-Augmented Agentic SOC — including institutional memory — in action.*[Request a demo](https://claude.ai/cowork/local_f87a299e-0024-411c-bfc1-291407ddbc9c#) *or talk to your Stellar Cyber representative about the Autonomous SOC add-on.* ## Related Posts [https://stellarcyber.ai/iam-identity-signals-managed-services/](https://stellarcyber.ai/iam-identity-signals-managed-services/) [https://stellarcyber.ai/your-next-identity-may-not-be-human/](https://stellarcyber.ai/your-next-identity-may-not-be-human/) [https://stellarcyber.ai/iam-protects-the-identity-itdr-protects-the-moment/](https://stellarcyber.ai/iam-protects-the-identity-itdr-protects-the-moment/)