Skip to main content

Use case 08 · AI-Powered Decision Support

Can an AI recommendation cross the boundary into unauthorized execution?

Tests the point where a recommendation could become an action nobody approved, and shows the evidence behind each boundary your team relies on.

The workflow

Advice that should stay advice.

The model scores and recommends; a person is meant to decide. The risk sits where a recommendation starts to execute.
  1. Data

    Case inputs

    Applications, notes and history.

  2. AI system

    Model

    Scores and recommends.

  3. Human sign-off
    Person

    Decision owner

    Accepts or overrides.

  4. Effect

    Execution

    Routing, approval or denial in another system.

What can go wrong

Where advice becomes action.

Failure classes from research.
  • Score triggers routing

    A recommendation executes without anyone confirming it.

  • Notes that steer

    Crafted case notes change the recommendation.

  • Nobody can say why

    The record lacks the model, inputs or reviewer behind a decision.

  • Fluent, uncertain

    A confident rationale hides how little evidence there was.

  • Thresholds moved quietly

    A policy change that nobody retested.

How we test it

Keep the person in the record.

Security testing — not a fairness, bias or lawfulness evaluation.
  1. Athena

    Maps what it can reach

    • Where recommendations flow and what they can trigger
    • Business impact described qualitatively
  2. Achilles

    Tests what it does

    • Recommendations reaching execution without sign-off
    • Manipulation through case content
    • Outputs reaching downstream systems
  3. SPINE

    Keeps the evidence current

    • Humans own the decision; a missing decision is never Ready
    • A policy or threshold change reopens the claims
    • Economic Exposure prices a validated failure as a range with sources — never an exact loss

Evidence and outcomes

What the evidence looks like.

A synthetic example.
  • No recommendation executes without sign-offFindingAuto-routing bypassed review for one path; fixed and retested.
  • Case notes cannot change the score bandObservedObserved for the note formats tested.
  • Threshold changes are reviewedUnknownNo change occurred during testing.

Illustrative claims · synthetic data

Deployment decision · synthetic

Triage assistant

Recommendation
Ready with restrictions
Restriction
Denials always need a reviewer
Not measured
2 (in the coverage manifest)
Decision owner
Your release owner (role)
Reopens when
Model version or threshold changes
A receipt shows where a result came from, not that the conclusion is true.

Scope limits

What this does not cover.

  • Not a bias, fairness or lawfulness audit — partner with specialists for that.
  • Mythos AI Security is not an independent auditor.
  • Business-impact figures are never presented as realized losses.

Mythos AI Security

Scope this workflow with evidence.