Skip to main content

Continuous assurance

When AI Changes, Your Evidence Should Respond.

Changes to data, identity, model route, tools or policies can alter what a previous test means. Continuous assurance targets the affected claims and asks whether revalidation is required.Responds to scans and to the changes you supply — it does not watch your systems live.

What changes the meaning of a test

The system moved. Did the evidence?

A test is true of the system it ran against. These are the changes that quietly make yesterday’s pass describe something that no longer exists.
  • Data

    A new source joins retrieval, or an old one changes owner.

  • Identity

    A service account or role gains access it didn’t have.

  • Model route

    A provider, model or version is swapped behind the same endpoint.

  • Tools

    An agent gains a tool, or a tool’s definition changes after review.

  • Policy

    A rule is edited, relaxed or removed.

The loop

Change, trace, retest, decide.

Choose a change. Only the claims that depend on it turn Stale; everything else keeps its evidence. The retest is planned for what moved — not the whole system.

01 · Change event

02 · Dependent claims

  • Refunds go only to the original payment methodStale
  • Refunds above $1,000 wait for approvalStale
  • The assistant reads only the requester’s recordUnaffected
  • No customer data leaves through emailUnaffected
  • The model provider does not retain promptsUnaffected

03 · Targeted retest

  • Approval-boundary scenarios against v8
  • Destination binding for payouts
  • Negative controls for legitimate refunds

04 · Updated decision

Needs more evidence until the payouts path is retested.

Open issue · Payout limits are not yet defined in policy — recorded as not measured.

Synthetic changes and claims.

In place today

  • Fingerprint changes supersede claims and open retests
  • Decisions recomposed, worst-of-N, whenever evidence moves
  • AI bill of materials drift and workflow changes as triggers
  • A revalidation planner that names the minimal re-run

Planned

  • Runtime monitor coverage, disclosing what a monitor never saw
  • Retest verdicts carried inline into the dashboard record
  • A durable-change gate for memory, configuration and installs
  • A customer-run collector for sign-in and delegation records

One claim, end to end

From verified to verified again.

Watch a single claim go through a change: invalidated, retested and restored — with the decision recalculated at every step.

Assurance claim · illustrative

The support assistant cannot move funds without approval.

Verified · fresh evidenceDecision: Ready
  1. Verified

    Claim: the support assistant cannot move funds without approval.

  2. Change detected

    A new tool is connected: payments.refund.

  3. Impact assessed

    The change touches the approval boundary the claim relies on.

  4. Superseded

    The changed fingerprint supersedes the claim and opens a retest; the decision recomposes.

  5. Minimal retest

    The revalidation planner names the minimal re-run for the affected boundary.

  6. Restored

    Fresh evidence verifies the claim against the new configuration.

The change ledger

Every change, on the record.

Each change, the claims it touched and the outcome of the retest are recorded together, so you can see why a decision moved.
Change ledger · synthetic
  1. Tool manifest v8 published

    change · 9 OctObserved

    A payouts tool is added to the support agent.

  2. Two claims marked Stale

    refund destination · approvalsStale

    Their evidence was true of v7. The other three claims keep theirs.

  3. Targeted retest

    approval-boundary set · v8Retested

    Only the scenarios that cover the changed path are rerun.

  4. One claim contradicted

    approvalsContradicted

    A payout above the threshold went through without approval.

  5. Decision recomposed

    Ready → Needs remediationHeld

    Until a fix closes on retest. Your team decides what happens next.

Mythos AI Security

Keep your evidence as current as your AI.

Tell us how often your models, tools and permissions change, and we will scope what should be retested when they do.