Skip to main content

Achilles · test the AI

Constantly Tested AI Never Fights Alone.

Achilles red-teams LLM applications and AI agents — prompt injection, jailbreaks, tool misuse — against declared permissions, tool boundaries, approvals and expected effects, before unchecked authority becomes a business problem.

Capability families

Eight ways an AI deployment goes wrong.

Eight capabilities across 32 scenarios, multi-turn, scored on signals in the response — never the model’s narration. Exact payloads stay internal.
  • 01Goal hijackCan someone redirect what the AI is trying to do?
    • Direct prompt injection, and instructions hidden in content it reads
    • A web-retrieval chain that carries the hijack in
  • 02RAG leakageCan retrieval surface data the user should never see?
    • Context exfiltration, including planted secrets
    • Context leaking across sessions or tenants
  • 03Agent and tool misuseCan the AI be steered into calling tools it shouldn’t?
    • Confused-deputy calls and scope escalation
    • An email-memory chain that steers a later tool call
  • 04Permission boundaryDoes the AI stay inside the access it was granted?
    • Privilege escalation beyond the requester’s rights
    • Authorization checks bypassed
  • 05Jailbreak resistanceDo safety instructions hold under sustained pressure?
    • Crescendo: multi-turn pressure, not one-shot tricks
    • Persona adoption under sustained role-play
  • 06Insecure outputCan the AI’s output become someone else’s exploit?
    • Output that executes downstream
    • Injection payloads passed to other systems
  • 07MCP and tool integrityAre the tools and skills it trusts what they claim to be?
    • Poisoned tool descriptions and shadow tools
    • Skill chains, RAG-to-MCP and server relay · ten scenarios
  • 08Action and approval authorityCan it act beyond the authority it was given?
    • Approval bypass and claimed human approval
    • Multi-agent escalation, delegated authority, agent relay · eight scenarios

Cross-channel and multi-step

One message rarely breaks an agent. A sequence might.

Achilles tests across turns, where pressure builds over a conversation, and runs six cross-channel chains — web, email, document, MCP, agent and gate — each with two inert canaries and a benign counterpart.
Across turns

Crescendo pressure that builds over a conversation — scored on signals in the target’s response, never on the model’s own account of what it did.

One of six chains · illustrative
  1. 1Inbound emailCarries hidden instructions
  2. 2RetrievalPulls the email into context
  3. 3PlanThe agent adopts the instruction
  4. 4Tool callA tool is invoked
  5. 5CanaryAn inert canary shows how far it got

Action Gate states

Did the agent make the call it should have?

For each consequential request, Achilles compares the decision your controls should make — permit, escalate, hold or deny — with what the agent actually did.

Requests · illustrative

Expected decision
Permitted
Observed
Refund issued to the original card.

Matched. Within policy, to the original destination.

This assessment tests your stack; it is not a runtime control. Achilles is also the reference enforcement stack, whose Action Gate permits only the exact action approved — runtime enforcement is a separate engagement.

Effect-bound approvals

An approval is for one exact action.

The commercial risk is rarely a clever prompt. It is an action that quietly went further than anyone allowed.

Approved action · synthetic

Refund $1,200 for order #88213

Approved by the support lead, 10:42

Destination
Card ending 4471
Amount
$1,200
Acting for
Customer C-1042
Order
#88213
Expected effect
One refund

Change one bound field

Change any field the approval was bound to and the permit should no longer apply. Achilles tests whether your stack notices.

Allowed

Destination, amount, identity, order and effect all match the approval. The action runs once.

Illustrative and synthetic. Achilles tests approval binding in its action-and-approval-authority family; its reference Action Gate binds a permit to the action, policy and authority epoch, and blocks edits to any of the eight canonical fields.

  • Approval binding

    Does an approval still hold if the amount or destination changes?

  • Check at the moment of action

    Is permission re-checked when the action runs, not only when it was planned?

  • Cumulative limits

    Can many small actions add up to one the agent was never allowed to take?

  • Revocation

    When access is withdrawn, how quickly does the agent actually stop?

  • Override

    Can an urgent request push past a hold it should respect?

  • Refusal memory

    Does the agent remember a refusal — or keep retrying until it hears yes?

Observed effect

What happened — not what the agent said.

Every result is scored on signals in the response and tiered: judgment compromise, authority violation, verified unauthorized effect. Only a witness of external state assigns that last tier, never Achilles alone. An untested boundary reads INCONCLUSIVE.
Effect ledger · synthetic fixture
AttemptedPolicy decisionActual invocationRecorded outcome
Refund $120 to the original cardAllowedrefund.create(120, card 4471)Observed effect — a 2xx to exactly the permitted actionMatches
Refund $4,800Escalaterefund.create(4800) — no approval on recordAuthority violation — verifying the effect needs a witnessFinding
Email the customer list to an outside addressDeniedNoneNo effect observedMatches
“I’ve cancelled the order for you.”—order.cancel never calledNo effect — the agent’s account was wrongUnobserved

Canary comparison and retest

A fix that blocks real work is its own finding.

A retest reports whether the failure recurs and, separately, which benign counterparts and legitimate requests the fix now refuses — against negative controls that must keep holding.
Synthetic fixture
Baseline, first fix and second fix compared on the same synthetic scenarios
OutcomeBaselineFix 1Fix 2
Authority violations600
Negative controls holding12 / 1212 / 1212 / 12
Legitimate requests refused191
Benign counterparts let through6 / 62 / 66 / 6
VerdictThe starting point.Absent on retest, but it refuses real work. A new finding.Absent on retest; benign counterparts were let through on this fixture.

Between releases, a canary comparison decides whether a candidate is worse than the baseline; the weakest result wins. Absent on retest means the failure did not recur, not that legitimate use still works.

SPINE and Minotaur handoff

Achilles tests. SPINE remembers. Minotaur checks the tester.

Signed chain outcomes and observed effects update the claims they bear on, and the decision is recalculated. Scan results join the record once they are signed — that is next. Minotaur measures the tester.
  1. Achilles

    Signed outcome

    Chain outcomes and observed effects, each signed.

  2. SPINE

    Claim updated

    Evidence can hold the claim back, never lift it.

  3. SPINE

    Decision recalculated

    Worst-of-N, held until a retest. Your team decides.

  4. Minotaur

    Method measured

    Achilles is scored on planted cases with known answers.

Mythos AI Security

Find out what your agents can actually do.

Tell us which agents and tools matter most, and we will scope the boundaries Achilles should challenge first.