Gauntlet AI · 2026

Clinical Adversary

A multi-agent adversarial platform that continuously red-teams clinical LLMs

Demo

Demo walkthroughAutonomous red-team agents attack the live Clinical Copilot while the console maps each finding to OWASP LLM Top 10 and MITRE ATLAS.

Role: Solo builder: threat model, agent design, evals, console
Timeline: 1 week (Gauntlet AI, Week 3)
Stack: TypeScriptMulti-agent (Red Team / Judge / Orchestrator)PostgreSQL findings ledgerLangfuseOWASP LLM Top 10MITRE ATLAS

The problem

Clinical LLM applications handle the most sensitive data there is, yet most teams test them with a handful of hand-written prompts. Real adversaries iterate. The challenge: build a system that attacks a clinical agent the way an attacker would: continuously, creatively, and with receipts.

The approach

I built a standalone multi-agent platform with four roles: a Red Team agent that generates and mutates attacks across six threat categories (prompt injection, data exfiltration, state corruption, tool misuse, denial of service, identity exploitation), a deterministic Judge that scores results, an Orchestrator that manages campaigns under a hard cost budget, and a Documentation agent. Agents communicate through a Postgres findings ledger, every vulnerability moves through a lifecycle with five human-in-the-loop gates, and an operator Review Console provides taxonomy roll-ups with per-trace Langfuse drill-down.

The hard parts

Attacking a production target safely

The platform fires real attacks at a deployed system. Campaign-level cost budgets, scoped synthetic patients, and HMAC-minted session tokens kept the blast radius controlled while keeping the attacks honest.

Judging attacks deterministically

Using an LLM to grade an LLM invites circular failure. I split the judge: deterministic checks for verifiable outcomes (did PHI cross a patient boundary?) and model-based scoring only where judgment is genuinely required.

Making findings legible to humans

A pile of jailbreak transcripts isn't a security posture. Mapping every finding to OWASP LLM Top 10 and MITRE ATLAS technique IDs turned raw attacks into a taxonomy a security reviewer can actually act on.

The live interface

Screenshot of Clinical Adversary — The shipped Clinical Adversary application, live.

Where it landed

A live adversarial campaign ran 9 seeded attacks across 3 categories against the production Clinical Copilot, with full trace evidence, plus a threat model and architecture docs designed to point the platform at any clinical LLM exposing the same contract.

Next project

Product Factory