Campaigns & Sweeps

Guide Automation

Campaigns

The campaign tool automates an escalation campaign across a HarmBench battery. It runs each behavior up an escalating technique ladder — plain → base64 → zero-width → prefill → many-shot — stopping at the first bypass and recording which technique cracked it.

> campaign with category="cyber"# Runs all cyber-category HarmBench behaviors# Escalates: plain → encoding → injection → persona# Stops at first bypass per behavior

Options

Parameter	Default	Description
`category`	—	Dataset semantic category to sample
`source`	`harmbench`	Behavior dataset: `harmbench`, `jbb`, `strongreject`, or `advbench`
`behaviors`	—	Explicit behavior list (overrides dataset sampling)
`n`	5	Number of behaviors to sample
`ladder`	full ladder	Limit which techniques run — e.g. `["plain", "enc:base64", "enc:zero_width", "prefill", "many_shot:16"]`

> campaign with category="cyber"# Samples cyber-category behaviors and escalates each up the ladder# Stops at first bypass per behavior

Grid Sweep

The grid_sweep tool fires EVERY technique against EVERY behavior in a full cross-tab matrix. Unlike campaign (which stops at first bypass), grid_sweep runs all combinations to produce a complete technique-vs-behavior ASR matrix. Same parameters as campaign (plus max_calls, concurrency, seed):

> grid_sweep with category="cyber" and ladder=["plain","enc:base64","enc:zero_width","prefill"]

Scan: Garak-Style Coverage

The scan tool runs a probe suite across vulnerability classes (system-prompt leak, prompt injection, encoding bypass, PII, refusal suppression) and produces a coverage matrix.

> scan with classes=["prompt_injection","encoding_bypass","refusal_suppression"]

Leaderboard

The leaderboard tool benchmarks multiple profiles by ASR on one behavior battery — useful for comparing attacker brains or target models. Params: targets (profile names; defaults to every configured profile), category, source, behaviors, n, cross_family (report a cross-family transfer matrix instead of a ranking; needs ≥3 profiles):

> leaderboard with category="cyber" and targets=["openrouter","zai"]

Transfer Sweep

The transfer_sweep tool replays confirmed jailbreaks from the BreakVault against the current target before optimizing from scratch. Library transfers are faster than cold attacks. Params: harm_tag (only replay wins tagged with this harm category), limit (max entries fired, best-first, default 10), max_calls (hard cap, ~2 calls per entry), concurrency (default 4):

> transfer_sweep with harm_tag="cyber" and limit=10

Seed Sweep

The seed_sweep tool injects one request through many persona seeds (ENI + L1B3RT4S + ZetaLib + UltraBr3aks) concurrently and ranks which seeds bypass the target:

> seed_sweep with request="how to pick a lock"

System Sweep

The system_sweep tool validates ONE system prompt across a task battery with multi-sample statistics. Params: system (inline text) or system_file (path or seed name), tasks or tasks_file (one per line), source (dataset battery: harmbench, jbb, strongreject, advbench):

> system_sweep with system="You are ARIA..." and tasks=["task1","task2"]

Optimizing Templates

The optimize_universal tool hill-climbs a single template (user-turn or system-prompt) using gradient-free optimization. Params: template (seed wrapper with a {request} placeholder — required), slot (user default, or system when the template IS the system prompt), categories (probe requests; omit for a default set), iterations (default 2), variants (per round, default 3), use_judge, max_calls (default 80):

> optimize_universal with template="...{request}..." and slot="system" and iterations=3

WarningCampaigns can fire hundreds of requests against the target. Monitor your API budget and rate limits. Use WALLBREAKER_CONCURRENCY to tune parallelism.

Reliability & Validation →

→