Running the use cases

There are two things you can run: the use-case demos (drive the real app through each use case) and the answer-quality benchmarks (score the compliance engine’s output). The package manager is pnpm.

What you need first

Requirement	Detail
App running	`server/` (API, `SERVER_PORT=3002`), `frontend/` (Vite dev on `:8125`), and `collector/`.
LLM provider key	Configured in `server/.env`.
Database	Postgres reachable via Prisma; the regulation library seeded (`pnpm seed:regulation` in `server/`).
A signed-in session	The demo capture drives a logged-in browser — you sign in once (below).

1. Use-case demos (drive the real app)

The use-case demos are defined as step scripts in the repo’s demo catalogue (frontend/e2e/manual-videos/catalog.mjs) and captured with Playwright. From frontend/:

Authenticate once — opens a browser; sign in and confirm the dashboard is visible:
Terminal window
```
pnpm manual:videos:auth
```
Capture the use cases (the three documented areas, or a subset):
Terminal window
```
pnpm manual:videos --only UC1,UC2,UC10 --allow-writes --overwrite
```
These demo-catalogue IDs map to the three areas: UC1 → Framework Compliance & Gap Analysis, UC2 → Local Health & Safety, UC10 → SAF Assessment & Evidence.

--allow-writes permits the mutating steps (adopt, scan, analyze, export). Per-use-case commands are on each use-case page.

2. Compliance answer-quality benchmark

Scores the compliance services’ answers against an 8-dimension review rubric (0/1/2 per dimension). From server/:

tsx server/scripts/audit_compliance_service.mts all --compact

It runs the real local services against the NYDFS and DORA records plus tagged internal policies, and writes a markdown report. See Results & maturity for the latest scores.

3. Compliance chat eval (routing + latency)

Runs deep compliance questions through the live compliance-guard agent and records which tools it routed to and how long each took. From server/:

SUITE=deep node llm_generated/scripts/compliance_chat_eval.mjs