Build an eval set from production transcripts

Turn real conversations into a Kno eval set with kno mine, content-hash ids, weak labels marked as derived, PII warnings built in.

Evals written by hand drift from reality. kno mine turns production transcripts into cases with the same JSONL shape every other stage expects.

kno mine --logs transcripts.jsonl --format jsonl-chat --mode immediate

What mine does

Content-hash ids, stable, deduplicable, no ID bookkeeping.
Weak labels marked, labels derived from signals (resolution status, follow-ups) are recorded as derived, so provenance survives ingestion.
Review path, --review walks each case as keep/edit/drop on a terminal (requires a TTY) and writes the decisions to a manifest beside the output, which re-mining reads back.
PII caveat, mined transcripts are the most PII-dense input Kno touches; the cookbook states the retention obligations plainly.

Modes

kno mine --logs transcripts.jsonl --format jsonl-chat --mode immediate   # label from the transcript itself
kno mine --logs transcripts.jsonl --mode resolution  # label from how it resolved

Then value against it

kno baseline --evals mined.jsonl
kno value --evals mined.jsonl --pool help-center.jsonl --baseline-run-id <run id>

A complete recipe against real vendor data: Value your Zendesk knowledge.