Restored job-posting extraction that was silently reading postings as blank

Situation

The shared job extractor turns a posting into structured requirements for both hiring ingestion (a hiring manager's roles, which drive candidate matching) and the new role fit check. The production model, Claude Sonnet 5, often wrapped its structured answer in an extra key. The schema treats every field as optional, so the wrapped answer validated as an empty job: no title, no requirements, and nothing raised an error. Hiring ingestion is live in production, so any role captured this way came through with nothing for matching to work from.

What I did

• Found it while validating the role fit check live. A clean posting came back "untitled", and I traced that to the raw tool answers.
• Measured it on staging code against the hiring-ingestion eval, then fixed it: unwrap a single stray envelope key, reject unknown top-level keys, and retry the model's output up to twice. Shipped on its own as qwestly-agent #440, ahead of the feature work, with a row in the hire matching log.
• While in there, found that production logs carried the first 80 characters of every member chat message, plus pay statements, upload filenames and LinkedIn usernames. Fixed that in qwestly-agent #439, along with turning on local logging.

Result

On the hiring-ingestion eval, failing fixtures fell from 16/35 to 5/35, and fixtures missing a role title from 13 to 0. The 5 that still fail are recall misses, not this bug. On synthetic postings, blank extractions went from 2/5 to 0/8. Production impact on real jobs is not yet measured.

Metrics

Metric	Baseline (2026-10-06, staging code)	Target	Latest (2026-10-06, fix)
Hiring-ingestion eval fixtures failing	16 / 35	≤ 5 / 35	5 / 35
Eval fixtures missing role.title	13 / 35	0	0 / 35
Synthetic postings extracted blank	2 / 5	0	0 / 8
Production hiring jobs saved with no title or criteria since the model change	not yet counted	0 new after #440 deploys	—

How to measure

• Eval: in qwestly-agent, run the hiring-ingestion eval (evals/hiring_ingestion/) through scripts/shared-env.sh --for qwestly-agent --, and compare failing fixtures and missing role.title counts. Raw answer shapes were recorded in /tmp/rf2/eval_raw_shapes*.jsonl (temporary).
• Production (admin-only): count hiring_portal.jobs created via agent ingestion that have an empty role.title or criteria, before and after #440's production release.

Log

• 2026-10-06: Baseline measured, fix merged to staging (#440). Log privacy fix opened (#439).