Table of Contents

Onboarding Human-in-the-Loop (HITL) Widgets (updated 7/24/2026)

During the authenticated onboarding agent chat (candidate app), the AI agent sometimes needs the candidate to make a decision before it can continue. Rather than asking in prose and hoping the candidate types the right thing, the agent surfaces a structured widget — a form, picker, or confirmation dialog embedded directly in the chat. These are called HITL (human-in-the-loop) actions.

How HITL works
Widget catalog
When each widget appears (flow order)
How HITL works technically
Blocking vs persistent widgets
Reload behavior
Key files

How HITL works

The agent decides it needs input. During the onboarding flow, a tool or deterministic code path determines that the candidate must make a choice before the conversation can advance.
The server emits a hitl_action event through the SSE (Server-Sent Events) stream. This event carries an action type (e.g. "vp_ranking", "reconciliation") and any parameters needed to render the widget.
The client renders the widget in the chat. The HitlRenderer in message-list.tsx matches the action type to the correct React component. The widget appears inline in the message list, looking like a chat bubble with interactive controls.
The candidate interacts and submits. Each widget has its own submission logic — some send a structured payload to the agent, others call dedicated API endpoints silently.
The agent processes the response and continues. On the next turn, the agent reads the submission (or queries the updated database state) and advances the flow.

VP Ranking

What it looks like: A list of 5 cards, each with a benefit-framed label and a one-line description. The labels are outcome-oriented — not the internal VP names; the VP id rides along invisibly as the card's id (defined in VP_RANKING_CARDS, complete_open_qa.py):

"A stronger online presence" (your_story) — LinkedIn About rewrite, resume polish, Qwestly card narrative
"See who's interested in my background" (your_wins) — Qwestly card optimized for hiring manager opens
"Find my perfect opportunity" (your_dream_job) — curated role feed, tighter matching
"A profile that shows how I really work" (your_skills) — AI readiness + how-you-think signals on card
"Intros to people worth meeting" (your_interests) — peer intros, warm intro paths by topic/industry

The candidate reorders them by importance using up/down arrow buttons (not drag-and-drop — the reorder animates as a swap, but there is no drag interaction).

When it appears: After the initial greeting and open Q&A, before LinkedIn intake begins. The agent needs to know which topics to prioritize during the VP interviews.

What the candidate does:

Reorder the cards to reflect their priorities
Click "Submit Rankings"

What happens after submission:

The rankings are sent as a system message (ranking_submission)
The server validates all 5 VPs are present
The agent advances to LinkedIn intake

Key facts:

Blocking: yes (chat input is disabled until submitted)
Ranking is used to order the VP interview sequence
Rankings persist in the conversation transcript, not in a separate profile

Preference Form (Location)

What it looks like: A free-text input with badge-style tags. The candidate types a location (e.g. "San Francisco, CA"), presses Enter, and it appears as a removable badge. Multiple locations can be added.

When it appears: During the your_dream_job VP interview. The system prompt instructs the agent to call show_preference_form("location"), but this is not enforced by any deterministic flow step — unlike all other HITL widgets. The LLM may choose to ask about location conversationally instead, call save_onboarding_preferences() directly with parsed free-text, or complete the VP without ever triggering the widget. This is why the form rarely appears in practice and typically requires the debug menu to manually inject.

What the candidate does:

Type one or more preferred locations, adding each as a badge
Click "Submit" to send them, or "Skip" to move on without preferences

What happens after submission:

Submitted: the values are sent as a natural-language chat message (e.g. "San Francisco, CA; New York, NY"). The agent parses them and calls save_onboarding_preferences(location=[...]).
Skipped: a system "action" message is sent, and the agent continues without location data.

Key facts:

Blocking: no. Unlike the other widgets, preference_form is not in isBlockingHitlAction, and show_preference_form.py emits no blocking flag — so the chat input stays enabled. The candidate can use the form or just answer in chat.
Purely LLM-driven — no deterministic gating. The agent's system prompt suggests it, but nothing in the flow (onboarding_flow.py or api/onboarding.py) forces the tool to fire. There is no STEP_AWAITING_PREFERENCES or equivalent.
Locations use Google Places-style free text, not a constrained dropdown
The form formerly included an "interested companies" field (now handled conversationally)

Reconciliation (LinkedIn vs Resume conflicts)

What it looks like: A multi-step widget showing one conflict at a time with a step counter (e.g. "2 of 4") and a row of clickable per-conflict progress bars. Each step shows:

The field in conflict (job title, employment date, or quantified metric)
Source A (LinkedIn) with its value, as a clickable option
Source B (uploaded resume) with its value, as a clickable option
A "Something else" free-text input for typing a custom value
A "I'm not sure — skip this one" button (appears when neither source has a concrete value)

Back and Next buttons allow reviewing previous conflicts before submitting.

When it appears: After the agent ingests both the candidate's LinkedIn profile and any uploaded documents. The agent runs an LLM analysis comparing the two sources. If conflicts are found in job titles, employment dates, or quantified metrics/achievements, the reconciliation widget appears. The flow parks at STEP_AWAITING_CONFLICT_RESOLUTION.

What the candidate does:

For each conflict, picks one source, types their own value (the "Something else…" input), or skips
Uses Back/Next to review all conflicts
Clicks the primary button — labeled "Next" mid-list, "Confirm" on the final conflict — to submit

What happens after submission:

A structured payload is built with provenance: each resolution records {field, value, source} where source is "linkedin", "resume", "manual", or "skipped"
The server clears only the resolved items (per-item accounting via stable IDs); any unsubmitted conflicts remain as partially_resolved
A system message is injected with provenance-labeled correction lines (e.g. "confirmed the LinkedIn value")
Corrections are threaded into every subsequent VP interview prompt — the first VP opener falls back to a neutral scripted opener to avoid restating a just-corrected fact
The flow advances to STEP_INTERVIEW_PREP and the first VP interview begins

Key facts:

Blocking: yes
Maximum 6 conflicts shown (MAX_RECONCILIATION_ITEMS)
Conflicts cover only job titles, employment dates, and quantified metrics — explicitly NOT names, companies, locations, education, skills
The LLM is only run once (to detect conflicts). Resolution is a deterministic code path — the LLM is not re-run after the candidate submits
Corrections are session-scoped (persist within the current chat, not across sessions)

Confirmation (Networking Opt-In)

What it looks like: A simple dialog titled "Peer networking" with a description ("If you opt in, we'll introduce you to a few other candidates with similar goals over email and LinkedIn. It's separate from your job matches, and you can opt out anytime.") and two buttons: "Yes" (id opt_in) and "No" (id skip).

When it appears: During the your_interests VP interview. The agent calls the show_networking_opt_in() tool.

What the candidate does:

Clicks "Yes" or "No"

What happens after submission:

Client-side (write-first): calls PATCH /api/candidates (via updateCandidateFields) to set either networking_opt_in_at or networking_skipped_at (clearing the other) on the candidate record. On failure the widget stays open for retry.
A local receipt bubble ("Networking opted in/skipped") is added, and a system message is sent to the agent: The user selected "Yes"/"No" for the networking_opt_in confirmation. Already saved — do not re-save.
The conversation continues past networking. There is no deterministic server handler for this confirmation — the agent (LLM) simply reads the system message on its next turn.

Key facts:

Blocking: yes
This is a product decision gate, not a data-conflict gate
The opt-in status is persisted to the candidate's profile, not just the chat transcript

Confirmation (Wrong Identity Upload)

What it looks like: A dialog with a warning icon (⚠️) and a message like: "Before I use resume.pdf, I need to double-check something: it looks like it belongs to 'Jane Smith', but I have you as 'John Doe'." Two buttons: "Delete it" and "Keep it." (The dialog's own title is "This document may not be yours".)

When it appears: After uploading a document during the background intake phase. Every upload is identity-checked against the candidate's known name (from the database, LinkedIn, and JWT). If the document header contains a completely different name, the check returns "mismatch". The document is quarantined (status forced to "pending", excluded from analysis), and this confirmation dialog blocks further progress.

What the candidate does:

Delete it — the document is removed. A 7-step cleanup runs: KB deletion, context tombstoning, chip flag clearing, state update, progress replay (re-folds the message stream without the removed document), history cache invalidation, and a visible receipt message. If no documents remain, the candidate is re-prompted to upload or skip.
Keep it — the document is accepted. Its identity status is set to "override" (never re-flagged), and the found name is added to identity_aliases (handles maiden/married names, preferred names). A system note confirms the override. Progress replays to include the previously quarantined document.

What happens after submission:

The widget shows "Deleting…" or "Keeping…" while the server processes
On success, a receipt bubble appears confirming the action
The flow returns to the verification beat and may surface the next mismatch or continue to analysis

Key facts:

Blocking: yes
The identity check has a deterministic pre-pass: if the known name (≥2 tokens) appears verbatim in the first 300 characters of the document, it's a match with zero LLM calls. Only if the pre-pass fails is an LLM consulted
Nicknames, initials, middle names, and spelling/transliteration variants are classified "match" (the LLM prompt treats trivial variants as the same person) — so they don't block. Only a clearly different name (a different surname counts) is a "mismatch"
Any failure (LLM timeout, parse error) degrades to "uncertain" — the check never blocks on a transient error
This check currently fires only during the background intake phase (before VP interviews); mid-VP uploads run the check but don't surface the dialog (follow-up work)

Companies Added (persistent chip list)

What it looks like: A persistent list of company chips showing every company the candidate has expressed interest in. Each chip shows the company name. On hover, an X (remove) and a dropdown arrow (swap for an alternative match) appear.

When it appears: Whenever the agent calls add_interested_companies(company_names=[...]). The widget shows the candidate's full current list, not just the batch just added. It stays visible throughout the conversation — it's a standing receipt, not a question.

What the candidate does:

Remove a company (hover → click X) — silent background mutation, no chat turn, no LLM involvement
Swap a company (hover → click dropdown → pick alternative) — replaces with a different directory match for the same name

What happens after submission:

Removal calls a dedicated API endpoint that removes the company server-side and returns the authoritative remaining list
Swap works the same way — replaces one chip, returns the new full list
These are background mutations: the chat input remains enabled and the conversation continues uninterrupted
Only the latest companies_added widget stays interactive; earlier ones are marked completed

Key facts:

Blocking: no — it's persistent, meaning it stays visible but doesn't disable input
De-duplication: old companies_added widgets are marked hitlCompleted so only the latest one is interactive
The agent picks up fresh state on the next message (it re-reads the candidate's preferences)

The onboarding flow has distinct phases. HITL widgets appear at specific steps:

1. Greeting + initial exchange (no widgets)

2. VP Ranking widget
   └─ Candidate ranks 5 value props → agent advances to LinkedIn intake

3. LinkedIn ingestion (no widgets)
   └─ Agent summarizes LinkedIn profile

4. Document upload (background intake)
   ├─ Identity Check (if wrong-identity upload)
   │   └─ Confirmation widget: Delete or Keep
   │
   └─ Reconciliation widget (if LinkedIn/resume conflicts found)
       └─ Candidate resolves each conflict → agent advances to VP interviews

5. VP Interviews (your_story, your_wins, your_dream_job, your_skills, your_interests)
   ├─ Preference Form widget (location, during your_dream_job)
   │   └─ LLM-driven only — no deterministic gating; rarely appears in practice
   │
   ├─ Companies Added widget (when candidate mentions companies)
   │   └─ Silent background mutations via chip remove/swap
   │
   └─ Confirmation widget (networking opt-in, during your_interests)
       └─ Candidate opts in or out

6. Interview complete

How HITL works technically

Server → Client: SSE events

The agent emits HITL actions through the SSE stream as JSON events:

{
  "type": "hitl_action",
  "action": "reconciliation",
  "request_id": "hitl_9f2a1c4b7e0d",
  "params": {
    "items": [
      {
        "id": "rec_a1b2c3d4",
        "field": "job_title",
        "source_a": { "origin": "linkedin", "value": "VP of Product" },
        "source_b": { "origin": "resume", "value": "Senior Product Manager" }
      }
    ]
  }
}

The request_id is a stable identifier used for deduplication, persistence, and the client's "mark this completed" flow.

Client: HitlRenderer dispatch

In message-list.tsx, the HitlRenderer component matches action to a component:

`action` value	Component
`"vp_ranking"`	`VpRankingWidget`
`"preference_form"`	`PreferenceFormWidget`
`"reconciliation"`	`ReconciliationWidget`
`"confirmation"`	`ConfirmationWidget`
`"companies_added"`	`CompaniesAddedWidget`

Client → Server: Submission

Submissions go through useOnboardingChat.ts → handleHitlSubmit():

The hook builds a structured payload specific to the widget type (rankings, resolutions, confirmation choice, etc.)
It calls streamChat() with that payload. The message type varies: reconciliation and the wrong-identity confirmation use system type "action" (so the server routes them deterministically); VP ranking rides the ranking_submission field with role "system"; preference form sends a plain user message; networking sends a role-"system" message after its client-side PATCH has landed.
On success, the widget is marked completed by request_id and (for reconciliation/confirmation) the persisted HITL is best-effort deleted via DELETE /api/conversations/{cid}/hitl
On failure, the widget rolls back and remains interactive for retry

Server: Turn handling

HITL submissions are orchestrated by the chat endpoint in api/index.py, which first calls handle_onboarding_turn (in api/onboarding.py) and then dispatches to the resolution helpers based on the resulting step and payload:

Step / payload	Where	Handler
`STEP_AWAITING_RANKING` (`ranking_submission`)	`handle_onboarding_turn`	Validates 5 unique VP ids, advances to LinkedIn intake
`STEP_AWAITING_CONFLICT_RESOLUTION` (`reconciliation_resolutions`)	`api/index.py` → `prepare_conflict_resolution_turn()`	Per-item accounting, correction injection, advance to first VP
`STEP_AWAITING_IDENTITY_RESOLUTION` (`identity_resolution`)	`api/index.py` → `prepare_identity_resolution_turn()`	Delete or override

handle_onboarding_turn itself also handles document uploads (running the identity checks and routing to verification), a pasted LinkedIn URL, and the LLM-export decision — but not the conflict/identity resolutions, which are dispatched by api/index.py after it returns.

The networking confirmation has no deterministic server branch at all: there is no confirmation field on ChatRequest (the HITL payload fields are ranking_submission, reconciliation_resolutions, and identity_resolution). The client writes the decision via PATCH /api/candidates and the agent reads a natural-language system message on its next turn.

Blocking vs persistent widgets

Widget	Blocking?	Input disabled?	Persisted on reload?
VP Ranking	Yes	Yes	Yes
Preference Form	No	No	Yes
Reconciliation	Yes	Yes	Yes
Confirmation (networking)	Yes	Yes	Yes
Confirmation (wrong identity)	Yes	Yes	Yes
Companies Added	No (persistent)	No	Yes (latest only)

Blocking widgets disable the chat input bar. The candidate must resolve the widget before continuing the conversation. This is enforced by setInputDisabled(true) in the hook when the widget flushes. Whether a widget blocks is decided by isBlockingHitlAction(action, params): reconciliation always blocks; vp_ranking and confirmation block only when the server sends params.blocking: true (it does for VP ranking, networking, and wrong-identity); preference_form and companies_added never block.

Persistent widgets (companies_added) stay visible as standing receipts. They don't block input. Old instances are marked hitlCompleted so only the latest one responds to interactions.

Reload behavior

If the candidate refreshes the page while a widget is active:

On mount, the client calls GET /api/conversations/{cid}/hitl to fetch any pending HITL actions
The widget is restored only if the current onboarding step matches the step it was created for — a reconciliation widget at interview_prep is stale and skipped
For blocking widgets, inputDisabled is re-set to true
For persistent widgets, only the latest companies_added is re-activated

Key files

Repo	File	Role
candidate	`src/app/briefing/_components/message-list.tsx`	`HitlRenderer` — dispatches action type to widget component
candidate	`src/app/briefing/_components/vp-ranking-widget.tsx`	VP ranking sortable card list
candidate	`src/app/briefing/_components/preference-form-widget.tsx`	Location preference input with badge tags
candidate	`src/app/briefing/_components/reconciliation-widget.tsx`	Multi-step conflict resolution (Back/Next, progressive disclosure)
candidate	`src/app/briefing/_components/confirmation-widget.tsx`	Generic confirmation dialog (networking + wrong-identity)
candidate	`src/app/briefing/_components/companies-added-widget.tsx`	Persistent company chip list with remove/swap
candidate	`src/app/briefing/_hooks/useOnboardingChat.ts`	`handleHitlSubmit`, optimistic UI, reload restoration
candidate	`src/types/onboarding-chat.d.ts`	`ReconciliationItem`, `ReconciliationResolution`, type definitions
qwestly-agent	`api/index.py`	Chat endpoint — orchestrates HITL dispatch (calls `handle_onboarding_turn`, then the resolution helpers)
qwestly-agent	`api/onboarding.py`	`handle_onboarding_turn` (ranking/uploads/LinkedIn/export), `prepare_conflict_resolution_turn`, `prepare_identity_resolution_turn`, identity confirmation builder
qwestly-agent	`lib/onboarding_flow.py`	Step constants, `phase_for_step`, `MAX_RECONCILIATION_ITEMS`, `identity_aliases` default
qwestly-agent	`tools/onboarding/complete_open_qa.py`	`VP_RANKING_CARDS` + emits the VP ranking HITL (advances open_qa → awaiting_ranking)
qwestly-agent	`tools/onboarding/analyze_background_sources.py`	Conflict detection, identity gate, reconciliation HITL builder
qwestly-agent	`tools/onboarding/summarize_known_linkedin.py`	LinkedIn snapshot builder (input to conflict detection)
qwestly-agent	`tools/onboarding/show_preference_form.py`	Preference form HITL generation
qwestly-agent	`tools/onboarding/show_networking_opt_in.py`	Networking opt-in HITL generation
qwestly-agent	`tools/onboarding/delete_rejected_upload.py`	7-step deletion + identity override
qwestly-agent	`lib/identity_check.py`	Upload identity verification

_private/qwestly-docs/Features/onboarding-human-in-the-loop/onboarding-hitl-overview.md

Onboarding Human-in-the-Loop (HITL) Widgets (updated 7/24/2026)

Table of Contents

How HITL works

Widget catalog

VP Ranking

Preference Form (Location)

Reconciliation (LinkedIn vs Resume conflicts)

Confirmation (Networking Opt-In)

Confirmation (Wrong Identity Upload)

Companies Added (persistent chip list)

When each widget appears (flow order)

How HITL works technically

Server → Client: SSE events

Client: HitlRenderer dispatch

Client → Server: Submission

Server: Turn handling

Blocking vs persistent widgets

Reload behavior

Key files