qwestly-workspace/docs/agent-conversations-schema-redesign.md

Table of Contents

Agent Conversations Schema Redesign

Problem

The current agent_conversations collection stores turns (not messages). Each document has user_message + response_parts + tool_calls + response_text. This has several issues:

  1. Not standard — OpenAI/Anthropic use {role, content} per message. Our "turn" model conflates user input, assistant output, tool calls, tool results, and file uploads into a single document.
  2. System messages are invisible — is_system: true stores user_message: "", losing the system trigger content. System-initialized turns produce no visible message in the transcript.
  3. Tool results are misordered — SSE streaming emits tool_results after the agent run completes, so they're at the end of response_parts. Reordering happens at read time (fixed in prior PR), but this is a symptom of the turn model being the wrong abstraction.
  4. HITL is separate — session_hitl collection stores pending file-upload prompts separately from message history. Undiscoverable, hard to reason about.
  5. No unique tool_call_ids — tool calls and results are correlated by tool_name alone. Same-tool-name calls in one turn overwrite each other.

Target Schema

New: agent_conversations (one doc per conversation)

Replaces the current agent_conversations collection. Stores high-level metadata only.

interface IAgentConversation {
  conversation_id: string;   // UUID
  user_id: string;
  name: string | null;       // display name (auto-generated from first message or LLM)
  created_at: Date;
  updated_at: Date;          // last message time
  turn_count: number;        // denormalized count for sidebar
  last_message_preview: string; // truncated preview of last user/assistant message
}

New: agent_messages (one doc per message)

Each message follows the OpenAI format (plus extensions for HITL, error, upload metadata).

interface IAgentMessage {
  conversation_id: string;   // UUID — groups messages into a conversation
  role: "system" | "user" | "assistant" | "tool";
  content: string | null;
  // Present on assistant messages that include tool calls:
  tool_calls?: {
    id: string;              // unique tool_call_id (from Pydantic AI)
    type: "function";
    function: {
      name: string;
      arguments: string;     // JSON string of args
    };
  }[] | null;
  // Present on tool messages (links back to the assistant tool_call):
  tool_call_id?: string | null;
  // Present on assistant messages that trigger a HITL widget
  // (file_upload, future: confirmation, form):
  hitl_action?: {
    action: string;
    params: Record<string, any>;
    request_id: string;
  } | null;
  // Present on any message that represents an error:
  error?: { message: string } | null;
  // Present on user messages that carry a file upload:
  upload?: {
    filename: string;
    file_type: string;
    size: number;
    parsed_text?: string;
  } | null;
  // Timing + cost metadata:
  created_at: Date;
  token_usage?: { input_tokens: number; output_tokens: number } | null;
  latency_ms?: number | null;
}

Deprecate: session_hitl

HITL state moves into agent_messages.hitl_action. Pending HITL is determined by querying for the most recent message with a hitl_action that hasn't been resolved yet. Resolved state can be tracked via a hitl_resolved_at field in the conversation doc or handled by the client.

Message Mapping

Event / Source Role Fields
System trigger (is_system: true) system content: request.message
User text input user content: "..."
User file upload user content: "I've uploaded...", upload: {...}
Assistant text response assistant content: "Here's your grade..."
Assistant tool call assistant content: null, tool_calls: [{id, function: {name, arguments}}]
Tool result tool tool_call_id: "...", content: "{...json...}"
HITL action (file upload prompt) assistant content: "Upload your resume...", hitl_action: {action: "file_upload", ...}
Error assistant content: null, error: {message: "..."}

Example conversation as stored messages

msg 1:  { role: "system",    content: "" }
msg 2:  { role: "assistant", content: "Hi there! Welcome...", token_usage: {...} }
msg 3:  { role: "user",      content: "dominickpham" }
msg 4:  { role: "assistant", content: null, tool_calls: [{ id: "tc1", function: { name: "ingest_linkedin_profile", arguments: "..." } }] }
msg 5:  { role: "tool",      tool_call_id: "tc1", content: "{ ...linkedin data... }" }
msg 6:  { role: "assistant", content: "Got your profile. Let me grade it..." }
msg 7:  { role: "assistant", content: null, tool_calls: [{ id: "tc2", function: { name: "grade_linkedin_profile", arguments: "{}" } }] }
msg 8:  { role: "tool",      tool_call_id: "tc2", content: "{ ...grade data... }" }
msg 9:  { role: "assistant", content: "Here's your grade: 5.7/10..." }
msg 10: { role: "assistant", content: "Upload your resume...", hitl_action: { action: "file_upload", request_id: "h1", params: {...} } }
msg 11: { role: "user",      content: "I've uploaded Career notes.pdf", upload: { filename: "Career notes.pdf", file_type: "pdf", size: 59690, parsed_text: "..." } }
msg 12: { role: "assistant", content: "Got your Career notes. Gap check: ..." }

Implementation Plan

Phase 1: Types & Schema (no data migration)

  1. Create TypeScript types in candidate/src/types/agent-message.d.ts:

    • IAgentMessage
    • IAgentConversation
    • IToolCall, IHitlAction, IFileUpload

  2. Update Python schemas in qwestly-agent/lib/schemas.py:

    • Add tool_call_id to ToolCallEvent and ToolResultEvent
    • Update ChatRequest (already has upload_result, is_system)

  3. Update SSE streaming in qwestly-agent/agents/base.py:

    • Read part.tool_call_id from Pydantic AI's ToolCallPart and pass it through SSE events
    • Read part.tool_call_id from ToolReturnPart and pass it through SSE result events
    • Update _collect_tool_results to track by tool_call_id instead of deduplicating by tool_name

  4. Update SSE ingest in qwestly-agent/lib/sse_ingest.py:

    • Store tool_call_id in both tool_call and tool_result entries

Phase 2: New logging layer

  1. Create AgentMessageLogger in qwestly-agent/lib/logging.py:

    • async def log_message(msg: AgentMessage) — inserts one doc into agent_messages
    • async def log_messages(messages: list[AgentMessage]) — bulk insert a turn's messages
    • async def upsert_conversation(conv_id, user_id, ...) — upserts conversation metadata

  2. Update chat endpoint (qwestly-agent/api/index.py):

    • After streaming completes, convert response_parts + user_message into a list of AgentMessage docs
    • Write messages individually to agent_messages
    • Upsert conversation metadata to agent_conversations
    • Remove old log_turn() call

  3. Update test routes (qwestly-agent/api/test_routes.py):

    • Same logging changes

Phase 3: Read paths

  1. Update conversation list (list_sessions in logging.py):

    • Query agent_conversations for metadata (simple find + sort)
    • Remove aggregation pipeline against turns

  2. Update transcript endpoint (get_conversation_transcript):

    • Query agent_messages.find({conversation_id}).sort({created_at: 1})
    • Messages are already in correct order (no reordering needed!)
    • Remove _interleave_tool_results and backfill logic — no longer needed
    • Map agent_messages docs to the transcript response format

  3. Update get_session — query agent_messages instead of turns

  4. Update delete_session — delete from both agent_messages and agent_conversations

  5. Update name_session — upsert into agent_conversations

  6. Update count_turns — read turn_count from agent_conversations (denormalized) or count agent_messages where role is user or system

  7. Update candidate admin route (src/app/api/admin/agent/conversations/[id]/route.ts):

  • Query agent_messages directly (simple find + sort)
  • Remove interleaveToolResults — no longer needed
  • Map to the response format

Phase 4: HITL consolidation

  1. Remove session_hitl collection:
  • HITL pending state is determined by finding the last hitl_action message without a corresponding resolved marker
  • Add hitl_resolved_at field to agent_conversations or track via a special tool message
  1. Update HITL API routes to read/write from agent_messages:
  • get_pending_hitl → find last message with hitl_action where not yet resolved
  • set_pending_hitl → already handled by logging the assistant message with hitl_action
  • clear_pending_hitl → insert a resolution marker or update conversation doc

Phase 5: Data migration

  1. Write migration script (Python, run manually in dev):
For each conversation_id group in old agent_conversations:
  1. Create agent_conversations doc with metadata
  2. For each turn (sorted by created_at):
    a. If user_message is non-empty → insert {role: "user", content}
       If user_message is empty → insert {role: "system", content: ""}
    b. For each part in response_parts:
       - text → {role: "assistant", content}
       - tool_call → {role: "assistant", content: null, tool_calls: [{id: new_uuid, function}]}
       - tool_result → {role: "tool", tool_call_id: matched_call_id, content: serialized_result}
       - file_upload → {role: "user", content: "File uploaded", upload: {...}}
  1. Verify migration — spot-check conversations render identically in the admin UI

Phase 6: Cleanup & docs

  1. Update candidate types — align ChatMessage interface to include new roles/fields

  2. Update admin transcript renderer — handle system role, updated tool message format

  3. Update live chat client — minor adjustments for tool_call_id in SSE events

  4. Update docs — message-format.md, CLAUDE.md, AGENTS.md

  5. Remove deprecated code — old log_turn, update_turn, _interleave_tool_results, backfill logic

  6. Delete claim-migration references — update user_id migration in capture_email.py to target agent_messages

Migration Compatibility

  • New code writes to both old and new collections during a transition period (controlled by env flag)
  • Old agent_conversations renamed to agent_conversations_deprecated after migration verified
  • Admin route reads from new collection first, falls back to old for unmigrated data

Open Questions

  1. HITL resolved state — How do we mark a HITL action as resolved? Options:

    • (A) hitl_resolved_at field on the conversation doc
    • (B) A special {role: "tool", content: "hitl_resolved", tool_call_id: request_id} message
    • (C) Rely on the next user message with upload metadata being the resolution → Leaning (C) — the file upload user message IS the resolution. No extra state needed.

  2. token_usage placement — Should token_usage be on every message, only on assistant messages, or only on the conversation doc? → On assistant messages only. System/user/tool messages don't consume tokens directly.

  3. Auto-name triggers — Currently name_session updates ALL turns. With per-message storage, name only exists on the conversation doc. Simpler.

  4. message_history_json — Currently stored per-turn for Pydantic AI cross-turn context. With per-message storage, we can reconstruct this from agent_messages. But Pydantic AI needs ModelMessage objects, not plain dicts. We'll keep using _mh_cache (in-memory) and the last document's message_history_json during migration, then switch to reconstructing from agent_messages.

  5. capture_email.py migration — Currently runs update_many({user_id: from}, {$set: {user_id: to}}) on agent_conversations. Needs to update both agent_conversations and agent_messages.

  6. provisioned-account.ts cascade delete — Currently runs deleteMany({user_id}) on agent_conversations. Needs to run on both collections.

Files Changed (estimated)

qwestly-agent

File Change
lib/logging.py Major: add AgentMessageLogger, modify ConversationLogger
lib/schemas.py Add tool_call_id fields, new AgentMessage model
lib/sse_ingest.py Store tool_call_id
agents/base.py Pass tool_call_id from Pydantic AI parts
api/index.py New logging, new transcript query, remove reorder/backfill
api/test_routes.py Same changes as index.py
tools/capture_email.py Update migration queries
docs/message-format.md Rewrite for new schema
tests/test_logging.py Update for new logger API

candidate

File Change
src/types/agent-message.d.ts New file — IAgentMessage, IAgentConversation types
src/types/chatbot_messages.d.ts Update ChatMessage to add system role
src/app/api/admin/agent/conversations/[id]/route.ts Query agent_messages, remove interleaveToolResults
src/app/agent/client.tsx Handle tool_call_id in SSE, minor type updates
src/app/agent/_components/message-bubble.tsx Handle system role rendering
src/app/admin/provision/user-initiated/client.tsx Update transcript renderer for new format
src/services/db/provisioned-account.ts Delete from both collections
src/services/db/claim-migration.ts Migrate both collections