_private/qwestly-private-docs/SOC2/incidents/2026-05-21-openai-key-compromise.md

Table of Contents

Incident Report — 2026-05-21

OpenAI API Key Compromise & VAPI Outage

Date: 2026-05-21 Reported by: Dominick Pham Severity: High (production services affected)

Incident 1: OpenAI API Key Leak

Summary

OpenAI notified us via email that one or more API keys associated with our account had been leaked/compromised. OpenAI did not disclose how the keys were compromised. They disabled the affected keys.

Impact

  • All services relying on the compromised keys stopped functioning
  • Production apps and staging environments were impacted

Resolution

  • Created new OpenAI API keys (separate keys for testing and production environments)
  • Redeployed all production applications and staging environments with the new keys
  • Services are now restored

Follow-up Actions

  • Investigate how keys were leaked (potential source: see Incident 2)
  • Rotate keys for any dev services also using the compromised key (e.g., LangSmith)
  • Review key management practices — consider vault/secrets manager
  • Notify team members using dev keys to update

Incident 2: VAPI Service Outage

Summary

VAPI is experiencing a full outage affecting both their dashboard and API. The dashboard is failing to load due to a CORS (Cross-Origin Resource Sharing) error — their frontend cannot reach their own backend. Their status page (slow but accessible) confirms both services are down.

This prevents accessing the dashboard to update our API keys, and all voice chat calls fail at the API layer before reaching key validation.

Impact

  • VAPI dashboard inaccessible (cannot manage settings or update keys)
  • Voice chat functionality is completely non-functional for all users
  • Call failures occur at the API layer, not at key validation

Status

  • Ongoing — VAPI status page confirms both dashboard and API are down
  • Root cause unknown; possible correlation with Incident 1 (speculative — VAPI may have experienced their own security incident that exposed our keys)

Follow-up Actions

  • Monitor VAPI status page and communications
  • Update OpenAI API key in VAPI once dashboard access is restored
  • Assess whether VAPI compromise may have been the source of our OpenAI key leak
  • Consider alternative voice API providers if outage is prolonged

Timeline

Time (EDT) Event
10:57 Received email from OpenAI: API keys compromised & disabled
10:57–12:20 Created new API keys; redeployed all production apps & staging environments
12:20 All services back online with new keys
~12:35 Attempted to update key in VAPI dashboard — dashboard fails to load (CORS error)
12:45 Checked VAPI status page (slow); confirms both dashboard and API are down
12:50 Incident documented; VAPI still down