Back to demoPortfolio home

Case study

Support Ticket Triage Console

End-to-end demo (local build recording)

Evaluation

In progress

Evaluation in progress - validated numbers coming soon.

Evaluation methodology

Synthetic tickets with sarcasm, forwards, staging vs production wording, and newsletter noise. Labels reflect human judgment, not rule output. Held-out splits are reported separately from dev tuning.

Fixtures: eval/fixtures/tickets-labeled.json.

In-house regression: npm run eval (internal splits in eval/results/latest.json, not published on the site).

Problem

Support queues mix production outages, billing disputes, how-to questions, and marketing noise. Without a first-pass sort, SLA risk hides behind low-impact tickets.

Who it is for

SaaS support and ops teams who want a human-in-the-loop console: machine suggests urgency, intent, and reply copy; a human approves before send.

What it does

Visitors triage a sample queue. The console surfaces urgency and intent, internal notes for escalations, editable reply drafts with variants, and owner-gated paste mode plus Markdown or JSON export for handoffs.

How I built it

Same pattern as inbox triage: local embeddings and ensemble classifiers in TypeScript, client-side for instant feedback and zero API keys in public demo mode.

Tradeoff: urgency is ordinal but trained with multiple heads; high vs normal separation is still the weakest blind-test slice.

Error analysis (in-house held-out)

Internal regression only (not published as headline metrics).

Representative urgency or intent misses from the in-house eval set. Common themes: newsletter text that mentions outages, staging vs production, and feature vs account how-to overlap.

  • tkt-003: urgency high→high, intent technical→account
  • tkt-005: urgency normal→normal, intent account→technical
  • tkt-009: urgency high→high, intent technical→account
  • tkt-012: urgency normal→critical, intent technical→technical
  • tkt-014: urgency high→critical, intent technical→account

Next steps: environment tag parsing, separate marketing detector from incident detector, and intent tie-breakers when billing and technical signals co-occur.

Limitations and next steps

  • No ticket system webhook integration in the public demo.
  • Urgency ordinal errors of two or more levels still occur.
  • Feature requests can be mislabeled as technical issues.

Architecture

[Sample / pasted tickets]
        │
        ▼
┌────────────────────┐
│ Urgency + intent   │  scored keyword rules (client)
└─────────┬──────────┘
          ▼
┌────────────────────┐     ┌──────────────────┐
│ Suggested reply    │────▶│ Human edit/copy  │
└─────────┬──────────┘     └──────────────────┘
          ▼
┌────────────────────┐
│ Export MD / JSON   │  (owner unlock)
└────────────────────┘