Monitoring and Oversight
Dashboard and Triage Overview
Overview: Visibility Before Action
Monitoring ANET requires two levels of visibility: Dashboard → Operational snapshot Triage → Call-level investigation The Dashboard helps you detect...
The Dashboard: Operational Snapshot
The Dashboard provides: Recent ANET calls Call status indicators Quick navigation to Triage Access to Analytics Common status types include: Fully han...
Daily Dashboard Review
Open Dashboard. Review: Total recent calls Status distribution Look for: Sudden spikes in transfers Increase in failures Unexpected fallback transfers...
Triage: Call-Level Investigation
Triage allows you to: Filter calls by status Filter by transfer reason Filter by score range Filter by tags Review timeline events Triage is used for ...
Structured Call Review
When reviewing a call: Open call in Triage. Read transcript from start to finish. Review timeline events. Confirm: Intent identified correctly Safety ...
When to Escalate from Dashboard to Admin
Escalate when you detect: Repeated agent_failure transfers Repeated guardrail timeouts Jurisdiction routing errors Repeated missing required field CAD...
Supervisor Best Practices
Review daily during early deployment Shift to weekly trend review once stable Always review fallback transfers Always review unknown transfer reasons ...
Key Takeaways
Dashboard identifies patterns. Triage diagnoses causes. Status does not equal quality. Escalate trends, not isolated calls.
Analytics and Performance Monitoring
Overview: Analytics Supports Oversight
Analytics aggregates structured data, including: Call volume Resolution vs transfer rate Intent distribution Safety escalations Guardrail transfers Sc...
Core Metrics to Monitor
Monitor: Total ANET calls Daily and weekly trends Volume spikes Use to: Validate adoption Identify seasonal changes Detect sudden surges Tracks: Fully...
Weekly Trend Review
Open the Analytics dashboard. Set time window (e.g., last 7 days). Review: Total call volume Resolution vs transfer Safety escalations Agent_failure r...
When to Consider Configuration Changes
Consider review when: Transfer rate increases significantly Specific intent dominates unexpectedly Safety escalations spike without a clear cause Agen...
Key Takeaways
Analytics reveals patterns. Trends matter more than daily fluctuations. Escalate degradation patterns. Do not optimize metrics at the expense of safet...
Scoring, Tags, and Observability
Overview: Scoring Is a Signal, Not a Verdict
ANET assigns scores across five categories: intent_identification conversation_quality language_handling location_handling information_completeness Sc...
Transfer Reasons vs Failure Tags
Transfer Reasons describe why routing occurred. Tags describe detected patterns or issues, such as: Handling issues Missing required data Safety-relat...
What Low Scores Mean
Scores below defined thresholds: Are logged as errors Contribute to monitoring May trigger alerts A single low score is not automatically a failure. S...
Review Low-Score Calls
Filter Triage by score < threshold. Select sample calls. Review transcript. Confirm: Was score justified? Was safety involved? Was intent unclear? Ide...
What Scoring Is Not
Scoring is not: A replacement for human QA A guarantee of correctness A disciplinary system Supervisors remain the final authority.
Monitoring Best Practices
Review low-score clusters weekly Correlate low scores with transfer reasons Track changes after configuration updates Use tags to identify configurati...
Limitations
Scoring: Reflects structured model evaluation May misinterpret nuanced human speech Cannot capture all contextual factors Human review remains essenti...
Key Takeaways
Scores provide structured quality signals. Review trends, not isolated calls. Correlate scoring with transfer patterns. Maintain human oversight autho...