feat(monitoring): multi-model dashboard v2.1 — tabs, per-model records, voice playback + MinIO dual-write

Problem: the dashboard was Jev-centric; NLU/arbiter/fallback DeepSeek, STT
providers and the audio-enhancer were barely visible, and stored texts/voices
had no real access path.

Bot (additive; reply path untouched):
- nlu.result now records latency_ms + the DeepSeek usage tokens
- matching.arbitration/fallback.result fill the model + latency_ms columns
  (payload unchanged; SQL coalesces column/payload so old rows still count)
- storage.py: MinIO dual-write layer — every voice (raw + denoised) and user
  attachment is mirrored fire-and-forget into bucket bot-media; the blocking
  minio SDK runs via asyncio.to_thread (senior-engineer review) so the event
  loop is never stalled; empty MINIO_* env = layer off; bucket auto-created
- requirements: minio>=7.2

Backend (admin-gated, read-only):
- GET /bot-metrics/models — one record per model (jev, deepseek-nlu,
  deepseek-arbiter, deepseek-fallback, audio-enhancer + dynamic per-provider
  STT) with calls/ok-rate/p50/p95/tokens/last-seen + version-drift vs the pin
- GET /bot-metrics/models/{id}/daily — per-model daily calls/fails/tokens/p50
- GET /bot-metrics/stages — every stage tagged with its Persian pipeline
  section (صوت/درک پیام/تطبیق/ابهام و انتخاب/پیش‌نویس)
- /traces now carries input_kind + a 120-char text excerpt (consensus →
  input.text → first STT transcript)
- media serving: POST /media-url mints a short-lived HMAC URL (audio/a tags
  cannot send Authorization); GET /media streams from MinIO bot-media with
  the read-only ./bot/data mount as fallback for pre-MinIO files; hard ref
  validation (voices|attachments only, no .., extension whitelist)
- router exempted from the global Bearer dependency (files.py pattern) —
  every endpoint keeps its own require_role('admin')

Frontend /bot-monitor (RTL, recharts):
- four tabs: نمای کلی / مدل‌ها (records table + per-model expandable daily
  charts: calls/fails, tokens, p50) / بخش‌ها (sections grouped) / پیام‌ها
- message drawer rebuilt: inline <audio controls> for raw + denoised voice,
  download links, every STT transcript + consensus side-by-side, user
  attachments, then the event timeline
- terminology: «ترس» → «پرونده‌ی پیام» with tooltips + a one-line legend

Grafana: bot-observability gains a «کارکرد مدل‌ها» table (stage→model
mapping for legacy rows); the explorer gains a message text column.

Seed: real (tiny) ogg files written + mirrored to MinIO, NLU usage tokens,
column-level model/latency on matching events.

Verified locally: /models returns all 7 records incl. per-STT and NLU tokens;
media 200 via signed URL (MinIO) and 401 without; ../ and .py rejected (400);
disk fallback serves a legacy file; pm-role 403 on every new endpoint;
bucket auto-created with 14 mirrored objects; tsc + vite build clean;
Grafana re-provisions 24 panels.
This commit is contained in:
2026-10-25 22:02:40 +03:30
parent 0adadd2cf8
commit cb621ca5d0
18 changed files with 2028 additions and 543 deletions
+16 -8
View File
@@ -8,6 +8,7 @@ import hashlib
import json
import logging
import re
import time
import httpx
@@ -220,6 +221,7 @@ async def understand(settings: Settings, text: str | list[dict], draft_summary:
})
try:
t0 = time.perf_counter()
async with httpx.AsyncClient(timeout=settings.LLM_TIMEOUT_SECONDS) as client:
resp = await client.post(
f"{settings.AI_BASE_URL.rstrip('/')}/chat/completions",
@@ -234,29 +236,35 @@ async def understand(settings: Settings, text: str | list[dict], draft_summary:
"response_format": {"type": "json_object"},
},
)
latency_ms = int((time.perf_counter() - t0) * 1000)
if resp.status_code != 200:
logger.error("LLM HTTP %s: %s", resp.status_code, resp.text[:300])
await obs.event(trace_id, "nlu.result", ok=False, model=settings.LLM_MODEL,
latency_ms=latency_ms,
payload={"error": f"HTTP {resp.status_code}", "body": resp.text[:300]})
raise NLUFailureError(f"HTTP {resp.status_code}")
content = ((resp.json() or {}).get("choices") or [{}])[0].get("message", {}).get("content", "")
resp_json = resp.json() or {}
content = ((resp_json.get("choices") or [{}])[0].get("message", {}) or {}).get("content", "")
parsed = _extract_json(content)
if not parsed or parsed.get("intent") not in INTENTS:
logger.warning("unparseable LLM output: %r", content[:200])
await obs.event(trace_id, "nlu.result", ok=False, model=settings.LLM_MODEL,
latency_ms=latency_ms,
payload={"error": "unparseable", "raw": content[:500]})
raise NLUFailureError("unparseable LLM output")
parsed.setdefault("patch", {})
parsed["entity_mentions"] = sanitize_mentions(parsed.get("entity_mentions"))
if not parsed.get("consensus_text"):
parsed["consensus_text"] = first_text
await obs.event(trace_id, "nlu.result", ok=True, model=settings.LLM_MODEL, payload={
"intent": parsed.get("intent"),
"confidence": parsed.get("confidence"),
"patch": parsed.get("patch"),
"consensus_text": parsed.get("consensus_text"),
"entity_mentions": parsed.get("entity_mentions"),
})
await obs.event(trace_id, "nlu.result", ok=True, model=settings.LLM_MODEL,
latency_ms=latency_ms, payload={
"intent": parsed.get("intent"),
"confidence": parsed.get("confidence"),
"patch": parsed.get("patch"),
"consensus_text": parsed.get("consensus_text"),
"entity_mentions": parsed.get("entity_mentions"),
"usage": resp_json.get("usage"),
})
return parsed
except Exception as exc:
if isinstance(exc, NLUFailureError):