feat(observability): phase-1 silent pipeline recorder + phase-0 Jev validation
Phase 0 — Jev validated with the real key (5 Persian scenarios, api.typesafe.ai): stale project name «منشیانه»->«منشانه» conf 0.73 (ambiguity band), STT spelling variants «مصطفوی/مستفدی/مسطفی»->«بازرگانی مصطفوی» conf 0.93, exact «دفتر» 0.88, proper _none rejection for «کاغذ A4» 0.92, and two mentions in ONE request (0.77/0.64). Exact model version to pin: jev-1.13.0. Latency 0.7-1.1s per field request; ~450-650 in / 64-133 out tokens. Phase 1 — Observability (DB-only, zero extra LLM tokens, nothing is ever deleted): - new tables bot_traces + bot_trace_events (idempotent DDL in db_init.py; ORM in models.py) - new app/observability.py: _jsonable (UUID/datetime/Decimal/set/enum/dataclass-safe), trace_start, event, log_resolution, link_draft, latest_trace_for_draft, set_trace_status, span() context manager, save_voice (Path.mkdir parents+exist_ok, day folders) - capture points wired: input.text; input.voice (+voice persisted under OBS_VOICE_DIR); stt.run + per-provider stt.result (success AND failure, latency, model); nlu.request + nlu.result with prompt_hash/prompt_version; entities.extracted; routing.decision (difflib baseline for the phase-6 A/B); ambiguity.sent; ambiguity.user_choice (buttons + free-text reply path); user_choice.pick; draft.render; draft.edit (wizard-side edits detected by the sync loop); draft.commit (quote/supplier/material/project + trace status/quote id); draft.cancel - compose: persistent ./bot/data:/srv/bot/data volume; bot/.gitignore: data/ (runtime data never committed, never deleted) - tests: bot/tools/test_observability.py -> 8/8 pass (jsonable cases, trace lifecycle, JSONB round-trip, draft linking, status update, save_voice day-folder mkdir, span task). Verified live: voice file lands on the host volume and is gitignored; rows + files persist across bot restart.
This commit is contained in:
@@ -6,6 +6,7 @@ Data grammar (≤64 bytes): op:{arg}:{d8} / pk:{d8}:{field}:{idx} / qc:{idx
|
||||
import logging
|
||||
|
||||
from app import drafts as draft_store
|
||||
from app import observability as obs
|
||||
from app.runtime import LinkedUser
|
||||
from app.texts import CANCELLED, UNLINKED, esc
|
||||
|
||||
@@ -77,6 +78,12 @@ async def _dispatch(ctx, chat_id: int, data: str, cq: dict) -> None:
|
||||
if not target_amb:
|
||||
return
|
||||
|
||||
tid = await obs.latest_trace_for_draft(draft.id)
|
||||
await obs.event(tid, "ambiguity.user_choice", payload={
|
||||
"amb_id": amb_id, "action": action, "kind": target_amb.get("kind"),
|
||||
"title": target_amb.get("title"),
|
||||
})
|
||||
|
||||
if action == "ok":
|
||||
proposed = target_amb.get("proposed_patch") or {}
|
||||
draft_store.merge_quote_patch(payload, proposed)
|
||||
@@ -126,6 +133,9 @@ async def _dispatch(ctx, chat_id: int, data: str, cq: dict) -> None:
|
||||
return
|
||||
if arg == "no":
|
||||
await draft_store.set_status(draft.id, "cancelled")
|
||||
tid = await obs.latest_trace_for_draft(draft.id)
|
||||
await obs.event(tid, "draft.cancel", payload={"draft_id": str(draft.id), "kind": draft.kind})
|
||||
await obs.set_trace_status(tid, "cancelled")
|
||||
ctx.pending.pop(chat_id, None)
|
||||
if draft.review_message_id:
|
||||
try:
|
||||
@@ -151,8 +161,10 @@ async def _apply_pick(ctx, chat_id: int, user: LinkedUser, draft, field: str, id
|
||||
await ctx.send(chat_id, "گزینه نامعتبر است.")
|
||||
return
|
||||
choice = options[idx]
|
||||
tid = await obs.latest_trace_for_draft(draft.id)
|
||||
await obs.event(tid, "user_choice.pick", payload={"field": field, "idx": idx, "choice": choice})
|
||||
if draft.kind == draft_store.KIND_QUOTE:
|
||||
await quote.apply_pick(ctx, chat_id, user, draft, field, choice)
|
||||
await quote.apply_pick(ctx, chat_id, user, draft, field, choice, trace_id=tid)
|
||||
elif draft.kind == draft_store.KIND_SUPPLIER:
|
||||
await supplier.apply_pick(ctx, chat_id, user, draft, field, choice)
|
||||
elif draft.kind == draft_store.KIND_MATERIAL:
|
||||
|
||||
Reference in New Issue
Block a user