feat(observability): phase-1 silent pipeline recorder + phase-0 Jev validation

Phase 0 — Jev validated with the real key (5 Persian scenarios, api.typesafe.ai): stale project name «منشیانه»->«منشانه» conf 0.73 (ambiguity band), STT spelling variants «مصطفوی/مستفدی/مسطفی»->«بازرگانی مصطفوی» conf 0.93, exact «دفتر» 0.88, proper _none rejection for «کاغذ A4» 0.92, and two mentions in ONE request (0.77/0.64). Exact model version to pin: jev-1.13.0. Latency 0.7-1.1s per field request; ~450-650 in / 64-133 out tokens.

Phase 1 — Observability (DB-only, zero extra LLM tokens, nothing is ever deleted):

- new tables bot_traces + bot_trace_events (idempotent DDL in db_init.py; ORM in models.py)

- new app/observability.py: _jsonable (UUID/datetime/Decimal/set/enum/dataclass-safe), trace_start, event, log_resolution, link_draft, latest_trace_for_draft, set_trace_status, span() context manager, save_voice (Path.mkdir parents+exist_ok, day folders)

- capture points wired: input.text; input.voice (+voice persisted under OBS_VOICE_DIR); stt.run + per-provider stt.result (success AND failure, latency, model); nlu.request + nlu.result with prompt_hash/prompt_version; entities.extracted; routing.decision (difflib baseline for the phase-6 A/B); ambiguity.sent; ambiguity.user_choice (buttons + free-text reply path); user_choice.pick; draft.render; draft.edit (wizard-side edits detected by the sync loop); draft.commit (quote/supplier/material/project + trace status/quote id); draft.cancel

- compose: persistent ./bot/data:/srv/bot/data volume; bot/.gitignore: data/ (runtime data never committed, never deleted)

- tests: bot/tools/test_observability.py -> 8/8 pass (jsonable cases, trace lifecycle, JSONB round-trip, draft linking, status update, save_voice day-folder mkdir, span task). Verified live: voice file lands on the host volume and is gitignored; rows + files persist across bot restart.
This commit is contained in:
2026-10-12 21:04:50 +03:30
parent 693e7511c6
commit d58b619107
16 changed files with 660 additions and 57 deletions
+45 -1
View File
@@ -5,7 +5,7 @@ container and owns its own state (links, drafts, message audit log).
"""
from datetime import datetime
from sqlalchemy import BigInteger, DateTime, Index, Integer, String, Text, func
from sqlalchemy import BigInteger, Boolean, DateTime, ForeignKey, Index, Integer, String, Text, func
from sqlalchemy.dialects.postgresql import JSONB, UUID
from sqlalchemy.orm import DeclarativeBase, Mapped, mapped_column
@@ -83,3 +83,47 @@ class BotMeta(Base):
key: Mapped[str] = mapped_column(String(64), primary_key=True)
value: Mapped[str | None] = mapped_column(Text)
updated_at: Mapped[datetime] = _timestamptz(onupdate=func.now())
class BotTrace(Base):
"""Observability: one row per incoming interaction (message) — the chain root.
Events of the whole pipeline hang off it (bot_trace_events). Linked to the
draft it produced so later button clicks can be attributed to the same chain.
"""
__tablename__ = "bot_traces"
id: Mapped[str] = _uuid_pk()
chat_id: Mapped[int] = mapped_column(BigInteger, nullable=False)
message_id: Mapped[int | None] = mapped_column(BigInteger)
kind: Mapped[str] = mapped_column(String(8), nullable=False) # text | voice
source: Mapped[str | None] = mapped_column(String(16)) # bale | wizard | api
draft_id: Mapped[str | None] = mapped_column(UUID(as_uuid=False))
draft_kind: Mapped[str | None] = mapped_column(String(16))
committed_quote_id: Mapped[str | None] = mapped_column(UUID(as_uuid=False))
status: Mapped[str] = mapped_column(String(16), nullable=False, default="open") # open|committed|cancelled
created_at: Mapped[datetime] = _timestamptz()
updated_at: Mapped[datetime] = _timestamptz(onupdate=func.now())
class BotTraceEvent(Base):
"""Observability: append-only pipeline event log (payload schemas by `stage`).
Written ONLY through app.observability (fire-and-forget, _jsonable-safe).
Nothing is ever deleted from here by the bot.
"""
__tablename__ = "bot_trace_events"
id: Mapped[int] = mapped_column(BigInteger, primary_key=True, autoincrement=True)
trace_id: Mapped[str] = mapped_column(
UUID(as_uuid=False), ForeignKey("bot_traces.id", ondelete="CASCADE"), nullable=False
)
stage: Mapped[str] = mapped_column(String(32), nullable=False)
ok: Mapped[bool] = mapped_column(Boolean, nullable=False, default=True)
latency_ms: Mapped[int | None] = mapped_column(Integer)
model: Mapped[str | None] = mapped_column(String(64))
field: Mapped[str | None] = mapped_column(String(32))
payload: Mapped[dict] = mapped_column(JSONB, nullable=False, default=dict)
created_at: Mapped[datetime] = _timestamptz()