feat(matching): Jev entity matching phases 2-5 (per-field fan-out, variants, candidate ambiguity buttons)

Phase 2 — jev.py client: POST /v1/systemone, PINNED model jev-1.13.0 (drift warning when the response model differs), one retry on 429/5xx, fail-soft (None on any error), usage/latency returned for observability. Config adds TYPESAFE_API_KEY / JEV_* (thresholds 0.85/0.60, 15 questions per field); no daily cap by owner decision. bot/.env is now mounted read-only into the container so the key never enters compose/git.

Phase 3 — matching.py: برون‌گروهی (one request per field, parallel), درون‌گروهی (one choice question per mention sharing the same full candidate list + «_none»), درون‌واریانسی (all STT spellings in one question). Routing: conf>=exact → bind; floor..exact → ambiguity; else/_none → new; missing/failed field → difflib fallback (bug fixed during testing: a failed request used to record 'new' instead of falling back). Resolver gains full cached lists for suppliers/materials (paged) plus brands (API+DB) and specs (EAV values + draft specs). NLU prompt now emits entity_mentions (6 fields) with the voice-variant grouping rule; sanitize_mentions guards the shape. handle_message runs matching only for register_quote/update_draft/add_material behind JEV_ENABLED; jev_map threads into quote/material resolve_payload (Jev hit skips difflib; exact writes ids; ambiguity skips difflib). Spec effect per owner: spelling normalization only (normalize_brand_spec).

Phase 4 — entity-match ambiguity sub-messages: create_ambiguities() builds deduped entries {kind: entity_match, field, variants, candidates, details with Persian percentages}; new keyboards.ambiguity_candidates_keyboard renders one button per candidate + «هیچ‌کدام — ایجاد جدید»; callbacks handle amb⛏️{prefix}:{amb_id}:{idx} (applies the candidate via matching.apply_choice then re-resolves/renders) and amb:new (rejected → auto-create path at commit). Every user choice lands in observability (ambiguity.user_choice with arg+candidates).

Phase 5 — voice variants: NLU groups different STT spellings of one entity into one variants[] (verified live: مصطفوی/مستفدی → one supplier mention); tools/test_matching.py live section (RUN_LIVE=1) exercises real DeepSeek + real Jev without touching Bale.

Tests: test_jev.py 5/5 (mock HTTP: no-key, happy path, 429-retry, 400, exception), test_matching.py 11/11 mock + 3/3 live (real NLU produced 7 mentions with correct fields; real Jev matched «50 کیلویی»→«۵۰ کیلویی» 0.98 and «کیسه» 1.0 and correctly returned new against the emptied catalog); observability regression 8/8.
This commit is contained in:
2026-10-12 21:24:03 +03:30
parent d58b619107
commit 1c853e50a9
16 changed files with 1029 additions and 70 deletions
+16 -2
View File
@@ -224,7 +224,20 @@ async def handle_message(ctx: BotContext, chat_id: int, message: dict) -> None:
result = await understand(settings, input_payload, draft_summary, trace_id=trace_id)
intent, patch = result.get("intent"), result.get("patch") or {}
consensus_text = result.get("consensus_text") or text
await obs.event(trace_id, "entities.extracted", payload={"patch": patch, "intent": intent})
mentions = result.get("entity_mentions") or []
await obs.event(trace_id, "entities.extracted",
payload={"patch": patch, "intent": intent, "entity_mentions": mentions})
# Phase 3: Jev per-field matching — only when it can be applied (flag-gated;
# a failure leaves jev_map empty and the difflib path runs unchanged).
jev_map: dict = {}
if settings.JEV_ENABLED and mentions and intent in ("register_quote", "update_draft", "add_material"):
try:
from app import matching
jev_map = await matching.resolve_mentions(ctx, patch, mentions, trace_id)
except Exception:
logger.exception("jev matching failed")
jev_map = {}
if source == "voice":
await ctx._log(chat_id, "in", "voice", consensus_text)
@@ -238,6 +251,7 @@ async def handle_message(ctx: BotContext, chat_id: int, message: dict) -> None:
incoming_msg_id=message_id,
target_draft=target_draft,
trace_id=trace_id,
jev_map=jev_map,
)
elif intent == "commit_draft":
active = target_draft or await draft_store.get_active(chat_id)
@@ -280,7 +294,7 @@ async def handle_message(ctx: BotContext, chat_id: int, message: dict) -> None:
elif intent == "add_supplier":
await supplier.handle_entities(ctx, chat_id, user, patch, source, trace_id=trace_id)
elif intent == "add_material":
await material.handle_entities(ctx, chat_id, user, patch, source, trace_id=trace_id)
await material.handle_entities(ctx, chat_id, user, patch, source, trace_id=trace_id, jev_map=jev_map)
elif intent == "add_project":
await project.handle_entities(ctx, chat_id, user, patch, source, trace_id=trace_id)
elif intent == "status_query":