Problem: the dashboard was Jev-centric; NLU/arbiter/fallback DeepSeek, STT
providers and the audio-enhancer were barely visible, and stored texts/voices
had no real access path.
Bot (additive; reply path untouched):
- nlu.result now records latency_ms + the DeepSeek usage tokens
- matching.arbitration/fallback.result fill the model + latency_ms columns
(payload unchanged; SQL coalesces column/payload so old rows still count)
- storage.py: MinIO dual-write layer — every voice (raw + denoised) and user
attachment is mirrored fire-and-forget into bucket bot-media; the blocking
minio SDK runs via asyncio.to_thread (senior-engineer review) so the event
loop is never stalled; empty MINIO_* env = layer off; bucket auto-created
- requirements: minio>=7.2
Backend (admin-gated, read-only):
- GET /bot-metrics/models — one record per model (jev, deepseek-nlu,
deepseek-arbiter, deepseek-fallback, audio-enhancer + dynamic per-provider
STT) with calls/ok-rate/p50/p95/tokens/last-seen + version-drift vs the pin
- GET /bot-metrics/models/{id}/daily — per-model daily calls/fails/tokens/p50
- GET /bot-metrics/stages — every stage tagged with its Persian pipeline
section (صوت/درک پیام/تطبیق/ابهام و انتخاب/پیشنویس)
- /traces now carries input_kind + a 120-char text excerpt (consensus →
input.text → first STT transcript)
- media serving: POST /media-url mints a short-lived HMAC URL (audio/a tags
cannot send Authorization); GET /media streams from MinIO bot-media with
the read-only ./bot/data mount as fallback for pre-MinIO files; hard ref
validation (voices|attachments only, no .., extension whitelist)
- router exempted from the global Bearer dependency (files.py pattern) —
every endpoint keeps its own require_role('admin')
Frontend /bot-monitor (RTL, recharts):
- four tabs: نمای کلی / مدلها (records table + per-model expandable daily
charts: calls/fails, tokens, p50) / بخشها (sections grouped) / پیامها
- message drawer rebuilt: inline <audio controls> for raw + denoised voice,
download links, every STT transcript + consensus side-by-side, user
attachments, then the event timeline
- terminology: «ترس» → «پروندهی پیام» with tooltips + a one-line legend
Grafana: bot-observability gains a «کارکرد مدلها» table (stage→model
mapping for legacy rows); the explorer gains a message text column.
Seed: real (tiny) ogg files written + mirrored to MinIO, NLU usage tokens,
column-level model/latency on matching events.
Verified locally: /models returns all 7 records incl. per-STT and NLU tokens;
media 200 via signed URL (MinIO) and 401 without; ../ and .py rejected (400);
disk fallback serves a legacy file; pm-role 403 on every new endpoint;
bucket auto-created with 14 mirrored objects; tsc + vite build clean;
Grafana re-provisions 24 panels.
189 lines
7.2 KiB
Python
189 lines
7.2 KiB
Python
"""Bot configuration.
|
|
|
|
All settings can be overridden with environment variables prefixed with
|
|
``PROCUREMENT_BOT_`` (e.g. ``PROCUREMENT_BOT_BOT_TOKEN=...``) or a local
|
|
``.env`` file. Docker/compose passes them in directly (same convention as
|
|
the api service's ``PROCUREMENT_*`` prefix).
|
|
"""
|
|
from pydantic_settings import BaseSettings, SettingsConfigDict
|
|
|
|
|
|
class Settings(BaseSettings):
|
|
model_config = SettingsConfigDict(
|
|
env_prefix="PROCUREMENT_BOT_",
|
|
env_file=".env",
|
|
env_file_encoding="utf-8",
|
|
extra="ignore",
|
|
)
|
|
|
|
# Bale Bot API — Telegram-compatible clone hosted at tapi.bale.ai.
|
|
BALE_API_BASE: str = "https://tapi.bale.ai"
|
|
BOT_TOKEN: str = "" # from Bale's BotFather
|
|
POLL_TIMEOUT_SECONDS: int = 25 # getUpdates long-poll window
|
|
POLL_ERROR_SLEEP_SECONDS: float = 3.0
|
|
|
|
# ERP API (in-network service URL inside Docker).
|
|
API_BASE_URL: str = "http://localhost:8000/api/v1"
|
|
API_TIMEOUT_SECONDS: float = 30.0
|
|
# Optional bearer for the day procurement endpoints start enforcing auth.
|
|
SERVICE_TOKEN: str = ""
|
|
|
|
# Frontend URL (for opening Wizard from Bale bot)
|
|
FRONTEND_URL: str = "http://localhost:8081"
|
|
|
|
# Security token for standalone wizard review URL
|
|
DRAFT_TOKEN_SECRET: str = "procurement-draft-secret-change-in-prod-v7"
|
|
DRAFT_TOKEN_EXPIRE_SECONDS: int = 7200 # 2 hours
|
|
|
|
# Persistent storage directory for draft attachments (proformas)
|
|
ATTACHMENTS_DIR: str = "data/attachments/proformas"
|
|
|
|
# OpenAI-compatible AI provider (LLM), e.g. OpenAI / GLM / DeepSeek / OpenRouter.
|
|
AI_BASE_URL: str = "" # e.g. https://api.openai.com/v1
|
|
AI_API_KEY: str = ""
|
|
LLM_MODEL: str = "gpt-4o-mini"
|
|
LLM_TIMEOUT_SECONDS: float = 60.0
|
|
|
|
# STT providers — support multiple models/APIs in parallel.
|
|
# 1. JSON list: [{"name": "m1", "base_url": "...", "api_key": "...", "model": "..."}, ...]
|
|
STT_PROVIDERS_JSON: str = ""
|
|
|
|
# 2. Individual provider variables (up to 3 distinct STT models)
|
|
STT_PROVIDER_1_NAME: str = "model-1"
|
|
STT_PROVIDER_1_BASE_URL: str = ""
|
|
STT_PROVIDER_1_API_KEY: str = ""
|
|
STT_PROVIDER_1_MODEL: str = ""
|
|
STT_PROVIDER_1_LANGUAGE: str = "fa"
|
|
|
|
STT_PROVIDER_2_NAME: str = "model-2"
|
|
STT_PROVIDER_2_BASE_URL: str = ""
|
|
STT_PROVIDER_2_API_KEY: str = ""
|
|
STT_PROVIDER_2_MODEL: str = ""
|
|
STT_PROVIDER_2_LANGUAGE: str = "fa"
|
|
|
|
STT_PROVIDER_3_NAME: str = "model-3"
|
|
STT_PROVIDER_3_BASE_URL: str = ""
|
|
STT_PROVIDER_3_API_KEY: str = ""
|
|
STT_PROVIDER_3_MODEL: str = ""
|
|
STT_PROVIDER_3_LANGUAGE: str = "fa"
|
|
|
|
# Legacy single STT provider fallback:
|
|
STT_BASE_URL: str = ""
|
|
STT_API_KEY: str = ""
|
|
STT_MODEL: str = "whisper-1"
|
|
STT_LANGUAGE: str = "fa"
|
|
|
|
# Audio Enhancement / Preprocessing (denoising & speech quality enhancement)
|
|
AUDIO_ENHANCE_ENABLED: bool = False
|
|
AUDIO_ENHANCE_BASE_URL: str = ""
|
|
AUDIO_ENHANCE_API_KEY: str = ""
|
|
AUDIO_ENHANCE_MODEL: str = ""
|
|
AUDIO_ENHANCE_TIMEOUT_SECONDS: float = 60.0
|
|
AUDIO_ENHANCE_TARGET_FORMAT: str = "" # empty = pass raw bytes directly; e.g. "wav" / "mp3"
|
|
AUDIO_ENHANCE_METHOD: str = "auto" # "auto" | "raw_stream" | "multipart"
|
|
|
|
@property
|
|
def stt_base_url(self) -> str:
|
|
return self.STT_BASE_URL or self.AI_BASE_URL
|
|
|
|
@property
|
|
def stt_api_key(self) -> str:
|
|
return self.STT_API_KEY or self.AI_API_KEY
|
|
|
|
@property
|
|
def stt_providers(self) -> list[dict]:
|
|
import json
|
|
if self.STT_PROVIDERS_JSON:
|
|
try:
|
|
data = json.loads(self.STT_PROVIDERS_JSON)
|
|
if isinstance(data, list) and len(data) > 0:
|
|
return data
|
|
except Exception:
|
|
pass
|
|
|
|
# Check explicit STT_PROVIDER_1 / 2 / 3
|
|
custom_list: list[dict] = []
|
|
for idx in (1, 2, 3):
|
|
b_url = getattr(self, f"STT_PROVIDER_{idx}_BASE_URL")
|
|
key = getattr(self, f"STT_PROVIDER_{idx}_API_KEY")
|
|
model = getattr(self, f"STT_PROVIDER_{idx}_MODEL")
|
|
name = getattr(self, f"STT_PROVIDER_{idx}_NAME")
|
|
lang = getattr(self, f"STT_PROVIDER_{idx}_LANGUAGE") or "fa"
|
|
if b_url and key:
|
|
custom_list.append({
|
|
"name": name or f"model-{idx}",
|
|
"base_url": b_url,
|
|
"api_key": key,
|
|
"model": model or self.STT_MODEL,
|
|
"language": lang,
|
|
"timeout": 30.0,
|
|
})
|
|
if custom_list:
|
|
return custom_list
|
|
|
|
# Fallback to single legacy STT provider if set
|
|
if self.stt_base_url and self.stt_api_key:
|
|
return [{
|
|
"name": "default",
|
|
"base_url": self.stt_base_url,
|
|
"api_key": self.stt_api_key,
|
|
"model": self.STT_MODEL,
|
|
"language": self.STT_LANGUAGE,
|
|
"timeout": 30.0,
|
|
}]
|
|
return []
|
|
|
|
# PostgreSQL — same instance/database as the api service.
|
|
DATABASE_URL: str = "postgresql+asyncpg://postgres:postgres@localhost:5432/procurement"
|
|
|
|
# Chats that may use the bot without Casdoor linking (comma separated).
|
|
ADMIN_CHAT_IDS: str = ""
|
|
|
|
# Drafts older than this are marked expired by the sweeper loop.
|
|
DRAFT_TTL_HOURS: int = 24
|
|
|
|
# Observability (phase 1) — silent pipeline recorder (DB only; no external
|
|
# tools, no extra LLM tokens). Data is NEVER deleted.
|
|
OBS_ENABLED: bool = True
|
|
OBS_VOICE_DIR: str = "data/voices" # persistent volume (./bot/data) — keep forever
|
|
OBS_PROMPT_VERSION: str = "" # manual label for the NLU prompt in use
|
|
# Read-only DB role for the Grafana datasource (dashboard phase 1); empty =
|
|
# role provisioning is skipped (local dev can point the datasource at the
|
|
# postgres user instead). Rotating = change env + restart the bot.
|
|
OBS_READER_PASSWORD: str = ""
|
|
|
|
# Second copy of every bot media file in the ERP's existing MinIO
|
|
# (dual-write with the on-disk archive; empty endpoint = mirroring off).
|
|
MINIO_ENDPOINT: str = "" # e.g. "minio:9000" inside the compose network
|
|
MINIO_ACCESS_KEY: str = ""
|
|
MINIO_SECRET_KEY: str = ""
|
|
MINIO_BUCKET: str = "bot-media"
|
|
MINIO_SECURE: bool = False
|
|
|
|
# Jev (TypeSafe) entity matching — phases 2-5. Model version is PINNED
|
|
# (phase 0 resolved jev-1.13.0) so upstream updates cannot silently change
|
|
# decisions; every response's actual model is recorded for drift detection.
|
|
TYPESAFE_API_KEY: str = ""
|
|
JEV_MODEL: str = "jev-1.13.0"
|
|
JEV_ENABLED: bool = True
|
|
JEV_TIMEOUT: float = 8.0
|
|
JEV_CONF_EXACT: float = 0.85
|
|
JEV_CONF_FLOOR: float = 0.65 # phase-6 calibration (2026-10-12): 0.60 let one
|
|
# wrong candidate (جعبه→بسته @0.61) through; the
|
|
# correct picks it drops are covered by difflib.
|
|
JEV_MAX_QUESTIONS_PER_FIELD: int = 15
|
|
|
|
LOG_LEVEL: str = "INFO"
|
|
|
|
@property
|
|
def admin_chat_ids(self) -> set[int]:
|
|
out: set[int] = set()
|
|
for part in self.ADMIN_CHAT_IDS.replace(";", ",").split(","):
|
|
part = part.strip()
|
|
if part.isdigit():
|
|
out.add(int(part))
|
|
return out
|
|
|
|
|
|
settings = Settings()
|