Files
ERP/bot/app/config.py
T
whqreza 0adadd2cf8 feat(monitoring): bot observability dashboards — Grafana + Persian RTL ERP page
Grafana (full use of the existing monitoring/ stack, per OSS policy):
- datasources.yml: BotPostgres (uid bot-postgres) over the shared network;
  grafana service joins 'shared' + carries the datasource env
- db_init.py: idempotent read-only role obs_reader (USAGE on schema public +
  SELECT on bot_traces/bot_trace_events; skipped when
  OBS_READER_PASSWORD empty; ALTER ROLE for rotation) + composite index
  ix_bot_trace_events_stage_time (stage, created_at DESC) — senior-engineer
  review applied with the column-name fix (event_type -> stage); (trace_id)
  already covered by the existing (trace_id, created_at) index
- dashboards/bot-observability.json: KPI row (real traffic, events, Jev/
  fallback errors, Jev p95, tokens, commits, archive guard), events/day,
  per-stage p50/p95, mention statuses (arbiter+fallback aggregates + legacy
  rows), DeepSeek arbiter actions, tokens/day, model-drift vs jev-1.13.0,
  user-choice A/B, STT + audio.enhance health, archive size guard
- dashboards/bot-trace-explorer.json: trace table + full event drill-down
  via the trace_id variable

ERP page (admin-only, Persian RTL, recharts@^3 for React 19):
- backend bot_metrics.py: read-only require_role('admin') endpoints
  (summary/routing/quality/jev/stt/traces + trace detail); aggregates count
  real traffic only (calibration/demo/chat_id=0 excluded)
- frontend BotMonitorPage at /bot-monitor (KPI cards, daily events,
  status donut, arbiter actions, user-choice A/B, STT/enhance tables,
  stage latency, tokens chart, traces table + detail drawer with payload
  viewer); entry link from the admin panel; MobileTabBar hidden there
- api.ts: typed botMetrics client section

Tooling/docs:
- tools/seed_observability_demo.py: deterministic demo traces (source='demo',
  every current pipeline stage incl. audio.enhance/jev.failure/arbitration/
  fallback/user choices) — excluded from real-traffic aggregates
- monitoring/README.md + bot/docs/DEPLOY-AND-MONITORING.md: setup, seed and
  rollback notes; alerts deliberately deferred (owner decision 2026-10-24)

Verified locally: obs_reader SELECT ok / INSERT denied; panels' SQL run as
obs_reader with data; datasource health OK; both dashboards survive a grafana
restart; endpoints 200 as admin and 403 without role; tsc + vite build clean;
page renders in-browser (admin session); bot startup provisions role+index.
2026-10-24 15:09:01 +03:30

181 lines
6.8 KiB
Python

"""Bot configuration.
All settings can be overridden with environment variables prefixed with
``PROCUREMENT_BOT_`` (e.g. ``PROCUREMENT_BOT_BOT_TOKEN=...``) or a local
``.env`` file. Docker/compose passes them in directly (same convention as
the api service's ``PROCUREMENT_*`` prefix).
"""
from pydantic_settings import BaseSettings, SettingsConfigDict
class Settings(BaseSettings):
model_config = SettingsConfigDict(
env_prefix="PROCUREMENT_BOT_",
env_file=".env",
env_file_encoding="utf-8",
extra="ignore",
)
# Bale Bot API — Telegram-compatible clone hosted at tapi.bale.ai.
BALE_API_BASE: str = "https://tapi.bale.ai"
BOT_TOKEN: str = "" # from Bale's BotFather
POLL_TIMEOUT_SECONDS: int = 25 # getUpdates long-poll window
POLL_ERROR_SLEEP_SECONDS: float = 3.0
# ERP API (in-network service URL inside Docker).
API_BASE_URL: str = "http://localhost:8000/api/v1"
API_TIMEOUT_SECONDS: float = 30.0
# Optional bearer for the day procurement endpoints start enforcing auth.
SERVICE_TOKEN: str = ""
# Frontend URL (for opening Wizard from Bale bot)
FRONTEND_URL: str = "http://localhost:8081"
# Security token for standalone wizard review URL
DRAFT_TOKEN_SECRET: str = "procurement-draft-secret-change-in-prod-v7"
DRAFT_TOKEN_EXPIRE_SECONDS: int = 7200 # 2 hours
# Persistent storage directory for draft attachments (proformas)
ATTACHMENTS_DIR: str = "data/attachments/proformas"
# OpenAI-compatible AI provider (LLM), e.g. OpenAI / GLM / DeepSeek / OpenRouter.
AI_BASE_URL: str = "" # e.g. https://api.openai.com/v1
AI_API_KEY: str = ""
LLM_MODEL: str = "gpt-4o-mini"
LLM_TIMEOUT_SECONDS: float = 60.0
# STT providers — support multiple models/APIs in parallel.
# 1. JSON list: [{"name": "m1", "base_url": "...", "api_key": "...", "model": "..."}, ...]
STT_PROVIDERS_JSON: str = ""
# 2. Individual provider variables (up to 3 distinct STT models)
STT_PROVIDER_1_NAME: str = "model-1"
STT_PROVIDER_1_BASE_URL: str = ""
STT_PROVIDER_1_API_KEY: str = ""
STT_PROVIDER_1_MODEL: str = ""
STT_PROVIDER_1_LANGUAGE: str = "fa"
STT_PROVIDER_2_NAME: str = "model-2"
STT_PROVIDER_2_BASE_URL: str = ""
STT_PROVIDER_2_API_KEY: str = ""
STT_PROVIDER_2_MODEL: str = ""
STT_PROVIDER_2_LANGUAGE: str = "fa"
STT_PROVIDER_3_NAME: str = "model-3"
STT_PROVIDER_3_BASE_URL: str = ""
STT_PROVIDER_3_API_KEY: str = ""
STT_PROVIDER_3_MODEL: str = ""
STT_PROVIDER_3_LANGUAGE: str = "fa"
# Legacy single STT provider fallback:
STT_BASE_URL: str = ""
STT_API_KEY: str = ""
STT_MODEL: str = "whisper-1"
STT_LANGUAGE: str = "fa"
# Audio Enhancement / Preprocessing (denoising & speech quality enhancement)
AUDIO_ENHANCE_ENABLED: bool = False
AUDIO_ENHANCE_BASE_URL: str = ""
AUDIO_ENHANCE_API_KEY: str = ""
AUDIO_ENHANCE_MODEL: str = ""
AUDIO_ENHANCE_TIMEOUT_SECONDS: float = 60.0
AUDIO_ENHANCE_TARGET_FORMAT: str = "" # empty = pass raw bytes directly; e.g. "wav" / "mp3"
AUDIO_ENHANCE_METHOD: str = "auto" # "auto" | "raw_stream" | "multipart"
@property
def stt_base_url(self) -> str:
return self.STT_BASE_URL or self.AI_BASE_URL
@property
def stt_api_key(self) -> str:
return self.STT_API_KEY or self.AI_API_KEY
@property
def stt_providers(self) -> list[dict]:
import json
if self.STT_PROVIDERS_JSON:
try:
data = json.loads(self.STT_PROVIDERS_JSON)
if isinstance(data, list) and len(data) > 0:
return data
except Exception:
pass
# Check explicit STT_PROVIDER_1 / 2 / 3
custom_list: list[dict] = []
for idx in (1, 2, 3):
b_url = getattr(self, f"STT_PROVIDER_{idx}_BASE_URL")
key = getattr(self, f"STT_PROVIDER_{idx}_API_KEY")
model = getattr(self, f"STT_PROVIDER_{idx}_MODEL")
name = getattr(self, f"STT_PROVIDER_{idx}_NAME")
lang = getattr(self, f"STT_PROVIDER_{idx}_LANGUAGE") or "fa"
if b_url and key:
custom_list.append({
"name": name or f"model-{idx}",
"base_url": b_url,
"api_key": key,
"model": model or self.STT_MODEL,
"language": lang,
"timeout": 30.0,
})
if custom_list:
return custom_list
# Fallback to single legacy STT provider if set
if self.stt_base_url and self.stt_api_key:
return [{
"name": "default",
"base_url": self.stt_base_url,
"api_key": self.stt_api_key,
"model": self.STT_MODEL,
"language": self.STT_LANGUAGE,
"timeout": 30.0,
}]
return []
# PostgreSQL — same instance/database as the api service.
DATABASE_URL: str = "postgresql+asyncpg://postgres:postgres@localhost:5432/procurement"
# Chats that may use the bot without Casdoor linking (comma separated).
ADMIN_CHAT_IDS: str = ""
# Drafts older than this are marked expired by the sweeper loop.
DRAFT_TTL_HOURS: int = 24
# Observability (phase 1) — silent pipeline recorder (DB only; no external
# tools, no extra LLM tokens). Data is NEVER deleted.
OBS_ENABLED: bool = True
OBS_VOICE_DIR: str = "data/voices" # persistent volume (./bot/data) — keep forever
OBS_PROMPT_VERSION: str = "" # manual label for the NLU prompt in use
# Read-only DB role for the Grafana datasource (dashboard phase 1); empty =
# role provisioning is skipped (local dev can point the datasource at the
# postgres user instead). Rotating = change env + restart the bot.
OBS_READER_PASSWORD: str = ""
# Jev (TypeSafe) entity matching — phases 2-5. Model version is PINNED
# (phase 0 resolved jev-1.13.0) so upstream updates cannot silently change
# decisions; every response's actual model is recorded for drift detection.
TYPESAFE_API_KEY: str = ""
JEV_MODEL: str = "jev-1.13.0"
JEV_ENABLED: bool = True
JEV_TIMEOUT: float = 8.0
JEV_CONF_EXACT: float = 0.85
JEV_CONF_FLOOR: float = 0.65 # phase-6 calibration (2026-10-12): 0.60 let one
# wrong candidate (جعبه→بسته @0.61) through; the
# correct picks it drops are covered by difflib.
JEV_MAX_QUESTIONS_PER_FIELD: int = 15
LOG_LEVEL: str = "INFO"
@property
def admin_chat_ids(self) -> set[int]:
out: set[int] = set()
for part in self.ADMIN_CHAT_IDS.replace(";", ",").split(","):
part = part.strip()
if part.isdigit():
out.add(int(part))
return out
settings = Settings()