feat(stack): full overhaul — hardening, ops tooling, monitoring, analytics, chatbot

Security: deep /api/health (DB+MinIO probes, 503 on failure) + /api/live; rate-limited public forms with honeypots; real client IP behind Traefik (--proxy-headers); TipTap URL sanitization on write (nh3); non-root containers + slim .dockerignore; prod MinIO console bound to 127.0.0.1; HSTS/nosniff on TLS routers; JWT 8h; no default credentials in code. Reliability: Alembic migrations (create_all baseline); graceful shutdown (exec uvicorn); hardened backups (checksums, sanity check, freshness gauge, optional rclone off-site) + restore.sh + hourly db-backup sidecar (ops profile); stack.sh ops tooling (deploy/logs/health/backup/restore/monitoring); .gitattributes LF for shell scripts. Orchestration: json-file log rotation, pinned images, per-service memory limits, edge/internal networks, Traefik ping/accesslog/metrics + flood limiter, conditional minio-setup, pgvector db image, one-shot db-init (umami + aramesh_chatbot databases). Chatbot: unified from site-chatbot branch (RAG + pgvector, SSE, ChatWidget). Monitoring: Prometheus/Alertmanager/Loki/Promtail/Grafana + cAdvisor, node/postgres/blackbox exporters, provisioned dashboards and 10 alert rules; alerts delivered to Bale (tapi.bale.ai — Telegram-compatible). Analytics: Umami on shared Postgres, build-time tracker injection, custom events (contact/newsletter/phone/chat/lang) via frontend/src/analytics.ts. Hygiene: CSV export for contact messages and subscribers; upload image optimization (Pillow); CI script + GitHub-compatible workflow; audit.sh; gen-env.sh for strong credentials.
2026-09-19 00:24:57 +03:30
parent 805f4594d4
commit ce0735d775
75 changed files with 4287 additions and 153 deletions
@@ -0,0 +1,50 @@
+"""مدل ری‌رنکر BGE — با لود تنبل و امتیازدهی دسته‌ای."""
+
+import logging
+import os
+
+from functools import lru_cache
+
+from .config import settings
+
+log = logging.getLogger(__name__)
+
+
+class _CrossEncoder:
+    """غلاف سبک روی AutoModelForSequenceClassification — API شبیه rank_bm25."""
+    def __init__(self) -> None:
+        import torch
+        from transformers import AutoModelForSequenceClassification, AutoTokenizer
+
+        os.makedirs(settings.hf_cache_dir, exist_ok=True)
+        self._torch = torch
+        self.tokenizer = AutoTokenizer.from_pretrained(settings.reranker_model, cache_dir=settings.hf_cache_dir)
+        self.model = AutoModelForSequenceClassification.from_pretrained(
+            settings.reranker_model, cache_dir=settings.hf_cache_dir
+        )
+        device = settings.reranker_device if settings.reranker_device == "cuda" and torch.cuda.is_available() else "cpu"
+        self.model.to(device)
+        self.model.eval()
+        self.device = device
+        log.info("Reranker loaded on %s", device)
+
+    def score(self, pairs: list[list[str]], batch_size: int = 16) -> list[float]:
+        torch = self._torch
+        out: list[float] = []
+        with torch.no_grad():
+            for i in range(0, len(pairs), batch_size):
+                inputs = self.tokenizer(
+                    pairs[i:i + batch_size],
+                    padding=True,
+                    truncation=True,
+                    max_length=512,
+                    return_tensors="pt",
+                ).to(self.device)
+                logits = self.model(**inputs).logits.squeeze(-1)
+                out.extend(logits.cpu().tolist())
+        return out
+
+
+@lru_cache(maxsize=1)
+def get_reranker() -> _CrossEncoder:
+    return _CrossEncoder()