Files
aramesh-backend/monitoring/prometheus/alerts.yml
T
MMD ce0735d775 feat(stack): full overhaul — hardening, ops tooling, monitoring, analytics, chatbot
Security: deep /api/health (DB+MinIO probes, 503 on failure) + /api/live;
rate-limited public forms with honeypots; real client IP behind Traefik
(--proxy-headers); TipTap URL sanitization on write (nh3); non-root
containers + slim .dockerignore; prod MinIO console bound to 127.0.0.1;
HSTS/nosniff on TLS routers; JWT 8h; no default credentials in code.

Reliability: Alembic migrations (create_all baseline); graceful shutdown
(exec uvicorn); hardened backups (checksums, sanity check, freshness gauge,
optional rclone off-site) + restore.sh + hourly db-backup sidecar (ops
profile); stack.sh ops tooling (deploy/logs/health/backup/restore/monitoring);
.gitattributes LF for shell scripts.

Orchestration: json-file log rotation, pinned images, per-service memory
limits, edge/internal networks, Traefik ping/accesslog/metrics + flood
limiter, conditional minio-setup, pgvector db image, one-shot db-init
(umami + aramesh_chatbot databases).

Chatbot: unified from site-chatbot branch (RAG + pgvector, SSE, ChatWidget).

Monitoring: Prometheus/Alertmanager/Loki/Promtail/Grafana + cAdvisor,
node/postgres/blackbox exporters, provisioned dashboards and 10 alert
rules; alerts delivered to Bale (tapi.bale.ai — Telegram-compatible).

Analytics: Umami on shared Postgres, build-time tracker injection, custom
events (contact/newsletter/phone/chat/lang) via frontend/src/analytics.ts.

Hygiene: CSV export for contact messages and subscribers; upload image
optimization (Pillow); CI script + GitHub-compatible workflow; audit.sh;
gen-env.sh for strong credentials.
2026-09-19 00:24:57 +03:30

119 lines
4.8 KiB
YAML
Raw Blame History

This file contains invisible Unicode characters
This file contains invisible Unicode characters that are indistinguishable to humans but may be processed differently by a computer. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# قواعد هشدار — از Prometheus به Alertmanager (تلگرام) می‌روند.
# آستانه‌ها محافظه‌کارانه‌اند؛ پس از یک هفته داده، دقیق‌تر تنظیم کنید.
groups:
- name: aramesh-availability
rules:
- alert: ServiceDown
expr: up == 0
for: 2m
labels:
severity: critical
annotations:
summary: "{{ $labels.job }} پاسخ نمی‌دهد"
description: "سرویس {{ $labels.job }} ({{ $labels.instance }}) حداقل ۲ دقیقه پایین است."
- alert: BackendHealthFailing
expr: probe_success{job="blackbox-internal-health"} == 0
for: 2m
labels:
severity: critical
annotations:
summary: "بررسی عمق سلامت بک‌اند شکست می‌خورد"
description: "‏/api/health اتصال DB یا MinIO را آزموده و 503 برگردانده — وابستگی‌ها را ببینید."
- alert: PublicSiteDown
expr: probe_success{job="blackbox-public"} == 0
for: 2m
labels:
severity: critical
annotations:
summary: "سایت از مسیر عمومی در دسترس نیست"
description: "پروب {{ $labels.instance }} ناموفق است — Traefik/TLS/DNS را بررسی کنید."
- name: aramesh-resources
rules:
- alert: HostDiskAlmostFull
expr: |
1 - (node_filesystem_avail_bytes{fstype!~"tmpfs|overlay|squashfs"} / node_filesystem_size_bytes{fstype!~"tmpfs|overlay|squashfs"}) > 0.85
for: 5m
labels:
severity: warning
annotations:
summary: "دیسک میزبان پر می‌شود"
description: "پرشدن بیش از ۸۵٪ روی {{ $labels.mountpoint }} — پشتیبان‌ها و لاگ‌ها را بررسی کنید."
- alert: HostMemoryPressure
expr: node_memory_MemAvailable_bytes / node_memory_MemTotal_bytes < 0.1
for: 5m
labels:
severity: warning
annotations:
summary: "RAM میزبان رو به تمام‌شدن است"
description: "کمتر از ۱۰٪ حافظه باقی مانده است."
- alert: ContainerMemoryNearLimit
expr: |
container_memory_usage_bytes{image!="", container!="", container_spec_memory_limit_bytes > 0}
/ container_spec_memory_limit_bytes{container!="", container_spec_memory_limit_bytes > 0} > 0.9
for: 10m
labels:
severity: warning
annotations:
summary: "کانتینر {{ $labels.container }} به سقف حافظه نزدیک است"
- name: aramesh-app
rules:
- alert: Http5xxRateHigh
expr: |
sum(rate(http_request_duration_seconds_count{status=~"5.."}[5m])) by (instance)
/ sum(rate(http_request_duration_seconds_count[5m])) by (instance) > 0.05
for: 5m
labels:
severity: warning
annotations:
summary: "نرخ خطای 5xx بک‌اند بالاست"
- alert: Proxy5xxRateHigh
expr: |
sum(rate(traefik_service_requests_total{code=~"5.."}[5m])) by (service)
/ sum(rate(traefik_service_requests_total[5m])) by (service) > 0.05
for: 5m
labels:
severity: warning
annotations:
summary: "نرخ خطای 5xx روی سرویس {{ $labels.service }} پشت Traefik بالاست"
- alert: HighBackendLatency
expr: histogram_quantile(0.95, sum(rate(http_request_duration_seconds_bucket[5m])) by (le)) > 2
for: 10m
labels:
severity: warning
annotations:
summary: "p95 تأخیر بک‌اند بالای ۲ ثانیه است"
- alert: PostgresTooManyConnections
expr: sum(pg_stat_activity_count) > 80
for: 5m
labels:
severity: warning
annotations:
summary: "تعداد اتصال‌های Postgres نزدیک سقف است"
- alert: TlsCertExpiringSoon
expr: traefik_tls_certs_not_after - time() < 1209600
labels:
severity: warning
annotations:
summary: "گواهی TLS کمتر از ۱۴ روز اعتبار دارد"
description: "تمدید خودکار Let's Encrypt کار نکرده — ACME/TLS را بررسی کنید."
- alert: BackupStale
expr: time() - backup_success_time > 93600
for: 30m
labels:
severity: critical
annotations:
summary: "آخرین پشتیبان موفق بیش از ۲۶ ساعت پیش است"
description: "کرون/اسکریپت پشتیبان‌گیری یا سرویس db-backup خفته — فوراً بررسی کنید."