ERP/bot/docs/SETUP.md

# Setup & Operations Guide

A Bale messenger bot that turns Persian **chat messages and voice notes** into
procurement actions in the ERP. It works **draft-first**: it captures what it
understood, tells you exactly **what it did** and **what is still empty**, and
**commits only after you approve** with a ✅ button.

Bale's Bot API is a Telegram-compatible clone hosted at `tapi.bale.ai`, so the
bot uses the familiar `getUpdates` long-polling — **no public port, no webhook,
no TLS setup**.

---

## What it can do (v1)

| Activity | Example (voice or text) | Committed via |
|---|---|---|
| Register a supplier quote | «سازان نو برای پروژه برج آسمان ۵۰۰ کیلو میلگرد A3 سایز ۱۴، هر کیلو ۴۵ تومن، پرداخت ۳ فاز» | `POST /quotes` (nested create) |
| Add a supplier | «تأمین‌کننده جدید پروفیل بر، دسته لوله و اتصالات، تلفن ۰۹۱۲…» | `POST /suppliers` |
| Add a material | «کالای جدید میلگرد A3 سایز ۱۸ با واحد کیلوگرم» | `POST /materials` |
| Status queries | «استعلام‌های در انتظار؟» / «کدام‌ها به‌زودی منقضی می‌شوند؟» | `GET /quotes?…` |
| Accept a quote | «استعلام سازان نو را قبول کن» | `PATCH /quotes/{id}/accept` |
| Announce a delivery | «اقلام سازان نو رسید، ۵۰۰ و ۲۰۰» | `PATCH /quotes/{id}/arrive-items` |

### The draft → approve loop (the core idea)

1. You send a message (or voice note). The bot converts voice to text
   (STT) and extracts an **intent + entities** (LLM).
2. Names are fuzzy-matched against the ERP (supplier / material / unit /
   project). Multiple candidates → inline **pick buttons**; no match → it is
   listed as missing.
3. The bot replies with a **review card**: everything it captured, plus a
   ✅/⚠️/❌ **list of what is still empty** («یادآوری چیزهای خالی»).
4. You keep chatting to refine («قیمت میلگرد ۴۷ شد», «پروژه برج آسمان») — the
   open draft is updated and re-rendered.
5. **Nothing is written to the ERP until you tap ✅ ثبت نهایی.** Partial
   commits are possible («ثبت فقط اقلام مشخص») because the API deliberately
   tolerates incomplete quotes. After commit you get the created quote id and
   the list of fields that were left empty.

### Team access & attribution

- Every user links their Bale chat to their ERP account once: `/start` →
  Casdoor **username** → **password** (validated through the real
  `POST /auth/login`; the password is used once and never stored).
- Mutating API calls carry `X-Actor-Id: <users.id>` so `quote_events.actor`
  records the **human** who asked, not the bot. (The bot's container talks to
  the api in-network: `http://api:8000/api/v1`.)
- `BALE_ADMIN_CHAT_IDS` may list chat ids that bypass linking.

### Commands

`/start` · `/help` · `/drafts` (open drafts) · `/cancel` (abort draft/flow) ·
`/whoami` · `/unlink`

---

## Setup

1. **Create the bot** in Bale's BotFather and copy the token.
2. **Configure an AI provider** — any OpenAI-compatible endpoint works for
   both STT (`POST {base}/audio/transcriptions`, Persian) and the LLM
   (`POST {base}/chat/completions`, JSON mode). Examples: OpenAI `whisper-1`
   + `gpt-4o-mini`, or any proxy/provider you already use.
3. **Compose** (dev):

   ```powershell
   # env passed at compose level (or a .env next to docker-compose.yml)
   $env:BALE_BOT_TOKEN = "<token>"
   $env:AI_BASE_URL    = "https://api.openai.com/v1"
   $env:AI_API_KEY     = "<key>"

   docker compose up -d --build bot
   docker compose logs -f bot
   ```

   For Coolify (`docker-compose.coolify.yml`) set the same variables as
   service-level env vars; the bot container needs no domain/labels.

### Environment variables (prefix `PROCUREMENT_BOT_`)

| Variable | Default | Purpose |
|---|---|---|
| `BOT_TOKEN` | — (required) | token from Bale's BotFather |
| `BALE_API_BASE` | `https://tapi.bale.ai` | Bot API base (overridable for tests) |
| `API_BASE_URL` | `http://localhost:8000/api/v1` | ERP api (in Docker: `http://api:8000/api/v1`) |
| `DATABASE_URL` | same postgres as the api | bot's own tables (`bot_*`) |
| `AI_BASE_URL` / `AI_API_KEY` | — | OpenAI-compatible STT+LLM provider |
| `STT_MODEL` / `LLM_MODEL` | `whisper-1` / `gpt-4o-mini` | model names at the provider |
| `ADMIN_CHAT_IDS` | — | comma-separated Bale chat ids allowed without linking |
| `SERVICE_TOKEN` | — | optional bearer for the day procurement endpoints enforce auth |
| `DRAFT_TTL_HOURS` | `24` | drafts older than this are marked expired |

In compose these are fed from `BALE_BOT_TOKEN`, `BALE_API_BASE`, `AI_BASE_URL`,
`AI_API_KEY`, `AI_STT_MODEL`, `AI_LLM_MODEL`, `BALE_ADMIN_CHAT_IDS`,
`BALE_SERVICE_TOKEN`.

---

## How it's built

```
bot/
├── Dockerfile                  python:3.13-slim + ffmpeg (voice conversion)
├── app/
│   ├── main.py                 ensure tables → sweeper task → poll loop
│   ├── config.py               pydantic-settings, PROCUREMENT_BOT_ prefix
│   ├── bale/client.py          getUpdates/sendMessage/edit/callbacks/getFile+download
│   ├── api/client.py           ERP /api/v1 client (X-Actor-Id attribution)
│   ├── stt.py                  ffmpeg (OGG/Opus → MP3) → /audio/transcriptions
│   ├── nlu.py                  Persian intent+entity extraction (JSON mode)
│   ├── resolve.py              fuzzy name→id matching (difflib + ilike search)
│   ├── drafts.py               draft store + QuoteCreate payload builder + missing-field logic
│   ├── keyboards.py            inline keyboards (callback_data ≤ 64 B op-codes)
│   ├── texts.py                Persian message templates
│   ├── runtime.py              context, link store, in-memory pending flows
│   ├── models.py / db_init.py  bot_links / bot_drafts / bot_messages / bot_meta (idempotent DDL)
│   └── handlers/               dispatch, commands, callbacks, links,
│                               quote, supplier, material, queries, decisions
└── tools/smoke_test.py         offline end-to-end harness (fake Bale/ERP/AI, in-memory store)
```

Design notes:

- **Draft state lives in postgres** (`bot_drafts.payload` JSONB in the shape of
  the eventual API request), so drafts survive bot restarts; short-lived
  interactions (linking, quote picking, accept/arrive confirmations) are
  in-memory with a 30-minute TTL.
- **Callback grammar** fits Bale's 64-byte `callback_data`:
  `q:ok:{draft8}` (commit), `pk:{draft8}:{field}:{idx}` (pick candidate),
  `ac:ts:{i}` (choose payment terms), `ar:full` / `ar:ok` (delivery)…
- **Persian handling**: Persian/Arabic digits are normalized before parsing;
  fuzzy matching folds ZWNJ and Arabic ya/ka; all user-facing text is Persian.
- **Voice notes** arrive as OGG/Opus; ffmpeg converts to mono 16 kHz MP3
  before hitting the STT endpoint (accepted by essentially all providers).
- The **accept flow echoes the termsheet's existing `schedule_steps`** back to
  `PATCH /quotes/{id}/accept` (the API replaces steps with what you send, so
  the bot must send the current ones).

### Attribution tweak in the api (backward compatible)

`backend/app/utils/security.py::get_actor` now prefers an `X-Actor-Id` header
over the raw bearer token when attributing `quote_events.actor`. Requests
without that header behave exactly as before.

---

## Testing

Offline harness (no Bale token, no DB, no AI key — stubs everything and drives
the real handler pipeline with fake updates):

```powershell
docker compose run --rm bot python tools/smoke_test.py
# or, with the requirements installed locally:
cd bot && python tools/smoke_test.py
```

Covers: linking, voice→draft, ambiguous-material pick buttons, unit fallback,
draft refinement, commit payload shape (temp_id cross-references), supplier /
material commits, status digest, accept payload (steps echoed, per-item price),
arrival quantities («۵۰۰ و ۲۰۰»), cancel and unknown-intent handling.

## Troubleshooting

| Symptom | Likely cause |
|---|---|
| Bot silent | wrong `BALE_BOT_TOKEN`; check `docker compose logs bot` |
| «پردازش هوشمند تنظیم نشده است» | `AI_BASE_URL` / `AI_API_KEY` missing |
| Voice → «نتوانستم پیام صوتی را تبدیل کنم» | provider rejected the model name (`STT_MODEL`) or key |
| «ERP API unreachable» | api container not up / wrong `API_BASE_URL` |
| Draft disappears | TTL expiry (`DRAFT_TTL_HOURS`) or someone pressed ❌ / `/cancel` |