Spaces:
Sleeping
Sleeping
Upload 44 files
Browse files- DEPLOY.md +19 -29
- README.md +18 -9
- app.py +23 -23
- src/llm_client.py +207 -133
DEPLOY.md
CHANGED
|
@@ -59,35 +59,25 @@ The Space will automatically build from `requirements.txt` and launch
|
|
| 59 |
|
| 60 |
## Enabling the LLM (recommended before sharing this project)
|
| 61 |
|
| 62 |
-
By default the Space runs in **retrieval-only fallback mode** — it still
|
| 63 |
-
|
| 64 |
-
|
| 65 |
-
|
| 66 |
-
|
| 67 |
-
|
| 68 |
-
|
| 69 |
-
|
| 70 |
-
|
| 71 |
-
|
| 72 |
-
|
| 73 |
-
|
| 74 |
-
|
| 75 |
-
|
| 76 |
-
|
| 77 |
-
|
| 78 |
-
|
| 79 |
-
|
| 80 |
-
|
| 81 |
-
through the now-deprecated `api-inference.huggingface.co` domain, which
|
| 82 |
-
fails with a DNS resolution error rather than a clear auth error.
|
| 83 |
-
- Hardcoding a specific provider (e.g. `"hf-inference"`) can fail with
|
| 84 |
-
*"Model not supported by provider ..."* for models that are actually
|
| 85 |
-
hosted via a different backend (Together, Fireworks, Novita, SambaNova,
|
| 86 |
-
etc.) under HF's Inference Providers system. `provider="auto"` lets HF's
|
| 87 |
-
router pick whichever backend actually serves each candidate model.
|
| 88 |
-
|
| 89 |
-
If you ever see errors in the "Test LLM connection" diagnostics, they'll
|
| 90 |
-
show you exactly which of the above it is per candidate model.
|
| 91 |
|
| 92 |
## Re-training / updating the models
|
| 93 |
|
|
|
|
| 59 |
|
| 60 |
## Enabling the LLM (recommended before sharing this project)
|
| 61 |
|
| 62 |
+
By default the Space runs in **retrieval-only fallback mode** — it still works, but answers are extractive
|
| 63 |
+
rather than LLM-generated. To turn on real LLM responses:
|
| 64 |
+
|
| 65 |
+
1. Create a **free** API key at https://console.groq.com (Groq has a free plan and an OpenAI-compatible API).
|
| 66 |
+
2. In your Space: **Settings → Variables and secrets → New secret**, name `GROQ_API_KEY`, value = your key.
|
| 67 |
+
3. (Optional) add `LLM_MODEL` to choose another Groq model if the default (`llama-3.1-8b-instant`) is ever retired.
|
| 68 |
+
4. Restart the Space, open **AI Assistant → LLM connection diagnostics → Test LLM connection**.
|
| 69 |
+
|
| 70 |
+
**Why not just use the Hugging Face token?** The app still supports `HF_TOKEN` (via HF's router), but two real
|
| 71 |
+
problems showed up in practice:
|
| 72 |
+
- `402 Payment Required` — Hugging Face Inference Providers bills against account credits, and free accounts
|
| 73 |
+
get almost none (a public demo exhausts it quickly). No code change can fix that.
|
| 74 |
+
- `Provider 'featherless-ai' not supported` — an old `InferenceClient` pinned in `requirements.txt` didn't know
|
| 75 |
+
providers that HF's router had started using. The client now speaks plain OpenAI-compatible HTTP instead of
|
| 76 |
+
the SDK, so there is no provider list to go stale.
|
| 77 |
+
|
| 78 |
+
Any other OpenAI-compatible service (OpenRouter, Together, Cerebras…) works with `LLM_API_KEY` + `LLM_BASE_URL`
|
| 79 |
+
+ `LLM_MODEL`. A 402/401 stops further attempts on that backend immediately; other errors (rate limit, unknown
|
| 80 |
+
model, server error, timeout) move on to the next candidate.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 81 |
|
| 82 |
## Re-training / updating the models
|
| 83 |
|
README.md
CHANGED
|
@@ -64,20 +64,29 @@ Open the printed local URL (usually `http://127.0.0.1:7860`).
|
|
| 64 |
|
| 65 |
## Enabling full LLM responses
|
| 66 |
|
| 67 |
-
The app works out of the box in **retrieval-only fallback mode** (no
|
| 68 |
-
|
|
|
|
| 69 |
|
| 70 |
-
1. Create a
|
| 71 |
-
2. Set it as
|
| 72 |
-
|
| 73 |
-
|
| 74 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 75 |
|
| 76 |
```bash
|
| 77 |
-
export
|
| 78 |
python app.py
|
| 79 |
```
|
| 80 |
|
|
|
|
|
|
|
| 81 |
## Deploying to Hugging Face Spaces
|
| 82 |
|
| 83 |
See [`DEPLOY.md`](./DEPLOY.md) for full step-by-step instructions.
|
|
@@ -98,7 +107,7 @@ smart-warehouse-ai/
|
|
| 98 |
│ ├── retriever.py # TF-IDF retriever
|
| 99 |
│ ├── intent_model.py # intent classifier (train/predict)
|
| 100 |
│ ├── anomaly_model.py # Isolation Forest anomaly detector
|
| 101 |
-
│ ├── llm_client.py #
|
| 102 |
│ └── inventory_db.py # NL -> structured query helpers
|
| 103 |
├── models/ # trained model artifacts (.joblib)
|
| 104 |
├── data/ # generated datasets + evaluation JSON
|
|
|
|
| 64 |
|
| 65 |
## Enabling full LLM responses
|
| 66 |
|
| 67 |
+
The app works out of the box in **retrieval-only fallback mode** (no external API calls). For real
|
| 68 |
+
LLM-generated answers it needs an API key for any **OpenAI-compatible** chat endpoint. The simplest free
|
| 69 |
+
option is Groq:
|
| 70 |
|
| 71 |
+
1. Create a free API key at https://console.groq.com
|
| 72 |
+
2. Set it as a Space secret (or env var) named `GROQ_API_KEY`. The default model is `llama-3.1-8b-instant`;
|
| 73 |
+
override with `LLM_MODEL` if Groq retires or renames it.
|
| 74 |
+
|
| 75 |
+
Other options (checked in this order; the first that answers wins):
|
| 76 |
+
|
| 77 |
+
| Secrets | Backend |
|
| 78 |
+
|---|---|
|
| 79 |
+
| `LLM_API_KEY` + `LLM_BASE_URL` + `LLM_MODEL` | any OpenAI-compatible API (OpenRouter, Together, Cerebras, …) |
|
| 80 |
+
| `GROQ_API_KEY` | Groq (free plan) |
|
| 81 |
+
| `HF_TOKEN` | Hugging Face Inference Providers router — **needs credits**; free accounts get almost none, which shows up as `402 Payment Required` |
|
| 82 |
|
| 83 |
```bash
|
| 84 |
+
export GROQ_API_KEY=gsk_xxxxxxxxxxxx
|
| 85 |
python app.py
|
| 86 |
```
|
| 87 |
|
| 88 |
+
The **Test LLM connection** button (AI Assistant tab) shows exactly what each backend returned.
|
| 89 |
+
|
| 90 |
## Deploying to Hugging Face Spaces
|
| 91 |
|
| 92 |
See [`DEPLOY.md`](./DEPLOY.md) for full step-by-step instructions.
|
|
|
|
| 107 |
│ ├── retriever.py # TF-IDF retriever
|
| 108 |
│ ├── intent_model.py # intent classifier (train/predict)
|
| 109 |
│ ├── anomaly_model.py # Isolation Forest anomaly detector
|
| 110 |
+
│ ├── llm_client.py # OpenAI-compatible LLM client (Groq / custom / HF router) + fallback
|
| 111 |
│ └── inventory_db.py # NL -> structured query helpers
|
| 112 |
├── models/ # trained model artifacts (.joblib)
|
| 113 |
├── data/ # generated datasets + evaluation JSON
|
app.py
CHANGED
|
@@ -5,7 +5,7 @@ A warehouse/intralogistics AI copilot demo, built for a Hugging Face
|
|
| 5 |
Space. Combines:
|
| 6 |
|
| 7 |
1. An LLM-powered assistant (RAG: TF-IDF retrieval + hosted LLM via the
|
| 8 |
-
|
| 9 |
2. A TF-IDF + Logistic Regression intent classifier that routes queries
|
| 10 |
into 8 operational categories.
|
| 11 |
3. A lightweight NL -> structured query layer over synthetic inventory /
|
|
@@ -36,7 +36,7 @@ from src.real_vision import (
|
|
| 36 |
from src.data_generation import generate_inventory_db, generate_orders_db
|
| 37 |
from src.intent_model import INTENT_DESCRIPTIONS, load_pipeline, predict as intent_predict
|
| 38 |
from src.inventory_db import query_inventory, query_orders
|
| 39 |
-
from src.llm_client import answer_query, test_connection,
|
| 40 |
from src.retriever import KBRetriever
|
| 41 |
|
| 42 |
ROOT = os.path.dirname(os.path.abspath(__file__))
|
|
@@ -112,13 +112,13 @@ retrieval_eval = _load_json("retrieval_eval.json")
|
|
| 112 |
latency_eval = _load_json("latency_eval.json")
|
| 113 |
vision_eval = _load_json("vision_eval.json")
|
| 114 |
|
| 115 |
-
|
| 116 |
|
| 117 |
# --------------------------------------------------------------------------
|
| 118 |
# ZeroGPU compatibility shim
|
| 119 |
# --------------------------------------------------------------------------
|
| 120 |
# This app is CPU-only by design (scikit-learn locally, LLM calls go to the
|
| 121 |
-
# remote
|
| 122 |
# free "ZeroGPU" hardware tier, which requires at least one function
|
| 123 |
# decorated with `@spaces.GPU` to be present or the platform's startup
|
| 124 |
# check fails with "No @spaces.GPU function detected". This is a harmless,
|
|
@@ -171,15 +171,14 @@ def test_llm_fn():
|
|
| 171 |
f"✅ **LLM connection working.** Model: `{response.model_id}` · "
|
| 172 |
f"{response.latency_s * 1000:.0f} ms\n\nSample answer: {response.answer}"
|
| 173 |
)
|
| 174 |
-
err_lines = "\n".join(f"-
|
| 175 |
return (
|
| 176 |
"❌ **LLM connection failed** — running in retrieval-only fallback mode.\n\n"
|
| 177 |
-
f"**
|
| 178 |
-
"**
|
| 179 |
-
"
|
| 180 |
-
"
|
| 181 |
-
"
|
| 182 |
-
"specific one."
|
| 183 |
)
|
| 184 |
|
| 185 |
|
|
@@ -459,7 +458,7 @@ being realistic enough to demonstrate the underlying ML techniques properly.
|
|
| 459 |
|---|---|---|---|
|
| 460 |
| **Intent classifier** | Decide what kind of request a query is (maintenance, safety, inventory, order status, navigation, picking, system status, general FAQ) | TF-IDF + Logistic Regression | Free text → intent label + confidence |
|
| 461 |
| **KB retriever** | Find the most relevant knowledge-base passage(s) for a query, to ground the LLM's answer | TF-IDF + cosine similarity | Free text → top-k ranked passages |
|
| 462 |
-
| **LLM assistant** | Generate a natural-language answer grounded in the retrieved context | Hosted instruct LLM
|
| 463 |
| **Inventory/order query** | Turn a question into a filtered table lookup | Regex slot extraction (SKU / order ID / zone) + intent routing | Free text → filtered inventory or orders table |
|
| 464 |
| **Anomaly detector** | Flag abnormal equipment sensor readings before they cause a stoppage | Isolation Forest (unsupervised), StandardScaler | 4 sensor features → anomaly / normal + anomaly score |
|
| 465 |
|
|
@@ -485,11 +484,12 @@ being realistic enough to demonstrate the underlying ML techniques properly.
|
|
| 485 |
## Tech stack
|
| 486 |
|
| 487 |
- **UI / deployment:** [Gradio](https://gradio.app) on Hugging Face Spaces
|
| 488 |
-
- **LLM:**
|
| 489 |
-
(`
|
| 490 |
-
|
| 491 |
-
|
| 492 |
-
|
|
|
|
| 493 |
- **Retrieval:** TF-IDF + cosine similarity over a small hand-written
|
| 494 |
warehouse-operations knowledge base (simple, fast, fully local RAG).
|
| 495 |
- **Intent classification:** TF-IDF + Logistic Regression (scikit-learn) --
|
|
@@ -517,7 +517,7 @@ being realistic enough to demonstrate the underlying ML techniques properly.
|
|
| 517 |
│ top-k passages
|
| 518 |
▼
|
| 519 |
┌───────────────────────┐
|
| 520 |
-
│ Hosted LLM │ (
|
| 521 |
│ (or extractive │
|
| 522 |
│ fallback if offline) │
|
| 523 |
└──────────┬────────────┘
|
|
@@ -763,12 +763,12 @@ with gr.Blocks(theme=gr.themes.Soft(primary_hue="blue"), css=CUSTOM_CSS, title="
|
|
| 763 |
gr.Markdown("<h1 id='title-banner'>🏭 Smart Warehouse AI Assistant</h1>")
|
| 764 |
subtitle_md = gr.Markdown(I18N["subtitle"]["en"])
|
| 765 |
lang_radio = gr.Radio(["English", "日本語"], value="English", label="🌐 Language / 言語", interactive=True)
|
| 766 |
-
if not
|
| 767 |
gr.Markdown(
|
| 768 |
-
"> ⚠️ **No
|
| 769 |
-
"
|
| 770 |
-
|
| 771 |
-
"
|
| 772 |
)
|
| 773 |
|
| 774 |
with gr.Tab("💬 AI Assistant"):
|
|
|
|
| 5 |
Space. Combines:
|
| 6 |
|
| 7 |
1. An LLM-powered assistant (RAG: TF-IDF retrieval + hosted LLM via the
|
| 8 |
+
an OpenAI-compatible API) for natural-language warehouse operations Q&A.
|
| 9 |
2. A TF-IDF + Logistic Regression intent classifier that routes queries
|
| 10 |
into 8 operational categories.
|
| 11 |
3. A lightweight NL -> structured query layer over synthetic inventory /
|
|
|
|
| 36 |
from src.data_generation import generate_inventory_db, generate_orders_db
|
| 37 |
from src.intent_model import INTENT_DESCRIPTIONS, load_pipeline, predict as intent_predict
|
| 38 |
from src.inventory_db import query_inventory, query_orders
|
| 39 |
+
from src.llm_client import answer_query, test_connection, llm_configured
|
| 40 |
from src.retriever import KBRetriever
|
| 41 |
|
| 42 |
ROOT = os.path.dirname(os.path.abspath(__file__))
|
|
|
|
| 112 |
latency_eval = _load_json("latency_eval.json")
|
| 113 |
vision_eval = _load_json("vision_eval.json")
|
| 114 |
|
| 115 |
+
LLM_CONFIGURED = llm_configured()
|
| 116 |
|
| 117 |
# --------------------------------------------------------------------------
|
| 118 |
# ZeroGPU compatibility shim
|
| 119 |
# --------------------------------------------------------------------------
|
| 120 |
# This app is CPU-only by design (scikit-learn locally, LLM calls go to the
|
| 121 |
+
# remote LLM API). Some Spaces accounts, however, only offer the
|
| 122 |
# free "ZeroGPU" hardware tier, which requires at least one function
|
| 123 |
# decorated with `@spaces.GPU` to be present or the platform's startup
|
| 124 |
# check fails with "No @spaces.GPU function detected". This is a harmless,
|
|
|
|
| 171 |
f"✅ **LLM connection working.** Model: `{response.model_id}` · "
|
| 172 |
f"{response.latency_s * 1000:.0f} ms\n\nSample answer: {response.answer}"
|
| 173 |
)
|
| 174 |
+
err_lines = "\n".join(f"- {e}" for e in response.debug_errors) or "(no error detail captured)"
|
| 175 |
return (
|
| 176 |
"❌ **LLM connection failed** — running in retrieval-only fallback mode.\n\n"
|
| 177 |
+
f"**What each configured backend reported:**\n{err_lines}\n\n"
|
| 178 |
+
"**Most reliable fix:** add a `GROQ_API_KEY` Space secret (free key at console.groq.com), then "
|
| 179 |
+
"restart the Space. Hugging Face's router needs inference credits, and free accounts get almost none "
|
| 180 |
+
"(that's the `402 Payment Required` error). Any OpenAI-compatible API also works via "
|
| 181 |
+
"`LLM_API_KEY` + `LLM_BASE_URL` + `LLM_MODEL`."
|
|
|
|
| 182 |
)
|
| 183 |
|
| 184 |
|
|
|
|
| 458 |
|---|---|---|---|
|
| 459 |
| **Intent classifier** | Decide what kind of request a query is (maintenance, safety, inventory, order status, navigation, picking, system status, general FAQ) | TF-IDF + Logistic Regression | Free text → intent label + confidence |
|
| 460 |
| **KB retriever** | Find the most relevant knowledge-base passage(s) for a query, to ground the LLM's answer | TF-IDF + cosine similarity | Free text → top-k ranked passages |
|
| 461 |
+
| **LLM assistant** | Generate a natural-language answer grounded in the retrieved context | Hosted instruct LLM via an OpenAI-compatible API (Groq / custom / HF router), with fallback | Query + context → grounded answer (or an extractive fallback if the LLM is unavailable) |
|
| 462 |
| **Inventory/order query** | Turn a question into a filtered table lookup | Regex slot extraction (SKU / order ID / zone) + intent routing | Free text → filtered inventory or orders table |
|
| 463 |
| **Anomaly detector** | Flag abnormal equipment sensor readings before they cause a stoppage | Isolation Forest (unsupervised), StandardScaler | 4 sensor features → anomaly / normal + anomaly score |
|
| 464 |
|
|
|
|
| 484 |
## Tech stack
|
| 485 |
|
| 486 |
- **UI / deployment:** [Gradio](https://gradio.app) on Hugging Face Spaces
|
| 487 |
+
- **LLM:** any hosted model behind an **OpenAI-compatible chat API** — Groq's
|
| 488 |
+
free tier (`GROQ_API_KEY`), a custom endpoint (`LLM_API_KEY` / `LLM_BASE_URL` /
|
| 489 |
+
`LLM_MODEL`), or Hugging Face's inference router. Plain HTTP rather than a
|
| 490 |
+
provider-specific SDK, so there's no provider list to go stale. It stops after
|
| 491 |
+
one attempt on a 402/401 (every model would fail the same way) and falls back
|
| 492 |
+
to a retrieval-only answer, so the public demo never just breaks.
|
| 493 |
- **Retrieval:** TF-IDF + cosine similarity over a small hand-written
|
| 494 |
warehouse-operations knowledge base (simple, fast, fully local RAG).
|
| 495 |
- **Intent classification:** TF-IDF + Logistic Regression (scikit-learn) --
|
|
|
|
| 517 |
│ top-k passages
|
| 518 |
▼
|
| 519 |
┌───────────────────────┐
|
| 520 |
+
│ Hosted LLM │ (OpenAI-compatible API)
|
| 521 |
│ (or extractive │
|
| 522 |
│ fallback if offline) │
|
| 523 |
└──────────┬────────────┘
|
|
|
|
| 763 |
gr.Markdown("<h1 id='title-banner'>🏭 Smart Warehouse AI Assistant</h1>")
|
| 764 |
subtitle_md = gr.Markdown(I18N["subtitle"]["en"])
|
| 765 |
lang_radio = gr.Radio(["English", "日本語"], value="English", label="🌐 Language / 言語", interactive=True)
|
| 766 |
+
if not LLM_CONFIGURED:
|
| 767 |
gr.Markdown(
|
| 768 |
+
"> ⚠️ **No LLM configured.** The AI Assistant tab runs in **retrieval-only fallback mode** "
|
| 769 |
+
"(still functional, just not LLM-generated prose). To enable full answers, add a Space secret "
|
| 770 |
+
"`GROQ_API_KEY` (free key at console.groq.com) — or `LLM_API_KEY` + `LLM_BASE_URL` + `LLM_MODEL` "
|
| 771 |
+
"for any OpenAI-compatible API, or an `HF_TOKEN` with Hugging Face inference credits."
|
| 772 |
)
|
| 773 |
|
| 774 |
with gr.Tab("💬 AI Assistant"):
|
src/llm_client.py
CHANGED
|
@@ -1,72 +1,76 @@
|
|
| 1 |
"""
|
| 2 |
llm_client.py
|
| 3 |
-------------
|
| 4 |
-
|
| 5 |
-
|
| 6 |
-
|
| 7 |
-
|
| 8 |
-
|
| 9 |
-
|
| 10 |
-
|
| 11 |
-
|
| 12 |
-
|
| 13 |
-
|
| 14 |
-
|
| 15 |
-
|
| 16 |
-
|
| 17 |
-
|
| 18 |
-
|
| 19 |
-
|
| 20 |
-
|
| 21 |
-
|
| 22 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 23 |
"""
|
| 24 |
|
| 25 |
import os
|
|
|
|
| 26 |
import time
|
| 27 |
from dataclasses import dataclass, field
|
| 28 |
-
from typing import List, Optional
|
|
|
|
|
|
|
| 29 |
|
| 30 |
from src.retriever import KBRetriever, RetrievedDoc
|
| 31 |
|
| 32 |
-
#
|
| 33 |
-
#
|
| 34 |
-
#
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 35 |
MODEL_CANDIDATES = [
|
| 36 |
"Qwen/Qwen2.5-7B-Instruct",
|
| 37 |
"meta-llama/Llama-3.1-8B-Instruct",
|
| 38 |
"meta-llama/Llama-3.2-3B-Instruct",
|
| 39 |
"mistralai/Mistral-7B-Instruct-v0.3",
|
| 40 |
-
"microsoft/Phi-3.5-mini-instruct",
|
| 41 |
-
"HuggingFaceH4/zephyr-7b-beta",
|
| 42 |
]
|
| 43 |
-
|
| 44 |
_env_model = os.environ.get("LLM_MODEL_ID")
|
| 45 |
if _env_model:
|
| 46 |
MODEL_CANDIDATES = [_env_model] + [m for m in MODEL_CANDIDATES if m != _env_model]
|
| 47 |
-
|
| 48 |
DEFAULT_MODEL_ID = MODEL_CANDIDATES[0]
|
| 49 |
|
| 50 |
-
# The token is normally
|
| 51 |
-
# common alternate secret names in case the Space was set up with a
|
| 52 |
-
# different name. First one found wins. Add your own name here if needed.
|
| 53 |
TOKEN_ENV_VAR_CANDIDATES = [
|
| 54 |
-
"HF_TOKEN",
|
| 55 |
-
"
|
| 56 |
-
"HUGGINGFACE_TOKEN",
|
| 57 |
-
"HUGGINGFACEHUB_API_TOKEN",
|
| 58 |
-
"HF_API_TOKEN",
|
| 59 |
-
"HUGGING_FACE_HUB_TOKEN",
|
| 60 |
-
"HF_ACCESS_TOKEN",
|
| 61 |
]
|
| 62 |
|
| 63 |
-
|
| 64 |
-
|
| 65 |
-
|
| 66 |
-
val = os.environ.get(var)
|
| 67 |
-
if val:
|
| 68 |
-
return val
|
| 69 |
-
return None
|
| 70 |
|
| 71 |
SYSTEM_PROMPT = (
|
| 72 |
"You are the Smart Warehouse AI Assistant, a helpful operations copilot "
|
|
@@ -74,10 +78,130 @@ SYSTEM_PROMPT = (
|
|
| 74 |
"sortation, and a WMS). Answer concisely and practically, in the tone of "
|
| 75 |
"an experienced warehouse operations engineer. Use the provided CONTEXT "
|
| 76 |
"when relevant, and say so plainly if the question is outside the "
|
| 77 |
-
"context. Prefer short paragraphs or bullet points over long prose."
|
|
|
|
| 78 |
)
|
| 79 |
|
| 80 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 81 |
@dataclass
|
| 82 |
class AssistantResponse:
|
| 83 |
answer: str
|
|
@@ -85,35 +209,23 @@ class AssistantResponse:
|
|
| 85 |
sources: List[RetrievedDoc]
|
| 86 |
latency_s: float
|
| 87 |
model_id: str
|
| 88 |
-
debug_errors: List[str] = field(default_factory=list) #
|
| 89 |
|
| 90 |
|
| 91 |
def _extractive_fallback(query: str, sources: List[RetrievedDoc]) -> str:
|
| 92 |
-
"""Deterministic answer used when no
|
| 93 |
-
Space always returns something useful instead of an error."""
|
| 94 |
if not sources:
|
| 95 |
-
return (
|
| 96 |
-
|
| 97 |
-
|
| 98 |
-
"picking strategy, safety incidents, or general warehouse "
|
| 99 |
-
"automation concepts."
|
| 100 |
-
)
|
| 101 |
-
lead = sources[0]
|
| 102 |
bullets = "\n".join(f"- **{s.title}**: {s.text}" for s in sources)
|
| 103 |
-
return (
|
| 104 |
-
|
| 105 |
-
f"see the diagnostics below.)\n\n"
|
| 106 |
-
f"Based on **{lead.title}**, here's the relevant information:\n\n{bullets}"
|
| 107 |
-
)
|
| 108 |
-
|
| 109 |
-
|
| 110 |
-
MAX_HISTORY_TURNS = 4 # prior (user, assistant) pairs carried into the LLM call
|
| 111 |
|
| 112 |
|
| 113 |
def _clean_history(history: Optional[List[dict]]) -> List[dict]:
|
| 114 |
-
"""Normalize Gradio's messages-format history
|
| 115 |
-
|
| 116 |
-
turns and keeps only the last MAX_HISTORY_TURNS pairs (bounded prompt)."""
|
| 117 |
if not history:
|
| 118 |
return []
|
| 119 |
cleaned = []
|
|
@@ -135,82 +247,44 @@ def answer_query(
|
|
| 135 |
max_tokens: int = 350,
|
| 136 |
history: Optional[List[dict]] = None,
|
| 137 |
) -> AssistantResponse:
|
| 138 |
-
"""`history`: prior turns (Gradio messages format)
|
| 139 |
-
|
| 140 |
start = time.time()
|
| 141 |
sources = retriever.retrieve(query, k=k)
|
| 142 |
context_block = "\n\n".join(f"[{s.title}]\n{s.text}" for s in sources)
|
| 143 |
-
prior_turns = _clean_history(history)
|
| 144 |
-
|
| 145 |
-
hf_token = _get_hf_token()
|
| 146 |
-
|
| 147 |
-
if not hf_token:
|
| 148 |
-
answer = _extractive_fallback(query, sources)
|
| 149 |
-
return AssistantResponse(
|
| 150 |
-
answer=answer,
|
| 151 |
-
used_llm=False,
|
| 152 |
-
sources=sources,
|
| 153 |
-
latency_s=time.time() - start,
|
| 154 |
-
model_id="extractive-fallback",
|
| 155 |
-
debug_errors=[
|
| 156 |
-
"No HF token secret found. Checked env vars: "
|
| 157 |
-
+ ", ".join(TOKEN_ENV_VAR_CANDIDATES)
|
| 158 |
-
+ ". Set one of these as a Space secret (Settings -> Variables and secrets)."
|
| 159 |
-
],
|
| 160 |
-
)
|
| 161 |
-
|
| 162 |
-
try:
|
| 163 |
-
from huggingface_hub import InferenceClient
|
| 164 |
-
except ImportError as e:
|
| 165 |
-
answer = _extractive_fallback(query, sources)
|
| 166 |
-
return AssistantResponse(
|
| 167 |
-
answer=answer, used_llm=False, sources=sources,
|
| 168 |
-
latency_s=time.time() - start, model_id="extractive-fallback",
|
| 169 |
-
debug_errors=[f"huggingface_hub not importable: {e}"],
|
| 170 |
-
)
|
| 171 |
|
| 172 |
messages = [{"role": "system", "content": SYSTEM_PROMPT}]
|
| 173 |
-
messages.extend(
|
| 174 |
messages.append({"role": "user", "content": f"CONTEXT:\n{context_block}\n\nQUESTION: {query}"})
|
| 175 |
|
| 176 |
-
|
| 177 |
-
|
| 178 |
-
|
| 179 |
-
|
| 180 |
-
|
| 181 |
-
|
| 182 |
-
|
| 183 |
-
|
| 184 |
-
|
| 185 |
-
|
| 186 |
-
|
| 187 |
-
|
| 188 |
-
|
| 189 |
-
|
| 190 |
-
|
| 191 |
-
|
| 192 |
-
|
| 193 |
-
|
| 194 |
-
|
| 195 |
-
|
| 196 |
-
|
| 197 |
-
|
| 198 |
-
|
| 199 |
-
|
| 200 |
-
|
| 201 |
-
answer = _extractive_fallback(query, sources)
|
| 202 |
-
return AssistantResponse(
|
| 203 |
-
answer=answer,
|
| 204 |
-
used_llm=False,
|
| 205 |
-
sources=sources,
|
| 206 |
-
latency_s=time.time() - start,
|
| 207 |
-
model_id="extractive-fallback",
|
| 208 |
-
debug_errors=errors,
|
| 209 |
-
)
|
| 210 |
|
| 211 |
|
| 212 |
def test_connection(retriever: Optional[KBRetriever] = None) -> AssistantResponse:
|
| 213 |
-
"""
|
| 214 |
-
'Test LLM connection' diagnostics button in the app."""
|
| 215 |
retriever = retriever or KBRetriever()
|
| 216 |
return answer_query("What is a WMS?", retriever, k=1, max_tokens=60)
|
|
|
|
| 1 |
"""
|
| 2 |
llm_client.py
|
| 3 |
-------------
|
| 4 |
+
Answers warehouse-operations questions with a hosted LLM, grounded in context
|
| 5 |
+
retrieved from the local knowledge base (simple RAG), with multi-turn memory.
|
| 6 |
+
|
| 7 |
+
Why this talks plain HTTP instead of `huggingface_hub.InferenceClient`
|
| 8 |
+
---------------------------------------------------------------------
|
| 9 |
+
An earlier version used InferenceClient(provider="auto"). That broke twice in
|
| 10 |
+
the wild: (1) Hugging Face's router started assigning models to providers
|
| 11 |
+
('featherless-ai', 'deepinfra') that the pinned client version didn't know,
|
| 12 |
+
raising "Provider ... not supported"; (2) the account had no Inference
|
| 13 |
+
Providers credits, giving HTTP 402. Everything modern (Groq, OpenRouter,
|
| 14 |
+
Together, Cerebras, HF's own router, ...) exposes the same OpenAI-compatible
|
| 15 |
+
`/chat/completions` protocol, so this module speaks that directly. There is
|
| 16 |
+
no provider enum to go stale, and any such endpoint can be used by setting
|
| 17 |
+
env vars / Space secrets.
|
| 18 |
+
|
| 19 |
+
Backends, tried in this order (first success wins)
|
| 20 |
+
--------------------------------------------------
|
| 21 |
+
1. Custom OpenAI-compatible endpoint:
|
| 22 |
+
LLM_API_KEY + LLM_BASE_URL + LLM_MODEL
|
| 23 |
+
or just `GROQ_API_KEY` (Groq has a free plan; base URL and a default model
|
| 24 |
+
are filled in; override the model with LLM_MODEL).
|
| 25 |
+
2. Hugging Face Inference Providers router (https://router.huggingface.co/v1),
|
| 26 |
+
using the HF token. NOTE: free HF accounts get very little monthly credit;
|
| 27 |
+
when it runs out the router answers 402 and this backend is skipped.
|
| 28 |
+
3. Extractive fallback: the retrieved knowledge-base passages are shown
|
| 29 |
+
directly, clearly labelled -- the demo never breaks.
|
| 30 |
+
|
| 31 |
+
Failure handling: a 402/401/403 means *every* model on that backend will fail
|
| 32 |
+
the same way, so the backend is abandoned after one attempt instead of
|
| 33 |
+
hammering six models. Error text never includes API keys.
|
| 34 |
"""
|
| 35 |
|
| 36 |
import os
|
| 37 |
+
import re
|
| 38 |
import time
|
| 39 |
from dataclasses import dataclass, field
|
| 40 |
+
from typing import List, Optional, Tuple
|
| 41 |
+
|
| 42 |
+
import requests
|
| 43 |
|
| 44 |
from src.retriever import KBRetriever, RetrievedDoc
|
| 45 |
|
| 46 |
+
# ---------------------------------------------------------------------------
|
| 47 |
+
# Configuration
|
| 48 |
+
# ---------------------------------------------------------------------------
|
| 49 |
+
GROQ_BASE_URL = "https://api.groq.com/openai/v1"
|
| 50 |
+
GROQ_DEFAULT_MODEL = "llama-3.1-8b-instant"
|
| 51 |
+
HF_ROUTER_BASE_URL = os.environ.get("HF_ROUTER_BASE_URL", "https://router.huggingface.co/v1")
|
| 52 |
+
|
| 53 |
+
# Models tried on the Hugging Face router, in order (LLM_MODEL_ID goes first if set).
|
| 54 |
MODEL_CANDIDATES = [
|
| 55 |
"Qwen/Qwen2.5-7B-Instruct",
|
| 56 |
"meta-llama/Llama-3.1-8B-Instruct",
|
| 57 |
"meta-llama/Llama-3.2-3B-Instruct",
|
| 58 |
"mistralai/Mistral-7B-Instruct-v0.3",
|
|
|
|
|
|
|
| 59 |
]
|
|
|
|
| 60 |
_env_model = os.environ.get("LLM_MODEL_ID")
|
| 61 |
if _env_model:
|
| 62 |
MODEL_CANDIDATES = [_env_model] + [m for m in MODEL_CANDIDATES if m != _env_model]
|
|
|
|
| 63 |
DEFAULT_MODEL_ID = MODEL_CANDIDATES[0]
|
| 64 |
|
| 65 |
+
# The HF token is normally `HF_TOKEN`; a few other names are accepted too.
|
|
|
|
|
|
|
| 66 |
TOKEN_ENV_VAR_CANDIDATES = [
|
| 67 |
+
"HF_TOKEN", "Smart_Warehouse", "HUGGINGFACE_TOKEN", "HUGGINGFACEHUB_API_TOKEN",
|
| 68 |
+
"HF_API_TOKEN", "HUGGING_FACE_HUB_TOKEN", "HF_ACCESS_TOKEN",
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 69 |
]
|
| 70 |
|
| 71 |
+
REQUEST_TIMEOUT_S = 40
|
| 72 |
+
TOTAL_BUDGET_S = 75 # stop trying more candidates after this long (UI shouldn't hang)
|
| 73 |
+
MAX_HISTORY_TURNS = 4 # prior (user, assistant) pairs carried into the call
|
|
|
|
|
|
|
|
|
|
|
|
|
| 74 |
|
| 75 |
SYSTEM_PROMPT = (
|
| 76 |
"You are the Smart Warehouse AI Assistant, a helpful operations copilot "
|
|
|
|
| 78 |
"sortation, and a WMS). Answer concisely and practically, in the tone of "
|
| 79 |
"an experienced warehouse operations engineer. Use the provided CONTEXT "
|
| 80 |
"when relevant, and say so plainly if the question is outside the "
|
| 81 |
+
"context. Prefer short paragraphs or bullet points over long prose. "
|
| 82 |
+
"Reply in the same language the user wrote in."
|
| 83 |
)
|
| 84 |
|
| 85 |
|
| 86 |
+
def _get_hf_token() -> Optional[str]:
|
| 87 |
+
for var in TOKEN_ENV_VAR_CANDIDATES:
|
| 88 |
+
val = os.environ.get(var)
|
| 89 |
+
if val:
|
| 90 |
+
return val
|
| 91 |
+
return None
|
| 92 |
+
|
| 93 |
+
|
| 94 |
+
@dataclass
|
| 95 |
+
class Backend:
|
| 96 |
+
label: str
|
| 97 |
+
base_url: str
|
| 98 |
+
api_key: str
|
| 99 |
+
models: List[str]
|
| 100 |
+
|
| 101 |
+
|
| 102 |
+
def get_backends() -> List[Backend]:
|
| 103 |
+
"""Backends configured via env vars / Space secrets, in priority order."""
|
| 104 |
+
backends: List[Backend] = []
|
| 105 |
+
key, url, model = os.environ.get("LLM_API_KEY"), os.environ.get("LLM_BASE_URL"), os.environ.get("LLM_MODEL")
|
| 106 |
+
if key and url:
|
| 107 |
+
host = re.sub(r"^https?://", "", url).split("/")[0]
|
| 108 |
+
backends.append(Backend(f"custom ({host})", url.rstrip("/"), key, [model] if model else []))
|
| 109 |
+
groq_key = os.environ.get("GROQ_API_KEY")
|
| 110 |
+
if groq_key:
|
| 111 |
+
backends.append(Backend("Groq", GROQ_BASE_URL, groq_key, [model or GROQ_DEFAULT_MODEL]))
|
| 112 |
+
hf = _get_hf_token()
|
| 113 |
+
if hf:
|
| 114 |
+
backends.append(Backend("Hugging Face router", HF_ROUTER_BASE_URL.rstrip("/"), hf, list(MODEL_CANDIDATES)))
|
| 115 |
+
return backends
|
| 116 |
+
|
| 117 |
+
|
| 118 |
+
def llm_configured() -> bool:
|
| 119 |
+
return bool(get_backends())
|
| 120 |
+
|
| 121 |
+
|
| 122 |
+
# ---------------------------------------------------------------------------
|
| 123 |
+
# Wire protocol (OpenAI-compatible /chat/completions)
|
| 124 |
+
# ---------------------------------------------------------------------------
|
| 125 |
+
class LLMError(Exception):
|
| 126 |
+
"""kind: credits | auth | rate | not_found | server | network | parse | empty"""
|
| 127 |
+
def __init__(self, kind: str, message: str, status: Optional[int] = None):
|
| 128 |
+
super().__init__(message)
|
| 129 |
+
self.kind, self.status = kind, status
|
| 130 |
+
|
| 131 |
+
|
| 132 |
+
# Failures after which trying more models on the SAME backend is pointless.
|
| 133 |
+
BACKEND_FATAL = {"credits", "auth"}
|
| 134 |
+
|
| 135 |
+
|
| 136 |
+
def _scrub(text: str, secret: str) -> str:
|
| 137 |
+
text = text.replace(secret, "***") if secret else text
|
| 138 |
+
return re.sub(r"(hf_|gsk_|sk-)[A-Za-z0-9_\-]{8,}", r"\1***", text)
|
| 139 |
+
|
| 140 |
+
|
| 141 |
+
def _error_message(resp: requests.Response) -> str:
|
| 142 |
+
try:
|
| 143 |
+
data = resp.json()
|
| 144 |
+
err = data.get("error", data)
|
| 145 |
+
if isinstance(err, dict):
|
| 146 |
+
return str(err.get("message") or err)[:240]
|
| 147 |
+
return str(err)[:240]
|
| 148 |
+
except Exception: # noqa: BLE001
|
| 149 |
+
return (resp.text or "")[:240].strip()
|
| 150 |
+
|
| 151 |
+
|
| 152 |
+
def _chat(backend: Backend, model: str, messages: list, max_tokens: int, temperature: float = 0.3) -> str:
|
| 153 |
+
try:
|
| 154 |
+
resp = requests.post(
|
| 155 |
+
f"{backend.base_url}/chat/completions",
|
| 156 |
+
headers={"Authorization": f"Bearer {backend.api_key}", "Content-Type": "application/json"},
|
| 157 |
+
json={"model": model, "messages": messages, "max_tokens": max_tokens, "temperature": temperature},
|
| 158 |
+
timeout=REQUEST_TIMEOUT_S,
|
| 159 |
+
)
|
| 160 |
+
except requests.exceptions.Timeout:
|
| 161 |
+
raise LLMError("network", f"timed out after {REQUEST_TIMEOUT_S}s")
|
| 162 |
+
except requests.exceptions.RequestException as e:
|
| 163 |
+
raise LLMError("network", _scrub(f"{type(e).__name__}: {e}", backend.api_key)[:200])
|
| 164 |
+
|
| 165 |
+
s = resp.status_code
|
| 166 |
+
if s == 200:
|
| 167 |
+
try:
|
| 168 |
+
content = resp.json()["choices"][0]["message"]["content"]
|
| 169 |
+
except Exception: # noqa: BLE001
|
| 170 |
+
raise LLMError("parse", "response was not a valid chat completion", s)
|
| 171 |
+
if not content or not str(content).strip():
|
| 172 |
+
raise LLMError("empty", "model returned an empty answer", s)
|
| 173 |
+
return str(content).strip()
|
| 174 |
+
|
| 175 |
+
detail = _scrub(_error_message(resp), backend.api_key)
|
| 176 |
+
if s == 402:
|
| 177 |
+
raise LLMError("credits", detail or "payment required", s)
|
| 178 |
+
if s in (401, 403):
|
| 179 |
+
raise LLMError("auth", detail or "unauthorized", s)
|
| 180 |
+
if s == 429:
|
| 181 |
+
raise LLMError("rate", detail or "rate limited", s)
|
| 182 |
+
if s in (400, 404, 410, 422):
|
| 183 |
+
raise LLMError("not_found", detail or "model/request not accepted", s)
|
| 184 |
+
raise LLMError("server", f"HTTP {s}: {detail}", s)
|
| 185 |
+
|
| 186 |
+
|
| 187 |
+
_HINTS = {
|
| 188 |
+
"credits": ("402 Payment Required — this account has no inference credits "
|
| 189 |
+
"(Hugging Face free accounts get almost none). Use a free-tier provider instead: "
|
| 190 |
+
"set a `GROQ_API_KEY` secret (free key at console.groq.com), or add credits."),
|
| 191 |
+
"auth": "key rejected (401/403) — check the secret's value and permissions.",
|
| 192 |
+
"rate": "rate limited (429) — the free tier's per-minute/day limit was hit; wait and retry.",
|
| 193 |
+
}
|
| 194 |
+
|
| 195 |
+
|
| 196 |
+
def _explain(backend: Backend, model: str, e: LLMError) -> str:
|
| 197 |
+
base = f"{backend.label} · {model or '(no model set)'}: {e.kind}"
|
| 198 |
+
hint = _HINTS.get(e.kind)
|
| 199 |
+
return f"{base} — {hint}" if hint else f"{base} — {e}"
|
| 200 |
+
|
| 201 |
+
|
| 202 |
+
# ---------------------------------------------------------------------------
|
| 203 |
+
# Public API
|
| 204 |
+
# ---------------------------------------------------------------------------
|
| 205 |
@dataclass
|
| 206 |
class AssistantResponse:
|
| 207 |
answer: str
|
|
|
|
| 209 |
sources: List[RetrievedDoc]
|
| 210 |
latency_s: float
|
| 211 |
model_id: str
|
| 212 |
+
debug_errors: List[str] = field(default_factory=list) # filled when used_llm is False
|
| 213 |
|
| 214 |
|
| 215 |
def _extractive_fallback(query: str, sources: List[RetrievedDoc]) -> str:
|
| 216 |
+
"""Deterministic answer used when no LLM is configured / reachable."""
|
|
|
|
| 217 |
if not sources:
|
| 218 |
+
return ("I don't have grounded context for that yet. Try asking about inventory, order status, "
|
| 219 |
+
"equipment maintenance, AGV routing, picking strategy, safety incidents, or general "
|
| 220 |
+
"warehouse automation concepts.")
|
|
|
|
|
|
|
|
|
|
|
|
|
| 221 |
bullets = "\n".join(f"- **{s.title}**: {s.text}" for s in sources)
|
| 222 |
+
return (f"(Showing retrieved knowledge instead of an LLM-generated answer -- see the diagnostics below.)\n\n"
|
| 223 |
+
f"Based on **{sources[0].title}**, here's the relevant information:\n\n{bullets}")
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 224 |
|
| 225 |
|
| 226 |
def _clean_history(history: Optional[List[dict]]) -> List[dict]:
|
| 227 |
+
"""Normalize Gradio's messages-format history: strip our own metadata footer from
|
| 228 |
+
assistant turns and keep only the last MAX_HISTORY_TURNS pairs (bounded prompt)."""
|
|
|
|
| 229 |
if not history:
|
| 230 |
return []
|
| 231 |
cleaned = []
|
|
|
|
| 247 |
max_tokens: int = 350,
|
| 248 |
history: Optional[List[dict]] = None,
|
| 249 |
) -> AssistantResponse:
|
| 250 |
+
"""`history`: prior turns (Gradio messages format) so follow-ups like
|
| 251 |
+
"what about Zone D instead?" work. Omit for single-turn."""
|
| 252 |
start = time.time()
|
| 253 |
sources = retriever.retrieve(query, k=k)
|
| 254 |
context_block = "\n\n".join(f"[{s.title}]\n{s.text}" for s in sources)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 255 |
|
| 256 |
messages = [{"role": "system", "content": SYSTEM_PROMPT}]
|
| 257 |
+
messages.extend(_clean_history(history))
|
| 258 |
messages.append({"role": "user", "content": f"CONTEXT:\n{context_block}\n\nQUESTION: {query}"})
|
| 259 |
|
| 260 |
+
backends = get_backends()
|
| 261 |
+
errors: List[str] = []
|
| 262 |
+
if not backends:
|
| 263 |
+
errors.append("No LLM configured. Set a `GROQ_API_KEY` secret (free at console.groq.com), "
|
| 264 |
+
"or `LLM_API_KEY` + `LLM_BASE_URL` + `LLM_MODEL` for any OpenAI-compatible API, "
|
| 265 |
+
"or an `HF_TOKEN` (needs Hugging Face inference credits).")
|
| 266 |
+
|
| 267 |
+
for backend in backends:
|
| 268 |
+
if not backend.models:
|
| 269 |
+
errors.append(f"{backend.label}: LLM_MODEL is not set.")
|
| 270 |
+
continue
|
| 271 |
+
for model in backend.models:
|
| 272 |
+
if time.time() - start > TOTAL_BUDGET_S:
|
| 273 |
+
errors.append(f"Stopped after {TOTAL_BUDGET_S}s without a successful answer.")
|
| 274 |
+
break
|
| 275 |
+
try:
|
| 276 |
+
text = _chat(backend, model, messages, max_tokens)
|
| 277 |
+
return AssistantResponse(text, True, sources, time.time() - start, f"{backend.label}: {model}")
|
| 278 |
+
except LLMError as e:
|
| 279 |
+
errors.append(_explain(backend, model, e))
|
| 280 |
+
if e.kind in BACKEND_FATAL:
|
| 281 |
+
break # every other model on this backend would fail identically
|
| 282 |
+
|
| 283 |
+
return AssistantResponse(_extractive_fallback(query, sources), False, sources,
|
| 284 |
+
time.time() - start, "extractive-fallback", debug_errors=errors)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 285 |
|
| 286 |
|
| 287 |
def test_connection(retriever: Optional[KBRetriever] = None) -> AssistantResponse:
|
| 288 |
+
"""One canned query through the full pipeline -- used by the diagnostics button."""
|
|
|
|
| 289 |
retriever = retriever or KBRetriever()
|
| 290 |
return answer_query("What is a WMS?", retriever, k=1, max_tokens=60)
|