feat(B-LLM): Zentraler LLM Client — llm_complete() + llm_embed() + Migration + Tests + Doku
Check Cross-Plugin Imports / check (push) Has been cancelled
Check Cross-Plugin Imports / check (push) Has been cancelled
B-LLM: llm_client.py um generische llm_complete() und llm_embed() erweitert - Provider-Auswahl, API-Key-Auflösung, Error-Handling, Cost-Tracking - Retry mit Exponential-Backoff für transient errors - Timeout konfigurierbar - Helper: get_api_credentials(), build_model(), _classify_error() B-LLM-MIG: Alle 8 direkten litellm.acompletion() Calls auf llm_complete() umgestellt - agent_runner.py, query_understanding.py (2x), ai_proactive (3x), ai_assistant (2x) - 0 verbleibende direkte litellm.acompletion() Calls außerhalb llm_client.py B-LLM-TEST: 39 Tests in test_llm_client.py — alle grün - Mock mode, error handling, embed, helpers, backward compat B-LLM-DOC: Plugin-Dev-Guide Kapitel 7 (LLM Integration) hinzugefügt
This commit is contained in:
@@ -850,4 +850,118 @@ class EventExamplePlugin(BasePlugin):
|
||||
|
||||
---
|
||||
|
||||
## 7. LLM Integration
|
||||
|
||||
LeoCRM stellt einen zentralen LLM-Client bereit über den alle LLM-Calls (Completion und Embedding) laufen. **Keine direkten `litellm.acompletion()` oder `litellm.aembedding()` Aufrufe in Plugin-Code.**
|
||||
|
||||
### 7.1 Completion
|
||||
|
||||
```python
|
||||
from app.ai.llm_client import llm_complete
|
||||
|
||||
result = await llm_complete(
|
||||
model="openai/gpt-4o", # oder None für Default-Model
|
||||
messages=[
|
||||
{"role": "system", "content": "You are a helpful assistant."},
|
||||
{"role": "user", "content": "Summarize this email."},
|
||||
],
|
||||
temperature=0.3,
|
||||
max_tokens=1000,
|
||||
# Optional: API-Key/Base aus DB holen
|
||||
db=db,
|
||||
tenant_id=tenant_id,
|
||||
# Optional: JSON-Response erzwingen
|
||||
response_format={"type": "json_object"},
|
||||
# Optional: Tools für Function-Calling
|
||||
tools=[{"type": "function", "function": {...}}],
|
||||
# Optional: Retry-Konfiguration
|
||||
timeout=30,
|
||||
max_retries=2,
|
||||
)
|
||||
|
||||
content = result["content"] # str — LLM-Response-Text
|
||||
usage = result["usage"] # dict — {prompt_tokens, completion_tokens, total_tokens}
|
||||
cost_usd = result["cost_usd"] # float — geschätzte Kosten
|
||||
model = result["model"] # str — verwendetes Modell
|
||||
raw_response = result["raw_response"] # litellm-Response-Objekt für erweiterte Nutzung
|
||||
```
|
||||
|
||||
### 7.2 Embedding
|
||||
|
||||
```python
|
||||
from app.ai.llm_client import llm_embed
|
||||
|
||||
# Einzelne Embedding
|
||||
embeddings = await llm_embed(
|
||||
texts="Text to embed",
|
||||
model="openai/text-embedding-3-small", # oder None für Default
|
||||
db=db,
|
||||
tenant_id=tenant_id,
|
||||
dimensions=768, # Optional, für text-embedding-3 Modelle
|
||||
)
|
||||
# → [[0.01, 0.02, ...]]
|
||||
|
||||
# Batch-Embedding
|
||||
embeddings = await llm_embed(
|
||||
texts=["Text 1", "Text 2", "Text 3"],
|
||||
db=db,
|
||||
tenant_id=tenant_id,
|
||||
)
|
||||
# → [[...], [...], [...]]
|
||||
```
|
||||
|
||||
### 7.3 Provider-Auswahl und API-Key-Auflösung
|
||||
|
||||
Der zentrale Client löst API-Keys automatisch aus der Datenbank (`AIProvider`-Tabelle) oder Environment-Variablen. Priorität:
|
||||
|
||||
1. Explizit übergebener `api_key` Parameter
|
||||
2. DB-Lookup über `get_api_credentials(db, tenant_id)`
|
||||
3. Environment-Variablen (`AI_API_KEY`, `AI_API_BASE`, `AI_PROVIDER`)
|
||||
4. Mock-Mode (kein API-Key → Keyword-basierte Fallback-Antworten)
|
||||
|
||||
```python
|
||||
from app.ai.llm_client import get_api_credentials, build_model
|
||||
|
||||
# API-Credentials aus DB holen
|
||||
api_key, api_base, provider_type = await get_api_credentials(db, tenant_id)
|
||||
|
||||
# Model-String bauen (provider/model)
|
||||
model = build_model("gpt-4o", provider_type) # → "openai/gpt-4o"
|
||||
```
|
||||
|
||||
### 7.4 Error-Handling
|
||||
|
||||
Der zentrale Client klassifiziert Errors automatisch:
|
||||
|
||||
- **Transient** (Timeout, Rate-Limit 429, Service-Unavailable 503) → Retry mit Exponential-Backoff
|
||||
- **Permanent** (Auth 401/403, Validation, Model-Not-Found) → Sofortiger Fehler, kein Retry
|
||||
|
||||
```python
|
||||
try:
|
||||
result = await llm_complete(model="openai/gpt-4o", messages=[...])
|
||||
except Exception as e:
|
||||
# Transient errors wurden bereits retried
|
||||
# Permanent errors kommen hier an
|
||||
logger.error(f"LLM call failed permanently: {e}")
|
||||
```
|
||||
|
||||
### 7.5 Cost-Tracking
|
||||
|
||||
`llm_complete()` gibt `cost_usd` zurück — automatisch berechnet aus Token-Usage. Plugins sollen diesen Wert in ihren Cost-Tracking-Mechanismus übernehmen.
|
||||
|
||||
```python
|
||||
result = await llm_complete(...)
|
||||
total_cost += result["cost_usd"]
|
||||
```
|
||||
|
||||
### 7.6 Was NICHT zu tun ist
|
||||
|
||||
- ❌ `import litellm` und direkte `litellm.acompletion()` / `litellm.aembedding()` Aufrufe
|
||||
- ❌ Eigene API-Key-Verwaltung — immer über `get_api_credentials()` oder `llm_complete(db=db, tenant_id=tenant_id)`
|
||||
- ❌ Eigene Retry-Logik — `llm_complete()` hat bereits Retry mit Backoff
|
||||
- ❌ Eigene Cost-Tracking-Logik — `llm_complete()` gibt `cost_usd` zurück
|
||||
- ❌ Eigene Provider-Auswahl — `build_model()` und `get_api_credentials()` zentralisieren das
|
||||
|
||||
---
|
||||
|
||||
*This document is authoritative for all plugin development at LeoCRM.*
|
||||
|
||||
Reference in New Issue
Block a user