feat(F): F-TEST + F-DOC + F-UI-TRIG — Phase F complete!
- F-TEST: tests/test_phase_f_agents.py (1425 lines, 45 tests, all pass) — ReAct Loop, Permissions, Approvals, Skills, Context Builder, Data Policy, Transparency, Workstream, Budget - F-DOC: docs/api-documentation.md (Phase F endpoints), docs/plugin-development-guide.md (Agent chapter 32), docs/test-strategy.md (Phase F test conventions) - F-UI-TRIG: trigger_dispatcher dispatches agents on ui.*/context.* events (already implemented in F-PROACTIVE) - Bug fix: approval.py metadata reserved attribute renamed to request_metadata - PROGRESS.md: Phase F marked done, ~155/223 tasks done (70%)
This commit is contained in:
@@ -343,6 +343,59 @@ with patch("app.plugins.builtins.unified_search.embedding.llm_embed", new_callab
|
||||
|
||||
**Regeln:**
|
||||
- `llm_complete` liefert ein Dict mit `normalized_query`, `facets`, `summary` (und optional `suggestions`).
|
||||
|
||||
## Phase F — Agent-System Test-Konventionen
|
||||
|
||||
### Neue Test-Datei: `tests/test_phase_f_agents.py` (45 Tests)
|
||||
|
||||
| Test-Gruppe | Tests | Status |
|
||||
|-------------|-------|--------|
|
||||
| ReAct Loop (Multi-Step, max_steps, Timeout, Error-Recovery, Cost, Dry-Run, Audit) | 8 | ✅ |
|
||||
| Agent-Permissions (Intersection, System-Admin, Visibility, Execute, Optimistic Lock) | 9 | ✅ |
|
||||
| Approval Requests (Create, Approve/Reject/Expire, List-Filter) | 6 | ✅ |
|
||||
| Skill Registry (Registration, get_by_names, keine Permission-Grants) | 3 | ✅ |
|
||||
| Context Builder (System-Prompt, ReAct-Format, Sensitive-Fields, Tool-Descriptions) | 5 | ✅ |
|
||||
| Data Policy (Sensitive-Fields, Provider-Compliance, Allowed-Categories) | 4 | ✅ |
|
||||
| Transparency (AI-Generated-Marking, AI-Participant-Erkennung) | 2 | ✅ |
|
||||
| Workstream (Message, Step, Result) | 3 | ✅ |
|
||||
| Budget Limits (Run-Stopp bei Budget, Cost-Akkumulation) | 2 | ✅ |
|
||||
|
||||
### Konventionen für Agent-Tests
|
||||
|
||||
1. **Keine echte DB / kein echtes LLM / kein Redis:** Alle externen Abhängigkeiten werden mit `AsyncMock` / `MagicMock` gemockt. Die Tests überschreiben die `conftest`-Fixtures `db_setup` und `clean_tables` mit No-Op-Fixtures, damit kein PostgreSQL/Redis benötigt wird.
|
||||
2. **Patch-Targets am Ursprungsmodul:** Funktionen, die innerhalb einer Funktion importiert werden, müssen am Ursprungsmodul gepatcht werden. Beispiel: `get_provider_compliance` wird in `enforce_data_policy` aus `app.ai.llm_client` importiert → Patch auf `app.ai.llm_client.get_provider_compliance`, nicht `app.ai.data_policy.get_provider_compliance`.
|
||||
3. **Mock-LLM-Responses:** `llm_complete` wird mit `AsyncMock` gemockt und liefert Dicts mit `content`, `usage`, `cost_usd`, `model`, `raw_response` (mit `choices[0].message.content` und `message.tool_calls`).
|
||||
4. **Tool-Calls:** Mock-Tool-Calls haben `id`, `function.name`, `function.arguments` (JSON-String). Tool-Handler werden als `AsyncMock` registriert.
|
||||
5. **Keine zufälligen UUIDs in Assertions:** Echte Entity-IDs aus Mocks verwenden; UUIDs nur als generierte Test-IDs.
|
||||
6. **SQLAlchemy-Modelle in Mocks:** Für `select(model.version)` in Optimistic-Lock-Tests `sqlalchemy.column()` verwenden, nicht Plain-Strings (sonst `ArgumentError`).
|
||||
7. **Reservierte Attributnamen:** SQLAlchemy-Modelle dürfen kein `metadata`-Attribut haben (reserviert in der Declarative API). `ApprovalRequest` nutzt `request_metadata` mit DB-Spaltenname `metadata` via `mapped_column("metadata", ...)`.
|
||||
|
||||
### Mock-Patterns für Agent-Tests
|
||||
|
||||
```python
|
||||
from unittest.mock import AsyncMock, MagicMock, patch
|
||||
|
||||
# LLM-Call
|
||||
with patch("app.ai.agent_loop.llm_complete", new_callable=AsyncMock) as mock_llm:
|
||||
mock_llm.return_value = {
|
||||
"content": "Final answer",
|
||||
"usage": {"total_tokens": 100},
|
||||
"cost_usd": 0.001,
|
||||
"model": "gpt-4o",
|
||||
"raw_response": raw_response,
|
||||
}
|
||||
# ... Test
|
||||
|
||||
# Provider-Compliance (in data_policy importiert aus llm_client)
|
||||
with patch("app.ai.llm_client.get_provider_compliance", new_callable=AsyncMock) as mock_compliance:
|
||||
mock_compliance.return_value = {"allowed_data_classes": ["internal"]}
|
||||
# ... Test
|
||||
|
||||
# Tool-Handler
|
||||
handler = AsyncMock(return_value="result")
|
||||
registry = MagicMock()
|
||||
registry.get = lambda name: MagicMock(handler=handler) if name == "search" else None
|
||||
```
|
||||
- `generate_embedding` / `llm_embed` liefern eine Liste von Floats (Embedding-Vektor).
|
||||
- Bei Fehlerpfaden: `mock_llm.side_effect = Exception("...")` oder `return_value = None` für Fallback-Verhalten testen.
|
||||
- DB-Session-Factory in AI-Tool-Handler-Tests: `patch("app.core.db.get_session_factory", return_value=sf)` mit `async_sessionmaker(bind=db_session.bind, ...)`.
|
||||
|
||||
Reference in New Issue
Block a user