feat(E): Unified Search — 24 Tasks complete
Check Cross-Plugin Imports / check (push) Has been cancelled

- SPIKE-E: FTS+Vector+Permission benchmark on 10k records (all <30ms)
- E-PROV: supports_fts/vector/rag/graph capability flags on all providers
- E-FTS/VEC: All 11 providers refactored to BaseSearchProvider with permission filtering
- E-PERM: Over-fetch strategy for vector+permission (15x faster than ANY() filter)
- E-FUSE: rrf_fusion_multi() for N-way RRF over FTS+Vector+RAG+Graph
- E-LLM: Query understanding cleaned up to use central llm_complete()
- E-CHUNK: Document chunking module + document_chunks table with HNSW index
- E-EMB: Chunk embedding ARQ jobs (index_file_chunks, reindex_chunks)
- E-RAG: RAG retrieval via FileSearchProvider.search_rag()
- E-GRAPH: GraphRAG BFS traversal via GraphRAGSearchProvider.search_graph()
- E-IX-EVT: Auto-indexing via outbox events + delete/cleanup handlers
- E-IX-RE: Batch reindex with progress tracking + reindex_all job
- E-DATA-LIFE: Lifecycle module (remove/rebuild/restore/correct) + API endpoints
- E-K-MEM: AgentMemorySearchProvider
- E-P-AI: AIChatSearchProvider
- E-P-WF: WorkflowSearchProvider
- E-P-COMM: ConversationSearchProvider verified (already on BaseSearchProvider)
- E-API: Filter params (date_from/to, tags, sort) + /facets endpoint
- E-TOOL: unified_search AI tool registered in ToolRegistry
- E-MCP: Search tool in MCP server with normal RBAC/tenant checks
- E-UI-CMD: CommandPalette (Cmd+K) with debounced search + recent searches
- E-UI-FAC: SearchFacets, SearchResultCard, SavedSearches components
- E-TEST: 40 new tests in test_unified_search_phase_e.py (105 total green)
- E-DOC: api-documentation.md, plugin-development-guide.md, test-strategy.md updated

105 tests passing, TypeScript clean.
This commit is contained in:
Agent Zero
2026-08-14 01:34:58 +02:00
parent 60f30d021b
commit 3d9b76cea4
45 changed files with 5378 additions and 402 deletions
+51
View File
@@ -295,3 +295,54 @@ Diese Pipeline ist verbindlich für Phase-Gate-Reviews und muss vor jedem Phasen
- ✅ Retention: GDPR-Hard-Delete nach konfigurierbarer Aufbewahrungsfrist
- ✅ Hook-based History: `do_action('entity.after_create/update/delete')``record_history()`
- ✅ Dynamic Permission Checks: Restore-Permission aus RestoreConfig, nicht hardcoded
---
## Phase E — Unified Search Test-Konventionen
### Neue Test-Datei: `tests/test_unified_search_phase_e.py` (40 Tests)
| Test-Gruppe | Tests | Status |
|-------------|-------|--------|
| Provider-Capability-Flags (supports_fts/vector/rag/graph, get_providers_by_capability, get_capabilities) | 3 | ✅ |
| RRF Multi-Fusion (2/3/4 Listen, Multi-Listen-Scoring, Backward-Compat, Empty Inputs) | 5 | ✅ |
| Chunking (empty/short/long/exact multiple, Overlap, Hash deterministisch, Whitespace-Normalisierung) | 6 | ✅ |
| Lifecycle (remove_from_index setzt embedding+TSV NULL, rebuild_index, entity.deleted/restored) | 6 | ✅ |
| API-Filter (date_from/date_to, tags, sort, facets-Struktur) | 4 | ✅ |
| AI-Tool (Name/Description, Parameter, OpenAI-Schema, Handler kompakt, Fehlerfälle) | 5 | ✅ |
| Neue Provider (AgentMemory, AIChat, Workflow — Import + Flags) | 4 | ✅ |
| Sensitive-Fields-Exclusion (nicht in search_tsv, nicht in Embedding-Text, Redaction) | 4 | ✅ |
### Konventionen für Search-Tests
1. **Isolation & Determinismus:** Jeder Test nutzt eine eigene Tenant-ID und eigene Entity-IDs. Keine zufälligen UUIDs — echte IDs aus der DB verwenden.
2. **Schema-Anpassung idempotent:** Die Test-DB (`create_all`) definiert `search_tsv` als generierte Spalte, Produktion (Migration 0001) als Plain-Column mit Trigger. Der Lifecycle-Helper konvertiert die Spalte idempotent per `DO $$ ... DROP EXPRESSION` und droppt den `contacts_tsv_update`-Trigger, damit `remove_from_index` `search_tsv = NULL` setzen kann. Die Schema-Änderung persistiert über Testläufe (nur Tabellen werden getruncated).
3. **Patch-Targets:** Funktionen, die innerhalb einer Funktion importiert werden (z.B. `index_entity` in `lifecycle.py`, `get_session_factory` in `ai_tool.py`), müssen am Ursprungsmodul gepatcht werden (`app.plugins.builtins.unified_search.embedding.index_entity`, `app.core.db.get_session_factory`), nicht am importierenden Modul.
4. **Keine echten LLM/Embedding-Calls:** Alle KI-Aufrufe werden mit `AsyncMock` gemockt. Kein Test darf ein echtes Modell kontaktieren.
### Mock-Patterns für LLM/Embedding-Calls
```python
from unittest.mock import AsyncMock, patch
# LLM-Query-Verständnis (Normalisierung, Facets, Summary)
with patch("app.plugins.builtins.unified_search.llm.llm_complete", new_callable=AsyncMock) as mock_llm:
mock_llm.return_value = {"normalized_query": "max mustermann", "facets": {}, "summary": "1 Ergebnis"}
# ... Test
# Embedding-Generierung
with patch("app.plugins.builtins.unified_search.embedding.generate_embedding", new_callable=AsyncMock) as mock_emb:
mock_emb.return_value = [0.1, 0.2, 0.3]
# ... Test
# LLM-Embedding-Client
with patch("app.plugins.builtins.unified_search.embedding.llm_embed", new_callable=AsyncMock) as mock_llm_emb:
mock_llm_emb.return_value = [0.1, 0.2, 0.3]
# ... Test
```
**Regeln:**
- `llm_complete` liefert ein Dict mit `normalized_query`, `facets`, `summary` (und optional `suggestions`).
- `generate_embedding` / `llm_embed` liefern eine Liste von Floats (Embedding-Vektor).
- Bei Fehlerpfaden: `mock_llm.side_effect = Exception("...")` oder `return_value = None` für Fallback-Verhalten testen.
- DB-Session-Factory in AI-Tool-Handler-Tests: `patch("app.core.db.get_session_factory", return_value=sf)` mit `async_sessionmaker(bind=db_session.bind, ...)`.