feat(E): Unified Search — 24 Tasks complete
Check Cross-Plugin Imports / check (push) Has been cancelled
Check Cross-Plugin Imports / check (push) Has been cancelled
- SPIKE-E: FTS+Vector+Permission benchmark on 10k records (all <30ms) - E-PROV: supports_fts/vector/rag/graph capability flags on all providers - E-FTS/VEC: All 11 providers refactored to BaseSearchProvider with permission filtering - E-PERM: Over-fetch strategy for vector+permission (15x faster than ANY() filter) - E-FUSE: rrf_fusion_multi() for N-way RRF over FTS+Vector+RAG+Graph - E-LLM: Query understanding cleaned up to use central llm_complete() - E-CHUNK: Document chunking module + document_chunks table with HNSW index - E-EMB: Chunk embedding ARQ jobs (index_file_chunks, reindex_chunks) - E-RAG: RAG retrieval via FileSearchProvider.search_rag() - E-GRAPH: GraphRAG BFS traversal via GraphRAGSearchProvider.search_graph() - E-IX-EVT: Auto-indexing via outbox events + delete/cleanup handlers - E-IX-RE: Batch reindex with progress tracking + reindex_all job - E-DATA-LIFE: Lifecycle module (remove/rebuild/restore/correct) + API endpoints - E-K-MEM: AgentMemorySearchProvider - E-P-AI: AIChatSearchProvider - E-P-WF: WorkflowSearchProvider - E-P-COMM: ConversationSearchProvider verified (already on BaseSearchProvider) - E-API: Filter params (date_from/to, tags, sort) + /facets endpoint - E-TOOL: unified_search AI tool registered in ToolRegistry - E-MCP: Search tool in MCP server with normal RBAC/tenant checks - E-UI-CMD: CommandPalette (Cmd+K) with debounced search + recent searches - E-UI-FAC: SearchFacets, SearchResultCard, SavedSearches components - E-TEST: 40 new tests in test_unified_search_phase_e.py (105 total green) - E-DOC: api-documentation.md, plugin-development-guide.md, test-strategy.md updated 105 tests passing, TypeScript clean.
This commit is contained in:
@@ -295,3 +295,54 @@ Diese Pipeline ist verbindlich für Phase-Gate-Reviews und muss vor jedem Phasen
|
||||
- ✅ Retention: GDPR-Hard-Delete nach konfigurierbarer Aufbewahrungsfrist
|
||||
- ✅ Hook-based History: `do_action('entity.after_create/update/delete')` → `record_history()`
|
||||
- ✅ Dynamic Permission Checks: Restore-Permission aus RestoreConfig, nicht hardcoded
|
||||
|
||||
---
|
||||
|
||||
## Phase E — Unified Search Test-Konventionen
|
||||
|
||||
### Neue Test-Datei: `tests/test_unified_search_phase_e.py` (40 Tests)
|
||||
|
||||
| Test-Gruppe | Tests | Status |
|
||||
|-------------|-------|--------|
|
||||
| Provider-Capability-Flags (supports_fts/vector/rag/graph, get_providers_by_capability, get_capabilities) | 3 | ✅ |
|
||||
| RRF Multi-Fusion (2/3/4 Listen, Multi-Listen-Scoring, Backward-Compat, Empty Inputs) | 5 | ✅ |
|
||||
| Chunking (empty/short/long/exact multiple, Overlap, Hash deterministisch, Whitespace-Normalisierung) | 6 | ✅ |
|
||||
| Lifecycle (remove_from_index setzt embedding+TSV NULL, rebuild_index, entity.deleted/restored) | 6 | ✅ |
|
||||
| API-Filter (date_from/date_to, tags, sort, facets-Struktur) | 4 | ✅ |
|
||||
| AI-Tool (Name/Description, Parameter, OpenAI-Schema, Handler kompakt, Fehlerfälle) | 5 | ✅ |
|
||||
| Neue Provider (AgentMemory, AIChat, Workflow — Import + Flags) | 4 | ✅ |
|
||||
| Sensitive-Fields-Exclusion (nicht in search_tsv, nicht in Embedding-Text, Redaction) | 4 | ✅ |
|
||||
|
||||
### Konventionen für Search-Tests
|
||||
|
||||
1. **Isolation & Determinismus:** Jeder Test nutzt eine eigene Tenant-ID und eigene Entity-IDs. Keine zufälligen UUIDs — echte IDs aus der DB verwenden.
|
||||
2. **Schema-Anpassung idempotent:** Die Test-DB (`create_all`) definiert `search_tsv` als generierte Spalte, Produktion (Migration 0001) als Plain-Column mit Trigger. Der Lifecycle-Helper konvertiert die Spalte idempotent per `DO $$ ... DROP EXPRESSION` und droppt den `contacts_tsv_update`-Trigger, damit `remove_from_index` `search_tsv = NULL` setzen kann. Die Schema-Änderung persistiert über Testläufe (nur Tabellen werden getruncated).
|
||||
3. **Patch-Targets:** Funktionen, die innerhalb einer Funktion importiert werden (z.B. `index_entity` in `lifecycle.py`, `get_session_factory` in `ai_tool.py`), müssen am Ursprungsmodul gepatcht werden (`app.plugins.builtins.unified_search.embedding.index_entity`, `app.core.db.get_session_factory`), nicht am importierenden Modul.
|
||||
4. **Keine echten LLM/Embedding-Calls:** Alle KI-Aufrufe werden mit `AsyncMock` gemockt. Kein Test darf ein echtes Modell kontaktieren.
|
||||
|
||||
### Mock-Patterns für LLM/Embedding-Calls
|
||||
|
||||
```python
|
||||
from unittest.mock import AsyncMock, patch
|
||||
|
||||
# LLM-Query-Verständnis (Normalisierung, Facets, Summary)
|
||||
with patch("app.plugins.builtins.unified_search.llm.llm_complete", new_callable=AsyncMock) as mock_llm:
|
||||
mock_llm.return_value = {"normalized_query": "max mustermann", "facets": {}, "summary": "1 Ergebnis"}
|
||||
# ... Test
|
||||
|
||||
# Embedding-Generierung
|
||||
with patch("app.plugins.builtins.unified_search.embedding.generate_embedding", new_callable=AsyncMock) as mock_emb:
|
||||
mock_emb.return_value = [0.1, 0.2, 0.3]
|
||||
# ... Test
|
||||
|
||||
# LLM-Embedding-Client
|
||||
with patch("app.plugins.builtins.unified_search.embedding.llm_embed", new_callable=AsyncMock) as mock_llm_emb:
|
||||
mock_llm_emb.return_value = [0.1, 0.2, 0.3]
|
||||
# ... Test
|
||||
```
|
||||
|
||||
**Regeln:**
|
||||
- `llm_complete` liefert ein Dict mit `normalized_query`, `facets`, `summary` (und optional `suggestions`).
|
||||
- `generate_embedding` / `llm_embed` liefern eine Liste von Floats (Embedding-Vektor).
|
||||
- Bei Fehlerpfaden: `mock_llm.side_effect = Exception("...")` oder `return_value = None` für Fallback-Verhalten testen.
|
||||
- DB-Session-Factory in AI-Tool-Handler-Tests: `patch("app.core.db.get_session_factory", return_value=sf)` mit `async_sessionmaker(bind=db_session.bind, ...)`.
|
||||
|
||||
Reference in New Issue
Block a user