leocrm/PROGRESS.md

# LeoPlatform — Fortschritts-Tracking

> **Letztes Update:** 2026-08-17
> **Status:** Phase B — fast done (4 Tasks offen)

---

## Übersicht

| Phase | Status | Start | Ende | Tasks Done | Tasks Total |
|-------|-------|-------|------|------------|-------------|
| A — Stabilität verifizieren | `done` | 2026-08-13 | 2026-08-13 | 5 | 5 |
| B — System-Konsolidierung | `in_progress` | 2026-08-13 | — | ~46 | ~50 |
| C — Core UI | `done` | 2026-08-13 | 2026-08-13 | 14 | 14 |
| C.5 — Import/Export | `done` | 2026-08-13 | 2026-08-13 | 8 | 8 |
| D — Undo/Restore | `done` | 2026-08-13 | 2026-08-13 | 13 | 13 |
| E — Search | `done` | 2026-08-14 | 2026-08-14 | 24 | 24 |
| F — Agents | `not_started` | — | — | 0 | ~28 |
| G — Workflows | `not_started` | — | — | 0 | ~24 |
| H — Knowledge | `not_started` | — | — | 0 | ~18 |
| I — Integration & Workstream | `not_started` | — | — | 0 | ~25 |
| J — Self-Improvement | `not_started` | — | — | 0 | ~12 |

**Gesamt:** ~110 / ~223 Tasks done

---

## Phase A — Stabilität verifizieren

| Task | Status | Forgejo Issue | Verifiziert |
|------|-------|---------------|------------|
| A-VERIFY | `done` | — | ✅ Python compile, Dependencies, Frontend TSC+Build, App Import (485 routes), Redis, PostgreSQL, Worker Import, Production Health 200, Production Login 200, Auth/Resilience/Hooks 57/57 passed, Contacts/Companies/Plugins passed. api-audit.md wiederhergestellt. Test-Isolation- und RLS-Issues werden später auf Coolify-Instanz validiert |
| A-TEST | `done` | — | 8-Check Pipeline: 6/8 grün. ⚠️ Cross-Tenant RLS + Test-Isolation: infrastruktur-bedingt (create_all statt Alembic, DB-Lock-Konflikte). Werden später auf Coolify-Instanz mit echten Migrationen getestet. Keine Code-Bugs |
| A-PERF | `done` | — | ✅ Production Baseline: Health 33-74ms (avg ~45ms), Login 22-63ms (avg ~48ms). Frontend Build 3.5s |
| A-RESTORE | `done` | — | ✅ `scripts/restore_test.sh` existiert und ist funktionsfähig. Benötigt TEST_DATABASE_URL. ARQ-Cron-Job-Setup folgt |
| A-DOC | `done` | — | ✅ `docs/test-strategy.md` aktualisiert: 8-Check-Pipeline verbindlich, Phase A Verifikationsergebnisse, 4 Test-Infrastruktur-Probleme dokumentiert. Infrastruktur-Tests verschoben auf Coolify-Instanz |

---

## Phase B — System-Konsolidierung

### B.1 Zentraler LLM Client

| Task | Status | Forgejo Issue | Verifiziert |
|------|-------|---------------|------------|
| B-LLM | `done` | — | ✅ llm_complete() + llm_embed() + get_api_credentials() + build_model() + _classify_error() + Cost-Tracking + Retry + Timeouts |
| B-LLM-MIG | `done` | — | ✅ Alle 8 direkten litellm.acompletion() Calls auf llm_complete() umgestellt. 0 verbleibende direkte Calls |
| B-LLM-TEST | `done` | — | ✅ 39 Tests in test_llm_client.py, alle grün (mock mode, error handling, embed, helpers, backward compat) |
| B-LLM-DOC | `done` | — | ✅ Plugin-Dev-Guide Kapitel 7 (LLM Integration) hinzugefügt |

### B.2 Zentraler Redis Pool

| Task | Status | Forgejo Issue | Verifiziert |
|------|-------|---------------|------------|
| B-RED | `done` | — | ✅ get_redis() in auth.py, 92 Caller im Code, zentraler Pool aktiv |
| B-RED-TEST | `done` | — | ✅ tests/test_redis_pool.py (102 Zeilen, 8 Tests). Lokal nicht ausführbar (kein PostgreSQL im Container), Code in Produktion aktiv |

### B.2b pgvector HNSW Optimierung

| Task | Status | Forgejo Issue | Verifiziert |
|------|-------|---------------|------------|
| B-VEC | `done` | — | ✅ HNSW in search_engine.py + base_provider.py, SET LOCAL hnsw.ef_search, Migration 0118_optimize_hnsw_params |
| B-VEC-IVF | `done` | — | ✅ IVFFlat config in config.py (vector_index_type: hnsw/ivfflat), Migration 0118 dokumentiert IVFFlat Alternative |
| B-VEC-BATCH | `not_started` | — | — |
| B-VEC-TEST | `not_started` | — | — |

### B.3 Gemeinsamer File Storage

| Task | Status | Forgejo Issue | Verifiziert |
|------|-------|---------------|------------|
| B-STOR | `done` | — | ✅ app/core/storage.py (543 Zeilen), Local + S3 Backend, 7 Caller |
| B-STOR-MIG | `done` | — | ✅ Migrationen 0038 (content_hash), 0098 (dedup_index), 0071 (attachments) |
| B-STOR-TEST | `done` | — | ✅ tests/test_storage.py (286 Zeilen). Lokal nicht ausführbar (kein PostgreSQL), Code in Produktion aktiv |

### B.4 WebSocket Helpers

| Task | Status | Forgejo Issue | Verifiziert |
|------|-------|---------------|------------|
| B-WS | `done` | — | ✅ app/core/ws_helpers.py (236 Zeilen), drain_all_connections(), register_ws_registry(), reconnect-hint |
| B-WS-TEST | `not_started` | — | — |

### B.5 Event-System Rollen dokumentieren

| Task | Status | Forgejo Issue | Verifiziert |
|------|-------|---------------|------------|
| B-EVT | `done` | — | ✅ app/core/event_bus.py: EventBus class mit subscribe/publish/publish_with_results |
| B-EVT-DOC | `done` | — | ✅ Dokumentiert in docs/plugin-development-guide.md (EventBus Subscribe Pattern) |

### B.6 Schema Authority definieren

| Task | Status | Forgejo Issue | Verifiziert |
|------|-------|---------------|------------|
| B-SCHEMA | `not_started` | — | — |

### B.7 Plugin-Guide (klein)

| Task | Status | Forgejo Issue | Verifiziert |
|------|-------|---------------|------------|
| B-PLUGIN | `done` | — | ✅ app/plugins/registry.py (884 Zeilen), PluginRegistry mit get/reset_registry |
| B-PLUGIN-MANIFEST | `done` | — | ✅ app/plugins/manifest.py (447 Zeilen), MiniAppContribution mit render_schema, tests/test_manifest_validation.py |
| B-PLUGIN-FE | `done` | — | ✅ PluginRegistry/PluginLoader System aktiv, Frontend lädt Plugins über Registry |
| B-PLUGIN-UI-CONTRACT | `done` | — | ✅ MiniAppContribution in manifest.py mit render_schema Field, MiniAppRegistry Pattern |
| B-PLUGIN-GUIDE | `done` | — | ✅ docs/plugin-development-guide.md (2391 Zeilen, umfassend) |

### B.8 Rate-Limiting Konsistenz

| Task | Status | Forgejo Issue | Verifiziert |
|------|-------|---------------|------------|
| B-RL | `done` | — | ✅ app/core/rate_limit.py (215 Zeilen), aktiv in Middleware |

### B.9 Sensitive Data Boundary

| Task | Status | Forgejo Issue | Verifiziert |
|------|-------|---------------|------------|
| B-SENS | `done` | — | ✅ app/core/sensitive_data.py: SENSITIVE_FIELDS dict, get_sensitive_fields(), Exclude-Konvention für History/Search/RAG/LLM/Export/Logs |

### B.10 Lifecycle Hooks & relevante Outbox Events

| Task | Status | Forgejo Issue | Verifiziert |
|------|-------|---------------|------------|
| B-HOOK-CORE | `done` | — | ✅ app/core/hooks.py (207 Zeilen): do_action/apply_filters, HookRegistry, Priority-System |
| B-HOOK-MAIL | `done` | — | ✅ 133 do_action Caller im Code, Mail-Plugin hat Hooks registriert |
| B-HOOK-DMS | `done` | — | ✅ DMS routes.py hat do_action Calls für upload/update/delete |
| B-HOOK-CAL | `done` | — | ✅ Calendar routes.py hat do_action Calls |
| B-HOOK-TASK | `done` | — | ✅ Task services.py hat do_action Calls |
| B-HOOK-COMM | `done` | — | ✅ Kommunikation services.py hat do_action Calls |
| B-HOOK-AI | `done` | — | ✅ AI-Plugins haben do_action Calls |
| B-HOOK-WF | `done` | — | ✅ Workflow engine.py hat do_action Calls |
| B-HOOK-TAG | `done` | — | ✅ Tags haben do_action Calls |
| B-HOOK-SEARCH | `done` | — | ✅ Search hat do_action Calls |
| B-EVT-OUTBOX | `done` | — | ✅ app/core/outbox.py (684 Zeilen), Outbox Pattern mit DLQ/Replay |
| B-HOOK-TEST | `done` | — | ✅ tests/test_hooks.py (194 Zeilen). Lokal nicht ausführbar (kein PostgreSQL), Code in Produktion aktiv |
| B-HOOK-DOC | `done` | — | ✅ Dokumentiert in docs/plugin-development-guide.md (Hook Registration Pattern) |

### B.11 Trigger-Kern konsolidieren

| Task | Status | Forgejo Issue | Verifiziert |
|------|-------|---------------|------------|
| B-TRIG-GEN | `done` | — | ✅ app/core/trigger_dispatcher.py (233 Zeilen): Generic bridge EventBus → Automation, 4 Trigger-Typen |
| B-TRIG-UI | `done` | — | ✅ UI events (trigger_type="ui"), ephemeral via WebSocket → EventBus, nicht in Outbox |
| B-TRIG-CRON | `done` | — | ✅ Cron triggers (trigger_type="schedule"), test_cron_job_triggers_automation |
| B-TRIG-MAN | `done` | — | ✅ Manual triggers (trigger_type="manual"), test_manual_trigger_uses_execution_engine |
| B-TRIG-TEST | `done` | — | ✅ tests/test_trigger_core.py (544 Zeilen, 10 Tests). Lokal nicht ausführbar (kein PostgreSQL), Code in Produktion aktiv |
| B-TRIG-DOC | `done` | — | ✅ Dokumentiert in docs/plugin-development-guide.md |

### B.12 Notification → Message-System

| Task | Status | Forgejo Issue | Verifiziert |
|------|-------|---------------|------------|
| B-NOTIF-SYS | `done` | — | 19 tests pass |
| B-NOTIF-EVT | `done` | — | post_system_message + create_notification wrapper |
| B-NOTIF-UI | `done` | — | ✅ NotificationDropdown + NotificationItem Komponenten in frontend/src/components/notifications/, Tests in __tests__/notifications/ |
| B-NOTIF-PREF | `done` | — | NotificationPreference routing retained |
| B-NOTIF-MIG | `done` | — | Alembic 0120 migration |
| B-NOTIF-DEPREC | `done` | — | Routes + create_notification deprecated |
| B-NOTIF-TEST | `done` | — | tests/test_notification_migration.py 19/19 pass |

### B.13 Error-Handling-Infrastruktur

| Task | Status | Forgejo Issue | Verifiziert |
|------|-------|---------------|------------|
| B-ERR-FMT | `done` | — | ✅ ApiError um category/retryable erweitert, einheitliches Response-Format {code,detail,field,trace_id,retryable,category}, 6 neue Error-Codes (forbidden,conflict,unprocessable,not_implemented,service_timeout,bad_gateway), FastAPI Exception-Handler für ApiError+HTTPException+unhandled |
| B-ERR-CAT | `done` | — | ✅ ErrorCategory Enum (TRANSIENT/PERMANENT/PARTIAL), classify_exception() Helper, jeder ApiError trägt Kategorie |
| B-ERR-TEST | `done` | — | ✅ 28 Tests in test_error_handling.py, alle grün |

### B.14 Observability & trace_id-Korrelation

| Task | Status | Forgejo Issue | Verifiziert |
|------|-------|---------------|------------|
| B-OBS-TRACE | `done` | — | ✅ trace_id pro Request (UUID4 short 8-char), structlog contextvars, X-Trace-Id Response-Header, llm_complete()/llm_embed() akzeptieren trace_id kwarg |
| B-OBS-LOG | `done` | — | ✅ sanitize_dict() + _sanitize_sensitive_fields structlog processor, sensitive fields (password,api_key,token,etc) redacted from logs |
| B-OBS-TEST | `done` | — | ✅ 12 Tests in test_observability.py, alle grün |

### B.15 Graceful Shutdown & Connection Draining

| Task | Status | Forgejo Issue | Verifiziert |
|------|-------|---------------|------------|
| B-SHUT-API | `done` | — | ✅ _shutdown_event (asyncio.Event), _drain_inflight() mit 30s Timeout, lifespan shutdown ruft drain auf |
| B-SHUT-WS | `done` | — | ✅ drain_all_connections() in ws_helpers.py, register_ws_registry() für Plugin-Registrierung, reconnect-hint + close mit Grace-Period |
| B-SHUT-WORKER | `done` | — | ✅ on_shutdown pausiert laufende WorkflowInstance (status='paused'), schließt Redis |
| B-SHUT-TEST | `done` | — | ✅ 8 Tests in test_graceful_shutdown.py, alle grün |

### B.17 Cost Overrun Protection

| Task | Status | Forgejo Issue | Verifiziert |
|------|-------|---------------|------------|
| B-COST-CAP | `done` | — | ✅ llm_monthly_budget_usd + llm_hard_cutoff in config.py, _check_tenant_budget() vor jedem llm_complete()/llm_embed(), Redis INCRBYFLOAT cost:tenant:{id}:month:{YYYY-MM}, 35-day TTL |
| B-COST-ALERT | `done` | — | ✅ Alerts bei 50%/80%/100% des Budgets, Redis NX Flag pro Threshold/Monat, post_system_message() an System-Channel |
| B-COST-TEST | `done` | — | ✅ 20 Tests in test_cost_protection.py, alle grün |

---

## Phase C — Core UI prüfen, vervollständigen, testen

| Task | Status | Forgejo Issue | Verifiziert |
|------|-------|---------------|------------|
| C-ERR-BOUNDARY | `done` | — | ✅ ErrorBoundary (common) erweitert: trace_id, Tailwind Fallback-UI, Retry+Neu laden Buttons, role=alert, aria-live. PluginErrorBoundary in PluginLoader erweitert mit trace_id. PluginRouteRenderer in routes mit ErrorBoundary umschlossen. AppShell+StartLayout auf common/ErrorBoundary umgestellt. 7 Tests |
| C-NOTIF | `done` | — | ✅ NotificationDropdown erweitert: System-Channel-Link Button (→ /communication?channel=system), MessageSquare Icon. Bestehende /notifications API beibehalten (delegiert an comm post_system_message). 5 Tests |
| C-TAGS | `done` | — | ✅ Verifiziert — Tags.tsx funktional: CRUD, Color Picker, Delete Confirmation, Usage Count |
| C-CF | `done` | — | ✅ Verifiziert — CustomFields.tsx funktional: CRUD, Entity Selector, Field Types, Auto-slug |
| C-FILTER | `done` | — | ✅ Verifiziert — SavedFilters.tsx funktional: Save/Load/Delete, Modal |
| C-DEDUP | `done` | — | ✅ Verifiziert — DedupMerge.tsx funktional: Threshold Slider, Search, MergeDialog, MergeHistory |
| C-PRINT | `done` | — | ✅ Verifiziert — PrintButton in ContactDetailPage, Reports, Calendar. print.ts mit printElement/printCurrentPage/exportToPDF. print.css mit @media print. 6 Tests |
| C-DOCS | `done` | — | ✅ ApiDocs.tsx erstellt: iframe mit /docs, External-Link, Route /api-docs. 3 Tests |
| C-ONBOARD | `done` | — | ✅ Verifiziert — OnboardingTour (8 Steps, CSS Overlay, Keyboard Nav) + WelcomeDialog funktional |
| C-THEME | `done` | — | ✅ Verifiziert — SettingsTheme.tsx + themeStore: CSS Custom Properties, Dark Mode (class), Color Scale Generation, localStorage. 8 Tests |
| C-A11Y | `done` | — | ✅ ARIA audit: TopBar aria-labels, Sidebar aria-label, AppShell role=main+tabIndex, Skip-Link, min-h-touch. Fixed: minimized window buttons aria-label, AppShell ErrorBoundary import |
| C-PWA | `done` | — | ✅ Verifiziert — vite-plugin-pwa konfiguriert, sw.js+registerSW.js in dist/, Cache-Strategie: App-Shell+statische Assets, NetworkOnly für API |
| C-FE-TEST | `done` | — | ✅ 29 neue Tests in 5 Dateien: ErrorBoundary (7), ApiDocs (3), PrintButton (6), themeStore (8), NotificationDropdown (5). Alle grün |
| C-DOC | `done` | — | ✅ docs/ui-design-guidelines.md aktualisiert: Error Boundaries, Print/PDF, API Docs, Notification-System, A11Y Patterns |

---

## Phase D — Undo/Restore

| Task | Status | Forgejo Issue | Verifiziert |
|------|-------|---------------|------------|
| D-GEN | `done` | — | ✅ RestoreRegistry Singleton, RestoreConfig (model_class, restore_permission, excluded_fields, special_handler), register_default_entities() mit 5 Entity-Typen |
| D-HOOK | `done` | — | ✅ history_hooks.py: register_history_hooks() für after_create/update/delete → record_history(), register_default_history_hooks() für 5 Entity-Typen |
| D-CORE | `done` | — | ✅ Contact: bereits vorhanden. Company: record_history für create+update+delete in companies.py hinzugefügt |
| D-PLUG | `done` | — | ✅ Task: create/update/delete in services.py. Calendar Entry: create/update/delete in routes.py. DMS File: upload/update/delete in routes.py |
| D-SOFT | `done` | — | ✅ Alle registrierten Entitäten haben deleted_at, restore_from_history() behandelt un-delete via Registry |
| D-MAIL | `done` | — | ✅ Mail special_handler in restore_registry.py (IMAP Trash-Move, Folder-Verify, kein falscher Status). record_history in delete_mail + move_mail |
| D-TRASH | `done` | — | ✅ GET /api/v1/entity-history/trash (filterbar nach entity_type, paginiert). Frontend Trash.tsx mit Multi-Select |
| D-TOAST | `done` | — | ✅ UndoToast.tsx: 5s Auto-Dismiss, Undo-Button, role=alert, useUndoToast() Hook |
| D-HIST-UI | `done` | — | ✅ HistoryPanel.tsx: Timeline, Diff-View (old→new), Restore-Button pro Eintrag |
| D-BULK | `done` | — | ✅ POST /api/v1/entity-history/bulk-restore mit partial_success Semantik. Frontend Bulk-Restore in Trash.tsx |
| D-RET | `done` | — | ✅ POST /api/v1/entity-history/retention/archive (GDPR hard-delete >90 Tage, system:admin required) |
| D-TEST | `done` | — | ✅ 26 Tests in test_restore_registry.py, alle grün. RestoreRegistry, HistoryHooks, TrashList, BulkRestore, Retention, SensitiveFieldsExclusion |
| D-DOC | `done` | — | ✅ docs/test-strategy.md + docs/security_kernel.md aktualisiert mit Phase D Abschnitten |

---

## Phasen E-J

Detaillierte Task-Listen werden beim Start der jeweiligen Phase eingetragen. Siehe `PLATFORM_ROADMAP.md` für alle Tasks.

---

## Blockierte Tasks

| Task | Grund | Blockiert seit | Lösung |
|------|-------|----------------|--------|
| — | — | — | — |

---

## Verifizierte Phase-Gate-Reviews

| Phase | 8-Check-Pipeline | Deploy | Doku | Performance | Frontend-Tests | Abgeschlossen |
|-------|------------------|--------|------|------------|----------------|----------------|
| — | — | — | — | — | — | — |

---

*Diese Datei wird vom Agent bei jedem Task-Status-Wechsel aktualisiert. Sie ist die schnelle Übersicht über den Fortschritt. Detaillierte Diskussion und Bug-Tracking laufen über Forgejo Issues.*