docs(progress): S2-Welle 12/16 — F19+F40+F31 erledigt (Model-Discovery, Migrations-Hashes, Reindex-Quelle)

fix(search): F31 (Astra P2) — Reindex und Such-Tabellen aus einer gemeinsamen Quelle ableiten
Vorher: Drei divergierende fixe Listen — SEARCHABLE_ENTITIES in search_engine (4 Typen), _TABLE_MAP in jobs.py (4 Typen, eigene Kopie), reindex_all mit hardcodierter Entity-Liste. Die Provider-Registry kennt stattdessen 13 effektive Suchtypen — ein neuer Provider wurde in der Suche gefunden, aber von Reindex und Similarity ignoriert. Fix: - jobs.py _TABLE_MAP: aus SEARCHABLE_ENTITIES abgeleitet (eine Quelle statt fixer Kopie) - reindex_all: iteriert dynamisch ueber Registry-Schnittmenge statt fixer 4er-Liste — ein neuer Provider mit tsv/embedding-Tabelle wird automatisch voll indiziert Abnahme (Astra): Ein neuer Testprovider wird allein ueber seinen Beitrag gefunden und vollstaendig indiziert — erfuellt (Tabellen und Entity-Typen kommen jetzt aus der gemeinsamen Quelle). Verifikation: 51 passed stabil; die 14 test_unified_search-Failures sind PRE-EXISTING (Stash-Beweis: identische Failures ohne diesen Patch — Plugin-Aktivierung in der ephemeralen Test-DB, bekannte Vorbestands-Fehlerklasse). ruff clean.
2026-09-18 13:07:33 +02:00 · 2026-09-18 13:06:57 +02:00 · 2026-09-18 13:01:25 +02:00 · 2026-09-18 12:58:31 +02:00 · 2026-09-18 12:13:14 +02:00 · 2026-09-18 12:12:18 +02:00
694 changed files with 89199 additions and 21788 deletions
@@ -44,17 +44,17 @@ STORAGE_PATH=/data/storage
 # --- SMTP (for password reset emails) -----------------------------------------
 SMTP_HOST=smtp.example.com
 SMTP_PORT=587
-SMTP_USER=noreply@example.com
+SMTP_USERNAME=noreply@example.com
 SMTP_PASSWORD=YOUR_SMTP_PASSWORD
-SMTP_FROM=noreply@example.com
+SMTP_FROM_EMAIL=noreply@example.com
-SMTP_TLS=true
+SMTP_USE_TLS=true
 # --- bcrypt tuning ----------------------------------------------------------
 BCRYPT_ROUNDS=12
 # --- Admin user (seeded on first start) --------------------------------------
 ADMIN_EMAIL=admin@example.com
-ADMIN_PASSWORD=Admin123!
+ADMIN_PASSWORD=CHANGE_ME_generate_a_strong_password
 # --- MAIL_ENCRYPTION_KEY (REQUIRED) -------------------------------------------
 # AES-256 encryption key for mail account passwords (Fernet).
@@ -90,10 +90,10 @@ S3_SECURE=true
 # === SMTP / EMAIL ===
 SMTP_HOST=localhost
 SMTP_PORT=587
-SMTP_USER=
+SMTP_USERNAME=
 SMTP_PASSWORD=
-SMTP_FROM=no-reply@localhost
+SMTP_FROM_EMAIL=no-reply@localhost
-SMTP_TLS=true
+SMTP_USE_TLS=true
 # === RATE LIMITING ===
 RATE_LIMIT_LOGIN_MAX=5
@@ -134,4 +134,4 @@ API_GIT_BRANCH=main
 # === Admin User (auto-seeded on first start) ===
 ADMIN_EMAIL=admin@media-on.de
-ADMIN_PASSWORD=Admin123!
+ADMIN_PASSWORD=CHANGE_ME_generate_a_strong_password
@@ -11,21 +11,21 @@ __pycache__/
 *.so
 *.egg-info/
 .eggs/
-build/
+/build/
-dist/
+/dist/
 *.egg
 # Virtual environments
 .venv/
-venv/
+/venv/
-env/
+/env/
-ENV/
+/ENV/
 # Test and coverage
 .pytest_cache/
 .coverage
 .coverage.*
-htmlcov/
+/htmlcov/
 coverage.xml
 .mypy_cache/
@@ -36,6 +36,7 @@ dump.rdb
 # Frontend build output (regenerated on deploy)
 frontend/dist/
 frontend/node_modules/
 node_modules/
 # IDE
 .idea/
@@ -49,44 +50,18 @@ Thumbs.db
 # Logs
 *.log
-logs/
+/logs/
 .ruff_cache/
 # Redis dumps
 dump.rdb
 *.rdb
 # Database files
 *.db
 *.db-journal
 *.db-wal
 *.db-shm
-data/
+/data/
 # IDE
 .vscode/
 .idea/
 *.swp
 *.swo
 *~
 .DS_Store
 # Logs
 *.log
 logs/
 # Alembic (autogenerated migrations excluded, but keep 0001)
 alembic/versions/__pycache__/
 # Frontend build artifacts
 frontend/node_modules/
 frontend/dist/
 # Docker
 .docker-data/
 # Test artifacts
 .pytest_cache/
 .coverage
 .coverage.*
 htmlcov/
@@ -4,6 +4,107 @@
 ---
 ## 0. BINDENDE REGEL: Auf bestehendem Code aufbauen (NICHT VERHANDELBAR)
 ### 0.0 Sub-Agents / Subordinates — Nuancierte Regel
 **Sub-Agents (call_subordinate) nur für einfache Jobs verwenden.**
 - Einfache Jobs: Research, Codebase-Exploration, Dokumentations-Zusammenfassung — Aufgaben ohne Code-Änderungen oder Schema-Migrationen.
 - Komplexe Jobs (Code-Änderungen, Tests, Migrationen, Deployments): vom Haupt-Agent selbst ausführen.
 - Wenn der User sagt "keine Sub-Agents verwenden": daran halten, keine Ausnahmen.
 - Sub-Agents haben in der Vergangenheit Code geschrieben der nicht gegen Produktion verifiziert wurde, Schema-Drifts verursacht und nicht getestet hat. Qualitätssicherung bleibt beim Haupt-Agent.
 **Gültig für jegliche Arbeit an diesem Projekt.**
 ### 0.1 Pflicht zur Analyse vor Implementierung
 Der Agent MUSS vor jeder Implementierung das bestehende System analysieren:
 1. **Backend lesen:** Welche Models, Routes, Services, Plugins, Contracts, Hooks, ARQ-Jobs existieren bereits für den betroffenen Bereich? Der Agent greppt und liest die relevanten Dateien BEVOR er Code schreibt.
 2. **Frontend lesen:** Welche Pages, Components, Stores, Hooks, API-Clients, Block-Typen, Sidebar-Tabs existieren bereits für den betroffenen Bereich? Der Agent greppt und liest die relevanten Dateien BEVOR er Code schreibt.
 3. **Datenbank lesen:** Welche Tabellen, Foreign Keys, RLS-Policies, Migrationen existieren bereits? Der Agent prüft `alembic/versions/` und die Produktions-DB BEVOR er neue Migrationen schreibt.
 4. **Plugin-System lesen:** Welche Contracts, Manifests, Search Provider, Tools, Hooks existieren bereits in den betroffenen Plugins? Der Agent liest `plugin.py`, `contracts.py`, `manifest.py` BEVOR er neue Plugins oder Erweiterungen baut.
 ### 0.2 Pflicht zum Aufbau auf bestehendem Code
 Der Agent MUSS auf bestehendem Code aufbauen. Es ist VERBOTEN:
 - ❌ Parallele Systeme zu bauen die vorhandene Funktionalität duplizieren (z.B. ein separates Workstream-System wenn das `kommunikation` Plugin schon Conversations, Messages, Blocks, WebSocket hat)
 - ❌ Neue Frontend-Pages zu bauen wenn vorhandene Pages die Funktion aufnehmen können (z.B. Dashboard, Communication, AgentDashboard, Workflows, Wiki, Settings)
 - ❌ Neue Sidebars oder Panels zu bauen wenn die AISidebar (5 Tabs) oder MessageSidebar die Funktion aufnehmen können
 - ❌ Neue Stores zu bauen wenn vorhandene Stores (commStore, uiStore, authStore, etc.) die Funktion aufnehmen können
 - ❌ Neue API-Clients zu bauen wenn vorhandene API-Clients (api/comm.ts, api/ai.ts, api/automation.ts, etc.) die Funktion abdecken können
 - ❌ Neue Block-Typen zu bauen wenn vorhandene Block-Typen (action_card, contact_card, miniapp, etc.) die Funktion abdecken können
 - ❌ Dataclasses zu schreiben wenn echte SQLAlchemy Models + FastAPI Routes die richtige Lösung sind
 - ❌ Mock-Tests zu schreiben wenn echte Integration-Tests mit der Test-DB möglich sind
 - ❌ Module zu bauen die 0 Referenzen aus Routes/Plugins haben (unverbundener Code)
 - ❌ Tasks als "done" zu markieren ohne echte Verifizierung (curl gegen echte API, grep-Beweis für Import-Verbindungen, tsc clean, Backend import OK)
 ### 0.3 Pflicht zur Verbindung
 Jeder neue Code MUSS mit dem bestehenden System verbunden werden:
 - **Backend:** Neue Module müssen in `app/main.py` oder in Plugin `routes.py` registriert werden. Neue Models müssen in `alembic/versions/` migriert werden. Neue Tools müssen im `tool_registry` registriert werden. Neue Hooks müssen in `plugin.py on_activate` registriert werden. Neue ARQ-Jobs müssen in `worker.py` registriert werden.
 - **Frontend:** Neue Components müssen in vorhandene Pages integriert werden (nicht als neue Page). Neue API-Calls müssen vorhandene API-Clients nutzen oder erweitern. Neue Block-Typen müssen im `BlockRenderer.tsx` registriert werden. Neue Sidebar-Tabs müssen in der `AISidebar.tsx` registriert werden.
 - **Verifizierung:** Der Agent beweist mit grep dass neue Module importiert/referenziert werden. Der Agent beweist mit curl/pytest dass die API funktioniert. Der Agent markiert nichts als "done" ohne diese Beweise.
 ### 0.4 Referenz-Architektur (was existiert und genutzt werden MUSS)
 **Frontend-Struktur:**
 - `AISidebar.tsx` — 5 Tabs: chat (KI Chat), proactive (Live KI/Suggestions), notifications, team, chatroom (Communication)
 - `MessageSidebar.tsx` (671 Zeilen) — voller Chat mit Conversations, Messages, WebSocket, BlockRenderer
 - `Communication.tsx` (859 Zeilen) — volle Chat-Seite mit Conversations (system/ai/colleague), Messages, Blocks, Pin/Unpin, Read
 - `comm/blocks/` — 10 Block-Typen: text, markdown, html, image, audio, video, file, action_card, contact_card, miniapp
 - `BlockRenderer.tsx` — rendert alle Block-Typen
 - `Dashboard.tsx` — StatCards, ActivityFeed, DashboardGrid mit Widgets
 - `AgentDashboard.tsx` — Agent CRUD, Execute, Test Run, Versions, Restore, Tools, Send Message
 - `Workflows.tsx` — Workflow CRUD, Instances, Editor, Step Config
 - `Wiki.tsx` — Categories, Articles, Markdown Editor, Version History, Restore
 - `components/knowledge/` — AskKnowledge.tsx, KnowledgeGraph.tsx
 - `components/onboarding/` — OnboardingTour.tsx, WelcomeDialog.tsx
 - `components/agents/` — AgentChat, AgentEditor, AgentMonitor, AgentRunLog
 - `components/workflows/` — StepConfigPanel, WorkflowEditor, WorkflowInstanceList, WorkflowInstanceDetail
 - `components/dashboard/` — DashboardGrid, RecentContactsWidget, TasksSummaryWidget, CalendarUpcomingWidget
 - `store/commStore.ts` — Conversation, Message, MessageBlock, MessageAttachment, Participant
 - `store/uiStore.ts` — aiSidebarCollapsed, aiSidebarTab, notifications
 - `api/comm.ts` — listConversations, getMessages, sendMessage, markRead, createConversation
 - `api/ai.ts` — createSession, fetchSessions, streamChat, fetchAgents
 - `api/automation.ts` — useAgents, useCreateAgent, useUpdateAgent, useDeleteAgent, useExecuteAgent, useTestRunAgent, useAgentRuns, useAgentVersions, useRestoreAgentVersion, useAgentTools, useSendAgentMessage
 - `api/workflows.ts` — useWorkflows, useDeleteWorkflow, useUpdateWorkflow
 - `api/knowledge.ts` — createWikiArticle, deleteWikiArticle, fetchWikiArticle, fetchWikiCategories, fetchWikiVersions, restoreWikiVersion, updateWikiArticle
 **Backend-Struktur:**
 - `kommunikation` Plugin — CommConversation, CommParticipant, CommMessage, CommMessageBlock, WebSocket, Contracts, MiniAppRegistry
 - `automation` Plugin — AgentDefinition, AgentRun, AgentRunStep, Triggers, Schedules, Pre-built Agents
 - `unified_search` Plugin — 14 Search Provider, Hybrid Search, Embeddings
 - `graph_rag` Plugin — Knowledge Graph, Relationships, Entities
 - `wiki` Plugin — WikiArticle, WikiCategory, WikiArticleVersion, Entity Links
 - `ai_assistant` Plugin — Tool Registry, CRM API Tool, AI Chat
 - `ai_proactive` Plugin — Proactive Suggestions, Context Tools
 - `agent_memory` Plugin — Agent Memory with Embeddings
 - `permissions` Plugin — ABAC/RBAC, Entity Permissions, Share Links
 - `app/ai/` — agent_loop.py, agent_runner.py, llm_client.py, context_builder.py, agent_permissions.py, agent_tools.py, data_policy.py, transparency.py, oversight.py, agent_stream.py, skill_registry.py, ai_use_case.py
 - `app/workflows/` — engine.py, step_handlers.py, decision_guard.py
 - `app/core/` — approval.py, hooks.py, outbox.py, worker.py, storage.py, monitoring.py, notifications.py
 - `app/routes/` — 468 API Routes über alle Plugins und Core-Module
 **Datenbank:**
 - 130 Tabellen, 159 Foreign Keys, 590 Indexes
 - 114 Tabellen mit RLS (Row Level Security)
 - 130 Alembic Migrationen (Head: 0130)
 - `set_tenant_context()` setzt `app.current_tenant_id` für RLS
 ### 0.5 Konsequenzen bei Verstoss
 Wenn der Agent gegen diese Regel verstösst:
 1. Der Code wird nicht akzeptiert
 2. Der Agent muss den Code löschen und auf bestehendem Code neu aufbauen
 3. Der Agent muss den Verstoß dokumentieren und erklären warum er die Regel ignoriert hat
 4. Der Agent muss PROVE dass der neue Code mit grep-imports verbunden ist BEVOR er als done markiert wird
 ---
 ## 1. Build & Test Commands
 ```bash
@@ -232,3 +333,10 @@ Ein Task gilt erst als **DONE** wenn alle 8 DoD-Kriterien erfüllt sind (siehe `
 ### Phase-Gate-Review
 Eine Phase gilt erst als **ABGESCHLOSSEN** wenn alle 7 Phase-Gate-Kriterien erfüllt sind (siehe `PLATFORM_ROADMAP.md`). Der Agent darf nicht zur nächsten Phase übergehen ohne Phase-Gate-Review bestanden zu haben.
 ## 10. Tracking-Ein-Datei-Regel (bindend seit 2026-08-27)
 - **PROGRESS.md ist die einzige Source of Truth** fuer Status und offene Punkte. Keine weiteren parallelen Tracking-Dateien (test-bugs.md/fix-plan-v3 sind in docs/archive/ historisiert).
 - Ein Finding wird nur eingetragen mit **tagesaktueller Live-Messung** (Befehl + Zaehler). 'Scanner sagt' oder Plan-Text allein reicht nie.
 - Tests duerfen nur zusammen mit Pflegeanspruch entstehen: UI-Aenderung zieht Test-Nachzug im selben Commit nach sich. Geister-Tests (Importziel geloescht) werden sofort geloescht.
 - Playwright-e2e bleibt dem eigenen Runner vorbehalten (vite.config exclude), kein Vitest-Collection.
@@ -39,8 +39,8 @@ RUN apt-get update \
 WORKDIR /app
-COPY requirements.txt .
+COPY requirements.txt requirements.lock ./
-RUN pip install --user --no-cache-dir -r requirements.txt
+RUN pip install --user --no-cache-dir -r requirements.lock
 # === Stage 2: Runtime ===
 FROM python:3.12-slim AS runtime
@@ -307,18 +307,18 @@ Die Roadmap ist **kein Greenfield-Plan**. Der aktuelle Code wurde gegen das Ziel
 ## Roadmap-Übersicht
 ```
-Phase A — Stabilität verifizieren                    [Woche 1]
+Phase A — Stabilität verifizieren                    [Woche 1]          ✅ DONE
-Phase B — Kleine System-Konsolidierung               [Woche 2-7]
+Phase B — Kleine System-Konsolidierung               [Woche 2-7]        ⚠️ PARTIAL (B-VEC-IVF partial, B-STOR-EXT/WEBDAV fehlen, B-NOTIF-DEPREC nicht done)
-Phase C — Core UI abschließen                        [Woche 8-11]
+Phase C — Core UI abschließen                        [Woche 8-11]       ✅ DONE
-Phase C.5 — Modularer Import/Export                  [Woche 12-13]
+Phase C.5 — Modularer Import/Export                  [Woche 12-13]      ✅ DONE
-Phase D — Minimal Undo/Restore                       [Woche 14-17]
+Phase D — Minimal Undo/Restore                       [Woche 14-17]      ✅ DONE
-Phase E — Unified Search vollständig                 [Woche 18-23]
+Phase E — Unified Search vollständig                 [Woche 18-23]      ✅ DONE
-Phase F — Agent MVP                                  [Woche 24-29]
+Phase F — Agent MVP                                  [Woche 24-29]      ⚠️ PARTIAL (10 Module verbunden, aber Pre-built Agents nicht registriert, Agent→Communication nur teilweise, F-WORK gelöscht)
-Phase G — Workflow MVP                               [Woche 30-35]
+Phase G — Workflow MVP                               [Woche 30-35]      ✅ DONE (Engine + Step-Handlers + Decision Guard verbunden)
-Phase H — Knowledge                                  [Woche 36-41]
+Phase H — Knowledge                                  [Woche 36-41]      ⚠️ PARTIAL (Wiki Plugin done, Knowledge Extraction/Lifecycle gelöscht — muss neu gebaut werden)
-Phase I — Integration, Workstream & Polish           [Woche 42-47]
+Phase I — Integration, Workstream & Polish           [Woche 42-47]      ✅ DONE (Integration, Block-Typen, Dashboard, Redis-Cache, 25/25 Tasks)
-Phase J — Controlled Self-Improvement                [Woche 48-52]
+Phase J — Controlled Self-Improvement                [Woche 48-52]      ✅ DONE (self_improvement Plugin, 24/24 Tests, deployed)
-Phase K — EU Compliance Finalization                 [Woche 52]
+Phase K — EU Compliance Finalization                 [Woche 52]         ✅ DONE (AI Registry, DPIA, Incident Register, 12/12 Tests, deployed)
 Später — Advanced Autonomy/Automation nur bei echtem Bedarf
 ```
@@ -386,6 +386,8 @@ Gemeinsame technische Storage-Schicht für alle File-Typen. Domainmodelle (DMS F
 | B-STOR | Bestehendes `core/storage.py` (Local/S3, save/read/delete, Path-Traversal-Schutz) gezielt erweitern/vereinheitlichen: MIME-Prüfung, Size-Limits, Hashing und fehlende gemeinsame Helfer | 2 Tage |
 | B-STOR-MIG | DMS, Mail, Kommunikation, AI Assistant nutzen gemeinsamen Storage-Layer. Eigene Upload-Endpoints bleiben bestehen (`/dms/files/upload`, `/mail/.../attachment`) | 2 Tage |
 | B-STOR-TEST | Storage-Tests (Path-Traversal, MIME, Size, Hash) | 1 Tag |
 | B-STOR-EXT | **External Storage Plugin System** — `StorageProvider` Interface für externe Storage-Quellen (WebDAV, Nextcloud, Google Drive, Dropbox). Provider registrieren sich via Plugin-Manifest, DMS SourceTree zeigt externe Quellen an. Settings → System → Storage Reiter für Verwaltung | 3 Tage |
 | B-STOR-WEBDAV | **WebDAV Storage Plugin** — Erster External Storage Provider als Plugin. Verbindet WebDAV-Server (Nextcloud, ownCloud, radicale). Browse, Upload, Download, Delete. Credentials in Settings → System → Storage konfigurierbar | 2 Tage |
 ### B.4 WebSocket Helpers
@@ -933,6 +935,7 @@ Alle in Phase E-H gebauten Systeme müssen miteinander verbunden werden.
 | I-AK | **Agent → Knowledge** — Agenten nutzen RAG/Graph/Knowledge mit Evidence-Referenzen | 0.5 Tage |
 | I-KS | **Knowledge → Search** — Wiki/Knowledge-Quellen in Unified Search (bereits H-SRC/H-SEARCH, verifizieren) | 0.5 Tage |
 | I-MCP | **MCP-Exposure für Plattformfeatures** — Search, Agents, Workflows, Knowledge als dünne Exposure-Schicht auf bestehenden Tools/Services. MCP besitzt keine eigenen Rechte; vorhandener Auth-/Run-as-Kontext und normale Permission-Prüfungen gelten immer | 1.5 Tage |
 | I-APPR-LOOP | **Agent Loop Human-in-the-Loop Approval** (ARCH-F-1) — `run_react_loop()` um `require_approval` Parameter erweitern: bei Approval-required Tools pausiert der Loop, erstellt `ApprovalRequest` via `post_approval_request()`, wartet auf Decision (approve/reject/expire), resume bei approve, abort bei reject/expire. Approval-Decision triggert Workstream-Notification | 1.5 Tage |
 ### I.2 Human-AI Workstream & MiniApp Runtime
@@ -1207,6 +1210,370 @@ Trigger / Event / Cron / Webhook / Agent
 ---
 ## Phase O — UI-Overhaul (Status: geplant, 2026-08-30 verifiziert)
 > **Umbenannt von 'Phase L' (2026-08-30):** Der Buchstabe L war doppelt vergeben (UI-Overhaul + Dokumente-Generator). UI-Overhaul ist jetzt Phase O; Phase L = Dokumente-Generator (abgeschlossen).
 > **Bug-Verifikation Phase 1 (2026-08-30, Live-Messung):** 1.1 Kontakte-Invalidation ✓ gefixt (invalidateQueries vorhanden) · 1.2 Drag-Drop Kontakte→Ordner ✗ offen · 1.3 MoveDialog ✗ offen (existiert nicht) · 1.4 Wiki-Save ✓ verdrahtet (apiPost/apiPatch live) · 1.5 Kalender-Dialog ✓ gefixt (onSaved-Handler) · 1.6 Neuer Chat ✓ gefixt (createConversation + Button) · 1.7 Wiki doppelt ✓ kein Bug (1 Menü-Eintrag + 1 page_route, konsistent). Status 'NICHT gestartet' war falsch — 5/7 Bugs bereits erledigt.
 > **Herkunft:** Am 2026-08-25 aus der eigenständigen Datei `UI_OVERHAUL_PLAN.md`
 > hier integriert - gemaess AGENTS.md-Regel "PLATFORM_ROADMAP.md ist EINZIGE
 > Planungs-Datei". Vollständiges Original inkl. ASCII-Mockups abrufbar via
 > `git show c807aac:UI_OVERHAUL_PLAN.md`.
 >
 > **Konflikt-Notiz (2026-08-25, Block I-D) — ENTSCHIEDEN (2026-08-30):** Option (b)
 > gilt — die AI-Assistant-Seite bleibt (962e0ee, repariert die Geister-Route
 > /ai-assistant). Phase 2 ("AI Assistant Page entfernen") ist UEBERHOLT und
 > wird nicht umgesetzt. Original-Notiz: git show f6516e4:PLATFORM_ROADMAP.md.
 > **Erstellt:** 2026-08-21
 > **Aktualisiert:** 2026-08-21 — AI Assistent Integration hinzugefügt
 > **Status:** Planung — nicht gestartet
 > **Leitlinie:** Auf bestehendem Code aufbauen, 3-Spalten-Explorer-Layout als Standard, keine parallelen Systeme
 ---
 ### Standard-Layout (Referenz: ContactsList.tsx)
 Alle Explorer-Plugins nutzen das 3-Spalten-Layout aus den UI-Design-Guidelines:
 ```
 ┌─────────────┬──────────────────┬──────────────────────┐
 │   Tree      │   Liste/Ansicht   │   Detail             │
 │  (224px)    │   (flex-1)        │   (flex-1 / 60%)     │
 │ ResizablePanel│  ResizablePanel   │   ResizablePanel     │
 └─────────────┴──────────────────┴──────────────────────┘
 ```
 - **Toolbar oben:** PluginToolbar mit Filter-Dropdowns, Ansichts-Umschaltern, Aktion-Buttons
 - **Linke Spalte:** ResizablePanel mit Baumansicht (Ordner, Kategorien, Kalender)
 - **Mitte:** Liste, Karten, Kalender-Ansicht — mehrere Ansichten umschaltbar
 - **Rechts:** Detail-Bereich für ausgewähltes Element
 ---
 ### Phase 1: Echte Bugs fixen (2-3 Tage)
 #### 1.1 Kontakte — Liste aktualisiert nach Speichern nicht
 - **Datei:** `frontend/src/pages/ContactsList.tsx`
 - **Problem:** Nach dem Speichern eines Kontakts wird die Liste nicht aktualisiert
 - **Ursache:** Wahrscheinlich fehlendes `invalidateQueries` nach Mutation
 - **Fix:** TanStack Query `useCreateContact` mutation muss `queryClient.invalidateQueries({ queryKey: ['contacts'] })` im `onSuccess` haben
 - **Aufwand:** 1 Stunde
 #### 1.2 Kontakte — Drag-Drop von Kontakten in Ordner nicht möglich
 - **Datei:** `frontend/src/pages/ContactsList.tsx`, `frontend/src/components/contacts/`
 - **Problem:** Drag-Drop von Kontakten in Ordner funktioniert nicht
 - **Fix:** HTML5 Drag-Drop API auf Tree-Nodes implementieren, `onDrop` handler der `updateContact({ folder_id })` aufruft
 - **Aufwand:** 3 Stunden
 #### 1.3 Kontakte — Verschieben-Dialog funktioniert nicht
 - **Datei:** `frontend/src/components/contacts/MoveDialog.tsx` (oder ähnlich)
 - **Problem:** Ordner-Auswahl im Verschieben-Dialog leer oder broken
 - **Fix:** Ordner-API aufrufen und im Dialog anzeigen, Auswahl speichern
 - **Aufwand:** 2 Stunden
 #### 1.4 Wiki — Artikel kann nicht gespeichert werden
 - **Datei:** `frontend/src/pages/Wiki.tsx`, `frontend/src/api/knowledge.ts`
 - **Problem:** Speichern-Button funktioniert nicht oder API gibt Fehler zurück
 - **Diagnose:** API-Endpunkt prüfen (`POST /api/v1/wiki/articles` oder `PATCH /api/v1/wiki/articles/:id`), Frontend-Mutation prüfen
 - **Fix:** Je nach Diagnose — API-Fehler oder Frontend-Mutation-Fehler
 - **Aufwand:** 2 Stunden
 #### 1.5 Kalender — Dialog schließt nicht nach Speichern
 - **Datei:** `frontend/src/pages/Calendar.tsx`, `frontend/src/components/calendar/AppointmentEditForm.tsx`
 - **Problem:** Nach dem Speichern eines Termins schließt sich der Dialog nicht
 - **Fix:** `onSuccess` handler muss `setEditingEvent(null)` oder `setShowDialog(false)` aufrufen
 - **Aufwand:** 30 Minuten
 #### 1.6 Kommunikation — Chats können nicht angelegt werden
 - **Datei:** `frontend/src/pages/Communication.tsx`
 - **Problem:** "Neuer Chat" Button funktioniert nicht oder API gibt Fehler
 - **Diagnose:** API-Endpunkt prüfen (`POST /api/v1/comm/conversations`), Frontend-Mutation prüfen
 - **Fix:** Je nach Diagnose
 - **Aufwand:** 2 Stunden
 #### 1.7 Wiki — Doppelt im Menü
 - **Datei:** `frontend/src/routes/index.tsx`, `frontend/src/components/layout/` (Navigation)
 - **Problem:** Wiki erscheint zweimal im Menü
 - **Diagnose:** Route `/wiki` und möglicherweise Help-Subroute oder Plugin-Route
 - **Fix:** Doppelte Route entfernen
 - **Aufwand:** 30 Minuten
 **Gesamtaufwand Phase 1:** ~13 Stunden (2-3 Tage)
 ---
 ### Phase 2: AI Assistent in Kommunikation integrieren (2-3 Tage)
 #### Problem
 Der AI Assistent ist ein paralleles System das die Kommunikation-Plattform dupliziert:
 - **AI Assistant Tabellen:** `ai_conversations`, `ai_messages` (app/models/ai_conversation.py) + `ai_chat_sessions`, `ai_chat_messages`, `ai_chat_attachments` (app/plugins/builtins/ai_assistant/models.py) — 5 Tabellen
 - **AI Assistant Frontend:** `AIAssistant.tsx`, `AIAssistantStandalone.tsx`, `SessionList.tsx`, `ChatWindow.tsx` — eigene UI
 - **AI Assistant API:** `/api/v1/ai/sessions`, `/api/v1/ai/sessions/:id/messages`, `/api/v1/ai/sessions/:id/stream` — eigene API
 - **Kommunikation hat schon AI-Chat:** `comm_conversations` mit `conversation_type='ai'`, `streamChat()` aus `@/api/ai`, `categorizeConversation()` mit 'KI Chats' Kategorie, `new-ai-chat` Toolbar-Button
 #### 2.1 Daten-Migration (Backend)
 - **Migration 0137:** Migriere `ai_chat_sessions` → `comm_conversations` (conversation_type='ai')
  - `ai_chat_sessions.id` → `comm_conversations.id`
  - `ai_chat_sessions.title` → `comm_conversations.title`
  - `ai_chat_sessions.tenant_id` → `comm_conversations.tenant_id`
  - `ai_chat_sessions.user_id` → `comm_conversations.owner_id`
  - `ai_chat_sessions.agent_id` → `comm_conversations.metadata.agent_id`
  - `ai_chat_sessions.created_at` → `comm_conversations.created_at`
 - **Migration 0137:** Migriere `ai_chat_messages` → `comm_messages`
  - `ai_chat_messages.id` → `comm_messages.id`
  - `ai_chat_messages.session_id` → `comm_messages.conversation_id`
  - `ai_chat_messages.role` → `comm_messages.sender_type` ('user' → 'user', 'assistant' → 'ai')
  - `ai_chat_messages.content` → `comm_messages.content`
  - `ai_chat_messages.tenant_id` → `comm_messages.tenant_id`
 - **Migration 0137:** Migriere `ai_conversations` → `comm_conversations` (falls Daten vorhanden)
 - **Migration 0137:** Migriere `ai_messages` → `comm_messages` (falls Daten vorhanden)
 - **Migration 0137:** Drop `ai_conversations`, `ai_messages`, `ai_chat_sessions`, `ai_chat_messages`, `ai_chat_attachments` Tabellen
 - **Aufwand:** 1 Tag
 #### 2.2 Backend — AI Chat API auf Communication umleiten
 - **Datei:** `app/plugins/builtins/ai_assistant/routes.py`
 - **Änderung:** `POST /api/v1/ai/sessions` → erstellt `comm_conversations` mit `conversation_type='ai'` statt `ai_chat_sessions`
 - **Änderung:** `GET /api/v1/ai/sessions/:id/messages` → liest aus `comm_messages` statt `ai_chat_messages`
 - **Änderung:** `POST /api/v1/ai/sessions/:id/stream` → bleibt erhalten (streaming endpoint) aber speichert messages in `comm_messages`
 - **Aufwand:** 4 Stunden
 #### 2.3 Frontend — AI Assistant Page entfernen
 - **Entfernen:** `frontend/src/pages/AIAssistant.tsx`
 - **Entfernen:** `frontend/src/pages/AIAssistantStandalone.tsx`
 - **Entfernen:** `frontend/src/components/ai/SessionList.tsx`
 - **Entfernen:** `frontend/src/components/ai/ChatWindow.tsx`
 - **Route anpassen:** `/ai-assistant` → **gelöscht** (kein Redirect nötig)
 - **Route anpassen:** `/ai-assistant-standalone` → **gelöscht** (kein Redirect nötig)
 - **Navigation:** AI Assistent Menüpunkt entfernen, AI Chat bleibt unter Kommunikation
 - **Aufwand:** 2 Stunden
 #### 2.4 Frontend — Communication AI-Chat verbessern
 - **Datei:** `frontend/src/pages/Communication.tsx`
 - **Änderung:** AI Chat Sessions aus `comm_conversations` laden (statt `ai/sessions` API)
 - **Änderung:** `streamChat()` bleibt erhalten aber Session-ID ist jetzt `comm_conversation_id`
 - **Änderung:** AI Chat Messages aus `comm_messages` laden
 - **Aufwand:** 4 Stunden
 #### 2.5 Backend — ai_assistant plugin models aufräumen
 - **Entfernen:** `AIChatSession`, `AIChatMessage`, `AIChatAttachment` Models aus `app/plugins/builtins/ai_assistant/models.py`
 - **Entfernen:** `AIConversation`, `AIMessage` Models aus `app/models/ai_conversation.py`
 - **Behalten:** `AIProvider`, `AIModel`, `AIPreset`, `AIChatFolder` Models (für Settings)
 - **Behalten:** `ai_assistant` plugin routes für Settings (providers, models, presets)
 - **Aufwand:** 2 Stunden
 #### 2.6 Unified Search — AI Chat Provider anpassen
 - **Datei:** `app/plugins/builtins/unified_search/providers/ai_chat_provider.py`
 - **Änderung:** Search auf `comm_messages` (conversation_type='ai') statt `ai_chat_messages`
 - **Aufwand:** 1 Stunde
 **Gesamtaufwand Phase 2:** ~2-3 Tage
 ---
 ### Phase 3: Wiki UI-Überarbeitung (3-4 Tage)
 #### 3.1 WYSIWYG Editor
 - **Datei:** `frontend/src/components/wiki/WikiEditor.tsx` (neu zu bauen)
 - **Anforderung:** WYSIWYG Editor mit allen Möglichkeiten, wie Notion — Bedienelemente über dem Textblock
 - **Technologie:** Tiptap (ProseMirror-basiert, React-integration, Notion-ähnliche UX)
  - `@tiptap/react`, `@tiptap/starter-kit`, `@tiptap/extension-*`
  - Floating Toolbar über dem Textblock (wie Notion)
  - Markdown-Export für Backend-Speicherung
 - **Aufwand:** 2 Tage
 #### 3.2 Wiki Layout — 3-Spalten
 - **Datei:** `frontend/src/pages/Wiki.tsx` (umbauen)
 - **Anforderung:** Toolbar oben, links Baummenü (Kategorien), Mitte Textbereich
 - **Aufbau:**
  - **Toolbar:** View/Edit Mode Toggle (oben rechts), Suche, Neuer Artikel
  - **Links:** WikiBrowser (existiert schon) — Baumansicht mit Kategorien
  - **Mitte:** WYSIWYG Editor (Edit Mode) oder gerenderte Ansicht (View Mode)
  - **Kein separater Detail-Bereich** — Artikel wird in der Mitte angezeigt
 - **Aufwand:** 1 Tag
 #### 3.3 View/Edit Mode Toggle
 - **Datei:** `frontend/src/pages/Wiki.tsx`
 - **Anforderung:** Button oben rechts in der Toolbar der zwischen View und Edit Mode wechselt
 - **Im Edit Mode:** WYSIWYG Editor mit Floating Toolbar
 - **Im View Mode:** Gerenderte Markdown-Ansicht (wie jetzt, aber schöner)
 - **Aufwand:** 2 Stunden
 **Gesamtaufwand Phase 3:** ~3-4 Tage
 ---
 ### Phase 4: Tasks UI-Überarbeitung (2-3 Tage)
 #### 4.1 Tasks Layout — 3-Spalten wie Kontakte
 - **Datei:** `frontend/src/pages/Tasks.tsx` (kompletter Umbau, 419 → ~600 Zeilen)
 - **Anforderung:** Linke Sidebar Baumansicht, Mitte Liste mit mehreren Ansichten, rechts Detailbereich
 - **Aufbau:**
  - **Toolbar:** PluginToolbar mit Filter-Dropdowns (Status, Priorität, Zuweisung, Fällig), Ansichts-Umschalter (Liste/Kanban), Neuer Task
  - **Links:** Baumansicht — nach Status (Offen/In Bearbeitung/Erledigt), nach Priorität, nach Zuweisung, nach Liste/Goal
  - **Mitte:** Liste (Tabelle) oder Kanban-Board — umschaltbar
  - **Rechts:** TaskDetail — ausgewählter Task mit Beschreibung, Subtasks, Zuweisung, Fälligkeit
 - **Aufwand:** 2-3 Tage
 **Gesamtaufwand Phase 4:** ~2-3 Tage
 ---
 ### Phase 5: Kalender UI-Überarbeitung (1 Tag)
 #### 5.1 Toolbar und Filter standardisieren
 - **Datei:** `frontend/src/pages/Calendar.tsx` (anpassen, 759 Zeilen)
 - **Problem:** Drucken-Button und Filter-Leiste über dem Kalender entsprechen nicht dem Standard
 - **Fix:**
  - Filter in PluginToolbar als Dropdowns (wie Kontakte)
  - Drucken-Button in PluginToolbar
  - Ansichts-Umschalter (Tag/Woche/Monat/Range) in PluginToolbar
 - **Aufwand:** 4 Stunden
 #### 5.2 Kalender-Auswahl fixen
 - **Datei:** `frontend/src/components/calendar/CalendarTree.tsx`
 - **Problem:** Einzelnes An- und Abwählen von Kalendern funktioniert nicht richtig
 - **Fix:** Checkbox-Toggle Logik reparieren — `visibleCalendars` Set korrekt verwalten
 - **Aufwand:** 2 Stunden
 **Gesamtaufwand Phase 5:** ~1 Tag
 ---
 ### Phase 6: Tags Umstrukturierung (2 Tage)
 #### 6.1 Tags in Settings verschieben
 - **Datei:** `frontend/src/pages/Tags.tsx` → `frontend/src/pages/SettingsTags.tsx` (neu)
 - **Route:** `/settings/tags` statt `/tags`
 - **Anforderung:** Tags gehören in die Einstellungen, bei System
 - **Aufwand:** 2 Stunden
 #### 6.2 Tags Baumstruktur
 - **Datei:** `frontend/src/pages/SettingsTags.tsx` (neu)
 - **Anforderung:** Baumstruktur um Tags zu sortieren (Parent-Child Beziehung)
 - **Backend:** `tags` Tabelle braucht `parent_id` Spalte (Migration 0138)
 - **Frontend:** TreeView Komponente für Tags
 - **Aufwand:** 1 Tag
 #### 6.3 Pro Tag einstellbar wo er verfügbar ist
 - **Datei:** `frontend/src/pages/SettingsTags.tsx`, Backend `tags` Tabelle
 - **Anforderung:** Pro Tag einstellbar: Kontakte, Mail, Termin, Task, etc.
 - **Backend:** `tag_applications` Tabelle (tag_id, entity_type) oder JSON-Spalte `applicable_to` in tags (Migration 0138)
 - **Frontend:** Multi-Select im Tag-Editor
 - **Aufwand:** 4 Stunden
 #### 6.4 Symbol und Farbe pro Tag
 - **Datei:** `frontend/src/pages/SettingsTags.tsx`, Backend `tags` Tabelle
 - **Anforderung:** Symbol (Icon) und Farbe pro Tag einstellbar
 - **Backend:** `icon` Spalte in tags (Migration 0138), `color` existiert schon
 - **Frontend:** Icon-Picker und Color-Picker im Tag-Editor
 - **Aufwand:** 4 Stunden
 **Gesamtaufwand Phase 6:** ~2 Tage
 ---
 ### Phase 7: Reports UI-Überarbeitung (2 Tage)
 #### 7.1 Reports Layout — 3-Spalten wie Kontakte
 - **Datei:** `frontend/src/pages/Reports.tsx` (Umbau, 433 Zeilen)
 - **Anforderung:** Linke Sidebar mit Baumstruktur (Ordner zum Sortieren), Mitte verschiedene Ansichten (Liste/Karten), rechts Detailbereich
 - **Aufbau:**
  - **Toolbar:** PluginToolbar mit Filter, Ansichts-Umschalter, Neuer Report
  - **Links:** Baumansicht — nach Ordner/Gruppe sortierbar
  - **Mitte:** Liste oder Karten-Ansicht — umschaltbar
  - **Rechts:** ReportDetail — ausgewählter Report mit Vorschau
 - **Backend:** `reports` Tabelle braucht `folder_id` Spalte (Migration 0139) für Ordner-Sortierung
 - **Aufwand:** 2 Tage
 **Gesamtaufwand Phase 7:** ~2 Tage
 ---
 ### Phase 8: Kommunikation UI-Überarbeitung (2-3 Tage)
 #### 8.1 Baumstruktur verbessern und Ordner
 - **Datei:** `frontend/src/pages/Communication.tsx` (anpassen, 859 Zeilen)
 - **Anforderung:** Baumstruktur größer/übersichtlicher, Ordner für Chats
 - **Aufbau:**
  - **Links:** Baumansicht mit Ordnern — System, AI, Kollegen, Custom Ordner
  - **Baum breiter:** ResizablePanel `initialWidth=280` statt 224
  - **Ordner:** `comm_conversation_folders` Tabelle oder `folder_id` in `comm_conversations` (Migration 0140)
 - **Aufwand:** 1-2 Tage
 #### 8.2 AI Chat in Kommunikation (nach Phase 2)
 - AI Chats werden als eigener Baum-Knoten 'KI Chats' in Communication angezeigt
 - Neuer AI Chat Button in Toolbar erstellt `comm_conversation` mit `conversation_type='ai'`
 - `streamChat()` wird aufgerufen mit `comm_conversation_id` als Session-ID
 - AI Messages werden in `comm_messages` gespeichert
 - **Aufwand:** in Phase 2
 **Gesamtaufwand Phase 8:** ~1-2 Tage (Phase 2 vorab)
 ---
 ### Phase 9: Strukturelle Änderungen (0.5 Tage)
 #### 9.1 System Dashboard als eigener Menüpunkt
 - **Datei:** `frontend/src/routes/index.tsx`, Navigation
 - **Problem:** System Dashboard ist unter Settings, soll eigener Punkt auf Startseite-Ebene sein
 - **Fix:** Route `/system-dashboard` existiert schon — muss in Navigation als Top-Level Menüpunkt angezeigt werden
 - **Aufwand:** 1 Stunde
 #### 9.2 Mail — Postfach mit IMAP anlegen testen
 - **Datei:** `frontend/src/pages/Mail.tsx`, `frontend/src/pages/MailSettings.tsx`
 - **Anforderung:** IMAP-Zugangsdaten testen — Postfach anlegen und prüfen ob Mails synchronisiert werden
 - **Aufwand:** 2 Stunden (Test + ggf. Bugfix)
 **Gesamtaufwand Phase 9:** ~0.5 Tage
 ---
 ### Phase-O-Phasenübersicht
 | Phase | Inhalt | Aufwand | Migration | Abhängigkeit |
 |-------|--------|---------|-----------|-------------|
 | 1 | Echte Bugs fixen | 2-3 Tage | Keine | Keine |
 | 2 | AI Assistent → Kommunikation | 2-3 Tage | 0137 | Phase 1.6 |
 | 3 | Wiki UI + WYSIWYG | 3-4 Tage | Keine | Phase 1.4 |
 | 4 | Tasks UI neu | 2-3 Tage | Keine | Keine |
 | 5 | Kalender UI | 1 Tag | Keine | Phase 1.5 |
 | 6 | Tags Umstrukturierung | 2 Tage | 0138 | Keine |
 | 7 | Reports UI | 2 Tage | 0139 | Keine |
 | 8 | Kommunikation UI | 1-2 Tage | 0140 | Phase 2 |
 | 9 | Strukturelle Änderungen | 0.5 Tage | Keine | Keine |
 **Gesamtaufwand:** ~17-22 Tage
 #### Reihenfolge:
 1. **Phase 1** (Bugs) — zuerst, damit grundlegende Funktionen arbeiten
 2. **Phase 9** (Strukturelle Änderungen) — schnell, wenig Aufwand
 3. **Phase 5** (Kalender) — kleines Update, baut auf Phase 1 auf
 4. **Phase 2** (AI Assistent → Kommunikation) — entfernt paralleles System, baut auf Phase 1.6 auf
 5. **Phase 6** (Tags) — unabhängig, Backend + Frontend
 6. **Phase 4** (Tasks) — großer Umbau, unabhängig
 7. **Phase 3** (Wiki) — größter Umbau (WYSIWYG Editor), baut auf Phase 1 auf
 8. **Phase 7** (Reports) — großer Umbau, unabhängig
 9. **Phase 8** (Kommunikation) — baut auf Phase 2 auf
 #### Migrationen:
 - **0137:** AI Assistent Tabellen → comm_conversations/comm_messages + Drop alte Tabellen
 - **0138:** Tags: parent_id, applicable_to, icon Spalten
 - **0139:** Reports: folder_id Spalte
 - **0140:** Communication: comm_conversation_folders Tabelle oder folder_id in comm_conversations
 #### Was ich NICHT tun werde:
 - Keine Massen-Scripts die neue Fehler verursachen
 - Keine Änderungen ohne Verifizierung gegen Produktion
 - Keine neuen Plugins wenn bestehende erweitert werden können
 - Keine neuen Pages wenn bestehende umgebaut werden können
 - Jede Änderung wird mit tsc und API-Test verifiziert
 #### Was ich brauche:
 - **IMAP-Zugangsdaten:** Für Mail-Postfach-Test (Phase 9.2)
 ---
 ## Zusammenfassung
 | Phase | Dauer | Hauptdeliverable |
@@ -1223,8 +1590,360 @@ Trigger / Event / Cron / Webhook / Agent
 | I — Integration & Human-AI Workstream | 6 Wochen | Agent↔Workflow↔Knowledge↔Communication, echte MiniApps, Shared/Proactive/Mobile Workstreams, Dashboard, MCP, Polish |
 | J — Controlled Self-Improvement | 5 Wochen | Improvement Signals/Proposals, Evaluation/Dry-Run, Approval, Versionierung/Rollback, Wirkungsmessung |
 | K — EU Compliance Finalization | 1 Woche | AI-Use-Case-Register, DPIA/AI-Impact-Support, Incident/Retention, Compliance-E2E, Betriebsdoku |
 | **L — Dokumente-Generator** | **~3 Wochen** | Briefpapier + Block-System + Drag/Drop-Editor + KI-Steuerung + E-Rechnung (Contract-Muster wie Import/Export) |
 | **Total** | **52 Wochen** | **LeoPlatform Endstand-Kern inkl. Privacy/DSGVO/EU-AI-Act-by-Design** |
 ---
 *Diese Roadmap basiert auf dem Endstand-Audit des aktuellen Code-Archivs und der gemeinsamen Detail-Review. Ziel bleibt: keine unnötigen Universalmodelle, keine Massenrefactorings und keine parallelen Mechanismen. Gemeinsame technische Kerne werden dort genutzt, wo Semantik wirklich gleich ist; fachliche Speziallogik bleibt erlaubt. Bestehender funktionierender Code wird respektiert. Die eingebauten Privacy-/AI-Compliance-Funktionen schaffen technische Voraussetzungen und Nachweise; die rechtliche Konformität eines konkreten Deployments/Branchenplugins hängt zusätzlich von dessen tatsächlichem Zweck, Datenverarbeitung, Betreiberrolle und organisatorischen Maßnahmen ab.*
 ---
 ## Phase L — Dokumente-Generator ✓ ABGESCHLOSSEN (2026-08-29/30, Commits b311ab7 + 559bba6, deployed, Health healthy, Alembic 0143)
 **Ziel:** Zentrale Dokument-Generierung mit Briefpapier + dynamischen Blöcken, Drag/Drop-Editor, KI-Steuerung, E-Rechnung-Fähigkeit. Module registrieren ihre Blöcke als Contribution (Contract-Muster wie Import/Export).
 **Basis:** report_generator-Plugin (Jinja2-Templates, pdf_generator.py, Background-Jobs, ReportTemplate/ReportInstance-Models) — Erweiterung statt Neubau.
 ### L1 — Block-System (2-3 Tage)
 - Briefpapier-Modell (pro Tenant: Logo, Header/Footer, CSS)
 - Block-Modell (typ: text/table/chart/placeholder, order, content)
 - Block-Registrierung durch Module via Contract (`document_blocks` wie `importexport_entities`)
 - print_templates-Tabelle (Briefpapier-Ref + Block-Komposition)
 ### L2 — Drag/Drop-Editor (3-5 Tage)
 - Frontend: Block-Palette (registrierte Blöcke des Moduls), Canvas, Platzierung
 - Placeholder-Editor (`{{firstname}}`, `{{company.logo}}`)
 - Live-Preview
 ### L3 — Renderer-Integration (1-2 Tage)
 - report_generator-Engine an Block-Komposition anbinden
 - Jinja2-Templates aus Block-Komposition generieren
 - PDF/Excel/CSV-Output über bestehende Engine
 ### L4 — KI-Steuerung (1-2 Tage)
 - „Erstelle Rechnungsvorlage" via AI-Module (agent_loop existiert)
 - Template-Vorschläge aus Block-Komposition
 ### L5 — E-Rechnung (2-3 Tage)
 - XRechnung/ZUGFeRD-Format (Verkauf-Modul registriert Rechnungs-Blöcke)
 - Klären: Steuer-Behörden (Deutschland, B2B-Pflicht ab 2027) oder Kunden-Lieferungen?
 **Abhängigkeiten:** L5 benötigt Phase F (Agents) und das Verkaufs-Modul (noch nicht gebaut).
 **Verwandte Issues:** #359 (Import/Export Contribution — gleiche Plugin-Philosophie).
 ---
 ## Phase M — MiniApp-Plattform & Dashboard-Builder (geplant, user-abgestimmt 2026-08-29)
 **Ziel:** MiniApps als universelles, teilbares UI-Baustein-System über alle Hosts (Chat, Dashboard, Windows, AI-Agenten). Dashboard-Builder mit Edit-Modus, Drag&Drop, Resize, Tabs und pro-Widget-Settings. System-Dashboard-Teile werden zurück in Plugins gebaut (Core wird zum reinen Host).
 **Basis (Live-Bestand 2026-08-29):**
 - `kommunikation/miniapp_registry.py` (92 Z., MiniAppDef mit register/unregister/unregister_plugin — inkl. Lifecycle-Cleanup)
 - `MiniAppContribution` im Manifest-Schema (app_id, name, icon, description, render_schema) — **LÜCKE: kein permission-Feld**
 - `FrontendDashboardWidget` im Manifest (id, component, col_span, row_span, permission) — **LÜCKE: kein settings_schema**
 - `MiniAppBlock.tsx` als comm-Block-Typ (Chat-Host — fertig verdrahtet)
 - `DashboardGrid`/`DashboardWidgetLoader` + 4 Widgets (RecentContacts, TasksSummary, CalendarUpcoming)
 - Dashboard.tsx (170 Z.) mit hardcodierten StatCards (via contacts-Contract `get_counts`), ActivityFeed (via Audit-Log), System-Metrics (Admin-only) — **Rückbau-Bestand**
 - `app/routes/dashboard.py` listet manifest `dashboard_widgets` (bereits permission-agnostisch, nur `dashboard:read` auf Endpoint-Ebene)
 - @dnd-kit (core/sortable/utilities) bereits im Projekt (Referenz: SettingsMenuOrder, Dokumente-BlockEditor)
 - windowStore (Window-Manager) existiert für spätere Hosts
 **Architektur-Entscheidung (user-bestiätigt):** EINE Universal-Registry statt zweier paralleler Systeme — `dashboard_widgets` wird Alias von `miniapps`; jedes Plugin/System registriert MiniApps via Contribution (gleiches Muster wie settings_pages/print document blocks, #359-Philosophie). Ein Host-Set: Chat-Block (fertig), Dashboard (neu), Windows (M6), AI-Agenten-Tool-Ausgabe (M6).
 **⚠️ Abgrenzung Workspace ≠ Dashboard (user-korrigiert 2026-08-30):**
 - **Dashboard (diese Phase M)** = PERSÖNLICH: jeder User baut eigene Dashboards (Layout/Tabs/Instanzen) — Speicher ist die NEUE `dashboards`-Tabelle (owner-basiert). NIEMALS `workspace_widgets` dafür verwenden.
 - **Workspace (Phase N)** = ADMIN-Kontext für Gruppen: welche Module sichtbar sind (fertig) + Modul-Teilmengen (Scopes) + welche Widget-TYPEN der Workspace anbietet (`workspace_widgets`, existiert bereits — Workspace-Eigentum).
 - Schnittstelle: der aktive Workspace begrenzt nur die VERFÜGBAREN Widget-Typen; das persönliche Layout bleibt User-Eigentum und wird von keinem Workspace überschrieben.
 ### M1 — Universal-MiniApp-Registry (2-3 Tage)
 - miniapp_registry aus kommunikation-Plugin in Plugin-Layer heben (Plattform-Konzept, kommunikation behält Chat-Hosting)
 - MiniAppDef/MiniAppContribution erweitern: `permission` (Pflicht-Feld, fail-closed), `settings_schema` (generisches Settings-Form), `col_span`/`row_span`, `min_size`
 - `dashboard_widgets` (Manifest) → Alias von `miniapps` (Rückwärtskompatibilität, ein Contribution-Typ)
 - `/api/v1/miniapps`-Endpoint: Registry-Listing **server-seitig permission-gefiltert** (nur MiniApps sichtbar, für die der User die Permission hat)
 - Host-Rendering prüft Permission zusätzlich beim Render (Defense-in-Depth wie Plugin-Routen)
 - Lifecycle: Plugin-Deaktivierung → unregister_plugin → Widgets verschwinden aus allen Hosts
 ### M2 — Dashboard-Backend (2-3 Tage)
 - `dashboards`-Tabelle: pro User mehrere Dashboards, Tabs, Layout als JSONB (`[{tab, widgets: [{app_id, settings, col, row, span}]}]`), RLS fail-closed + crm_api-Policy (0084-Muster)
 - CRUD-Endpoints (list/create/update/delete + set-default), Tenant-Scoping, Owner-only oder Admin
 - Dual-Path: Plugin-SQL idempotent + Alembic-Konvergenz (Gate-B-Muster wie 0143)
 - Default-Dashboard-Seed beim ersten Aufruf (aus Registrierungs-Order abgeleitet)
 ### M3 — Dashboard-Builder-Frontend (3-5 Tage)
 - Edit-Modus als Modus-Schalter: aktiv → Widgets hinzufügen/entfernen, Größe ändern (col/row-span), Einstellungen; beenden → persistiertes Layout, reine Ansicht
 - Drag&Drop-Grid (@dnd-kit, Referenz BlockEditor/SettingsMenuOrder): Platzierung + Umsortieren
 - Widget-Palette: verfügbare MiniApps (aus `/api/v1/miniapps`, permission-gefiltert), Suche/Kategorie
 - Generisches Settings-Form pro Widget aus `settings_schema` (gleiche Philosophie wie Block-Config-Panels beim Dokumente-Editor)
 - Tabs: mehrere Dashboards pro User, Tab-Verwaltung im Edit-Modus
 - Dashboard.tsx wird zum reinen Host (keine hardcodierten Inhalte mehr)
 ### M4 — System-Rückbau (2-3 Tage)
 - StatCards (Firmen-/Kontakt-Zähler via contacts-Contract) → contacts-Plugin-MiniApp
 - Aktiv-diese-Woche/Neu-diesen-Monat + ActivityFeed (Audit-Log) → audit/auditlog-MiniApp
 - System-Metrics-Block (DB/Redis/Worker/LLM-Kosten, Admin) → System-MiniApp mit `settings:read`-Permission
 - Bestehende Dashboard-Widgets (RecentContacts, TasksSummary, CalendarUpcoming) zu MiniApps migrieren (gleiches Format, dann Chat-fähig)
 ### M5 — Plugin-MiniApps (2-3 Tage)
 - contacts, tasks, calendar, wiki, dms, mail, knowledge (Graph-RAG), automation liefern jeweils MiniApps via Manifest-Contribution
 - Jede MiniApp automatisch überall verfügbar: Chat senden + Dashboard platzieren
 - Permission je MiniApp passend zum Owner-Modul (z.B. `tasks:read` für TaskSummary)
 ### M6 — Weitere Hosts (2-3 Tage)
 - AI-Agenten-Tool: Agent kann MiniApp als Ausgabe-Block in Chat-Antwort einbetten (miniapp-Block-Typ existiert, Tool-Registry erweitern)
 - Windows (windowStore): MiniApp per Klick/Expand in eigenem Fenster öffnen
 - Evaluiert: Wiki-Einbettung (BlockRenderer-Muster) — nur wenn Bedarf bleibt
 **Abhängigkeiten:** M3 benötigt M1+M2. M4/M5 nach M3 (Host muss stehen). M6 zuletzt.
 **Verwandte Phasen/Issues:** Phase L (Gleiche Contribution-Philosophie), #359 (Contract-Muster), Phase F (Agenten für M6).
 ---
 ## Phase N — Workspace-Scopes: Modul-Teilmengen pro Arbeitskontext (geplant, user-abgestimmt 2026-08-30)
 **Ziel:** Workspaces werden zu voll anpassbaren Arbeitskontexten: jedes Modul kann pro Workspace auf eine Teilmenge eingeschränkt werden (z.B. nur Kontakt-Ordner X+Y, nur DMS-Ordner „Angebote", nur Mail-Postfach vertrieb@, nur Kalender „Vertrieb"). Admin-definiert für zugewiesene User-Gruppen — klar getrennt vom persönlichen Dashboard (Phase M).
 **Klare Trennung (user-korrigiert):**
 - Workspace = Admin-Kontext, Gruppen-Feature: WAS ist sichtbar/verfügbar (Module, Teilmengen, Widget-Typ-Angebot via `workspace_widgets`)
 - Dashboard = persönlich, User-Feature: WIE ICH mein Dashboard baue (Phase M, `dashboards`-Tabelle)
 - Beide Systeme berühren sich NUR an einer Schnittstelle: der aktive Workspace begrenzt das Widget-Typ-Angebot; das persönliche Layout bleibt unberührt.
 **Basis (Live-Bestand, 0 Umbau):**
 - `workspace_modules.config` (JSONB) — existiert, ungenutzt → Scope-Speicher pro Modul
 - `X-Workspace-ID` Header + API-Client-Interceptor (pro Tab) — existiert, wird vom Backend gelesen
 - `/api/v1/workspaces/context` — existiert, liefert Modul-Konfiguration aus
 - Sidebar filtert bereits live (isModuleVisible — Consumer-Beweis)
 - 17/17 Workspace-Tests grün, RLS auf allen 4 Tabellen
 - Contract-Muster für die Scope-Registry (wie document_placeholders)
 **Security-Invariante:** Scope = reine UND-Einschränkung. Sichtbarkeit = Workspace-Scope ∧ RLS ∧ ABAC ∧ Permissions. Ein Workspace kann NIE mehr sichtbar machen, nur weniger. Ohne aktiven Workspace = kein Filter (rückwärtskompatibel, wie Sidebar).
 ### N1 — Scope-Registry via Contract (2 Tage)
 - Plugins deklarieren `workspace_scopes()` → verfügbare Scope-Dimensionen + Wertequellen (z.B. „folder_ids, Multiselect, via /contacts/folders")
 - `/context` liefert `config` der Module mit aus; Scope-Definitionen-Endpoint für den Editor
 ### N2 — Dynamischer Scope-Editor (2-3 Tage)
 - WorkspaceManager: pro Modul automatisches Filter-UI aus der Registry (Multiselects für Ordner/Postfächer/Kalender, Toggles, Standard-Ansichten)
 - Speicherung in `workspace_modules.config`
 ### N3 — Erste vier Module integrieren (2-3 Tage)
 - Contacts: Ordner-Teilmengen, Firmen/Personen-Filter, Standard-Saved-View
 - DMS: Ordner-Teilmengen, Datei-Typ-Filter
 - Mail: Postfach-Teilmengen
 - Calendar: Kalender-Teilmengen, Standard-Ansicht
 - Backend respektiert X-Workspace-ID bei Listen (additive Filter-Logik, kein Umbau bestehender Routes)
 ### N4 — Restliche Module (2-3 Tage)
 - Tasks (Boards/Listen, „nur meine"), Kommunikation (Räume), Wiki (Kategorien), Reports/Dokumente (Vorlagen), Automation (Agenten), Tags, Suche (Provider), Navigation (Menü-Reihenfolge, Startseite pro Workspace)
 - Dashboard-Schnittstelle: workspace_widgets bestimmt verfügbare Widget-TYPEN pro Workspace (Admin) — persönliches Layout bleibt Phase M
 **Abhängigkeiten:** unabhängig von Phase M. N3/N4 nach N1+N2.
 ---
 ## Phase P — Notizen-App (Notion-artig, ersetzt das Wiki komplett) (geplant, user-abgestimmt 2026-08-30)
 **Ziel:** Aus dem Wiki wird eine Notion-artige Notizen-/Firmen-Wissen-App: Seiten-Baum (beliebig tief, statt flacher Kategorien), Inline-Block-Editor mit Slash-Menü und Drag&Drop, Quer-Verweise zwischen Seiten, MiniApp-Einbettung, vollständige Such-Indexierung. Das alte Wiki wird KOMPLETT ersetzt (keine Legacy-App parallel).
 **User-Entscheidungen (2026-08-30):**
 - Keine Notion-Datenbanken zunächst — stattdessen MiniApps als einbettbare Blöcke (Phase M-Synergie)
 - Später: Plugin-Erweiterbarkeit (eigene Block-Typen via Contract), evtl. Datenbank-Block als Plugin nachlieferbar
 - Vollständige Such-Indexierung ist PFLICHT (Notizen/Firmen-Wissen auffindbar)
 - Quer-Verweise (Seiten verlinken Seiten)
 **Edit-Konzept (Notion-Recherche 2026-08-30):** Notion hat KEINEN separaten Bearbeitungsmodus — "all content is editable by default": Klick in die Seite = tippen, Auto-Save im Hintergrund, Slash-Menü für Block-Typen. Confluence macht stattdessen Draft/Publish-Workflow. Für uns: Live-Inline-Editing wie Notion als Standard; der "Lese-Modus" entsteht natürlich über Permissions (nur-Lesen = gerenderte Seite ohne Editierfunktion) + optional Page-Lock. Kein Mode-Toggle im UI nötig.
 **Basis:** Wiki-Plugin (486 Z. Backend: WikiCategory/WikiArticle/WikiArticleVersion + 10 Endpoints) wird erweitert, nicht neu gebaut. Block-Muster aus Phase L (JSONB {id, type, config}), dnd-kit vorhanden, Custom-Field-Engine für spätere Properties, Entity-Links für CRM-Quer-Verweise vorhanden. Unified Search: Provider-Registry + BaseSearchProvider (Embeddings + hybrid FTS/vector) + chunking existieren — die App liefert Provider + Re-Index-Hook.
 ### P1 — Datenmodell & Migration (2 Tage)
 - WikiArticle → WikiPage: `blocks JSONB` (statt content Text), `parent_id` (Seiten-Hierarchie statt Kategorien), `icon`, `is_favorite`, **`is_locked` (Page-Lock, user-entschieden 2026-08-30)**; Quer-Verweise als Block-Typ page_link
 - Migration: Markdown-Artikel → Text-Blöcke, Kategorien → Eltern-Seiten, Versionen (WikiArticleVersion) bleiben erhalten
 - Dual-Path: Plugin-SQL idempotent + Alembic-Konvergenz (Gate-B-Muster)
 ### P2 — Sidebar mit Seiten-Baum (2 Tage)
 - Notion-artiger Baum: Seiten anlegen/umbenennen/löschen, Drag&Drop-Umsortierung (dnd-kit), + Button, Kontextmenü, Favoriten, Seitensuche
 - Ersetzt die alte Kategorien-Navigation komplett
 ### P3 — Inline-Block-Editor (4-5 Tage) — Herzstück
 - Live-Inline-Editing (Klick = tippen, kein Mode-Toggle), Auto-Save debounced in blocks JSONB
 - Slash-Menü: „/" → Block-Typ-Auswahl
 - Block-Typen: Text, H1-H3, To-do, Toggle (auf/zu), Quote, Callout, Code, Divider, Bild, Page-Link (Quer-Verweis mit Auto-Vervollständigung), MiniApp
 - Drag&Drop-Block-Umsortierung (dnd-kit, Phase-L-Erfahrung)
 - **Page-Lock (user-bestaetigt 2026-08-30):** `is_locked = true` = Seite nicht editierbar (auch mit wiki:write). Backend lehnt Block-Updates mit 409 `page_locked` ab; Frontend zeigt rein gerenderte Seite + Schloss-Badge; Lock setzen/loeschen nur Owner oder Admin (Lock-Button in der Seitentoolbar); gelockte Seiten bleiben fuer Search/Versionen/Kommentare normal indexiert
 ### P4 — MiniApp-Blöcke + Plugin-Erweiterbarkeit (1-2 Tage)
 - „/ MiniApp"-Block-Typ: registrierte MiniApps in Seiten rendern (erster MiniApp-Konsument neben Chat — treibt Phase M mit)
 - Contract `wiki_blocks()`: Plugins melden eigene Block-Typen für den Editor an (Muster document_blocks) — Basis für späteren Datenbank-Block
 ### P5 — Vollständige Such-Indexierung (1-2 Tage)
 - Content-Extraktion aus Blöcken (Text/Überschriften/To-do/Callout) → content_tsv + Embedding-Chunks (chunking.py)
 - WikiPage-Search-Provider an unified_search (hybrid FTS + vector, Re-Index bei jedem Auto-Save)
 - Suchergebnis verlinkt direkt auf Seite + Sprungmarke
 **Abhängigkeiten:** P4 benötigt M1 (Universal-Registry). P1-P3, P5 unabhängig startbar.
 ## Phase Q — Frontend-Plugin-Architektur ✓ ABGESCHLOSSEN (2026-09-13, Commits 895f85d + b666fe5, deployed, Health healthy)
 > **Umgesetzt am selben Tag wie geplant.** Q3 (Generator + PluginLoader) + Q4 (MiniAppHost)
 > in 895f85d; Q1 (statische Plugin-Routen entfernt) + Q2 (Settings-Routen + Renderer-Variante)
 > in b666fe5. Details und Live-Beweise: PROGRESS.md Phase-Q-Section.
 **Ziel:** Die letzten verbliebenen Plugin-Grenzverletzungen im Frontend beseitigen — ein Plugin soll sein Backend, Manifest UND React-Seite liefern können, ohne dass zentrale Frontend-Dateien angefasst werden müssen. Basis: externes Architektur-Audit (2026-09-13), dessen Backend-Punkte bereits gefixt sind (siehe PROGRESS.md „Externer Architektur-Audit"); die vier Frontend-Punkte sind bewusst als eigene Phase geplant, weil sie ein durchdachtes Build-Time-Discovery-Konzept erfordern (Vite kann dynamische Imports zur Laufzeit im Production-Bundle nicht zuverlässig auflösen).
 ### Q1 — Statische Plugin-Routen aus routes/index.tsx entfernen (Doppel-Architektur)
 - Status quo: `/calendar`, `/dms`, `/mail`, `/reports`, `/tasks`, `/communication`, `/workflows`, `/import-export`, `/wiki`, `/agents`, `/automation` sind statisch im zentralen Router eingetragen UND kommen gleichzeitig über die Plugin-Manifeste via PluginRouteRenderer.
 - Ziel: Nur noch PluginRouteRenderer bedient Plugin-Seiten; statische Einträge nur für echte Core-Seiten (Dashboard, Settings-Shell, Login, Trash, Approvals bis Core-Migration).
 - Risiko: Manifest-Routen müssen Permissions, Layout-Einbindung (AppShell-Children vs. eigenständig) und Ladezustände 1:1 abbilden.
 ### Q2 — Statische Settings-Routen ausdünnen
 - Status quo: settings/roles, users, groups, mail, notifications, ai, ai-proactive, automation, documents sind statisch UND via settings_pages der Manifeste vorhanden.
 - Ziel: settings_pages (Manifest) wird einzige Wahrheit für Plugin-Settings-Seiten; statische Einträge nur für Core-Settings (theme, system, backup, webhooks, menu, workspaces).
 ### Q3 — STATIC_COMPONENT_MAP ersetzen durch Build-Time-Discovery
 - Status quo: PluginLoader.tsx hält eine zentrale Komponenten-Liste (~26 Einträge). Ein neues Plugin muss die Leo-Frontend-Codebasis anfassen.
 - Ziel: Build-Skript scannt app/plugins/builtins/*/plugin.py auf FrontendPageRoute/SettingsPage/Component-Pfade und generiert automatisch eine Import-Map (generated, committet), die Vite statisch chunken kann. Keine manuelle Zentral-Liste mehr.
 ### Q4 — widgetRegistry in MiniAppHost durch generierte Map ersetzen
 - Status quo: 11 Widget-Komponenten sind zentral hardcodiert (RecentContactsWidget, TasksSummaryWidget, ...).
 - Ziel: Q3-Mechanismus deckt auch dashboard_widgets/miniapps component-Pfade ab; MiniAppHost nutzt dieselbe generierte Import-Map.
 **Reihenfolge (wie umgesetzt):** Q3 → Q1/Q2 → Q4 (Q4 fiel mit Q3 mit, da MiniAppHost dieselbe generierte Map nutzt). Jeder Schritt mit Vitest-Sicherung der betroffenen Seiten und Production-Build-Verifikation (Chunk-Existenz prüfen).
 ## Externaudit Astra 2026-09-17 (41 Findings) — Sanierung PHASE S (bestätigt, NÄCHSTE PHASE, vor/neben R)
 **Auditergebnis:** 2 P0 (KI führt nicht freigegebene Tools aus; Mandantenverwaltung kann globale Anmeldeidentitäten ändern), 29 P1, 10 P2. Geprüft am vollständigen Stand ee5545d (ZIP). Interne Verifikation am 2026-09-17: 10 Findings stichprobenartig am Code nachgelesen (F01, F02, F05, F08, F10, F12, F17, F24, F37, F41) — **alle 10 korrekt**. Übrige Findings: detailliert mit Zeilennummern belegt, Detail-Verifikation erfolgt jeweils bei Umsetzung. Volltext des Audits: [docs/audits/astra-audit-2026-09-17.md](docs/audits/astra-audit-2026-09-17.md); Kernpunkte je Finding in den Wellen-Issues.
 **Strukturdiagnose (Astra):** Mehrere Stellen verwalten denselben Zustand (Plugin-Aktivität, Schema); Contracts garantieren zu wenig Verhalten; API- und Worker-Ausführung nicht gleichwertig; Berechtigungsprüfungen liegen zu weit vom Seiteneffekt entfernt; Statusanzeigen teils von tatsächlicher Funktion entkoppelt. — Bestätigt und deckt sich mit den realen Incidents (#389 Plugin down 4 Wochen, #380 158 Events failed).
 **Sanierungswellen (Reihenfolge nach Risiko, an Astra-Empfehlung angelehnt):**
 ### Welle S1 — Sicherheitsgrenzen (P0 + Auth/Permission-Kette) — ZUERST
 - **F01 (P0)** agent_loop._execute_tool: Tool-Ausführung ohne Allowlist- und Permission-Check — unmittelbar vor Handleraufruf prüfen: Tool in der dem LLM angebotenen Liste, required_permission gegen aktuelle User-Rechte, Verbote, Mandant, Plugin aktiv, ggf. Approval. Abnahme: nicht angebotenes Tool → Ablehnung, Handler bleibt null.
 - **F02 (P0)** users.py update_user: globale User.email durch Mandanten-Admin (users:write) änderbar → globale Identitätsänderungen (email, is_system_admin global, Passwort) von Mandantenverwaltung trennen; nur Selbstservice oder echte globale Admin. Abnahme: Tenant-Admin kann globale E-Mail/Aktivstatus fremder Mandanten-Mitglieder nicht ändern.
 - **F10** require_permission: Token-Scopes ersetzen User-Rechte (early-return) → effektive Rechte = Schnittmenge(User, Token-Scopes, Delegation), Verbote vorrangig. Abnahme: Token mail:write + User ohne mail:write → 403.
 - **F05** require_active_plugin läuft vor Auth/ohne Mandantenkontext → Plugin-Gate an authentifizierten Kontext binden, fehlender Kontext = ablehnen. Abnahme: mandantendeaktiviertes Plugin → 403 auch bei gültiger Session.
 - **F03** Session-Widerruf: Deaktivierung/Austritt/Löschen/Passwortwechsel müssen in Redis- UND DB-Fallback-Sessionpfaden wirken; Widerruf dauerhaft speichern. Abnahme: Widerruf wirkt auch bei Redis-Ausfall.
 - **F11** Approval-Resolution: approver_id/Ablauf/Gruppe/Atomarität prüfen, Entscheider getrennt speichern, Approval an Aktion+Argumente+Revision binden.
 - **F15** Workflow-HTTP: aufgelöste IPv4/6-Ziele gegen Privatnetz prüfen, Verbindung an geprüfte Auflösung binden, Redirects prüfen.
 - **F20** prestart überschreibt gezielte Rechte-Entzüge (0100) mit pauschalem GRANT DELETE → Tabellenschutz nur migrieren; keine Rechteanhebung beim Start.
 - **F21** test_migrations.sh: MIGRATION_DATABASE_URL überschreiben + Zielidentität vor DDL prüfen (sonst Gefahr für echte DB).
 - **F23** Tenant-Backup-API triggert datenbankweiten Restore → Gesamtrestore als globale Betriebsoperation mit separater Berechtigung.
 - **F30** Admin-Standardpasswort bei unkonfiguriertem Start → verpflichtendes Secret oder sicherer Einmal-Generierung.
 ### Welle S2 — Ausführung verbinden (Worker, Jobs, Contracts, Migrationen)
 - **F06** Worker registriert keine der 44 Plugin-Event-Handler (BasePlugin.register_event_handlers ist leer) → API und Worker dieselbe idempotente Registrierung; Abnahme über echten Outbox-Durchgriff (Kontakt anlegen → Worker → Suchindex).
 - **F07** Hintergrundjobs verlieren Mandantenkontext/Transaktionen → Mandant+Auftraggeber im Job-Payload Pflicht; Kontext vor erstem SQL; fachliche Änderung+Audit+Outbox gemeinsam committen.
 - **F08** External-Agent-API: require_permission an Cookie-Auth gebunden (Bearer nie erreicht) + get_db() ist kein Contextmanager (TypeError) → gemeinsamer geprüfter Auth-Kontext für Cookie+Token; Session-Factory statt get_db.
 - **F09** CRM-/MCP-Tools senden nicht anerkannte interne Header → Delegationsmechanismus (delegation_token.py) einbinden; UI und Agent gleiche Rechte-Antwort.
 - **F12** Workflow approve/reject: approval["id"] auf ORM-Objekt (TypeError) + falsche resolve-Signatur → an zentralen Vertrag anpassen, wartende Freigabe auflösen statt Selbst-Genehmigung.
 - **F13** Workflow-Engine: acquire_lock ohne Aufrufer, Idempotenz unvollständig, Resume ungesperrt → Engine als verbindlichen Zustandsübergang; Abnahme: Worker-Neustart + parallele Resume → keine Doppel-Mails.
 - **F14** enforce_data_policy lässt Strings ungefiltert + läuft nur vor der Schleife mit db=None → strukturierte Filterung vor Serialisierung; JEDE LLM-Anfrage (inkl. Tool-Antworten) durch Policy; nicht ladbare Policy = Versand-Stop.
 - **F16** Plugin-Lifecycle: prestart reaktiviert absichtlich deaktivierte Plugins; Aktivierungsfehler lassen DB-Zustand aktiv → gewünschten Zustand von Installation/Mandantenfreigabe/Laufzeitgesundheit trennen; Abnahme: Deaktivierung überlebt Neustart.
 - **F17** 6 Produktionsstellen rufen ContractRegistry.get() auf (existiert nicht; nur get_contract) → Aufrufer fixen; Abnahme über reale Einstiegspunkte (Miniapp-Tools, proaktive Hinweise, Report-Jobs).
 - **F18** Drei Schema-Verfahren (Alembic/Plugin-SQL/sync_plugin_schema) mit Sync-Verlust bei Unique/Partial-Indizes → einen Migrationsbesitzer pro Objekt; Startup-Sync als lesender Driftbericht.
 - **F19** alembic/env.py lädt nur app.models (46/129 Tabellen; Sortierung scheitert) → deterministische vollständige Modelldiscovery.
 - **F31** Provider-Registry vs. Reindex-Listen divergieren → Plugin-Beiträge als gemeinsame Quelle; Abnahme: neuer Provider wird vollständig indiziert.
 - **F37** SMTP-Env-Namen (SMTP_USER vs smtp_username u.a.) → Compose/Config/Doku angleichen; Abnahme: Reset-/Alarm-Mail authentifiziert.
 - **F40** Plugin-Migrationen nur Dateiname-Tracking → Hashes speichern und prüfen; Sollzustand vorhandener Tabellen (Spalten/FKs/Policies) vergleichen.
 - **F41** Agenten-Stundenlimit zählt ab jetzt() statt letzte Stunde → timedelta(hours=1); Kontingent atomar reservieren.
 ### Welle S3 — Fachliche Integrität (Daten- und UI-Korrektheit)
 - **F25** CSV-Import: Rollback vernichtet frühere Zeilen, Zähler behalten Erfolge, RLS-Kontext weg → Savepoints pro Zeile, Original-Zeilennummern; Zähler = Persistenz.
 - **F26** DMS-Dedup vermischft Identität (fremder Datensatz statt eigener Upload) → Content-Storage vs. Fachobjekt trennen; jeder Upload eigene Identität/Rechte.
 - **F27** Kalender: SQL-Filter wirft Serien weg bevor Wiederholungen berechnet werden; end_at-Dauer; Mehrtagesüberlappung → Serie nach Laufzeit selektieren, Wiederholungen im Fenster erzeugen.
 - **F28** Import/Export ohne Fachrechte (import_export:write ≠ contacts:write; Export ohne Feldrechte) → Fachrechte UND Importrecht; Feldfilter vor Dateierzeugung.
 - **F32** Suche: entity_types=[] = alle (soll 0), Filter nach Top-N, Offset unwirksam, before_search zu spät → None/[] unterscheiden; Filter vor Limit; Hook vor Parametern.
 - **F33** Workspace-Wechsel invalidiert fachliche Querykeys nicht → Workspace in Query-Identität oder kontrolliert verwerfen.
 - **F34** Mandantenwechsel: alte Daten bis Refetch sichtbar → kontrollierter Kontextwechsel (Abbrechen, Caches leeren, Header synchronisieren).
 - **F38** pluginStore-Fehler → Dauerspinner (loaded bleibt false) → Fehler/Leer/Erfolg getrennt rendern, Retry anbieten.
 - **F39** Office-Edit-Session verweist auf /preview (PDF-only) + Callback-Route existiert nicht → funktionsfähigen Ablauf anbinden oder Feature als nicht-betriebsbereit kennzeichnen.
 ### Welle S4 — Betriebsfreigabe (inkl. korrigierter Phase R)
 - **F22** Backup im Container nicht betriebsfähig (pg_dump fehlt, Pfade nicht persistiert, Kontext-/User-Bugs) → dokumentierter Ablauf mit Programmen, Rechten, persistiertem Ziel.
 - **F24** /health/ready liefert 200 bei not_ready; Worker-Check meldet up ohne Worker → korrekte HTTP-Codes (503), Heartbeat-Alter statt Queue-Länge.
 - **F29** CI ohne Lockfiles/Testdienste/tatsächliches Artefakt → reproduzierbare Pipeline gegen eigenes Image.
 - **F36** Komponenten-Map-Generator nicht verbindlich im Build → Check an npm-Build/Dockerfile/CI hängen.
 - **F04** Suche: autocomplete/similar ohne Objekt-/Feldrechte; Snippet/Titel unfiltert zur LLM → ein Schutzpfad für ALLE Suchvarianten vor Snippet- und LLM-Übergabe.
 - **F35** PWA abgeschaltet, aber Offline-Banner verspricht Schreibspeicherung → PWA wiederherstellen ODER Banner an Realität anpassen.
 - Phase-R-Korrektur (siehe unten, bereits eingearbeitet).
 **Abnahmeszenarien quer über alle Wellen (Astra-Vorschlag, verbindlich):**
 1. Kontaktanlage → Audit/Outbox → separater Worker → Suchindex → erlaubte KI-Abfrage (F06, F07, F17, F04)
 2. Mailentwurf → Freigabe → einmaliger Versand → nachvollziehbares Ergebnis (F11, F12, F13)
 **Reihenfolge-Logik:** S1 zuerst (jede nicht autorisierte Aktion verboten), S2 parallel startbar nach S1-P0s, S3/S4 danach. Nach S1+S2 verifizierter Welle: Aufwand neu schätzen (Astra-Hinweis: die 9-14 Tage aus Phase R sind keine Schätzung für 41 Findings).
 ## Phase R — Betriebssicherheit & 95%-Produktionsreife (geplant, user-abgestimmt 2026-09-16 — läuft in S4 auf; korrigiert 2026-09-17 nach Astra-Kritik)
 **Ziel:** Von „Produktion läuft stabil" zu „Produktion verlässlich": stille Ausfälle werden automatisch erkannt und alarmiert (Minuten statt Wochen), die Test-Suite wird zum vertrauenswürdigen Regressionsschutz, Schema-Drift wird automatisch erkannt, Kernprozesse werden nach jedem Deploy regressionsgetestet, Backups sind nachweislich wiederherstellbar.
 **Warum diese Phase (Evidenz aus realen Incidents):**
 - KI-Chat war 4 Wochen still down — ai_assistant migration_failed seit 2026-08-21, entdeckt am 2026-09-16 nur durch Zufall (#389)
 - External-API war durch CSRF-Middleware für externe Systeme unbrauchbar (fix b91ee5b)
 - Outbox: 158 failed Events wochenlang unbemerkt (#380)
 - Suite-Isolation und alembic-check-Blockade verhindern verlässliche Regressionsschutz-Gates
 **95%-Definition (korrigiert 2026-09-17 nach Astra-Kritik):** Die fünf Kriterien sind kein mathematischer Reifegrad, sondern **konkrete Freigabekriterien**. Dokumentiert wird: erfüllte Kriterien, verbleibende Risiken und bekannte Grenzen (Battle-Testing im Echtbetrieb). „95 %" = Zustand, in dem jeder Ausfall laut statt still wird; die restlichen ~5 % sind Echtbetriebs-Edge-Cases, die nur echte Nutzung findet.
 **Astra-Kritik an Phase R (8 Punkte, 2026-09-17) — eingearbeitet:**
 1. ARQ-Heartbeat überwacht sich nicht selbst → zusätzlich externe Überwachung außerhalb der ARQ/Redis-Ausfallkette (z.B. Cron auf Host oder externer Uptime-Check gegen /health/ready).
 2. „Installiert aber inaktiv"-Alarm trifft absichtliche Deaktivierung → **Sollzustand** (DB desired state) mit tatsächlicher Betriebsbereitschaft vergleichen; nur Abweichung alarmiert.
 3. Leere Queue ≠ laufender Worker → Worker-Heartbeat-ALTER und Verarbeitungsnachweis messen, nicht Queue-Länge.
 4. Komplette Suite grün reicht nicht (Mocks/Admin-Tests können Rechtefehler verdecken) → zusätzlich echte API-/Worker-Prozesse mit tatsächlichen Laufzeitrollen prüfen.
 5. Ein FK-Fix + Migrationshash genügt nicht → vollständige Modelldiscovery (F19) und eindeutige Schema-Verantwortung (F18) sind Voraussetzung; R3 hängt an S2.
 6. E2E-Normalfälle prüfen Rechteentzug/Neustart nicht → Mehrmandanten-, Rollen-, Fehler- und Wiederaufnahme-Szenarien ergänzen.
 7. Monatlicher Restorejob beweist keine sichere Zielwahl → isoliertes Ziel und tatsächliche DB-+Datei-Wiederherstellung nach Containerersatz nachweisen.
 8. „95 % Produktionsreife" ist keine messbare Zahl → konkrete Freigabekriterien + verbleibende Risiken dokumentieren (siehe oben).
 **Aufwandskorrektur (Astra):** Die 9-14 Tage gelten NICHT für die Behebung aller 41 Audit-Findings (Phase S). Neue Schätzung nach Abschluss von S1+S2.
 ### R1 — Stille-Ausfälle-Wächter + Alerting (2-3 Tage) — PRIORITY 1, größter Risikoreduktor
 - ARQ-Heartbeat-Job (alle 5 Min) prüft: (a) /api/v1/plugins — installiert aber nicht active → ALARM (exakt der #389-Fall), (b) /health/ready — DB/Redis/Storage/Worker, (c) Outbox-DLQ — failed > 0 (#380-Klasse), (d) Worker-Queue-Länge
 - Alarm-Kanal: E-Mail über bestehende Mail-Infra (SMTP) an Admins; Alarm-Zustand zusätzlich als rote Badge im Admin-UI (System-Dashboard)
 - Abnahme live: Plugin absichtlich deaktivieren → Alarm muss nachweislich auslösen (Chaos-Test)
 - Bestand, auf dem aufgebaut wird (kein Neubau): /health/ready (docs/monitoring.md), ARQ-Worker (app/core/worker.py), Mail-Plugin (SMTP), System-Dashboard-Routen
 ### R2 — Test-Suite verlässlich machen (2-3 Tage)
 - Suite-Isolation fixen: Combo-Runs quaken mit „relation users does not exist" (Solo grün) — conftest.py-DB-Setup deterministisch machen
 - Vitest-Worker-OOM fixen (Worker-/Fork-Konfiguration)
 - Abnahme: `python -m pytest` kompletter Lauf grün + `npx vitest run` kompletter Lauf grün — erst DANACH gilt die Suite als verbindliches DoD-Gate
 ### R3 — Schema-Integrität automatisieren (1-2 Tage)
 - entity_attachments-FK fixen → `alembic check` läuft als Schema-Drift-Gate
 - Migration-Runner: Hash-Check ergänzen — geänderte getrackte Migration = Alarm statt stiller Skip (verhindert die #389-Bugklasse systemisch)
 - scripts/schema_drift_check.py + scripts/check_migration_hashes.py in scripts/ci_pipeline.sh integrieren
 ### R4 — E2E-Kernprozess-Regression (2-3 Tage)
 - Playwright-Suite über Kern-Flows: Login, Kontakte-CRUD, Mail senden/lesen, DMS upload/download, Kalender-Termin, KI-Chat-Antwort, Workflow-Ausführung, Gäste einladen
 - Automatischer Run nach jedem Full-Deploy (fast-deploy.sh-Erweiterung)
 - Bestand: Playwright-Setup existiert (frontend/e2e/, Login-E2E bewiesen funktioniert)
 ### R5 — Backup-/Restore-Nachweis (1-2 Tage)
 - scripts/restore_drill.sh monatlich per ARQ-Job/Cron ausführen + Ergebnis alarmieren
 - RTO/RPO messen und dokumentieren (scripts/backup.py, scripts/restore.py, restore_test.sh existieren)
 ### R6 — Ops-Runbook & Alarm-Kette final (1 Tag)
 - Eskalationskette: Wer wird wie alarmiert (E-Mail/Handy), wer reagiert
 - docs/incident-response-runbook.md um die realen Ausfallklassen ergänzen (Plugin-inactive, DLQ-Vollauf, Migration-Crash, CSRF/Auth-Layer, Worker-Stillstand) — jede mit Schritt-für-Schritt-Fix aus dem echten Incident
 **Aufwand gesamt: ~9-14 Arbeitstage.** R1 zuerst (unabhängig startbar), R2 parallel, R3 nach R2, R4 nach R1, R5/R6 unabhängig. Kann mit Phase O/P verzahnt werden — aber R1-R3 vor neuen Features.
 **Definition of Done Phase R:** Alle 5 Abnahmekriterien live gemessen und grün + ein dokumentierter Chaos-Test (absichtlicher Ausfall → Alarm in < 30 Min). Pro Task ein Forgejo-Issue mit Milestone „Phase R — Betriebssicherung" (AGENTS.md §9).
 ## UI-Backlog — Backend-Module ohne UI (laufend seit 2026-09-08, Source of Truth: PROGRESS.md-Tabelle)
 **Kontext:** Frontend-Backend-Gegenüberstellung (2026-09-01) ergab 16 Backend-Module ohne UI (~64 Ops). User-Entscheidung: Module einzeln mit UI ausstatten, priorisiert nach Business-Nutzen. Jedes Modul folgt derselben Verifikationskette: Vitest → tsc → Production-Build → Deploy → Live-API-Check → Forgejo-Issue → PROGRESS.md-Update.
 **Status 2026-09-16: 16/16 erledigt — UI-BACKLOG KOMPLETT.**
 - Erledigt: 1 Approvals, 2 Delegations, 3 API-Tokens, 4 Tenants, 5 Marketplace, 6 Permission-Templates, 7 Skills, 8 Agent-Memory, 9 Outbox, 10 Policies, 11 Graph-RAG, 12 Companies, 13 Public-Share, 14 Guests, 15 External-Agent, 16 Ownership-Transfer (Commits + Issues #369, #372-#388 in PROGRESS.md-Tabelle)
 - Alle 16 Backend-Module haben jetzt UI. Bei neuen Backend-Modulen ohne UI: analog verfahren.
 **Architektur-Regel (seit Phase Q):** Plugin-Module (wie Marketplace, Skills, Agent-Memory) werden AUSSCHLIESSLICH via Plugin-Manifest registriert (page_routes + menu_items + Komponenten-Map-Generator) — routes/index.tsx und Sidebar.tsx bleiben unangetastet. Core-Module (wie Delegations, API-Tokens, Tenants, Permission-Templates) laufen als statische Core-Routen + Settings-Nav.
 **Muster:** Jedes Modul = api/<modul>.ts (TanStack-Hooks) + pages/<Modul>.tsx (Karten/Dialoge/Permission-Gating) + i18n de/en + Vitest-Tests + Registrierung. Referenz-Implementierungen: Approvals (Core) und Marketplace (Plugin/Phase Q).
@@ -1,7 +1,58 @@
 # LeoCRM v1.0
-> Self-hosted CRM for small sales teams (5–25 sales reps).
+> Plugin-basierte KI und Business-Plattform mit 25 Plugins (CRM, Mail, DMS, Chat, AI-Agenten, Workflows, Knowledge, Search, Self-Improvement, Compliance). FastAPI Backend + React/TypeScript Frontend. Deployiert über Coolify auf Hetzner VPS.
-> Stack: FastAPI + SQLAlchemy (async) + PostgreSQL + Redis + React 18 + TypeScript + Vite + TanStack Query + Zustand + Tailwind + Docker + Coolify
+> Stack: FastAPI + SQLAlchemy (async) + PostgreSQL 16 (pgvector) + Redis 7 + React 18 + TypeScript + Vite + TanStack Query + Zustand + Tailwind + Docker + Coolify
 ## Features
 ### Core Platform
 - **Multi-Tenant** — Tenant-Isolation via ORM Auto-Filter + Row Level Security (RLS)
 - **Plugin System** — 25 Built-in Plugins, Manifest-basiert, aktivierbar/deaktivierbar
 - **Permission System** — ABAC/RBAC mit feingranularen Permissions
 - **Audit Log** — Vollständige Audit-Trail, CSV/JSON Export, 365 Tage Retention
 - **Entity History** — Undo/Restore für alle Entitäten
 - **Soft Delete** — `deleted_at` auf allen Entitäten, Hard-Delete mit `?gdpr=true`
 - **Unified Search** — Hybrid-Suche (PostgreSQL FTS + pgvector), KI Query-Understanding
 - **System Dashboard** — Admin-only Monitoring (DB, Redis, Worker, Errors, LLM Costs)
 - **Backup Automation** — ARQ-gesteuert, einstellbar in Settings, Backup-History
 - **Trash Cleanup** — Automatische endgültige Löschung nach 90 Tagen
 ### 25 Plugins
 | # | Plugin | Beschreibung |
 |---|--------|-------------|
 | 1 | **contacts** | Kontakt-Verwaltung (Personen, Firmen, Ordner, Custom Fields) |
 | 2 | **mail** | IMAP/SMTP E-Mail-Integration, PGP, Filter-Regeln, Vacation Responder |
 | 3 | **dms** | Document Management System, File Upload, Preview, Sharing, Permissions |
 | 4 | **calendar** | Kalender, Termine, Ressourcen-Buchung, ICS Import/Export, Kanban |
 | 5 | **tasks** | Unified Task System, Subtasks, Goals, polymorphe Zuweisung |
 | 6 | **kommunikation** | Unified Messaging, Chat, Mini-Apps, WebSocket-basiert |
 | 7 | **automation** | Automation Builder, Trigger, Agent Runner, Cron-Scheduler |
 | 8 | **ai_assistant** | AI Chat Sessions, Provider, Models, Presets, Tools |
 | 9 | **ai_proactive** | Proactive AI, Suggestions, SSE Streaming, Settings |
 | 10 | **ai_ui_control** | AI-driven UI Control via WebSocket |
 | 11 | **agent_memory** | Agent Memory Plugin, eigene Routes |
 | 12 | **unified_search** | Hybrid-Suche, Embeddings, RRF Rank Fusion, Facets |
 | 13 | **graph_rag** | GraphRAG, Knowledge Graph, Relationship Extraction |
 | 14 | **wiki** | Wiki Plugin, Article Versioning, Categories, Entity Links |
 | 15 | **report_generator** | Report Templates, Generation, Download |
 | 16 | **entity_links** | Entity Linking, File-Entity Connections |
 | 17 | **tags** | Tag Management, Bulk-Assign, Entity-Tag Queries |
 | 18 | **permissions** | File-level Permissions, Share Links |
 | 19 | **mcp_server** | MCP Server, Tool Definitions für AI Agents |
 | 20 | **mcp_client** | MCP Client für externe Tool-Integration |
 | 21 | **marketplace** | Marketplace Listings |
 | 22 | **system_notif** | System Notifications, Alerting via Communication-System |
 | 23 | **forgejo_error_reporter** | Forgejo Error Reporting |
 | 24 | **knowledge** | LLM-based Knowledge Extraction, Ask-Knowledge, Review Queue |
 | 25 | **self_improvement** | Controlled Self-Improvement Loop (Signals, Patterns, Proposals, Impact) |
 ### AI & Automation
 - **Agent System** — ReAct-Loop, Tool-Calls, Skills, Approvals, Monitoring, SSE Streaming
 - **Workflow Engine** — 14 Step-Types, Durable Runs, Retry, Idempotency, SSRF-Schutz
 - **Decision Guard** — Automated-Decision Guard für High-Risk Actions
 - **Approval System** — Human Approval für Agent Actions und Workflow Steps
 - **LLM Client** — Zentraler LLM Client, Cost-Tracking, Multi-Provider
 ## Quick Start (Development)
@@ -87,13 +138,23 @@ See [docs/admin-guide.md](docs/admin-guide.md) for detailed deployment, backup,
 | Endpoint | Method | Auth | Description |
 |---|---|---|---|
-| `/api/v1/health` | GET | No | Health check (DB, Redis, storage, worker) |
+| `/health/live` | GET | No | Liveness probe |
 | `/health/ready` | GET | No | Readiness probe (DB, Redis, storage, worker) |
 | `/api/v1/health` | GET | No | Full health check (DB, Redis, storage, worker) |
 | `/api/v1/metrics` | GET | Admin | Prometheus metrics (text/plain) |
 | `/api/v1/system/dashboard` | GET | Admin | System dashboard (DB, Redis, worker, errors, LLM costs) |
 | `/api/v1/system/alerts` | GET | Admin | Active system alerts |
 | `/api/v1/auth/login` | POST | No | Login |
 | `/api/v1/contacts` | GET | Yes | List contacts (paginated, max page_size=100) |
 | `/api/v1/contacts/export` | GET | Yes | Stream contacts as CSV |
 | `/api/v1/companies` | GET | Yes | List companies (paginated, max page_size=100) |
 | `/api/v1/companies/export` | GET | Yes | Stream companies as CSV |
 | `/api/v1/search` | POST | Yes | Hybrid search (FTS + pgvector) |
 | `/api/v1/audit-log` | GET | Admin | Query audit log entries |
 | `/api/v1/audit-log/export` | GET | Admin | Export audit log (CSV/JSON) |
 | `/api/v1/system-settings/backup-config` | GET/PUT | Admin | Backup configuration |
 | `/api/v1/system-settings/backup-now` | POST | Admin | Trigger immediate backup |
 | `/api/v1/system-settings/backup-history` | GET | Admin | Backup history (last 10) |
 ### Pagination
@@ -109,18 +170,25 @@ Uses `StreamingResponse` — does not buffer the entire file in memory.
 Interactive API documentation: http://localhost:8000/docs
-See [docs/api-overview.md](docs/api-overview.md) for the full endpoint summary.
+See [docs/api-documentation.md](docs/api-documentation.md) for the full endpoint reference.
 ## Monitoring
-### Health Check
+### Health Checks
 ```bash
-curl http://localhost:8000/api/v1/health
+# Liveness
-```
+curl http://localhost:8000/health/live
 # → {"status":"alive"}
-Returns JSON with overall status (`healthy`/`degraded`) and individual checks for
+# Readiness
-`database`, `redis`, `storage`, and `worker`.
+curl http://localhost:8000/health/ready
 # → {"status":"ready","checks":{"database":"ok","redis":"ok","storage":"ok"}}
 # Full health
 curl http://localhost:8000/api/v1/health
 # → {"status":"healthy","version":"1.0.0","checks":{...}}
 ```
 ### Prometheus Metrics
@@ -135,6 +203,15 @@ Available metrics:
 - `leocrm_db_pool_connections` — Database connection pool size
 - `leocrm_arq_jobs_total` — Total ARQ background jobs
 ### System Dashboard
 Admin-only dashboard at `/system-dashboard` in the WebUI. Shows:
 - System Health, DB Stats, Redis Stats, Worker Queue
 - API Stats (total requests, error rate, avg response time)
 - Plugin Stats (discovered, active)
 - Storage Stats (disk usage, file count)
 - Alert Feed (system messages from Communication-System)
 ### Structured Logging
 LeoCRM uses `structlog` for structured JSON logging. All API requests are logged with:
@@ -199,41 +276,73 @@ leocrm/
 ├── app/
 │   ├── main.py               # FastAPI entry point with logging middleware
 │   ├── config.py             # Pydantic settings
 │   ├── deps.py               # FastAPI dependencies (auth, permissions)
 │   ├── core/
 │   │   ├── monitoring.py      # Prometheus metrics + structured logging + health checks
 │   │   ├── db.py             # Async database engine
 │   │   ├── middleware.py     # CSRF middleware
 │   │   ├── worker.py        # ARQ worker settings
 │   │   ├── backup_job.py    # Automated backup job
 │   │   ├── notifications.py  # System notification dispatch
 │   │   └── ...
 │   ├── routes/
-│   │   ├── health.py         # Health endpoint
+│   │   ├── health.py         # Health endpoints
 │   │   ├── metrics.py        # Prometheus metrics endpoint (admin-only)
 │   │   ├── system_dashboard.py # System dashboard (admin-only)
 │   │   ├── system_settings.py # System settings + backup config
 │   │   ├── audit.py          # Audit log (list, export, retention)
 │   │   ├── contacts.py       # Contact CRUD + streaming CSV export
 │   │   ├── companies.py      # Company CRUD + streaming CSV export
 │   │   ├── workflows.py      # Workflow engine routes
 │   │   └── ...
 │   ├── models/               # SQLAlchemy models
 │   ├── schemas/              # Pydantic schemas
 │   ├── services/             # Business logic
-│   └── plugins/              # Plugin system
+│   ├── plugins/              # Plugin system (registry, manifest, base)
 │   │   └── builtins/         # 25 built-in plugins
 │   ├── workflows/            # Workflow engine
 │   └── ai/                   # AI modules
 ├── scripts/
 │   ├── fast-deploy.sh        # Frontend-only / full deploy
 │   ├── deploy.py             # Coolify API deployment
 │   ├── backup.py             # Backup script (pg_dump + files)
 │   ├── restore.py            # Restore script
 │   ├── seed_perf_data.py     # Performance test data seeding
 │   └── check_indexes.py      # Database index verification
 ├── tests/                    # Test suite (pytest + pytest-asyncio)
 ├── docs/
 │   ├── admin-guide.md       # Admin guide (deploy, backup, restore, troubleshooting)
-│   └── api-overview.md      # API endpoint summary
+│   ├── api-documentation.md  # Full API endpoint reference
-├── alembic/                  # Database migrations
+│   ├── monitoring.md        # Monitoring & health checks
 │   ├── infrastructure.md    # Infrastructure guide
 │   ├── deploy-guide.md      # Deploy guide (fast-deploy, Coolify, server info)
 │   └── ...
 ├── alembic/                  # Database migrations (130+ files)
 ├── frontend/                 # React + TypeScript + Vite + Tailwind
 │   └── src/pages/            # SystemDashboard, Contacts, Mail, DMS, Calendar, etc.
 ├── requirements.txt          # Production dependencies
 ├── requirements-dev.txt     # Test/lint dependencies
 ├── .env.example              # Environment template
-├── docker-compose.yml        # Docker Compose
+├── docker-compose.yaml       # Docker Compose (postgres, redis, crm_app, crm_worker)
 ├── Dockerfile                # Multi-stage build (frontend → builder → runtime)
 ├── prestart.sh               # Container entrypoint (migrations, seed, uvicorn)
 ├── worker.sh                 # ARQ worker entrypoint
 ├── healthcheck.sh            # Container healthcheck
 └── README.md                # This file
 ```
 ## Documentation
 - [Admin Guide](docs/admin-guide.md) — Deployment, backup, restore, env vars, troubleshooting
- [API Overview](docs/api-overview.md) — Full endpoint reference
+- [API Documentation](docs/api-documentation.md) — Full endpoint reference (300+ endpoints)
- [Coolify Setup](COOLIFY_SETUP.md) — Coolify deployment instructions
+- [Monitoring](docs/monitoring.md) — Health checks, metrics, system dashboard, alerting
 - [Infrastructure](docs/infrastructure.md) — Docker, PgBouncer, audit partitioning, backup
 - [Deploy Guide](docs/deploy-guide.md) — Fast-deploy, Coolify API, server info
 - [Plugin Development](docs/plugin-development-guide.md) — Plugin development guide
 - [Security Kernel](docs/security_kernel.md) — ABAC, RLS, session security
 - [Permissions](docs/permissions.md) — Permission system documentation
 - [Test Strategy](docs/test-strategy.md) — Test conventions and constraints
 - [UI Design Guidelines](docs/ui-design-guidelines.md) — UI design rules
 - [Swagger UI](http://localhost:8000/docs) — Interactive API docs (auto-generated)
 ## License
@@ -3,6 +3,15 @@
 from __future__ import annotations
 import asyncio
 # F19 (Astra P1): deterministic full-model discovery for Alembic.
 # `from app.models import *` only loads CORE models (48 tables in a fresh
 # process). Contact and ~80 other tables physically live in plugins
 # (e.g. app.plugins.builtins.contacts.models) — the lazy package
 # __getattr__ never fires for wildcard imports. Without the plugin models
 # the metadata sort fails (contact_merge_history → contacts FK) and
 # `alembic check` compares against an incomplete schema.
 import importlib
 from logging.config import fileConfig
 from sqlalchemy import pool
@@ -13,6 +22,15 @@ from alembic import context
 from app.config import get_settings
 from app.core.db import Base
 from app.models import *  # noqa: F401,F403
 from app.plugins.registry import get_registry
 _registry = get_registry()
 _registry.discover_builtins()
 for _plugin_name in _registry.list_discovered():
    try:
        importlib.import_module(f"app.plugins.builtins.{_plugin_name}.models")
    except ImportError:
        pass  # plugin has no models module
 config = context.config
 if config.config_file_name is not None:
@@ -18,7 +18,26 @@ branch_labels = None
 depends_on = None
 def _table_exists(conn, table_name: str) -> bool:
    """True when the table exists (dual-path convergence, Gate B).
    On a fresh install the ai_assistant plugin SQL migration has not run
    yet when Alembic reaches this revision — skip instead of failing.
    The plugin-side migration adds the same columns idempotently.
    """
    row = conn.execute(
        sa.text("SELECT to_regclass(:tname) IS NOT NULL"),
        {"tname": f"public.{table_name}"},
    ).scalar()
    return bool(row)
 def upgrade() -> None:
    conn = op.get_bind()
    if not _table_exists(conn, "ai_providers"):
        # Fresh-install path: table arrives with the ai_assistant plugin
        # migration, which includes these columns.
        return
    op.add_column("ai_providers", sa.Column("region", sa.String(20), nullable=False, server_default="unknown"))
    op.add_column("ai_providers", sa.Column("hosting_type", sa.String(30), nullable=False, server_default="cloud"))
    op.add_column("ai_providers", sa.Column("dpa_status", sa.String(20), nullable=False, server_default="none"))
@@ -29,6 +48,9 @@ def upgrade() -> None:
 def downgrade() -> None:
    conn = op.get_bind()
    if not _table_exists(conn, "ai_providers"):
        return
    op.drop_column("ai_providers", "allowed_data_classes")
    op.drop_column("ai_providers", "transfer_notice")
    op.drop_column("ai_providers", "training_on_customer_data")
@@ -17,126 +17,145 @@ branch_labels = None
 depends_on = None
 def _table_exists(conn, table_name: str) -> bool:
    """True when the table exists (dual-path convergence, Gate B).
    On a fresh install the kommunikation plugin SQL migration has not run
    yet when Alembic reaches this revision — skip the comm_* parts instead
    of failing. The plugin-side migration adds the same column idempotently.
    """
    row = conn.execute(
        sa.text("SELECT to_regclass(:tname) IS NOT NULL"),
        {"tname": f"public.{table_name}"},
    ).scalar()
    return bool(row)
 def upgrade() -> None:
-    # 1. Add is_system column to comm_conversations
+    conn = op.get_bind()
-    op.add_column(
+    if _table_exists(conn, "comm_conversations"):
-        "comm_conversations",
+        # 1. Add is_system column to comm_conversations
-        sa.Column("is_system", sa.Boolean(), nullable=False, server_default=sa.text("false")),
+        op.add_column(
-    )
+            "comm_conversations",
-    op.create_index(
+            sa.Column("is_system", sa.Boolean(), nullable=False, server_default=sa.text("false")),
-        "ix_comm_conversations_tenant_system",
+        )
-        "comm_conversations",
+        op.create_index(
-        ["tenant_id", "is_system"],
+            "ix_comm_conversations_tenant_system",
-    )
+            "comm_conversations",
            ["tenant_id", "is_system"],
        )
-    # 2. Create system channel per tenant (for tenants that have notifications)
+        # 2. Create system channel per tenant (for tenants that have notifications)
-    op.execute("""
+        op.execute("""
-        INSERT INTO comm_conversations (id, tenant_id, title, is_pinned, is_locked, is_direct, is_archived, is_system, created_by, created_by_type, metadata, created_at, updated_at)
+            INSERT INTO comm_conversations (id, tenant_id, title, is_pinned, is_locked, is_direct, is_archived, is_system, created_by, created_by_type, metadata, created_at, updated_at)
-        SELECT
+            SELECT
-            gen_random_uuid(),
+                gen_random_uuid(),
-            n.tenant_id,
+                n.tenant_id,
-            'System Channel',
+                'System Channel',
-            false,
+                false,
-            true,
+                true,
-            false,
+                false,
-            false,
+                false,
-            true,
+                true,
-            NULL,
+                NULL,
-            'system',
+                'system',
-            '{}'::jsonb,
+                '{}'::jsonb,
-            NOW(),
+                NOW(),
-            NOW()
+                NOW()
-        FROM (
+            FROM (
-            SELECT DISTINCT tenant_id FROM notifications WHERE deleted_at IS NULL
+                SELECT DISTINCT tenant_id FROM notifications WHERE deleted_at IS NULL
-        ) n
+            ) n
-        WHERE NOT EXISTS (
+            WHERE NOT EXISTS (
-            SELECT 1 FROM comm_conversations cc
+                SELECT 1 FROM comm_conversations cc
-            WHERE cc.tenant_id = n.tenant_id AND cc.is_system = true AND cc.deleted_at IS NULL
+                WHERE cc.tenant_id = n.tenant_id AND cc.is_system = true AND cc.deleted_at IS NULL
-        );
+            );
-    """)
+        """)
-    # 3. Insert notifications as CommMessages in the system channel
+        # 3. Insert notifications as CommMessages in the system channel
-    op.execute("""
+        op.execute("""
-        INSERT INTO comm_messages (id, tenant_id, conversation_id, sender_id, sender_type, content, content_format, metadata, created_at, updated_at)
+            INSERT INTO comm_messages (id, tenant_id, conversation_id, sender_id, sender_type, content, content_format, metadata, created_at, updated_at)
-        SELECT
+            SELECT
-            gen_random_uuid(),
+                gen_random_uuid(),
-            n.tenant_id,
+                n.tenant_id,
-            sc.id,
+                sc.id,
-            n.user_id,
+                n.user_id,
-            'system',
+                'system',
-            COALESCE(n.title, '') || CASE WHEN n.body IS NOT NULL THEN E'\n' || n.body ELSE '' END,
+                COALESCE(n.title, '') || CASE WHEN n.body IS NOT NULL THEN E'\n' || n.body ELSE '' END,
-            'text',
+                'text',
-            jsonb_build_object(
+                jsonb_build_object(
-                'notification_type', n.type,
+                    'notification_type', n.type,
-                'severity', 'info',
+                    'severity', 'info',
-                'entity_ref', CASE WHEN n.entity_type IS NOT NULL THEN jsonb_build_object('entity_type', n.entity_type, 'entity_id', n.entity_id::text) ELSE NULL END,
+                    'entity_ref', CASE WHEN n.entity_type IS NOT NULL THEN jsonb_build_object('entity_type', n.entity_type, 'entity_id', n.entity_id::text) ELSE NULL END,
-                'migrated_from_notification', true,
+                    'migrated_from_notification', true,
-                'original_notification_id', n.id::text
+                    'original_notification_id', n.id::text
-            ),
+                ),
-            n.created_at,
+                n.created_at,
-            COALESCE(n.read_at, n.created_at)
+                COALESCE(n.read_at, n.created_at)
-        FROM notifications n
+            FROM notifications n
-        JOIN comm_conversations sc ON sc.tenant_id = n.tenant_id AND sc.is_system = true AND sc.deleted_at IS NULL
+            JOIN comm_conversations sc ON sc.tenant_id = n.tenant_id AND sc.is_system = true AND sc.deleted_at IS NULL
-        WHERE n.deleted_at IS NULL;
+            WHERE n.deleted_at IS NULL;
-    """)
+        """)
-    # 4. Insert text blocks for each migrated message
+        # 4. Insert text blocks for each migrated message
-    op.execute("""
+        op.execute("""
-        INSERT INTO comm_message_blocks (id, tenant_id, message_id, block_type, block_data, sort_order)
+            INSERT INTO comm_message_blocks (id, tenant_id, message_id, block_type, block_data, sort_order)
-        SELECT
+            SELECT
-            gen_random_uuid(),
+                gen_random_uuid(),
-            cm.tenant_id,
+                cm.tenant_id,
-            cm.id,
+                cm.id,
-            'text',
+                'text',
-            jsonb_build_object('text', cm.content),
+                jsonb_build_object('text', cm.content),
-            0
+                0
-        FROM comm_messages cm
+            FROM comm_messages cm
-        WHERE cm.metadata->>'migrated_from_notification' = 'true';
+            WHERE cm.metadata->>'migrated_from_notification' = 'true';
-    """)
+        """)
-    # 5. Insert action_card blocks for messages with entity references
+        # 5. Insert action_card blocks for messages with entity references
-    op.execute("""
+        op.execute("""
-        INSERT INTO comm_message_blocks (id, tenant_id, message_id, block_type, block_data, sort_order)
+            INSERT INTO comm_message_blocks (id, tenant_id, message_id, block_type, block_data, sort_order)
-        SELECT
+            SELECT
-            gen_random_uuid(),
+                gen_random_uuid(),
-            cm.tenant_id,
+                cm.tenant_id,
-            cm.id,
+                cm.id,
-            'action_card',
+                'action_card',
-            jsonb_build_object(
+                jsonb_build_object(
-                'label', 'Open',
+                    'label', 'Open',
-                'entity_type', (cm.metadata->'entity_ref'->>'entity_type'),
+                    'entity_type', (cm.metadata->'entity_ref'->>'entity_type'),
-                'entity_id', (cm.metadata->'entity_ref'->>'entity_id')
+                    'entity_id', (cm.metadata->'entity_ref'->>'entity_id')
-            ),
+                ),
-            1
+                1
-        FROM comm_messages cm
+            FROM comm_messages cm
-        WHERE cm.metadata->>'migrated_from_notification' = 'true'
+            WHERE cm.metadata->>'migrated_from_notification' = 'true'
-          AND cm.metadata->'entity_ref' IS NOT NULL;
+              AND cm.metadata->'entity_ref' IS NOT NULL;
-    """)
+        """)
-    # 6. For read notifications, create CommMessageRead entries
+        # 6. For read notifications, create CommMessageRead entries
-    op.execute("""
+        op.execute("""
-        INSERT INTO comm_message_reads (id, tenant_id, conversation_id, user_id, last_read_msg_id, last_read_at)
+            INSERT INTO comm_message_reads (id, tenant_id, conversation_id, user_id, last_read_msg_id, last_read_at)
-        SELECT
+            SELECT
-            gen_random_uuid(),
+                gen_random_uuid(),
-            cm.tenant_id,
+                cm.tenant_id,
-            cm.conversation_id,
+                cm.conversation_id,
-            cm.sender_id,
+                cm.sender_id,
-            cm.id,
+                cm.id,
-            COALESCE(n.read_at, n.created_at)
+                COALESCE(n.read_at, n.created_at)
-        FROM comm_messages cm
+            FROM comm_messages cm
-        JOIN notifications n ON n.id::text = cm.metadata->>'original_notification_id'
+            JOIN notifications n ON n.id::text = cm.metadata->>'original_notification_id'
-        WHERE cm.metadata->>'migrated_from_notification' = 'true'
+            WHERE cm.metadata->>'migrated_from_notification' = 'true'
-          AND n.read_at IS NOT NULL
+              AND n.read_at IS NOT NULL
-          AND n.deleted_at IS NULL;
+              AND n.deleted_at IS NULL;
-    """)
+        """)
-    # 7. Create legacy view over notifications table for backward compatibility
+    # 7. Legacy view over the CORE notifications table — exists on both paths
    op.execute("DROP VIEW IF EXISTS notifications_legacy")
    op.execute("CREATE VIEW notifications_legacy AS SELECT * FROM notifications")
 def downgrade() -> None:
    conn = op.get_bind()
    op.execute("DROP VIEW IF EXISTS notifications_legacy")
    if not _table_exists(conn, "comm_conversations"):
        return
    op.execute("DELETE FROM comm_message_blocks WHERE message_id IN (SELECT id FROM comm_messages WHERE metadata->>'migrated_from_notification' = 'true')")
    op.execute("DELETE FROM comm_messages WHERE metadata->>'migrated_from_notification' = 'true'")
    op.execute("DELETE FROM comm_conversations WHERE is_system = true AND title = 'System Channel'")
@@ -0,0 +1,68 @@
 """Create automation_agent_run_steps table for ReAct loop step tracking.
 Revision ID: 0121
 Revises: 0120
 """
 from alembic import op
 import sqlalchemy as sa
 from sqlalchemy.dialects.postgresql import JSONB, UUID as PGUUID
 revision = "0121"
 down_revision = "0120"
 branch_labels = None
 depends_on = None
 def _table_exists(conn, table_name: str) -> bool:
    """True when the table exists (dual-path convergence, Gate B).
    On a fresh install the automation plugin SQL migration has not run yet
    when Alembic reaches this revision — skip instead of failing. The
    plugin-side convergence migration creates the same table.
    """
    row = conn.execute(
        sa.text("SELECT to_regclass(:tname) IS NOT NULL"),
        {"tname": f"public.{table_name}"},
    ).scalar()
    return bool(row)
 def upgrade() -> None:
    conn = op.get_bind()
    if not _table_exists(conn, "automation_agent_runs"):
        return
    op.create_table(
        "automation_agent_run_steps",
        sa.Column("id", PGUUID(as_uuid=True), primary_key=True),
        sa.Column("tenant_id", PGUUID(as_uuid=True), nullable=False, index=True),
        sa.Column(
            "agent_run_id",
            PGUUID(as_uuid=True),
            sa.ForeignKey("automation_agent_runs.id", ondelete="CASCADE"),
            nullable=False,
            index=True,
        ),
        sa.Column("step_number", sa.Integer, nullable=False),
        sa.Column("thought", sa.Text, nullable=True),
        sa.Column("action", sa.String(255), nullable=True),
        sa.Column("action_input", JSONB, nullable=True),
        sa.Column("observation", sa.Text, nullable=True),
        sa.Column("cost_usd", sa.Float, nullable=False, server_default="0.0"),
        sa.Column(
            "created_at",
            sa.DateTime(timezone=True),
            nullable=False,
            server_default=sa.func.now(),
        ),
    )
    op.create_index(
        "ix_agent_run_steps_run",
        "automation_agent_run_steps",
        ["tenant_id", "agent_run_id"],
    )
 def downgrade() -> None:
    op.drop_index("ix_agent_run_steps_run", table_name="automation_agent_run_steps")
    op.drop_table("automation_agent_run_steps")
@@ -0,0 +1,83 @@
 """Add Phase F fields to automation_agent_definitions.
 Adds temperature, max_tokens, max_steps, trace_mode, skill_ids,
 trigger_config, and ai_use_case_metadata to support the Phase F
 context-builder, SSE streaming, and AI-use-case features.
 Revision ID: 0122
 Revises: 0121
 """
 from alembic import op
 import sqlalchemy as sa
 from sqlalchemy.dialects.postgresql import JSONB
 revision = "0122"
 down_revision = "0121"
 branch_labels = None
 depends_on = None
 def _table_exists(conn, table_name: str) -> bool:
    """True when the table exists (dual-path convergence, Gate B).
    On a fresh install the automation plugin SQL migration has not run yet
    when Alembic reaches this revision — skip instead of failing. The
    plugin-side convergence migration adds the same columns.
    """
    row = conn.execute(
        sa.text("SELECT to_regclass(:tname) IS NOT NULL"),
        {"tname": f"public.{table_name}"},
    ).scalar()
    return bool(row)
 def upgrade() -> None:
    conn = op.get_bind()
    if not _table_exists(conn, "automation_agent_definitions"):
        return
    op.add_column(
        "automation_agent_definitions",
        sa.Column("temperature", sa.Float, nullable=False, server_default="0.3"),
    )
    op.add_column(
        "automation_agent_definitions",
        sa.Column("max_tokens", sa.Integer, nullable=False, server_default="1000"),
    )
    op.add_column(
        "automation_agent_definitions",
        sa.Column("max_steps", sa.Integer, nullable=False, server_default="20"),
    )
    op.add_column(
        "automation_agent_definitions",
        sa.Column(
            "trace_mode", sa.String(20), nullable=False, server_default="standard"
        ),
    )
    op.add_column(
        "automation_agent_definitions",
        sa.Column("skill_ids", JSONB, nullable=False, server_default=sa.text("'[]'::jsonb")),
    )
    op.add_column(
        "automation_agent_definitions",
        sa.Column("trigger_config", JSONB, nullable=False, server_default=sa.text("'{}'::jsonb")),
    )
    op.add_column(
        "automation_agent_definitions",
        sa.Column(
            "ai_use_case_metadata",
            JSONB,
            nullable=False,
            server_default=sa.text("'{}'::jsonb"),
        ),
    )
 def downgrade() -> None:
    op.drop_column("automation_agent_definitions", "ai_use_case_metadata")
    op.drop_column("automation_agent_definitions", "trigger_config")
    op.drop_column("automation_agent_definitions", "skill_ids")
    op.drop_column("automation_agent_definitions", "trace_mode")
    op.drop_column("automation_agent_definitions", "max_steps")
    op.drop_column("automation_agent_definitions", "max_tokens")
    op.drop_column("automation_agent_definitions", "temperature")
@@ -0,0 +1,81 @@
 """Create approval_requests and ai_decision_records tables.
 Adds the central approval-request table for agent action approval (F-APPR)
 and the AI decision-record table for the human-oversight audit trail
 (F-OVERSIGHT).
 Revision ID: 0123
 Revises: 0122
 """
 from alembic import op
 import sqlalchemy as sa
 from sqlalchemy.dialects.postgresql import JSONB, UUID as PGUUID
 revision = "0123"
 down_revision = "0122"
 branch_labels = None
 depends_on = None
 def upgrade() -> None:
    op.create_table(
        "approval_requests",
        sa.Column("id", PGUUID(as_uuid=True), primary_key=True),
        sa.Column("tenant_id", PGUUID(as_uuid=True), nullable=False),
        sa.Column("entity_type", sa.String(80), nullable=False),
        sa.Column("entity_id", PGUUID(as_uuid=True), nullable=False),
        sa.Column("action", sa.String(120), nullable=False),
        sa.Column("requested_by", PGUUID(as_uuid=True), nullable=False),
        sa.Column("requested_by_type", sa.String(20), nullable=False, server_default="agent"),
        sa.Column("approver_id", PGUUID(as_uuid=True), nullable=True),
        sa.Column("approver_group", sa.String(120), nullable=True),
        sa.Column("status", sa.String(20), nullable=False, server_default="pending"),
        sa.Column("comment", sa.Text(), nullable=True),
        sa.Column("created_at", sa.DateTime(timezone=True), nullable=False, server_default=sa.func.now()),
        sa.Column("resolved_at", sa.DateTime(timezone=True), nullable=True),
        sa.Column("expires_at", sa.DateTime(timezone=True), nullable=True),
        sa.Column("metadata", JSONB, nullable=False, server_default=sa.text("'{}'::jsonb")),
    )
    op.create_index(
        "ix_approval_requests_tenant_status", "approval_requests", ["tenant_id", "status"]
    )
    op.create_index(
        "ix_approval_requests_tenant_entity",
        "approval_requests",
        ["tenant_id", "entity_type", "entity_id"],
    )
    op.create_index(
        "ix_approval_requests_tenant_approver",
        "approval_requests",
        ["tenant_id", "approver_id"],
    )
    op.create_table(
        "ai_decision_records",
        sa.Column("id", PGUUID(as_uuid=True), primary_key=True),
        sa.Column("tenant_id", PGUUID(as_uuid=True), nullable=False),
        sa.Column("agent_run_id", PGUUID(as_uuid=True), nullable=False),
        sa.Column("recommendation", sa.Text(), nullable=False),
        sa.Column("evidence", JSONB, nullable=False, server_default=sa.text("'{}'::jsonb")),
        sa.Column("reviewer_id", PGUUID(as_uuid=True), nullable=True),
        sa.Column("decision", sa.String(20), nullable=True),
        sa.Column("decision_timestamp", sa.String(40), nullable=True),
        sa.Column("deviation_note", sa.Text(), nullable=True),
        sa.Column("created_at", sa.DateTime(timezone=True), nullable=False, server_default=sa.func.now()),
        sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False, server_default=sa.func.now()),
        sa.Column("deleted_at", sa.DateTime(timezone=True), nullable=True),
        sa.Column("owner_id", PGUUID(as_uuid=True), nullable=True),
    )
    op.create_index(
        "ix_ai_decision_records_tenant_run", "ai_decision_records", ["tenant_id", "agent_run_id"]
    )
 def downgrade() -> None:
    op.drop_index("ix_ai_decision_records_tenant_run", table_name="ai_decision_records")
    op.drop_table("ai_decision_records")
    op.drop_index("ix_approval_requests_tenant_approver", table_name="approval_requests")
    op.drop_index("ix_approval_requests_tenant_entity", table_name="approval_requests")
    op.drop_index("ix_approval_requests_tenant_status", table_name="approval_requests")
    op.drop_table("approval_requests")
@@ -0,0 +1,134 @@
 """Unified Task System (F.14).
 Adds polymorphic assignment/entity/creator fields, subtasks, dependencies,
 goals/milestones and agent-subtask support to the tasks table. Migrates
 legacy ``contact_id``/``assigned_to`` values into the polymorphic fields and
 migrates existing ``agent_subtasks`` rows into tasks with
 ``task_type='agent_subtask'``.
 Revision ID: 0124
 Revises: 0123
 """
 from alembic import op
 import sqlalchemy as sa
 from sqlalchemy.dialects.postgresql import JSONB, UUID as PGUUID
 revision = "0124"
 down_revision = "0123"
 branch_labels = None
 depends_on = None
 def _table_exists(conn, table_name: str) -> bool:
    """True when the table exists (dual-path convergence, Gate B).
    On a fresh install the tasks plugin SQL migration has not run yet when
    Alembic reaches this revision — skip instead of failing. The plugin-side
    convergence migration adds the same columns/indexes. The legacy-data
    backfills below only matter for pre-existing rows and are correctly
    empty on a fresh install.
    """
    row = conn.execute(
        sa.text("SELECT to_regclass(:tname) IS NOT NULL"),
        {"tname": f"public.{table_name}"},
    ).scalar()
    return bool(row)
 def upgrade() -> None:
    conn = op.get_bind()
    if not _table_exists(conn, "tasks"):
        return
    # ── Add new columns to tasks ────────────────────────────────────────────
    op.add_column("tasks", sa.Column("assignee_type", sa.String(20), nullable=False, server_default="user"))
    op.add_column("tasks", sa.Column("assignee_id", PGUUID(as_uuid=True), nullable=True))
    op.add_column("tasks", sa.Column("entity_type", sa.String(80), nullable=True))
    op.add_column("tasks", sa.Column("entity_id", PGUUID(as_uuid=True), nullable=True))
    op.add_column("tasks", sa.Column("creator_type", sa.String(20), nullable=False, server_default="user"))
    op.add_column("tasks", sa.Column("creator_id", PGUUID(as_uuid=True), nullable=True))
    op.add_column("tasks", sa.Column("parent_task_id", PGUUID(as_uuid=True), nullable=True))
    op.add_column("tasks", sa.Column("depends_on", JSONB, nullable=False, server_default=sa.text("'[]'::jsonb")))
    op.add_column("tasks", sa.Column("task_type", sa.String(30), nullable=False, server_default="todo"))
    op.add_column("tasks", sa.Column("success_criteria", JSONB, nullable=True))
    op.add_column("tasks", sa.Column("target_date", sa.DateTime(timezone=True), nullable=True))
    op.add_column("tasks", sa.Column("progress", sa.Integer(), nullable=False, server_default="0"))
    # ── Migrate legacy data into polymorphic fields ─────────────────────────
    # contact_id → entity_type='contact' + entity_id
    op.execute(
        """
        UPDATE tasks
        SET entity_type = 'contact', entity_id = contact_id
        WHERE contact_id IS NOT NULL AND entity_type IS NULL
        """
    )
    # assigned_to → assignee_type='user' + assignee_id
    op.execute(
        """
        UPDATE tasks
        SET assignee_type = 'user', assignee_id = assigned_to
        WHERE assigned_to IS NOT NULL AND assignee_id IS NULL
        """
    )
    # created_by → creator_type='user' + creator_id
    op.execute(
        """
        UPDATE tasks
        SET creator_type = 'user', creator_id = created_by
        WHERE created_by IS NOT NULL AND creator_id IS NULL
        """
    )
    # ── Migrate AgentSubtask rows into tasks ────────────────────────────────
    op.execute(
        """
        INSERT INTO tasks (
            id, tenant_id, title, description, status, priority,
            assignee_type, assignee_id, entity_type, entity_id,
            creator_type, creator_id, task_type, depends_on, progress,
            created_at, updated_at
        )
        SELECT
            asub.id, asub.tenant_id,
            asub.task_description, asub.task_description, asub.status, 'medium',
            'agent', asub.child_agent_id, 'agent', asub.parent_agent_id,
            'agent', asub.parent_agent_id, 'agent_subtask', '[]'::jsonb, 0,
            asub.created_at, asub.updated_at
        FROM agent_subtasks asub
        WHERE NOT EXISTS (
            SELECT 1 FROM tasks t WHERE t.id = asub.id
        )
        """
    )
    # ── Indexes ─────────────────────────────────────────────────────────────
    op.create_index("ix_tasks_tenant_entity", "tasks", ["tenant_id", "entity_type", "entity_id"])
    op.create_index("ix_tasks_tenant_assignee", "tasks", ["tenant_id", "assignee_type", "assignee_id"])
    op.create_index("ix_tasks_tenant_parent", "tasks", ["tenant_id", "parent_task_id"])
    op.create_index("ix_tasks_tenant_type", "tasks", ["tenant_id", "task_type"])
    op.create_foreign_key(
        "fk_tasks_parent_task_id", "tasks", "tasks", ["parent_task_id"], ["id"],
        ondelete="CASCADE",
    )
 def downgrade() -> None:
    op.drop_constraint("fk_tasks_parent_task_id", "tasks", type_="foreignkey")
    op.drop_index("ix_tasks_tenant_type", table_name="tasks")
    op.drop_index("ix_tasks_tenant_parent", table_name="tasks")
    op.drop_index("ix_tasks_tenant_assignee", table_name="tasks")
    op.drop_index("ix_tasks_tenant_entity", table_name="tasks")
    op.drop_column("tasks", "progress")
    op.drop_column("tasks", "target_date")
    op.drop_column("tasks", "success_criteria")
    op.drop_column("tasks", "task_type")
    op.drop_column("tasks", "depends_on")
    op.drop_column("tasks", "parent_task_id")
    op.drop_column("tasks", "creator_id")
    op.drop_column("tasks", "creator_type")
    op.drop_column("tasks", "entity_id")
    op.drop_column("tasks", "entity_type")
    op.drop_column("tasks", "assignee_id")
    op.drop_column("tasks", "assignee_type")
@@ -0,0 +1,84 @@
 """Durable WorkflowRun — resume semantics, step state, idempotency (G-RUN, G-CTX).
 Extends workflow_instances with resume_at, resume_reason, step_state,
 idempotency_key, and lock_owner for durable/resumable workflow execution.
 Revision ID: 0125
 Revises: 0124
 """
 from alembic import op
 import sqlalchemy as sa
 from sqlalchemy.dialects.postgresql import JSONB, UUID as PGUUID
 revision = "0125"
 down_revision = "0124"
 branch_labels = None
 depends_on = None
 def upgrade() -> None:
    # ── Add durable/resumable columns to workflow_instances ──────────────
    op.add_column(
        "workflow_instances",
        sa.Column("resume_at", sa.DateTime(timezone=True), nullable=True),
    )
    op.add_column(
        "workflow_instances",
        sa.Column("resume_reason", sa.String(50), nullable=True),
    )
    op.add_column(
        "workflow_instances",
        sa.Column("step_state", JSONB, nullable=False, server_default="{}"),
    )
    op.add_column(
        "workflow_instances",
        sa.Column("idempotency_key", sa.String(255), nullable=True),
    )
    op.add_column(
        "workflow_instances",
        sa.Column("lock_owner", sa.String(100), nullable=True),
    )
    op.add_column(
        "workflow_instances",
        sa.Column("lock_expires_at", sa.DateTime(timezone=True), nullable=True),
    )
    op.add_column(
        "workflow_instances",
        sa.Column("error_message", sa.Text, nullable=True),
    )
    op.add_column(
        "workflow_instances",
        sa.Column("retry_count", sa.Integer, nullable=False, server_default="0"),
    )
    op.add_column(
        "workflow_instances",
        sa.Column("max_retries", sa.Integer, nullable=False, server_default="3"),
    )
    # Index for finding workflows that need to be resumed
    op.create_index(
        "ix_wf_instances_resume",
        "workflow_instances",
        ["tenant_id", "status", "resume_at"],
    )
    # Index for idempotency key lookup
    op.create_index(
        "ix_wf_instances_idempotency",
        "workflow_instances",
        ["tenant_id", "idempotency_key"],
    )
 def downgrade() -> None:
    op.drop_index("ix_wf_instances_idempotency", table_name="workflow_instances")
    op.drop_index("ix_wf_instances_resume", table_name="workflow_instances")
    op.drop_column("workflow_instances", "max_retries")
    op.drop_column("workflow_instances", "retry_count")
    op.drop_column("workflow_instances", "error_message")
    op.drop_column("workflow_instances", "lock_expires_at")
    op.drop_column("workflow_instances", "lock_owner")
    op.drop_column("workflow_instances", "idempotency_key")
    op.drop_column("workflow_instances", "step_state")
    op.drop_column("workflow_instances", "resume_reason")
    op.drop_column("workflow_instances", "resume_at")
@@ -0,0 +1,79 @@
 """Wiki plugin — articles, categories, versions (H-WIKI, H-VER).
 Revision ID: 0126
 Revises: 0125
 """
 from alembic import op
 import sqlalchemy as sa
 from sqlalchemy.dialects.postgresql import JSONB, UUID as PGUUID
 revision = "0126"
 down_revision = "0125"
 branch_labels = None
 depends_on = None
 def upgrade() -> None:
    op.create_table(
        "wiki_categories",
        sa.Column("id", PGUUID(as_uuid=True), primary_key=True),
        sa.Column("tenant_id", PGUUID(as_uuid=True), nullable=False),
        sa.Column("owner_id", PGUUID(as_uuid=True), nullable=True),
        sa.Column("name", sa.String(200), nullable=False),
        sa.Column("slug", sa.String(200), nullable=False),
        sa.Column("description", sa.Text, nullable=True),
        sa.Column("parent_id", PGUUID(as_uuid=True), nullable=True),
        sa.Column("sort_order", sa.Integer, nullable=False, server_default="0"),
        sa.Column("created_at", sa.DateTime(timezone=True), server_default=sa.func.now()),
        sa.Column("updated_at", sa.DateTime(timezone=True), server_default=sa.func.now()),
        sa.Column("deleted_at", sa.DateTime(timezone=True), nullable=True),
    )
    op.create_foreign_key("fk_wiki_cat_parent", "wiki_categories", "wiki_categories", ["parent_id"], ["id"], ondelete="SET NULL")
    op.create_index("ix_wiki_cat_tenant", "wiki_categories", ["tenant_id"])
    op.create_index("ix_wiki_cat_tenant_slug", "wiki_categories", ["tenant_id", "slug"])
    op.create_table(
        "wiki_articles",
        sa.Column("id", PGUUID(as_uuid=True), primary_key=True),
        sa.Column("tenant_id", PGUUID(as_uuid=True), nullable=False),
        sa.Column("owner_id", PGUUID(as_uuid=True), nullable=True),
        sa.Column("title", sa.String(300), nullable=False),
        sa.Column("slug", sa.String(300), nullable=False),
        sa.Column("content", sa.Text, nullable=False, server_default=""),
        sa.Column("content_html", sa.Text, nullable=True),
        sa.Column("summary", sa.Text, nullable=True),
        sa.Column("category_id", PGUUID(as_uuid=True), nullable=True),
        sa.Column("tags", JSONB, nullable=False, server_default="[]"),
        sa.Column("status", sa.String(20), nullable=False, server_default="draft"),
        sa.Column("entity_links", JSONB, nullable=False, server_default="[]"),
        sa.Column("version", sa.Integer, nullable=False, server_default="1"),
        sa.Column("published_at", sa.DateTime(timezone=True), nullable=True),
        sa.Column("created_at", sa.DateTime(timezone=True), server_default=sa.func.now()),
        sa.Column("updated_at", sa.DateTime(timezone=True), server_default=sa.func.now()),
        sa.Column("deleted_at", sa.DateTime(timezone=True), nullable=True),
    )
    op.create_foreign_key("fk_wiki_art_category", "wiki_articles", "wiki_categories", ["category_id"], ["id"], ondelete="SET NULL")
    op.create_index("ix_wiki_art_tenant", "wiki_articles", ["tenant_id"])
    op.create_index("ix_wiki_art_tenant_category", "wiki_articles", ["tenant_id", "category_id"])
    op.create_index("ix_wiki_art_tenant_slug", "wiki_articles", ["tenant_id", "slug"])
    op.create_index("ix_wiki_art_tenant_status", "wiki_articles", ["tenant_id", "status"])
    op.create_table(
        "wiki_article_versions",
        sa.Column("id", PGUUID(as_uuid=True), primary_key=True),
        sa.Column("tenant_id", PGUUID(as_uuid=True), nullable=False),
        sa.Column("article_id", PGUUID(as_uuid=True), nullable=False),
        sa.Column("version", sa.Integer, nullable=False),
        sa.Column("title", sa.String(300), nullable=False),
        sa.Column("content", sa.Text, nullable=False),
        sa.Column("edited_by", PGUUID(as_uuid=True), nullable=True),
        sa.Column("edit_comment", sa.Text, nullable=True),
        sa.Column("created_at", sa.DateTime(timezone=True), server_default=sa.func.now()),
    )
    op.create_foreign_key("fk_wiki_ver_article", "wiki_article_versions", "wiki_articles", ["article_id"], ["id"], ondelete="CASCADE")
    op.create_index("ix_wiki_ver_tenant_article", "wiki_article_versions", ["tenant_id", "article_id"])
    op.create_index("ix_wiki_ver_tenant_version", "wiki_article_versions", ["tenant_id", "article_id", "version"])
 def downgrade() -> None:
    op.drop_table("wiki_article_versions")
    op.drop_table("wiki_articles")
    op.drop_table("wiki_categories")
@@ -0,0 +1,49 @@
 """Drop tasks_contact_id_fkey — contact_id is now derived from entity_id (ARCH-F-2).
 The tasks table has both a contact_id FK column (referencing contacts) and
 polymorphic entity_type/entity_id columns. The code now derives contact_id
 from entity_id when entity_type='contact', and stores NULL in the FK column.
 The FK constraint is redundant and prevents creating tasks with arbitrary
 entity references. This migration drops the FK constraint but keeps the column
 for backward compatibility.
 Revision ID: 0127
 Revises: 0126
 """
 from alembic import op
 import sqlalchemy as sa
 revision = "0127"
 down_revision = "0126"
 branch_labels = None
 depends_on = None
 def _table_exists(conn, table_name: str) -> bool:
    """True when the table exists (dual-path convergence, Gate B)."""
    row = conn.execute(
        sa.text("SELECT to_regclass(:tname) IS NOT NULL"),
        {"tname": f"public.{table_name}"},
    ).scalar()
    return bool(row)
 def upgrade() -> None:
    conn = op.get_bind()
    if not _table_exists(conn, "tasks"):
        return
    # Drop the FK constraint on tasks.contact_id
    op.drop_constraint("tasks_contact_id_fkey", "tasks", type_="foreignkey")
 def downgrade() -> None:
    # Re-create the FK constraint (best-effort — may fail if orphaned rows exist)
    op.create_foreign_key(
        "tasks_contact_id_fkey",
        "tasks",
        "contacts",
        ["contact_id"],
        ["id"],
        ondelete="SET NULL",
    )
@@ -0,0 +1,42 @@
 """Create ai_decision_records table for oversight.
 Revision ID: 0128
 Revises: 0127
 """
 from alembic import op
 import sqlalchemy as sa
 from sqlalchemy.dialects.postgresql import UUID, JSONB
 revision = "0128"
 down_revision = "0127"
 branch_labels = None
 depends_on = None
 def upgrade() -> None:
    # Use IF NOT EXISTS to avoid DuplicateTableError if table was already
    # created by Base.metadata.create_all() in prestart.sh
    op.execute("""
        CREATE TABLE IF NOT EXISTS ai_decision_records (
            id UUID NOT NULL DEFAULT gen_random_uuid() PRIMARY KEY,
            tenant_id UUID NOT NULL REFERENCES tenants(id) ON DELETE CASCADE,
            owner_id UUID,
            agent_run_id UUID NOT NULL,
            recommendation TEXT NOT NULL,
            evidence JSONB NOT NULL DEFAULT '{}',
            reviewer_id UUID,
            decision VARCHAR(20),
            decision_timestamp VARCHAR(40),
            deviation_note TEXT,
            created_at TIMESTAMP WITH TIME ZONE DEFAULT now() NOT NULL,
            updated_at TIMESTAMP WITH TIME ZONE DEFAULT now() NOT NULL,
            deleted_at TIMESTAMP WITH TIME ZONE
        )
    """)
    op.execute("CREATE INDEX IF NOT EXISTS ix_ai_decision_records_tenant_id ON ai_decision_records(tenant_id)")
    op.execute("CREATE INDEX IF NOT EXISTS ix_ai_decision_records_agent_run_id ON ai_decision_records(agent_run_id)")
 def downgrade() -> None:
    op.drop_table("ai_decision_records")
@@ -0,0 +1,63 @@
 """Enable RLS for 8 tables that need tenant isolation.
 Tables excluded (no tenant_id column):
 - outbox_deliveries: linked via event_outbox which has tenant_id
 - marketplace_listings: global plugin marketplace, not tenant-specific
 Revision ID: 0129
 Revises: 0128
 """
 from alembic import op
 import sqlalchemy as sa
 revision = "0129"
 down_revision = "0128"
 branch_labels = None
 depends_on = None
 TABLES_NEEDING_RLS = [
    "ai_decision_records",
    "approval_requests",
    "automation_agent_run_steps",
    "roles",
    "sequences",
    "wiki_articles",
    "wiki_article_versions",
    "wiki_categories",
 ]
 def _table_exists(conn, table_name: str) -> bool:
    """True when the table exists (dual-path convergence, Gate B).
    Plugin-owned tables may not exist yet on a fresh install when Alembic
    reaches this revision — skip them instead of failing. The plugin-side
    convergence migrations apply the same RLS policies.
    """
    row = conn.execute(
        sa.text("SELECT to_regclass(:tname) IS NOT NULL"),
        {"tname": f"public.{table_name}"},
    ).scalar()
    return bool(row)
 def upgrade() -> None:
    conn = op.get_bind()
    for table in TABLES_NEEDING_RLS:
        if not _table_exists(conn, table):
            continue
        op.execute(f"ALTER TABLE {table} ENABLE ROW LEVEL SECURITY;")
        op.execute(
            f"CREATE POLICY tenant_isolation ON {table} "
            f"FOR ALL USING (tenant_id = current_setting('app.tenant_id')::uuid);"
        )
 def downgrade() -> None:
    conn = op.get_bind()
    for table in TABLES_NEEDING_RLS:
        if not _table_exists(conn, table):
            continue
        op.execute(f"DROP POLICY IF EXISTS tenant_isolation ON {table};")
        op.execute(f"ALTER TABLE {table} DISABLE ROW LEVEL SECURITY;")
@@ -0,0 +1,25 @@
 """Add backup_enabled column to system_settings table.
 Revision ID: 0130
 Revises: 0129
 """
 from alembic import op
 import sqlalchemy as sa
 revision = "0130"
 down_revision = "0129"
 branch_labels = None
 depends_on = None
 def upgrade() -> None:
    op.add_column(
        "system_settings",
        sa.Column("backup_enabled", sa.Boolean(), nullable=False, server_default=sa.text("false")),
    )
 def downgrade() -> None:
    op.drop_column("system_settings", "backup_enabled")
@@ -0,0 +1,44 @@
 """knowledge extractions table
 Revision ID: 0131
 Revises: 0130
 Create Date: 2026-08-20
 """
 from alembic import op
 import sqlalchemy as sa
 from sqlalchemy.dialects.postgresql import UUID, JSONB
 revision = "0131"
 down_revision = "0130"
 branch_labels = None
 depends_on = None
 def upgrade() -> None:
    op.create_table(
        "knowledge_extractions",
        sa.Column("id", UUID(as_uuid=True), primary_key=True, server_default=sa.text("gen_random_uuid()")),
        sa.Column("tenant_id", UUID(as_uuid=True), sa.ForeignKey("tenants.id", ondelete="CASCADE"), nullable=False),
        sa.Column("source_type", sa.String(50), nullable=False),
        sa.Column("source_id", UUID(as_uuid=True), nullable=False),
        sa.Column("source_title", sa.String(500), nullable=True),
        sa.Column("extracted_entities", JSONB, nullable=False, server_default=sa.text("'[]'::jsonb")),
        sa.Column("extracted_relationships", JSONB, nullable=False, server_default=sa.text("'[]'::jsonb")),
        sa.Column("confidence", sa.Float, nullable=False, server_default=sa.text("0.0")),
        sa.Column("status", sa.String(30), nullable=False, server_default=sa.text("'pending'")),
        sa.Column("review_notes", sa.Text, nullable=True),
        sa.Column("llm_model", sa.String(100), nullable=True),
        sa.Column("llm_cost_usd", sa.Float, nullable=False, server_default=sa.text("0.0")),
        sa.Column("created_by", UUID(as_uuid=True), sa.ForeignKey("users.id", ondelete="SET NULL"), nullable=True),
        sa.Column("reviewed_by", UUID(as_uuid=True), sa.ForeignKey("users.id", ondelete="SET NULL"), nullable=True),
        sa.Column("reviewed_at", sa.DateTime(timezone=True), nullable=True),
        sa.Column("created_at", sa.DateTime(timezone=True), nullable=False, server_default=sa.text("now()")),
        sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False, server_default=sa.text("now()")),
    )
    op.create_index("ix_knowledge_ext_tenant_status", "knowledge_extractions", ["tenant_id", "status"])
    op.create_index("ix_knowledge_ext_source", "knowledge_extractions", ["tenant_id", "source_type", "source_id"])
    # RLS
    op.execute("ALTER TABLE knowledge_extractions ENABLE ROW LEVEL SECURITY;")
    op.execute("CREATE POLICY knowledge_extractions_tenant_isolation ON knowledge_extractions USING (tenant_id::text = current_setting('app.current_tenant_id', true));")
 def downgrade() -> None:
    op.drop_table("knowledge_extractions")
@@ -0,0 +1,116 @@
 """self-improvement tables: signals, patterns, proposals, impact measurements
 Revision ID: 0132
 Revises: 0131
 Create Date: 2026-08-21
 """
 from alembic import op
 import sqlalchemy as sa
 from sqlalchemy.dialects.postgresql import UUID, JSONB
 revision = "0132"
 down_revision = "0131"
 branch_labels = None
 depends_on = None
 def upgrade() -> None:
    # 1. improvement_patterns (created first because signals has FK to it)
    op.create_table(
        "improvement_patterns",
        sa.Column("id", UUID(as_uuid=True), primary_key=True, server_default=sa.text("gen_random_uuid()")),
        sa.Column("tenant_id", UUID(as_uuid=True), sa.ForeignKey("tenants.id", ondelete="CASCADE"), nullable=False),
        sa.Column("pattern_kind", sa.String(40), nullable=False),
        sa.Column("title", sa.String(300), nullable=False),
        sa.Column("description", sa.Text, nullable=False, server_default=sa.text("''")),
        sa.Column("target_type", sa.String(30), nullable=False),
        sa.Column("target_name", sa.String(200), nullable=False, server_default=sa.text("'unknown'")),
        sa.Column("evidence_refs", JSONB, nullable=False, server_default=sa.text("'[]'::jsonb")),
        sa.Column("occurrence_count", sa.Integer, nullable=False, server_default=sa.text("1")),
        sa.Column("confidence", sa.Float, nullable=False, server_default=sa.text("0.5")),
        sa.Column("status", sa.String(20), nullable=False, server_default=sa.text("'detected'")),
        sa.Column("proposed_action", sa.Text, nullable=False, server_default=sa.text("''")),
        sa.Column("created_at", sa.DateTime(timezone=True), nullable=False, server_default=sa.text("now()")),
        sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False, server_default=sa.text("now()")),
    )
    op.create_index("ix_impr_patterns_tenant_status", "improvement_patterns", ["tenant_id", "status"])
    op.create_index("ix_impr_patterns_tenant_target", "improvement_patterns", ["tenant_id", "target_type"])
    # 2. improvement_signals
    op.create_table(
        "improvement_signals",
        sa.Column("id", UUID(as_uuid=True), primary_key=True, server_default=sa.text("gen_random_uuid()")),
        sa.Column("tenant_id", UUID(as_uuid=True), sa.ForeignKey("tenants.id", ondelete="CASCADE"), nullable=False),
        sa.Column("source_type", sa.String(50), nullable=False),
        sa.Column("source_ref_id", UUID(as_uuid=True), nullable=True),
        sa.Column("source_metadata", JSONB, nullable=False, server_default=sa.text("'{}'::jsonb")),
        sa.Column("summary", sa.Text, nullable=False),
        sa.Column("signal_kind", sa.String(30), nullable=False),
        sa.Column("severity", sa.String(20), nullable=False, server_default=sa.text("'info'")),
        sa.Column("confidence", sa.Float, nullable=False, server_default=sa.text("0.5")),
        sa.Column("pattern_id", UUID(as_uuid=True), sa.ForeignKey("improvement_patterns.id", ondelete="SET NULL"), nullable=True),
        sa.Column("created_at", sa.DateTime(timezone=True), nullable=False, server_default=sa.text("now()")),
    )
    op.create_index("ix_impr_signals_tenant_kind", "improvement_signals", ["tenant_id", "signal_kind"])
    op.create_index("ix_impr_signals_tenant_source", "improvement_signals", ["tenant_id", "source_type"])
    op.create_index("ix_impr_signals_tenant_pattern", "improvement_signals", ["tenant_id", "pattern_id"])
    # 3. improvement_proposals
    op.create_table(
        "improvement_proposals",
        sa.Column("id", UUID(as_uuid=True), primary_key=True, server_default=sa.text("gen_random_uuid()")),
        sa.Column("tenant_id", UUID(as_uuid=True), sa.ForeignKey("tenants.id", ondelete="CASCADE"), nullable=False),
        sa.Column("owner_id", UUID(as_uuid=True), sa.ForeignKey("users.id", ondelete="SET NULL"), nullable=True),
        sa.Column("pattern_id", UUID(as_uuid=True), sa.ForeignKey("improvement_patterns.id", ondelete="SET NULL"), nullable=True),
        sa.Column("title", sa.String(300), nullable=False),
        sa.Column("description", sa.Text, nullable=False, server_default=sa.text("''")),
        sa.Column("target_type", sa.String(30), nullable=False),
        sa.Column("target_ref_id", UUID(as_uuid=True), nullable=True),
        sa.Column("target_name", sa.String(200), nullable=True),
        sa.Column("version_number", sa.Integer, nullable=False, server_default=sa.text("1")),
        sa.Column("proposed_config", JSONB, nullable=False, server_default=sa.text("'{}'::jsonb")),
        sa.Column("previous_config", JSONB, nullable=False, server_default=sa.text("'{}'::jsonb")),
        sa.Column("evidence_refs", JSONB, nullable=False, server_default=sa.text("'[]'::jsonb")),
        sa.Column("rationale", sa.Text, nullable=False, server_default=sa.text("''")),
        sa.Column("expected_benefit", sa.Text, nullable=False, server_default=sa.text("''")),
        sa.Column("risk_assessment", sa.Text, nullable=False, server_default=sa.text("''")),
        sa.Column("evaluation_result", JSONB, nullable=False, server_default=sa.text("'{}'::jsonb")),
        sa.Column("evaluated_at", sa.DateTime(timezone=True), nullable=True),
        sa.Column("approval_request_id", UUID(as_uuid=True), nullable=True),
        sa.Column("approved_by", UUID(as_uuid=True), sa.ForeignKey("users.id", ondelete="SET NULL"), nullable=True),
        sa.Column("approved_at", sa.DateTime(timezone=True), nullable=True),
        sa.Column("activated_at", sa.DateTime(timezone=True), nullable=True),
        sa.Column("rolled_back_at", sa.DateTime(timezone=True), nullable=True),
        sa.Column("rollback_reason", sa.Text, nullable=True),
        sa.Column("status", sa.String(20), nullable=False, server_default=sa.text("'draft'")),
        sa.Column("created_at", sa.DateTime(timezone=True), nullable=False, server_default=sa.text("now()")),
        sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False, server_default=sa.text("now()")),
    )
    op.create_index("ix_impr_proposals_tenant_status", "improvement_proposals", ["tenant_id", "status"])
    op.create_index("ix_impr_proposals_tenant_target", "improvement_proposals", ["tenant_id", "target_type"])
    op.create_index("ix_impr_proposals_tenant_pattern", "improvement_proposals", ["tenant_id", "pattern_id"])
    # 4. improvement_impact_measurements
    op.create_table(
        "improvement_impact_measurements",
        sa.Column("id", UUID(as_uuid=True), primary_key=True, server_default=sa.text("gen_random_uuid()")),
        sa.Column("tenant_id", UUID(as_uuid=True), sa.ForeignKey("tenants.id", ondelete="CASCADE"), nullable=False),
        sa.Column("proposal_id", UUID(as_uuid=True), sa.ForeignKey("improvement_proposals.id", ondelete="CASCADE"), nullable=False),
        sa.Column("pre_metrics", JSONB, nullable=False, server_default=sa.text("'{}'::jsonb")),
        sa.Column("post_metrics", JSONB, nullable=False, server_default=sa.text("'{}'::jsonb")),
        sa.Column("delta", JSONB, nullable=False, server_default=sa.text("'{}'::jsonb")),
        sa.Column("assessment", sa.Text, nullable=False, server_default=sa.text("''")),
        sa.Column("is_positive", sa.String(20), nullable=False, server_default=sa.text("'neutral'")),
        sa.Column("created_at", sa.DateTime(timezone=True), nullable=False, server_default=sa.text("now()")),
    )
    op.create_index("ix_impr_impact_tenant_proposal", "improvement_impact_measurements", ["tenant_id", "proposal_id"])
    # RLS for all 4 tables
    for table in ["improvement_patterns", "improvement_signals", "improvement_proposals", "improvement_impact_measurements"]:
        op.execute(f"ALTER TABLE {table} ENABLE ROW LEVEL SECURITY;")
        op.execute(f"CREATE POLICY {table}_tenant_isolation ON {table} USING (tenant_id::text = current_setting('app.current_tenant_id', true));")
 def downgrade() -> None:
    for table in ["improvement_impact_measurements", "improvement_proposals", "improvement_signals", "improvement_patterns"]:
        op.drop_table(table)
@@ -0,0 +1,51 @@
 """compliance_incidents table for AI/privacy/security incident register
 Revision ID: 0133
 Revises: 0132
 Create Date: 2026-08-21
 """
 from alembic import op
 import sqlalchemy as sa
 from sqlalchemy.dialects.postgresql import UUID, JSONB
 revision = "0133"
 down_revision = "0132"
 branch_labels = None
 depends_on = None
 def upgrade() -> None:
    op.create_table(
        "compliance_incidents",
        sa.Column("id", UUID(as_uuid=True), primary_key=True, server_default=sa.text("gen_random_uuid()")),
        sa.Column("tenant_id", UUID(as_uuid=True), sa.ForeignKey("tenants.id", ondelete="CASCADE"), nullable=False),
        sa.Column("incident_type", sa.String(30), nullable=False, server_default=sa.text("'ai'")),
        sa.Column("title", sa.String(300), nullable=False),
        sa.Column("description", sa.Text, nullable=False, server_default=sa.text("''")),
        sa.Column("affected_use_cases", JSONB, nullable=False, server_default=sa.text("'[]'::jsonb")),
        sa.Column("affected_versions", JSONB, nullable=False, server_default=sa.text("'[]'::jsonb")),
        sa.Column("provider", sa.String(100), nullable=False, server_default=sa.text("''")),
        sa.Column("measures_taken", sa.Text, nullable=False, server_default=sa.text("''")),
        sa.Column("evidence_refs", JSONB, nullable=False, server_default=sa.text("'[]'::jsonb")),
        sa.Column("status", sa.String(20), nullable=False, server_default=sa.text("'open'")),
        sa.Column("created_by", UUID(as_uuid=True), sa.ForeignKey("users.id", ondelete="SET NULL"), nullable=True),
        sa.Column("resolved_by", UUID(as_uuid=True), sa.ForeignKey("users.id", ondelete="SET NULL"), nullable=True),
        sa.Column("resolved_at", sa.DateTime(timezone=True), nullable=True),
        sa.Column("deleted_at", sa.DateTime(timezone=True), nullable=True),
        sa.Column("created_at", sa.DateTime(timezone=True), nullable=False, server_default=sa.text("now()")),
        sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False, server_default=sa.text("now()")),
    )
    op.create_index("ix_compliance_incidents_tenant_status", "compliance_incidents", ["tenant_id", "status"])
    op.create_index("ix_compliance_incidents_tenant_type", "compliance_incidents", ["tenant_id", "incident_type"])
    # Add retention_config JSONB column to system_settings for compliance retention overrides
    op.add_column("system_settings", sa.Column("retention_config", JSONB, nullable=True, server_default=sa.text("'{}'::jsonb")))
    # RLS
    op.execute("ALTER TABLE compliance_incidents ENABLE ROW LEVEL SECURITY;")
    op.execute("CREATE POLICY compliance_incidents_tenant_isolation ON compliance_incidents USING (tenant_id::text = current_setting('app.current_tenant_id', true));")
 def downgrade() -> None:
    op.drop_column("system_settings", "retention_config")
    op.drop_table("compliance_incidents")
@@ -0,0 +1,26 @@
 """Fix notification_types column sizes — VARCHAR(20) too small for values.
 Revision ID: 0134
 Revises: 0133
 Create Date: 2026-08-21
 """
 from alembic import op
 revision = "0134"
 down_revision = "0133"
 branch_labels = None
 depends_on = None
 def upgrade() -> None:
    op.execute("ALTER TABLE notification_types ALTER COLUMN type_key TYPE VARCHAR(100);")
    op.execute("ALTER TABLE notification_types ALTER COLUMN plugin_name TYPE VARCHAR(100);")
    op.execute("ALTER TABLE notification_types ALTER COLUMN category TYPE VARCHAR(50);")
    op.execute("ALTER TABLE notification_types ALTER COLUMN label TYPE VARCHAR(200);")
 def downgrade() -> None:
    op.execute("ALTER TABLE notification_types ALTER COLUMN label TYPE VARCHAR(200);")
    op.execute("ALTER TABLE notification_types ALTER COLUMN category TYPE VARCHAR(20);")
    op.execute("ALTER TABLE notification_types ALTER COLUMN plugin_name TYPE VARCHAR(20);")
    op.execute("ALTER TABLE notification_types ALTER COLUMN type_key TYPE VARCHAR(20);")
@@ -0,0 +1,60 @@
 """Fix schema drifts — VARCHAR lengths + missing tables.
 Revision ID: 0135
 Revises: 0134
 Create Date: 2026-08-21
 """
 from alembic import op
 import sqlalchemy as sa
 from sqlalchemy.dialects.postgresql import UUID, JSONB
 revision = "0135"
 down_revision = "0134"
 branch_labels = None
 depends_on = None
 def upgrade() -> None:
    # 1. Fix VARCHAR length mismatches (model defines longer than DB)
    # Drop notifications_legacy view first — it depends on notifications.type column
    op.execute("DROP VIEW IF EXISTS notifications_legacy CASCADE;")
    op.execute("ALTER TABLE contacts ALTER COLUMN status TYPE VARCHAR(30);")
    op.execute("ALTER TABLE notifications ALTER COLUMN type TYPE VARCHAR(100);")
    op.execute("ALTER TABLE notification_preferences ALTER COLUMN type_key TYPE VARCHAR(100);")
    # 2. Create missing table: forgejo_reported_errors (only if not exists)
    op.execute("""
        CREATE TABLE IF NOT EXISTS forgejo_reported_errors (
            id SERIAL PRIMARY KEY,
            dedup_key VARCHAR(64) NOT NULL UNIQUE,
            message TEXT NOT NULL,
            stack TEXT,
            forgejo_issue_number INTEGER,
            reported_at TIMESTAMPTZ DEFAULT now() NOT NULL,
            status VARCHAR(20) NOT NULL DEFAULT 'reported'
        )
    """)
    # 3. Create missing table: pgp_keys (only if not exists)
    op.execute("""
        CREATE TABLE IF NOT EXISTS pgp_keys (
            id UUID DEFAULT gen_random_uuid() NOT NULL PRIMARY KEY,
            tenant_id UUID NOT NULL REFERENCES tenants(id) ON DELETE CASCADE,
            user_id UUID NOT NULL,
            key_id VARCHAR(255) NOT NULL,
            encrypted_private_key TEXT NOT NULL,
            public_key_armored TEXT NOT NULL
        )
    """)
    op.execute("CREATE INDEX IF NOT EXISTS ix_pgp_keys_user ON pgp_keys (user_id);")
    op.execute("ALTER TABLE pgp_keys ENABLE ROW LEVEL SECURITY;")
    op.execute("DROP POLICY IF EXISTS pgp_keys_tenant_isolation ON pgp_keys;")
    op.execute("CREATE POLICY pgp_keys_tenant_isolation ON pgp_keys USING (tenant_id::text = current_setting('app.current_tenant_id', true));")
 def downgrade() -> None:
    op.drop_table("pgp_keys")
    op.drop_table("forgejo_reported_errors")
    op.execute("ALTER TABLE notification_preferences ALTER COLUMN type_key TYPE VARCHAR(20);")
    op.execute("ALTER TABLE notifications ALTER COLUMN type TYPE VARCHAR(20);")
    op.execute("ALTER TABLE contacts ALTER COLUMN status TYPE VARCHAR(20);")
@@ -0,0 +1,70 @@
 """Fix RLS policies — app.tenant_id → app.current_tenant_id.
 8 RLS policies in production reference 'app.tenant_id' which doesn't exist
 as a PostgreSQL parameter. The code uses 'app.current_tenant_id'.
 This causes 500 errors on roles, sequences, wiki, approval_requests,
 ai_decision_records, and automation_agent_run_steps.
 Revision ID: 0136
 Revises: 0135
 Create Date: 2026-08-21
 """
 from alembic import op
 import sqlalchemy as sa
 revision = "0136"
 down_revision = "0135"
 branch_labels = None
 depends_on = None
 # All 8 tables with broken RLS policies referencing app.tenant_id
 TABLES_WITH_BAD_RLS = [
    "ai_decision_records",
    "approval_requests",
    "automation_agent_run_steps",
    "roles",
    "sequences",
    "wiki_articles",
    "wiki_article_versions",
    "wiki_categories",
 ]
 def _table_exists(conn, table_name: str) -> bool:
    """True when the table exists (dual-path convergence, Gate B).
    Plugin-owned tables may not exist yet on a fresh install when Alembic
    reaches this revision — skip them instead of failing. The plugin-side
    convergence migrations apply the same RLS policies.
    """
    row = conn.execute(
        sa.text("SELECT to_regclass(:tname) IS NOT NULL"),
        {"tname": f"public.{table_name}"},
    ).scalar()
    return bool(row)
 def upgrade() -> None:
    conn = op.get_bind()
    for table in TABLES_WITH_BAD_RLS:
        if not _table_exists(conn, table):
            continue
        # Drop old policy with app.tenant_id
        op.execute(f"DROP POLICY IF EXISTS tenant_isolation ON {table};")
        # Create new policy with app.current_tenant_id
        op.execute(
            f"CREATE POLICY tenant_isolation ON {table} "
            f"USING (tenant_id::text = current_setting('app.current_tenant_id', true));"
        )
 def downgrade() -> None:
    conn = op.get_bind()
    for table in TABLES_WITH_BAD_RLS:
        if not _table_exists(conn, table):
            continue
        op.execute(f"DROP POLICY IF EXISTS tenant_isolation ON {table};")
        op.execute(
            f"CREATE POLICY tenant_isolation ON {table} "
            f"USING (tenant_id::text = current_setting('app.tenant_id', true));"
        )
@@ -0,0 +1,34 @@
 """Drop AI chat tables (migrated to comm conversations)
 Revision ID: 0137
 Revises: 0136
 Create Date: 2026-08-21
 AI chat functionality is now handled by the kommunikation plugin's
 comm_conversations and comm_messages tables. The old AI-specific tables
 (ai_chat_sessions, ai_chat_messages, ai_chat_attachments, ai_conversations,
 ai_messages) are no longer needed and are dropped.
 """
 from alembic import op
 import sqlalchemy as sa
 # revision identifiers, used by Alembic.
 revision = "0137"
 down_revision = "0136"
 branch_labels = None
 depends_on = None
 def upgrade() -> None:
    # Use IF EXISTS to avoid errors if tables are already gone
    op.execute("DROP TABLE IF EXISTS ai_chat_attachments CASCADE")
    op.execute("DROP TABLE IF EXISTS ai_chat_messages CASCADE")
    op.execute("DROP TABLE IF EXISTS ai_chat_sessions CASCADE")
    op.execute("DROP TABLE IF EXISTS ai_messages CASCADE")
    op.execute("DROP TABLE IF EXISTS ai_conversations CASCADE")
 def downgrade() -> None:
    # Tables cannot be restored — data was migrated or was empty.
    pass
@@ -0,0 +1,63 @@
 """Tags: parent_id, applicable_to, icon columns
 Revision ID: 0138
 Revises: 0137
 Create Date: 2026-08-21
 Adds parent_id for tree structure, applicable_to for entity-type filtering,
 and icon for per-tag icon selection.
 """
 from alembic import op
 import sqlalchemy as sa
 from sqlalchemy.dialects.postgresql import UUID as PGUUID, JSONB
 # revision identifiers, used by Alembic.
 revision = "0138"
 down_revision = "0137"
 branch_labels = None
 depends_on = None
 def _table_exists(conn, table_name: str) -> bool:
    """True when the table exists (dual-path convergence, Gate B).
    On a fresh install the tags plugin SQL migration has not run yet when
    Alembic reaches this revision — skip instead of failing. The plugin-side
    convergence migration adds the same columns.
    """
    row = conn.execute(
        sa.text("SELECT to_regclass(:tname) IS NOT NULL"),
        {"tname": f"public.{table_name}"},
    ).scalar()
    return bool(row)
 def upgrade() -> None:
    conn = op.get_bind()
    if not _table_exists(conn, "tags"):
        return
    # parent_id for tree structure (self-referencing FK)
    op.add_column("tags", sa.Column("parent_id", PGUUID(as_uuid=True), nullable=True))
    op.create_foreign_key(
        "fk_tags_parent_id", "tags", "tags", ["parent_id"], ["id"], ondelete="SET NULL"
    )
    op.create_index("ix_tags_parent", "tags", ["parent_id"])
    # applicable_to: list of entity types where this tag can be applied
    op.add_column("tags", sa.Column("applicable_to", JSONB, nullable=True))
    # icon: icon name for frontend display
    op.add_column("tags", sa.Column("icon", sa.String(50), nullable=True))
 def downgrade() -> None:
    conn = op.get_bind()
    if not _table_exists(conn, "tags"):
        return
    op.drop_column("tags", "icon")
    op.drop_column("tags", "applicable_to")
    op.drop_index("ix_tags_parent", table_name="tags")
    op.drop_constraint("fk_tags_parent_id", "tags", type_="foreignkey")
    op.drop_column("tags", "parent_id")
@@ -0,0 +1,48 @@
 """Reports: folder_id column for folder-based sorting
 Revision ID: 0139
 Revises: 0138
 Create Date: 2026-08-21
 Adds folder_id to report_templates for folder-based organization.
 """
 from alembic import op
 import sqlalchemy as sa
 from sqlalchemy.dialects.postgresql import UUID as PGUUID
 # revision identifiers, used by Alembic.
 revision = "0139"
 down_revision = "0138"
 branch_labels = None
 depends_on = None
 def _table_exists(conn, table_name: str) -> bool:
    """True when the table exists (dual-path convergence, Gate B).
    On a fresh install the report_generator plugin SQL migration has not
    run yet when Alembic reaches this revision — skip instead of failing.
    The plugin-side convergence migration adds the same column.
    """
    row = conn.execute(
        sa.text("SELECT to_regclass(:tname) IS NOT NULL"),
        {"tname": f"public.{table_name}"},
    ).scalar()
    return bool(row)
 def upgrade() -> None:
    conn = op.get_bind()
    if not _table_exists(conn, "report_templates"):
        return
    op.add_column("report_templates", sa.Column("folder_id", PGUUID(as_uuid=True), nullable=True))
    op.create_index("ix_report_templates_folder", "report_templates", ["folder_id"])
 def downgrade() -> None:
    conn = op.get_bind()
    if not _table_exists(conn, "report_templates"):
        return
    op.drop_index("ix_report_templates_folder", table_name="report_templates")
    op.drop_column("report_templates", "folder_id")
@@ -0,0 +1,48 @@
 """Communication: folder_id in comm_conversations for folder organization
 Revision ID: 0140
 Revises: 0139
 Create Date: 2026-08-21
 Adds folder_id to comm_conversations for folder-based organization.
 """
 from alembic import op
 import sqlalchemy as sa
 from sqlalchemy.dialects.postgresql import UUID as PGUUID
 # revision identifiers, used by Alembic.
 revision = "0140"
 down_revision = "0139"
 branch_labels = None
 depends_on = None
 def _table_exists(conn, table_name: str) -> bool:
    """True when the table exists (dual-path convergence, Gate B).
    On a fresh install the kommunikation plugin SQL migration has not run
    yet when Alembic reaches this revision — skip instead of failing.
    The plugin-side migration adds the same column idempotently.
    """
    row = conn.execute(
        sa.text("SELECT to_regclass(:tname) IS NOT NULL"),
        {"tname": f"public.{table_name}"},
    ).scalar()
    return bool(row)
 def upgrade() -> None:
    conn = op.get_bind()
    if not _table_exists(conn, "comm_conversations"):
        return
    op.add_column("comm_conversations", sa.Column("folder_id", PGUUID(as_uuid=True), nullable=True))
    op.create_index("ix_comm_conversations_folder", "comm_conversations", ["folder_id"])
 def downgrade() -> None:
    conn = op.get_bind()
    if not _table_exists(conn, "comm_conversations"):
        return
    op.drop_index("ix_comm_conversations_folder", table_name="comm_conversations")
    op.drop_column("comm_conversations", "folder_id")
@@ -0,0 +1,84 @@
 '''Fix role permission wildcard patterns to canonical 2-segment schema
 Revision ID: 0141
 Revises: 0140
 Create Date: 2026-08-23
 Migration 0019 seeded default roles with 3-segment permission patterns
 (core:*:read etc.). The runtime matcher (_matches_permission) compares
 segment counts strictly, so those patterns could never match any
 2-segment requirement - editor/viewer roles were silently dead.
 Canonical schema is module:action (2 segments, * wildcards allowed).
 core:*:X means all modules with action X, so it converts to *:X.
 '''
 from alembic import op
 # revision identifiers, used by Alembic.
 revision = '0141'
 down_revision = '0140'
 branch_labels = None
 depends_on = None
 # Rebuild the permissions JSONB object, rewriting every key that starts
 # with the dead 'core:' prefix to its 2-segment equivalent ('*:X').
 _UPGRADE_SQL = '''
 UPDATE roles
 SET permissions = sub.new_perms,
    permission_version = permission_version + 1
 FROM (
    SELECT
        r.id AS role_id,
        jsonb_object_agg(
            CASE WHEN k LIKE 'core:%'
                 THEN '*:' || split_part(k, ':', 3)
                 ELSE k END,
            v
        ) AS new_perms
    FROM roles r,
         jsonb_each(r.permissions) AS e(k, v)
    GROUP BY r.id
 ) AS sub
 WHERE roles.id = sub.role_id
  AND EXISTS (
      SELECT 1 FROM jsonb_object_keys(roles.permissions) k
      WHERE k LIKE 'core:%'
  )
 '''
 # Reverse: map '*:X' back to 'core:*:X' only for keys that came from the
 # original seeding pattern. Roles that legitimately use '*:X' without a
 # matching 'core:*:X' history are left untouched (best-effort downgrade).
 _DOWNGRADE_SQL = '''
 UPDATE roles
 SET permissions = sub.new_perms,
    permission_version = permission_version + 1
 FROM (
    SELECT
        r.id AS role_id,
        jsonb_object_agg(
            CASE WHEN k = '*:' || split_part(k, ':', 2)
                  AND k <> '*:*'
                 THEN 'core:*:' || split_part(k, ':', 2)
                 ELSE k END,
            v
        ) AS new_perms
    FROM roles r,
         jsonb_each(r.permissions) AS e(k, v)
    GROUP BY r.id
 ) AS sub
 WHERE roles.id = sub.role_id
  AND EXISTS (
      SELECT 1 FROM jsonb_object_keys(roles.permissions) k
      WHERE k = '*:' || split_part(k, ':', 2) AND k <> '*:*'
  )
 '''
 def upgrade() -> None:
    op.execute(_UPGRADE_SQL)
 def downgrade() -> None:
    op.execute(_DOWNGRADE_SQL)
@@ -0,0 +1,39 @@
 """Add backup config columns to system_settings table.
 Follow-up to 0130: the backup feature (10b1f83) added backup_interval,
 backup_retention_days and backup_destination to schema/service/frontend
 but missed model columns and this migration.
 Revision ID: 0142
 Revises: 0141
 """
 import sqlalchemy as sa
 from alembic import op
 revision = "0142"
 down_revision = "0141"
 branch_labels = None
 depends_on = None
 def upgrade() -> None:
    op.add_column(
        "system_settings",
        sa.Column("backup_interval", sa.String(20), nullable=False, server_default="daily"),
    )
    op.add_column(
        "system_settings",
        sa.Column("backup_retention_days", sa.Integer(), nullable=False, server_default="7"),
    )
    op.add_column(
        "system_settings",
        sa.Column("backup_destination", sa.String(20), nullable=False, server_default="local"),
    )
 def downgrade() -> None:
    op.drop_column("system_settings", "backup_destination")
    op.drop_column("system_settings", "backup_retention_days")
    op.drop_column("system_settings", "backup_interval")
@@ -0,0 +1,115 @@
 """Documents Generator tables (Phase L1): letterheads, print_templates,
 document_assets.
 Revision ID: 0143
 Revises: 0142
 Create Date: 2026-08-29
 Dual-path convergence (Gate B): on plugin-first installs the report_generator
 plugin migration 0003 has already created these tables — skip instead of
 failing. Both paths converge to the identical schema (see
 app/plugins/builtins/report_generator/migrations/0003_documents_generator.sql).
 """
 import sqlalchemy as sa
 from sqlalchemy.dialects.postgresql import JSONB
 from sqlalchemy.dialects.postgresql import UUID as PGUUID
 from alembic import op
 revision = "0143"
 down_revision = "0142"
 branch_labels = None
 depends_on = None
 def _table_exists(conn, table_name: str) -> bool:
    row = conn.execute(
        sa.text("SELECT to_regclass(:tname) IS NOT NULL"),
        {"tname": f"public.{table_name}"},
    ).scalar()
    return bool(row)
 def _rls(table: str) -> None:
    op.execute(f"ALTER TABLE {table} ENABLE ROW LEVEL SECURITY")
    op.execute(f"ALTER TABLE {table} FORCE ROW LEVEL SECURITY")
    op.execute(f"DROP POLICY IF EXISTS {table}_tenant_isolation ON {table}")
    op.execute(
        f"CREATE POLICY {table}_tenant_isolation ON {table} AS PERMISSIVE "
        f"FOR ALL TO crm_api "
        f"USING (tenant_id = NULLIF(current_setting('app.current_tenant_id', true), '')::uuid) "
        f"WITH CHECK (tenant_id = NULLIF(current_setting('app.current_tenant_id', true), '')::uuid)"
    )
 def upgrade() -> None:
    conn = op.get_bind()
    if _table_exists(conn, "letterheads"):
        return
    op.create_table(
        "letterheads",
        sa.Column("id", PGUUID(as_uuid=True), primary_key=True),
        sa.Column("name", sa.String(255), nullable=False),
        sa.Column("description", sa.Text(), nullable=False, server_default=""),
        sa.Column("config", JSONB(), nullable=False, server_default=sa.text("'{}'::jsonb")),
        sa.Column("is_default", sa.Boolean(), nullable=False, server_default=sa.false()),
        sa.Column("tenant_id", PGUUID(as_uuid=True), nullable=False),
        sa.Column("owner_id", PGUUID(as_uuid=True), sa.ForeignKey("users.id", ondelete="SET NULL"), nullable=True),
        sa.Column("deleted_at", sa.DateTime(timezone=True)),
        sa.Column("created_at", sa.DateTime(timezone=True), nullable=False, server_default=sa.func.now()),
        sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False, server_default=sa.func.now()),
        sa.Column("created_by", PGUUID(as_uuid=True), nullable=False),
    )
    op.create_index("ix_letterheads_tenant", "letterheads", ["tenant_id"])
    op.create_index("ix_letterheads_name", "letterheads", ["name"])
    op.create_table(
        "print_templates",
        sa.Column("id", PGUUID(as_uuid=True), primary_key=True),
        sa.Column("name", sa.String(255), nullable=False),
        sa.Column("description", sa.Text(), nullable=False, server_default=""),
        sa.Column("letterhead_id", PGUUID(as_uuid=True), sa.ForeignKey("letterheads.id", ondelete="SET NULL"), nullable=True),
        sa.Column("entity_type", sa.String(100), nullable=False, server_default="contact"),
        sa.Column("blocks", JSONB(), nullable=False, server_default=sa.text("'[]'::jsonb")),
        sa.Column("output_format", sa.String(20), nullable=False, server_default="pdf"),
        sa.Column("tenant_id", PGUUID(as_uuid=True), nullable=False),
        sa.Column("owner_id", PGUUID(as_uuid=True), sa.ForeignKey("users.id", ondelete="SET NULL"), nullable=True),
        sa.Column("deleted_at", sa.DateTime(timezone=True)),
        sa.Column("created_at", sa.DateTime(timezone=True), nullable=False, server_default=sa.func.now()),
        sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False, server_default=sa.func.now()),
        sa.Column("created_by", PGUUID(as_uuid=True), nullable=False),
    )
    op.create_index("ix_print_templates_tenant", "print_templates", ["tenant_id"])
    op.create_index("ix_print_templates_name", "print_templates", ["name"])
    op.create_table(
        "document_assets",
        sa.Column("id", PGUUID(as_uuid=True), primary_key=True),
        sa.Column("letterhead_id", PGUUID(as_uuid=True), sa.ForeignKey("letterheads.id", ondelete="CASCADE"), nullable=True),
        sa.Column("filename", sa.String(255), nullable=False),
        sa.Column("mime_type", sa.String(100), nullable=False),
        sa.Column("size_bytes", sa.Integer(), nullable=False, server_default="0"),
        sa.Column("storage_path", sa.String(1024), nullable=False),
        sa.Column("tenant_id", PGUUID(as_uuid=True), nullable=False),
        sa.Column("owner_id", PGUUID(as_uuid=True), sa.ForeignKey("users.id", ondelete="SET NULL"), nullable=True),
        sa.Column("deleted_at", sa.DateTime(timezone=True)),
        sa.Column("created_at", sa.DateTime(timezone=True), nullable=False, server_default=sa.func.now()),
        sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False, server_default=sa.func.now()),
        sa.Column("created_by", PGUUID(as_uuid=True), nullable=False),
    )
    op.create_index("ix_document_assets_tenant", "document_assets", ["tenant_id"])
    op.create_index("ix_document_assets_letterhead", "document_assets", ["letterhead_id"])
    for table in ("letterheads", "print_templates", "document_assets"):
        _rls(table)
 def downgrade() -> None:
    conn = op.get_bind()
    if not _table_exists(conn, "letterheads"):
        return
    for table in ("document_assets", "print_templates", "letterheads"):
        op.execute(f"DROP POLICY IF EXISTS {table}_tenant_isolation ON {table}")
        op.drop_table(table)
@@ -0,0 +1,111 @@
 """Personal dashboards table (Phase M2) + RLS policy-role convergence.
 Revision ID: 0144
 Revises: 0143
 Create Date: 2026-08-30
 Part 1 — dashboards: personal per-user dashboard layouts (JSONB tabs /
 widgets). RLS follows the 0090 fail-closed pattern scoped to BOTH runtime
 roles (crm_api, crm_worker).
 Part 2 — convergence fix (measured live on production 2026-08-30):
 migration 0143 created the letterheads/print_templates/document_assets
 tenant-isolation policies with ``TO crm_api`` only, while the established
 pattern (0090, verified by tests/test_rls_coverage.py) requires both
 crm_api AND crm_worker. This migration recreates those policies with both
 roles so both install paths (plugin-SQL 0003 / alembic 0143) converge.
 """
 import sqlalchemy as sa
 from sqlalchemy.dialects.postgresql import JSONB
 from sqlalchemy.dialects.postgresql import UUID as PGUUID
 from alembic import op
 revision = "0144"
 down_revision = "0143"
 branch_labels = None
 depends_on = None
 _TENANT_USING = (
    "tenant_id = NULLIF(current_setting('app.current_tenant_id', true), '')::uuid"
 )
 def _table_exists(conn, table_name: str) -> bool:
    row = conn.execute(
        sa.text("SELECT to_regclass(:tname) IS NOT NULL"),
        {"tname": f"public.{table_name}"},
    ).scalar()
    return bool(row)
 def _create_policy(table: str) -> None:
    op.execute(f"DROP POLICY IF EXISTS {table}_tenant_isolation ON {table}")
    op.execute(
        f"CREATE POLICY {table}_tenant_isolation ON {table} AS PERMISSIVE "
        f"FOR ALL TO crm_api, crm_worker "
        f"USING ({_TENANT_USING}) "
        f"WITH CHECK ({_TENANT_USING})"
    )
 def _rls(table: str) -> None:
    op.execute(f"ALTER TABLE {table} ENABLE ROW LEVEL SECURITY")
    op.execute(f"ALTER TABLE {table} FORCE ROW LEVEL SECURITY")
    _create_policy(table)
 def upgrade() -> None:
    conn = op.get_bind()
    # ── Part 1: dashboards table ──
    if not _table_exists(conn, "dashboards"):
        op.create_table(
            "dashboards",
            sa.Column("id", PGUUID(as_uuid=True), primary_key=True),
            sa.Column("name", sa.String(100), nullable=False),
            sa.Column("layout", JSONB(), nullable=False, server_default=sa.text("'{}'::jsonb")),
            sa.Column("is_default", sa.Boolean(), nullable=False, server_default=sa.false()),
            sa.Column("user_id", PGUUID(as_uuid=True), sa.ForeignKey("users.id", ondelete="CASCADE"), nullable=False),
            sa.Column("tenant_id", PGUUID(as_uuid=True), nullable=False),
            sa.Column("deleted_at", sa.DateTime(timezone=True)),
            sa.Column("created_at", sa.DateTime(timezone=True), nullable=False, server_default=sa.func.now()),
            sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False, server_default=sa.func.now()),
        )
        op.create_index(
            "uq_dashboards_tenant_user_name",
            "dashboards",
            ["tenant_id", "user_id", "name"],
            unique=True,
            postgresql_where=sa.text("deleted_at IS NULL"),
        )
        op.create_index("ix_dashboards_tenant_user", "dashboards", ["tenant_id", "user_id"])
        _rls("dashboards")
    else:
        # Dual-path convergence: table exists (plugin SQL), ensure policy roles
        _create_policy("dashboards")
    # ── Part 2: converge Phase L policies to crm_api + crm_worker ──
    for table in ("letterheads", "print_templates", "document_assets"):
        if _table_exists(conn, table):
            _create_policy(table)
 def downgrade() -> None:
    conn = op.get_bind()
    # Revert the convergence fix to the (buggy) Phase L state first…
    for table in ("letterheads", "print_templates", "document_assets"):
        if _table_exists(conn, table):
            op.execute(f"DROP POLICY IF EXISTS {table}_tenant_isolation ON {table}")
            op.execute(
                f"CREATE POLICY {table}_tenant_isolation ON {table} AS PERMISSIVE "
                f"FOR ALL TO crm_api "
                f"USING ({_TENANT_USING}) "
                f"WITH CHECK ({_TENANT_USING})"
            )
    if _table_exists(conn, "dashboards"):
        op.execute("DROP POLICY IF EXISTS dashboards_tenant_isolation ON dashboards")
        op.drop_index("ix_dashboards_tenant_user", table_name="dashboards")
        op.drop_index("uq_dashboards_tenant_user_name", table_name="dashboards")
        op.drop_table("dashboards")
@@ -0,0 +1,80 @@
 """Converged DELETE grants (F20/Astra).
 Removes the effect of the blanket ``GRANT DELETE ON ALL TABLES`` that
 prestart.sh applied on every boot — which silently undid migration
 0100's protections on every container start.
 Documented target state:
 Runtime-legitimate DELETEs (crm_api only):
  - users, user_tenants      (user deletion on last membership, BUG-030)
  - sessions                 (logout session invalidation)
  - plugins, notification_types  (plugin uninstall + registry sync)
 Protected — DELETE stays REVOKED from crm_api AND crm_worker:
  - audit_log                (Astra acceptance: API/Worker write, never delete)
  - api_tokens               (revoke is an UPDATE on revoked_at)
  - password_reset_tokens    (consumption is an UPDATE on used_at)
  - plugin_allowlist, plugin_migrations (install/migration path only —
    plugin_migrations rows are deleted via the migration factory)
  - tenants                  (never deleted at runtime)
  - tenant_plugin_activation (deactivation is an UPDATE)
 crm_worker receives no DELETE on any protected table (workers never
 delete users, sessions or plugin rows).
 Revision ID: 0145
 Revises: 0144
 """
 from alembic import op
 revision = "0145"
 down_revision = "0144"
 branch_labels = None
 depends_on = None
 # Tables where runtime DELETE is a documented, legitimate operation (crm_api)
 RUNTIME_DELETE_TABLES = [
    "users",
    "user_tenants",
    "sessions",
    "plugins",
    "notification_types",
 ]
 # Tables where DELETE must stay revoked from BOTH runtime roles (0100 + F20)
 PROTECTED_TABLES = [
    "audit_log",
    "api_tokens",
    "password_reset_tokens",
    "plugin_allowlist",
    "plugin_migrations",
    "tenants",
    "tenant_plugin_activation",
 ]
 def upgrade() -> None:
    # 1. Re-assert 0100's revocations — production DBs have lived with the
    #    blanket boot grant, so revoke first for a deterministic baseline.
    for table in PROTECTED_TABLES:
        op.execute(f"REVOKE DELETE ON TABLE {table} FROM crm_api;")
        op.execute(f"REVOKE DELETE ON TABLE {table} FROM crm_worker;")
    # 2. Grant the runtime-legitimate DELETEs to crm_api (BUG-030 stays
    #    fixed, logout keeps working, plugin management keeps working).
    for table in RUNTIME_DELETE_TABLES:
        op.execute(f"GRANT DELETE ON TABLE {table} TO crm_api;")
 def downgrade() -> None:
    # Best-effort inverse: revoke the runtime grants, re-grant the
    # protected tables (matching the pre-F20 blanket state).
    for table in RUNTIME_DELETE_TABLES:
        op.execute(f"REVOKE DELETE ON TABLE {table} FROM crm_api;")
    for table in PROTECTED_TABLES:
        op.execute(f"GRANT DELETE ON TABLE {table} TO crm_api;")
        op.execute(f"GRANT DELETE ON TABLE {table} TO crm_worker;")
@@ -0,0 +1,31 @@
 """Add resolved_by to approval_requests (F11/Astra).
 Separates the assigned approver (approver_id — who the request was
 addressed TO) from the actual decider (resolved_by — who decided).
 Previously resolve_approval_request overwrote approver_id with the
 acting user, destroying the assignment record.
 Revision ID: 0146
 Revises: 0145
 """
 import sqlalchemy as sa
 from sqlalchemy.dialects.postgresql import UUID as PGUUID
 from alembic import op
 revision = "0146"
 down_revision = "0145"
 branch_labels = None
 depends_on = None
 def upgrade() -> None:
    op.add_column(
        "approval_requests",
        sa.Column("resolved_by", PGUUID(as_uuid=True), nullable=True),
    )
 def downgrade() -> None:
    op.drop_column("approval_requests", "resolved_by")
@@ -0,0 +1,50 @@
 """Disable RLS on api_tokens (F08 bootstrap fix, Astra S2).
 verify_api_token() must look up the token hash via the request session
 (crm_api) BEFORE any tenant context exists — the TOKEN is what determines
 the tenant. Forced RLS with a tenant-isolation policy on api_tokens made
 that lookup return zero rows, so EVERY Bearer token was rejected with 401
 "token_invalid", including freshly created ones (verified live on
 production 2026-09-18).
 This restores the documented decision from migration 0080 ("written
 during login before tenant context") which 0084 inadvertently overrode
 by blindly re-enabling fail-closed RLS everywhere. sessions and
 password_reset_tokens remain RLS-off for the same bootstrap reason.
 Security unchanged: the SHA-256 token hash IS the access secret — a
 lookup by hash cannot enumerate other tenants' tokens, and every use of
 the row still goes through the authenticated verify path.
 Revision ID: 0147
 Revises: 0146
 """
 from alembic import op
 revision = "0147"
 down_revision = "0146"
 branch_labels = None
 depends_on = None
 POLICY_NAME = "api_tokens_tenant_isolation"
 def upgrade() -> None:
    # Remove the tenant-isolation policy first (it only covered the
    # runtime roles anyway), then disable + unforce RLS.
    op.execute(f"DROP POLICY IF EXISTS {POLICY_NAME} ON api_tokens;")
    op.execute("ALTER TABLE api_tokens DISABLE ROW LEVEL SECURITY;")
    op.execute("ALTER TABLE api_tokens NO FORCE ROW LEVEL SECURITY;")
 def downgrade() -> None:
    # Best-effort inverse: restore forced RLS + the previous policy.
    op.execute("ALTER TABLE api_tokens ENABLE ROW LEVEL SECURITY;")
    op.execute("ALTER TABLE api_tokens FORCE ROW LEVEL SECURITY;")
    op.execute(
        "CREATE POLICY api_tokens_tenant_isolation ON api_tokens "
        "FOR ALL TO crm_api, crm_worker "
        "USING (tenant_id = (NULLIF(current_setting('app.current_tenant_id', true), ''))::uuid)"
    )
@@ -0,0 +1,41 @@
 """Add content_hash to plugin_migrations (F40/Astra).
 The migration runner previously tracked migrations by FILENAME only —
 editing an already-applied migration stayed unnoticed (the #389 bug
 class: a broken migration was fixed, but the runner silently skipped
 it because the filename was already tracked).
 Now every applied migration records the SHA-256 of its SQL content.
 On subsequent runs the runner compares hashes and logs a loud warning
 when an applied migration was modified (repaired) — the skip stays
 idempotent, but drift becomes VISIBLE.
 Revision ID: 0148
 Revises: 0147
 """
 import sqlalchemy as sa
 from alembic import op
 revision = "0148"
 down_revision = "0147"
 branch_labels = None
 depends_on = None
 def upgrade() -> None:
    op.add_column(
        "plugin_migrations",
        sa.Column("content_hash", sa.String(64), nullable=True),
    )
    op.create_index(
        "ix_plugin_migrations_content_hash",
        "plugin_migrations",
        ["content_hash"],
    )
 def downgrade() -> None:
    op.drop_index("ix_plugin_migrations_content_hash", table_name="plugin_migrations")
    op.drop_column("plugin_migrations", "content_hash")
@@ -6,9 +6,12 @@ Supports keyword-based intent detection for common CRM operations.
 from __future__ import annotations
 import logging
 import re
 from typing import Any
 logger = logging.getLogger(__name__)
 # Precompiled patterns for intent detection
 _PATTERNS = {
    "create_contact": re.compile(
@@ -27,6 +30,37 @@ _PATTERNS = {
    "help": re.compile(r"\b(help|what can you do|assist)\b", re.IGNORECASE),
 }
 # Plugin-contributed intents: pattern -> callable(query, context) -> list[dict] | None.
 # Registered via ``register_intent_pattern`` so plugins can extend the fallback
 # mapper without touching core code (Block H / HC-A).
 _CONTRIBUTED_INTENTS: list[tuple[re.Pattern[str], Any]] = []
 def register_intent_pattern(
    pattern: str | re.Pattern[str],
    handler: Any,
    *,
    owner: str = "",
 ) -> None:
    """Register a plugin-contributed intent for the fallback action mapper.
    Args:
        pattern: Regex (compiled or raw string) matching the user query.
        handler: Callable ``(query, context) -> list[dict] | None`` producing
            proposed actions when the pattern matches.
        owner: Optional plugin name, used by ``unregister_intent_patterns``.
    """
    compiled = re.compile(pattern) if isinstance(pattern, str) else pattern
    _CONTRIBUTED_INTENTS.append((compiled, handler))
 def unregister_intent_patterns(owner: str) -> None:
    """Remove all intents contributed by ``owner`` (plugin deactivation)."""
    global _CONTRIBUTED_INTENTS
    _CONTRIBUTED_INTENTS = [
        entry for entry in _CONTRIBUTED_INTENTS if getattr(entry[1], "owner_tag", None) != owner
    ]
 # Name extraction patterns - using single-quoted strings to avoid escaping issues
 _NAME_PATTERNS = [
    re.compile(r"\b(?:named|called|for)\s+['\"]?([^'\".,]+)['\"]?", re.IGNORECASE),
@@ -147,6 +181,16 @@ def map_query_to_actions(query: str, context: dict[str, Any] | None = None) -> l
            }
        )
    # --- Plugin-contributed intents (Block H / HC-A) ---
    for pattern, handler in _CONTRIBUTED_INTENTS:
        try:
            if pattern.search(q):
                contributed = handler(query, context)
                if contributed:
                    actions.extend(contributed)
        except Exception:
            logger.warning("Contributed intent handler failed", exc_info=True)
    # --- Generic fallback ---
    if not actions:
        if _PATTERNS["help"].search(q):
@@ -0,0 +1,658 @@
 """Core ReAct (Reasoning + Acting) loop for AI agents.
 Implements a true ReAct loop that alternates between LLM reasoning and tool
 execution. Each step records the thought (LLM content), action (tool name),
 action_input (tool arguments), and observation (tool result).
 The loop terminates when:
 - The LLM returns a final response without tool calls (completed)
 - max_steps is reached (stopped_max_steps)
 - timeout is exceeded (stopped_timeout)
 - A permanent error occurs (stopped_error)
 Error handling uses ``ErrorCategory`` from ``app.core.error_codes``:
 - TRANSIENT → retry the LLM call (up to 3 retries per step)
 - PERMANENT → stop the loop immediately
 - PARTIAL → continue with partial results
 Usage::
    from app.ai.agent_loop import run_react_loop
    result = await run_react_loop(
        agent_definition=agent,
        messages=[{"role": "user", "content": "Summarize recent emails"}],
        tools=tool_schemas,
        tool_registry=registry,
        db=db_session,
        tenant_id=tenant_id,
        user_id=user_id,
    )
    print(result.final_content, result.total_cost_usd, result.steps_taken)
 """
 from __future__ import annotations
 import asyncio
 import json
 import logging
 import time
 import uuid
 from dataclasses import dataclass, field
 from datetime import UTC, datetime
 from typing import TYPE_CHECKING, Any
 from app.ai.llm_client import llm_complete
 from app.core.error_codes import ErrorCategory, classify_exception
 if TYPE_CHECKING:
    from collections.abc import Callable
    from sqlalchemy.ext.asyncio import AsyncSession
    from app.ai.tool_registry import ToolRegistry
 logger = logging.getLogger(__name__)
 # Maximum retries for transient errors per LLM step
 _MAX_TRANSIENT_RETRIES = 3
 # ──────────────────────────────────────────────────────────────────────────
 # Data structures
 # ──────────────────────────────────────────────────────────────────────────
@dataclass
 class ReActStep:
    """A single step in the ReAct loop (Thought → Action → Observation)."""
    step_number: int
    thought: str  # LLM content before tool calls
    action: str | None  # Tool name (None if final response)
    action_input: dict[str, Any] | None  # Tool arguments
    observation: str | None  # Tool result
    cost_usd: float
    timestamp: str  # ISO format
@dataclass
 class ReActResult:
    """Final result of the ReAct loop."""
    final_content: str
    steps: list[ReActStep] = field(default_factory=list)
    total_cost_usd: float = 0.0
    steps_taken: int = 0
    status: str = "completed"  # completed | stopped_max_steps | stopped_timeout | stopped_error
    error: str | None = None
 # ──────────────────────────────────────────────────────────────────────────
 # Core loop
 # ──────────────────────────────────────────────────────────────────────────
 def _extract_tool_calls(raw_response: Any) -> list[dict[str, Any]]:
    """Extract tool calls from a LiteLLM raw response.
    Returns a list of dicts with keys: ``id``, ``name``, ``arguments``.
    """
    tool_calls: list[dict[str, Any]] = []
    try:
        msg = raw_response.choices[0].message
        if hasattr(msg, "tool_calls") and msg.tool_calls:
            for tc in msg.tool_calls:
                tool_calls.append({
                    "id": tc.id or "",
                    "name": tc.function.name if tc.function else "",
                    "arguments": tc.function.arguments if tc.function and tc.function.arguments else "{}",
                })
    except (AttributeError, IndexError, TypeError) as exc:
        logger.debug("Failed to extract tool calls from response: %s", exc)
    return tool_calls
 async def _execute_tool(
    tool_registry: ToolRegistry,
    tool_name: str,
    arguments: dict[str, Any],
    context: dict[str, Any],
 ) -> str:
    """Execute a single tool call via the registry.
    Returns the tool result as a string, or an error message.
    Note: access control (allowlist + required_permission) is enforced by
    ``_check_tool_access`` in ``run_react_loop`` BEFORE any execution path
    (dry-run, approval, execute) is reached.
    """
    tool = tool_registry.get(tool_name)
    if tool is None:
        return f"Error: Tool '{tool_name}' not found"
    try:
        result = await tool.handler(arguments=arguments, context=context)
        return result if isinstance(result, str) else json.dumps(result)
    except Exception as exc:
        logger.exception("Tool '%s' execution failed", tool_name)
        return f"Error: {exc}"
 def _filter_observation(observation: str) -> str:
    """F14 (Astra P1): sanitize a tool observation before it re-enters
    the LLM conversation.
    Tool responses are raw data (CRM records, mail payloads, settings) and
    may contain sensitive fields (smtp_password, api keys, ...). The data
    policy runs BEFORE the loop — observations arise INSIDE it and used to
    reach the provider verbatim. Parse JSON observations and strip
    sensitive fields (same SENSITIVE_FIELDS set as the data policy).
    """
    stripped = observation.strip()
    if not stripped.startswith(("{", "[")):
        return observation
    try:
        import json
        parsed = json.loads(stripped)
    except (json.JSONDecodeError, ValueError):
        return observation
    from app.core.sensitive_data import SENSITIVE_FIELDS
    sensitive_names: set[str] = set()
    for fields in SENSITIVE_FIELDS.values():
        sensitive_names |= fields
    def _strip(data: Any) -> Any:
        if isinstance(data, dict):
            return {
                k: _strip(v)
                for k, v in data.items()
                if k not in sensitive_names
            }
        if isinstance(data, list):
            return [_strip(v) for v in data]
        return data
    import json
    return json.dumps(_strip(parsed))
 def _extract_allowed_tool_names(tools: list[dict[str, Any]] | None) -> set[str]:
    """Extract the tool names actually offered to the LLM.
    Always returns a set (possibly empty) — empty means no tools were
    offered, so the caller fails closed on ANY tool call.
    """
    if not tools:
        return set()
    names: set[str] = set()
    for t in tools:
        fn = (t or {}).get("function") or {}
        name = fn.get("name")
        if name:
            names.add(str(name))
    return names
 def _normalize_permissions(ctx: dict[str, Any] | None) -> dict[str, Any]:
    """Normalize a user/agent context into the resolved-permissions shape
    expected by ``check_permission`` (permissions / denied / is_system_admin).
    Session user contexts carry ``denied_permissions`` while resolved
    permission dicts use ``denied`` — both are accepted here.
    """
    if not isinstance(ctx, dict):
        return {"permissions": [], "denied": [], "is_system_admin": False}
    return {
        "permissions": list(ctx.get("permissions", []) or []),
        "denied": list(ctx.get("denied", ctx.get("denied_permissions", [])) or []),
        "is_system_admin": bool(ctx.get("is_system_admin", False)),
    }
 def _check_tool_access(
    tool_registry: ToolRegistry,
    tool_name: str,
    allowed_tools: set[str] | None,
    user_permissions: dict[str, Any] | None,
 ) -> str | None:
    """F01 guard: enforce allowlist + required_permission before execution.
    Must run before EVERY execution path (dry-run, approval, execute).
    Returns None when access is granted, otherwise an error observation.
    Checks (fail-closed):
    1. Allowlist — the tool must be among the schemas actually offered to
       the LLM. A hallucinated/injected tool name never reaches a handler.
    2. required_permission — when the tool declares one, the acting user's
       CURRENT permissions must grant it. Without a permission context the
       call is rejected (deny list first, system admin bypass).
    """
    tool = tool_registry.get(tool_name)
    if tool is None:
        return None  # "not found" is handled by _execute_tool
    # 1. Allowlist: only tools offered to the LLM may run.
    if allowed_tools is not None and tool_name not in allowed_tools:
        logger.warning(
            "F01 guard: tool '%s' is registered but NOT offered to this agent — rejected",
            tool_name,
        )
        return f"Error: Tool '{tool_name}' is not available to this agent"
    # 2. required_permission: enforce against the user's CURRENT permissions.
    required = getattr(tool, "required_permission", None)
    if isinstance(required, str) and required:
        resolved = _normalize_permissions(user_permissions)
        from app.core.permissions import check_permission
        if not check_permission(resolved, required):
            logger.warning(
                "F01 guard: tool '%s' requires '%s' which the acting user lacks — rejected",
                tool_name,
                required,
            )
            return (
                f"Error: Permission '{required}' required for tool '{tool_name}' "
                "and not granted to the acting user"
            )
    return None
 async def run_react_loop(
    agent_definition: Any,  # AgentDefinition from automation models
    messages: list[dict[str, Any]],
    tools: list[dict[str, Any]],
    tool_registry: ToolRegistry,
    db: AsyncSession,
    tenant_id: uuid.UUID,
    user_id: uuid.UUID,
    agent_run_id: uuid.UUID | None = None,
    max_steps: int = 20,
    timeout_seconds: int = 300,
    trace_id: str | None = None,
    on_step: Callable | None = None,
    dry_run: bool = False,
    require_approval: bool = False,
    approval_tools: list[str] | None = None,
    user_permissions: dict[str, Any] | None = None,
 ) -> ReActResult:
    """Execute a ReAct loop: LLM reasoning → tool execution → repeat.
    Args:
        agent_definition: AgentDefinition with llm_model, system_prompt, etc.
        messages: Initial chat messages (without system prompt).
        tools: OpenAI-format tool schemas for function calling.
        tool_registry: ToolRegistry instance for tool execution.
        db: Async DB session.
        tenant_id: Tenant ID for multi-tenancy.
        user_id: User ID for permission context.
        agent_run_id: Optional AgentRun ID for step persistence.
        max_steps: Maximum loop iterations (default 20).
        timeout_seconds: Overall timeout (default 300).
        trace_id: Optional trace ID for correlation.
        on_step: Optional async callback fired after each step.
        dry_run: When True, tool execution is simulated — tool handlers are
            NOT called. A mock result is returned instead and steps are still
            logged with real LLM cost.
    Returns:
        ReActResult with final content, steps, cost, and status.
    """
    from app.core.hooks import do_action
    result = ReActResult(final_content="", status="completed")
    start_time = time.monotonic()
    # Build LLM parameters from agent definition
    litellm_model = getattr(agent_definition, "llm_model", None) or "gpt-4o"
    system_prompt = getattr(agent_definition, "system_prompt", "") or "You are a helpful AI assistant."
    api_key = getattr(agent_definition, "api_key", None)
    api_base = getattr(agent_definition, "api_base", None)
    provider = getattr(agent_definition, "provider", None)
    max_tokens = getattr(agent_definition, "max_tokens", None) or 1000
    # Build the full message list with system prompt prepended
    full_messages: list[dict[str, Any]] = [
        {"role": "system", "content": system_prompt},
        *messages,
    ]
    tool_context: dict[str, Any] = {
        "tenant_id": str(tenant_id),
        "user_id": str(user_id),
        "db": db,
        "agent_name": getattr(agent_definition, "name", "Agent"),
    }
    # F01 (Astra P0): allowlist — only tools actually offered to the LLM
    # may ever execute. Empty set = no tools offered = every call rejected.
    allowed_tool_names: set[str] = _extract_allowed_tool_names(tools)
    # Audit helper — records every tool call in the audit log.
    async def _audit_tool_call(
        step_number: int,
        tool_name: str,
        arguments: dict[str, Any],
        result: str,
        cost_usd: float,
    ) -> None:
        """Create an audit log entry for a single tool call."""
        try:
            from app.core.audit import log_audit
            await log_audit(
                db=db,
                tenant_id=tenant_id,
                user_id=user_id,
                action="agent.tool_call",
                entity_type="agent_run",
                entity_id=agent_run_id,
                details={
                    "agent_run_id": str(agent_run_id) if agent_run_id else None,
                    "step_number": step_number,
                    "tool_name": tool_name,
                    "arguments": arguments,
                    "result": result[:2000],
                    "cost_usd": cost_usd,
                    "dry_run": dry_run,
                },
            )
        except Exception:
            logger.exception("Failed to audit tool call '%s'", tool_name)
    for step_num in range(1, max_steps + 1):
        # ── Timeout check ──
        elapsed = time.monotonic() - start_time
        if elapsed >= timeout_seconds:
            result.status = "stopped_timeout"
            result.error = f"Timeout after {elapsed:.1f}s (limit {timeout_seconds}s)"
            logger.warning("ReAct loop timed out at step %d: %s", step_num, result.error)
            break
        # ── LLM call with transient retry ──
        llm_result: dict[str, Any] | None = None
        last_error: str | None = None
        for retry in range(_MAX_TRANSIENT_RETRIES + 1):
            try:
                llm_result = await llm_complete(
                    model=litellm_model,
                    messages=full_messages,
                    tools=tools if tools else None,
                    temperature=0.3,
                    max_tokens=max_tokens,
                    api_key=api_key,
                    api_base=api_base,
                    provider=provider,
                    trace_id=trace_id,
                    tenant_id=tenant_id,
                    db=db,
                )
                break
            except Exception as exc:
                last_error = str(exc)
                category = classify_exception(exc)
                if category == ErrorCategory.PERMANENT:
                    result.status = "stopped_error"
                    result.error = f"Permanent error at step {step_num}: {exc}"
                    logger.error("ReAct loop permanent error: %s", result.error)
                    return result
                if category == ErrorCategory.TRANSIENT and retry < _MAX_TRANSIENT_RETRIES:
                    backoff = 2 ** retry
                    logger.warning(
                        "Transient error at step %d (retry %d/%d): %s — retrying in %ds",
                        step_num, retry + 1, _MAX_TRANSIENT_RETRIES, exc, backoff,
                    )
                    await asyncio.sleep(backoff)
                    continue
                # PARTIAL or exhausted retries
                if category == ErrorCategory.PARTIAL:
                    logger.warning("Partial error at step %d: %s — continuing", step_num, exc)
                    last_error = str(exc)
                    break
                # Exhausted transient retries
                result.status = "stopped_error"
                result.error = f"Error after {retry + 1} retries at step {step_num}: {exc}"
                logger.error("ReAct loop error: %s", result.error)
                return result
        if llm_result is None:
            result.status = "stopped_error"
            result.error = f"LLM call failed at step {step_num}: {last_error}"
            return result
        # ── Extract response data ──
        content = llm_result.get("content", "")
        cost_usd = llm_result.get("cost_usd", 0.0)
        result.total_cost_usd += cost_usd
        tool_calls = _extract_tool_calls(llm_result.get("raw_response"))
        # ── No tool calls → final response ──
        if not tool_calls:
            step = ReActStep(
                step_number=step_num,
                thought=content,
                action=None,
                action_input=None,
                observation=None,
                cost_usd=cost_usd,
                timestamp=datetime.now(UTC).isoformat(),
            )
            result.steps.append(step)
            result.final_content = content
            result.steps_taken = step_num
            # Fire hook
            await do_action(
                "agent.step",
                agent_id=str(getattr(agent_definition, "id", "")),
                step_number=step_num,
                thought=content,
                action=None,
                observation=None,
                cost_usd=cost_usd,
                agent_run_id=str(agent_run_id) if agent_run_id else None,
                trace_id=trace_id,
            )
            # Callback
            if on_step:
                try:
                    await on_step(step)
                except Exception:
                    logger.debug("on_step callback failed", exc_info=True)
            break
        # ── Execute tool calls ──
        # Append assistant message with tool calls to conversation
        full_messages.append({
            "role": "assistant",
            "content": content,
            "tool_calls": [
                {
                    "id": tc["id"],
                    "type": "function",
                    "function": {"name": tc["name"], "arguments": tc["arguments"]},
                }
                for tc in tool_calls
            ],
        })
        # Execute each tool call and collect observations
        observations: list[str] = []
        for tc in tool_calls:
            tool_name = tc["name"]
            try:
                args = json.loads(tc["arguments"]) if tc["arguments"] else {}
            except json.JSONDecodeError:
                args = {}
                logger.warning("Invalid JSON arguments for tool '%s': %s", tool_name, tc["arguments"])
            # F01 (Astra P0): enforce allowlist + required_permission before
            # every execution path (dry-run, approval, execute). Fail-closed:
            # a hallucinated or injected tool name never reaches a handler.
            guard_error = _check_tool_access(
                tool_registry, tool_name, allowed_tool_names, user_permissions
            )
            if guard_error is not None:
                observation = guard_error
            elif dry_run:
                observation = json.dumps(
                    {
                        "dry_run": True,
                        "would_execute": tool_name,
                        "arguments": args,
                    }
                )
            elif require_approval and (approval_tools is None or tool_name in (approval_tools or [])):
                # I-APPR-LOOP: Human-in-the-Loop Approval
                # Create an ApprovalRequest and pause the loop
                try:
                    from app.core.approval import create_approval_request
                    pass  # agent_workstream removed
                    approval = await create_approval_request(
                        db=db,
                        tenant_id=tenant_id,
                        entity_type="agent_run",
                        entity_id=agent_run_id or uuid.uuid4(),
                        action=f"tool:{tool_name}",
                        requested_by=user_id,
                        requested_by_type="agent",
                    )
                    # Post approval request to Communication (I-WORK-HANDOFF)
                    if agent_run_id:
                        try:
                            from app.plugins.builtins.contracts import get_contract_registry
                            komm = get_contract_registry().get_contract("kommunikation")
                            if komm:
                                agent_id = getattr(agent_definition, "id", uuid.uuid4())
                                room_title = f"Agent: {getattr(agent_definition, 'name', 'Agent')}"
                                conv_id = await komm.find_locked_room_id(
                                    db=db,
                                    tenant_id=tenant_id,
                                    plugin_name="automation",
                                    title=room_title,
                                )
                                if conv_id:
                                    await komm.send_message(
                                        db=db,
                                        tenant_id=tenant_id,
                                        conversation_id=conv_id,
                                        sender_id=agent_id,
                                        sender_type="agent",
                                        content=f"Approval required for tool '{tool_name}'",
                                        content_format="text",
                                        blocks=[
                                            {
                                                "block_type": "approval_request",
                                                "block_data": {
                                                    "title": f"Approval: {tool_name}",
                                                    "description": f"Agent wants to execute tool '{tool_name}' with arguments: {json.dumps(args)[:300]}",
                                                    "approval_id": str(approval.id),
                                                    "status": "pending",
                                                },
                                                "sort_order": 0,
                                            }
                                        ],
                                        metadata={"approval_id": str(approval.id), "agent_run_id": str(agent_run_id)},
                                    )
                        except Exception:
                            logger.warning("Failed to post approval request to communication", exc_info=True)
                    # Pause the loop — return with waiting_for_approval status
                    result.status = "waiting_for_approval"
                    result.error = f"Tool '{tool_name}' requires human approval (request_id: {approval.id})"
                    result.steps_taken = step_num
                    result.final_content = f"I need approval to execute tool '{tool_name}'. Approval request {approval.id} has been created."
                    logger.info("Agent loop paused for approval on tool '%s' (request: %s)", tool_name, approval.id)
                    return result
                except Exception as e:
                    logger.warning("Failed to create approval request for tool '%s': %s", tool_name, e)
                    observation = json.dumps({"error": f"Approval required but failed to create request: {e}"})
            else:
                observation = await _execute_tool(tool_registry, tool_name, args, tool_context)
            # F14 (Astra P1): tool responses are raw data — sanitize the
            # observation before it re-enters the LLM conversation.
            observation = _filter_observation(observation)
            observations.append(observation)
            # Audit every tool call (real or simulated)
            await _audit_tool_call(
                step_number=step_num,
                tool_name=tool_name,
                arguments=args,
                result=observation,
                cost_usd=cost_usd / len(tool_calls) if tool_calls else cost_usd,
            )
            # Feed tool result back into conversation
            full_messages.append({
                "role": "tool",
                "tool_call_id": tc["id"],
                "content": observation,
            })
            # Record step
            step = ReActStep(
                step_number=step_num,
                thought=content,
                action=tool_name,
                action_input=args,
                observation=observation,
                cost_usd=cost_usd / len(tool_calls) if tool_calls else cost_usd,
                timestamp=datetime.now(UTC).isoformat(),
            )
            result.steps.append(step)
            # Fire hook
            await do_action(
                "agent.step",
                agent_id=str(getattr(agent_definition, "id", "")),
                step_number=step_num,
                thought=content,
                action=tool_name,
                observation=observation,
                cost_usd=cost_usd,
                agent_run_id=str(agent_run_id) if agent_run_id else None,
                trace_id=trace_id,
            )
            # Callback
            if on_step:
                try:
                    await on_step(step)
                except Exception:
                    logger.debug("on_step callback failed", exc_info=True)
        result.steps_taken = step_num
        # If this was the last allowed step, stop gracefully
        if step_num >= max_steps:
            result.status = "stopped_max_steps"
            result.error = f"Reached max_steps limit ({max_steps})"
            result.final_content = content
            logger.warning("ReAct loop stopped at max_steps=%d", max_steps)
            break
    # If loop completed without a final response (e.g. all steps had tool calls)
    if not result.final_content and result.steps:
        result.final_content = result.steps[-1].thought or ""
    if result.status == "completed" and not result.final_content:
        result.final_content = ""
    return result
@@ -0,0 +1,230 @@
 """Agent permission context resolution for AI agents.
 Resolves the effective permissions available to an agent run as the
 intersection of the user's (or run-as user's) RBAC permissions, the agent's
 configured tools/skills, and the tools each skill is allowed to use.
 Effective = User/Run-as ∩ Agent ∩ Skill ∩ Tool
 Key principles:
 - Skills orchestrate tools but NEVER grant additional permissions.
 - Every tool/service call re-checks permissions — rights are NOT frozen
  for a run.
 - System admins get all tools.
 """
 from __future__ import annotations
 import logging
 import uuid
 from dataclasses import dataclass
 from typing import Any
 from sqlalchemy import select
 from sqlalchemy.ext.asyncio import AsyncSession
 from app.core.error_codes import ApiError
 from app.core.permissions import check_permission, resolve_permissions
 logger = logging.getLogger(__name__)
@dataclass
 class AgentPermissionContext:
    """Effective permission context for a single agent run."""
    user_id: uuid.UUID
    tenant_id: uuid.UUID
    run_as_user_id: uuid.UUID | None
    user_permissions: dict[str, Any]  # RBAC permissions from Role
    agent_tool_ids: list[str]
    agent_skill_ids: list[str]
    effective_tool_ids: list[str]  # After intersection
    is_system_admin: bool = False
    def has_permission(self, permission: str) -> bool:
        """Check whether the run-as user has the given RBAC permission."""
        return check_permission(self.user_permissions, permission)
    def can_use_tool(self, tool_id: str) -> bool:
        """Check whether the agent may call the given tool."""
        return tool_id in self.effective_tool_ids
 def _resolve_effective_tool_ids(
    agent_definition: Any,
    user_permissions: dict[str, Any],
 ) -> list[str]:
    """Compute the effective tool IDs after User ∩ Agent ∩ Skill ∩ Tool.
    Mirrors the semantics of ``app.ai.agent_tools.get_agent_tools``: skills
    orchestrate tools but never grant additional permissions.
    """
    from app.ai.skill_registry import get_skill_registry
    from app.ai.tool_registry import get_tool_registry
    agent_tool_ids: list[str] = list(getattr(agent_definition, "tool_ids", None) or [])
    agent_skill_ids: list[str] = list(getattr(agent_definition, "skill_ids", None) or [])
    skill_registry = get_skill_registry()
    skills = skill_registry.get_by_names(agent_skill_ids)
    direct_tool_ids = set(agent_tool_ids)
    skill_tool_ids: set[str] = set()
    for skill in skills:
        skill_tool_ids.update(skill.allowed_tool_ids or [])
    # Tools directly on the agent, plus tools reachable via skills that are
    # also directly on the agent (skills never widen the agent's tool set).
    available_tool_ids = direct_tool_ids | (direct_tool_ids & skill_tool_ids)
    tool_registry = get_tool_registry()
    tools = tool_registry.get_by_names(sorted(available_tool_ids))
    permitted = [
        tool
        for tool in tools
        if not getattr(tool, "required_permission", None)
        or check_permission(user_permissions, tool.required_permission)
    ]
    return [tool.name for tool in permitted]
 async def resolve_agent_permissions(
    db: AsyncSession,
    tenant_id: uuid.UUID,
    user_id: uuid.UUID,
    agent_definition: Any,
    run_as_user_id: uuid.UUID | None = None,
 ) -> AgentPermissionContext:
    """Resolve effective permissions for an agent run.
    Effective = User/Run-as ∩ Agent ∩ Skill ∩ Tool.
    Args:
        db: Async DB session.
        tenant_id: Tenant ID for multi-tenancy.
        user_id: The user requesting the run (permission source).
        agent_definition: AgentDefinition with tool_ids and skill_ids.
        run_as_user_id: Optional user the agent runs as. When provided, the
            run-as user's permissions are used instead of the requester's.
    Returns:
        AgentPermissionContext with the resolved effective tool IDs.
    """
    effective_user_id = run_as_user_id or user_id
    user_permissions = await resolve_permissions(db, effective_user_id, tenant_id)
    is_system_admin = bool(user_permissions.get("is_system_admin", False))
    agent_tool_ids: list[str] = list(getattr(agent_definition, "tool_ids", None) or [])
    agent_skill_ids: list[str] = list(getattr(agent_definition, "skill_ids", None) or [])
    if is_system_admin:
        effective_tool_ids = list(agent_tool_ids)
    else:
        effective_tool_ids = _resolve_effective_tool_ids(agent_definition, user_permissions)
    return AgentPermissionContext(
        user_id=user_id,
        tenant_id=tenant_id,
        run_as_user_id=run_as_user_id,
        user_permissions=user_permissions,
        agent_tool_ids=agent_tool_ids,
        agent_skill_ids=agent_skill_ids,
        effective_tool_ids=effective_tool_ids,
        is_system_admin=is_system_admin,
    )
 async def check_entity_lock(
    db: AsyncSession,
    entity_type: str,
    entity_id: uuid.UUID,
    expected_version: int,
 ) -> bool:
    """Optimistic-lock check: raise ApiError('conflict') on version mismatch.
    Loads the entity's ``version`` column. If the current version differs from
    ``expected_version``, raises ``ApiError`` with code ``conflict``. Models
    without a ``version`` column are treated as unlocked (no-op).
    Returns True when the lock check passes.
    """
    from app.services.entity_permission_service import ENTITY_MODELS
    model = ENTITY_MODELS.get(entity_type)
    if model is None or not hasattr(model, "version"):
        return True
    result = await db.execute(select(model.version).where(model.id == entity_id))
    current_version = result.scalar_one_or_none()
    if current_version is None:
        raise ApiError(code="not_found", detail=f"{entity_type} not found")
    if int(current_version) != int(expected_version):
        raise ApiError(
            code="conflict",
            detail=(
                f"{entity_type} {entity_id} was modified concurrently "
                f"(expected version {expected_version}, current {current_version})"
            ),
        )
    return True
 async def filter_visible_agents(
    db: AsyncSession,
    tenant_id: uuid.UUID,
    user_id: uuid.UUID,
    agents: list,
 ) -> list:
    """Filter agents visible to a user based on agents:read + EntityPermission.
    A user sees an agent when they have the ``agents:read`` permission AND the
    agent is visible via ownership, tenant-ownership, or entity_permissions.
    System admins see all agents.
    """
    user_permissions = await resolve_permissions(db, user_id, tenant_id)
    if user_permissions.get("is_system_admin"):
        return list(agents)
    if not check_permission(user_permissions, "agents:read"):
        return []
    from app.services.permission_resolver import get_visible_ids
    visible_ids, _ = await get_visible_ids(db, tenant_id, user_id, "agent_definition")
    return [a for a in agents if a.id in visible_ids]
 async def check_agent_execute_permission(
    db: AsyncSession,
    tenant_id: uuid.UUID,
    user_id: uuid.UUID,
    agent_id: uuid.UUID,
 ) -> bool:
    """Check if a user has agents:execute permission for a specific agent.
    Requires the ``agents:execute`` RBAC permission AND entity-level access
    (owner, tenant-owned, or shared via entity_permissions). System admins
    always pass.
    """
    user_permissions = await resolve_permissions(db, user_id, tenant_id)
    if user_permissions.get("is_system_admin"):
        return True
    if not check_permission(user_permissions, "agents:execute"):
        return False
    from app.services.permission_resolver import check_entity_access
    return await check_entity_access(
        db, tenant_id, user_id, "agent_definition", agent_id, required_level="read"
    )
 __all__ = [
    "AgentPermissionContext",
    "resolve_agent_permissions",
    "check_entity_lock",
    "filter_visible_agents",
    "check_agent_execute_permission",
 ]
@@ -0,0 +1,158 @@
 """SSE streaming for the ReAct agent loop.
 Wraps ``run_react_loop`` from ``app.ai.agent_loop`` and emits Server-Sent
 Events (SSE) for each step, plus a final ``done`` or ``error`` event.
 Events emitted:
 - ``event: step``   — JSON {step_number, thought, action, action_input, observation, cost_usd}
 - ``event: status`` — JSON {status: "running", step: N}
 - ``event: done``   — JSON {status, total_cost, steps_taken, final_content}
 - ``event: error``  — JSON {error, trace_id}
 Trace modes:
 - ``standard`` — step events include action + result only (no thought)
 - ``extended`` — step events also include the thought/reasoning
 """
 from __future__ import annotations
 import asyncio
 import json
 import logging
 import uuid
 from collections.abc import AsyncGenerator
 from typing import TYPE_CHECKING, Any
 from app.ai.agent_loop import ReActStep, run_react_loop
 if TYPE_CHECKING:
    from sqlalchemy.ext.asyncio import AsyncSession
 logger = logging.getLogger(__name__)
 # ──────────────────────────────────────────────────────────────────────────
 # SSE helpers
 # ──────────────────────────────────────────────────────────────────────────
 def _sse(event: str, data: dict[str, Any]) -> str:
    """Format a single SSE event as ``event: <name>\ndata: <json>\n\n``."""
    return f"event: {event}\ndata: {json.dumps(data, ensure_ascii=False)}\n\n"
 def _step_event(step: ReActStep, trace_mode: str) -> str:
    """Build the SSE ``step`` event for a ReAct step."""
    data: dict[str, Any] = {
        "step_number": step.step_number,
        "action": step.action,
        "action_input": step.action_input,
        "observation": step.observation,
        "cost_usd": step.cost_usd,
    }
    if trace_mode == "extended":
        data["thought"] = step.thought
    return _sse("step", data)
 # ──────────────────────────────────────────────────────────────────────────
 # Streaming loop
 # ──────────────────────────────────────────────────────────────────────────
 async def stream_react_loop(
    agent_definition: Any,
    user_message: str,
    tools: list[dict],
    tool_registry: Any,
    db: AsyncSession | None,
    tenant_id: uuid.UUID,
    user_id: uuid.UUID,
    agent_run_id: uuid.UUID | None = None,
    max_steps: int = 20,
    timeout_seconds: int = 300,
    trace_id: str | None = None,
    user_permissions: dict[str, Any] | None = None,
 ) -> AsyncGenerator[str, None]:
    """Run the ReAct loop and yield SSE-formatted events.
    Args:
        agent_definition: AgentDefinition with llm_model, system_prompt, etc.
        user_message: The user's message to the agent.
        tools: OpenAI-format tool schemas for function calling.
        tool_registry: ToolRegistry instance for tool execution.
        db: Async DB session.
        tenant_id: Tenant ID for multi-tenancy.
        user_id: User ID for permission context.
        agent_run_id: Optional AgentRun ID for step persistence.
        max_steps: Maximum loop iterations (default 20).
        timeout_seconds: Overall timeout (default 300).
        trace_id: Optional trace ID for correlation.
    Yields:
        SSE-formatted event strings.
    """
    trace_mode = getattr(agent_definition, "trace_mode", "standard") or "standard"
    queue: asyncio.Queue[str | None] = asyncio.Queue()
    async def on_step(step: ReActStep) -> None:
        """Push step + status events into the queue."""
        await queue.put(_step_event(step, trace_mode))
        await queue.put(
            _sse("status", {"status": "running", "step": step.step_number})
        )
    async def _producer() -> None:
        """Run the loop and push the final done/error event."""
        try:
            result = await run_react_loop(
                agent_definition=agent_definition,
                messages=[{"role": "user", "content": user_message}],
                tools=tools,
                tool_registry=tool_registry,
                db=db,
                tenant_id=tenant_id,
                user_id=user_id,
                agent_run_id=agent_run_id,
                max_steps=max_steps,
                timeout_seconds=timeout_seconds,
                trace_id=trace_id,
                on_step=on_step,
                user_permissions=user_permissions,  # F01: enforce at execution time
            )
            await queue.put(
                _sse(
                    "done",
                    {
                        "status": result.status,
                        "total_cost": result.total_cost_usd,
                        "steps_taken": result.steps_taken,
                        "final_content": result.final_content,
                    },
                )
            )
        except Exception as exc:  # noqa: BLE001 — stream must not crash the consumer
            logger.exception("ReAct streaming loop failed")
            await queue.put(
                _sse("error", {"error": str(exc), "trace_id": trace_id})
            )
        finally:
            await queue.put(None)  # sentinel
    producer_task = asyncio.create_task(_producer())
    try:
        while True:
            event = await queue.get()
            if event is None:
                break
            yield event
    finally:
        if not producer_task.done():
            producer_task.cancel()
            try:
                await producer_task
            except asyncio.CancelledError:
                pass
 __all__ = ["stream_react_loop"]
@@ -0,0 +1,117 @@
 """Tool-/Skill-Binding for AI agents.
 Resolves the effective capabilities available to an agent as the intersection
 of the user's permissions, the agent's configured tools/skills, and the tools
 that each skill is allowed to use.
 Key principle: Skills orchestrate tools but NEVER grant additional permissions.
 If a user does not have ``mail:read``, no skill can give them access to a
 mail-reading tool.
 """
 from __future__ import annotations
 import logging
 from typing import Any
 from app.ai.skill_registry import SkillDefinition, SkillRegistry
 logger = logging.getLogger(__name__)
 def _user_has_permission(
    user_permissions: dict[str, Any],
    required_permission: str | None,
 ) -> bool:
    """Check whether the user has the required permission for a tool.
    A tool without a required permission is always allowed. The check uses the
    same semantics as ``app.core.permissions.check_permission``: system admins
    pass, denied permissions block, and wildcards are supported.
    """
    if not required_permission:
        return True
    if user_permissions.get("is_system_admin"):
        return True
    denied = set(user_permissions.get("denied_permissions", []) or [])
    if any(_permission_matches(denied_perm, required_permission) for denied_perm in denied):
        return False
    granted = set(user_permissions.get("permissions", []) or [])
    return any(_permission_matches(granted_perm, required_permission) for granted_perm in granted)
 def _permission_matches(granted: str, required: str) -> bool:
    """Match a granted permission against a required one, supporting wildcards."""
    if granted == required:
        return True
    g_parts = granted.split(":")
    r_parts = required.split(":")
    if len(g_parts) != len(r_parts):
        return False
    for g_part, r_part in zip(g_parts, r_parts, strict=False):
        if g_part == "*":
            continue
        if g_part != r_part:
            return False
    return True
 def get_agent_tools(
    agent_definition: Any,
    tool_registry: Any,
    skill_registry: SkillRegistry,
    user_permissions: dict[str, Any],
 ) -> tuple[list[dict[str, Any]], list[SkillDefinition]]:
    """Get the tools and skills available to this agent.
    Effective capabilities = User/Run-as ∩ Agent ∩ Skill ∩ Tool.
    Args:
        agent_definition: AgentDefinition with ``tool_ids`` and ``skill_ids``.
        tool_registry: ToolRegistry with ``get_by_names`` and ``get``.
        skill_registry: SkillRegistry used to resolve skill names.
        user_permissions: Resolved permission dict (``permissions``,
            ``denied_permissions``, ``is_system_admin``).
    Returns:
        A tuple of (tool_schemas, skills). ``tool_schemas`` is the list of
        OpenAI-format tool schemas the agent may actually call. ``skills`` is
        the list of resolved SkillDefinitions the agent may use.
    """
    agent_tool_ids: list[str] = list(getattr(agent_definition, "tool_ids", None) or [])
    agent_skill_ids: list[str] = list(getattr(agent_definition, "skill_ids", None) or [])
    # 1. Resolve the agent's skills to SkillDefinitions.
    skills = skill_registry.get_by_names(agent_skill_ids)
    # 2. Collect the tool IDs available directly on the agent.
    direct_tool_ids = set(agent_tool_ids)
    # 3. For each skill, collect its allowed tool IDs.
    skill_tool_ids: set[str] = set()
    for skill in skills:
        skill_tool_ids.update(skill.allowed_tool_ids or [])
    # 4. Intersect: agent.tool_ids ∩ skill.allowed_tool_ids → tools via skills.
    #    Tools directly in agent.tool_ids (not via skills) are also available.
    available_tool_ids = direct_tool_ids | (direct_tool_ids & skill_tool_ids)
    # 5. Resolve the available tools from the registry.
    tools = tool_registry.get_by_names(sorted(available_tool_ids))
    # 6. Filter by user permissions: only tools where the user has the
    #    required permission. Skills never grant additional permissions.
    permitted_tools = [
        tool
        for tool in tools
        if _user_has_permission(user_permissions, getattr(tool, "required_permission", None))
    ]
    # 7. Build OpenAI-format schemas and return.
    tool_schemas = [tool.to_openai_schema() for tool in permitted_tools]
    return tool_schemas, skills
 __all__ = ["get_agent_tools"]
@@ -0,0 +1,156 @@
 """AI use-case metadata and validation.
 Defines the structured metadata that describes *why* and *how* an AI agent
 may process data. This is the governance contract for an agent definition:
 which data categories it may touch, which providers/models/actions are
 allowed, and whether human oversight is required.
 Used by:
 - ``app/ai/data_policy.py`` — runtime enforcement of allowed data categories
 - ``app/ai/oversight.py`` — human-review policy (``oversight_policy``)
 - ``app/plugins/builtins/automation/agent_routes.py`` — PATCH/GET endpoints
 """
 from __future__ import annotations
 from typing import Any
 from pydantic import BaseModel, Field
 # ──────────────────────────────────────────────────────────────────────────
 # Constants
 # ──────────────────────────────────────────────────────────────────────────
 # Known data categories an agent may declare it processes.
 KNOWN_DATA_CATEGORIES = (
    "contact_data",
    "email_content",
    "calendar",
    "tasks",
    "dms",
    "communication",
    "financial",
    "public",
 )
 # Valid oversight policies.
 OVERSIGHT_POLICIES = ("always_required", "on_high_risk", "never")
 # Valid risk classes.
 RISK_CLASSES = ("low", "medium", "high")
 # Valid allowed actions.
 KNOWN_ACTIONS = ("read", "summarize", "draft", "send", "create", "update", "delete")
 class AIUseCaseMetadata(BaseModel):
    """Structured metadata describing an AI agent's intended use case.
    Attributes:
        intended_purpose: Human-readable description of the use case.
        owner: User ID or email responsible for the use case.
        data_categories: Data categories the agent may process.
        allowed_providers: Provider IDs the agent may use (empty = any).
        allowed_models: Model names the agent may use (empty = any).
        allowed_actions: Actions the agent may perform (empty = any).
        oversight_policy: When human review is required.
        risk_class: Risk classification of the use case.
        human_review_required: Whether a human must review outputs.
    """
    intended_purpose: str = Field(default="", max_length=1000)
    owner: str = Field(default="", max_length=255)
    data_categories: list[str] = Field(default_factory=list)
    allowed_providers: list[str] = Field(default_factory=list)
    allowed_models: list[str] = Field(default_factory=list)
    allowed_actions: list[str] = Field(default_factory=list)
    oversight_policy: str = Field(default="never")
    risk_class: str = Field(default="low")
    human_review_required: bool = False
    @classmethod
    def from_dict(cls, data: dict[str, Any] | None) -> "AIUseCaseMetadata":
        """Build metadata from a raw dict (e.g. the agent's JSONB column)."""
        if not data:
            return cls()
        # Only pass known fields so unknown keys don't break validation.
        known = {k: v for k, v in data.items() if k in cls.model_fields}
        return cls(**known)
    def to_dict(self) -> dict[str, Any]:
        """Serialize to a plain dict for JSONB storage."""
        return self.model_dump()
 # ──────────────────────────────────────────────────────────────────────────
 # Validation
 # ──────────────────────────────────────────────────────────────────────────
 def validate_ai_use_case(metadata: AIUseCaseMetadata, agent_definition: Any) -> list[str]:
    """Validate metadata against an agent configuration.
    Returns a list of human-readable warnings. An empty list means the
    metadata is consistent with the agent definition.
    Checks performed:
    - ``intended_purpose`` and ``owner`` are set.
    - ``data_categories`` are known values.
    - ``oversight_policy`` and ``risk_class`` are valid.
    - ``allowed_models`` (if non-empty) include the agent's configured model.
    - ``allowed_providers`` (if non-empty) include the agent's provider.
    - ``human_review_required`` is consistent with ``oversight_policy``.
    """
    warnings: list[str] = []
    if not metadata.intended_purpose.strip():
        warnings.append("intended_purpose is empty — describe the AI use case")
    if not metadata.owner.strip():
        warnings.append("owner is empty — set a responsible user or email")
    for cat in metadata.data_categories:
        if cat not in KNOWN_DATA_CATEGORIES:
            warnings.append(f"data_category '{cat}' is not a known category")
    if metadata.oversight_policy not in OVERSIGHT_POLICIES:
        warnings.append(
            f"oversight_policy '{metadata.oversight_policy}' is invalid "
            f"(expected one of {OVERSIGHT_POLICIES})"
        )
    if metadata.risk_class not in RISK_CLASSES:
        warnings.append(
            f"risk_class '{metadata.risk_class}' is invalid "
            f"(expected one of {RISK_CLASSES})"
        )
    # Model / provider consistency (only if the agent pins allowed values).
    agent_model = getattr(agent_definition, "llm_model", None)
    if metadata.allowed_models and agent_model:
        # Strip provider prefix for comparison (e.g. "openai/gpt-4o" -> "gpt-4o").
        bare_model = agent_model.split("/", 1)[-1]
        if agent_model not in metadata.allowed_models and bare_model not in metadata.allowed_models:
            warnings.append(
                f"agent model '{agent_model}' is not in allowed_models {metadata.allowed_models}"
            )
    agent_provider = getattr(agent_definition, "provider", None)
    if metadata.allowed_providers and agent_provider:
        if agent_provider not in metadata.allowed_providers:
            warnings.append(
                f"agent provider '{agent_provider}' is not in allowed_providers "
                f"{metadata.allowed_providers}"
            )
    # Oversight consistency.
    if metadata.oversight_policy == "always_required" and not metadata.human_review_required:
        warnings.append(
            "oversight_policy is 'always_required' but human_review_required is False"
        )
    if metadata.oversight_policy == "never" and metadata.human_review_required:
        warnings.append(
            "oversight_policy is 'never' but human_review_required is True"
        )
    return warnings
@@ -0,0 +1,280 @@
 """Context builder — assembles the message list for an AI agent run.
 Builds the full chat context (system prompt + user message) for a ReAct agent
 from its ``AgentDefinition`` plus runtime context (user, tenant, memory,
 tools).  Sensitive fields are never included in the context — the builder
 respects ``SENSITIVE_FIELDS`` from ``app.core.sensitive_data``.
 Usage::
    from app.ai.context_builder import build_agent_context
    messages = await build_agent_context(
        agent_definition=agent,
        user_message="Summarize recent emails",
        db=db_session,
        tenant_id=tenant_id,
        user_id=user_id,
        memory_items=[{"content": "...", "metadata": {...}}],
    )
 """
 from __future__ import annotations
 import logging
 import uuid
 from typing import TYPE_CHECKING, Any
 if TYPE_CHECKING:
    from sqlalchemy.ext.asyncio import AsyncSession
 logger = logging.getLogger(__name__)
 # Default ReAct instruction prefix appended to the agent's system prompt.
 _REACT_PREFIX = (
    "You operate in a ReAct (Reasoning + Acting) loop. For each step you must "
    "produce a Thought, then an Action (a tool call), then observe the result "
    "and continue. When you have enough information to answer the user, stop "
    "calling tools and provide your final answer directly.\n\n"
    "Format:\n"
    "Thought: <your reasoning>\n"
    "Action: <tool name>\n"
    "Action Input: <JSON arguments>\n"
    "Observation: <tool result>\n"
    "... (repeat as needed) ...\n"
    "Final Answer: <your response to the user>\n"
 )
 class ReActSystemPromptBuilder:
    """Builds the ReAct system prompt for an agent definition.
    Sections:
    - Agent identity (name, description, capabilities)
    - Available tools (name + description only — schemas come via the tools param)
    - ReAct format instructions
    - Constraints (max_steps, budget, what the agent can/cannot do)
    """
    def __init__(
        self,
        agent_definition: Any,
        tool_descriptions: list[dict[str, str]] | None = None,
        max_steps: int | None = None,
        budget_limit_usd: float | None = None,
    ) -> None:
        self.agent_definition = agent_definition
        self.tool_descriptions = tool_descriptions or []
        self.max_steps = max_steps
        self.budget_limit_usd = budget_limit_usd
    def build(self) -> str:
        """Return the full system prompt string."""
        sections: list[str] = []
        # 1. Agent identity
        sections.append(self._identity_section())
        # 2. Available tools
        sections.append(self._tools_section())
        # 3. ReAct format instructions
        sections.append(_REACT_PREFIX)
        # 4. Constraints
        sections.append(self._constraints_section())
        # 5. Base system prompt from the agent definition
        base_prompt = getattr(self.agent_definition, "system_prompt", "") or ""
        if base_prompt:
            sections.append(base_prompt)
        return "\n\n".join(s for s in sections if s)
    def _identity_section(self) -> str:
        """Agent identity: name, description, capabilities."""
        name = getattr(self.agent_definition, "name", "") or "AI Agent"
        description = getattr(self.agent_definition, "description", "") or ""
        capabilities = getattr(self.agent_definition, "capabilities", None) or []
        lines = [f"You are {name}."]
        if description:
            lines.append(f"Description: {description}")
        if capabilities:
            caps = ", ".join(str(c) for c in capabilities)
            lines.append(f"Capabilities: {caps}")
        return "\n".join(lines)
    def _tools_section(self) -> str:
        """Available tools — name + description only (no full schema)."""
        if not self.tool_descriptions:
            return "You have no tools available. Answer from your own knowledge."
        lines = ["Available tools:"]
        for tool in self.tool_descriptions:
            name = tool.get("name", "")
            description = tool.get("description", "")
            if name:
                lines.append(f"- {name}: {description}")
        return "\n".join(lines)
    def _constraints_section(self) -> str:
        """Constraints: max_steps, budget, and behavioral limits."""
        constraints: list[str] = []
        max_steps = self.max_steps or getattr(
            self.agent_definition, "max_steps", None
        ) or 20
        constraints.append(f"- Maximum {max_steps} reasoning steps per run.")
        budget = self.budget_limit_usd
        if budget is None:
            budget = getattr(self.agent_definition, "budget_limit_usd", None)
        if budget is not None and budget > 0:
            constraints.append(f"- Budget limit: ${float(budget):.2f} per run.")
        constraints.append(
            "- Only call tools that are listed as available. Do not invent tools."
        )
        constraints.append(
            "- Never expose or request passwords, API keys, tokens, or other "
            "sensitive credentials."
        )
        constraints.append(
            "- Respect tenant data boundaries. Do not access data outside the "
            "current tenant."
        )
        return "Constraints:\n" + "\n".join(constraints)
 async def _load_user_context(
    db: AsyncSession | None,
    tenant_id: uuid.UUID,
    user_id: uuid.UUID,
    run_as_user_id: uuid.UUID | None,
 ) -> dict[str, Any]:
    """Load user + tenant context from the DB with graceful fallbacks."""
    context: dict[str, Any] = {
        "user_name": None,
        "tenant_name": None,
        "role": None,
    }
    if db is None:
        return context
    try:
        from sqlalchemy import select
        from app.models.tenant import Tenant
        from app.models.user import User, UserTenant
        effective_user_id = run_as_user_id or user_id
        user_result = await db.execute(
            select(User).where(User.id == effective_user_id).limit(1)
        )
        user = user_result.scalar_one_or_none()
        if user is not None:
            context["user_name"] = user.name or user.email
        tenant_result = await db.execute(
            select(Tenant).where(Tenant.id == tenant_id).limit(1)
        )
        tenant = tenant_result.scalar_one_or_none()
        if tenant is not None:
            context["tenant_name"] = tenant.name
        role_result = await db.execute(
            select(UserTenant.role)
            .where(UserTenant.user_id == effective_user_id)
            .where(UserTenant.tenant_id == tenant_id)
            .limit(1)
        )
        role = role_result.scalar_one_or_none()
        if role:
            context["role"] = role
    except Exception:
        logger.debug("Failed to load user/tenant context", exc_info=True)
    return context
 async def build_agent_context(
    agent_definition: Any,  # AgentDefinition from automation models
    user_message: str | None,
    db: AsyncSession | None,
    tenant_id: uuid.UUID,
    user_id: uuid.UUID,
    run_as_user_id: uuid.UUID | None = None,
    memory_items: list[dict] | None = None,
    trace_id: str | None = None,
 ) -> list[dict[str, Any]]:
    """Build the full message list for an agent run.
    Returns a list of chat messages: a system message (built by
    ``ReActSystemPromptBuilder``) followed by the user message.  Sensitive
    fields are redacted from any injected context.
    """
    # 1. Resolve the tools the agent has access to (filtered by tool_ids).
    tool_descriptions: list[dict[str, str]] = []
    tool_ids = list(getattr(agent_definition, "tool_ids", None) or [])
    try:
        from app.ai.tool_registry import get_tool_registry
        registry = get_tool_registry()
        if tool_ids:
            tools = registry.get_by_names(tool_ids)
        else:
            tools = registry.get_all()
        tool_descriptions = [
            {"name": t.name, "description": t.description} for t in tools
        ]
    except Exception:
        logger.debug("Failed to load tool descriptions", exc_info=True)
    # 2. Build the system prompt.
    builder = ReActSystemPromptBuilder(
        agent_definition=agent_definition,
        tool_descriptions=tool_descriptions,
    )
    system_prompt = builder.build()
    # 3. Load user/tenant context.
    user_ctx = await _load_user_context(
        db, tenant_id, user_id, run_as_user_id
    )
    # 4. Assemble the context block (redacting sensitive fields).
    context_lines: list[str] = []
    if user_ctx.get("user_name"):
        context_lines.append(f"Current user: {user_ctx['user_name']}")
    if user_ctx.get("tenant_name"):
        context_lines.append(f"Current tenant: {user_ctx['tenant_name']}")
    if user_ctx.get("role"):
        context_lines.append(f"Current user role: {user_ctx['role']}")
    if memory_items:
        context_lines.append("Relevant memory items:")
        for item in memory_items:
            content = item.get("content", "") if isinstance(item, dict) else str(item)
            if content:
                context_lines.append(f"- {content}")
    # 5. Build the final message list.
    messages: list[dict[str, Any]] = [
        {"role": "system", "content": system_prompt}
    ]
    if context_lines:
        context_block = "\n".join(context_lines)
        messages.append(
            {"role": "system", "content": f"Context:\n{context_block}"}
        )
    if user_message:
        messages.append({"role": "user", "content": user_message})
    return messages
 __all__ = ["ReActSystemPromptBuilder", "build_agent_context"]
@@ -0,0 +1,254 @@
 """Runtime provider / data policy enforcement for AI agents.
 Filters messages and context before they reach the LLM based on:
 - Sensitive fields (``app.core.sensitive_data.SENSITIVE_FIELDS``)
 - AI use-case metadata (allowed data categories)
 - Provider compliance (data residency / allowed data classes)
 This is the enforcement layer that guarantees an agent never sends data it
 is not permitted to process to a provider that is not approved for it.
 """
 from __future__ import annotations
 import logging
 import uuid
 from typing import Any
 from sqlalchemy.ext.asyncio import AsyncSession
 from app.ai.ai_use_case import AIUseCaseMetadata
 from app.core.sensitive_data import (
    SENSITIVE_FIELDS,
 )
 logger = logging.getLogger(__name__)
 # Data categories that map to entity types for sensitive-field filtering.
 _CATEGORY_ENTITY_MAP = {
    "contact_data": "contact",
    "email_content": "mail_account",
    "communication": "mail_account",
 }
 async def enforce_data_policy(
    db: AsyncSession,
    tenant_id: uuid.UUID,
    messages: list[dict[str, Any]],
    agent_definition: Any,
 ) -> list[dict[str, Any]]:
    """Filter messages/context based on the data policy.
    Steps:
    1. Remove sensitive fields from any dict content in the messages.
    2. Check AI use-case metadata for allowed data categories.
    3. Check provider compliance for data residency requirements.
    Args:
        db: Async DB session (may be ``None`` in tests / mock mode).
        tenant_id: Tenant ID for provider lookup.
        messages: The chat messages to filter.
        agent_definition: AgentDefinition with ``ai_use_case_metadata``.
    Returns:
        A new list of messages with disallowed data removed.
    """
    metadata = AIUseCaseMetadata.from_dict(
        getattr(agent_definition, "ai_use_case_metadata", None)
    )
    # Provider compliance (data residency / allowed data classes).
    compliance: dict[str, Any] | None = None
    if db is not None and tenant_id is not None:
        try:
            from app.ai.llm_client import get_provider_compliance
            compliance = await get_provider_compliance(db, tenant_id)
        except Exception:
            logger.debug("Failed to load provider compliance — skipping residency check")
    filtered: list[dict[str, Any]] = []
    for msg in messages:
        content = msg.get("content", "")
        if isinstance(content, dict):
            content = _filter_dict_content(
                content, metadata, compliance, agent_definition
            )
        elif isinstance(content, list):
            content = [
                _filter_dict_content(c, metadata, compliance, agent_definition)
                if isinstance(c, dict)
                else c
                for c in content
            ]
        elif isinstance(content, str):
            # F14 (Astra P1): JSON-serialized strings passed through
            # UNFILTERED before — a payload like '{"smtp_password": ...}'
            # reached the provider verbatim. Parse, filter, re-serialize;
            # non-JSON strings stay unchanged (plain prose is fine).
            content = _filter_json_string_content(
                content, metadata, compliance, agent_definition
            )
        new_msg = dict(msg)
        new_msg["content"] = content
        filtered.append(new_msg)
    return filtered
 def _filter_json_string_content(
    content: str,
    metadata: AIUseCaseMetadata,
    compliance: dict[str, Any] | None,
    agent_definition: Any,
 ) -> str:
    """Filter a JSON-serialized string payload (F14).
    Tries to parse the string as a JSON object/array and runs the SAME
    dict-level filtering on it. Non-JSON strings are returned unchanged.
    """
    stripped = content.strip()
    if not stripped.startswith(("{", "[")):
        return content
    try:
        import json
        parsed = json.loads(stripped)
    except (json.JSONDecodeError, ValueError):
        return content  # not JSON — plain string content is not a leak vector
    if isinstance(parsed, dict):
        filtered = _filter_dict_content(parsed, metadata, compliance, agent_definition)
        import json
        return json.dumps(filtered)
    if isinstance(parsed, list):
        filtered = [
            _filter_dict_content(item, metadata, compliance, agent_definition)
            if isinstance(item, dict)
            else item
            for item in parsed
        ]
        import json
        return json.dumps(filtered)
    return content
 def _filter_dict_content(
    data: dict[str, Any],
    metadata: AIUseCaseMetadata,
    compliance: dict[str, Any] | None,
    agent_definition: Any,
 ) -> dict[str, Any]:
    """Filter a single dict (entity payload) against the data policy."""
    # 1. Remove sensitive fields (always blocked from LLM context).
    result = _strip_sensitive_fields(data)
    # 2. Enforce allowed data categories from AI use-case metadata.
    if metadata.data_categories:
        result = _filter_by_allowed_categories(result, metadata.data_categories)
    # 3. Provider compliance — block fields whose data class the provider
    #    is not approved to process.
    if compliance is not None:
        result = _filter_by_provider_compliance(result, compliance)
    return result
 def _strip_sensitive_fields(data: dict[str, Any]) -> dict[str, Any]:
    """Recursively remove any key that matches a sensitive field name."""
    sensitive_names = set()
    for fields in SENSITIVE_FIELDS.values():
        sensitive_names |= fields
    result: dict[str, Any] = {}
    for key, value in data.items():
        if key in sensitive_names:
            continue
        if isinstance(value, dict):
            result[key] = _strip_sensitive_fields(value)
        elif isinstance(value, list):
            result[key] = [
                _strip_sensitive_fields(v) if isinstance(v, dict) else v
                for v in value
            ]
        else:
            result[key] = value
    return result
 def _filter_by_allowed_categories(
    data: dict[str, Any], allowed_categories: list[str]
 ) -> dict[str, Any]:
    """Remove entity-type payloads whose category is not allowed.
    Uses the category→entity mapping to decide whether a dict represents a
    disallowed entity type. Unknown dicts are kept (fail-open for generic
    context that has no clear entity type).
    """
    # Determine the entity type of this dict by checking for known keys.
    entity_type = _guess_entity_type(data)
    if entity_type is None:
        return data
    category = _entity_to_category(entity_type)
    if category is not None and category not in allowed_categories:
        return {}
    return data
 def _filter_by_provider_compliance(
    data: dict[str, Any], compliance: dict[str, Any]
 ) -> dict[str, Any]:
    """Remove fields whose data class the provider may not process."""
    allowed_classes = compliance.get("allowed_data_classes") or []
    if not allowed_classes:
        return data  # No restriction configured (fail-open).
    from app.core.sensitive_data import check_provider_compliance
    result: dict[str, Any] = {}
    for key, value in data.items():
        if isinstance(value, dict):
            result[key] = _filter_by_provider_compliance(value, compliance)
            continue
        # Determine data class for this field (best-effort).
        data_class = _guess_data_class(key, value)
        if check_provider_compliance(allowed_classes, data_class):
            result[key] = value
    return result
 def _guess_entity_type(data: dict[str, Any]) -> str | None:
    """Best-effort guess of the entity type from dict keys."""
    if any(k in data for k in ("email", "smtp_password", "imap_password")):
        return "mail_account"
    if any(k in data for k in ("first_name", "last_name", "company_id")):
        return "contact"
    if any(k in data for k in ("secret_key", "encryption_key")):
        return "system_settings"
    return None
 def _entity_to_category(entity_type: str) -> str | None:
    """Map an entity type to a data category."""
    for category, entity in _CATEGORY_ENTITY_MAP.items():
        if entity == entity_type:
            return category
    return None
 def _guess_data_class(key: str, value: Any) -> str:
    """Best-effort data class for a field (defaults to 'internal')."""
    # Sensitive field names are always critical.
    for fields in SENSITIVE_FIELDS.values():
        if key in fields:
            return "critical"
    # Heuristic: values that look like credentials/tokens are critical.
    if isinstance(value, str) and any(
        marker in key.lower() for marker in ("password", "token", "secret", "key")
    ):
        return "critical"
    return "internal"
@@ -0,0 +1,63 @@
 """Knowledge extraction — extract entities and relationships from content."""
 from __future__ import annotations
 import uuid
 from dataclasses import dataclass, field
 from typing import Any
 LOW_CONFIDENCE_THRESHOLD = 0.6
@dataclass
 class ExtractedEntity:
    """An entity extracted from content."""
    name: str
    entity_type: str
    confidence: float = 0.0
    mentions: list[str] = field(default_factory=list)
    metadata: dict[str, Any] = field(default_factory=dict)
@dataclass
 class ExtractedRelationship:
    """A relationship extracted from content."""
    source_entity: str
    source_type: str
    target_entity: str
    target_type: str
    relationship_type: str
    confidence: float = 0.0
    evidence: str = ""
@dataclass
 class ExtractionResult:
    """Result of a knowledge extraction operation."""
    source_type: str = ""
    source_id: str = ""
    tenant_id: str = ""
    entities: list[ExtractedEntity] = field(default_factory=list)
    relationships: list[ExtractedRelationship] = field(default_factory=list)
    overall_confidence: float = 0.0
 def is_low_confidence(score: float) -> bool:
    """Check if a confidence score is below the threshold."""
    return score < LOW_CONFIDENCE_THRESHOLD
 def filter_high_confidence(
    items: list[ExtractedRelationship], threshold: float = LOW_CONFIDENCE_THRESHOLD
 ) -> tuple[list[ExtractedRelationship], list[ExtractedRelationship]]:
    """Split items into (high, low) confidence lists."""
    high = [i for i in items if i.confidence >= threshold]
    low = [i for i in items if i.confidence < threshold]
    return high, low
 async def extract_knowledge(text: str, tenant_id: uuid.UUID) -> ExtractionResult:
    """Extract knowledge from text. Returns empty result for empty/short text."""
    if not text or len(text) < 10:
        return ExtractionResult(tenant_id=str(tenant_id))
    return ExtractionResult(tenant_id=str(tenant_id))
@@ -0,0 +1,55 @@
 """Knowledge lifecycle — manage retention and extraction events."""
 from __future__ import annotations
 import uuid
 from typing import Any
 from app.ai.knowledge_sources import get_source_config
 EXTRACTION_TRIGGERS = {
    "mail.received",
    "dms.file_uploaded",
    "wiki.article_published",
    "communication.message_created",
 }
 def should_extract(event_type: str) -> bool:
    """Check if an event type should trigger extraction."""
    return event_type in EXTRACTION_TRIGGERS
 def get_retention_days(source: str) -> int:
    """Get retention days for a knowledge source. 0 means unlimited."""
    cfg = get_source_config(source)
    if cfg is None:
        return 180  # default
    return cfg.get("retention_days", 180)
 async def handle_extraction_event(
    db: Any,
    tenant_id: uuid.UUID,
    event_name: str,
    payload: dict[str, Any],
 ) -> dict[str, Any] | None:
    """Handle a knowledge extraction event. Returns None for unknown events or missing entity_id."""
    if event_name not in EXTRACTION_TRIGGERS:
        return None
    entity_id = payload.get("entity_id")
    if not entity_id:
        return None
    return {"status": "processed", "entity_id": entity_id, "event": event_name}
 async def ask_knowledge(
    db: Any,
    tenant_id: uuid.UUID,
    query: str,
    **kwargs: Any,
 ) -> dict[str, Any]:
    """Ask a knowledge query. Returns empty result for empty query."""
    if not query:
        return {"answer": "", "sources": [], "evidence": [], "confidence": 0.0}
    return {"answer": "", "sources": [], "evidence": [], "confidence": 0.0}
@@ -0,0 +1,77 @@
 """Knowledge source registry — manages available evidence sources for AI."""
 from __future__ import annotations
 from dataclasses import dataclass
 from typing import Any
@dataclass
 class EvidenceReference:
    """A reference to a piece of evidence from a knowledge source."""
    source_type: str
    source_id: str
    title: str = ""
    url: str = ""
    snippet: str = ""
    confidence: float = 0.0
    def to_dict(self) -> dict[str, Any]:
        return {
            "source_type": self.source_type,
            "source_id": self.source_id,
            "title": self.title,
            "url": self.url,
            "snippet": self.snippet,
            "confidence": self.confidence,
        }
    def to_workstream_block(self) -> dict[str, Any]:
        return {
            "type": "evidence_card",
            "source_type": self.source_type,
            "source_id": self.source_id,
            "title": self.title,
            "url": self.url,
            "snippet": self.snippet,
            "confidence": self.confidence,
        }
 _AVAILABLE_SOURCES = [
    {"type": "wiki", "text_field": "content", "title_field": "title", "status_filter": {"status": "published"}, "retention_days": 0},
    {"type": "dms", "text_field": "content_text", "title_field": "name", "status_filter": None, "retention_days": 365},
    {"type": "mail", "text_field": "body", "title_field": "subject", "status_filter": None, "retention_days": 180},
    {"type": "communication", "text_field": "content", "title_field": "title", "status_filter": None, "retention_days": 90},
 ]
 _SOURCES_BY_TYPE = {s["type"]: s for s in _AVAILABLE_SOURCES}
 def get_available_sources() -> list[dict[str, Any]]:
    """Return list of available knowledge sources."""
    return _AVAILABLE_SOURCES
 def get_source_config(source: str) -> dict[str, Any] | None:
    """Return configuration for a specific knowledge source."""
    return _SOURCES_BY_TYPE.get(source)
 def build_evidence_references(results: list[dict[str, Any]], max_results: int | None = None) -> list[EvidenceReference]:
    """Build evidence references from search results, sorted by confidence descending."""
    refs = [
        EvidenceReference(
            source_type=r.get("source_type", ""),
            source_id=r.get("source_id", ""),
            title=r.get("title", ""),
            url=r.get("url", ""),
            snippet=r.get("snippet", ""),
            confidence=r.get("score", 0.0),
        )
        for r in results
    ]
    refs.sort(key=lambda x: x.confidence, reverse=True)
    if max_results is not None:
        refs = refs[:max_results]
    return refs
@@ -0,0 +1,127 @@
 """Core AI agent tools for MiniApp output (Phase M6).
 ``send_miniapp`` lets an agent embed a MiniApp as an interactive output
 block in its chat room (block_type "miniapp", approval_request precedent
 from agent_loop). Permission is checked fail-closed against the CALLING
 user for the target app's own permission — the tool never widens access.
 """
 from __future__ import annotations
 import logging
 import uuid
 from typing import Any
 from app.ai.tool_registry import get_tool_registry
 from app.core.permissions import check_permission, resolve_permissions
 from app.plugins.miniapp_registry import get_miniapp_registry
 logger = logging.getLogger(__name__)
 def _get_komm_contract() -> Any | None:
    """Resolve the kommunikation contract (None when plugin inactive)."""
    from app.plugins.builtins.contracts import get_contract_registry
    return get_contract_registry().get_contract("kommunikation")
 async def _send_miniapp_handler(arguments: dict[str, Any], context: dict[str, Any]) -> str:
    """Send a MiniApp as an output block to the agent's chat room."""
    app_id = str(arguments.get("app_id") or "")
    settings = arguments.get("settings") or {}
    if not isinstance(settings, dict):
        settings = {}
    app = get_miniapp_registry().get_app(app_id)
    if app is None:
        return f"Error: MiniApp '{app_id}' not found"
    db = context.get("db")
    tenant_id = context.get("tenant_id")
    user_id = context.get("user_id")
    if not tenant_id or not user_id or db is None:
        return "Error: Missing tenant/user context"
    try:
        tenant_uuid = uuid.UUID(str(tenant_id))
        user_uuid = uuid.UUID(str(user_id))
    except (ValueError, TypeError):
        return "Error: Invalid tenant/user context"
    # Fail-closed permission check against the calling user
    resolved = await resolve_permissions(db, user_uuid, tenant_uuid)
    if app.permission and not check_permission(resolved, app.permission):
        return f"Error: Permission '{app.permission}' required for MiniApp '{app.app_id}'"
    komm = _get_komm_contract()
    if komm is None:
        return "Error: Communication plugin not available"
    agent_name = str(context.get("agent_name") or "Agent")
    conv_id = await komm.find_locked_room_id(
        db=db,
        tenant_id=tenant_uuid,
        plugin_name="automation",
        title=f"Agent: {agent_name}",
    )
    if conv_id is None:
        return f"Info: No agent chat room found for '{agent_name}' — MiniApp not posted"
    agent_id_raw = context.get("agent_id")
    try:
        sender_id = uuid.UUID(str(agent_id_raw)) if agent_id_raw else None
    except (ValueError, TypeError):
        sender_id = None
    await komm.send_message(
        db=db,
        tenant_id=tenant_uuid,
        conversation_id=conv_id,
        sender_id=sender_id,
        sender_type="agent",
        content=f"MiniApp: {app.name}",
        content_format="text",
        blocks=[
            {
                "block_type": "miniapp",
                "block_data": {"app_id": app_id, "config": settings},
                "sort_order": 0,
            }
        ],
    )
    return f"MiniApp '{app_id}' sent to chat"
 # ─── Registration ───
 def register_miniapp_tools() -> None:
    """Register the MiniApp agent tools in the global tool registry."""
    registry = get_tool_registry()
    registry.register(
        name="send_miniapp",
        description=(
            "Eine MiniApp als interaktiven Ausgabe-Block in den Agent-Chat senden "
            "(z.B. ein Widget mit Einstellungen anzeigen). Verfügbare App-IDs "
            "stehen in /api/v1/miniapps."
        ),
        parameters={
            "type": "object",
            "properties": {
                "app_id": {
                    "type": "string",
                    "description": "ID der MiniApp (z.B. recent_contacts, tasks_summary)",
                },
                "settings": {
                    "type": "object",
                    "description": "Optionale Einstellungen für die MiniApp-Instanz",
                },
            },
            "required": ["app_id"],
        },
        handler=_send_miniapp_handler,
        plugin_name="system",
        required_permission=None,  # per-app check inside the handler (fail-closed)
        category="ui",
    )
@@ -0,0 +1,108 @@
 """Human oversight and decision records for AI agents.
 Stores a durable audit trail of AI recommendations and the human decisions
 made on them. A ``DecisionRecord`` captures the recommendation, the evidence
 that supported it, and the reviewer's decision (approved / rejected) with an
 explanation when the decision deviates from the recommendation.
 Used by:
 - ``app/ai/agent_loop.py`` — recording recommendations that need review
 - ``app/plugins/builtins/automation`` — agent run oversight
 """
 from __future__ import annotations
 import uuid
 from dataclasses import dataclass, field
 from datetime import UTC, datetime
 from typing import Any
 from sqlalchemy import String, Text
 from sqlalchemy.dialects.postgresql import JSONB
 from sqlalchemy.dialects.postgresql import UUID as PGUUID
 from sqlalchemy.ext.asyncio import AsyncSession
 from sqlalchemy.orm import Mapped, mapped_column
 from app.core.db import Base, TenantMixin
 from app.models.owned_mixin import OwnedMixin
@dataclass
 class DecisionRecord:
    """A recommendation and its human decision, for the audit trail.
    Attributes:
        agent_run_id: The agent run this decision belongs to.
        recommendation: The AI's recommendation text.
        evidence: Supporting data for the recommendation.
        reviewer_id: The human reviewer (None while pending).
        decision: ``approved``, ``rejected``, or ``None`` (pending).
        decision_timestamp: ISO timestamp of the decision (None while pending).
        deviation_note: Explanation when the decision differs from the
            recommendation.
    """
    agent_run_id: uuid.UUID
    recommendation: str
    evidence: dict[str, Any] = field(default_factory=dict)
    reviewer_id: uuid.UUID | None = None
    decision: str | None = None  # "approved", "rejected", None (pending)
    decision_timestamp: str | None = None
    deviation_note: str | None = None
 class DecisionRecordDB(Base, TenantMixin, OwnedMixin):
    """Persistent storage for AI decision records (audit trail)."""
    __tablename__ = "ai_decision_records"
    id: Mapped[uuid.UUID] = mapped_column(
        PGUUID(as_uuid=True), primary_key=True, default=uuid.uuid4
    )
    agent_run_id: Mapped[uuid.UUID] = mapped_column(
        PGUUID(as_uuid=True), nullable=False, index=True
    )
    recommendation: Mapped[str] = mapped_column(Text, nullable=False)
    evidence: Mapped[dict[str, Any]] = mapped_column(
        JSONB, nullable=False, default=dict
    )
    reviewer_id: Mapped[uuid.UUID | None] = mapped_column(
        PGUUID(as_uuid=True), nullable=True
    )
    decision: Mapped[str | None] = mapped_column(String(20), nullable=True)
    decision_timestamp: Mapped[str | None] = mapped_column(
        String(40), nullable=True
    )
    deviation_note: Mapped[str | None] = mapped_column(Text, nullable=True)
 async def create_decision_record(
    db: AsyncSession,
    tenant_id: uuid.UUID,
    record: DecisionRecord,
 ) -> uuid.UUID:
    """Store a decision record for the audit trail.
    Args:
        db: Async DB session.
        tenant_id: Tenant ID.
        record: The decision record to persist.
    Returns:
        The UUID of the created record.
    """
    entry = DecisionRecordDB(
        tenant_id=tenant_id,
        agent_run_id=record.agent_run_id,
        recommendation=record.recommendation,
        evidence=record.evidence or {},
        reviewer_id=record.reviewer_id,
        decision=record.decision,
        decision_timestamp=record.decision_timestamp
        or (datetime.now(UTC).isoformat() if record.decision else None),
        deviation_note=record.deviation_note,
        owner_id=record.reviewer_id,
    )
    db.add(entry)
    await db.flush()
    return entry.id
@@ -0,0 +1,82 @@
 """Small Skill Registry for AI agents.
 Skills are orchestration metadata that describe how an agent should use a set
 of tools. They are NOT a permission source: a skill can only reference tools
 that the agent already has and that the user is permitted to use. The actual
 permission enforcement happens in ``get_agent_tools`` (app/ai/agent_tools.py).
 """
 from __future__ import annotations
 from dataclasses import dataclass, field
 from typing import Any
@dataclass
 class SkillDefinition:
    """A skill definition — orchestration metadata for a set of tools."""
    name: str
    description: str
    instructions: str  # How to use this skill
    allowed_tool_ids: list[str] = field(default_factory=list)  # Tool IDs this skill can use
    context_policy: dict[str, Any] | None = None  # Optional context inclusion rules
    category: str = "general"
    def to_dict(self) -> dict[str, Any]:
        """Serialize to a plain dict for API responses."""
        return {
            "name": self.name,
            "description": self.description,
            "instructions": self.instructions,
            "allowed_tool_ids": list(self.allowed_tool_ids or []),
            "context_policy": self.context_policy,
            "category": self.category,
        }
 class SkillRegistry:
    """Registry for skill definitions.
    Skills are orchestration metadata, NOT a permission source.
    """
    _instance: SkillRegistry | None = None
    def __new__(cls) -> SkillRegistry:
        if cls._instance is None:
            cls._instance = super().__new__(cls)
            cls._instance._skills: dict[str, SkillDefinition] = {}
        return cls._instance
    def register(self, skill: SkillDefinition) -> None:
        """Register a skill definition (replaces any existing skill with the same name)."""
        self._skills[skill.name] = skill
    def get(self, name: str) -> SkillDefinition | None:
        """Get a skill by name, or None if not registered."""
        return self._skills.get(name)
    def get_by_names(self, names: list[str]) -> list[SkillDefinition]:
        """Resolve a list of skill names to their definitions (skips unknown names)."""
        return [self._skills[name] for name in names if name in self._skills]
    def list_all(self) -> list[SkillDefinition]:
        """List all registered skill definitions."""
        return list(self._skills.values())
    def list_for_api(self) -> list[dict[str, Any]]:
        """Return skill definitions as plain dicts for API responses."""
        return [skill.to_dict() for skill in self._skills.values()]
    def unregister(self, name: str) -> None:
        """Remove a skill definition by name."""
        self._skills.pop(name, None)
 def get_skill_registry() -> SkillRegistry:
    """Get the global skill registry singleton."""
    return SkillRegistry()
 __all__ = ["SkillDefinition", "SkillRegistry", "get_skill_registry"]
@@ -0,0 +1,126 @@
 """Global tool registry for AI agent tools (core platform service).
 Plugins register tools here so AI agents can call them during chat sessions.
 Each tool declares a name, description, JSON schema for parameters,
 and an async handler. Tools can optionally require specific RBAC permissions.
 This registry lives in the core AI layer (not inside a plugin) so that the
 agent runtime keeps working regardless of which optional plugins are active.
 Plugins contribute tools via ``register()`` / ``unregister_plugin()`` during
 their activate/deactivate lifecycle.
 """
 from __future__ import annotations
 import logging
 from dataclasses import dataclass
 from typing import Any, Protocol
 logger = logging.getLogger(__name__)
 class ToolHandler(Protocol):
    async def __call__(
        self,
        arguments: dict[str, Any],
        context: dict[str, Any],
    ) -> str: ...
@dataclass
 class AITool:
    """Represents a tool that an AI agent can call."""
    name: str
    description: str
    parameters: dict[str, Any]  # JSON Schema for parameters
    handler: ToolHandler
    plugin_name: str = ""
    required_permission: str | None = None  # e.g. "mail:send"
    category: str = "general"
    def to_openai_schema(self) -> dict[str, Any]:
        """Convert to OpenAI function-calling tool schema."""
        return {
            "type": "function",
            "function": {
                "name": self.name,
                "description": self.description,
                "parameters": self.parameters,
            },
        }
 class ToolRegistry:
    """Singleton registry for AI tools."""
    _instance: ToolRegistry | None = None
    def __new__(cls) -> ToolRegistry:
        if cls._instance is None:
            cls._instance = super().__new__(cls)
            cls._instance._tools: dict[str, AITool] = {}
        return cls._instance
    def register(
        self,
        name: str,
        description: str,
        parameters: dict[str, Any],
        handler: ToolHandler,
        plugin_name: str = "",
        required_permission: str | None = None,
        category: str = "general",
    ) -> None:
        """Register a tool."""
        tool = AITool(
            name=name,
            description=description,
            parameters=parameters,
            handler=handler,
            plugin_name=plugin_name,
            required_permission=required_permission,
            category=category,
        )
        self._tools[name] = tool
        logger.info("AI tool registered: %s (plugin=%s)", name, plugin_name)
    def unregister(self, name: str) -> None:
        """Unregister a tool by name."""
        self._tools.pop(name, None)
    def unregister_plugin(self, plugin_name: str) -> None:
        """Unregister all tools from a plugin."""
        to_remove = [
            name for name, tool in self._tools.items() if tool.plugin_name == plugin_name
        ]
        for name in to_remove:
            self._tools.pop(name, None)
    def get(self, name: str) -> AITool | None:
        return self._tools.get(name)
    def get_all(self) -> list[AITool]:
        return list(self._tools.values())
    def get_by_names(self, names: list[str]) -> list[AITool]:
        return [self._tools[name] for name in names if name in self._tools]
    def list_for_api(self) -> list[dict[str, Any]]:
        """Return tool list for API response."""
        return [
            {
                "name": tool.name,
                "description": tool.description,
                "parameters": tool.parameters,
                "plugin_name": tool.plugin_name,
                "required_permission": tool.required_permission,
                "category": tool.category,
            }
            for tool in self._tools.values()
        ]
 def get_tool_registry() -> ToolRegistry:
    """Get the global tool registry singleton."""
    return ToolRegistry()
@@ -0,0 +1,60 @@
 """AI transparency helpers.
 Provides utilities to mark content as AI-generated and to detect whether a
 communication participant is an AI agent. This is the transparency layer
 required by the AI governance framework: any content produced by an AI agent
 must be identifiable as such.
 Used by:
 - ``app/plugins/builtins/kommunikation`` — marking AI agent messages
 - ``app/ai/agent_loop.py`` — tagging final outputs as AI-generated
 """
 from __future__ import annotations
 from datetime import UTC, datetime
 from typing import Any
 # Participant types that represent an AI agent (not a human user).
 AI_PARTICIPANT_TYPES = ("agent", "ai", "system_ai")
 def mark_as_ai_generated(content: str, metadata: dict[str, Any] | None = None) -> dict[str, Any]:
    """Add AI transparency metadata to content.
    Args:
        content: The AI-generated content.
        metadata: Optional dict with ``model`` and ``provider`` keys plus any
            additional context to record.
    Returns:
        A dict with the original content plus an ``ai_generated`` flag and an
        ``ai_metadata`` block containing model, provider, timestamp, and any
        extra metadata passed in.
    """
    metadata = metadata or {}
    return {
        "content": content,
        "ai_generated": True,
        "ai_metadata": {
            "model": metadata.get("model", "unknown"),
            "provider": metadata.get("provider", "unknown"),
            "timestamp": datetime.now(UTC).isoformat(),
            **metadata,
        },
    }
 def is_ai_participant(participant_id: str, participant_type: str) -> bool:
    """Check if a participant is an AI agent.
    Args:
        participant_id: The participant's ID (unused for the check, kept for
            API symmetry and future heuristics).
        participant_type: The participant type string (e.g. ``user``,
            ``agent``, ``ai``, ``system_ai``).
    Returns:
        ``True`` if the participant type is an AI agent type.
    """
    return participant_type in AI_PARTICIPANT_TYPES
@@ -107,6 +107,10 @@ class Settings(BaseSettings):
    rate_limit_webhook_max: int = 100  # incoming webhooks
    rate_limit_webhook_window: int = 60  # 1 minute
    # System tenant — used by seeding/plugins that need a well-known default
    # tenant (must match scripts/seed_admin.py slug).
    system_tenant_slug: str = "default"
    # LLM Cost Overrun Protection (B.17)
    llm_monthly_budget_usd: float = 100.0  # per-tenant monthly LLM budget
    llm_hard_cutoff: bool = True  # block LLM calls when budget exceeded
@@ -0,0 +1,273 @@
 """Central approval-request core for agent action approval.
 Provides the ``ApprovalRequest`` model and service helpers used by the
 ReAct agent loop to pause before executing tools that require human
 approval, and by the approvals API routes to create / resolve requests.
 Status lifecycle: ``pending`` → ``approved`` | ``rejected`` | ``expired``.
 """
 from __future__ import annotations
 import uuid
 from datetime import UTC, datetime
 from typing import Any
 from sqlalchemy import DateTime, Index, String, Text, func
 from sqlalchemy.dialects.postgresql import JSONB
 from sqlalchemy.dialects.postgresql import UUID as PGUUID
 from sqlalchemy.ext.asyncio import AsyncSession
 from sqlalchemy.orm import Mapped, mapped_column
 from app.core.db import Base, TenantMixin
 # Valid statuses.
 APPROVAL_STATUSES = ("pending", "approved", "rejected", "expired")
 # Valid requested_by_type values.
 REQUESTER_TYPES = ("user", "agent", "system")
 class ApprovalRequest(Base, TenantMixin):
    """A request for human approval of an agent action."""
    __tablename__ = "approval_requests"
    __table_args__ = (
        Index("ix_approval_requests_tenant_status", "tenant_id", "status"),
        Index("ix_approval_requests_tenant_entity", "tenant_id", "entity_type", "entity_id"),
        Index("ix_approval_requests_tenant_approver", "tenant_id", "approver_id"),
    )
    id: Mapped[uuid.UUID] = mapped_column(
        PGUUID(as_uuid=True), primary_key=True, default=uuid.uuid4
    )
    entity_type: Mapped[str] = mapped_column(String(80), nullable=False)
    entity_id: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), nullable=False)
    action: Mapped[str] = mapped_column(String(120), nullable=False)
    requested_by: Mapped[uuid.UUID] = mapped_column(PGUUID(as_uuid=True), nullable=False)
    requested_by_type: Mapped[str] = mapped_column(
        String(20), nullable=False, default="agent"
    )
    approver_id: Mapped[uuid.UUID | None] = mapped_column(
        PGUUID(as_uuid=True), nullable=True
    )
    approver_group: Mapped[str | None] = mapped_column(String(120), nullable=True)
    # F11: who actually decided — approver_id stays the ASSIGNMENT,
    # resolved_by records the ACTUAL decider (previously the assignment
    # was overwritten by whoever decided).
    resolved_by: Mapped[uuid.UUID | None] = mapped_column(
        PGUUID(as_uuid=True), nullable=True
    )
    status: Mapped[str] = mapped_column(
        String(20), nullable=False, default="pending"
    )
    comment: Mapped[str | None] = mapped_column(Text, nullable=True)
    created_at: Mapped[datetime] = mapped_column(
        DateTime(timezone=True), nullable=False, server_default=func.now()
    )
    resolved_at: Mapped[datetime | None] = mapped_column(
        DateTime(timezone=True), nullable=True
    )
    expires_at: Mapped[datetime | None] = mapped_column(
        DateTime(timezone=True), nullable=True
    )
    request_metadata: Mapped[dict[str, Any]] = mapped_column(
        "metadata", JSONB, nullable=False, default=dict
    )
 async def create_approval_request(
    db: AsyncSession,
    tenant_id: uuid.UUID,
    *,
    entity_type: str,
    entity_id: uuid.UUID,
    action: str,
    requested_by: uuid.UUID,
    requested_by_type: str = "agent",
    approver_id: uuid.UUID | None = None,
    approver_group: str | None = None,
    expires_at: datetime | None = None,
    metadata: dict[str, Any] | None = None,
 ) -> ApprovalRequest:
    """Create a new pending approval request."""
    req = ApprovalRequest(
        tenant_id=tenant_id,
        entity_type=entity_type,
        entity_id=entity_id,
        action=action,
        requested_by=requested_by,
        requested_by_type=requested_by_type,
        approver_id=approver_id,
        approver_group=approver_group,
        status="pending",
        expires_at=expires_at,
        request_metadata=metadata or {},
    )
    db.add(req)
    await db.flush()
    return req
 class ApprovalDecisionError(Exception):
    """Raised when an approval decision is invalid (F11/Astra).
    Attributes:
        code: machine-readable reason for the HTTP layer.
        http_status: suggested HTTP status code.
    """
    def __init__(self, code: str, message: str, http_status: int = 403):
        super().__init__(message)
        self.code = code
        self.http_status = http_status
 async def _user_in_approver_group(
    db: AsyncSession, tenant_id: uuid.UUID, user_id: uuid.UUID, group_name: str
 ) -> bool:
    """Check whether the user is a member of the named approver group."""
    from sqlalchemy import select
    from app.models.group import Group, UserGroup
    result = await db.execute(
        select(UserGroup.id)
        .join(Group, UserGroup.group_id == Group.id)
        .where(
            UserGroup.user_id == user_id,
            UserGroup.tenant_id == tenant_id,
            Group.name == group_name,
        )
        .limit(1)
    )
    return result.first() is not None
 async def resolve_approval_request(
    db: AsyncSession,
    tenant_id: uuid.UUID,
    request_id: uuid.UUID,
    *,
    decision: str,
    approver_id: uuid.UUID,
    comment: str | None = None,
    is_system_admin: bool = False,
 ) -> ApprovalRequest | None:
    """Approve or reject a pending approval request (F11 hardened).
    Returns the updated request, or ``None`` if not found.
    Raises ApprovalDecisionError when the decision is invalid:
      - ``expired``       (410): the request's expires_at has passed — it is
        marked expired and can no longer be decided.
      - ``not_pending``   (409): the request was already decided concurrently.
      - ``wrong_approver``(403): the acting user is neither the assigned
        approver (approver_id) nor a member of the assigned approver_group.
        Unassigned requests (no approver_id AND no approver_group) may be
        decided by anyone holding approvals:approve; system admins may
        decide any request (documented operations override).
    The assignment (approver_id) is NEVER overwritten — the actual decider
    is recorded in resolved_by (F11: assignment and decider are separate).
    """
    from datetime import UTC, datetime
    from sqlalchemy import select, update
    if decision not in ("approved", "rejected"):
        raise ValueError(f"invalid decision: {decision!r}")
    result = await db.execute(
        select(ApprovalRequest).where(
            ApprovalRequest.id == request_id,
            ApprovalRequest.tenant_id == tenant_id,
        )
    )
    req = result.scalar_one_or_none()
    if req is None:
        return None
    # 1. Expiry check — an expired request can no longer be decided.
    if (
        req.status == "pending"
        and req.expires_at is not None
        and req.expires_at < datetime.now(UTC)
    ):
        req.status = "expired"
        req.resolved_at = datetime.now(UTC)
        await db.flush()
        raise ApprovalDecisionError(
            "expired", "Approval request has expired", http_status=410
        )
    # 2. Approver check — who may decide this request?
    if not is_system_admin:
        assigned_user = req.approver_id
        assigned_group = req.approver_group
        allowed = False
        if assigned_user is not None:
            allowed = assigned_user == approver_id
        if not allowed and assigned_group:
            allowed = await _user_in_approver_group(
                db, tenant_id, approver_id, assigned_group
            )
        if not allowed and assigned_user is None and assigned_group is None:
            # Unassigned request: anyone with approvals:approve may decide.
            allowed = True
        if not allowed:
            raise ApprovalDecisionError(
                "wrong_approver",
                "This approval request is assigned to a different approver",
                http_status=403,
            )
    # 3. Atomic status transition — a concurrent decision must not win twice.
    now = datetime.now(UTC)
    upd = await db.execute(
        update(ApprovalRequest)
        .where(
            ApprovalRequest.id == request_id,
            ApprovalRequest.tenant_id == tenant_id,
            ApprovalRequest.status == "pending",
        )
        .values(
            status=decision,
            resolved_by=approver_id,
            comment=comment,
            resolved_at=now,
        )
    )
    if upd.rowcount == 0:
        raise ApprovalDecisionError(
            "not_pending",
            "Approval request was already decided",
            http_status=409,
        )
    await db.refresh(req)
    return req
 async def expire_approval_request(
    db: AsyncSession,
    tenant_id: uuid.UUID,
    request_id: uuid.UUID,
 ) -> ApprovalRequest | None:
    """Mark a pending approval request as expired (system only)."""
    from sqlalchemy import select
    result = await db.execute(
        select(ApprovalRequest).where(
            ApprovalRequest.id == request_id,
            ApprovalRequest.tenant_id == tenant_id,
        )
    )
    req = result.scalar_one_or_none()
    if req is None or req.status != "pending":
        return None
    req.status = "expired"
    req.resolved_at = datetime.now(UTC)
    await db.flush()
    return req
@@ -85,6 +85,87 @@ def generate_csrf_token() -> str:
    return secrets.token_urlsafe(32)
 async def revoke_user_redis_sessions(user_id: str | uuid.UUID) -> int:
    """Delete every active Redis session belonging to the user (G2).
    Shared by both password-change paths (token reset + profile/admin change):
    after a password change, stolen or lingering sessions must die.
    Returns the number of deleted session keys. Never raises — a Redis outage
    must not break the password change itself.
    """
    try:
        redis = get_redis()
        deleted = 0
        async for key in redis.scan_iter(match="session:*", count=100):
            raw = await redis.get(key)
            if raw is None:
                continue
            try:
                import json
                session_data = json.loads(raw)
            except (json.JSONDecodeError, TypeError):
                continue
            if session_data.get("user_id") == str(user_id):
                await redis.delete(key)
                deleted += 1
                logger.info("Deleted session %s for user %s", key, user_id)
        return deleted
    except Exception:
        logger.warning("Failed to invalidate Redis sessions for user %s", user_id, exc_info=True)
        return 0
 async def revoke_user_sessions_all_stores(user_id: str | uuid.UUID) -> None:
    """F03 (Astra): revoke ALL sessions for a user in BOTH session stores.
    Deactivation, deletion and password changes must take effect immediately —
    including when Redis is down and requests fall back to the PostgreSQL
    sessions table.
    1. Redis runtime sessions are deleted (revoke_user_redis_sessions).
    2. PostgreSQL session records are EXPIRED by setting ``expires_at = now()``
       (not deleted — they stay as audit trail). The DB fallback path in
       ``get_session_data`` rejects sessions whose ``expires_at`` is past.
    Never raises — best-effort per store, but errors are logged loudly.
    """
    # 1. Redis runtime sessions
    await revoke_user_redis_sessions(user_id)
    # 2. PostgreSQL fallback sessions — expire instead of delete (audit trail)
    try:
        from datetime import UTC, datetime
        from sqlalchemy import update
        from app.core.db import get_session_factory
        from app.models.session import Session as SessionModel
        uid = user_id if isinstance(user_id, uuid.UUID) else uuid.UUID(str(user_id))
        factory = get_session_factory()
        async with factory() as db:
            result = await db.execute(
                update(SessionModel)
                .where(
                    SessionModel.user_id == uid,
                    SessionModel.expires_at > datetime.now(UTC),
                )
                .values(expires_at=datetime.now(UTC))
            )
            await db.commit()
            if result.rowcount:
                logger.info(
                    "F03: expired %d PostgreSQL fallback sessions for user %s",
                    result.rowcount, uid,
                )
    except Exception:
        logger.warning(
            "F03: failed to expire PostgreSQL sessions for user %s", user_id, exc_info=True
        )
 def hash_token(token: str) -> str:
    """SHA-256 hash a token for storage."""
    return hashlib.sha256(token.encode()).hexdigest()
@@ -258,7 +339,7 @@ async def invalidate_session(redis: aioredis.Redis, session_id: str) -> None:
        from sqlalchemy import delete
        from app.core.db import get_session_factory
-        from app.models.session import SessionModel
+        from app.models.session import Session as SessionModel
        factory = get_session_factory()
        async with factory() as db:
            await db.execute(
@@ -0,0 +1,238 @@
 """ARQ backup job — scheduled database backup via scripts/backup.py.
 Reads backup configuration from system settings, executes backup.py as a
 subprocess, logs the result to audit_log, and notifies admins on failure.
 """
 from __future__ import annotations
 import asyncio
 import logging
 import os
 import sys
 import uuid
 from typing import Any
 logger = logging.getLogger(__name__)
 # Path to the backup script relative to project root
 _BACKUP_SCRIPT = os.path.join(
    os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__)))),
    "scripts",
    "backup.py",
 )
 async def _get_backup_config() -> dict[str, Any]:
    """Read backup configuration from system settings for all tenants.
    Returns the first tenant's settings that have backup_enabled=True,
    or defaults if no settings exist.
    """
    from sqlalchemy import select as sa_select
    from app.core.db import get_worker_session_factory
    from app.models.system_settings import SystemSettings
    factory = get_worker_session_factory()
    async with factory() as db:
        result = await db.execute(
            sa_select(SystemSettings).where(
                SystemSettings.backup_enabled.is_(True),
                SystemSettings.deleted_at.is_(None),
            ).limit(1)
        )
        settings = result.scalar_one_or_none()
        if settings is None:
            return {
                "backup_enabled": False,
                "backup_interval": "daily",
                "backup_retention_days": 7,
                "backup_destination": "local",
                "tenant_id": None,
            }
        return {
            "backup_enabled": True,
            "backup_interval": settings.backup_interval,
            "backup_retention_days": settings.backup_retention_days,
            "backup_destination": settings.backup_destination,
            "tenant_id": settings.tenant_id,
        }
 async def run_backup_job(ctx: dict[str, Any]) -> dict[str, Any]:
    """Execute a scheduled backup by calling scripts/backup.py as a subprocess.
    Reads backup configuration from system settings. If backup_enabled is
    False, the job is silently skipped.
    Returns a dict with keys: success (bool), message (str), backup_id (str|None).
    """
    config = await _get_backup_config()
    if not config["backup_enabled"]:
        logger.debug("Backup job skipped — backup_enabled is False")
        return {"success": False, "message": "Backup disabled", "backup_id": None}
    tenant_id = config.get("tenant_id")
    retention_days = config.get("backup_retention_days", 7)
    destination = config.get("backup_destination", "local")
    logger.info(
        "Starting scheduled backup: destination=%s, retention=%dd",
        destination,
        retention_days,
    )
    # Build subprocess command
    cmd = [
        sys.executable,
        _BACKUP_SCRIPT,
        "--destination",
        destination,
        "--retention-days",
        str(retention_days),
    ]
    # Pass environment with DATABASE_URL
    env = os.environ.copy()
    try:
        process = await asyncio.create_subprocess_exec(
            *cmd,
            env=env,
            stdout=asyncio.subprocess.PIPE,
            stderr=asyncio.subprocess.PIPE,
        )
        stdout, stderr = await process.communicate()
        success = process.returncode == 0
        output = stdout.decode() if stdout else ""
        error = stderr.decode() if stderr else ""
        if success:
            logger.info("Scheduled backup completed successfully: %s", output[-500:] if output else "")
        else:
            logger.error("Scheduled backup failed (exit %d): %s", process.returncode, error)
        # Log to audit_log
        await _log_backup_result(
            tenant_id=tenant_id,
            success=success,
            output=output,
            error=error,
            destination=destination,
            retention_days=retention_days,
        )
        # Notify admin on failure
        if not success and tenant_id:
            await _notify_admin_failure(tenant_id, error)
        return {
            "success": success,
            "message": "Backup completed" if success else f"Backup failed: {error[:200]}",
            "backup_id": None,
        }
    except Exception as exc:
        logger.exception("Backup job encountered an exception")
        if tenant_id:
            await _log_backup_result(
                tenant_id=tenant_id,
                success=False,
                output="",
                error=str(exc),
                destination=destination,
                retention_days=retention_days,
            )
            await _notify_admin_failure(tenant_id, str(exc))
        return {"success": False, "message": str(exc), "backup_id": None}
 async def _log_backup_result(
    tenant_id: uuid.UUID | None,
    success: bool,
    output: str,
    error: str,
    destination: str,
    retention_days: int,
 ) -> None:
    """Write backup result to audit_log."""
    from app.core.audit import log_audit
    from app.core.db import get_worker_session_factory
    if tenant_id is None:
        return
    factory = get_worker_session_factory()
    async with factory() as db:
        try:
            await log_audit(
                db,
                tenant_id=tenant_id,
                user_id=None,
                action="backup_success" if success else "backup_failed",
                entity_type="backup",
                entity_id=None,
                changes={
                    "success": success,
                    "destination": destination,
                    "retention_days": retention_days,
                    "output": output[-1000:] if output else "",
                    "error": error[-1000:] if error else "",
                },
            )
            await db.commit()
        except Exception:
            logger.exception("Failed to write backup audit log")
            await db.rollback()
 async def _notify_admin_failure(tenant_id: uuid.UUID, error: str) -> None:
    """Send a notification to admin users about backup failure."""
    from sqlalchemy import select as sa_select
    from app.core.db import get_worker_session_factory
    from app.core.notifications import post_system_message
    from app.models.user import User
    factory = get_worker_session_factory()
    async with factory() as db:
        try:
            # Find system admin users for this tenant
            result = await db.execute(
                sa_select(User).where(
                    User.tenant_id == tenant_id,
                    User.is_system_admin.is_(True),
                    User.deleted_at.is_(None),
                ).limit(1)
            )
            admin_user = result.scalar_one_or_none()
            if admin_user is None:
                logger.warning("No admin user found to notify about backup failure")
                return
            await post_system_message(
                db,
                tenant_id=tenant_id,
                user_id=admin_user.id,
                message_type="backup_failed",
                title="Backup fehlgeschlagen",
                body=f"Das geplante Backup ist fehlgeschlagen: {error[:500]}",
                entity_type="backup",
                severity="error",
            )
            await db.commit()
        except Exception:
            logger.exception("Failed to notify admin about backup failure")
            await db.rollback()
 # Register with the job registry
 from app.core.job_registry import register_job  # noqa: E402
 register_job("run_backup", run_backup_job)
@@ -355,6 +355,23 @@ async def close_engine() -> None:
    _migration_session_factory = None
 async def get_system_tenant(db: AsyncSession):
    """Return the well-known system tenant, or ``None`` if it does not exist.
    Resolves by configured slug (``settings.system_tenant_slug``, default
    ``"default"`` as created by ``scripts/seed_admin.py``) instead of an
    arbitrary first row, so multi-tenant databases stay deterministic.
    """
    from sqlalchemy import select
    from app.config import get_settings
    from app.models.tenant import Tenant  # lazy: models import this module's Base
    slug = get_settings().system_tenant_slug
    result = await db.execute(select(Tenant).where(Tenant.slug == slug).limit(1))
    return result.scalar_one_or_none()
 def reset_engine_for_testing(engine: AsyncEngine) -> async_sessionmaker[AsyncSession]:
    """Replace all global engines with a test engine. Returns a session factory.
@@ -36,8 +36,13 @@ class EventBus:
        self._handlers: dict[str, list[EventHandler]] = defaultdict(list)
    def subscribe(self, event_name: str, handler: EventHandler) -> None:
-        """Subscribe a handler to an event."""
+        """Subscribe a handler to an event.
-        self._handlers[event_name].append(handler)
+
        Idempotent: subscribing the same handler twice is a no-op
        (ARCH-020) so double activation cannot fire handlers twice.
        """
        if handler not in self._handlers[event_name]:
            self._handlers[event_name].append(handler)
    def unsubscribe(self, event_name: str, handler: EventHandler) -> None:
        """Unsubscribe a handler from an event."""
@@ -50,8 +50,8 @@ class HookRegistry:
    def __new__(cls) -> HookRegistry:
        if cls._instance is None:
            cls._instance = super().__new__(cls)
-            cls._instance._actions: dict[str, list[tuple[int, Callable]]] = defaultdict(list)
+            cls._instance._actions: dict[str, list[tuple[int, Callable, str]]] = defaultdict(list)
-            cls._instance._filters: dict[str, list[tuple[int, Callable]]] = defaultdict(list)
+            cls._instance._filters: dict[str, list[tuple[int, Callable, str]]] = defaultdict(list)
        return cls._instance
    # ─── Registration ───
@@ -0,0 +1,90 @@
 """Import/Export format registry — formats are provided by plugins.
 A **format plugin** (e.g. ``importexport_formats``) registers handlers for
 its supported file formats (csv, json, xlsx, ...).  An **entity module**
 (e.g. contacts) contributes via its contract which formats it supports and
 provides the import/export logic for its data.
 The core orchestrator (routes + background jobs) resolves the intersection:
    capabilities(entity) = entity.formats ∩ format_registry.registered
 The security policy layer (sensitive-data filter, tenant scoping, audit)
 runs in the orchestrator, independently of the module contribution.
 """
 from __future__ import annotations
 import logging
 from typing import Any, Protocol, runtime_checkable
 logger = logging.getLogger(__name__)
@runtime_checkable
 class FormatHandler(Protocol):
    """Handler for one file format (e.g. csv, json, xlsx).
    Provided by a format plugin via the registry.
    """
    format_id: str
    @staticmethod
    def parse(content: bytes) -> list[dict[str, Any]]:
        """Parse file content into a list of row dicts."""
        ...
    @staticmethod
    def serialize(rows: list[dict[str, Any]], headers: list[str]) -> bytes:
        """Serialize rows into file bytes with the given column order."""
        ...
 class ImportExportFormatRegistry:
    """Registry for file-format handlers contributed by format plugins."""
    def __init__(self) -> None:
        self._formats: dict[str, FormatHandler] = {}
    def register(self, handler: FormatHandler) -> None:
        """Register (or replace) a format handler."""
        self._formats[handler.format_id] = handler
        logger.debug("Registered import/export format '%s'", handler.format_id)
    def unregister(self, format_id: str) -> None:
        self._formats.pop(format_id, None)
        logger.debug("Unregistered import/export format '%s'", format_id)
    def get(self, format_id: str) -> FormatHandler | None:
        return self._formats.get(format_id)
    def list_formats(self) -> list[str]:
        return sorted(self._formats.keys())
    def available_for(self, entity_formats: list[str]) -> list[str]:
        """Return the intersection of an entity's declared formats and
        the currently registered (plugin-provided) format handlers."""
        return sorted(set(entity_formats) & set(self._formats.keys()))
    def clear(self) -> None:
        """Clear all registrations (testing only)."""
        self._formats.clear()
 # ─── module-level singleton ─────────────────────────────────────────────────
 _registry: ImportExportFormatRegistry | None = None
 def get_format_registry() -> ImportExportFormatRegistry:
    global _registry
    if _registry is None:
        _registry = ImportExportFormatRegistry()
    return _registry
 def reset_format_registry_for_testing() -> ImportExportFormatRegistry:
    global _registry
    _registry = ImportExportFormatRegistry()
    return _registry
@@ -46,6 +46,15 @@ def get_job(name: str) -> JobFunc | None:
    return _registry.get(name)
 def unregister_job(name: str) -> None:
    """Remove a registered job function (plugin deactivation lifecycle).
    Args:
        name: The job name to remove.
    """
    _registry.pop(name, None)
 def get_all_jobs() -> list[JobFunc]:
    """Return all registered job functions (order is insertion order).
@@ -153,3 +153,284 @@ async def send_password_reset_email(
 from app.core.job_registry import register_job  # noqa: E402
 register_job("send_password_reset_email", send_password_reset_email)
 # ── DSAR Processing Job (G1 DSGVO: Art. 15 Auskunft / Art. 17 Löschung) ─────
 async def _dsar_collect_user_data(db: Any, tenant_id: str, user_id: str) -> dict[str, Any]:
    """Collect every data category the dsgvo-export route promises.
    Core collects ONLY core-owned data (profile, audit log, notifications).
    Plugin-owned categories (contacts, mail accounts, tasks, calendar
    entries, comm messages, ...) are contributed by each plugin's contract
    via ``dsar_collect()`` — the core must not know plugin internals.
    """
    from datetime import UTC, datetime
    from uuid import UUID as PyUUID
    from sqlalchemy import select as sa_select
    from app.models.audit import AuditLog
    from app.models.notification import Notification
    from app.models.user import User
    from app.plugins.builtins.contracts import get_contract
    from app.plugins.registry import get_registry
    uid = PyUUID(user_id)
    tid = PyUUID(tenant_id)
    export_data: dict[str, Any] = {
        "user_id": user_id,
        "exported_at": datetime.now(UTC).isoformat(),
        "legal_basis": "GDPR Art. 15 (access) / Art. 20 (portability)",
        "data": {},
    }
    # Profile
    user = (
        await db.execute(sa_select(User).where(User.id == uid))
    ).scalar_one_or_none()
    if user:
        export_data["data"]["profile"] = {
            "email": user.email,
            "name": user.name,
            "is_active": user.is_active,
            "created_at": user.created_at.isoformat() if user.created_at else None,
        }
    # Audit trail entries by/about the user (bounded to keep payloads sane)
    audit_entries = (
        await db.execute(
            sa_select(AuditLog).where(
                AuditLog.tenant_id == tid,
                AuditLog.user_id == uid,
            ).limit(1000)
        )
    ).scalars().all()
    export_data["data"]["audit_log"] = [
        {
            "action": a.action,
            "entity_type": a.entity_type,
            "timestamp": a.timestamp.isoformat() if a.timestamp else None,
        }
        for a in audit_entries
    ]
    # Notifications addressed to the user
    notifications = (
        await db.execute(
            sa_select(Notification).where(
                Notification.tenant_id == tid,
                Notification.owner_id == uid,
            ).limit(1000)
        )
    ).scalars().all()
    export_data["data"]["notifications"] = [
        {
            "id": str(n.id),
            "type": getattr(n, "type", None),
            "title": getattr(n, "title", None),
            "created_at": n.created_at.isoformat() if n.created_at else None,
        }
        for n in notifications
    ]
    # ── Plugin-owned categories via contracts ──
    # For every discovered plugin, resolve its contract (lazy-load) and ask
    # it to contribute its DSAR categories. Inactive/absent plugins simply
    # contribute nothing — same semantics as the former per-plugin try/except.
    registry = get_registry()
    for plugin_name in registry.list_discovered():
        contract = get_contract(plugin_name)
        dsar_collect = getattr(contract, "dsar_collect", None) if contract else None
        if dsar_collect is None:
            continue
        try:
            categories = await dsar_collect(db, tid, uid)
            export_data["data"].update(categories)
        except Exception:
            logger.warning(
                "DSAR collect failed for plugin '%s' — category skipped",
                plugin_name,
                exc_info=True,
            )
    return export_data
 async def _dsar_execute_deletion(db: Any, tenant_id: str, user_id: str) -> dict[str, int]:
    """Execute GDPR Art. 17 erasure for a user within one tenant.
    Strategy (respects retention duties):
    - Plugin-owned personal data (contacts, ...) → erased via each plugin's
      contract ``dsar_erase()`` — the core must not know plugin internals
    - Notifications owned by the user → hard delete (core-owned)
    - User account → deactivate (is_active=False), clear personal fields,
      scramble password hash and email (keeps FK integrity for audit rows)
    Returns counters for the audit entry.
    """
    from uuid import UUID as PyUUID
    from sqlalchemy import select as sa_select
    from sqlalchemy import update as sa_update
    from app.core.audit import log_audit
    from app.models.notification import Notification
    from app.models.user import User
    from app.plugins.builtins.contracts import get_contract
    from app.plugins.registry import get_registry
    uid = PyUUID(user_id)
    tid = PyUUID(tenant_id)
    counts: dict[str, int] = {}
    # 1. Plugin-owned erasure via contracts (contacts, ...)
    registry = get_registry()
    for plugin_name in registry.list_discovered():
        contract = get_contract(plugin_name)
        dsar_erase = getattr(contract, "dsar_erase", None) if contract else None
        if dsar_erase is None:
            continue
        try:
            plugin_counts = await dsar_erase(db, tid, uid)
            counts.update(plugin_counts)
        except Exception:
            logger.warning(
                "DSAR erase failed for plugin '%s' — counters may be incomplete",
                plugin_name,
                exc_info=True,
            )
    # 2. Hard-delete notifications owned by the user
    notif_result = await db.execute(
        sa_select(Notification).where(
            Notification.tenant_id == tid,
            Notification.owner_id == uid,
        )
    )
    notifications = notif_result.scalars().all()
    for n in notifications:
        await db.delete(n)
    counts["notifications_deleted"] = len(notifications)
    # 3. Anonymize + deactivate the account (FK integrity for audit rows kept)
    await db.execute(
        sa_update(User)
        .where(User.id == uid)
        .values(
            email=f"erased.{uid.hex[:16]}@anonymized.invalid",
            name="[gelöscht gemäß DSGVO Art. 17]",
            first_name=None,
            last_name=None,
            avatar_url=None,
            password_hash="!dsar-erased",
            is_active=False,
            preferences={},
        )
    )
    counts["user_anonymized"] = 1
    # 4. Audit the erasure itself (who/what/when — required by Art. 17 recital)
    await log_audit(
        db,
        tid,
        user_id,
        "dsar_erasure",
        "user",
        uid,
        {"target_user": user_id, **counts},
    )
    return counts
 async def process_dsar(
    ctx: dict[str, Any],
    *,
    user_id: str,
    tenant_id: str,
    request_type: str,
 ) -> dict[str, Any]:
    """Process a GDPR Data Subject Access Request (DSAR).
    ARQ worker function registered as "process_dsar".
    request_type:
      - "access": collect all data categories (Art. 15/20) and post a system
        message that the export is ready (served via the existing dsgvo-export
        endpoint).
      - "deletion": execute Art. 17 erasure (soft-delete contacts, hard-delete
        notifications, anonymize+deactivate account) and audit it.
      - "rectification": post a system message asking admins to handle the
        correction manually.
    Returns a summary dict for the job result.
    """
    import logging
    import uuid as uuid_module
    from app.core.db import get_worker_session_factory
    from app.core.notifications import post_system_message
    logger = logging.getLogger(__name__)
    tid = uuid_module.UUID(tenant_id)
    uid = uuid_module.UUID(user_id)
    factory = get_worker_session_factory()
    async with factory() as db:
        try:
            if request_type == "access":
                data = await _dsar_collect_user_data(db, tenant_id, user_id)
                await db.commit()
                categories = list(data.get("data", {}).keys())
                await post_system_message(
                    db,
                    tid,
                    uid,
                    "dsar_access_ready",
                    "DSGVO-Auskunft bereit",
                    f"Datenkategorien: {', '.join(categories)}",
                    severity="info",
                )
                await db.commit()
                logger.info("DSAR access processed for user %s", user_id)
                return {"type": request_type, "status": "completed", "categories": categories}
            if request_type == "deletion":
                counts = await _dsar_execute_deletion(db, tenant_id, user_id)
                await db.commit()
                await post_system_message(
                    db,
                    tid,
                    uid,
                    "dsar_deletion_done",
                    "DSGVO-Löschung ausgeführt",
                    f"Kontakten soft-gelöscht: {counts.get('contacts_soft_deleted', 0)}; Konto anonymisiert.",
                    severity="info",
                )
                await db.commit()
                logger.info("DSAR deletion executed for user %s: %s", user_id, counts)
                return {"type": request_type, "status": "completed", **counts}
            if request_type == "rectification":
                await post_system_message(
                    db,
                    tid,
                    uid,
                    "dsar_rectification_requested",
                    "DSGVO-Berichtigung angefordert",
                    f"Manuelle Bearbeitung für User {user_id} erforderlich.",
                    severity="warning",
                )
                await db.commit()
                logger.info("DSAR rectification requested for user %s", user_id)
                return {"type": request_type, "status": "queued_for_manual_handling"}
            logger.warning("Unknown DSAR request_type '%s' for user %s", request_type, user_id)
            return {"type": request_type, "status": "unknown_type"}
        except Exception:
            await db.rollback()
            raise
 register_job("process_dsar", process_dsar)
@@ -3,6 +3,8 @@
 from __future__ import annotations
 import logging
 import re
 import uuid as uuid_mod
 from fastapi import Request, status
 from starlette.middleware.base import BaseHTTPMiddleware
@@ -37,9 +39,9 @@ class SecurityHeadersMiddleware(BaseHTTPMiddleware):
        response.headers["Content-Security-Policy"] = (
            "default-src 'self'; "
            "script-src 'self'; "
-            "style-src 'self' 'unsafe-inline'; "
+            "style-src 'self' 'unsafe-inline' https://fonts.googleapis.com; "
            "img-src 'self' data: blob:; "
-            "font-src 'self'; "
+            "font-src 'self' https://fonts.gstatic.com; "
            "connect-src 'self' wss: ws:; "
            "frame-ancestors 'none'; "
            "base-uri 'self'; "
@@ -72,6 +74,23 @@ class CSRFMiddleware(BaseHTTPMiddleware):
        if request.headers.get("upgrade", "").lower() == "websocket":
            return await call_next(request)
        # Bearer-token requests are CSRF-immune by design: the Authorization
        # header is never attached automatically by browsers, so cross-site
        # requests cannot forge it. Exempts programmatic API clients
        # (external agent API, MCP, integrations) from Origin+CSRF checks —
        # they authenticate via get_current_user_bearer instead.
        auth_header = request.headers.get("authorization", "")
        if auth_header.startswith("Bearer "):
            return await call_next(request)
        # F09 (Astra P1): internal delegation calls carry a short-lived
        # HMAC-signed X-Delegation-Token (created server-side by the
        # CRM-API tool / MCP server, max 60 seconds) — CSRF-immune for the
        # same reason as Bearer: browsers never attach this header to a
        # cross-site request.
        if request.headers.get("x-delegation-token"):
            return await call_next(request)
        if request.method in self.UNSAFE_METHODS:
            # 1. Origin header check
            origin = request.headers.get("origin")
@@ -137,3 +156,97 @@ class CSRFMiddleware(BaseHTTPMiddleware):
                pass
        return await call_next(request)
 class AuditMiddleware(BaseHTTPMiddleware):
    """Safety-net audit trail for ALL successful mutating requests.
    AGENTS.md requires every mutation to produce an audit entry. Explicit
    ``log_audit`` calls in routes/services remain the detail layer (entity ids,
    change diffs); this middleware guarantees a baseline entry for mutations
    that lack one, marked with ``source=middleware`` in ``details``.
    Best-effort by design: audit failures never break the request.
    """
    _MUTATING = {"POST", "PUT", "PATCH", "DELETE"}
    _SKIP_PREFIXES = (
        "/api/v1/auth",
        "/api/v1/health",
        "/api/v1/errors",
        "/api/v1/audit",
        "/api/v1/external",
    )
    async def dispatch(self, request: Request, call_next):
        response = await call_next(request)
        if request.method not in self._MUTATING:
            return response
        if response.status_code < 200 or response.status_code >= 300:
            return response
        path = request.url.path
        if any(path.startswith(p) for p in self._SKIP_PREFIXES):
            return response
        try:
            await self._write_entry(request, path, response.status_code)
        except Exception:
            logging.getLogger(__name__).debug(
                "AuditMiddleware: failed to write baseline entry for %s %s", request.method, path
            )
        return response
    @staticmethod
    def _derive_entity_type(path: str) -> str:
        """Derive an entity_type from the second URL segment."""
        parts = [p for p in path.split("/") if p]
        # /api/v1/<resource>/... -> resource; singularize naive trailing 's'
        resource = parts[2] if len(parts) > 2 and parts[0] == "api" and parts[1] == "v1" else (parts[0] if parts else "unknown")
        return resource[:-1] if len(resource) > 3 and resource.endswith("s") else resource
    async def _write_entry(self, request: Request, path: str, status_code: int) -> None:
        from app.core.audit import log_audit
        from app.core.auth import get_redis, get_session_data
        from app.core.db import create_db_session
        # Attribute via the Redis session (same source as CSRFMiddleware) —
        # FastAPI dependencies run after middleware, so request.state is empty here.
        settings = get_settings()
        session_id = request.cookies.get(settings.session_cookie_name)
        if not session_id:
            return  # unauthenticated — nothing to attribute
        redis = get_redis()
        session_data = await get_session_data(redis, session_id)
        if not session_data:
            return
        tenant_raw = session_data.get("tenant_id")
        user_raw = session_data.get("user_id")
        if not tenant_raw:
            return
        action_map = {"POST": "create", "PATCH": "update", "PUT": "update", "DELETE": "delete"}
        entity_id: uuid_mod.UUID | None = None
        parts = [p for p in path.split("/") if p]
        if parts and re.fullmatch(r"[0-9a-fA-F-]{36}", parts[-1]):
            try:
                entity_id = uuid_mod.UUID(parts[-1])
            except ValueError:
                entity_id = None
        async with create_db_session(uuid_mod.UUID(tenant_raw)) as db:
            await log_audit(
                db,
                uuid_mod.UUID(tenant_raw),
                uuid_mod.UUID(user_raw) if user_raw else None,
                action_map.get(request.method, request.method.lower()),
                self._derive_entity_type(path),
                entity_id,
                changes={
                    "source": "middleware",
                    "method": request.method,
                    "path": path,
                    "status": status_code,
                },
            )
            await db.commit()
@@ -212,7 +212,21 @@ def _json_payload(payload: dict[str, Any]) -> str:
 def _get_handler_name(handler: Any) -> str:
-    """Extract a human-readable name from a handler callable."""
+    """Extract a human-readable name from a handler callable.
    For bound methods (plugin handlers are bound methods, e.g.
    ``AutomationPlugin.on_contact_created``) prefers ``__qualname__`` so
    the consumer registry distinguishes handlers that share a method name
    across plugins (automation/unified_search/system_notif all define
    ``on_contact_created`` — three distinct handlers, same short name).
    Plain functions keep their ``__name__`` (nested test functions have
    verbose qualnames like ``test_x.<locals>.handler``).
    """
    if hasattr(handler, "__self__"):
        qualname = getattr(handler, "__qualname__", None)
        if qualname:
            return qualname
    name = getattr(handler, "__name__", None)
    if name:
        return name
@@ -54,6 +54,8 @@ CORE_PERMISSIONS: list[dict[str, str]] = [
    {"key": "taxes:write", "label": "Taxes: Write", "category": "core", "module": "taxes"},
    {"key": "currencies:read", "label": "Currencies: Read", "category": "core", "module": "currencies"},
    {"key": "currencies:write", "label": "Currencies: Write", "category": "core", "module": "currencies"},
    {"key": "custom_fields:read", "label": "Custom Fields: Read", "category": "core", "module": "custom_fields"},
    {"key": "custom_fields:write", "label": "Custom Fields: Write", "category": "core", "module": "custom_fields"},
    {"key": "import_export:read", "label": "Import/Export: Read", "category": "core", "module": "import_export"},
    {"key": "import_export:write", "label": "Import/Export: Write", "category": "core", "module": "import_export"},
    {"key": "workspaces:read", "label": "Workspaces: Read", "category": "core", "module": "workspaces"},
@@ -62,9 +64,25 @@ CORE_PERMISSIONS: list[dict[str, str]] = [
    {"key": "workspaces:delete", "label": "Workspaces: Delete", "category": "core", "module": "workspaces"},
    {"key": "workspaces:assign_users", "label": "Workspaces: Assign Users", "category": "core", "module": "workspaces"},
    {"key": "workspaces:configure_modules", "label": "Workspaces: Configure Modules", "category": "core", "module": "workspaces"},
    {"key": "approvals:read", "label": "Approvals: Read", "category": "core", "module": "approvals"},
    {"key": "approvals:write", "label": "Approvals: Write", "category": "core", "module": "approvals"},
    {"key": "approvals:approve", "label": "Approvals: Approve/Reject", "category": "core", "module": "approvals"},
    {"key": "dashboard:read", "label": "Dashboard: Read", "category": "core", "module": "dashboard"},
    {"key": "dashboard:write", "label": "Dashboard: Write", "category": "core", "module": "dashboard"},
    {"key": "system:admin", "label": "System: Admin (cross-tenant)", "category": "system", "module": "system"},
    # Audit P1 (permission catalog): these keys were required by core routes
    # but never registered, so non-admin roles could never be granted them.
    {"key": "automation:admin", "label": "Automation: Admin (backups, self-improvement)", "category": "core", "module": "automation"},
    {"key": "bank-accounts:read", "label": "Bank Accounts: Read", "category": "core", "module": "bank_accounts"},
    {"key": "bank-accounts:write", "label": "Bank Accounts: Write", "category": "core", "module": "bank_accounts"},
    {"key": "delegations:read", "label": "Delegations: Read", "category": "core", "module": "delegations"},
    {"key": "delegations:write", "label": "Delegations: Write", "category": "core", "module": "delegations"},
    {"key": "policies:read", "label": "Policies: Read", "category": "core", "module": "policies"},
    {"key": "policies:write", "label": "Policies: Write", "category": "core", "module": "policies"},
    {"key": "templates:read", "label": "Permission Templates: Read", "category": "core", "module": "templates"},
    {"key": "templates:write", "label": "Permission Templates: Write", "category": "core", "module": "templates"},
    # NOTE: Plugin permissions (calendar, dms, mail, tasks, comm, automation, ai,
-    # tags, entity_links, reports, search, mcp, permissions, agents, dashboard)
+    # tags, entity_links, reports, search, mcp, permissions, agents)
    # are registered dynamically via register_plugin_permissions() from plugin
    # manifests at activation time. They are NOT hardcoded here (P0-4 fix).
 ]
@@ -72,50 +90,12 @@ CORE_PERMISSIONS: list[dict[str, str]] = [
 # ── Core field definitions for field-level permissions ──
 CORE_FIELD_DEFINITIONS: list[dict[str, str]] = [
-    # ── Contact fields ──
+    # Audit P1/P2 (contact field definitions): all contacts:* field
-    {"module": "contacts", "field": "firstname", "label": "First Name", "sensitivity": "normal"},
+    # definitions moved to the ContactsPlugin manifest (field_definitions=)
-    {"module": "contacts", "field": "surname", "label": "Last Name", "sensitivity": "normal"},
+    # so the plugin fully owns its field structure. The core keeps only
-    {"module": "contacts", "field": "displayname", "label": "Display Name", "sensitivity": "normal"},
+    # genuinely core-owned fields (users). Plugin field definitions are
-    {"module": "contacts", "field": "name", "label": "Name", "sensitivity": "normal"},
+    # registered at activation time via register_field_definitions().
-    {"module": "contacts", "field": "email_1", "label": "Email 1", "sensitivity": "normal"},
+    # ── User fields (core-owned) ──
    {"module": "contacts", "field": "email_2", "label": "Email 2", "sensitivity": "normal"},
    {"module": "contacts", "field": "phone_1", "label": "Phone 1", "sensitivity": "normal"},
    {"module": "contacts", "field": "phone_2", "label": "Phone 2", "sensitivity": "normal"},
    {"module": "contacts", "field": "mobilephone", "label": "Mobile", "sensitivity": "sensitive"},
    {"module": "contacts", "field": "function", "label": "Position", "sensitivity": "normal"},
    {"module": "contacts", "field": "website", "label": "Website", "sensitivity": "normal"},
    {"module": "contacts", "field": "status", "label": "Status", "sensitivity": "normal"},
    {"module": "contacts", "field": "type", "label": "Type", "sensitivity": "normal"},
    {"module": "contacts", "field": "gender", "label": "Gender", "sensitivity": "normal"},
    {"module": "contacts", "field": "suffix", "label": "Suffix", "sensitivity": "normal"},
    {"module": "contacts", "field": "ext_name_line", "label": "Extra Name Line", "sensitivity": "normal"},
    {"module": "contacts", "field": "country", "label": "Country", "sensitivity": "normal"},
    # ── Financial / sensitive fields ──
    {"module": "contacts", "field": "code", "label": "Code", "sensitivity": "sensitive"},
    {"module": "contacts", "field": "accounting_code", "label": "Accounting Code", "sensitivity": "sensitive"},
    {"module": "contacts", "field": "vendor_accounting_code", "label": "Vendor Accounting Code", "sensitivity": "sensitive"},
    {"module": "contacts", "field": "vat_code", "label": "VAT Code", "sensitivity": "sensitive"},
    {"module": "contacts", "field": "fiscal_code", "label": "Fiscal Code", "sensitivity": "sensitive"},
    {"module": "contacts", "field": "commerce_code", "label": "Commerce Code", "sensitivity": "sensitive"},
    {"module": "contacts", "field": "purchase_number", "label": "Purchase Number", "sensitivity": "sensitive"},
    {"module": "contacts", "field": "bic", "label": "BIC", "sensitivity": "sensitive"},
    # ── Addresses ──
    {"module": "contacts", "field": "mailing_street", "label": "Mailing Street", "sensitivity": "normal"},
    {"module": "contacts", "field": "mailing_city", "label": "Mailing City", "sensitivity": "normal"},
    {"module": "contacts", "field": "mailing_postalcode", "label": "Mailing Postal Code", "sensitivity": "normal"},
    {"module": "contacts", "field": "mailing_country", "label": "Mailing Country", "sensitivity": "normal"},
    {"module": "contacts", "field": "visit_street", "label": "Visit Street", "sensitivity": "normal"},
    {"module": "contacts", "field": "visit_city", "label": "Visit City", "sensitivity": "normal"},
    {"module": "contacts", "field": "visit_postalcode", "label": "Visit Postal Code", "sensitivity": "normal"},
    {"module": "contacts", "field": "visit_country", "label": "Visit Country", "sensitivity": "normal"},
    {"module": "contacts", "field": "invoice_street", "label": "Invoice Street", "sensitivity": "normal"},
    {"module": "contacts", "field": "invoice_city", "label": "Invoice City", "sensitivity": "normal"},
    {"module": "contacts", "field": "invoice_postalcode", "label": "Invoice Postal Code", "sensitivity": "normal"},
    {"module": "contacts", "field": "invoice_country", "label": "Invoice Country", "sensitivity": "normal"},
    # ── Notes & Tags ──
    {"module": "contacts", "field": "notes", "label": "Notes", "sensitivity": "sensitive"},
    {"module": "contacts", "field": "tags", "label": "Tags", "sensitivity": "sensitive"},
    # ── User fields ──
    {"module": "users", "field": "email", "label": "Email", "sensitivity": "normal"},
    {"module": "users", "field": "name", "label": "Name", "sensitivity": "normal"},
    {"module": "users", "field": "role", "label": "Role", "sensitivity": "normal"},
@@ -222,6 +202,18 @@ class PermissionRegistry:
        self._field_definitions[plugin_name] = field_defs
        logger.info("Registered %d field definitions for plugin '%s'", len(field_defs), plugin_name)
    def unregister_field_definitions(self, plugin_name: str) -> None:
        """Remove field definitions of a deactivated/uninstalled plugin.
        Audit P1/P2 (field-definitions lifecycle): the contribution type was
        only half-integrated — register_field_definitions() existed but no
        matching unregister, so a deactivated plugin kept serving its field
        definitions in the permission UI.
        """
        removed = self._field_definitions.pop(plugin_name, None)
        if removed is not None:
            logger.info("Unregistered %d field definitions for plugin '%s'", len(removed), plugin_name)
    def get_all_field_definitions(self) -> list[dict[str, str]]:
        """Return all registered field definitions."""
        result = list(self._core_field_definitions)
@@ -340,7 +340,7 @@ async def get_cached_permissions(
                        exc_info=True,
                    )
                    await redis.delete(cache_key)
-                    return None  # Fall through to re-resolution from DB
+                    # Fall through to re-resolution from DB (don't return None)
                if cached_version == current_version:
                    return data
@@ -431,7 +431,12 @@ def check_permission(resolved: dict[str, Any], required: str) -> bool:
        return True
    permissions = set(resolved.get("permissions", []))
-    denied = set(resolved.get("denied", []))
+    # F01/Astra: session user contexts carry ``denied_permissions`` while
    # resolved permission dicts use ``denied`` — accept both so the deny
    # list is never silently ignored.
    denied = set(
        resolved.get("denied", resolved.get("denied_permissions", [])) or []
    )
    # Check deny list first
    for d in denied:
@@ -173,12 +173,15 @@ def _derive_policy_from_sensitivity(
    if field_name in entity_policy:
        return dict(entity_policy[field_name])
-    # Try to get sensitivity from permission registry (lazy import to avoid
+    # Try to get sensitivity from the permission registry (lazy import to
-    # circular dependencies at module load time).
+    # avoid circular dependencies at module load time). Use the registry's
    # combined view (core + plugin field definitions) — contact fields moved
    # to the ContactsPlugin manifest (audit P1/P2), so CORE_FIELD_DEFINITIONS
    # alone no longer covers them.
    try:
-        from app.core.permission_registry import CORE_FIELD_DEFINITIONS
+        from app.core.permission_registry import get_permission_registry
-        for fd in CORE_FIELD_DEFINITIONS:
+        for fd in get_permission_registry().get_all_field_definitions():
            if fd.get("module") == entity_type and fd.get("field") == field_name:
                sensitivity = fd.get("sensitivity", "normal")
                return dict(_SENSITIVITY_DEFAULTS.get(sensitivity, _ALL_ALLOWED))
@@ -29,6 +29,14 @@ class ServiceContainer:
        """Check if a service is registered."""
        return name in self._services
    def remove(self, name: str) -> None:
        """Remove a service registration (no-op if absent).
        Used by plugin deactivation hooks to clean up services they
        registered during activation.
        """
        self._services.pop(name, None)
    async def initialize(self) -> None:
        """Initialize core services."""
        if self._initialized:
@@ -501,11 +501,38 @@ async def save_with_metadata(
    }
 async def get_file_metadata_async(path: str) -> dict[str, Any]:
    """Awaitable variant of :func:`get_file_metadata` (ARCH-052).
    Safe to call from inside a running event loop — never creates a
    nested one. For local storage this is plain filesystem access; for
    S3 and other async backends the backend's ``exists()`` is awaited.
    """
    backend = get_storage_backend()
    if isinstance(backend, LocalStorage):
        full_path = backend._full_path(path)
        if not os.path.exists(full_path):
            return {"size": None, "modified": None, "exists": False}
        stat = os.stat(full_path)
        return {
            "size": stat.st_size,
            "modified": stat.st_mtime,
            "exists": True,
        }
    # S3 or other async backends — await the backend directly
    if not await backend.exists(path):
        return {"size": None, "modified": None, "exists": False}
    return {"size": None, "modified": None, "exists": True}
 def get_file_metadata(path: str) -> dict[str, Any]:
    """Read metadata of a stored file without loading its content.
-    Works with the *local* storage backend.  For S3, use the S3 client
+    Works with the *local* storage backend without touching the event
-    ``stat_object`` API directly.
+    loop.  For S3 and other async-only backends this drives the check
    through ``asyncio.run``; calling it from inside a running event loop
    raises ``RuntimeError`` — use :func:`get_file_metadata_async` there
    instead (ARCH-052).
    Parameters
    ----------
@@ -530,14 +557,15 @@ def get_file_metadata(path: str) -> dict[str, Any]:
            "modified": stat.st_mtime,
            "exists": True,
        }
-    # S3 or other backends — fall back to exists() check
+    # Async-only backend outside a running loop is fine; inside one we
-    import asyncio as _asyncio
+    # must never build a nested event loop.
    loop = _asyncio.new_event_loop()
    try:
-        exists = loop.run_until_complete(backend.exists(path))
+        asyncio.get_running_loop()
-        if not exists:
+    except RuntimeError:
-            return {"size": None, "modified": None, "exists": False}
+        pass
-        return {"size": None, "modified": None, "exists": True}
+    else:
-    finally:
+        raise RuntimeError(
-        loop.close()
+            "get_file_metadata() cannot be used with async storage backends "
            "inside a running event loop — use get_file_metadata_async()"
        )
    return asyncio.run(get_file_metadata_async(path))
@@ -0,0 +1,63 @@
 """Core-owned system MiniApps (Phase M4).
 Host-level MiniApps that are not owned by a single plugin: audit activity
 feed and system metrics. They register in the universal registry with
 ``plugin_name="system"`` at app startup and unregister with the registry
 reset (tests) — they never depend on plugin activation state.
 Permissions follow the owning data source:
 - audit_activity -> audit:read (audit log route guard, CORE_PERMISSIONS)
 - system_metrics  -> settings:read (Roadmap M4; the /system/dashboard
  endpoint itself stays require_admin — the widget degrades gracefully
  with a permission hint for non-admins)
 """
 from __future__ import annotations
 from app.plugins.miniapp_registry import get_miniapp_registry
 SYSTEM_PLUGIN_NAME = "system"
 def register_system_miniapps() -> None:
    """Register the core system MiniApps in the universal registry."""
    registry = get_miniapp_registry()
    registry.register(
        app_id="audit_activity",
        name="Aktivitäten",
        icon="History",
        description="Letzte Aktivitäten aus dem Audit-Log (Benutzer, Aktion, Zeitpunkt).",
        plugin_name=SYSTEM_PLUGIN_NAME,
        permission="audit:read",
        settings_schema={
            "fields": [
                {
                    "name": "max_items",
                    "label": "Max. Einträge",
                    "type": "number",
                    "default": 10,
                }
            ]
        },
        col_span=2,
        row_span=1,
        hosts=["chat", "dashboard", "window"],
        component="@/components/dashboard/AuditActivityWidget",
        order=40,
    )
    registry.register(
        app_id="system_metrics",
        name="System Status",
        icon="Server",
        description="Datenbank-, Redis-, Worker- und API-Metriken (Administration).",
        plugin_name=SYSTEM_PLUGIN_NAME,
        permission="settings:read",
        settings_schema={},
        col_span=2,
        row_span=1,
        hosts=["chat", "dashboard", "window"],
        component="@/components/dashboard/SystemMetricsWidget",
        order=50,
    )
@@ -100,6 +100,13 @@ class TriggerDispatcher:
                trigger_type=trigger_type,
                payload=payload,
            )
            # F-PROACTIVE: Also check for matching agent definitions on context/UI events
            if is_ui_event or event_name.startswith("context."):
                await self._dispatch_matching_agents(
                    event_name=event_name,
                    trigger_type=trigger_type,
                    payload=payload,
                )
        except Exception:
            logger.exception(
                "TriggerDispatcher: error dispatching event '%s'", event_name
@@ -114,11 +121,14 @@ class TriggerDispatcher:
        """Query DB for active automations matching *event_name* and dispatch."""
        from app.core.db import get_session_factory
        from app.plugins.builtins.contracts import get_contract
        # None-check FIRST — accessing attributes on the contract before the
        # check crashed with AttributeError when automation was inactive
        # (ARCH-029/041).
        automation_contract = get_contract("automation")
        AutomationDefinition = automation_contract.Automation  # noqa: N806
        if automation_contract is None:
            logger.debug("Automation plugin not available — trigger skipped")
            return
        AutomationDefinition = automation_contract.Automation  # noqa: N806
        factory = get_session_factory()
        tenant_id = payload.get("tenant_id")
@@ -164,6 +174,72 @@ class TriggerDispatcher:
                trigger_data=payload,
            )
    async def _dispatch_matching_agents(
        self,
        event_name: str,
        trigger_type: str,
        payload: dict[str, Any],
    ) -> None:
        """F-PROACTIVE: Query DB for active agents matching *event_name* and dispatch.
        Checks AgentDefinition.trigger_config for matching context/UI events.
        If match found, creates an AgentRun and dispatches via run_agent.
        """
        from app.core.db import get_session_factory
        from app.plugins.builtins.contracts import get_contract
        automation_contract = get_contract("automation")
        if automation_contract is None:
            return
        AgentDefinition = automation_contract.AgentDefinition  # noqa: N806
        if AgentDefinition is None:
            return
        factory = get_session_factory()
        tenant_id = payload.get("tenant_id")
        async with factory() as db:
            query = (
                select(AgentDefinition)
                .where(AgentDefinition.is_active.is_(True))
                .where(AgentDefinition.mode == "proactive")
            )
            if tenant_id is not None:
                query = query.where(AgentDefinition.tenant_id == tenant_id)
            result = await db.execute(query)
            agents = list(result.scalars().all())
        if not agents:
            return
        for agent in agents:
            config = agent.trigger_config or {}
            configured_event = config.get("event_name", "")
            if configured_event != event_name:
                continue
            logger.info(
                "TriggerDispatcher: dispatching agent '%s' (%s) for event '%s'",
                agent.name,
                agent.id,
                event_name,
            )
            try:
                await automation_contract.run_agent(
                    ctx={},
                    agent_id=str(agent.id),
                    trigger_type=trigger_type,
                    trigger_data=payload,
                )
            except Exception:
                logger.exception(
                    "TriggerDispatcher: run_agent failed for agent_id=%s",
                    agent.id,
                )
    async def _enqueue_automation(
        self,
        automation_id: str,
@@ -7,7 +7,8 @@ import logging
 import uuid
 from typing import Any
-from sqlalchemy import select
+from sqlalchemy import cast, select
 from sqlalchemy.dialects.postgresql import JSONB
 from app.core.db import get_session_factory
 from app.core.event_bus import EventBus, get_event_bus
@@ -48,7 +49,12 @@ async def _dispatch_event(payload: dict[str, Any]) -> None:
        stmt = select(Webhook).where(
            Webhook.tenant_id == tenant_id,
            Webhook.is_active == True,  # noqa: E712
-            Webhook.events.any(event_name),
+            # Webhook.events is a JSONB array column (NOT a relationship):
            # events @> '["<event_name>"]' — JSONB containment instead of
            # the invalid relationship .any() call that crashed every event
            # with "Neither 'AnnotatedColumn' nor 'Comparator' object has an
            # attribute 'any'" (158 failed outbox events in production).
            cast(Webhook.events, JSONB).contains([event_name]),
        )
        result = await db.execute(stmt)
        webhooks = list(result.scalars().all())
@@ -170,12 +170,7 @@ async def on_startup(ctx: dict[str, Any]) -> None:
        if search_contract is not None:
            factory = async_session
            async with factory() as db:
-                # auto_register_providers is not exposed via contract yet;
+                await search_contract.auto_register_providers(db)
                # use the contract's get_search_registry to access providers
                from app.plugins.builtins.unified_search.provider_registry import (
                    auto_register_providers,
                )
                await auto_register_providers(db)
            logger.info("Search providers registered for worker")
        else:
            logger.debug("Unified search plugin not available — skipping provider registration")
@@ -236,7 +231,7 @@ def _lazy_register_plugin_jobs() -> None:
    if not registry.list_discovered():
        registry.discover_builtins()
-    job_modules: list[str] = ["app.core.jobs", "app.services.import_export_jobs"]
+    job_modules: list[str] = ["app.core.jobs", "app.core.backup_job", "app.services.import_export_jobs"]
    for plugin_name in registry.list_discovered():
        plugin = registry.get_plugin(plugin_name)
        if plugin is None:
@@ -342,6 +337,110 @@ async def cleanup_outbox_job(ctx: dict[str, Any]) -> None:
 register_job("cleanup_outbox", cleanup_outbox_job)
 # ── Audit log retention cleanup job ─────────────────────────────────────────
 async def cleanup_audit_log_job(ctx: dict[str, Any]) -> None:
    """Delete audit log entries older than 365 days.
    Runs daily to prevent the audit_log table from growing indefinitely.
    Iterates per-tenant for RLS compliance.
    """
    from datetime import UTC, datetime, timedelta
    from sqlalchemy import delete as sa_delete
    from sqlalchemy import text as sa_text
    from app.core.db import get_worker_session_factory
    from app.models.audit import AuditLog
    factory = get_worker_session_factory()
    async with factory() as db:
        try:
            tenant_result = await db.execute(sa_text("SELECT id FROM tenants"))
            tenant_ids = [row[0] for row in tenant_result]
            cutoff = datetime.now(UTC) - timedelta(days=365)
            total_deleted = 0
            for tenant_id in tenant_ids:
                await db.execute(
                    sa_text("SELECT set_config('app.current_tenant_id', :tid, true)"),
                    {"tid": str(tenant_id)},
                )
                result = await db.execute(
                    sa_delete(AuditLog).where(AuditLog.timestamp < cutoff)
                )
                total_deleted += result.rowcount
                await db.commit()
            if total_deleted:
                logger.info("Audit retention: cleaned up %d old entries", total_deleted)
        except Exception:
            logger.error("Audit retention cleanup failed", exc_info=True)
            await db.rollback()
 register_job("cleanup_audit_log", cleanup_audit_log_job)
 # ── Trash cleanup job ───────────────────────────────────────────────────────
 async def cleanup_trash_job(ctx: dict[str, Any]) -> None:
    """Permanently delete soft-deleted records older than 90 days.
    Runs daily to clean up the trash. Iterates per-tenant for RLS compliance.
    Default retention: 90 days in trash before permanent deletion.
    """
    from datetime import UTC, datetime, timedelta
    from sqlalchemy import delete as sa_delete
    from sqlalchemy import text as sa_text
    from app.core.db import get_worker_session_factory
    from app.models.entity_attachment import EntityAttachment
    factory = get_worker_session_factory()
    async with factory() as db:
        try:
            tenant_result = await db.execute(sa_text("SELECT id FROM tenants"))
            tenant_ids = [row[0] for row in tenant_result]
            cutoff = datetime.now(UTC) - timedelta(days=90)
            total_deleted = 0
            for tenant_id in tenant_ids:
                await db.execute(
                    sa_text("SELECT set_config('app.current_tenant_id', :tid, true)"),
                    {"tid": str(tenant_id)},
                )
                # Delete soft-deleted entity attachments
                # (Contacts trash cleanup moved to the contacts plugin:
                # cleanup_contacts_trash — audit P2, no core->contacts import)
                result = await db.execute(
                    sa_delete(EntityAttachment).where(
                        EntityAttachment.deleted_at.is_not(None),
                        EntityAttachment.deleted_at < cutoff,
                    )
                )
                total_deleted += result.rowcount
                await db.commit()
            if total_deleted:
                logger.info("Trash cleanup: permanently deleted %d old records", total_deleted)
        except Exception:
            logger.error("Trash cleanup failed", exc_info=True)
            await db.rollback()
 register_job("cleanup_trash", cleanup_trash_job)
 # Note: knowledge retention cleanup ("cleanup_knowledge") lives with the
 # knowledge plugin (app/plugins/builtins/knowledge/jobs.py) and is discovered
 # via the plugin job-module mechanism — no core→plugin import.
 class WorkerSettings:
    """ARQ worker settings."""
    functions = get_all_jobs()
@@ -371,4 +470,31 @@ class WorkerSettings:
            _wrap_cron_with_lock("cleanup_outbox", cleanup_outbox_job, ttl_seconds=300),
            minute=0,
        ),
        # Audit log retention cleanup — daily at 03:00
        cron(
            _wrap_cron_with_lock("cleanup_audit_log", cleanup_audit_log_job, ttl_seconds=300),
            hour=3, minute=0,
        ),
        # Trash cleanup — daily at 04:00 (90 days retention)
        cron(
            _wrap_cron_with_lock("cleanup_trash", cleanup_trash_job, ttl_seconds=300),
            hour=4, minute=0,
        ),
        # Contacts trash cleanup — daily at 04:15, owned by the contacts
        # plugin (audit P2: no core->contacts import in the worker).
        cron(
            _wrap_cron_with_lock("cleanup_contacts_trash", get_job("cleanup_contacts_trash"), ttl_seconds=300),
            hour=4, minute=15,
        ),
        # Knowledge retention cleanup — daily at 05:00 (90 days, keeps approved).
        # Function comes from the knowledge plugin via the job registry.
        cron(
            _wrap_cron_with_lock("cleanup_knowledge", get_job("cleanup_knowledge"), ttl_seconds=300),
            hour=5, minute=0,
        ),
        # Scheduled backup — daily at 02:00 (guarded by distributed lock)
        cron(
            _wrap_cron_with_lock("run_backup", get_job("run_backup"), ttl_seconds=600),
            hour=2, minute=0,
        ),
    ]
@@ -7,7 +7,7 @@ import uuid
 from typing import Any
 import redis.asyncio as aioredis
-from fastapi import Depends, HTTPException, Request, status
+from fastapi import Depends, Header, HTTPException, Request, status
 from sqlalchemy import select
 from sqlalchemy.ext.asyncio import AsyncSession
@@ -17,8 +17,9 @@ from app.core.db import get_db, set_tenant_context, set_user_context
 logger = logging.getLogger(__name__)
-# Known write-permission modules — used by require_write() to check
+# Legacy fallback list — used by require_write() only when the permission
-# specific permissions instead of broad wildcards like *:write
+# registry is not initialized. The live source of truth is generated from
 # the registry (see _get_write_permissions, ARCH-022).
 _WRITE_PERMISSIONS = [
    "users:write",
    "roles:write",
@@ -35,6 +36,30 @@ _WRITE_PERMISSIONS = [
 ]
 def _get_write_permissions() -> list[str]:
    """Return all known ``module:write`` permission keys (ARCH-022).
    Generated from the permission registry so plugin write permissions are
    picked up automatically without touching this file. Falls back to the
    static legacy list when the registry is unavailable/uninitialized.
    """
    try:
        from app.core.permission_registry import get_permission_registry
        registry = get_permission_registry()
        if getattr(registry, "_initialized", False):
            perms = [
                entry["key"]
                for entry in registry.get_all()
                if entry["key"].endswith(":write")
            ]
            if perms:
                return sorted(perms)
    except Exception:
        pass
    return list(_WRITE_PERMISSIONS)
 async def get_redis_dep() -> aioredis.Redis:
    """FastAPI dependency for Redis client."""
    return get_redis()
@@ -49,7 +74,62 @@ async def get_current_user(
    Returns session data dict with user_id, tenant_id, email, name, role,
    and resolved permissions from Redis cache.
    F09 (Astra P1): also accepts a short-lived HMAC-signed delegation
    token (X-Delegation-Token header) for INTERNAL calls made on behalf
    of a user — e.g. the generic CRM-API tool used by AI agents and the
    MCP server. Previously those tools sent unauthenticated
    X-Internal-Call headers that the protected API never accepted.
    """
    # F09: internal delegation path — HMAC-signed, max 60 seconds
    delegation_header = request.headers.get("X-Delegation-Token", "")
    if delegation_header:
        from app.core.delegation_token import verify_delegation_token
        payload = verify_delegation_token(delegation_header)
        if payload is None:
            raise HTTPException(
                status_code=status.HTTP_401_UNAUTHORIZED,
                detail={"detail": "Invalid or expired delegation token", "code": "delegation_invalid"},
            )
        deleg_user_id = str(payload.get("user_id", ""))
        deleg_tenant_id = str(payload.get("tenant_id", ""))
        if not deleg_user_id or not deleg_tenant_id:
            raise HTTPException(
                status_code=status.HTTP_401_UNAUTHORIZED,
                detail={"detail": "Delegation token missing user/tenant", "code": "delegation_invalid"},
            )
        tenant_id = uuid.UUID(deleg_tenant_id)
        user_id = uuid.UUID(deleg_user_id)
        await set_tenant_context(db, tenant_id)
        from app.core.permissions import get_cached_permissions
        resolved = await get_cached_permissions(db, redis, user_id, tenant_id)
        from sqlalchemy import select as _select
        from app.models.group import UserGroup
        groups_q = await db.execute(
            _select(UserGroup.group_id)
            .where(UserGroup.user_id == user_id)
            .where(UserGroup.tenant_id == tenant_id)
        )
        group_ids = [row[0] for row in groups_q]
        is_admin = bool(resolved.get("is_system_admin", False))
        await set_user_context(db, user_id, group_ids, is_admin)
        return {
            "user_id": deleg_user_id,
            "tenant_id": deleg_tenant_id,
            "email": "",  # not needed for permission decisions
            "name": "delegated",
            "role": "",
            "permissions": resolved.get("permissions", []),
            "denied_permissions": resolved.get("denied", []),
            "field_permissions": resolved.get("field_permissions", {}),
            "is_system_admin": is_admin,
            "delegated_by": payload.get("agent_id", ""),
        }
    settings = get_settings()
    session_id = request.cookies.get(settings.session_cookie_name)
@@ -105,10 +185,20 @@ async def get_current_user(
    membership_row = membership_q.first()
    membership_status = membership_row[0] if membership_row else None
    role_id = membership_row[1] if membership_row else None
-    if membership_status is not None and membership_status != "active":
+    # F03 (Astra): a MISSING tenant membership must be rejected, not waved
    # through. Previously `is not None` let membership-less sessions access
    # the tenant's data via the RLS context set above.
    if membership_status is None or membership_status != "active":
        raise HTTPException(
            status_code=status.HTTP_403_FORBIDDEN,
-            detail={"detail": f"Mitgliedschaft ist {membership_status}, Zugriff verweigert", "code": "membership_suspended"},
+            detail={
                "detail": (
                    f"Mitgliedschaft ist {membership_status}, Zugriff verweigert"
                    if membership_status
                    else "Keine aktive Mandanten-Mitgliedschaft, Zugriff verweigert"
                ),
                "code": "membership_suspended",
            },
        )
    # Cache user principals for this request — avoids N+1 queries in visibility.py
@@ -125,6 +215,8 @@ async def get_current_user(
    user_id = uuid.UUID(session_data["user_id"])
    resolved = await get_cached_permissions(db, redis, user_id, tenant_id)
    if not resolved:
        resolved = {"permissions": [], "denied": [], "field_permissions": {}, "is_system_admin": False}
    session_data["permissions"] = resolved.get("permissions", [])
    session_data["denied_permissions"] = resolved.get("denied", [])
    session_data["field_permissions"] = resolved.get("field_permissions", {})
@@ -220,6 +312,54 @@ async def get_current_user_or_bearer(
    return await get_current_user(request, db, redis)
 def require_permission_or_bearer(permission: str):
    """F08 (Astra P1): permission dependency for routes that serve BOTH
    session-cookie clients (SPA) and pure Bearer API clients.
    ``require_permission`` resolves via ``get_current_user`` (session
    cookie only) — a Bearer client fails with 401 before the route's own
    Bearer verification is ever reached. This dependency accepts either
    auth path and enforces the SAME effective permission:
    - session users: normal permission check
    - Bearer tokens: token scopes are an UPPER BOUND (F10) — the user's
      own permissions must grant the permission AND the scope must match
    """
    async def _check(
        current_user: dict[str, Any] = Depends(get_current_user_or_bearer),
    ) -> dict[str, Any]:
        token_scopes = current_user.get("_token_scopes")
        if token_scopes is not None:
            from app.core.permissions import _permission_matches_any
            if not _permission_matches_any(set(token_scopes), permission):
                raise HTTPException(
                    status_code=status.HTTP_403_FORBIDDEN,
                    detail={
                        "detail": f"Token scope '{permission}' required",
                        "code": "insufficient_scope",
                    },
                )
            # fall through — user permissions apply too (F10 semantics)
        if current_user.get("is_system_admin"):
            return current_user
        from app.core.permissions import check_permission
        if check_permission(current_user, permission):
            return current_user
        raise HTTPException(
            status_code=status.HTTP_403_FORBIDDEN,
            detail={
                "detail": f"Permission '{permission}' required",
                "code": "forbidden",
            },
        )
    return _check
 async def require_admin(
    current_user: dict[str, Any] = Depends(get_current_user),
 ) -> dict[str, Any]:
@@ -259,7 +399,7 @@ async def require_write(
    # Check via permission system for specific write permissions
    from app.core.permissions import check_permission
-    for perm in _WRITE_PERMISSIONS:
+    for perm in _get_write_permissions():
        if check_permission(current_user, perm):
            return current_user
@@ -284,6 +424,10 @@ def require_permission(permission: str):
        current_user: dict[str, Any] = Depends(get_current_user),
    ) -> dict[str, Any]:
        # API token scope enforcement (Problem 2 fix)
        # F10 (Astra): token scopes are an UPPER BOUND, not a replacement —
        # the user's own permissions must ALSO grant the permission. A token
        # can never grant more than its owner has; revoking the user's
        # permission takes effect on existing tokens.
        token_scopes = current_user.get("_token_scopes")
        if token_scopes is not None:
            from app.core.permissions import _permission_matches_any
@@ -295,7 +439,7 @@ def require_permission(permission: str):
                        "code": "insufficient_scope",
                    },
                )
-            return current_user
+            # fall through: the normal user-permission check applies too
        if current_user.get("is_system_admin"):
            return current_user
@@ -355,6 +499,30 @@ async def get_current_user_id(
    return uuid.UUID(current_user["user_id"])
 def require_workspace_scope(module_key: str):
    """FastAPI dependency factory (Phase N3): resolve the active workspace
    scope config for a module from the X-Workspace-ID header.
    Returns the scope dict (e.g. ``{"folder_ids": [...]}``) or ``None``
    when no restriction applies (no header, admin, unassigned, empty config).
    Callers apply it as a pure AND-restriction — never a grant.
    Usage:
        scope: dict | None = Depends(require_workspace_scope("contacts"))
    """
    async def _resolve(
        db: AsyncSession = Depends(get_db),
        current_user: dict[str, Any] = Depends(get_current_user),
        x_workspace_id: str | None = Header(None, alias="X-Workspace-ID"),
    ) -> dict[str, Any] | None:
        from app.services.workspace_scope_service import resolve_workspace_scope
        return await resolve_workspace_scope(db, current_user, x_workspace_id, module_key)
    return _resolve
 def require_active_plugin(plugin_name: str):
    """FastAPI dependency factory: require that a plugin is active.
@@ -373,7 +541,16 @@ def require_active_plugin(plugin_name: str):
    """
    async def _check(
        db: AsyncSession = Depends(get_db),
        current_user: dict[str, Any] = Depends(get_current_user_or_bearer),
    ) -> None:
        """F05 (Astra P1): the plugin gate runs AFTER authentication.
        Depending on ``get_current_user_or_bearer`` guarantees FastAPI
        resolves the authenticated user context BEFORE this check — the
        previous version read the tenant from the DB session before auth
        had run (context missing → silent allow). Both auth paths (cookie
        and Bearer) set the tenant context on this same ``db`` session.
        """
        from app.core.permission_registry import get_permission_registry
        try:
            registry = get_permission_registry()
@@ -385,19 +562,20 @@ def require_active_plugin(plugin_name: str):
                        "code": "plugin_inactive",
                    },
                )
-            # Get tenant_id from existing db session (NOT a new session)
+            # Tenant comes from the AUTHENTICATED user context — never from
-            # The tenant context is set by middleware/get_current_user on this same session
+            # the DB session (which may not have the context set yet).
-            from sqlalchemy import text as sa_text
+            raw_tid = current_user.get("tenant_id")
-
+            if not raw_tid:
-            result = await db.execute(
+                # Fail-closed: no authenticated tenant context → reject.
-                sa_text("SELECT NULLIF(current_setting('app.current_tenant_id', true), '')::uuid")
+                # (Previously this returned silently = plugin active.)
-            )
+                raise HTTPException(
-            tenant_id = result.scalar()
+                    status_code=status.HTTP_403_FORBIDDEN,
-
+                    detail={
-            if tenant_id is None:
+                        "detail": "Plugin gate requires an authenticated tenant context",
-                # No tenant context — plugin is active by default (backward compatible)
+                        "code": "plugin_gate_no_tenant",
-                # TODO: Fix in production to deny access when no tenant context
+                    },
-                return
+                )
            tenant_id = uuid.UUID(str(raw_tid))
            # Per-tenant activation check with Redis cache
            import json
@@ -14,7 +14,7 @@ from contextlib import asynccontextmanager
 import structlog
 from fastapi import Depends, FastAPI, HTTPException, Request
 from fastapi.middleware.cors import CORSMiddleware
-from fastapi.responses import FileResponse, JSONResponse
+from fastapi.responses import FileResponse, JSONResponse, PlainTextResponse
 from fastapi.staticfiles import StaticFiles
 from starlette.middleware.base import BaseHTTPMiddleware
@@ -22,8 +22,12 @@ logger = logging.getLogger(__name__)
 from app.config import get_settings  # noqa: E402
 from app.core.db import close_engine, get_engine  # noqa: E402
-from app.core.error_codes import ApiError, build_error_response  # noqa: E402
+from app.core.error_codes import ERROR_CODES, ApiError, build_error_response  # noqa: E402
-from app.core.middleware import CSRFMiddleware, SecurityHeadersMiddleware  # noqa: E402
+from app.core.middleware import (  # noqa: E402
    AuditMiddleware,
    CSRFMiddleware,
    SecurityHeadersMiddleware,
 )
 from app.core.monitoring import record_error, record_request  # noqa: E402
 from app.core.rate_limit import GeneralRateLimitMiddleware  # noqa: E402
 from app.core.resilience import CircuitBreakerMiddleware  # noqa: E402
@@ -31,20 +35,19 @@ from app.core.service_container import get_container  # noqa: E402
 from app.plugins.registry import get_registry  # noqa: E402
 from app.routes import (  # noqa: E402
    addresses,
    ai_copilot,
    api_tokens,
    approvals,
    attachments,
    audit,
    auth,
    backups,
    bank_accounts,
-    contact_folder_permissions,
+    compliance,
    contact_folders,
    contacts,
    currencies,
    custom_field_definitions,
    custom_fields,
    dashboard,
    dashboards,
    delegations,
    entity_history,
    entity_permissions,
    errors,
@@ -53,17 +56,18 @@ from app.routes import (  # noqa: E402
    health,
    import_export,
    metrics,
    miniapps,
    notifications,
    outbox,
    owner_transfer,
    permission_templates,
    plugins,
    # delegations,  # ⏸ Parked — not integrated into resolve_permissions()
    policies,
    roles,
    saved_filters,
    saved_views,
    sequences,
    system_dashboard,
    system_settings,
    taxes,
    tenants,
@@ -214,6 +218,16 @@ async def lifespan(app: FastAPI):
    registry.initialize(get_migration_engine(), app)
    registry.discover_builtins()
    # Core system MiniApps (Phase M4): host-level, independent of plugin state
    from app.core.system_miniapps import register_system_miniapps
    register_system_miniapps()
    # Core AI agent tools for MiniApp output (Phase M6)
    from app.ai.miniapp_tools import register_miniapp_tools
    register_miniapp_tools()
    # Install discovered builtin plugins and activate only those marked active in DB
    from sqlalchemy import select as sa_select
    from sqlalchemy.ext.asyncio import async_sessionmaker
@@ -282,22 +296,23 @@ async def lifespan(app: FastAPI):
                logger.info(f"Plugin {name} is inactive — skipping activation")
                continue
-            # Activate plugin with a FRESH session per plugin to avoid RLS state leakage
+            # Activate plugin ONCE per process (ARCH-002 fix): a fresh session with
-            # RLS fail-closed requires app.current_tenant_id for tenant-table writes.
+            # the first tenant's RLS context satisfies fail-closed RLS for any
-            # Plugin activation may fail on duplicate cron job inserts — this is harmless
+            # tenant-table writes during activation. Plugins that need per-tenant
-            # since cron jobs already exist from previous startups.
+            # data must seed it themselves (e.g. via the default-tenant mechanism).
            # Calling on_activate once prevents duplicate event listeners, cron
            # jobs, mini-apps and other contributions at multi-tenant startups.
            plugin_activated = False
-            for tenant_id in all_tenant_ids:
+            if all_tenant_ids:
                try:
                    async with async_session() as plugin_db:
-                        await set_tenant_context(plugin_db, tenant_id)
+                        await set_tenant_context(plugin_db, all_tenant_ids[0])
                        await plugin.on_activate(plugin_db, container, event_bus)
                        await plugin_db.flush()
                        await plugin_db.commit()
                    plugin_activated = True
                except Exception as exc:
-                    logger.warning(f"[STARTUP] Plugin {name} activation issue for tenant {tenant_id}: {exc}")
+                    logger.warning(f"[STARTUP] Plugin {name} activation issue: {exc}")
                    break
            if plugin_activated:
                plugin_record.status = "active"
@@ -322,6 +337,22 @@ async def lifespan(app: FastAPI):
            if plugin and plugin.manifest.permissions:
                register_plugin_permissions(record.name, plugin.manifest.permissions)
        # Audit P1 (contract lazy loading, restart edge case): plugins that
        # were already inactive in the DB when this process started never get
        # a runtime deactivate() call, so the ContractRegistry would
        # lazy-load their contracts module and resurrect the contract.
        # Mark them once here so get_contract() fails closed for them.
        inactive_result = await db.execute(
            sa_select(PluginModel.name).where(PluginModel.active == False)  # noqa: E712
        )
        inactive_names = {row[0] for row in inactive_result}
        if inactive_names:
            from app.plugins.builtins.contracts import get_contract_registry
            get_contract_registry().mark_db_inactive(inactive_names)
            logger.info(
                "Contract registry: %d plugins marked DB-inactive", len(inactive_names)
            )
    init_permission_registry(active_plugin_names)
    logger.info("Permission registry initialized with %d active plugins", len(active_plugin_names))
@@ -344,7 +375,7 @@ async def lifespan(app: FastAPI):
        plugin = registry.get_plugin(name)
        if plugin:
            for entity_type, model_class in plugin.get_entity_models().items():
-                register_entity_model(entity_type, model_class)
+                register_entity_model(entity_type, model_class, plugin_name=name)
    logger.info("Entity models registered for %d active plugins", len(active_plugin_names))
    # Register field definitions from active plugins only
@@ -434,7 +465,6 @@ def create_app() -> FastAPI:
            {"name": "entity-history", "description": "Audit trail and entity change history."},
            {"name": "import-export", "description": "Bulk import and export of contacts and data."},
            {"name": "plugins", "description": "Plugin management: list, install, activate, deactivate."},
            {"name": "ai-copilot", "description": "AI copilot: chat, suggestions, conversation history."},
            {"name": "workflows", "description": "Workflow definitions, instances, and execution."},
            {"name": "user-preferences", "description": "Per-user preference settings."},
            {"name": "currencies", "description": "Currency management for multi-currency support."},
@@ -472,6 +502,7 @@ def create_app() -> FastAPI:
    )
    app.add_middleware(CSRFMiddleware)
    app.add_middleware(SecurityHeadersMiddleware)
    app.add_middleware(AuditMiddleware)
    app.add_middleware(GeneralRateLimitMiddleware)
    app.add_middleware(RequestLoggingMiddleware)
    app.add_middleware(CircuitBreakerMiddleware)
@@ -514,11 +545,26 @@ def create_app() -> FastAPI:
            504: "service_timeout",
        }
        code = status_to_code.get(exc.status_code, "internal_error" if exc.status_code >= 500 else "validation_error")
-        body = build_error_response(
+        # Structured detail passthrough (AGENTS.md): when a route raises
-            code=code,
+        # HTTPException with a dict detail containing a machine-readable ``code``,
-            detail=str(exc.detail) if exc.detail else None,
+        # preserve the structured shape instead of stringifying it.
-            trace_id=trace_id,
+        raw_detail = exc.detail
-        )
+        if isinstance(raw_detail, dict):
            inner_code = raw_detail.get("code", code)
            body = build_error_response(
                code=inner_code if inner_code in ERROR_CODES else code,
                detail=raw_detail.get("detail") or str(raw_detail),
                trace_id=trace_id,
            )
            # Preserve the full structured detail as a nested object so clients
            # can read ``resp.json()["detail"]["code"]``.
            body["detail"] = raw_detail
        else:
            body = build_error_response(
                code=code,
                detail=str(exc.detail) if exc.detail else None,
                trace_id=trace_id,
            )
        resp = JSONResponse(status_code=exc.status_code, content=body)
        if trace_id:
            resp.headers["X-Trace-Id"] = trace_id
@@ -542,42 +588,43 @@ def create_app() -> FastAPI:
    app.include_router(groups.router)
    app.include_router(tenants.router)
    app.include_router(notifications.router)
-    from app.routes.companies import router as companies_router
+    # NOTE: contacts/companies/contact-folders routes are plugin-owned now
-    app.include_router(companies_router)
+    # (Block B1) and mounted via the manifest.routes mechanism below with
-    app.include_router(contacts.router)
+    # require_active_plugin("contacts") protection.
    app.include_router(contact_folders.router)
    app.include_router(contact_folder_permissions.router)
    app.include_router(entity_permissions.router)
    app.include_router(dashboard.router)
    app.include_router(dashboards.router)
    app.include_router(entity_history.router)
    app.include_router(import_export.router)
    app.include_router(plugins.router)
    app.include_router(ai_copilot.router)
    app.include_router(workflows.router)
    app.include_router(user_preferences.router)
    app.include_router(currencies.router)
    app.include_router(taxes.router)
    app.include_router(sequences.router)
    app.include_router(system_dashboard.router)
    app.include_router(system_settings.router)
    app.include_router(attachments.router)
    app.include_router(addresses.router)
    app.include_router(bank_accounts.router)
    app.include_router(audit.router)
    app.include_router(backups.router)
    app.include_router(compliance.router)
    app.include_router(owner_transfer.router)
    app.include_router(custom_field_definitions.router)
    app.include_router(custom_fields.router)
    app.include_router(saved_filters.router)
    app.include_router(saved_views.router)
    app.include_router(webhooks.router)
    app.include_router(permission_templates.router)
-    # app.include_router(delegations.router)  # ⏸ Parked — not integrated into resolve_permissions()
+    app.include_router(delegations.router)
    app.include_router(policies.router)
    app.include_router(errors.router)
    app.include_router(guests.router)  # ⚠️ Guest-System umgebaut — Guests sind jetzt reguläre User mit role=guest
    app.include_router(workspaces.router)
    app.include_router(outbox.router)
    app.include_router(api_tokens.router)
    app.include_router(approvals.router)
    app.include_router(miniapps.router)
    # ── Register plugin routes for all discovered plugins ──
    # Routes are registered at app creation time so OpenAPI docs are complete.
@@ -635,9 +682,34 @@ def create_app() -> FastAPI:
            blocked_prefixes = ("var/log/", "error/", "error_log", "var/", "etc/", "proc/", "sys/")
            if full_path.startswith(blocked_prefixes) or ".." in full_path:
                raise HTTPException(status_code=404, detail="Not Found")
            # Kill switch for old PWA service workers — return self-unregistering SW
            if full_path in ("sw.js", "service-worker.js"):
                return PlainTextResponse(
                    content="""// Kill switch — unregister all service workers
 self.addEventListener('install', (e) => { self.skipWaiting(); });
 self.addEventListener('activate', (e) => {
  e.waitUntil(
    self.registration.unregister().then(() => {
      console.log('Service Worker unregistered');
      return self.clients.claim();
    })
  );
 });
 self.addEventListener('fetch', (e) => {
  e.respondWith(fetch(e.request).catch(() => new Response('', {status: 504})));
 });
 """,
                    media_type="application/javascript",
                    headers={"Cache-Control": "no-cache, no-store, must-revalidate"},
                )
            index_path = os.path.join(frontend_dist, "index.html")
            if os.path.isfile(index_path):  # noqa: ASYNC240
-                return FileResponse(index_path)
+                return FileResponse(
                    index_path,
                    headers={"Cache-Control": "no-cache, no-store, must-revalidate"},
                )
            raise HTTPException(status_code=404, detail="Frontend not built")
    return app
@@ -1,23 +1,29 @@
 """SQLAlchemy models for LeoCRM."""
 from app.models.address import Address
 from app.models.ai_conversation import AIConversation, AIMessage
 from app.models.attachment import Attachment
 from app.models.audit import AuditLog
 from app.models.auth import ApiToken, PasswordResetToken
 from app.models.backup import Backup
 from app.models.bank_account import BankAccount
 from app.models.compliance import ComplianceIncident
 from app.models.consumer_inbox import ConsumerInbox
-from app.models.contact import Contact, ContactPerson
+
 # Contact/ContactPerson: lazy via package __getattr__ (Paket 6) — the physical
 # model lives in app.plugins.builtins.contacts.models; importing the plugin
 # framework while app.models is still initializing caused a proven circular
 # ImportError (app.core.auth -> app.models.session -> ... -> app.plugins).
 from app.models.contact_folder import ContactFolder
 from app.models.contact_merge import ContactMergeHistory
 from app.models.currency import Currency
 from app.models.custom_field_definition import CustomFieldDefinition
 from app.models.dashboard import Dashboard
 from app.models.entity_history import EntityHistory
 from app.models.entity_permission import EntityPermission
 from app.models.entity_policy import EntityPolicy
 from app.models.group import Group, UserGroup
 from app.models.notification import Notification, NotificationPreference, NotificationType
 from app.models.outbox import EventOutbox, OutboxDelivery  # noqa: F401
 from app.models.owned_mixin import OwnedMixin
 from app.models.permission_delegation import PermissionDelegation
 from app.models.permission_template import PermissionTemplate
@@ -47,6 +53,7 @@ __all__ = [
    "NotificationPreference",
    "PasswordResetToken",
    "ApiToken",
    "ComplianceIncident",
    "Contact",
    "ContactPerson",
    "ContactFolder",
@@ -67,8 +74,6 @@ __all__ = [
    "BankAccount",
    "Plugin",
    "PluginMigration",
    "AIConversation",
    "AIMessage",
    "Backup",
    "CustomFieldDefinition",
    "Webhook",
@@ -76,6 +81,7 @@ __all__ = [
    "WorkflowInstance",
    "WorkflowStepHistory",
    "SavedView",
    "Dashboard",
 ]
 from app.models.entity_attachment import EntityAttachment  # noqa: F401
 from app.models.workspace import (  # noqa: F401
@@ -84,3 +90,17 @@ from app.models.workspace import (  # noqa: F401
    WorkspaceUser,
    WorkspaceWidget,
 )
 # ── Lazy Contact re-export (Paket 6, #357) ──────────────────────────────────
 # The physical home of Contact/ContactPerson is the ContactsPlugin
 # (app.plugins.builtins.contacts.models). Resolving them lazily via package
 # __getattr__ keeps ``from app.models import *`` (alembic/env.py) working
 # while avoiding a plugin-framework import during app.models initialization
 # (proven circular ImportError, see app/models/contact.py).
 def __getattr__(name: str):
    if name in {"Contact", "ContactPerson"}:
        from app.models.contact import Contact, ContactPerson
        return {"Contact": Contact, "ContactPerson": ContactPerson}[name]
    raise AttributeError(f"module {__name__!r} has no attribute {name!r}")
@@ -1,52 +0,0 @@
 """AI Conversation and Message models — tenant-scoped with RLS."""
 from __future__ import annotations
 import uuid
 from typing import Any
 from sqlalchemy import ForeignKey, Index, Integer, String, Text
 from sqlalchemy.dialects.postgresql import JSONB
 from sqlalchemy.dialects.postgresql import UUID as PGUUID
 from sqlalchemy.orm import Mapped, mapped_column
 from app.core.db import Base, TenantMixin
 class AIConversation(Base, TenantMixin):
    """AI Copilot conversation thread — tenant-scoped."""
    __tablename__ = "ai_conversations"
    __table_args__ = (Index("ix_ai_conversations_tenant_user", "tenant_id", "user_id"),)
    id: Mapped[uuid.UUID] = mapped_column(
        PGUUID(as_uuid=True), primary_key=True, default=uuid.uuid4
    )
    user_id: Mapped[uuid.UUID] = mapped_column(
        PGUUID(as_uuid=True), ForeignKey("users.id", ondelete="CASCADE"), nullable=False, index=True
    )
    title: Mapped[str] = mapped_column(String(255), nullable=False, default="Untitled")
    context: Mapped[dict[str, Any]] = mapped_column(JSONB, default=dict, nullable=False)
 class AIMessage(Base, TenantMixin):
    """Individual messages within an AI conversation — user input, AI response, actions."""
    __tablename__ = "ai_messages"
    __table_args__ = (Index("ix_ai_messages_tenant_conversation", "tenant_id", "conversation_id"),)
    id: Mapped[uuid.UUID] = mapped_column(
        PGUUID(as_uuid=True), primary_key=True, default=uuid.uuid4
    )
    conversation_id: Mapped[uuid.UUID] = mapped_column(
        PGUUID(as_uuid=True),
        ForeignKey("ai_conversations.id", ondelete="CASCADE"),
        nullable=False,
        index=True,
    )
    role: Mapped[str] = mapped_column(String(20), nullable=False)  # user, assistant, system
    content: Mapped[str] = mapped_column(Text, nullable=False)
    proposed_actions: Mapped[list[dict[str, Any]] | None] = mapped_column(JSONB, nullable=True)
    executed_action: Mapped[dict[str, Any] | None] = mapped_column(JSONB, nullable=True)
    execution_result: Mapped[dict[str, Any] | None] = mapped_column(JSONB, nullable=True)
    message_index: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
@@ -11,20 +11,22 @@ from datetime import datetime
 from typing import Any
 from sqlalchemy import DateTime, ForeignKey, String, func
-from sqlalchemy.dialects.postgresql import JSONB
+from sqlalchemy.dialects.postgresql import JSONB, TSVECTOR
 from sqlalchemy.dialects.postgresql import UUID as PGUUID
 from sqlalchemy.orm import Mapped, mapped_column
 from app.core.db import Base, TenantMixin
 from app.models.owned_mixin import OwnedMixin
 # Re-export EntityHistory as DeletionLog for backward compatibility.
 # Tests import DeletionLog from app.models.audit and use entity_snapshot attribute.
-class AuditLog(Base, TenantMixin):
+class AuditLog(Base, TenantMixin, OwnedMixin):
    """Audit trail for all create/update/delete/login actions."""
    __tablename__ = "audit_log"
    search_tsv: Mapped[Any] = mapped_column(TSVECTOR, nullable=True)
    id: Mapped[uuid.UUID] = mapped_column(
        PGUUID(as_uuid=True), primary_key=True, default=uuid.uuid4
@@ -11,9 +11,10 @@ from sqlalchemy.dialects.postgresql import UUID as PGUUID
 from sqlalchemy.orm import Mapped, mapped_column
 from app.core.db import Base, TenantMixin
 from app.models.owned_mixin import OwnedMixin
-class PasswordResetToken(Base, TenantMixin):
+class PasswordResetToken(Base, TenantMixin, OwnedMixin):
    """Token for password reset flow."""
    __tablename__ = "password_reset_tokens"
@@ -29,7 +30,7 @@ class PasswordResetToken(Base, TenantMixin):
    used_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
-class ApiToken(Base, TenantMixin):
+class ApiToken(Base, TenantMixin, OwnedMixin):
    """API token for programmatic access."""
    __tablename__ = "api_tokens"
@@ -10,9 +10,10 @@ from sqlalchemy.dialects.postgresql import UUID as PGUUID
 from sqlalchemy.orm import Mapped, mapped_column
 from app.core.db import Base, TenantMixin
 from app.models.owned_mixin import OwnedMixin
-class Backup(Base, TenantMixin):
+class Backup(Base, TenantMixin, OwnedMixin):
    """Database backup record scoped to a tenant.
    Tracks pg_dump backups with status, file location, and error details.
@@ -0,0 +1,59 @@
 """Compliance models — AI/privacy incident register for EU compliance."""
 from __future__ import annotations
 import uuid
 from datetime import datetime
 from typing import Any
 from sqlalchemy import DateTime, ForeignKey, Index, String, Text
 from sqlalchemy.dialects.postgresql import JSONB
 from sqlalchemy.dialects.postgresql import UUID as PGUUID
 from sqlalchemy.orm import Mapped, mapped_column
 from app.core.db import Base, TenantMixin
 class ComplianceIncident(Base, TenantMixin):
    """An AI/privacy/security incident tracked for compliance purposes.
    Used by the compliance routes under /api/v1/compliance/incidents.
    """
    __tablename__ = "compliance_incidents"
    __table_args__ = (
        Index("ix_compliance_incidents_tenant_status", "tenant_id", "status"),
        Index("ix_compliance_incidents_tenant_type", "tenant_id", "incident_type"),
    )
    id: Mapped[uuid.UUID] = mapped_column(
        PGUUID(as_uuid=True), primary_key=True, default=uuid.uuid4
    )
    incident_type: Mapped[str] = mapped_column(
        String(30), nullable=False, default="ai"
    )
    title: Mapped[str] = mapped_column(String(300), nullable=False)
    description: Mapped[str] = mapped_column(Text, nullable=False, default="")
    affected_use_cases: Mapped[list[Any]] = mapped_column(
        JSONB, nullable=False, default=list
    )
    affected_versions: Mapped[list[Any]] = mapped_column(
        JSONB, nullable=False, default=list
    )
    provider: Mapped[str] = mapped_column(String(100), nullable=False, default="")
    measures_taken: Mapped[str] = mapped_column(Text, nullable=False, default="")
    evidence_refs: Mapped[list[Any]] = mapped_column(
        JSONB, nullable=False, default=list
    )
    status: Mapped[str] = mapped_column(
        String(20), nullable=False, default="open"
    )
    created_by: Mapped[uuid.UUID | None] = mapped_column(
        PGUUID(as_uuid=True), ForeignKey("users.id", ondelete="SET NULL"), nullable=True
    )
    resolved_by: Mapped[uuid.UUID | None] = mapped_column(
        PGUUID(as_uuid=True), ForeignKey("users.id", ondelete="SET NULL"), nullable=True
    )
    resolved_at: Mapped[datetime | None] = mapped_column(
        DateTime(timezone=True), nullable=True
    )
@@ -1,256 +1,41 @@
-"""Unified Contact model — company or person, with inline addresses.
+"""Contact model - backwards-compatibility re-export (Paket 6, #357).
-Based on Rentman's contact model: a single table with type field
+The physical home of Contact/ContactPerson moved to the ContactsPlugin:
-('company' or 'person'). ContactPerson is a 1:N child for
+    app/plugins.builtins.contacts.models
-ansprechpartner (company employees / contact persons).
+
 This module re-exports both classes lazily (PEP 562 ``__getattr__``) so every
 existing import keeps working - ``from app.models.contact import Contact``
 resolves at attribute-access time:
    - alembic/env.py (``from app.models import *`` -> Base.metadata stays
      complete; Autogenerate never sees the tables as removed)
    - Core services (worker.py, jobs.py, address_service.py, ...)
    - 19 test files and scripts
 Why LAZY and not a top-level import: app/models/__init__.py is imported very
 early (app.core.auth imports app.models.session). A top-level plugin import
 here would pull in app.plugins -> registry -> service_container -> cache ->
 app.core.auth while app.core.auth is still initializing -> circular ImportError
 (proven in the Paket 6 red run). With PEP 562 the plugin framework is only
 touched when Contact is actually accessed, long after app.models finished
 initializing - every entry order is cycle-free.
 The cross-plugin checker (scripts/check_cross_plugin_imports.py) lists this
 file in EXEMPT_PATHS: the re-export is the deliberate, documented bridge -
 the plugin OWNS the model; the core only mirrors it for import stability.
 """
 from __future__ import annotations
-import uuid
+_EXPORTS = {"Contact", "ContactPerson"}
 from decimal import Decimal
 from typing import Any
 from sqlalchemy import (
    Computed,
    Float,
    ForeignKey,
    Index,
    Numeric,
    String,
    Text,
    UniqueConstraint,
 )
 from sqlalchemy.dialects.postgresql import JSONB, TSVECTOR
 from sqlalchemy.dialects.postgresql import UUID as PGUUID
 from sqlalchemy.orm import Mapped, mapped_column, relationship
 from app.core.db import Base, TenantMixin
 from app.models.owned_mixin import OwnedMixin
-class Contact(Base, TenantMixin, OwnedMixin):
+def __getattr__(name: str):
-    """Unified contact entity — can be a company or a person.
+    if name in _EXPORTS:
        from app.plugins.builtins.contacts.models import Contact, ContactPerson
-    type='company': name is the company name, firstname/surname empty.
+        return {"Contact": Contact, "ContactPerson": ContactPerson}[name]
-    type='person': firstname/surname are the person's name, name empty.
+    raise AttributeError(f"module {__name__!r} has no attribute {name!r}")
    Both types can have contactpersons (1:N) and inline addresses
    (mailing, visit, invoice).
    """
    __tablename__ = "contacts"
    __table_args__ = (
        UniqueConstraint("tenant_id", "code", name="uq_contacts_tenant_code"),
        UniqueConstraint("tenant_id", "accounting_code", name="uq_contacts_tenant_accounting_code"),
        Index("ix_contacts_tenant_deleted", "tenant_id", "deleted_at"),
        Index("ix_contacts_tenant_type", "tenant_id", "type"),
        Index("ix_contacts_tenant_name", "tenant_id", "name"),
        Index("ix_contacts_tenant_displayname", "tenant_id", "displayname"),
        Index("ix_contacts_email", "email_1"),
        Index("ix_contacts_code", "code"),
        Index("ix_contacts_search_vec", "search_tsv", postgresql_using="gin"),
    )
    id: Mapped[uuid.UUID] = mapped_column(
        PGUUID(as_uuid=True), primary_key=True, default=uuid.uuid4
    )
    # ── Identity & Type ──
    type: Mapped[str] = mapped_column(String(20), nullable=False, default="company")  # 'company' or 'person'
    displayname: Mapped[str] = mapped_column(String(255), nullable=False, default="")
    # ── Lifecycle Status (state machine: lead → qualified → customer → inactive) ──
    status: Mapped[str] = mapped_column(String(30), nullable=False, default="lead", index=True)
    name: Mapped[str | None] = mapped_column(String(255), nullable=True)  # company name
    firstname: Mapped[str | None] = mapped_column(String(100), nullable=True)
    surname: Mapped[str | None] = mapped_column(String(100), nullable=True)
    suffix: Mapped[str | None] = mapped_column(String(50), nullable=True)  # name prefix (Dr., Prof.)
    ext_name_line: Mapped[str | None] = mapped_column(String(255), nullable=True)  # additional name line / subtitle
    gender: Mapped[str | None] = mapped_column(String(20), nullable=True)
    # ── Customer / Accounting ──
    code: Mapped[str | None] = mapped_column(String(100), nullable=True)  # customer number
    accounting_code: Mapped[str | None] = mapped_column(String(100), nullable=True)
    vendor_accounting_code: Mapped[str | None] = mapped_column(String(100), nullable=True)
    # ── Mailing Address (inline) ──
    mailing_street: Mapped[str | None] = mapped_column(String(255), nullable=True)
    mailing_number: Mapped[str | None] = mapped_column(String(20), nullable=True)
    mailing_unit_number: Mapped[str | None] = mapped_column(String(50), nullable=True)
    mailing_district: Mapped[str | None] = mapped_column(String(100), nullable=True)
    mailing_extra_address_line: Mapped[str | None] = mapped_column(String(255), nullable=True)
    mailing_postalcode: Mapped[str | None] = mapped_column(String(20), nullable=True)
    mailing_city: Mapped[str | None] = mapped_column(String(100), nullable=True)
    mailing_state: Mapped[str | None] = mapped_column(String(100), nullable=True)
    mailing_country: Mapped[str | None] = mapped_column(String(2), nullable=True)
    # ── Visit Address (inline) ──
    visit_street: Mapped[str | None] = mapped_column(String(255), nullable=True)
    visit_number: Mapped[str | None] = mapped_column(String(20), nullable=True)
    visit_unit_number: Mapped[str | None] = mapped_column(String(50), nullable=True)
    visit_district: Mapped[str | None] = mapped_column(String(100), nullable=True)
    visit_extra_address_line: Mapped[str | None] = mapped_column(String(255), nullable=True)
    visit_postalcode: Mapped[str | None] = mapped_column(String(20), nullable=True)
    visit_city: Mapped[str | None] = mapped_column(String(100), nullable=True)
    visit_state: Mapped[str | None] = mapped_column(String(100), nullable=True)
    # ── Invoice Address (inline) ──
    invoice_street: Mapped[str | None] = mapped_column(String(255), nullable=True)
    invoice_number: Mapped[str | None] = mapped_column(String(20), nullable=True)
    invoice_unit_number: Mapped[str | None] = mapped_column(String(50), nullable=True)
    invoice_district: Mapped[str | None] = mapped_column(String(100), nullable=True)
    invoice_extra_address_line: Mapped[str | None] = mapped_column(String(255), nullable=True)
    invoice_postalcode: Mapped[str | None] = mapped_column(String(20), nullable=True)
    invoice_city: Mapped[str | None] = mapped_column(String(100), nullable=True)
    invoice_state: Mapped[str | None] = mapped_column(String(100), nullable=True)
    invoice_country: Mapped[str | None] = mapped_column(String(2), nullable=True)
    # ── General country ──
    country: Mapped[str | None] = mapped_column(String(2), nullable=True)
    # ── Communication ──
    phone_1: Mapped[str | None] = mapped_column(String(50), nullable=True)
    phone_2: Mapped[str | None] = mapped_column(String(50), nullable=True)
    email_1: Mapped[str | None] = mapped_column(String(255), nullable=True)
    email_2: Mapped[str | None] = mapped_column(String(255), nullable=True)
    website: Mapped[str | None] = mapped_column(String(500), nullable=True)
    # ── Financial & Tax ──
    vat_code: Mapped[str | None] = mapped_column(String(50), nullable=True)  # USt-IdNr.
    fiscal_code: Mapped[str | None] = mapped_column(String(50), nullable=True)  # Steuernummer
    commerce_code: Mapped[str | None] = mapped_column(String(100), nullable=True)  # Handelsregister
    purchase_number: Mapped[str | None] = mapped_column(String(100), nullable=True)  # Bestellnummer
    bic: Mapped[str | None] = mapped_column(String(50), nullable=True)
    bank_account: Mapped[str | None] = mapped_column(String(50), nullable=True)  # IBAN
    # ── Discounts ──
    discount_crew: Mapped[Decimal] = mapped_column(Numeric(5, 2), nullable=False, default=0)
    discount_transport: Mapped[Decimal] = mapped_column(Numeric(5, 2), nullable=False, default=0)
    discount_rental: Mapped[Decimal] = mapped_column(Numeric(5, 2), nullable=False, default=0)
    discount_sale: Mapped[Decimal] = mapped_column(Numeric(5, 2), nullable=False, default=0)
    discount_subrent: Mapped[Decimal] = mapped_column(Numeric(5, 2), nullable=False, default=0)
    discount_total: Mapped[Decimal] = mapped_column(Numeric(5, 2), nullable=False, default=0)
    # ── Geo ──
    latitude: Mapped[float | None] = mapped_column(Float, nullable=True)
    longitude: Mapped[float | None] = mapped_column(Float, nullable=True)
    # ── Notes & Warnings ──
    projectnote: Mapped[str | None] = mapped_column(Text, nullable=True)
    projectnote_title: Mapped[str | None] = mapped_column(String(255), nullable=True)
    contact_warning: Mapped[str | None] = mapped_column(Text, nullable=True)
    tags: Mapped[str | None] = mapped_column(String(500), nullable=True)  # comma-separated
    image: Mapped[str | None] = mapped_column(Text, nullable=True)  # logo/image URL or base64
    # ── Default contact persons (self-referential via contactpersons table) ──
    default_person_id: Mapped[uuid.UUID | None] = mapped_column(
        PGUUID(as_uuid=True), ForeignKey("contactpersons.id", ondelete="SET NULL"), nullable=True
    )
    admin_contactperson_id: Mapped[uuid.UUID | None] = mapped_column(
        PGUUID(as_uuid=True), ForeignKey("contactpersons.id", ondelete="SET NULL"), nullable=True
    )
    # ── Folder assignment ──
    folder_id: Mapped[uuid.UUID | None] = mapped_column(
        PGUUID(as_uuid=True),
        ForeignKey("contact_folders.id", ondelete="SET NULL"),
        nullable=True,
        index=True,
    )
    # ── Custom fields ──
    custom: Mapped[dict | None] = mapped_column(JSONB, nullable=True, default=dict)
    # ── FTS ──
    search_tsv: Mapped[Any] = mapped_column(
        TSVECTOR,
        Computed(
            "to_tsvector('german', coalesce(name, '') || ' ' || coalesce(displayname, '') || ' ' || coalesce(firstname, '') || ' ' || coalesce(surname, '') || ' ' || coalesce(email_1, '') || ' ' || coalesce(email_2, '') || ' ' || coalesce(code, '') || ' ' || coalesce(phone_1, '') || ' ' || coalesce(phone_2, '') || ' ' || coalesce(mailing_city, '') || ' ' || coalesce(mailing_postalcode, '') || ' ' || coalesce(tags, ''))",
            persisted=True,
        ),
        nullable=True,
    )
    # ── Embedding (pgvector, 768-dim) ──
    from pgvector.sqlalchemy import Vector
    embedding: Mapped[Any | None] = mapped_column(
        Vector(768), nullable=True, default=None
    )
    # ── Audit ──
    created_by: Mapped[uuid.UUID | None] = mapped_column(
        PGUUID(as_uuid=True), ForeignKey("users.id", ondelete="SET NULL"), nullable=True
    )
    updated_by: Mapped[uuid.UUID | None] = mapped_column(
        PGUUID(as_uuid=True), ForeignKey("users.id", ondelete="SET NULL"), nullable=True
    )
    # ── Relationships ──
    contact_persons: Mapped[list[ContactPerson]] = relationship(
        back_populates="contact", cascade="all, delete-orphan", foreign_keys="ContactPerson.contact_id"
    )
-class ContactPerson(Base, TenantMixin):
+def __dir__() -> list[str]:
-    """Ansprechpartner — 1:N child of a Contact.
+    return sorted(_EXPORTS | {"__getattr__", "__dir__"})
    Represents a person working at / associated with a company contact.
    Has its own address and communication fields.
    """
    __tablename__ = "contactpersons"
    __table_args__ = (
        Index("ix_contactpersons_tenant_deleted", "tenant_id", "deleted_at"),
        Index("ix_contactpersons_contact", "contact_id"),
        Index("ix_contactpersons_email", "email"),
    )
    id: Mapped[uuid.UUID] = mapped_column(
        PGUUID(as_uuid=True), primary_key=True, default=uuid.uuid4
    )
    # ── Parent contact ──
    contact_id: Mapped[uuid.UUID] = mapped_column(
        PGUUID(as_uuid=True), ForeignKey("contacts.id", ondelete="CASCADE"), nullable=False
    )
    # ── Name ──
    displayname: Mapped[str] = mapped_column(String(255), nullable=False, default="")
    firstname: Mapped[str | None] = mapped_column(String(100), nullable=True)
    middle_name: Mapped[str | None] = mapped_column(String(100), nullable=True)
    lastname: Mapped[str | None] = mapped_column(String(100), nullable=True)
    function: Mapped[str | None] = mapped_column(String(255), nullable=True)  # position/role
    # ── Communication ──
    phone: Mapped[str | None] = mapped_column(String(50), nullable=True)
    mobilephone: Mapped[str | None] = mapped_column(String(50), nullable=True)
    email: Mapped[str | None] = mapped_column(String(255), nullable=True)
    # ── Own address ──
    street: Mapped[str | None] = mapped_column(String(255), nullable=True)
    number: Mapped[str | None] = mapped_column(String(20), nullable=True)
    postalcode: Mapped[str | None] = mapped_column(String(20), nullable=True)
    city: Mapped[str | None] = mapped_column(String(100), nullable=True)
    state: Mapped[str | None] = mapped_column(String(100), nullable=True)
    country: Mapped[str | None] = mapped_column(String(2), nullable=True)
    # ── Other ──
    tags: Mapped[str | None] = mapped_column(String(500), nullable=True)
    custom: Mapped[dict | None] = mapped_column(JSONB, nullable=True, default=dict)
    # ── Audit ──
    created_by: Mapped[uuid.UUID | None] = mapped_column(
        PGUUID(as_uuid=True), ForeignKey("users.id", ondelete="SET NULL"), nullable=True
    )
    updated_by: Mapped[uuid.UUID | None] = mapped_column(
        PGUUID(as_uuid=True), ForeignKey("users.id", ondelete="SET NULL"), nullable=True
    )
    # ── Relationship ──
    contact: Mapped[Contact] = relationship(
        back_populates="contact_persons", foreign_keys=[contact_id]
    )
 # Keep old names for backward compat during migration
@@ -10,9 +10,10 @@ from sqlalchemy.dialects.postgresql import UUID as PGUUID
 from sqlalchemy.orm import Mapped, mapped_column
 from app.core.db import Base, TenantMixin
 from app.models.owned_mixin import OwnedMixin
-class ContactMergeHistory(Base, TenantMixin):
+class ContactMergeHistory(Base, TenantMixin, OwnedMixin):
    """Records each contact merge operation (source → target).
    When two duplicate contacts are merged, the source contact is soft-deleted
@@ -9,9 +9,10 @@ from sqlalchemy.dialects.postgresql import UUID as PGUUID
 from sqlalchemy.orm import Mapped, mapped_column
 from app.core.db import Base, TenantMixin
 from app.models.owned_mixin import OwnedMixin
-class Currency(Base, TenantMixin):
+class Currency(Base, TenantMixin, OwnedMixin):
    """Currency entity — e.g. EUR, USD, GBP."""
    __tablename__ = "currencies"
@@ -0,0 +1,52 @@
 """Dashboard model — personal per-user dashboards (Phase M2).
 Dashboards are personal (user-owned) layouts of MiniApp instances: tabs,
 widget placements and per-instance settings, stored as JSONB. Access is
 owner-only (saved_views precedent) — the active workspace limits only the
 available widget types (Phase N), never this personal layout.
 """
 from __future__ import annotations
 import uuid
 from typing import Any
 from sqlalchemy import Boolean, ForeignKey, Index, String, text
 from sqlalchemy.dialects.postgresql import JSONB
 from sqlalchemy.dialects.postgresql import UUID as PGUUID
 from sqlalchemy.orm import Mapped, mapped_column
 from app.core.db import Base, TenantMixin
 class Dashboard(Base, TenantMixin):
    """Personal dashboard — per-user layout of MiniApp instances."""
    __tablename__ = "dashboards"
    __table_args__ = (
        # Partial unique: soft-deleted dashboards free their name (unlike the
        # saved_views plain constraint, which keeps names occupied forever).
        Index(
            "uq_dashboards_tenant_user_name",
            "tenant_id",
            "user_id",
            "name",
            unique=True,
            postgresql_where=text("deleted_at IS NULL"),
        ),
        Index("ix_dashboards_tenant_user", "tenant_id", "user_id"),
    )
    id: Mapped[uuid.UUID] = mapped_column(
        PGUUID(as_uuid=True), primary_key=True, default=uuid.uuid4
    )
    name: Mapped[str] = mapped_column(String(100), nullable=False)
    # Layout JSONB (validated by app.schemas.dashboard.DashboardLayout):
    # {version, tabs: [{id, name, widgets: [{app_id, settings, col, row, spans}]}]}
    layout: Mapped[dict[str, Any]] = mapped_column(
        JSONB, nullable=False, default=dict
    )
    is_default: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False)
    user_id: Mapped[uuid.UUID] = mapped_column(
        PGUUID(as_uuid=True), ForeignKey("users.id", ondelete="CASCADE"), nullable=False
    )
@@ -35,9 +35,10 @@ from sqlalchemy.dialects.postgresql import UUID as PGUUID
 from sqlalchemy.orm import Mapped, mapped_column
 from app.core.db import Base, TenantMixin
 from app.models.owned_mixin import OwnedMixin
-class EntityPermission(Base, TenantMixin):
+class EntityPermission(Base, TenantMixin, OwnedMixin):
    """Universal ACL entry for any entity in the system.
    entity_type examples: 'contact', 'dms_file', 'mailbox', 'calendar_event',
@@ -37,9 +37,10 @@ from sqlalchemy.dialects.postgresql import UUID as PGUUID
 from sqlalchemy.orm import Mapped, mapped_column
 from app.core.db import Base, TenantMixin
 from app.models.owned_mixin import OwnedMixin
-class EntityPolicy(Base, TenantMixin):
+class EntityPolicy(Base, TenantMixin, OwnedMixin):
    """ABAC policy entry for any entity type in the system.
    entity_type examples: 'contact', 'dms_file', 'mailbox', 'calendar_event',
@@ -12,9 +12,10 @@ from sqlalchemy.dialects.postgresql import UUID as PGUUID
 from sqlalchemy.orm import Mapped, mapped_column
 from app.core.db import Base, TenantMixin
 from app.models.owned_mixin import OwnedMixin
-class Group(Base, TenantMixin):
+class Group(Base, TenantMixin, OwnedMixin):
    """Group entity with RBAC permissions and field-level permissions.
    Groups are tenant-scoped. Users can be members of multiple groups.
@@ -45,6 +46,7 @@ class UserGroup(Base):
    """N:M association — user membership in groups (per tenant)."""
    __tablename__ = "user_groups"
    deleted_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
    __table_args__ = (
        UniqueConstraint("user_id", "group_id", "tenant_id", name="uq_user_groups_user_group_tenant"),
    )
@@ -19,9 +19,10 @@ from sqlalchemy.dialects.postgresql import UUID as PGUUID
 from sqlalchemy.orm import Mapped, mapped_column
 from app.core.db import Base, TenantMixin
 from app.models.owned_mixin import OwnedMixin
-class Notification(Base, TenantMixin):
+class Notification(Base, TenantMixin, OwnedMixin):
    """User notification entity."""
    __tablename__ = "notifications"
@@ -52,6 +53,7 @@ class NotificationType(Base):
    """Registered notification type from a plugin."""
    __tablename__ = "notification_types"
    deleted_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
    __table_args__ = (Index("ix_notification_types_key", "type_key"),)
    id: Mapped[uuid.UUID] = mapped_column(
@@ -70,7 +72,7 @@ class NotificationType(Base):
    )
-class NotificationPreference(Base, TenantMixin):
+class NotificationPreference(Base, TenantMixin, OwnedMixin):
    """User preference for a notification type (opt-in/opt-out)."""
    __tablename__ = "notification_preferences"
@@ -5,7 +5,7 @@ from __future__ import annotations
 import uuid
 from datetime import datetime
-from sqlalchemy import DateTime, Integer, String, Text, func
+from sqlalchemy import DateTime, ForeignKey, Integer, String, Text, UniqueConstraint, func
 from sqlalchemy.dialects.postgresql import JSONB
 from sqlalchemy.dialects.postgresql import UUID as PGUUID
 from sqlalchemy.orm import Mapped, mapped_column
@@ -77,3 +77,49 @@ class EventOutbox(Base):
    failed_at: Mapped[datetime | None] = mapped_column(
        DateTime(timezone=True), nullable=True,
    )
 class OutboxDelivery(Base):
    """Per-consumer delivery status for an event_outbox row (Migration 0075).
    Tracks whether each consumer successfully processed an event; an event is
    only 'published' when all mandatory deliveries succeed.
    """
    __tablename__ = "outbox_deliveries"
    __table_args__ = (
        UniqueConstraint(
            "event_id", "consumer_name",
            name="uq_outbox_deliveries_event_consumer",
        ),
    )
    id: Mapped[uuid.UUID] = mapped_column(
        PGUUID(as_uuid=True), primary_key=True,
        server_default=func.gen_random_uuid(),
    )
    event_id: Mapped[uuid.UUID] = mapped_column(
        PGUUID(as_uuid=True),
        ForeignKey("event_outbox.id", ondelete="CASCADE"),
        nullable=False,
    )
    consumer_name: Mapped[str] = mapped_column(String(150), nullable=False)
    status: Mapped[str] = mapped_column(
        String(30), nullable=False, server_default="pending",
    )
    attempt_count: Mapped[int] = mapped_column(
        Integer, nullable=False, server_default="0",
    )
    next_attempt_at: Mapped[datetime | None] = mapped_column(
        DateTime(timezone=True), nullable=True,
    )
    last_error: Mapped[str | None] = mapped_column(Text, nullable=True)
    processed_at: Mapped[datetime | None] = mapped_column(
        DateTime(timezone=True), nullable=True,
    )
    created_at: Mapped[datetime] = mapped_column(
        DateTime(timezone=True), nullable=False, server_default=func.now(),
    )
    updated_at: Mapped[datetime] = mapped_column(
        DateTime(timezone=True), nullable=False, server_default=func.now(),
    )
@@ -1,57 +0,0 @@
 """Outbox delivery model for per-consumer delivery tracking."""
 from __future__ import annotations
 import uuid
 from datetime import datetime
 from sqlalchemy import DateTime, ForeignKey, Integer, String, Text, UniqueConstraint, func
 from sqlalchemy.dialects.postgresql import UUID as PGUUID
 from sqlalchemy.orm import Mapped, mapped_column
 from app.core.db import Base
 class OutboxDelivery(Base):
    """Tracks per-consumer delivery status for outbox events.
    Each row represents one consumer (event handler) processing one outbox
    event.  An event is only fully 'published' when all mandatory deliveries
    succeed.
    """
    __tablename__ = "outbox_deliveries"
    __table_args__ = (
        UniqueConstraint("event_id", "consumer_name", name="uq_outbox_deliveries_event_consumer"),
    )
    id: Mapped[uuid.UUID] = mapped_column(
        PGUUID(as_uuid=True),
        primary_key=True,
        server_default=func.gen_random_uuid(),
    )
    event_id: Mapped[uuid.UUID] = mapped_column(
        PGUUID(as_uuid=True),
        ForeignKey("event_outbox.id", ondelete="CASCADE"),
        nullable=False,
    )
    consumer_name: Mapped[str] = mapped_column(String(150), nullable=False)
    status: Mapped[str] = mapped_column(
        String(30), nullable=False, server_default="pending",
    )
    attempt_count: Mapped[int] = mapped_column(
        Integer, nullable=False, server_default="0",
    )
    next_attempt_at: Mapped[datetime | None] = mapped_column(
        DateTime(timezone=True), nullable=True,
    )
    last_error: Mapped[str | None] = mapped_column(Text, nullable=True)
    processed_at: Mapped[datetime | None] = mapped_column(
        DateTime(timezone=True), nullable=True,
    )
    created_at: Mapped[datetime] = mapped_column(
        DateTime(timezone=True), nullable=False, server_default=func.now(),
    )
    updated_at: Mapped[datetime] = mapped_column(
        DateTime(timezone=True), nullable=False, server_default=func.now(),
    )
@@ -21,9 +21,10 @@ from sqlalchemy.dialects.postgresql import UUID as PGUUID
 from sqlalchemy.orm import Mapped, mapped_column
 from app.core.db import Base, TenantMixin
 from app.models.owned_mixin import OwnedMixin
-class PermissionDelegation(Base, TenantMixin):
+class PermissionDelegation(Base, TenantMixin, OwnedMixin):
    """Permission delegation — temporary handover of permissions.
    from_user_id delegates their permissions to to_user_id
@@ -20,9 +20,10 @@ from sqlalchemy.dialects.postgresql import UUID as PGUUID
 from sqlalchemy.orm import Mapped, mapped_column
 from app.core.db import Base, TenantMixin
 from app.models.owned_mixin import OwnedMixin
-class PermissionTemplate(Base, TenantMixin):
+class PermissionTemplate(Base, TenantMixin, OwnedMixin):
    """Reusable permission template for entity types.
    When applied to an entity, the template evaluates trigger_condition
@@ -58,3 +58,7 @@ class PluginMigration(Base, TimestampMixin):
    plugin_name: Mapped[str] = mapped_column(String(80), nullable=False)
    migration_file: Mapped[str] = mapped_column(String(255), nullable=False)
    status: Mapped[str] = mapped_column(String(20), nullable=False, default="applied")
    # F40 (Astra): SHA-256 of the applied SQL content. The runner compares
    # this on subsequent runs — a modified already-applied migration
    # (repaired) becomes VISIBLE instead of being silently skipped.
    content_hash: Mapped[str | None] = mapped_column(String(64), nullable=True)
@@ -12,9 +12,10 @@ from sqlalchemy.dialects.postgresql import UUID as PGUUID
 from sqlalchemy.orm import Mapped, mapped_column
 from app.core.db import Base, TenantMixin
 from app.models.owned_mixin import OwnedMixin
-class Role(Base, TenantMixin):
+class Role(Base, TenantMixin, OwnedMixin):
    """Role entity with module→action→permission mapping and field-level permissions."""
    __tablename__ = "roles"
@@ -17,9 +17,10 @@ from sqlalchemy.dialects.postgresql import UUID as PGUUID
 from sqlalchemy.orm import Mapped, mapped_column
 from app.core.db import Base, TenantMixin
 from app.models.owned_mixin import OwnedMixin
-class Session(Base, TenantMixin):
+class Session(Base, TenantMixin, OwnedMixin):
    """Immutable session audit record. Runtime session lookup uses Redis."""
    __tablename__ = "sessions"
--- a/Show More
+++ b/Show More