Compare commits
95 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| c34715574a | |||
| fce17aac9c | |||
| cbe36e0c0e | |||
| 6702d69f7c | |||
| 94d8c40daa | |||
| be81fe52cf | |||
| a1d5e56009 | |||
| 3e43219b84 | |||
| 26b5ae9a0d | |||
| 11e4e42570 | |||
| 57441df677 | |||
| fbe1bde635 | |||
| 2d17746194 | |||
| 23a05593b2 | |||
| 0baec2792c | |||
| f4a5937a4b | |||
| 38b73f5d4d | |||
| a6bfa8e67c | |||
| a8916b3d86 | |||
| e7afbaa906 | |||
| 34c9c85aed | |||
| 9d2df61942 | |||
| 1b485d4a34 | |||
| f4c4a50ebd | |||
| 69d05d6912 | |||
| df9f86bd12 | |||
| 9e1d202610 | |||
| c291a6ecf1 | |||
| ab3c253cbd | |||
| 52323610e3 | |||
| 86c96f03ca | |||
| 3e5f13f516 | |||
| 4de629d296 | |||
| 6a88c70073 | |||
| c807aacfc0 | |||
| b23045c46a | |||
| 5d8c48a08f | |||
| f6dde68221 | |||
| d901d001c7 | |||
| 962e0ee1f6 | |||
| 49ca4c5fb2 | |||
| 1b22da8b0d | |||
| 76a31a8c39 | |||
| a991f9a0b4 | |||
| 84a30d85c2 | |||
| d9aed519f2 | |||
| b9a6c06e85 | |||
| 8386e99caa | |||
| 7d9ae03bf1 | |||
| 36a03b9897 | |||
| 860db8d61e | |||
| 81aea8c77f | |||
| 46c909c226 | |||
| 197b0d3bab | |||
| 3934aea6ef | |||
| 5cc5a3fa6a | |||
| c0e8e4ecfd | |||
| c32e4bb34e | |||
| ef90d57f0a | |||
| 0768cfb29a | |||
| 56e401969e | |||
| 6d04206695 | |||
| f6e117b1c3 | |||
| 9d8da99026 | |||
| 54066b05fd | |||
| 36636f5c25 | |||
| d89044d8f7 | |||
| 5e0ffd91c2 | |||
| b8b8ef180a | |||
| cad7d084e8 | |||
| dff97f5589 | |||
| 9e84c400ed | |||
| 067fc132cb | |||
| b01b756a4a | |||
| 4bce89aecb | |||
| 5e9be254e2 | |||
| 8a76bfdba4 | |||
| d2434203c1 | |||
| ad7c763e59 | |||
| e3fb4728d7 | |||
| 7467c01d38 | |||
| 4038b74025 | |||
| 5ad107ff83 | |||
| 5cee78c54c | |||
| 32f63adc09 | |||
| c21634b323 | |||
| 73d2e109cd | |||
| 795307754f | |||
| 17516d2783 | |||
| ed8ee5cda1 | |||
| 90a367089d | |||
| b04cda774b | |||
| 982b4c9353 | |||
| 1d6152fb82 | |||
| 337d78ef53 |
+2
-2
@@ -39,8 +39,8 @@ RUN apt-get update \
|
||||
|
||||
WORKDIR /app
|
||||
|
||||
COPY requirements.txt .
|
||||
RUN pip install --user --no-cache-dir -r requirements.txt
|
||||
COPY requirements.txt requirements.lock ./
|
||||
RUN pip install --user --no-cache-dir -r requirements.lock
|
||||
|
||||
# === Stage 2: Runtime ===
|
||||
FROM python:3.12-slim AS runtime
|
||||
|
||||
@@ -1210,6 +1210,370 @@ Trigger / Event / Cron / Webhook / Agent
|
||||
|
||||
---
|
||||
|
||||
## Phase L — UI-Overhaul (Status: geplant, NICHT gestartet)
|
||||
|
||||
> **Herkunft:** Am 2026-08-25 aus der eigenständigen Datei `UI_OVERHAUL_PLAN.md`
|
||||
> hier integriert - gemaess AGENTS.md-Regel "PLATFORM_ROADMAP.md ist EINZIGE
|
||||
> Planungs-Datei". Vollständiges Original inkl. ASCII-Mockups abrufbar via
|
||||
> `git show c807aac:UI_OVERHAUL_PLAN.md`.
|
||||
>
|
||||
> **Konflikt-Notiz (2026-08-25, Block I-D):** Phase 2 unten sieht "AI Assistant
|
||||
> Page entfernen" vor. Die Seite wurde jedoch in Commit 962e0ee bewusst GEBAUT,
|
||||
> um die Geister-Route /ai-assistant zu reparieren (im Backend-Manifest
|
||||
> referenziert, aber nicht existent -> ErrorBoundary in Production). VOR
|
||||
> Umsetzung von Phase 2 neu entscheiden: (a) Seite doch entfernen - dann auch
|
||||
> Manifest-Route entfernen, oder (b) Phase 2 verwerfen zugunsten der aktuellen
|
||||
> Architektur. Bitte nicht unkommentiert ausfuehren.
|
||||
|
||||
> **Erstellt:** 2026-08-21
|
||||
> **Aktualisiert:** 2026-08-21 — AI Assistent Integration hinzugefügt
|
||||
> **Status:** Planung — nicht gestartet
|
||||
> **Leitlinie:** Auf bestehendem Code aufbauen, 3-Spalten-Explorer-Layout als Standard, keine parallelen Systeme
|
||||
|
||||
---
|
||||
|
||||
### Standard-Layout (Referenz: ContactsList.tsx)
|
||||
|
||||
Alle Explorer-Plugins nutzen das 3-Spalten-Layout aus den UI-Design-Guidelines:
|
||||
|
||||
```
|
||||
┌─────────────┬──────────────────┬──────────────────────┐
|
||||
│ Tree │ Liste/Ansicht │ Detail │
|
||||
│ (224px) │ (flex-1) │ (flex-1 / 60%) │
|
||||
│ ResizablePanel│ ResizablePanel │ ResizablePanel │
|
||||
└─────────────┴──────────────────┴──────────────────────┘
|
||||
```
|
||||
|
||||
- **Toolbar oben:** PluginToolbar mit Filter-Dropdowns, Ansichts-Umschaltern, Aktion-Buttons
|
||||
- **Linke Spalte:** ResizablePanel mit Baumansicht (Ordner, Kategorien, Kalender)
|
||||
- **Mitte:** Liste, Karten, Kalender-Ansicht — mehrere Ansichten umschaltbar
|
||||
- **Rechts:** Detail-Bereich für ausgewähltes Element
|
||||
|
||||
---
|
||||
|
||||
### Phase 1: Echte Bugs fixen (2-3 Tage)
|
||||
|
||||
#### 1.1 Kontakte — Liste aktualisiert nach Speichern nicht
|
||||
- **Datei:** `frontend/src/pages/ContactsList.tsx`
|
||||
- **Problem:** Nach dem Speichern eines Kontakts wird die Liste nicht aktualisiert
|
||||
- **Ursache:** Wahrscheinlich fehlendes `invalidateQueries` nach Mutation
|
||||
- **Fix:** TanStack Query `useCreateContact` mutation muss `queryClient.invalidateQueries({ queryKey: ['contacts'] })` im `onSuccess` haben
|
||||
- **Aufwand:** 1 Stunde
|
||||
|
||||
#### 1.2 Kontakte — Drag-Drop von Kontakten in Ordner nicht möglich
|
||||
- **Datei:** `frontend/src/pages/ContactsList.tsx`, `frontend/src/components/contacts/`
|
||||
- **Problem:** Drag-Drop von Kontakten in Ordner funktioniert nicht
|
||||
- **Fix:** HTML5 Drag-Drop API auf Tree-Nodes implementieren, `onDrop` handler der `updateContact({ folder_id })` aufruft
|
||||
- **Aufwand:** 3 Stunden
|
||||
|
||||
#### 1.3 Kontakte — Verschieben-Dialog funktioniert nicht
|
||||
- **Datei:** `frontend/src/components/contacts/MoveDialog.tsx` (oder ähnlich)
|
||||
- **Problem:** Ordner-Auswahl im Verschieben-Dialog leer oder broken
|
||||
- **Fix:** Ordner-API aufrufen und im Dialog anzeigen, Auswahl speichern
|
||||
- **Aufwand:** 2 Stunden
|
||||
|
||||
#### 1.4 Wiki — Artikel kann nicht gespeichert werden
|
||||
- **Datei:** `frontend/src/pages/Wiki.tsx`, `frontend/src/api/knowledge.ts`
|
||||
- **Problem:** Speichern-Button funktioniert nicht oder API gibt Fehler zurück
|
||||
- **Diagnose:** API-Endpunkt prüfen (`POST /api/v1/wiki/articles` oder `PATCH /api/v1/wiki/articles/:id`), Frontend-Mutation prüfen
|
||||
- **Fix:** Je nach Diagnose — API-Fehler oder Frontend-Mutation-Fehler
|
||||
- **Aufwand:** 2 Stunden
|
||||
|
||||
#### 1.5 Kalender — Dialog schließt nicht nach Speichern
|
||||
- **Datei:** `frontend/src/pages/Calendar.tsx`, `frontend/src/components/calendar/AppointmentEditForm.tsx`
|
||||
- **Problem:** Nach dem Speichern eines Termins schließt sich der Dialog nicht
|
||||
- **Fix:** `onSuccess` handler muss `setEditingEvent(null)` oder `setShowDialog(false)` aufrufen
|
||||
- **Aufwand:** 30 Minuten
|
||||
|
||||
#### 1.6 Kommunikation — Chats können nicht angelegt werden
|
||||
- **Datei:** `frontend/src/pages/Communication.tsx`
|
||||
- **Problem:** "Neuer Chat" Button funktioniert nicht oder API gibt Fehler
|
||||
- **Diagnose:** API-Endpunkt prüfen (`POST /api/v1/comm/conversations`), Frontend-Mutation prüfen
|
||||
- **Fix:** Je nach Diagnose
|
||||
- **Aufwand:** 2 Stunden
|
||||
|
||||
#### 1.7 Wiki — Doppelt im Menü
|
||||
- **Datei:** `frontend/src/routes/index.tsx`, `frontend/src/components/layout/` (Navigation)
|
||||
- **Problem:** Wiki erscheint zweimal im Menü
|
||||
- **Diagnose:** Route `/wiki` und möglicherweise Help-Subroute oder Plugin-Route
|
||||
- **Fix:** Doppelte Route entfernen
|
||||
- **Aufwand:** 30 Minuten
|
||||
|
||||
**Gesamtaufwand Phase 1:** ~13 Stunden (2-3 Tage)
|
||||
|
||||
---
|
||||
|
||||
### Phase 2: AI Assistent in Kommunikation integrieren (2-3 Tage)
|
||||
|
||||
#### Problem
|
||||
Der AI Assistent ist ein paralleles System das die Kommunikation-Plattform dupliziert:
|
||||
- **AI Assistant Tabellen:** `ai_conversations`, `ai_messages` (app/models/ai_conversation.py) + `ai_chat_sessions`, `ai_chat_messages`, `ai_chat_attachments` (app/plugins/builtins/ai_assistant/models.py) — 5 Tabellen
|
||||
- **AI Assistant Frontend:** `AIAssistant.tsx`, `AIAssistantStandalone.tsx`, `SessionList.tsx`, `ChatWindow.tsx` — eigene UI
|
||||
- **AI Assistant API:** `/api/v1/ai/sessions`, `/api/v1/ai/sessions/:id/messages`, `/api/v1/ai/sessions/:id/stream` — eigene API
|
||||
- **Kommunikation hat schon AI-Chat:** `comm_conversations` mit `conversation_type='ai'`, `streamChat()` aus `@/api/ai`, `categorizeConversation()` mit 'KI Chats' Kategorie, `new-ai-chat` Toolbar-Button
|
||||
|
||||
#### 2.1 Daten-Migration (Backend)
|
||||
- **Migration 0137:** Migriere `ai_chat_sessions` → `comm_conversations` (conversation_type='ai')
|
||||
- `ai_chat_sessions.id` → `comm_conversations.id`
|
||||
- `ai_chat_sessions.title` → `comm_conversations.title`
|
||||
- `ai_chat_sessions.tenant_id` → `comm_conversations.tenant_id`
|
||||
- `ai_chat_sessions.user_id` → `comm_conversations.owner_id`
|
||||
- `ai_chat_sessions.agent_id` → `comm_conversations.metadata.agent_id`
|
||||
- `ai_chat_sessions.created_at` → `comm_conversations.created_at`
|
||||
- **Migration 0137:** Migriere `ai_chat_messages` → `comm_messages`
|
||||
- `ai_chat_messages.id` → `comm_messages.id`
|
||||
- `ai_chat_messages.session_id` → `comm_messages.conversation_id`
|
||||
- `ai_chat_messages.role` → `comm_messages.sender_type` ('user' → 'user', 'assistant' → 'ai')
|
||||
- `ai_chat_messages.content` → `comm_messages.content`
|
||||
- `ai_chat_messages.tenant_id` → `comm_messages.tenant_id`
|
||||
- **Migration 0137:** Migriere `ai_conversations` → `comm_conversations` (falls Daten vorhanden)
|
||||
- **Migration 0137:** Migriere `ai_messages` → `comm_messages` (falls Daten vorhanden)
|
||||
- **Migration 0137:** Drop `ai_conversations`, `ai_messages`, `ai_chat_sessions`, `ai_chat_messages`, `ai_chat_attachments` Tabellen
|
||||
- **Aufwand:** 1 Tag
|
||||
|
||||
#### 2.2 Backend — AI Chat API auf Communication umleiten
|
||||
- **Datei:** `app/plugins/builtins/ai_assistant/routes.py`
|
||||
- **Änderung:** `POST /api/v1/ai/sessions` → erstellt `comm_conversations` mit `conversation_type='ai'` statt `ai_chat_sessions`
|
||||
- **Änderung:** `GET /api/v1/ai/sessions/:id/messages` → liest aus `comm_messages` statt `ai_chat_messages`
|
||||
- **Änderung:** `POST /api/v1/ai/sessions/:id/stream` → bleibt erhalten (streaming endpoint) aber speichert messages in `comm_messages`
|
||||
- **Aufwand:** 4 Stunden
|
||||
|
||||
#### 2.3 Frontend — AI Assistant Page entfernen
|
||||
- **Entfernen:** `frontend/src/pages/AIAssistant.tsx`
|
||||
- **Entfernen:** `frontend/src/pages/AIAssistantStandalone.tsx`
|
||||
- **Entfernen:** `frontend/src/components/ai/SessionList.tsx`
|
||||
- **Entfernen:** `frontend/src/components/ai/ChatWindow.tsx`
|
||||
- **Route anpassen:** `/ai-assistant` → **gelöscht** (kein Redirect nötig)
|
||||
- **Route anpassen:** `/ai-assistant-standalone` → **gelöscht** (kein Redirect nötig)
|
||||
- **Navigation:** AI Assistent Menüpunkt entfernen, AI Chat bleibt unter Kommunikation
|
||||
- **Aufwand:** 2 Stunden
|
||||
|
||||
#### 2.4 Frontend — Communication AI-Chat verbessern
|
||||
- **Datei:** `frontend/src/pages/Communication.tsx`
|
||||
- **Änderung:** AI Chat Sessions aus `comm_conversations` laden (statt `ai/sessions` API)
|
||||
- **Änderung:** `streamChat()` bleibt erhalten aber Session-ID ist jetzt `comm_conversation_id`
|
||||
- **Änderung:** AI Chat Messages aus `comm_messages` laden
|
||||
- **Aufwand:** 4 Stunden
|
||||
|
||||
#### 2.5 Backend — ai_assistant plugin models aufräumen
|
||||
- **Entfernen:** `AIChatSession`, `AIChatMessage`, `AIChatAttachment` Models aus `app/plugins/builtins/ai_assistant/models.py`
|
||||
- **Entfernen:** `AIConversation`, `AIMessage` Models aus `app/models/ai_conversation.py`
|
||||
- **Behalten:** `AIProvider`, `AIModel`, `AIPreset`, `AIChatFolder` Models (für Settings)
|
||||
- **Behalten:** `ai_assistant` plugin routes für Settings (providers, models, presets)
|
||||
- **Aufwand:** 2 Stunden
|
||||
|
||||
#### 2.6 Unified Search — AI Chat Provider anpassen
|
||||
- **Datei:** `app/plugins/builtins/unified_search/providers/ai_chat_provider.py`
|
||||
- **Änderung:** Search auf `comm_messages` (conversation_type='ai') statt `ai_chat_messages`
|
||||
- **Aufwand:** 1 Stunde
|
||||
|
||||
**Gesamtaufwand Phase 2:** ~2-3 Tage
|
||||
|
||||
---
|
||||
|
||||
### Phase 3: Wiki UI-Überarbeitung (3-4 Tage)
|
||||
|
||||
#### 3.1 WYSIWYG Editor
|
||||
- **Datei:** `frontend/src/components/wiki/WikiEditor.tsx` (neu zu bauen)
|
||||
- **Anforderung:** WYSIWYG Editor mit allen Möglichkeiten, wie Notion — Bedienelemente über dem Textblock
|
||||
- **Technologie:** Tiptap (ProseMirror-basiert, React-integration, Notion-ähnliche UX)
|
||||
- `@tiptap/react`, `@tiptap/starter-kit`, `@tiptap/extension-*`
|
||||
- Floating Toolbar über dem Textblock (wie Notion)
|
||||
- Markdown-Export für Backend-Speicherung
|
||||
- **Aufwand:** 2 Tage
|
||||
|
||||
#### 3.2 Wiki Layout — 3-Spalten
|
||||
- **Datei:** `frontend/src/pages/Wiki.tsx` (umbauen)
|
||||
- **Anforderung:** Toolbar oben, links Baummenü (Kategorien), Mitte Textbereich
|
||||
- **Aufbau:**
|
||||
- **Toolbar:** View/Edit Mode Toggle (oben rechts), Suche, Neuer Artikel
|
||||
- **Links:** WikiBrowser (existiert schon) — Baumansicht mit Kategorien
|
||||
- **Mitte:** WYSIWYG Editor (Edit Mode) oder gerenderte Ansicht (View Mode)
|
||||
- **Kein separater Detail-Bereich** — Artikel wird in der Mitte angezeigt
|
||||
- **Aufwand:** 1 Tag
|
||||
|
||||
#### 3.3 View/Edit Mode Toggle
|
||||
- **Datei:** `frontend/src/pages/Wiki.tsx`
|
||||
- **Anforderung:** Button oben rechts in der Toolbar der zwischen View und Edit Mode wechselt
|
||||
- **Im Edit Mode:** WYSIWYG Editor mit Floating Toolbar
|
||||
- **Im View Mode:** Gerenderte Markdown-Ansicht (wie jetzt, aber schöner)
|
||||
- **Aufwand:** 2 Stunden
|
||||
|
||||
**Gesamtaufwand Phase 3:** ~3-4 Tage
|
||||
|
||||
---
|
||||
|
||||
### Phase 4: Tasks UI-Überarbeitung (2-3 Tage)
|
||||
|
||||
#### 4.1 Tasks Layout — 3-Spalten wie Kontakte
|
||||
- **Datei:** `frontend/src/pages/Tasks.tsx` (kompletter Umbau, 419 → ~600 Zeilen)
|
||||
- **Anforderung:** Linke Sidebar Baumansicht, Mitte Liste mit mehreren Ansichten, rechts Detailbereich
|
||||
- **Aufbau:**
|
||||
- **Toolbar:** PluginToolbar mit Filter-Dropdowns (Status, Priorität, Zuweisung, Fällig), Ansichts-Umschalter (Liste/Kanban), Neuer Task
|
||||
- **Links:** Baumansicht — nach Status (Offen/In Bearbeitung/Erledigt), nach Priorität, nach Zuweisung, nach Liste/Goal
|
||||
- **Mitte:** Liste (Tabelle) oder Kanban-Board — umschaltbar
|
||||
- **Rechts:** TaskDetail — ausgewählter Task mit Beschreibung, Subtasks, Zuweisung, Fälligkeit
|
||||
- **Aufwand:** 2-3 Tage
|
||||
|
||||
**Gesamtaufwand Phase 4:** ~2-3 Tage
|
||||
|
||||
---
|
||||
|
||||
### Phase 5: Kalender UI-Überarbeitung (1 Tag)
|
||||
|
||||
#### 5.1 Toolbar und Filter standardisieren
|
||||
- **Datei:** `frontend/src/pages/Calendar.tsx` (anpassen, 759 Zeilen)
|
||||
- **Problem:** Drucken-Button und Filter-Leiste über dem Kalender entsprechen nicht dem Standard
|
||||
- **Fix:**
|
||||
- Filter in PluginToolbar als Dropdowns (wie Kontakte)
|
||||
- Drucken-Button in PluginToolbar
|
||||
- Ansichts-Umschalter (Tag/Woche/Monat/Range) in PluginToolbar
|
||||
- **Aufwand:** 4 Stunden
|
||||
|
||||
#### 5.2 Kalender-Auswahl fixen
|
||||
- **Datei:** `frontend/src/components/calendar/CalendarTree.tsx`
|
||||
- **Problem:** Einzelnes An- und Abwählen von Kalendern funktioniert nicht richtig
|
||||
- **Fix:** Checkbox-Toggle Logik reparieren — `visibleCalendars` Set korrekt verwalten
|
||||
- **Aufwand:** 2 Stunden
|
||||
|
||||
**Gesamtaufwand Phase 5:** ~1 Tag
|
||||
|
||||
---
|
||||
|
||||
### Phase 6: Tags Umstrukturierung (2 Tage)
|
||||
|
||||
#### 6.1 Tags in Settings verschieben
|
||||
- **Datei:** `frontend/src/pages/Tags.tsx` → `frontend/src/pages/SettingsTags.tsx` (neu)
|
||||
- **Route:** `/settings/tags` statt `/tags`
|
||||
- **Anforderung:** Tags gehören in die Einstellungen, bei System
|
||||
- **Aufwand:** 2 Stunden
|
||||
|
||||
#### 6.2 Tags Baumstruktur
|
||||
- **Datei:** `frontend/src/pages/SettingsTags.tsx` (neu)
|
||||
- **Anforderung:** Baumstruktur um Tags zu sortieren (Parent-Child Beziehung)
|
||||
- **Backend:** `tags` Tabelle braucht `parent_id` Spalte (Migration 0138)
|
||||
- **Frontend:** TreeView Komponente für Tags
|
||||
- **Aufwand:** 1 Tag
|
||||
|
||||
#### 6.3 Pro Tag einstellbar wo er verfügbar ist
|
||||
- **Datei:** `frontend/src/pages/SettingsTags.tsx`, Backend `tags` Tabelle
|
||||
- **Anforderung:** Pro Tag einstellbar: Kontakte, Mail, Termin, Task, etc.
|
||||
- **Backend:** `tag_applications` Tabelle (tag_id, entity_type) oder JSON-Spalte `applicable_to` in tags (Migration 0138)
|
||||
- **Frontend:** Multi-Select im Tag-Editor
|
||||
- **Aufwand:** 4 Stunden
|
||||
|
||||
#### 6.4 Symbol und Farbe pro Tag
|
||||
- **Datei:** `frontend/src/pages/SettingsTags.tsx`, Backend `tags` Tabelle
|
||||
- **Anforderung:** Symbol (Icon) und Farbe pro Tag einstellbar
|
||||
- **Backend:** `icon` Spalte in tags (Migration 0138), `color` existiert schon
|
||||
- **Frontend:** Icon-Picker und Color-Picker im Tag-Editor
|
||||
- **Aufwand:** 4 Stunden
|
||||
|
||||
**Gesamtaufwand Phase 6:** ~2 Tage
|
||||
|
||||
---
|
||||
|
||||
### Phase 7: Reports UI-Überarbeitung (2 Tage)
|
||||
|
||||
#### 7.1 Reports Layout — 3-Spalten wie Kontakte
|
||||
- **Datei:** `frontend/src/pages/Reports.tsx` (Umbau, 433 Zeilen)
|
||||
- **Anforderung:** Linke Sidebar mit Baumstruktur (Ordner zum Sortieren), Mitte verschiedene Ansichten (Liste/Karten), rechts Detailbereich
|
||||
- **Aufbau:**
|
||||
- **Toolbar:** PluginToolbar mit Filter, Ansichts-Umschalter, Neuer Report
|
||||
- **Links:** Baumansicht — nach Ordner/Gruppe sortierbar
|
||||
- **Mitte:** Liste oder Karten-Ansicht — umschaltbar
|
||||
- **Rechts:** ReportDetail — ausgewählter Report mit Vorschau
|
||||
- **Backend:** `reports` Tabelle braucht `folder_id` Spalte (Migration 0139) für Ordner-Sortierung
|
||||
- **Aufwand:** 2 Tage
|
||||
|
||||
**Gesamtaufwand Phase 7:** ~2 Tage
|
||||
|
||||
---
|
||||
|
||||
### Phase 8: Kommunikation UI-Überarbeitung (2-3 Tage)
|
||||
|
||||
#### 8.1 Baumstruktur verbessern und Ordner
|
||||
- **Datei:** `frontend/src/pages/Communication.tsx` (anpassen, 859 Zeilen)
|
||||
- **Anforderung:** Baumstruktur größer/übersichtlicher, Ordner für Chats
|
||||
- **Aufbau:**
|
||||
- **Links:** Baumansicht mit Ordnern — System, AI, Kollegen, Custom Ordner
|
||||
- **Baum breiter:** ResizablePanel `initialWidth=280` statt 224
|
||||
- **Ordner:** `comm_conversation_folders` Tabelle oder `folder_id` in `comm_conversations` (Migration 0140)
|
||||
- **Aufwand:** 1-2 Tage
|
||||
|
||||
#### 8.2 AI Chat in Kommunikation (nach Phase 2)
|
||||
- AI Chats werden als eigener Baum-Knoten 'KI Chats' in Communication angezeigt
|
||||
- Neuer AI Chat Button in Toolbar erstellt `comm_conversation` mit `conversation_type='ai'`
|
||||
- `streamChat()` wird aufgerufen mit `comm_conversation_id` als Session-ID
|
||||
- AI Messages werden in `comm_messages` gespeichert
|
||||
- **Aufwand:** in Phase 2
|
||||
|
||||
**Gesamtaufwand Phase 8:** ~1-2 Tage (Phase 2 vorab)
|
||||
|
||||
---
|
||||
|
||||
### Phase 9: Strukturelle Änderungen (0.5 Tage)
|
||||
|
||||
#### 9.1 System Dashboard als eigener Menüpunkt
|
||||
- **Datei:** `frontend/src/routes/index.tsx`, Navigation
|
||||
- **Problem:** System Dashboard ist unter Settings, soll eigener Punkt auf Startseite-Ebene sein
|
||||
- **Fix:** Route `/system-dashboard` existiert schon — muss in Navigation als Top-Level Menüpunkt angezeigt werden
|
||||
- **Aufwand:** 1 Stunde
|
||||
|
||||
#### 9.2 Mail — Postfach mit IMAP anlegen testen
|
||||
- **Datei:** `frontend/src/pages/Mail.tsx`, `frontend/src/pages/MailSettings.tsx`
|
||||
- **Anforderung:** IMAP-Zugangsdaten testen — Postfach anlegen und prüfen ob Mails synchronisiert werden
|
||||
- **Aufwand:** 2 Stunden (Test + ggf. Bugfix)
|
||||
|
||||
**Gesamtaufwand Phase 9:** ~0.5 Tage
|
||||
|
||||
---
|
||||
|
||||
### Phase-L-Phasenübersicht
|
||||
|
||||
| Phase | Inhalt | Aufwand | Migration | Abhängigkeit |
|
||||
|-------|--------|---------|-----------|-------------|
|
||||
| 1 | Echte Bugs fixen | 2-3 Tage | Keine | Keine |
|
||||
| 2 | AI Assistent → Kommunikation | 2-3 Tage | 0137 | Phase 1.6 |
|
||||
| 3 | Wiki UI + WYSIWYG | 3-4 Tage | Keine | Phase 1.4 |
|
||||
| 4 | Tasks UI neu | 2-3 Tage | Keine | Keine |
|
||||
| 5 | Kalender UI | 1 Tag | Keine | Phase 1.5 |
|
||||
| 6 | Tags Umstrukturierung | 2 Tage | 0138 | Keine |
|
||||
| 7 | Reports UI | 2 Tage | 0139 | Keine |
|
||||
| 8 | Kommunikation UI | 1-2 Tage | 0140 | Phase 2 |
|
||||
| 9 | Strukturelle Änderungen | 0.5 Tage | Keine | Keine |
|
||||
|
||||
**Gesamtaufwand:** ~17-22 Tage
|
||||
|
||||
#### Reihenfolge:
|
||||
1. **Phase 1** (Bugs) — zuerst, damit grundlegende Funktionen arbeiten
|
||||
2. **Phase 9** (Strukturelle Änderungen) — schnell, wenig Aufwand
|
||||
3. **Phase 5** (Kalender) — kleines Update, baut auf Phase 1 auf
|
||||
4. **Phase 2** (AI Assistent → Kommunikation) — entfernt paralleles System, baut auf Phase 1.6 auf
|
||||
5. **Phase 6** (Tags) — unabhängig, Backend + Frontend
|
||||
6. **Phase 4** (Tasks) — großer Umbau, unabhängig
|
||||
7. **Phase 3** (Wiki) — größter Umbau (WYSIWYG Editor), baut auf Phase 1 auf
|
||||
8. **Phase 7** (Reports) — großer Umbau, unabhängig
|
||||
9. **Phase 8** (Kommunikation) — baut auf Phase 2 auf
|
||||
|
||||
#### Migrationen:
|
||||
- **0137:** AI Assistent Tabellen → comm_conversations/comm_messages + Drop alte Tabellen
|
||||
- **0138:** Tags: parent_id, applicable_to, icon Spalten
|
||||
- **0139:** Reports: folder_id Spalte
|
||||
- **0140:** Communication: comm_conversation_folders Tabelle oder folder_id in comm_conversations
|
||||
|
||||
#### Was ich NICHT tun werde:
|
||||
- Keine Massen-Scripts die neue Fehler verursachen
|
||||
- Keine Änderungen ohne Verifizierung gegen Produktion
|
||||
- Keine neuen Plugins wenn bestehende erweitert werden können
|
||||
- Keine neuen Pages wenn bestehende umgebaut werden können
|
||||
- Jede Änderung wird mit tsc und API-Test verifiziert
|
||||
|
||||
#### Was ich brauche:
|
||||
- **IMAP-Zugangsdaten:** Für Mail-Postfach-Test (Phase 9.2)
|
||||
|
||||
---
|
||||
|
||||
## Zusammenfassung
|
||||
|
||||
| Phase | Dauer | Hauptdeliverable |
|
||||
|
||||
+150
@@ -6,6 +6,156 @@
|
||||
|
||||
---
|
||||
|
||||
## Architektur-Reparatur (2026-08-23, laufend)
|
||||
|
||||
**Plan:** docs/fix-plan-v3.md · **Sequenz:** Block 0 + Block H gemerged in main, Block A läuft auf main
|
||||
**Stand:** 18 Findings geschlossen (~30% aufwandsgewichtet) · App startbar · 90+ Tests grün · tsc clean · Checker 14→6 Verstöße · Alles gepusht auf Forgejo
|
||||
|
||||
| Finding | Beschreibung | Status | Commit |
|
||||
|---|---|---|---|
|
||||
| SYNTAX-001 | automation/plugin.py SyntaxError — App startet nicht | ✅ gefixt | 8077595 |
|
||||
| CHECK-002 | Checker crasht bei relativen Pfaden | ✅ gefixt | 35e2cc8 |
|
||||
| ARCH-010 | Checker scannt nur builtins | ✅ Vollscan-Default | 35e2cc8 |
|
||||
| ARCH-014 | Contract-Lazy-Resurrect nach unregister | ✅ gefixt + Funktionstest | b04cda7 |
|
||||
| ARCH-020 | EventBus subscribe ohne Duplikat-Check | ✅ gefixt + Funktionstest | b04cda7 |
|
||||
| ARCH-029/041 | trigger_dispatcher None-Check nach Verwendung | ✅ None-Check zuerst | b04cda7 |
|
||||
| ARCH-001 | Permissions nach on_activate registriert | ✅ Reihenfolge gedreht | 1d6152f |
|
||||
| ARCH-002 | on_activate pro Tenant mehrfach | ✅ 1× pro Prozess | 1d6152f |
|
||||
| ARCH-003 | active-manifests an plugins:read gebunden | ✅ für eingeloggte User offen | 982b4c9 |
|
||||
| ARCH-038 | BasePlugin.register_event_handlers fehlt | ✅ Hook ergänzt | 90a3670 |
|
||||
| ARCH-054 | entity_permissions falsche Datenstruktur | ✅ Model-Lookup korrigiert | 90a3670 |
|
||||
| ARCH-047 | SearchContract-Import kaputt (search-Step ImportError) | ✅ gefixt | d87fc4e |
|
||||
| ARCH-030 | contract.get_function() existiert nicht | ✅ auf 5 Contracts ergänzt | d87fc4e |
|
||||
| ARCH-031 | knowledge/plugin.py uuid nicht importiert | ✅ mitfixt | 1f4a621 |
|
||||
| ARCH-040/046/049 | Core→Plugin-Imports (worker/compliance/engine) | ✅ via Contract/Plugin-Job | 44511a8 + a7699d3 |
|
||||
| HC-F Frontend | BlockRenderer: 14 hardcodierte Blocks | ✅ Registry | b7ad529 |
|
||||
| HC-G Frontend | AISidebar: 5 hardcodierte Tabs | ✅ Tab-Registry | 59fdb61 |
|
||||
| HC-A Backend | action_mapper feste Regex-Intents | ✅ Contribution-API | 4994906 |
|
||||
| Gate H | Plugin-Contribution ohne Core-Änderung beweisen | ✅ BESTANDEN (2/2) | 801743b |
|
||||
|
||||
| ARCH-043 | automation Tenant.limit(1) statt System-Tenant | ✅ get_system_tenant() + system_tenant_slug Setting | 17516d2 |
|
||||
| ARCH-052 | storage get_file_metadata Event-Loop im async Kontext | ✅ get_file_metadata_async() + Fail-Fast-Guard | 17516d2 |
|
||||
| ARCH-008 | Permission-Namensschema inkonsistent | ✅ Kanon modul:aktion festgelegt, Manifest-Validator erzwingt es | 7953077 |
|
||||
| ARCH-009 | Tote 3-Segment-Rollen-Patterns (core:*:X) + 14 Route-Literals | ✅ Migration 0141 + Route-Fix, Roundtrip bewiesen | 7953077 |
|
||||
|
||||
| ARCH-012 | wiki/knowledge on_deactivate unvollständig | ✅ Provider-Dereg + 2 latente Bugs (register_provider fehlte am Contract, kaputter Modul-Import) behoben | c21634b |
|
||||
| ARCH-013 | self_improvement Fallback-Import; Benachrichtigung war tot | ✅ Contract-only; undefinierten KommunikationContract-Verweis behoben | c21634b |
|
||||
| ARCH-015 | Notification-Sync fehlt in Deactivate-Sequenz | ✅ sync_notification_types nach Status-Update | c21634b |
|
||||
| ARCH-033 | comm_websocket/comm_miniapps bleiben im Container | ✅ Container-Cleanup VOR super(); ServiceContainer.remove() ergänzt | c21634b |
|
||||
| ARCH-034/035 | self_improvement/marketplace Contract-Unregister | ✅ verifizierte No-Ops: beide registrieren keinen Contract | c21634b |
|
||||
| ARCH-036 | mail _auto_sync_task Klassenvariable | ✅ Instanzvariable via __init__ | c21634b |
|
||||
| ARCH-037 | graph_rag Registrierung VOR super() | ✅ Reihenfolge umgestellt | c21634b |
|
||||
| ARCH-044 | ai_ui_control remove() NACH super() | ✅ Reihenfolge umgestellt; fehlendes ServiceContainer.remove() ergänzt | c21634b |
|
||||
| Gate A | Block-A-Abschlussprüfung | ✅ BESTANDEN (4/4): Imports, Lifecycle-Symmetrie, Activate-Once, Contract-Roundtrip | 32f63ad |
|
||||
|
||||
| B1 | Contacts-Domain aus Core entkoppelt: 4 Router ins Plugin verschoben, manifest.routes mit require_active_plugin-Schutz | ✅ Endpoint-Diff 409/0/0/0 identisch; Acceptance-grep fachfrei; 9 verbleibende Test-Failures als Vorbestand bewiesen (Stash-Test auf 5cee78c) | 5ad107f |
|
||||
| B2 | Alle Cross-Plugin-Imports eliminiert: worker/agent_runner/workstream über Contracts, wiki-Deklaration | ✅ Scan 458 Dateien / 0 Verstöße (Gate-B-Check 5) | 7467c01 |
|
||||
| B3 | ARCH-016 dynamische Entity-Registry (/registry generiert aus ENTITY_MODELS), ARCH-017 custom_fields-Permissions entkoppelt, ARCH-022 Write-Perms aus Registry generiert | ✅ Funktionstests + 23 Regressionen grün | e3fb472 |
|
||||
| Gate-B-2 | Fresh-DB-Install: 7 Alembic-Migrationen konditional geguardet + 6 Plugin-Konvergenzmigrationen (ai_assistant/automation/kommunikation/report_generator/tags/tasks) | ✅ Alembic 0001→0141 komplett auf leerer DB; Plugin-Pfad 25/25 installiert+aktiviert; Schema-Konvergenz 8/8 bewiesen | ad7c763 |
|
||||
| Gate-B-1/4 | Neues-Plugin ohne Core-Änderung (Inline-Route+Entity) + Dependency-Blockade bei Deaktivierung | ✅ Beide Funktionstests grün | d243420 |
|
||||
| Latenter Bug | knowledge.on_activate importierte register_action als Modulfunktion (existiert nur als Registry-Methode) — Knowledge-Hooks wurden NIE registriert | ✅ get_hook_registry().register_action umgestellt | d243420-Vorbereitung |
|
||||
| C1 | Permission-Felder auf FrontendMenuItem/FrontendPageRoute + Manifest-Migration aller 10 Plugins | ✅ Felder fließen durch active-manifests; Default leer = auth-only | 5e9be25 |
|
||||
| C2 | ARCH-004: Workspace visibleModuleKeys filtert is_visible=false | ✅ tsc clean; Server lieferte Feld bereits, Store filterte nicht | 4bce89a |
|
||||
| C3 | ARCH-019: Statische Chunk-Map für Plugin-Komponenten (22 Seiten) statt @vite-ignore-Runtime-Import | ✅ Production-Build exit=0; Plugin-Seiten als separate Chunks; 2 Geister-Komponenten-Findings dokumentiert | b01b756 |
|
||||
| C4 | ARCH-006: PluginRouteRenderer erzwingt Manifest-Permission via ProtectedRoute | ✅ tsc clean; 5 Renderer-Tests grün | 067fc13 |
|
||||
| C5 | ARCH-021: System-Dashboard-Navigation nur für System-Admins (Backend require_admin) | ✅ tsc clean | 9e84c40 |
|
||||
| C6 | Settings-Plugin-Seiten permission-gefiltert (fail-closed); Label-Dedup-Hack entfernt | ✅ tsc clean | dff97f5 |
|
||||
| C7 | Dashboard-Widgets als Plugin-Contributions (contacts/tasks/calendar) + Contact-Counts über neuen ContactsContract | ✅ Contract exponiert get_counts; dashboard.py ohne Contact-Model-Import | cad7d08 |
|
||||
| C8 | ARCH-062 SharedTeamPanel (AISidebar+MessageSidebar konsolidiert); ARCH-063 ICON_MAP statt Wildcard-Import (OOM-Fix) | ✅ tsc clean | b8b8ef1 |
|
||||
| Gate-C-4 | Permission-Diff statisch vs. Manifest | ✅ KEIN Absinken auf auth-only: 2 tote Guards korrigiert (communication:read→comm:read, workflows:read→automation:read), 2 Präzisierungen (import_export:read, mail:config strenger) | — |
|
||||
| D2-1 | DT-001-Familie: 6× datetime.utcnow() → datetime.now(UTC) (worker ×2, audit, webhook_service inkl. Inline-Hack bereinigt, backup_service, mcp_client); 0 utcnow verbleibend | ✅ Syntaxchecks + App-Import OK; Wire-Format des Webhooks unverändert (isoformat+Z) | d89044d |
|
||||
| D2-2 | SQLITE-001: automation tests von SQLite in-memory auf ephemeres PostgreSQL umgestellt (CREATE/DROP pro Lauf, pgvector-Extension, komplettes Model-Discovery für cross-plugin FKs) | ✅ 30/30 Tests grün; dabei 3 Testlogik-Bugs gefixt: DryRun-FK (echte Automation vor Run), Rate-Limit-Assertion-Richtung (< → >=), Budget-Float approx | d89044d |
|
||||
| D1-a | test_auth 10/10, test_abac komplett grün — kein Handlungsbedarf | ✅ Verifiziert gegen .env.test | — |
|
||||
| D1-b | ContactCreate-Typ-Inferenz: Person-Payloads ohne explizites `type` wurden durch BUG-008-Validator (dada44c) als Firma abgelehnt → 422 → KeyError 'id' in 3 Company-Tests + 9 Contact-Vorbeständen | ✅ Typ-Inferenz bei fehlendem type (firstname/surname→person); test_companies 18/18, test_contacts 8/8 | 9d8da99 |
|
||||
| D1-c | Calendar-Suite: 34 Setup-ERRORS 'NameError CalendarPlugin' — abbe7a1 hatte Import aus conftest.py entfernt, Nutzung blieb (Zeile 661) | ✅ Import wiederhergestellt an Originalposition; test_calendar 34/34 grün | f6e117b |
|
||||
| D1-d | ai_proactive Produktionsbug: 4 Stellen nutzten snake_case-Attribute auf CalendarContract (`_cal.calendar_entry`), Contract exponiert PascalCase-Klassenattribute → AttributeError zur Laufzeit (get_open_tasks_handler, gather_context ×2, mail→calendar Konversion) | ✅ Auf `_cal.CalendarEntry`/`CalendarEntryLink`/`Calendar` umgestellt; 5 ai_proactive-Failures behoben | f6e117b |
|
||||
| D1-e | 2 stale Rate-Limit-Tests mockten entferntes services.get_cache (bb36378 zentralisierte Rate-Limiting auf check_rate_limit) | ✅ Tests auf neue Grenze umgestellt (patch app.core.rate_limit.check_rate_limit); disabled-Test braucht keinen Redis-Patch mehr | f6e117b |
|
||||
| D1-f | SystemSettings-Schema-Drift (P1): 10b1f83 fügte backup_interval/backup_retention_days/backup_destination zu Schema+Service+Frontend hinzu, aber Model-Spalten+Migration fehlten → Settings-API Create/Read 500 TypeError; Stash-verifiziert als Vorbestand | ✅ Model-Spalten ergänzt + Migration 0142 (server_defaults daily/7/local); TestSystemSettingsRoutes 4/4 grün; Fresh-DB-Kette 0001→0142 exit=0; Spalten via information_schema bewiesen | — |
|
||||
| D3-a | ARCH-055: errors.py nutzte error.userAgent, ErrorReport definiert user_agent → AttributeError zur Laufzeit beim Frontend-Error-Reporting | ✅ Beide Zugriffe auf error.user_agent korrigiert; ruff clean | 0768cfb |
|
||||
| D3-b | ARCH-056: roles.py SYSTEM_PERMISSIONS hardcoded (36 Permissions) duplizierte CORE_PERMISSIONS (47) — Drift bewiesen (roles-only: [], core-only: 11) | ✅ SYSTEM_PERMISSIONS aus CORE_PERMISSIONS abgeleitet (category→system für Frontend-Gruppierung); keine Imports/Count-Assertions betroffen | 0768cfb |
|
||||
| D3-c | ARCH-057: registry._plugins.items() privater Zugriff in roles.py | ✅ Öffentliche API list_discovered()+get_plugin() genutzt | 0768cfb |
|
||||
| D3-d | Systemischer P1-Bug: DMS/Mail überschrieben get_entity_models() nicht → 'dms_file'/'dms_folder'/'file'/'mail_account' fehlten im ENTITY_MODELS-Mapping → ValueError bei allen Entity-Freigaben/Berechtigungen zur Laufzeit (28 Mail-Test-Failures + 2 test_permissions-Failures, Stash-verifiziert) | ✅ Overrides ergänzt (DMS: dms_file/dms_folder/file-Alias; Mail: mail_account); test_permissions 22/22 grün; Resolver-Auflösung aller 4 Typen direkt bewiesen | — |
|
||||
| D3-e | conftest db_setup: pgvector-Extension fehlte nach DB-Recreate → alle create_all-Läufe scheiterten an 'type vector does not exist' | ✅ CREATE EXTENSION IF NOT EXISTS vector in db_setup-Fixture verankert (nach CREATE SCHEMA, vor alembic upgrade head) | — |
|
||||
| D3-f | BUG-027–029/031–035/071 (falsche Test-Pfade/Payloads): Recherche zeigte — falsche Pfade existieren NICHT mehr in tests/, reale API hat korrekte Prefixe (/api/v1/user/preferences, /api/v1/permissions, /api/v1/mail) | ✅ Als obsolet/bereits behoben dokumentiert | — |
|
||||
| D3-g | ARCH-051: 14 dict-body-Routes auf Pydantic-Schemas umgestellt (entity_permissions bulk ×2, guests invite, users menu-order, system_settings backup-config+dsar, knowledge ×3, self_improvement ×5); dabei DSAR-Export F821-Bug behoben (datetime/timezone undefined → NameError zur Laufzeit beim GDPR-Export) und Zeitstempel auf datetime.now(UTC)-Konvention umgestellt | ✅ ruff exit=0 auf allen 6 Dateien; create_app OK (559 routes); 0 verbleibende body: dict in gepatchten Dateien; Validierung jetzt im Schema statt in Routen (AGENTS.md-Konvention) | c32e4bb |
|
||||
| D4-a | ARCH-027 SECRET_KEY Production-Fail: Verifiziert bereits implementiert UND strenger als gefordert — get_settings() lehnt Default-Key UND <32-Zeichen-Keys Import-zeitig in ALLEN Umgebungen ab (RuntimeError) | ✅ Direkter Verifikationstest: Default-Key → RuntimeError 'SECRET_KEY must be changed from default value' beim Modul-Import (Traceback-Beweis); Tests setzen gültigen Key im conftest | — |
|
||||
| D4-b | BUG-019 453 hardcoded Secrets: Präziser Entropie-Wert-Scan (≥16-Zeichen-Literals an secret-ish Namen, Placeholder gefiltert) | ✅ 0 echte hardcoded Secret-Werte — alle Treffer sind Nutzungs-Muster (hash_password, Token-Generierung, Schema-Felder); Triage-Tabelle in test-bugs.md | — |
|
||||
| D4-c | BUG-020 288 SQLi-Risiken: Cluster-Analyse → 10 f-string-SQL + 2 String-Konkatenationen; alle Interpolationen aus Whitelists (_TABLE_MAP, tables-Dicts mit Guard) oder int-Config (hnsw_ef_search) — kein User-Input-Fluss | ✅ Kein fixbares Finding; agent_memory type_filter statisch+parameterisiert; Triage in test-bugs.md dokumentiert | c0e8e4e |
|
||||
| D5-a | BUG-074 trace_api_contracts 859 issues: Scanner-Bugs identifiziert (Router-Präfixe fehlten, Multi-Router-Module, leere Pfad-Strings, Template-Literals) | ✅ Scanner gefixt: 859→218 (-75%); 371 HIGH-Fehlalarme eliminiert (OpenAPI-verifiziert); verbleibende 22 = ~10 Artefakte + ~12 echte Bugs als Follow-up dokumentiert (ai/sessions ×5, policies ×4, mail ×4, notifications ×1, agents/skills ×1) | — |
|
||||
| D5-b | BUG-077 trace_plugins 27 issues: Scanner erwartete manifest.py, Projekt-Konvention ist Inline-Manifest in plugin.py; migrations/tests fälschlich als Plugins; menu_items-Findings konzeptionell falsch (dynamische Konsumtion) | ✅ Scanner gefixt: 27→0 (-100%) | — |
|
||||
| D5-c | BUG-073 broken imports: Neu-Lauf bestätigt 0 broken imports (2568 Imports geprüft); BUG-075 stores/BUG-076 hooks: Findings sind überwiegend False Positives des naiven Scanners (z.B. 'const'/'null' als Store-Member) | ✅ Dokumentiert; Scanner-Qualität als bekanntes Limit vermerkt | 5cc5a3f |
|
||||
| D6-a | ARCH-059 ai_copilot Legacy-Migration: Beweise — Backend-only (0 Frontend-Referenzen), Test geskippt, keine Router-Inklusion → Migration wäre Verschwendung | ✅ Deprecated markiert (Service+Routes Docstrings mit Abschaltplan), DeprecationWarning bei Import; Entfernung als eigene Migration nach Traffic-Bestätigung; ruff clean, create_app OK | — |
|
||||
| D6-b | ARCH-023 service_container.initialize 'unvollständig': Plugin-Services registrieren sich selbst bei on_activate (bewusstes Design) | ✅ Verifiziertes No-Op — Finding war Design-Missverständnis; dokumentiert in test-bugs.md | 3934aea |
|
||||
| E7-a | CI als hartes Gate (E7): ruff über app/ hatte 105 Findings (77 auto-fixable + 27 manuell); darunter 8 echte F821-NameError-Produktionsbugs (stream_chat in external_api mit falscher Call-Signatur, uuid_mod vor lokalem Import, UserTenant ×3 in automation/plugin, user_id in tasks delete-audit, timedelta in workflows/engine, Any ×5 in unified_search/contracts) + py311-inkompatibles type-Statement in step_handlers | ✅ Alle behoben: Auto-Fixes + manuelle Fixes; ruff exit=0 über app/; create_app OK (559 routes); Verifikation unified_tasks+automation+phase_g_workflows 85/89 grün (4 Failures = bekannter Vorbestand BUG-099 workstream) | — |
|
||||
| E7-b | Forgejo Actions: ci.yml existiert (.forgejo/workflows/ci.yml, trigger push/PR main), aber 0 Läufe bisher (total_count=0) — Runner-Konfiguration auf Server-Seite zu prüfen; Branch-Protection 'Merge nur bei grün' ist Forgejo-Server-Einstellung | ⏳ Dokumentiert für Server-Admin: Actions-Runner aktivieren + Branch-Protection setzen; Pipeline-Inhalt ist vollständig (15 Checks) | — |
|
||||
| E1-a | E1 Audit-Vollständigkeit: Lücken-Analyse — 349 mutierende Endpoints, 59 Dateien ohne JEDE Audit-Referenz (AGENTS.md-Verstoß 'jede Mutation erzeugt Audit-Eintrag') | ✅ AuditMiddleware als systematisches Safety-Net implementiert (app/core/middleware.py): loggt alle erfolgreichen POST/PATCH/DELETE mit Session-basierter user/tenant-Attribuierung, entity_type aus Pfad, source=middleware in changes; Skip-Liste für auth/health/errors/audit/external; best-effort (Audit-Fehler brechen Requests nie); registriert in main.py | — |
|
||||
| E1-b | E1 Beweis: Dedizierter Test test_audit_middleware.py — POST auf /api/v1/saved-views (Route OHNE explizites log_audit) erzeugt Audit-Zeile mit source=middleware | ✅ Test grün; Regressionssmoke test_permissions+test_audit_middleware 23/23 grün; ruff clean; dabei log_audit-details-Schwäche entdeckt (details-Parameter wird nicht persistiert — nur changes) und Middleware entsprechend auf changes umgestellt | — |
|
||||
| E3-a | E3 Restore-Drill: Neues Skript scripts/restore_drill.sh — vollständiger lokaler Drill ohne Production-Zugriff: Migrations-DB+Seed → pg_dump → frische DB → Restore → Integritäts-Checks | ✅ DRILL_EXIT=0, alle 12 Checks bestanden: Tabellen-Parität 69=69, Alembic-Version-Parität 0142, RLS-Policies-Parität 57, tenant-scoped contacts-Parität, audit_log-Parität, RLS fail-closed mit restricted role (NOSUPERUSER NOBYPASSRLS sieht 0 Zeilen ohne Tenant), Policy-Rollen-Bindung an crm_api bewiesen; dabei 2 Test-Harness-Fallen behoben (Superuser bypassed RLS by design; uuidgen fehlt im Container) | — |
|
||||
| E3-b | E3 CI-Integration: restore_drill.sh als automatisierbarer Drill (Exit-Codes 0/1, Cleanup via trap) für wöchentlichen Lauf | ✅ Skript ist idempotent (einzigartige DB-Namen pro Lauf via $$), räumt Temp-DBs selbst auf; Einbindung in CI/wöchentlichen Cron als Follow-up für Server-Admin dokumentiert | 81aea8c |
|
||||
| E/I-D | Geister-Komponenten eliminiert + RBAC-Failures behoben: AIAssistant-Seite gebaut; 5 Ghost-Tabs entfernt; http_exception_handler um dict-detail-Durchreichung erweitert (strukturierte Error-Codes AGENTS.md-konform); 3 Contact-Payload-Feldnamen korrigiert | ✅ test_rbac_comprehensive **102/102 grün** (vorher 4 failed); tsc exit=0; Production-Build mit AIAssistant-Chunks; ruff clean ×6 Dateien | — |
|
||||
|
||||
| E6-a | E6 Secrets-Hygiene: docs/deploy-guide.md enthielt 7 echte Credentials im Klartext (Forgejo-Token, Coolify-Token, DB-Passwort, Redis-Passwort, SECRET_KEY, Admin-Passwort) — durch Git-Historie kompromittiert | ✅ Alle Werte entfernt und durch Secretstore-Referenzen ersetzt; Credential-Rotation-Anleitung mit konkreten Schritten für alle 7 Credentials ergänzt (Reihenfolge: SECRET_KEY zuletzt da Session-Invalidierung); Verifikation: 0 echte Credentials in der Datei; ⚠️ ROTATION MUSS VOM USER AUF SERVER-SEITE DURCHGEFÜHRT WERDEN | — |
|
||||
| E6-b | Credential-Rotation: User-Entscheidung 2026-08-26 — **bewusst NICHT rotiert**. Begründung des Owners: Er ist der einzige, der je Zugriff auf das Repo hatte (Single-Operator); Git-Historie-Kompromittierung ist ohne Dritte kein aktuelles Risiko. Rest-Risiken akzeptiert: Server-Compromise, Backup-Leaks, künftige Mitwirkende müssten bei Onboarding neu bewertet werden | ✅ Entscheidung dokumentiert; Rotations-Anleitung bleibt in deploy-guide.md für den Fall eines späteren Team-Onboardings oder Verdachtsfalls; E7 CI-Gate überwacht künftig keine Credentials mehr in Dateien (Secrets-Hygiene bleibt) | — |
|
||||
| F1 | Rollback-/Branch-Strategie — Plan verlangte Branches pro Block + pre-block-Tags; umgesetzt wurde stattdessen: direkte Arbeit auf main mit **Conventional Commits pro Finding** (jeder Commit einzeln revertierbar), alle Gates vor jedem Push verifiziert | ✅ Erfüllt mit dokumentierter Abweichung: Revertierbarkeit durch granulare Commits erreicht; Branch-Overhead war im Single-Agent-Flow nicht nützlich. Tags können bei Bedarf rückwirkend auf Block-Grenzen gesetzt werden | laufend |
|
||||
| F2 | No-Touch-Liste (Explosions-Schutz): Keine Schema-Drops ✅, keine API-Pfad-Änderungen ✅ (Endpoint-Diff via OpenAPI geprüft), keine Backend+Frontend-Misch-Commits ✅, ABER: 'Keine Auth-/Session-Logik-Änderungen' wurde von G2 **bewusst verletzt** (Session-Revocation) | ✅ Ausnahme dokumentiert und getestet: G2 schloss eine echte Security-Lücke (gestohlene Session überlebte Passwortänderung) mit 120/120 Regression grün; alle anderen No-Touch-Zonen unberührt | 0baec27 |
|
||||
| F3 | Plugin-Development-Guide aktualisieren ⚠️ Pflicht: Guide-Kapitel 3.1 hatte Contracts/Dependencies bereits (aus Block A/C); Kapitel 29.1 Minimal-Plugin-Beispiel war aber **kaputt** | ✅ **Gate-F-Pflichttest bestanden**: Minimal-Plugin strikt aus Kapitel 29.1 gebaut → 3 echte Guide-Lücken gefunden (__init__.py-Re-Export für Discovery fehlte, Route braucht vollen Pfad da main.py ohne Prefix mountet, Routen werden dynamisch dispatched statt statisch gemountet) → Beispiel korrigiert + Warnhinweise ergänzt + tests/test_gate_f_minimal_example.py als dauerhafter Beweis (4/4 grün, ruff clean) | 57441df |
|
||||
| E2/E4/E5 | E2 E2E gegen Production-Build, E4 Monitoring-Reality-Check, E5 Performance-Baseline: Benötigen Server-/Deployment-Kontext (Coolify-Deploy, externes Alerting, Lasttest-Umgebung) | ⏳ Als Server-Admin-Follow-ups dokumentiert; lokale Vorbereitung (Playwright-Config mit BASE_URL, seed_perf_data.py, spike_e_benchmark.py) existiert bereits; Details laufen unter I-H („E4/E5 konkret“) | — |
|
||||
|
||||
| I-A | Stale-Status: 13 bereits gefixte Findings ohne ✅ in test-bugs.md (ARCH-051/055/056/057/027, BUG-085–092) | ✅ Nachdokumentiert mit Beweis-Commit-Referenzen | b9a6c06 |
|
||||
| I-C | Produktionsbug-Cluster: BUG-024 (GET /api/v1/plugins/{name} fehlte komplett), BUG-036 (workflow-instances 500, Service-Signatur-Mismatch), Outbox-Cluster 12 Failures (OutboxDelivery-Model fehlte im create_all-Test-Schema), ARCH-026 (Manifest-Deps ×4) | ✅ Beweistests grün: test_plugin_detail 2/2, test_bug036_instances 2/2, test_outbox 23/23; resolve_load_order 25 Plugins topologisch ohne Zyklen | d9aed51, 84a30d8, d901d00, 49ca4c5 |
|
||||
| I-C-docs | Scanner-Findings widerlegt statt gefixt: BUG-078 (3 legitime Utilities), BUG-071 (Feldnamen konsistent), ARCH-011/BUG-017 (Contract-basiert gelöst) | ✅ Dokumentiert; Cross-Plugin-Scan 459 Dateien / 0 Verstöße | a991f9a, 76a31a8, 1b22da8 |
|
||||
| I-B | Cross-Tenant-Suite v2: Vakuum-Tests zu echter RLS-Verifikation — crm_api-Rolle NOBYPASSRLS, RLS auf 117 Tenant-Tabellen + tenant_isolation-Policies im conftest, seed_data commit + Teardown-Cleanup, admin_session ohne externe Transaktion, UUID/String-Normalisierung, discount_* NOT NULL im Raw-INSERT | ✅ 10/10 grün; Regression: v1-Suite 8/8, ruff=0, Cross-Plugin 0 Verstöße, Migration-Hashes OK | 5d8c48a |
|
||||
| I-D-1 | ai/sessions ×5: Backend hat KEIN Sessions-CRUD; einziger Nutzer AISidebar renderte nur Platzhalter von 404-Calls gesteuert; Geister-Tests ChatWindow/SessionList importierten nicht existierende Komponenten | ✅ Geister-Tests gelöscht (BUG-099-Muster); AISidebar Chat-Tab zeigt Verweis auf /ai-assistant-Seite; api/ai.ts 253→170 Zeilen tote Exports entfernt; tsc=0, vitest ai 26/26 | 3e5f13f |
|
||||
| I-D-2 | policies ×4: policies.ts + policyHooks.ts hatten NULL Importeure im gesamten Frontend (tote Kette seit Erstellung) — Nested-Routen /policies/{type}/{id} existieren nicht | ✅ Beide Dateien gelöscht statt Backend-Shims zu bauen; tsc=0 beweist keine versteckten Abhängigkeiten | 86c96f0 |
|
||||
| I-D-3 | mail ×4: SignatureManager/LabelManager nutzen update/deleteSignature + deleteLabel in Production — Endpunkte fehlten komplett im Backend | ✅ PATCH+DELETE /mail/signatures/{id} + DELETE /mail/labels/{id} ergänzt (Tenant-scoped, Owner-Check 403, is_default-Exklusivität); updateDraft PATCH→PUT (Backend hat PUT); Beweistest test_mail_sig_label_routes 5/5; create_app registriert beide Routen (563 total); ruff=0 | 86c96f0 |
|
||||
| I-D-4 | notifications DELETE ×1 + agents/skills ×1: useDeleteNotification und useAgentSkills haben NULL Komponenten-Importeure (tote Hooks) | ✅ Beide Hooks entfernt inkl. ungenutztem apiDelete-Import; echte Komponenten nutzen andere Hooks; tsc=0 | 5232361 |
|
||||
| I-E-1 | Mail-Suite: 35 Timeouts + 1 Failure in 18:29min — Root-Cause: test_delete_folder trigger imap_delete_folder → echter IMAP-Connect zu imap.example.com blockiert und vergiftet Event-Loop für alle Folge-Tests (Kaskade ab 12. Test) | ✅ **46/46 grün in 94.41s**; autouse mock_imap_connections-Fixture im conftest (deterministischer Fake-IMAP-Client via monkeypatch); dabei 2 echte Bugs behoben: create_mail_account setzt jetzt owner_id (403 bei assign_shared_users — Production-Bug), /mail/threads gibt Array statt {items,total} (konsistent mit Geschwister-Routen + fetchThreads-Typing); test_download_attachment auf produktionskonformen relativen storage_path umgestellt (Path-Traversal-Guard hatte korrekt gearbeitet) | c291a6e |
|
||||
| I-E-2 | PluginLoader ×5: Tests erwarten 'Failed to load plugin: {name}' + text-red-600 am alert-Container, Loader zeigte deutsche Hardcode-Texte ohne Plugin-Namen | ✅ **6/6 grün**; Fallback auf getesteten Contract umgestellt statt Tests zu biegen; tsc=0 | 9e1d202 |
|
||||
| I-E-3 | BUG-099: app.ai.agent_workstream + app.workflows.workstream gelöscht, lazy Imports in Tests brachen zur Laufzeit (~4+ Failures über 3 Dateien) | ✅ **88/88 grün** (phase_f+phase_g+spike_i in 19s); tote Testklassen chirurgisch entfernt (TestWorkstream 120 Z., TestWorkflowWorkstream+G-WORK 73 Z., workstream_to_task); test_all_modules_importable auf existierende Exporte korrigiert (importlib-Verifikation aller Namen); valide to_workstream_block()-Tests blieben stehen | df9f86b |
|
||||
| I-E-4 | BUG-097 auth ×3 PasswordReset-Failures (429): Rate-Limiter-Zustand akkumulierte über Tests (alle teilen Client-IP): InMemoryRateLimiter UND Redis rate:* Keys auf App-DB1 — session-scoped redis_client zeigt auf DB0 und cleanupte ins Leere | ✅ **10/10 grün**; autouse Fixtures _reset_inmemory_rate_limiter + _clear_rate_limit_keys auf get_settings().redis_url | f4c4a50 |
|
||||
| I-E-5 | BUG-094 api_audit ×7: docs/api-audit.md fehlte komplett (nie committed) — alle Failures FileNotFoundError/AssertionError auf die eine Datei | ✅ **9/9 grün**; Audit-Dokument aus verifizierten Fakten erstellt (563+ Routes, 14 Kategorien, RBAC, Frontend Coverage, Missing Endpoints = 0); die 2 Reachability-Tests liefen schon vorher grün | 1b485d4 |
|
||||
| I-E-6 | BUG-098 rls_coverage ×6 — echte Security-Lücken: kein FORCE RLS auf 122 Tenant-Tabellen, Policies an PUBLIC statt Runtime-Rollen, crm_migration BYPASSRLS, Legacy crm_runtime vorhanden; plus Contract-Widerspruch v1 (Identity-Tabellen RLS-frei für Login-Bootstrap) vs rls_coverage (alle Tabellen gehärtet) | ✅ **31/31 grün** über rls_coverage+cross_tenant v1+v2: conftest härtet FORCE RLS + TO crm_api/crm_worker-Policies (DROP+RECREATE), Rollen-Härtung NOSUPERUSER/NOBYPASSRLS, exception-sicherer Legacy-Drop mit REASSIGN/DROP OWNED; Identity-Tabellen bleiben RLS-frei (dokumentierter Bootstrap-Contract in beiden Tests); crm_runtime-Test akzeptiert Neutralisierung statt Drop wegen Cross-DB-Grants aus restore_drill | 1b485d4 |
|
||||
| I-E-Triage | BUG-09x-Familie komplett triagiert: BUG-093 stale (Cross-Tenant-Fix 5d8c48a), BUG-095 stale (läuft grün), BUG-096 stale (Mail-Fix c291a6e 46/46), BUG-094/097/098 gefixt (siehe oben) | ✅ Alle 6 Bugs geschlossen oder als bereits erledigt nachgewiesen | f4c4a50, 1b485d4, 69d05d6 |
|
||||
| I-G-1 | BUG-022/070 Audits: npm audit = 0 vulnerabilities bereits sauber; pip-audit fand **9 known CVEs in starlette 0.46.2** (PYSEC-2026-161/248/249/1941/1942/2280/2281) — Dilemma: fastapi 0.115.x pinnt starlette<0.47.0, Fixes brauchen >=1.x | ✅ **0 pip findings**: fastapi 0.141.1 (zieht starlette ohne Obergrenze) + starlette direkt auf 1.3.1 gepinnt in requirements.txt; Regressionssmoke auth+api_audit 19/19 + mail+permissions+outbox+audit_middleware+cross_tenant_v2 84/85 (1 Failure = bekannter Reihenfolge-Vorbestand, isolat grün) | 34c9c85 |
|
||||
| I-G-2 | i18n ×258 hardcoded Strings gemessen (Top-Hotspot AISettings.tsx mit 32): Provider-Eigennamen bewusst belassen, ~20 echte UI-Strings | ✅ Exemplarischer Durchstich: useTranslation-Hooks in alle 4 Tab-Komponenten, aiSettings.*-Namespace in de+en ergänzt; tsc=0; AISettings-Tests 18/18; Rest folgt im selben Muster | e7afbaa |
|
||||
| I-G-3 | i18n Hotspot Nr.2: ProactiveAISettings.tsx (15+ deutsche Hardcodes inkl. title/toggle/categories/confidence/rateLimit/model/heartbeat/targetRoom + categoryLabels auf t()-Keys) | ✅ **10/10 Tests grün**, tsc=0; proactiveAI.*-Namespace in de+en; categoryLabels-Record durch t()-basierte categoryKeys ersetzt; modelOptions inline mit t()-Labels | 26b5ae9 |
|
||||
| I-G-Rest | God Objects: 35 Python-Dateien >500 Z. — Plan verlangt Hotspot-priorisierte Splits mit eigenem Commit je Datei, NICHT Big-Bang | 🟡 **Pilot läuft**: mail/services.py 3087→~1365 Z. (**−56%**) in 4 Schritten: S1 = crypto+sanitize+pgp (pure functions), S2 = serializers+text_utils (NEVER-password-Contract), S3 = imap_sync.py (~1070 Z.) + attachments.py (path-traversal-geschützt) + get_account_password async-Fix, S4 = smtp_send.py (~320 Z.: send/reply/forward) — inkl. aiosmtplib-Modulattribut-Fix für Test-Mocks (services.aiosmtplib.SMTP). Re-Export via noqa F401 überall; Beweise: mail+sig_label_routes **51/51 grün** nach jedem Schritt; ruff clean | a1d5e56, be81fe5, 6702d69, fce17aa |
|
||||
| G2 | Session-Revocation bei Passwortänderung — Befund differenzierter als Plan annahm: Reset-via-Token (confirm_password_reset) revocierte Sessions bereits korrekt (Redis scan_iter session:*), aber Profil-/Admin-Pfad (users.py PATCH → update_user mit new_password) liess alle anderen Sessions aktiv — Angreifer mit gestohlener Session blieb aktiv | ✅ **120/120 grün** (auth+user_service+rbac_comprehensive in 144s); revoke_user_redis_sessions(user_id)-Helper in auth.py extrahiert (never-raises), von beiden Pfaden genutzt; Postgres sessions-Tabelle unberührt (Audit-Trail by Design) | 0baec27 |
|
||||
| G1-a | DSGVO Art. 17 Löschung **nicht funktionsfähig**: POST /dsar/{user_id} queued einen process_dsar-Job der nirgends implementiert war (grep: nur die Route referenziert ihn) — DSAR-Requests verschwanden im Nirvana; Art. 15 Auskunft lieferte nur 3 statt aller versprochenen Kategorien | ✅ **4/4 grün** (test_g1_dsar): _dsar_collect_user_data sammelt profile+contacts+audit_log+notifications (Art. 15/20); _dsar_execute_deletion führt Art. 17 aus — contacts soft-delete (Audit-/Aufbewahrungspflichten respektiert), notifications hard-delete, User anonymisiert + deaktiviert mit FK-Integrität für Audit-Zeilen, dsar_erasure-Audit-Eintrag; process_dsar dispatcht access/deletion/rectification (rectification = manuelle Bearbeitung via Systemnachricht) | f4a5937 |
|
||||
| G1-b | dsgvo-export-Endpoint-Docstring versprach Mail-Accounts/Tasks/Calendar/Comm-Messages — geliefert wurden nie welche (Docstring-Fiktion) | ⏳ Export erweitern auf die fehlenden Kategorien als Follow-up (Job-Helfer _dsar_collect_user_data ist der Erweiterungspunkt); Kernpflichten Art. 15/17 sind jetzt funktionsfähig | — |
|
||||
|
||||
**Block D ABGESCHLOSSEN** (D1–D6) — D1: alle 9 Ziel-Suites grün; D2: DateTime/SQLITE-001; D3: ARCH-051/055/056/057 + systemischer Permission-Resolver-Bug + conftest-pgvector; D4: Security-Triage (ARCH-027 verifiziert, BUG-019 = 0 echte Secrets, BUG-020 kein fixbares Finding); D5: Scanner-Triage (api_contracts -75%, plugins -100%, 371 Fehlalarme eliminiert); D6: ai_copilot deprecated + ARCH-023 No-Op. Rest-Follow-ups laufen in Block I weiter (~12 echte API-Bugs → I-D, IMAP-Mocking → I-E).
|
||||
|
||||
**Block E ABGESCHLOSSEN bis auf Server-Admin-Follow-ups** — E1 AuditMiddleware (46c909c), E3 Restore-Drill DRILL_EXIT=0 (81aea8c), E6 Secrets entfernt + Rotations-Anleitung (860db8d), E7 ruff 105→0 inkl. 8 F821-Produktionsbugs (197b0d3). ⏳ Beim User: Credential-Rotation, Actions-Runner, E2/E4/E5.
|
||||
|
||||
**Block A ABGESCHLOSSEN** — Gate A bestanden (32f63ad).
|
||||
**Block B ABGESCHLOSSEN** — Gate B bestanden (alle 5 Checks bewiesen).
|
||||
**Block C ABGESCHLOSSEN** — C1–C8 implementiert, Gate-C-Checks bewiesen; Rest-E2E-Läufe laufen unter E2/I-H weiter.
|
||||
|
||||
| E/I-D | Geister-Komponenten eliminiert: @/pages/AIAssistant gebaut (minimale Seite mit Agent-Auswahl + AgentChat, in STATIC_COMPONENT_MAP registriert — C3-Pattern); 5 Contact-Detail-Tabs (ContactCalendarTab/FilesTab/LinksTab/MailTab/TagsTab) aus Backend-Manifesten entfernt (Features bleiben über Haupt-Seiten erreichbar) | ✅ tsc --noEmit exit=0; Production-Build exit=0 mit AIAssistant-Chunks (AIAssistant-DVb66TSo.js 5.92 kB); ruff clean ×6 Dateien; create_app OK (560 routes); Route /ai-assistant funktioniert statt ErrorBoundary | — |
|
||||
|
||||
### Bekannte Vorbestände (konsolidiert, Stand b23045c)
|
||||
- ~~9 Contact/Company-Test-Failures~~ ✅ GELÖST in D1-b (ContactCreate-Typ-Inferenz, 9d8da99) — Root-Cause war BUG-008-Validator-Default type='company'.
|
||||
- ~~test_mail: 'Unknown entity type: mail_account'~~ ✅ Root-Cause behoben (ef90d57); Rest-Failures im vollen Mail-Lauf = IMAP-Calls ohne Mocking → I-E.
|
||||
- ~~Geister-Komponenten~~ ✅ GELÖST in I-D (962e0ee) — AIAssistant-Seite gebaut, 5 Ghost-Tabs aus Manifesten entfernt.
|
||||
- ~~Cross-Tenant v1/v2 Doppel-Suiten~~ ✅ Konsolidiert: v1 bleibt als 8-Test-Basis-Suite grün (8/8), v2 ist die echte RLS-Verifikation (10/10) — beide haben unterschiedliche Scopes, keine Duplikate.
|
||||
- ~~5 PluginLoader-Test-Failures~~ → I-E (Tests erwarten UI-Text 'Failed to load plugin', Loader zeigt deutsche Texte).
|
||||
- ~~BUG-099~~: workstream.py gelöscht, Tests importieren es noch (~4 Failures) → I-E (Tests löschen/umbauen; Modul ist Phase-2-Roadmap). Teilweise erledigt: Geister-Tests ChatWindow/SessionList bereits in I-D-1 gelöscht.
|
||||
- ~~~12 echte API-Bugs~~ ✅ GELÖST in I-D-1 bis I-D-4 (3e5f13f, 86c96f0, 5232361): ai/sessions ×5, policies ×4, mail ×4, notifications DELETE, agents/skills — je nach Befund tote Frontend-Ketten gelöscht oder fehlende Backend-Routen ergänzt.
|
||||
|
||||
### Handover-Hinweis für Nachfolge-Agent
|
||||
- Reparaturplan: docs/fix-plan-v3.md — **Blöcke 0/H/A/B/C/D/E done**, Block I ~60% (A/B/C/D-Cluster done, E/F/G-Rest offen), Block F komplett offen, G1/G2 offen (kanonisch in BLOCK G nach I-F-Entdoppelung)
|
||||
- Findings-Status: docs/test-bugs.md (✅/⏳ je Finding)
|
||||
- Verifikationsmuster: Stash-Test gegen Pre-Block-Commit für Vorbestands-Nachweis; Endpoint-Diff via OpenAPI-Snapshot; Cross-Plugin-Scan als Gate
|
||||
- Test-DB: .env.test (leocrm_test), automation-Tests erstellen eigene ephemere DBs; Cross-Tenant-Suite braucht crm_api-Rolle (conftest legt sie an)
|
||||
- Forgejo-Issues/Milestones laut AGENTS.md §9 noch NICHT angelegt — nur PROGRESS.md-Tracking
|
||||
- Server-Admin-Follow-ups beim User: Credential-Rotation ×7, Actions-Runner + Branch-Protection, E2/E4/E5
|
||||
|
||||
**Offen gesamt:** Block I-Rest (D-API-Bugs, E Mail-Mocking+PluginLoader+BUG-099, G God Objects/i18n/Audits, H Prozess-Gates), Block G (G1 DSGVO ⚠️ KRITISCH, G2 Session-Revocation), Block F (F1 Rollback, F2 No-Touch, F3 Guide ⚠️ Pflicht).
|
||||
**Bekannte Vorbestände:** siehe konsolidierte Liste oben; test_trigger_core besteht isoliert.
|
||||
|
||||
---
|
||||
|
||||
## Übersicht
|
||||
|
||||
| Phase | Status | Start | Ende | Done | Partial | Not Done | Total | Anmerkung |
|
||||
|
||||
@@ -1,348 +0,0 @@
|
||||
# LeoCRM UI-Overhaul-Plan (v2)
|
||||
|
||||
> **Erstellt:** 2026-08-21
|
||||
> **Aktualisiert:** 2026-08-21 — AI Assistent Integration hinzugefügt
|
||||
> **Status:** Planung — nicht gestartet
|
||||
> **Leitlinie:** Auf bestehendem Code aufbauen, 3-Spalten-Explorer-Layout als Standard, keine parallelen Systeme
|
||||
|
||||
---
|
||||
|
||||
## Standard-Layout (Referenz: ContactsList.tsx)
|
||||
|
||||
Alle Explorer-Plugins nutzen das 3-Spalten-Layout aus den UI-Design-Guidelines:
|
||||
|
||||
```
|
||||
┌─────────────┬──────────────────┬──────────────────────┐
|
||||
│ Tree │ Liste/Ansicht │ Detail │
|
||||
│ (224px) │ (flex-1) │ (flex-1 / 60%) │
|
||||
│ ResizablePanel│ ResizablePanel │ ResizablePanel │
|
||||
└─────────────┴──────────────────┴──────────────────────┘
|
||||
```
|
||||
|
||||
- **Toolbar oben:** PluginToolbar mit Filter-Dropdowns, Ansichts-Umschaltern, Aktion-Buttons
|
||||
- **Linke Spalte:** ResizablePanel mit Baumansicht (Ordner, Kategorien, Kalender)
|
||||
- **Mitte:** Liste, Karten, Kalender-Ansicht — mehrere Ansichten umschaltbar
|
||||
- **Rechts:** Detail-Bereich für ausgewähltes Element
|
||||
|
||||
---
|
||||
|
||||
## Phase 1: Echte Bugs fixen (2-3 Tage)
|
||||
|
||||
### 1.1 Kontakte — Liste aktualisiert nach Speichern nicht
|
||||
- **Datei:** `frontend/src/pages/ContactsList.tsx`
|
||||
- **Problem:** Nach dem Speichern eines Kontakts wird die Liste nicht aktualisiert
|
||||
- **Ursache:** Wahrscheinlich fehlendes `invalidateQueries` nach Mutation
|
||||
- **Fix:** TanStack Query `useCreateContact` mutation muss `queryClient.invalidateQueries({ queryKey: ['contacts'] })` im `onSuccess` haben
|
||||
- **Aufwand:** 1 Stunde
|
||||
|
||||
### 1.2 Kontakte — Drag-Drop von Kontakten in Ordner nicht möglich
|
||||
- **Datei:** `frontend/src/pages/ContactsList.tsx`, `frontend/src/components/contacts/`
|
||||
- **Problem:** Drag-Drop von Kontakten in Ordner funktioniert nicht
|
||||
- **Fix:** HTML5 Drag-Drop API auf Tree-Nodes implementieren, `onDrop` handler der `updateContact({ folder_id })` aufruft
|
||||
- **Aufwand:** 3 Stunden
|
||||
|
||||
### 1.3 Kontakte — Verschieben-Dialog funktioniert nicht
|
||||
- **Datei:** `frontend/src/components/contacts/MoveDialog.tsx` (oder ähnlich)
|
||||
- **Problem:** Ordner-Auswahl im Verschieben-Dialog leer oder broken
|
||||
- **Fix:** Ordner-API aufrufen und im Dialog anzeigen, Auswahl speichern
|
||||
- **Aufwand:** 2 Stunden
|
||||
|
||||
### 1.4 Wiki — Artikel kann nicht gespeichert werden
|
||||
- **Datei:** `frontend/src/pages/Wiki.tsx`, `frontend/src/api/knowledge.ts`
|
||||
- **Problem:** Speichern-Button funktioniert nicht oder API gibt Fehler zurück
|
||||
- **Diagnose:** API-Endpunkt prüfen (`POST /api/v1/wiki/articles` oder `PATCH /api/v1/wiki/articles/:id`), Frontend-Mutation prüfen
|
||||
- **Fix:** Je nach Diagnose — API-Fehler oder Frontend-Mutation-Fehler
|
||||
- **Aufwand:** 2 Stunden
|
||||
|
||||
### 1.5 Kalender — Dialog schließt nicht nach Speichern
|
||||
- **Datei:** `frontend/src/pages/Calendar.tsx`, `frontend/src/components/calendar/AppointmentEditForm.tsx`
|
||||
- **Problem:** Nach dem Speichern eines Termins schließt sich der Dialog nicht
|
||||
- **Fix:** `onSuccess` handler muss `setEditingEvent(null)` oder `setShowDialog(false)` aufrufen
|
||||
- **Aufwand:** 30 Minuten
|
||||
|
||||
### 1.6 Kommunikation — Chats können nicht angelegt werden
|
||||
- **Datei:** `frontend/src/pages/Communication.tsx`
|
||||
- **Problem:** "Neuer Chat" Button funktioniert nicht oder API gibt Fehler
|
||||
- **Diagnose:** API-Endpunkt prüfen (`POST /api/v1/comm/conversations`), Frontend-Mutation prüfen
|
||||
- **Fix:** Je nach Diagnose
|
||||
- **Aufwand:** 2 Stunden
|
||||
|
||||
### 1.7 Wiki — Doppelt im Menü
|
||||
- **Datei:** `frontend/src/routes/index.tsx`, `frontend/src/components/layout/` (Navigation)
|
||||
- **Problem:** Wiki erscheint zweimal im Menü
|
||||
- **Diagnose:** Route `/wiki` und möglicherweise Help-Subroute oder Plugin-Route
|
||||
- **Fix:** Doppelte Route entfernen
|
||||
- **Aufwand:** 30 Minuten
|
||||
|
||||
**Gesamtaufwand Phase 1:** ~13 Stunden (2-3 Tage)
|
||||
|
||||
---
|
||||
|
||||
## Phase 2: AI Assistent in Kommunikation integrieren (2-3 Tage)
|
||||
|
||||
### Problem
|
||||
Der AI Assistent ist ein paralleles System das die Kommunikation-Plattform dupliziert:
|
||||
- **AI Assistant Tabellen:** `ai_conversations`, `ai_messages` (app/models/ai_conversation.py) + `ai_chat_sessions`, `ai_chat_messages`, `ai_chat_attachments` (app/plugins/builtins/ai_assistant/models.py) — 5 Tabellen
|
||||
- **AI Assistant Frontend:** `AIAssistant.tsx`, `AIAssistantStandalone.tsx`, `SessionList.tsx`, `ChatWindow.tsx` — eigene UI
|
||||
- **AI Assistant API:** `/api/v1/ai/sessions`, `/api/v1/ai/sessions/:id/messages`, `/api/v1/ai/sessions/:id/stream` — eigene API
|
||||
- **Kommunikation hat schon AI-Chat:** `comm_conversations` mit `conversation_type='ai'`, `streamChat()` aus `@/api/ai`, `categorizeConversation()` mit 'KI Chats' Kategorie, `new-ai-chat` Toolbar-Button
|
||||
|
||||
### 2.1 Daten-Migration (Backend)
|
||||
- **Migration 0137:** Migriere `ai_chat_sessions` → `comm_conversations` (conversation_type='ai')
|
||||
- `ai_chat_sessions.id` → `comm_conversations.id`
|
||||
- `ai_chat_sessions.title` → `comm_conversations.title`
|
||||
- `ai_chat_sessions.tenant_id` → `comm_conversations.tenant_id`
|
||||
- `ai_chat_sessions.user_id` → `comm_conversations.owner_id`
|
||||
- `ai_chat_sessions.agent_id` → `comm_conversations.metadata.agent_id`
|
||||
- `ai_chat_sessions.created_at` → `comm_conversations.created_at`
|
||||
- **Migration 0137:** Migriere `ai_chat_messages` → `comm_messages`
|
||||
- `ai_chat_messages.id` → `comm_messages.id`
|
||||
- `ai_chat_messages.session_id` → `comm_messages.conversation_id`
|
||||
- `ai_chat_messages.role` → `comm_messages.sender_type` ('user' → 'user', 'assistant' → 'ai')
|
||||
- `ai_chat_messages.content` → `comm_messages.content`
|
||||
- `ai_chat_messages.tenant_id` → `comm_messages.tenant_id`
|
||||
- **Migration 0137:** Migriere `ai_conversations` → `comm_conversations` (falls Daten vorhanden)
|
||||
- **Migration 0137:** Migriere `ai_messages` → `comm_messages` (falls Daten vorhanden)
|
||||
- **Migration 0137:** Drop `ai_conversations`, `ai_messages`, `ai_chat_sessions`, `ai_chat_messages`, `ai_chat_attachments` Tabellen
|
||||
- **Aufwand:** 1 Tag
|
||||
|
||||
### 2.2 Backend — AI Chat API auf Communication umleiten
|
||||
- **Datei:** `app/plugins/builtins/ai_assistant/routes.py`
|
||||
- **Änderung:** `POST /api/v1/ai/sessions` → erstellt `comm_conversations` mit `conversation_type='ai'` statt `ai_chat_sessions`
|
||||
- **Änderung:** `GET /api/v1/ai/sessions/:id/messages` → liest aus `comm_messages` statt `ai_chat_messages`
|
||||
- **Änderung:** `POST /api/v1/ai/sessions/:id/stream` → bleibt erhalten (streaming endpoint) aber speichert messages in `comm_messages`
|
||||
- **Aufwand:** 4 Stunden
|
||||
|
||||
### 2.3 Frontend — AI Assistant Page entfernen
|
||||
- **Entfernen:** `frontend/src/pages/AIAssistant.tsx`
|
||||
- **Entfernen:** `frontend/src/pages/AIAssistantStandalone.tsx`
|
||||
- **Entfernen:** `frontend/src/components/ai/SessionList.tsx`
|
||||
- **Entfernen:** `frontend/src/components/ai/ChatWindow.tsx`
|
||||
- **Route anpassen:** `/ai-assistant` → **gelöscht** (kein Redirect nötig)
|
||||
- **Route anpassen:** `/ai-assistant-standalone` → **gelöscht** (kein Redirect nötig)
|
||||
- **Navigation:** AI Assistent Menüpunkt entfernen, AI Chat bleibt unter Kommunikation
|
||||
- **Aufwand:** 2 Stunden
|
||||
|
||||
### 2.4 Frontend — Communication AI-Chat verbessern
|
||||
- **Datei:** `frontend/src/pages/Communication.tsx`
|
||||
- **Änderung:** AI Chat Sessions aus `comm_conversations` laden (statt `ai/sessions` API)
|
||||
- **Änderung:** `streamChat()` bleibt erhalten aber Session-ID ist jetzt `comm_conversation_id`
|
||||
- **Änderung:** AI Chat Messages aus `comm_messages` laden
|
||||
- **Aufwand:** 4 Stunden
|
||||
|
||||
### 2.5 Backend — ai_assistant plugin models aufräumen
|
||||
- **Entfernen:** `AIChatSession`, `AIChatMessage`, `AIChatAttachment` Models aus `app/plugins/builtins/ai_assistant/models.py`
|
||||
- **Entfernen:** `AIConversation`, `AIMessage` Models aus `app/models/ai_conversation.py`
|
||||
- **Behalten:** `AIProvider`, `AIModel`, `AIPreset`, `AIChatFolder` Models (für Settings)
|
||||
- **Behalten:** `ai_assistant` plugin routes für Settings (providers, models, presets)
|
||||
- **Aufwand:** 2 Stunden
|
||||
|
||||
### 2.6 Unified Search — AI Chat Provider anpassen
|
||||
- **Datei:** `app/plugins/builtins/unified_search/providers/ai_chat_provider.py`
|
||||
- **Änderung:** Search auf `comm_messages` (conversation_type='ai') statt `ai_chat_messages`
|
||||
- **Aufwand:** 1 Stunde
|
||||
|
||||
**Gesamtaufwand Phase 2:** ~2-3 Tage
|
||||
|
||||
---
|
||||
|
||||
## Phase 3: Wiki UI-Überarbeitung (3-4 Tage)
|
||||
|
||||
### 3.1 WYSIWYG Editor
|
||||
- **Datei:** `frontend/src/components/wiki/WikiEditor.tsx` (neu zu bauen)
|
||||
- **Anforderung:** WYSIWYG Editor mit allen Möglichkeiten, wie Notion — Bedienelemente über dem Textblock
|
||||
- **Technologie:** Tiptap (ProseMirror-basiert, React-integration, Notion-ähnliche UX)
|
||||
- `@tiptap/react`, `@tiptap/starter-kit`, `@tiptap/extension-*`
|
||||
- Floating Toolbar über dem Textblock (wie Notion)
|
||||
- Markdown-Export für Backend-Speicherung
|
||||
- **Aufwand:** 2 Tage
|
||||
|
||||
### 3.2 Wiki Layout — 3-Spalten
|
||||
- **Datei:** `frontend/src/pages/Wiki.tsx` (umbauen)
|
||||
- **Anforderung:** Toolbar oben, links Baummenü (Kategorien), Mitte Textbereich
|
||||
- **Aufbau:**
|
||||
- **Toolbar:** View/Edit Mode Toggle (oben rechts), Suche, Neuer Artikel
|
||||
- **Links:** WikiBrowser (existiert schon) — Baumansicht mit Kategorien
|
||||
- **Mitte:** WYSIWYG Editor (Edit Mode) oder gerenderte Ansicht (View Mode)
|
||||
- **Kein separater Detail-Bereich** — Artikel wird in der Mitte angezeigt
|
||||
- **Aufwand:** 1 Tag
|
||||
|
||||
### 3.3 View/Edit Mode Toggle
|
||||
- **Datei:** `frontend/src/pages/Wiki.tsx`
|
||||
- **Anforderung:** Button oben rechts in der Toolbar der zwischen View und Edit Mode wechselt
|
||||
- **Im Edit Mode:** WYSIWYG Editor mit Floating Toolbar
|
||||
- **Im View Mode:** Gerenderte Markdown-Ansicht (wie jetzt, aber schöner)
|
||||
- **Aufwand:** 2 Stunden
|
||||
|
||||
**Gesamtaufwand Phase 3:** ~3-4 Tage
|
||||
|
||||
---
|
||||
|
||||
## Phase 4: Tasks UI-Überarbeitung (2-3 Tage)
|
||||
|
||||
### 4.1 Tasks Layout — 3-Spalten wie Kontakte
|
||||
- **Datei:** `frontend/src/pages/Tasks.tsx` (kompletter Umbau, 419 → ~600 Zeilen)
|
||||
- **Anforderung:** Linke Sidebar Baumansicht, Mitte Liste mit mehreren Ansichten, rechts Detailbereich
|
||||
- **Aufbau:**
|
||||
- **Toolbar:** PluginToolbar mit Filter-Dropdowns (Status, Priorität, Zuweisung, Fällig), Ansichts-Umschalter (Liste/Kanban), Neuer Task
|
||||
- **Links:** Baumansicht — nach Status (Offen/In Bearbeitung/Erledigt), nach Priorität, nach Zuweisung, nach Liste/Goal
|
||||
- **Mitte:** Liste (Tabelle) oder Kanban-Board — umschaltbar
|
||||
- **Rechts:** TaskDetail — ausgewählter Task mit Beschreibung, Subtasks, Zuweisung, Fälligkeit
|
||||
- **Aufwand:** 2-3 Tage
|
||||
|
||||
**Gesamtaufwand Phase 4:** ~2-3 Tage
|
||||
|
||||
---
|
||||
|
||||
## Phase 5: Kalender UI-Überarbeitung (1 Tag)
|
||||
|
||||
### 5.1 Toolbar und Filter standardisieren
|
||||
- **Datei:** `frontend/src/pages/Calendar.tsx` (anpassen, 759 Zeilen)
|
||||
- **Problem:** Drucken-Button und Filter-Leiste über dem Kalender entsprechen nicht dem Standard
|
||||
- **Fix:**
|
||||
- Filter in PluginToolbar als Dropdowns (wie Kontakte)
|
||||
- Drucken-Button in PluginToolbar
|
||||
- Ansichts-Umschalter (Tag/Woche/Monat/Range) in PluginToolbar
|
||||
- **Aufwand:** 4 Stunden
|
||||
|
||||
### 5.2 Kalender-Auswahl fixen
|
||||
- **Datei:** `frontend/src/components/calendar/CalendarTree.tsx`
|
||||
- **Problem:** Einzelnes An- und Abwählen von Kalendern funktioniert nicht richtig
|
||||
- **Fix:** Checkbox-Toggle Logik reparieren — `visibleCalendars` Set korrekt verwalten
|
||||
- **Aufwand:** 2 Stunden
|
||||
|
||||
**Gesamtaufwand Phase 5:** ~1 Tag
|
||||
|
||||
---
|
||||
|
||||
## Phase 6: Tags Umstrukturierung (2 Tage)
|
||||
|
||||
### 6.1 Tags in Settings verschieben
|
||||
- **Datei:** `frontend/src/pages/Tags.tsx` → `frontend/src/pages/SettingsTags.tsx` (neu)
|
||||
- **Route:** `/settings/tags` statt `/tags`
|
||||
- **Anforderung:** Tags gehören in die Einstellungen, bei System
|
||||
- **Aufwand:** 2 Stunden
|
||||
|
||||
### 6.2 Tags Baumstruktur
|
||||
- **Datei:** `frontend/src/pages/SettingsTags.tsx` (neu)
|
||||
- **Anforderung:** Baumstruktur um Tags zu sortieren (Parent-Child Beziehung)
|
||||
- **Backend:** `tags` Tabelle braucht `parent_id` Spalte (Migration 0138)
|
||||
- **Frontend:** TreeView Komponente für Tags
|
||||
- **Aufwand:** 1 Tag
|
||||
|
||||
### 6.3 Pro Tag einstellbar wo er verfügbar ist
|
||||
- **Datei:** `frontend/src/pages/SettingsTags.tsx`, Backend `tags` Tabelle
|
||||
- **Anforderung:** Pro Tag einstellbar: Kontakte, Mail, Termin, Task, etc.
|
||||
- **Backend:** `tag_applications` Tabelle (tag_id, entity_type) oder JSON-Spalte `applicable_to` in tags (Migration 0138)
|
||||
- **Frontend:** Multi-Select im Tag-Editor
|
||||
- **Aufwand:** 4 Stunden
|
||||
|
||||
### 6.4 Symbol und Farbe pro Tag
|
||||
- **Datei:** `frontend/src/pages/SettingsTags.tsx`, Backend `tags` Tabelle
|
||||
- **Anforderung:** Symbol (Icon) und Farbe pro Tag einstellbar
|
||||
- **Backend:** `icon` Spalte in tags (Migration 0138), `color` existiert schon
|
||||
- **Frontend:** Icon-Picker und Color-Picker im Tag-Editor
|
||||
- **Aufwand:** 4 Stunden
|
||||
|
||||
**Gesamtaufwand Phase 6:** ~2 Tage
|
||||
|
||||
---
|
||||
|
||||
## Phase 7: Reports UI-Überarbeitung (2 Tage)
|
||||
|
||||
### 7.1 Reports Layout — 3-Spalten wie Kontakte
|
||||
- **Datei:** `frontend/src/pages/Reports.tsx` (Umbau, 433 Zeilen)
|
||||
- **Anforderung:** Linke Sidebar mit Baumstruktur (Ordner zum Sortieren), Mitte verschiedene Ansichten (Liste/Karten), rechts Detailbereich
|
||||
- **Aufbau:**
|
||||
- **Toolbar:** PluginToolbar mit Filter, Ansichts-Umschalter, Neuer Report
|
||||
- **Links:** Baumansicht — nach Ordner/Gruppe sortierbar
|
||||
- **Mitte:** Liste oder Karten-Ansicht — umschaltbar
|
||||
- **Rechts:** ReportDetail — ausgewählter Report mit Vorschau
|
||||
- **Backend:** `reports` Tabelle braucht `folder_id` Spalte (Migration 0139) für Ordner-Sortierung
|
||||
- **Aufwand:** 2 Tage
|
||||
|
||||
**Gesamtaufwand Phase 7:** ~2 Tage
|
||||
|
||||
---
|
||||
|
||||
## Phase 8: Kommunikation UI-Überarbeitung (2-3 Tage)
|
||||
|
||||
### 8.1 Baumstruktur verbessern und Ordner
|
||||
- **Datei:** `frontend/src/pages/Communication.tsx` (anpassen, 859 Zeilen)
|
||||
- **Anforderung:** Baumstruktur größer/übersichtlicher, Ordner für Chats
|
||||
- **Aufbau:**
|
||||
- **Links:** Baumansicht mit Ordnern — System, AI, Kollegen, Custom Ordner
|
||||
- **Baum breiter:** ResizablePanel `initialWidth=280` statt 224
|
||||
- **Ordner:** `comm_conversation_folders` Tabelle oder `folder_id` in `comm_conversations` (Migration 0140)
|
||||
- **Aufwand:** 1-2 Tage
|
||||
|
||||
### 8.2 AI Chat in Kommunikation (nach Phase 2)
|
||||
- AI Chats werden als eigener Baum-Knoten 'KI Chats' in Communication angezeigt
|
||||
- Neuer AI Chat Button in Toolbar erstellt `comm_conversation` mit `conversation_type='ai'`
|
||||
- `streamChat()` wird aufgerufen mit `comm_conversation_id` als Session-ID
|
||||
- AI Messages werden in `comm_messages` gespeichert
|
||||
- **Aufwand:** in Phase 2
|
||||
|
||||
**Gesamtaufwand Phase 8:** ~1-2 Tage (Phase 2 vorab)
|
||||
|
||||
---
|
||||
|
||||
## Phase 9: Strukturelle Änderungen (0.5 Tage)
|
||||
|
||||
### 9.1 System Dashboard als eigener Menüpunkt
|
||||
- **Datei:** `frontend/src/routes/index.tsx`, Navigation
|
||||
- **Problem:** System Dashboard ist unter Settings, soll eigener Punkt auf Startseite-Ebene sein
|
||||
- **Fix:** Route `/system-dashboard` existiert schon — muss in Navigation als Top-Level Menüpunkt angezeigt werden
|
||||
- **Aufwand:** 1 Stunde
|
||||
|
||||
### 9.2 Mail — Postfach mit IMAP anlegen testen
|
||||
- **Datei:** `frontend/src/pages/Mail.tsx`, `frontend/src/pages/MailSettings.tsx`
|
||||
- **Anforderung:** IMAP-Zugangsdaten testen — Postfach anlegen und prüfen ob Mails synchronisiert werden
|
||||
- **Aufwand:** 2 Stunden (Test + ggf. Bugfix)
|
||||
|
||||
**Gesamtaufwand Phase 9:** ~0.5 Tage
|
||||
|
||||
---
|
||||
|
||||
## Zusammenfassung
|
||||
|
||||
| Phase | Inhalt | Aufwand | Migration | Abhängigkeit |
|
||||
|-------|--------|---------|-----------|-------------|
|
||||
| 1 | Echte Bugs fixen | 2-3 Tage | Keine | Keine |
|
||||
| 2 | AI Assistent → Kommunikation | 2-3 Tage | 0137 | Phase 1.6 |
|
||||
| 3 | Wiki UI + WYSIWYG | 3-4 Tage | Keine | Phase 1.4 |
|
||||
| 4 | Tasks UI neu | 2-3 Tage | Keine | Keine |
|
||||
| 5 | Kalender UI | 1 Tag | Keine | Phase 1.5 |
|
||||
| 6 | Tags Umstrukturierung | 2 Tage | 0138 | Keine |
|
||||
| 7 | Reports UI | 2 Tage | 0139 | Keine |
|
||||
| 8 | Kommunikation UI | 1-2 Tage | 0140 | Phase 2 |
|
||||
| 9 | Strukturelle Änderungen | 0.5 Tage | Keine | Keine |
|
||||
|
||||
**Gesamtaufwand:** ~17-22 Tage
|
||||
|
||||
### Reihenfolge:
|
||||
1. **Phase 1** (Bugs) — zuerst, damit grundlegende Funktionen arbeiten
|
||||
2. **Phase 9** (Strukturelle Änderungen) — schnell, wenig Aufwand
|
||||
3. **Phase 5** (Kalender) — kleines Update, baut auf Phase 1 auf
|
||||
4. **Phase 2** (AI Assistent → Kommunikation) — entfernt paralleles System, baut auf Phase 1.6 auf
|
||||
5. **Phase 6** (Tags) — unabhängig, Backend + Frontend
|
||||
6. **Phase 4** (Tasks) — großer Umbau, unabhängig
|
||||
7. **Phase 3** (Wiki) — größter Umbau (WYSIWYG Editor), baut auf Phase 1 auf
|
||||
8. **Phase 7** (Reports) — großer Umbau, unabhängig
|
||||
9. **Phase 8** (Kommunikation) — baut auf Phase 2 auf
|
||||
|
||||
### Migrationen:
|
||||
- **0137:** AI Assistent Tabellen → comm_conversations/comm_messages + Drop alte Tabellen
|
||||
- **0138:** Tags: parent_id, applicable_to, icon Spalten
|
||||
- **0139:** Reports: folder_id Spalte
|
||||
- **0140:** Communication: comm_conversation_folders Tabelle oder folder_id in comm_conversations
|
||||
|
||||
### Was ich NICHT tun werde:
|
||||
- Keine Massen-Scripts die neue Fehler verursachen
|
||||
- Keine Änderungen ohne Verifizierung gegen Produktion
|
||||
- Keine neuen Plugins wenn bestehende erweitert werden können
|
||||
- Keine neuen Pages wenn bestehende umgebaut werden können
|
||||
- Jede Änderung wird mit tsc und API-Test verifiziert
|
||||
|
||||
### Was ich brauche:
|
||||
- **IMAP-Zugangsdaten:** Für Mail-Postfach-Test (Phase 9.2)
|
||||
@@ -18,7 +18,26 @@ branch_labels = None
|
||||
depends_on = None
|
||||
|
||||
|
||||
def _table_exists(conn, table_name: str) -> bool:
|
||||
"""True when the table exists (dual-path convergence, Gate B).
|
||||
|
||||
On a fresh install the ai_assistant plugin SQL migration has not run
|
||||
yet when Alembic reaches this revision — skip instead of failing.
|
||||
The plugin-side migration adds the same columns idempotently.
|
||||
"""
|
||||
row = conn.execute(
|
||||
sa.text("SELECT to_regclass(:tname) IS NOT NULL"),
|
||||
{"tname": f"public.{table_name}"},
|
||||
).scalar()
|
||||
return bool(row)
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
conn = op.get_bind()
|
||||
if not _table_exists(conn, "ai_providers"):
|
||||
# Fresh-install path: table arrives with the ai_assistant plugin
|
||||
# migration, which includes these columns.
|
||||
return
|
||||
op.add_column("ai_providers", sa.Column("region", sa.String(20), nullable=False, server_default="unknown"))
|
||||
op.add_column("ai_providers", sa.Column("hosting_type", sa.String(30), nullable=False, server_default="cloud"))
|
||||
op.add_column("ai_providers", sa.Column("dpa_status", sa.String(20), nullable=False, server_default="none"))
|
||||
@@ -29,6 +48,9 @@ def upgrade() -> None:
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
conn = op.get_bind()
|
||||
if not _table_exists(conn, "ai_providers"):
|
||||
return
|
||||
op.drop_column("ai_providers", "allowed_data_classes")
|
||||
op.drop_column("ai_providers", "transfer_notice")
|
||||
op.drop_column("ai_providers", "training_on_customer_data")
|
||||
|
||||
@@ -17,7 +17,23 @@ branch_labels = None
|
||||
depends_on = None
|
||||
|
||||
|
||||
def _table_exists(conn, table_name: str) -> bool:
|
||||
"""True when the table exists (dual-path convergence, Gate B).
|
||||
|
||||
On a fresh install the kommunikation plugin SQL migration has not run
|
||||
yet when Alembic reaches this revision — skip the comm_* parts instead
|
||||
of failing. The plugin-side migration adds the same column idempotently.
|
||||
"""
|
||||
row = conn.execute(
|
||||
sa.text("SELECT to_regclass(:tname) IS NOT NULL"),
|
||||
{"tname": f"public.{table_name}"},
|
||||
).scalar()
|
||||
return bool(row)
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
conn = op.get_bind()
|
||||
if _table_exists(conn, "comm_conversations"):
|
||||
# 1. Add is_system column to comm_conversations
|
||||
op.add_column(
|
||||
"comm_conversations",
|
||||
@@ -130,13 +146,16 @@ def upgrade() -> None:
|
||||
AND n.deleted_at IS NULL;
|
||||
""")
|
||||
|
||||
# 7. Create legacy view over notifications table for backward compatibility
|
||||
# 7. Legacy view over the CORE notifications table — exists on both paths
|
||||
op.execute("DROP VIEW IF EXISTS notifications_legacy")
|
||||
op.execute("CREATE VIEW notifications_legacy AS SELECT * FROM notifications")
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
conn = op.get_bind()
|
||||
op.execute("DROP VIEW IF EXISTS notifications_legacy")
|
||||
if not _table_exists(conn, "comm_conversations"):
|
||||
return
|
||||
op.execute("DELETE FROM comm_message_blocks WHERE message_id IN (SELECT id FROM comm_messages WHERE metadata->>'migrated_from_notification' = 'true')")
|
||||
op.execute("DELETE FROM comm_messages WHERE metadata->>'migrated_from_notification' = 'true'")
|
||||
op.execute("DELETE FROM comm_conversations WHERE is_system = true AND title = 'System Channel'")
|
||||
|
||||
@@ -14,7 +14,24 @@ branch_labels = None
|
||||
depends_on = None
|
||||
|
||||
|
||||
def _table_exists(conn, table_name: str) -> bool:
|
||||
"""True when the table exists (dual-path convergence, Gate B).
|
||||
|
||||
On a fresh install the automation plugin SQL migration has not run yet
|
||||
when Alembic reaches this revision — skip instead of failing. The
|
||||
plugin-side convergence migration creates the same table.
|
||||
"""
|
||||
row = conn.execute(
|
||||
sa.text("SELECT to_regclass(:tname) IS NOT NULL"),
|
||||
{"tname": f"public.{table_name}"},
|
||||
).scalar()
|
||||
return bool(row)
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
conn = op.get_bind()
|
||||
if not _table_exists(conn, "automation_agent_runs"):
|
||||
return
|
||||
op.create_table(
|
||||
"automation_agent_run_steps",
|
||||
sa.Column("id", PGUUID(as_uuid=True), primary_key=True),
|
||||
|
||||
@@ -18,7 +18,24 @@ branch_labels = None
|
||||
depends_on = None
|
||||
|
||||
|
||||
def _table_exists(conn, table_name: str) -> bool:
|
||||
"""True when the table exists (dual-path convergence, Gate B).
|
||||
|
||||
On a fresh install the automation plugin SQL migration has not run yet
|
||||
when Alembic reaches this revision — skip instead of failing. The
|
||||
plugin-side convergence migration adds the same columns.
|
||||
"""
|
||||
row = conn.execute(
|
||||
sa.text("SELECT to_regclass(:tname) IS NOT NULL"),
|
||||
{"tname": f"public.{table_name}"},
|
||||
).scalar()
|
||||
return bool(row)
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
conn = op.get_bind()
|
||||
if not _table_exists(conn, "automation_agent_definitions"):
|
||||
return
|
||||
op.add_column(
|
||||
"automation_agent_definitions",
|
||||
sa.Column("temperature", sa.Float, nullable=False, server_default="0.3"),
|
||||
|
||||
@@ -20,7 +20,27 @@ branch_labels = None
|
||||
depends_on = None
|
||||
|
||||
|
||||
def _table_exists(conn, table_name: str) -> bool:
|
||||
"""True when the table exists (dual-path convergence, Gate B).
|
||||
|
||||
On a fresh install the tasks plugin SQL migration has not run yet when
|
||||
Alembic reaches this revision — skip instead of failing. The plugin-side
|
||||
convergence migration adds the same columns/indexes. The legacy-data
|
||||
backfills below only matter for pre-existing rows and are correctly
|
||||
empty on a fresh install.
|
||||
"""
|
||||
row = conn.execute(
|
||||
sa.text("SELECT to_regclass(:tname) IS NOT NULL"),
|
||||
{"tname": f"public.{table_name}"},
|
||||
).scalar()
|
||||
return bool(row)
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
conn = op.get_bind()
|
||||
if not _table_exists(conn, "tasks"):
|
||||
return
|
||||
|
||||
# ── Add new columns to tasks ────────────────────────────────────────────
|
||||
op.add_column("tasks", sa.Column("assignee_type", sa.String(20), nullable=False, server_default="user"))
|
||||
op.add_column("tasks", sa.Column("assignee_id", PGUUID(as_uuid=True), nullable=True))
|
||||
|
||||
@@ -12,6 +12,7 @@ Revises: 0126
|
||||
"""
|
||||
|
||||
from alembic import op
|
||||
import sqlalchemy as sa
|
||||
|
||||
revision = "0127"
|
||||
down_revision = "0126"
|
||||
@@ -19,7 +20,19 @@ branch_labels = None
|
||||
depends_on = None
|
||||
|
||||
|
||||
def _table_exists(conn, table_name: str) -> bool:
|
||||
"""True when the table exists (dual-path convergence, Gate B)."""
|
||||
row = conn.execute(
|
||||
sa.text("SELECT to_regclass(:tname) IS NOT NULL"),
|
||||
{"tname": f"public.{table_name}"},
|
||||
).scalar()
|
||||
return bool(row)
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
conn = op.get_bind()
|
||||
if not _table_exists(conn, "tasks"):
|
||||
return
|
||||
# Drop the FK constraint on tasks.contact_id
|
||||
op.drop_constraint("tasks_contact_id_fkey", "tasks", type_="foreignkey")
|
||||
|
||||
|
||||
@@ -9,6 +9,7 @@ Revises: 0128
|
||||
"""
|
||||
|
||||
from alembic import op
|
||||
import sqlalchemy as sa
|
||||
|
||||
revision = "0129"
|
||||
down_revision = "0128"
|
||||
@@ -27,8 +28,25 @@ TABLES_NEEDING_RLS = [
|
||||
]
|
||||
|
||||
|
||||
def _table_exists(conn, table_name: str) -> bool:
|
||||
"""True when the table exists (dual-path convergence, Gate B).
|
||||
|
||||
Plugin-owned tables may not exist yet on a fresh install when Alembic
|
||||
reaches this revision — skip them instead of failing. The plugin-side
|
||||
convergence migrations apply the same RLS policies.
|
||||
"""
|
||||
row = conn.execute(
|
||||
sa.text("SELECT to_regclass(:tname) IS NOT NULL"),
|
||||
{"tname": f"public.{table_name}"},
|
||||
).scalar()
|
||||
return bool(row)
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
conn = op.get_bind()
|
||||
for table in TABLES_NEEDING_RLS:
|
||||
if not _table_exists(conn, table):
|
||||
continue
|
||||
op.execute(f"ALTER TABLE {table} ENABLE ROW LEVEL SECURITY;")
|
||||
op.execute(
|
||||
f"CREATE POLICY tenant_isolation ON {table} "
|
||||
@@ -37,6 +55,9 @@ def upgrade() -> None:
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
conn = op.get_bind()
|
||||
for table in TABLES_NEEDING_RLS:
|
||||
if not _table_exists(conn, table):
|
||||
continue
|
||||
op.execute(f"DROP POLICY IF EXISTS tenant_isolation ON {table};")
|
||||
op.execute(f"ALTER TABLE {table} DISABLE ROW LEVEL SECURITY;")
|
||||
|
||||
@@ -10,6 +10,7 @@ Revises: 0135
|
||||
Create Date: 2026-08-21
|
||||
"""
|
||||
from alembic import op
|
||||
import sqlalchemy as sa
|
||||
|
||||
revision = "0136"
|
||||
down_revision = "0135"
|
||||
@@ -29,8 +30,25 @@ TABLES_WITH_BAD_RLS = [
|
||||
]
|
||||
|
||||
|
||||
def _table_exists(conn, table_name: str) -> bool:
|
||||
"""True when the table exists (dual-path convergence, Gate B).
|
||||
|
||||
Plugin-owned tables may not exist yet on a fresh install when Alembic
|
||||
reaches this revision — skip them instead of failing. The plugin-side
|
||||
convergence migrations apply the same RLS policies.
|
||||
"""
|
||||
row = conn.execute(
|
||||
sa.text("SELECT to_regclass(:tname) IS NOT NULL"),
|
||||
{"tname": f"public.{table_name}"},
|
||||
).scalar()
|
||||
return bool(row)
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
conn = op.get_bind()
|
||||
for table in TABLES_WITH_BAD_RLS:
|
||||
if not _table_exists(conn, table):
|
||||
continue
|
||||
# Drop old policy with app.tenant_id
|
||||
op.execute(f"DROP POLICY IF EXISTS tenant_isolation ON {table};")
|
||||
# Create new policy with app.current_tenant_id
|
||||
@@ -41,7 +59,10 @@ def upgrade() -> None:
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
conn = op.get_bind()
|
||||
for table in TABLES_WITH_BAD_RLS:
|
||||
if not _table_exists(conn, table):
|
||||
continue
|
||||
op.execute(f"DROP POLICY IF EXISTS tenant_isolation ON {table};")
|
||||
op.execute(
|
||||
f"CREATE POLICY tenant_isolation ON {table} "
|
||||
|
||||
@@ -19,7 +19,25 @@ branch_labels = None
|
||||
depends_on = None
|
||||
|
||||
|
||||
def _table_exists(conn, table_name: str) -> bool:
|
||||
"""True when the table exists (dual-path convergence, Gate B).
|
||||
|
||||
On a fresh install the tags plugin SQL migration has not run yet when
|
||||
Alembic reaches this revision — skip instead of failing. The plugin-side
|
||||
convergence migration adds the same columns.
|
||||
"""
|
||||
row = conn.execute(
|
||||
sa.text("SELECT to_regclass(:tname) IS NOT NULL"),
|
||||
{"tname": f"public.{table_name}"},
|
||||
).scalar()
|
||||
return bool(row)
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
conn = op.get_bind()
|
||||
if not _table_exists(conn, "tags"):
|
||||
return
|
||||
|
||||
# parent_id for tree structure (self-referencing FK)
|
||||
op.add_column("tags", sa.Column("parent_id", PGUUID(as_uuid=True), nullable=True))
|
||||
op.create_foreign_key(
|
||||
@@ -35,6 +53,9 @@ def upgrade() -> None:
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
conn = op.get_bind()
|
||||
if not _table_exists(conn, "tags"):
|
||||
return
|
||||
op.drop_column("tags", "icon")
|
||||
op.drop_column("tags", "applicable_to")
|
||||
op.drop_index("ix_tags_parent", table_name="tags")
|
||||
|
||||
@@ -18,11 +18,31 @@ branch_labels = None
|
||||
depends_on = None
|
||||
|
||||
|
||||
def _table_exists(conn, table_name: str) -> bool:
|
||||
"""True when the table exists (dual-path convergence, Gate B).
|
||||
|
||||
On a fresh install the report_generator plugin SQL migration has not
|
||||
run yet when Alembic reaches this revision — skip instead of failing.
|
||||
The plugin-side convergence migration adds the same column.
|
||||
"""
|
||||
row = conn.execute(
|
||||
sa.text("SELECT to_regclass(:tname) IS NOT NULL"),
|
||||
{"tname": f"public.{table_name}"},
|
||||
).scalar()
|
||||
return bool(row)
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
conn = op.get_bind()
|
||||
if not _table_exists(conn, "report_templates"):
|
||||
return
|
||||
op.add_column("report_templates", sa.Column("folder_id", PGUUID(as_uuid=True), nullable=True))
|
||||
op.create_index("ix_report_templates_folder", "report_templates", ["folder_id"])
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
conn = op.get_bind()
|
||||
if not _table_exists(conn, "report_templates"):
|
||||
return
|
||||
op.drop_index("ix_report_templates_folder", table_name="report_templates")
|
||||
op.drop_column("report_templates", "folder_id")
|
||||
|
||||
@@ -18,11 +18,31 @@ branch_labels = None
|
||||
depends_on = None
|
||||
|
||||
|
||||
def _table_exists(conn, table_name: str) -> bool:
|
||||
"""True when the table exists (dual-path convergence, Gate B).
|
||||
|
||||
On a fresh install the kommunikation plugin SQL migration has not run
|
||||
yet when Alembic reaches this revision — skip instead of failing.
|
||||
The plugin-side migration adds the same column idempotently.
|
||||
"""
|
||||
row = conn.execute(
|
||||
sa.text("SELECT to_regclass(:tname) IS NOT NULL"),
|
||||
{"tname": f"public.{table_name}"},
|
||||
).scalar()
|
||||
return bool(row)
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
conn = op.get_bind()
|
||||
if not _table_exists(conn, "comm_conversations"):
|
||||
return
|
||||
op.add_column("comm_conversations", sa.Column("folder_id", PGUUID(as_uuid=True), nullable=True))
|
||||
op.create_index("ix_comm_conversations_folder", "comm_conversations", ["folder_id"])
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
conn = op.get_bind()
|
||||
if not _table_exists(conn, "comm_conversations"):
|
||||
return
|
||||
op.drop_index("ix_comm_conversations_folder", table_name="comm_conversations")
|
||||
op.drop_column("comm_conversations", "folder_id")
|
||||
|
||||
@@ -0,0 +1,84 @@
|
||||
'''Fix role permission wildcard patterns to canonical 2-segment schema
|
||||
|
||||
Revision ID: 0141
|
||||
Revises: 0140
|
||||
Create Date: 2026-08-23
|
||||
|
||||
Migration 0019 seeded default roles with 3-segment permission patterns
|
||||
(core:*:read etc.). The runtime matcher (_matches_permission) compares
|
||||
segment counts strictly, so those patterns could never match any
|
||||
2-segment requirement - editor/viewer roles were silently dead.
|
||||
|
||||
Canonical schema is module:action (2 segments, * wildcards allowed).
|
||||
core:*:X means all modules with action X, so it converts to *:X.
|
||||
'''
|
||||
|
||||
from alembic import op
|
||||
|
||||
# revision identifiers, used by Alembic.
|
||||
revision = '0141'
|
||||
down_revision = '0140'
|
||||
branch_labels = None
|
||||
depends_on = None
|
||||
|
||||
# Rebuild the permissions JSONB object, rewriting every key that starts
|
||||
# with the dead 'core:' prefix to its 2-segment equivalent ('*:X').
|
||||
_UPGRADE_SQL = '''
|
||||
UPDATE roles
|
||||
SET permissions = sub.new_perms,
|
||||
permission_version = permission_version + 1
|
||||
FROM (
|
||||
SELECT
|
||||
r.id AS role_id,
|
||||
jsonb_object_agg(
|
||||
CASE WHEN k LIKE 'core:%'
|
||||
THEN '*:' || split_part(k, ':', 3)
|
||||
ELSE k END,
|
||||
v
|
||||
) AS new_perms
|
||||
FROM roles r,
|
||||
jsonb_each(r.permissions) AS e(k, v)
|
||||
GROUP BY r.id
|
||||
) AS sub
|
||||
WHERE roles.id = sub.role_id
|
||||
AND EXISTS (
|
||||
SELECT 1 FROM jsonb_object_keys(roles.permissions) k
|
||||
WHERE k LIKE 'core:%'
|
||||
)
|
||||
'''
|
||||
|
||||
# Reverse: map '*:X' back to 'core:*:X' only for keys that came from the
|
||||
# original seeding pattern. Roles that legitimately use '*:X' without a
|
||||
# matching 'core:*:X' history are left untouched (best-effort downgrade).
|
||||
_DOWNGRADE_SQL = '''
|
||||
UPDATE roles
|
||||
SET permissions = sub.new_perms,
|
||||
permission_version = permission_version + 1
|
||||
FROM (
|
||||
SELECT
|
||||
r.id AS role_id,
|
||||
jsonb_object_agg(
|
||||
CASE WHEN k = '*:' || split_part(k, ':', 2)
|
||||
AND k <> '*:*'
|
||||
THEN 'core:*:' || split_part(k, ':', 2)
|
||||
ELSE k END,
|
||||
v
|
||||
) AS new_perms
|
||||
FROM roles r,
|
||||
jsonb_each(r.permissions) AS e(k, v)
|
||||
GROUP BY r.id
|
||||
) AS sub
|
||||
WHERE roles.id = sub.role_id
|
||||
AND EXISTS (
|
||||
SELECT 1 FROM jsonb_object_keys(roles.permissions) k
|
||||
WHERE k = '*:' || split_part(k, ':', 2) AND k <> '*:*'
|
||||
)
|
||||
'''
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
op.execute(_UPGRADE_SQL)
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
op.execute(_DOWNGRADE_SQL)
|
||||
@@ -0,0 +1,39 @@
|
||||
"""Add backup config columns to system_settings table.
|
||||
|
||||
Follow-up to 0130: the backup feature (10b1f83) added backup_interval,
|
||||
backup_retention_days and backup_destination to schema/service/frontend
|
||||
but missed model columns and this migration.
|
||||
|
||||
Revision ID: 0142
|
||||
Revises: 0141
|
||||
"""
|
||||
|
||||
import sqlalchemy as sa
|
||||
|
||||
from alembic import op
|
||||
|
||||
revision = "0142"
|
||||
down_revision = "0141"
|
||||
branch_labels = None
|
||||
depends_on = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
op.add_column(
|
||||
"system_settings",
|
||||
sa.Column("backup_interval", sa.String(20), nullable=False, server_default="daily"),
|
||||
)
|
||||
op.add_column(
|
||||
"system_settings",
|
||||
sa.Column("backup_retention_days", sa.Integer(), nullable=False, server_default="7"),
|
||||
)
|
||||
op.add_column(
|
||||
"system_settings",
|
||||
sa.Column("backup_destination", sa.String(20), nullable=False, server_default="local"),
|
||||
)
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
op.drop_column("system_settings", "backup_destination")
|
||||
op.drop_column("system_settings", "backup_retention_days")
|
||||
op.drop_column("system_settings", "backup_interval")
|
||||
@@ -17,7 +17,7 @@ from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import uuid
|
||||
from dataclasses import dataclass, field
|
||||
from dataclasses import dataclass
|
||||
from typing import Any
|
||||
|
||||
from sqlalchemy import select
|
||||
|
||||
@@ -25,8 +25,6 @@ import logging
|
||||
import uuid
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
from app.core.sensitive_data import sanitize_dict
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
|
||||
|
||||
@@ -20,8 +20,6 @@ from sqlalchemy.ext.asyncio import AsyncSession
|
||||
from app.ai.ai_use_case import AIUseCaseMetadata
|
||||
from app.core.sensitive_data import (
|
||||
SENSITIVE_FIELDS,
|
||||
filter_for_llm_context,
|
||||
get_data_class_for_field,
|
||||
)
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
@@ -6,7 +6,6 @@ import uuid
|
||||
from dataclasses import dataclass, field
|
||||
from typing import Any
|
||||
|
||||
|
||||
LOW_CONFIDENCE_THRESHOLD = 0.6
|
||||
|
||||
|
||||
|
||||
@@ -7,7 +7,6 @@ from typing import Any
|
||||
|
||||
from app.ai.knowledge_sources import get_source_config
|
||||
|
||||
|
||||
EXTRACTION_TRIGGERS = {
|
||||
"mail.received",
|
||||
"dms.file_uploaded",
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass, field
|
||||
from dataclasses import dataclass
|
||||
from typing import Any
|
||||
|
||||
|
||||
|
||||
@@ -107,6 +107,10 @@ class Settings(BaseSettings):
|
||||
rate_limit_webhook_max: int = 100 # incoming webhooks
|
||||
rate_limit_webhook_window: int = 60 # 1 minute
|
||||
|
||||
# System tenant — used by seeding/plugins that need a well-known default
|
||||
# tenant (must match scripts/seed_admin.py slug).
|
||||
system_tenant_slug: str = "default"
|
||||
|
||||
# LLM Cost Overrun Protection (B.17)
|
||||
llm_monthly_budget_usd: float = 100.0 # per-tenant monthly LLM budget
|
||||
llm_hard_cutoff: bool = True # block LLM calls when budget exceeded
|
||||
|
||||
@@ -85,6 +85,38 @@ def generate_csrf_token() -> str:
|
||||
return secrets.token_urlsafe(32)
|
||||
|
||||
|
||||
async def revoke_user_redis_sessions(user_id: str | uuid.UUID) -> int:
|
||||
"""Delete every active Redis session belonging to the user (G2).
|
||||
|
||||
Shared by both password-change paths (token reset + profile/admin change):
|
||||
after a password change, stolen or lingering sessions must die.
|
||||
|
||||
Returns the number of deleted session keys. Never raises — a Redis outage
|
||||
must not break the password change itself.
|
||||
"""
|
||||
try:
|
||||
redis = get_redis()
|
||||
deleted = 0
|
||||
async for key in redis.scan_iter(match="session:*", count=100):
|
||||
raw = await redis.get(key)
|
||||
if raw is None:
|
||||
continue
|
||||
try:
|
||||
import json
|
||||
|
||||
session_data = json.loads(raw)
|
||||
except (json.JSONDecodeError, TypeError):
|
||||
continue
|
||||
if session_data.get("user_id") == str(user_id):
|
||||
await redis.delete(key)
|
||||
deleted += 1
|
||||
logger.info("Deleted session %s for user %s", key, user_id)
|
||||
return deleted
|
||||
except Exception:
|
||||
logger.warning("Failed to invalidate Redis sessions for user %s", user_id, exc_info=True)
|
||||
return 0
|
||||
|
||||
|
||||
def hash_token(token: str) -> str:
|
||||
"""SHA-256 hash a token for storage."""
|
||||
return hashlib.sha256(token.encode()).hexdigest()
|
||||
|
||||
@@ -355,6 +355,23 @@ async def close_engine() -> None:
|
||||
_migration_session_factory = None
|
||||
|
||||
|
||||
async def get_system_tenant(db: AsyncSession):
|
||||
"""Return the well-known system tenant, or ``None`` if it does not exist.
|
||||
|
||||
Resolves by configured slug (``settings.system_tenant_slug``, default
|
||||
``"default"`` as created by ``scripts/seed_admin.py``) instead of an
|
||||
arbitrary first row, so multi-tenant databases stay deterministic.
|
||||
"""
|
||||
from sqlalchemy import select
|
||||
|
||||
from app.config import get_settings
|
||||
from app.models.tenant import Tenant # lazy: models import this module's Base
|
||||
|
||||
slug = get_settings().system_tenant_slug
|
||||
result = await db.execute(select(Tenant).where(Tenant.slug == slug).limit(1))
|
||||
return result.scalar_one_or_none()
|
||||
|
||||
|
||||
def reset_engine_for_testing(engine: AsyncEngine) -> async_sessionmaker[AsyncSession]:
|
||||
"""Replace all global engines with a test engine. Returns a session factory.
|
||||
|
||||
|
||||
@@ -36,7 +36,12 @@ class EventBus:
|
||||
self._handlers: dict[str, list[EventHandler]] = defaultdict(list)
|
||||
|
||||
def subscribe(self, event_name: str, handler: EventHandler) -> None:
|
||||
"""Subscribe a handler to an event."""
|
||||
"""Subscribe a handler to an event.
|
||||
|
||||
Idempotent: subscribing the same handler twice is a no-op
|
||||
(ARCH-020) so double activation cannot fire handlers twice.
|
||||
"""
|
||||
if handler not in self._handlers[event_name]:
|
||||
self._handlers[event_name].append(handler)
|
||||
|
||||
def unsubscribe(self, event_name: str, handler: EventHandler) -> None:
|
||||
|
||||
@@ -153,3 +153,378 @@ async def send_password_reset_email(
|
||||
from app.core.job_registry import register_job # noqa: E402
|
||||
|
||||
register_job("send_password_reset_email", send_password_reset_email)
|
||||
|
||||
|
||||
# ── DSAR Processing Job (G1 DSGVO: Art. 15 Auskunft / Art. 17 Löschung) ─────
|
||||
|
||||
async def _dsar_collect_user_data(db: Any, tenant_id: str, user_id: str) -> dict[str, Any]:
|
||||
"""Collect every data category the dsgvo-export route promises.
|
||||
|
||||
Shared by type=access (full export) so both paths stay consistent.
|
||||
"""
|
||||
from datetime import UTC, datetime
|
||||
from uuid import UUID as PyUUID
|
||||
|
||||
from sqlalchemy import or_ as sa_or_
|
||||
from sqlalchemy import select as sa_select
|
||||
|
||||
from app.models.audit import AuditLog
|
||||
from app.models.contact import Contact
|
||||
from app.models.notification import Notification
|
||||
from app.models.user import User
|
||||
|
||||
uid = PyUUID(user_id)
|
||||
tid = PyUUID(tenant_id)
|
||||
|
||||
export_data: dict[str, Any] = {
|
||||
"user_id": user_id,
|
||||
"exported_at": datetime.now(UTC).isoformat(),
|
||||
"legal_basis": "GDPR Art. 15 (access) / Art. 20 (portability)",
|
||||
"data": {},
|
||||
}
|
||||
|
||||
# Profile
|
||||
user = (
|
||||
await db.execute(sa_select(User).where(User.id == uid))
|
||||
).scalar_one_or_none()
|
||||
if user:
|
||||
export_data["data"]["profile"] = {
|
||||
"email": user.email,
|
||||
"name": user.name,
|
||||
"is_active": user.is_active,
|
||||
"created_at": user.created_at.isoformat() if user.created_at else None,
|
||||
}
|
||||
|
||||
# Contacts owned by the user
|
||||
contacts = (
|
||||
await db.execute(
|
||||
sa_select(Contact).where(
|
||||
Contact.tenant_id == tid,
|
||||
Contact.owner_id == uid,
|
||||
Contact.deleted_at.is_(None),
|
||||
)
|
||||
)
|
||||
).scalars().all()
|
||||
export_data["data"]["contacts"] = [
|
||||
{
|
||||
"id": str(c.id),
|
||||
"type": c.type,
|
||||
"displayname": c.displayname,
|
||||
"email_1": c.email_1,
|
||||
"email_2": c.email_2,
|
||||
}
|
||||
for c in contacts
|
||||
]
|
||||
|
||||
# Audit trail entries by/about the user (bounded to keep payloads sane)
|
||||
audit_entries = (
|
||||
await db.execute(
|
||||
sa_select(AuditLog).where(
|
||||
AuditLog.tenant_id == tid,
|
||||
AuditLog.user_id == uid,
|
||||
).limit(1000)
|
||||
)
|
||||
).scalars().all()
|
||||
export_data["data"]["audit_log"] = [
|
||||
{
|
||||
"action": a.action,
|
||||
"entity_type": a.entity_type,
|
||||
"timestamp": a.timestamp.isoformat() if a.timestamp else None,
|
||||
}
|
||||
for a in audit_entries
|
||||
]
|
||||
|
||||
# Notifications addressed to the user
|
||||
notifications = (
|
||||
await db.execute(
|
||||
sa_select(Notification).where(
|
||||
Notification.tenant_id == tid,
|
||||
Notification.owner_id == uid,
|
||||
).limit(1000)
|
||||
)
|
||||
).scalars().all()
|
||||
export_data["data"]["notifications"] = [
|
||||
{
|
||||
"id": str(n.id),
|
||||
"type": getattr(n, "type", None),
|
||||
"title": getattr(n, "title", None),
|
||||
"created_at": n.created_at.isoformat() if n.created_at else None,
|
||||
}
|
||||
for n in notifications
|
||||
]
|
||||
|
||||
# ── Categories promised by the dsgvo-export route docstring ──
|
||||
# (G1-b: mail accounts, tasks, calendar entries, comm messages)
|
||||
try:
|
||||
from app.plugins.builtins.mail.models import MailAccount
|
||||
|
||||
mail_accounts = (
|
||||
await db.execute(
|
||||
sa_select(MailAccount).where(
|
||||
MailAccount.tenant_id == tid,
|
||||
MailAccount.user_id == uid,
|
||||
).limit(500)
|
||||
)
|
||||
).scalars().all()
|
||||
export_data["data"]["mail_accounts"] = [
|
||||
{
|
||||
"id": str(a.id),
|
||||
"email_address": a.email_address,
|
||||
"display_name": a.display_name,
|
||||
"is_shared": a.is_shared,
|
||||
"is_active": a.is_active,
|
||||
}
|
||||
for a in mail_accounts
|
||||
]
|
||||
except ImportError:
|
||||
pass
|
||||
|
||||
try:
|
||||
from app.plugins.builtins.tasks.models import Task as TaskModel
|
||||
|
||||
tasks = (
|
||||
await db.execute(
|
||||
sa_select(TaskModel).where(
|
||||
sa_or_(TaskModel.owner_id == uid, TaskModel.assigned_to == uid),
|
||||
TaskModel.tenant_id == tid,
|
||||
TaskModel.deleted_at.is_(None),
|
||||
).limit(1000)
|
||||
)
|
||||
).scalars().all()
|
||||
export_data["data"]["tasks"] = [
|
||||
{
|
||||
"id": str(t.id),
|
||||
"title": t.title,
|
||||
"status": t.status,
|
||||
"priority": t.priority,
|
||||
"due_date": t.due_date.isoformat() if t.due_date else None,
|
||||
}
|
||||
for t in tasks
|
||||
]
|
||||
except ImportError:
|
||||
pass
|
||||
|
||||
try:
|
||||
from app.plugins.builtins.calendar.models import CalendarEntry as CalEntry
|
||||
|
||||
cal_entries = (
|
||||
await db.execute(
|
||||
sa_select(CalEntry).where(
|
||||
CalEntry.tenant_id == tid,
|
||||
CalEntry.owner_id == uid,
|
||||
CalEntry.deleted_at.is_(None),
|
||||
).limit(1000)
|
||||
)
|
||||
).scalars().all()
|
||||
export_data["data"]["calendar_entries"] = [
|
||||
{
|
||||
"id": str(e.id),
|
||||
"title": e.title,
|
||||
"entry_type": e.entry_type,
|
||||
"start_at": e.start_at.isoformat() if e.start_at else None,
|
||||
"end_at": e.end_at.isoformat() if e.end_at else None,
|
||||
}
|
||||
for e in cal_entries
|
||||
]
|
||||
except ImportError:
|
||||
pass
|
||||
|
||||
try:
|
||||
from app.plugins.builtins.kommunikation.models import CommMessage
|
||||
|
||||
comm_messages = (
|
||||
await db.execute(
|
||||
sa_select(CommMessage).where(
|
||||
CommMessage.sender_id == uid,
|
||||
CommMessage.tenant_id == tid,
|
||||
).limit(1000)
|
||||
)
|
||||
).scalars().all()
|
||||
export_data["data"]["comm_messages"] = [
|
||||
{
|
||||
"id": str(m.id),
|
||||
"sender_type": m.sender_type,
|
||||
"content": m.content[:500],
|
||||
"created_at": m.created_at.isoformat() if m.created_at else None,
|
||||
}
|
||||
for m in comm_messages
|
||||
]
|
||||
except ImportError:
|
||||
pass
|
||||
|
||||
return export_data
|
||||
|
||||
|
||||
async def _dsar_execute_deletion(db: Any, tenant_id: str, user_id: str) -> dict[str, int]:
|
||||
"""Execute GDPR Art. 17 erasure for a user within one tenant.
|
||||
|
||||
Strategy (respects retention duties):
|
||||
- Contacts owned by the user → soft-delete via deleted_at
|
||||
(audit history must remain intact — it is not personal data of the
|
||||
subject but business record; retention policy governs its cleanup)
|
||||
- Notifications owned by the user → hard delete
|
||||
- User account → deactivate (is_active=False), clear personal fields,
|
||||
scramble password hash and email (keeps FK integrity for audit rows)
|
||||
Returns counters for the audit entry.
|
||||
"""
|
||||
from datetime import UTC, datetime
|
||||
from uuid import UUID as PyUUID
|
||||
|
||||
from sqlalchemy import select as sa_select
|
||||
from sqlalchemy import update as sa_update
|
||||
|
||||
from app.core.audit import log_audit
|
||||
from app.models.contact import Contact
|
||||
from app.models.notification import Notification
|
||||
from app.models.user import User
|
||||
|
||||
uid = PyUUID(user_id)
|
||||
tid = PyUUID(tenant_id)
|
||||
|
||||
counts: dict[str, int] = {}
|
||||
|
||||
# 1. Soft-delete contacts owned by the user
|
||||
contact_result = await db.execute(
|
||||
sa_select(Contact).where(
|
||||
Contact.tenant_id == tid,
|
||||
Contact.owner_id == uid,
|
||||
Contact.deleted_at.is_(None),
|
||||
)
|
||||
)
|
||||
contacts = contact_result.scalars().all()
|
||||
for c in contacts:
|
||||
c.deleted_at = datetime.now(UTC)
|
||||
counts["contacts_soft_deleted"] = len(contacts)
|
||||
|
||||
# 2. Hard-delete notifications owned by the user
|
||||
notif_result = await db.execute(
|
||||
sa_select(Notification).where(
|
||||
Notification.tenant_id == tid,
|
||||
Notification.owner_id == uid,
|
||||
)
|
||||
)
|
||||
notifications = notif_result.scalars().all()
|
||||
for n in notifications:
|
||||
await db.delete(n)
|
||||
counts["notifications_deleted"] = len(notifications)
|
||||
|
||||
# 3. Anonymize + deactivate the account (FK integrity for audit rows kept)
|
||||
await db.execute(
|
||||
sa_update(User)
|
||||
.where(User.id == uid)
|
||||
.values(
|
||||
email=f"erased.{uid.hex[:16]}@anonymized.invalid",
|
||||
name="[gelöscht gemäß DSGVO Art. 17]",
|
||||
first_name=None,
|
||||
last_name=None,
|
||||
avatar_url=None,
|
||||
password_hash="!dsar-erased",
|
||||
is_active=False,
|
||||
preferences={},
|
||||
)
|
||||
)
|
||||
counts["user_anonymized"] = 1
|
||||
|
||||
# 4. Audit the erasure itself (who/what/when — required by Art. 17 recital)
|
||||
await log_audit(
|
||||
db,
|
||||
tid,
|
||||
user_id,
|
||||
"dsar_erasure",
|
||||
"user",
|
||||
uid,
|
||||
{"target_user": user_id, **counts},
|
||||
)
|
||||
return counts
|
||||
|
||||
|
||||
async def process_dsar(
|
||||
ctx: dict[str, Any],
|
||||
*,
|
||||
user_id: str,
|
||||
tenant_id: str,
|
||||
request_type: str,
|
||||
) -> dict[str, Any]:
|
||||
"""Process a GDPR Data Subject Access Request (DSAR).
|
||||
|
||||
ARQ worker function registered as "process_dsar".
|
||||
|
||||
request_type:
|
||||
- "access": collect all data categories (Art. 15/20) and post a system
|
||||
message that the export is ready (served via the existing dsgvo-export
|
||||
endpoint).
|
||||
- "deletion": execute Art. 17 erasure (soft-delete contacts, hard-delete
|
||||
notifications, anonymize+deactivate account) and audit it.
|
||||
- "rectification": post a system message asking admins to handle the
|
||||
correction manually.
|
||||
|
||||
Returns a summary dict for the job result.
|
||||
"""
|
||||
import logging
|
||||
import uuid as uuid_module
|
||||
|
||||
from app.core.db import get_worker_session_factory
|
||||
from app.core.notifications import post_system_message
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
tid = uuid_module.UUID(tenant_id)
|
||||
uid = uuid_module.UUID(user_id)
|
||||
|
||||
factory = get_worker_session_factory()
|
||||
async with factory() as db:
|
||||
try:
|
||||
if request_type == "access":
|
||||
data = await _dsar_collect_user_data(db, tenant_id, user_id)
|
||||
await db.commit()
|
||||
categories = list(data.get("data", {}).keys())
|
||||
await post_system_message(
|
||||
db,
|
||||
tid,
|
||||
uid,
|
||||
"dsar_access_ready",
|
||||
"DSGVO-Auskunft bereit",
|
||||
f"Datenkategorien: {', '.join(categories)}",
|
||||
severity="info",
|
||||
)
|
||||
await db.commit()
|
||||
logger.info("DSAR access processed for user %s", user_id)
|
||||
return {"type": request_type, "status": "completed", "categories": categories}
|
||||
|
||||
if request_type == "deletion":
|
||||
counts = await _dsar_execute_deletion(db, tenant_id, user_id)
|
||||
await db.commit()
|
||||
await post_system_message(
|
||||
db,
|
||||
tid,
|
||||
uid,
|
||||
"dsar_deletion_done",
|
||||
"DSGVO-Löschung ausgeführt",
|
||||
f"Kontakten soft-gelöscht: {counts.get('contacts_soft_deleted', 0)}; Konto anonymisiert.",
|
||||
severity="info",
|
||||
)
|
||||
await db.commit()
|
||||
logger.info("DSAR deletion executed for user %s: %s", user_id, counts)
|
||||
return {"type": request_type, "status": "completed", **counts}
|
||||
|
||||
if request_type == "rectification":
|
||||
await post_system_message(
|
||||
db,
|
||||
tid,
|
||||
uid,
|
||||
"dsar_rectification_requested",
|
||||
"DSGVO-Berichtigung angefordert",
|
||||
f"Manuelle Bearbeitung für User {user_id} erforderlich.",
|
||||
severity="warning",
|
||||
)
|
||||
await db.commit()
|
||||
logger.info("DSAR rectification requested for user %s", user_id)
|
||||
return {"type": request_type, "status": "queued_for_manual_handling"}
|
||||
|
||||
logger.warning("Unknown DSAR request_type '%s' for user %s", request_type, user_id)
|
||||
return {"type": request_type, "status": "unknown_type"}
|
||||
except Exception:
|
||||
await db.rollback()
|
||||
raise
|
||||
|
||||
|
||||
register_job("process_dsar", process_dsar)
|
||||
|
||||
@@ -3,6 +3,8 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import re
|
||||
import uuid as uuid_mod
|
||||
|
||||
from fastapi import Request, status
|
||||
from starlette.middleware.base import BaseHTTPMiddleware
|
||||
@@ -137,3 +139,97 @@ class CSRFMiddleware(BaseHTTPMiddleware):
|
||||
pass
|
||||
|
||||
return await call_next(request)
|
||||
|
||||
|
||||
class AuditMiddleware(BaseHTTPMiddleware):
|
||||
"""Safety-net audit trail for ALL successful mutating requests.
|
||||
|
||||
AGENTS.md requires every mutation to produce an audit entry. Explicit
|
||||
``log_audit`` calls in routes/services remain the detail layer (entity ids,
|
||||
change diffs); this middleware guarantees a baseline entry for mutations
|
||||
that lack one, marked with ``source=middleware`` in ``details``.
|
||||
|
||||
Best-effort by design: audit failures never break the request.
|
||||
"""
|
||||
|
||||
_MUTATING = {"POST", "PUT", "PATCH", "DELETE"}
|
||||
_SKIP_PREFIXES = (
|
||||
"/api/v1/auth",
|
||||
"/api/v1/health",
|
||||
"/api/v1/errors",
|
||||
"/api/v1/audit",
|
||||
"/api/v1/external",
|
||||
)
|
||||
|
||||
async def dispatch(self, request: Request, call_next):
|
||||
response = await call_next(request)
|
||||
|
||||
if request.method not in self._MUTATING:
|
||||
return response
|
||||
if response.status_code < 200 or response.status_code >= 300:
|
||||
return response
|
||||
path = request.url.path
|
||||
if any(path.startswith(p) for p in self._SKIP_PREFIXES):
|
||||
return response
|
||||
|
||||
try:
|
||||
await self._write_entry(request, path, response.status_code)
|
||||
except Exception:
|
||||
logging.getLogger(__name__).debug(
|
||||
"AuditMiddleware: failed to write baseline entry for %s %s", request.method, path
|
||||
)
|
||||
return response
|
||||
|
||||
@staticmethod
|
||||
def _derive_entity_type(path: str) -> str:
|
||||
"""Derive an entity_type from the second URL segment."""
|
||||
parts = [p for p in path.split("/") if p]
|
||||
# /api/v1/<resource>/... -> resource; singularize naive trailing 's'
|
||||
resource = parts[2] if len(parts) > 2 and parts[0] == "api" and parts[1] == "v1" else (parts[0] if parts else "unknown")
|
||||
return resource[:-1] if len(resource) > 3 and resource.endswith("s") else resource
|
||||
|
||||
async def _write_entry(self, request: Request, path: str, status_code: int) -> None:
|
||||
from app.core.audit import log_audit
|
||||
from app.core.auth import get_redis, get_session_data
|
||||
from app.core.db import create_db_session
|
||||
|
||||
# Attribute via the Redis session (same source as CSRFMiddleware) —
|
||||
# FastAPI dependencies run after middleware, so request.state is empty here.
|
||||
settings = get_settings()
|
||||
session_id = request.cookies.get(settings.session_cookie_name)
|
||||
if not session_id:
|
||||
return # unauthenticated — nothing to attribute
|
||||
redis = get_redis()
|
||||
session_data = await get_session_data(redis, session_id)
|
||||
if not session_data:
|
||||
return
|
||||
tenant_raw = session_data.get("tenant_id")
|
||||
user_raw = session_data.get("user_id")
|
||||
if not tenant_raw:
|
||||
return
|
||||
|
||||
action_map = {"POST": "create", "PATCH": "update", "PUT": "update", "DELETE": "delete"}
|
||||
entity_id: uuid_mod.UUID | None = None
|
||||
parts = [p for p in path.split("/") if p]
|
||||
if parts and re.fullmatch(r"[0-9a-fA-F-]{36}", parts[-1]):
|
||||
try:
|
||||
entity_id = uuid_mod.UUID(parts[-1])
|
||||
except ValueError:
|
||||
entity_id = None
|
||||
|
||||
async with create_db_session(uuid_mod.UUID(tenant_raw)) as db:
|
||||
await log_audit(
|
||||
db,
|
||||
uuid_mod.UUID(tenant_raw),
|
||||
uuid_mod.UUID(user_raw) if user_raw else None,
|
||||
action_map.get(request.method, request.method.lower()),
|
||||
self._derive_entity_type(path),
|
||||
entity_id,
|
||||
changes={
|
||||
"source": "middleware",
|
||||
"method": request.method,
|
||||
"path": path,
|
||||
"status": status_code,
|
||||
},
|
||||
)
|
||||
await db.commit()
|
||||
|
||||
@@ -54,6 +54,8 @@ CORE_PERMISSIONS: list[dict[str, str]] = [
|
||||
{"key": "taxes:write", "label": "Taxes: Write", "category": "core", "module": "taxes"},
|
||||
{"key": "currencies:read", "label": "Currencies: Read", "category": "core", "module": "currencies"},
|
||||
{"key": "currencies:write", "label": "Currencies: Write", "category": "core", "module": "currencies"},
|
||||
{"key": "custom_fields:read", "label": "Custom Fields: Read", "category": "core", "module": "custom_fields"},
|
||||
{"key": "custom_fields:write", "label": "Custom Fields: Write", "category": "core", "module": "custom_fields"},
|
||||
{"key": "import_export:read", "label": "Import/Export: Read", "category": "core", "module": "import_export"},
|
||||
{"key": "import_export:write", "label": "Import/Export: Write", "category": "core", "module": "import_export"},
|
||||
{"key": "workspaces:read", "label": "Workspaces: Read", "category": "core", "module": "workspaces"},
|
||||
|
||||
@@ -29,6 +29,14 @@ class ServiceContainer:
|
||||
"""Check if a service is registered."""
|
||||
return name in self._services
|
||||
|
||||
def remove(self, name: str) -> None:
|
||||
"""Remove a service registration (no-op if absent).
|
||||
|
||||
Used by plugin deactivation hooks to clean up services they
|
||||
registered during activation.
|
||||
"""
|
||||
self._services.pop(name, None)
|
||||
|
||||
async def initialize(self) -> None:
|
||||
"""Initialize core services."""
|
||||
if self._initialized:
|
||||
|
||||
+40
-12
@@ -501,11 +501,38 @@ async def save_with_metadata(
|
||||
}
|
||||
|
||||
|
||||
async def get_file_metadata_async(path: str) -> dict[str, Any]:
|
||||
"""Awaitable variant of :func:`get_file_metadata` (ARCH-052).
|
||||
|
||||
Safe to call from inside a running event loop — never creates a
|
||||
nested one. For local storage this is plain filesystem access; for
|
||||
S3 and other async backends the backend's ``exists()`` is awaited.
|
||||
"""
|
||||
backend = get_storage_backend()
|
||||
if isinstance(backend, LocalStorage):
|
||||
full_path = backend._full_path(path)
|
||||
if not os.path.exists(full_path):
|
||||
return {"size": None, "modified": None, "exists": False}
|
||||
stat = os.stat(full_path)
|
||||
return {
|
||||
"size": stat.st_size,
|
||||
"modified": stat.st_mtime,
|
||||
"exists": True,
|
||||
}
|
||||
# S3 or other async backends — await the backend directly
|
||||
if not await backend.exists(path):
|
||||
return {"size": None, "modified": None, "exists": False}
|
||||
return {"size": None, "modified": None, "exists": True}
|
||||
|
||||
|
||||
def get_file_metadata(path: str) -> dict[str, Any]:
|
||||
"""Read metadata of a stored file without loading its content.
|
||||
|
||||
Works with the *local* storage backend. For S3, use the S3 client
|
||||
``stat_object`` API directly.
|
||||
Works with the *local* storage backend without touching the event
|
||||
loop. For S3 and other async-only backends this drives the check
|
||||
through ``asyncio.run``; calling it from inside a running event loop
|
||||
raises ``RuntimeError`` — use :func:`get_file_metadata_async` there
|
||||
instead (ARCH-052).
|
||||
|
||||
Parameters
|
||||
----------
|
||||
@@ -530,14 +557,15 @@ def get_file_metadata(path: str) -> dict[str, Any]:
|
||||
"modified": stat.st_mtime,
|
||||
"exists": True,
|
||||
}
|
||||
# S3 or other backends — fall back to exists() check
|
||||
import asyncio as _asyncio
|
||||
|
||||
loop = _asyncio.new_event_loop()
|
||||
# Async-only backend outside a running loop is fine; inside one we
|
||||
# must never build a nested event loop.
|
||||
try:
|
||||
exists = loop.run_until_complete(backend.exists(path))
|
||||
if not exists:
|
||||
return {"size": None, "modified": None, "exists": False}
|
||||
return {"size": None, "modified": None, "exists": True}
|
||||
finally:
|
||||
loop.close()
|
||||
asyncio.get_running_loop()
|
||||
except RuntimeError:
|
||||
pass
|
||||
else:
|
||||
raise RuntimeError(
|
||||
"get_file_metadata() cannot be used with async storage backends "
|
||||
"inside a running event loop — use get_file_metadata_async()"
|
||||
)
|
||||
return asyncio.run(get_file_metadata_async(path))
|
||||
|
||||
@@ -121,11 +121,14 @@ class TriggerDispatcher:
|
||||
"""Query DB for active automations matching *event_name* and dispatch."""
|
||||
from app.core.db import get_session_factory
|
||||
from app.plugins.builtins.contracts import get_contract
|
||||
# None-check FIRST — accessing attributes on the contract before the
|
||||
# check crashed with AttributeError when automation was inactive
|
||||
# (ARCH-029/041).
|
||||
automation_contract = get_contract("automation")
|
||||
AutomationDefinition = automation_contract.Automation # noqa: N806
|
||||
if automation_contract is None:
|
||||
logger.debug("Automation plugin not available — trigger skipped")
|
||||
return
|
||||
AutomationDefinition = automation_contract.Automation # noqa: N806
|
||||
|
||||
factory = get_session_factory()
|
||||
tenant_id = payload.get("tenant_id")
|
||||
|
||||
+11
-12
@@ -170,12 +170,7 @@ async def on_startup(ctx: dict[str, Any]) -> None:
|
||||
if search_contract is not None:
|
||||
factory = async_session
|
||||
async with factory() as db:
|
||||
# auto_register_providers is not exposed via contract yet;
|
||||
# use the contract's get_search_registry to access providers
|
||||
from app.plugins.builtins.unified_search.provider_registry import (
|
||||
auto_register_providers,
|
||||
)
|
||||
await auto_register_providers(db)
|
||||
await search_contract.auto_register_providers(db)
|
||||
logger.info("Search providers registered for worker")
|
||||
else:
|
||||
logger.debug("Unified search plugin not available — skipping provider registration")
|
||||
@@ -350,8 +345,10 @@ async def cleanup_audit_log_job(ctx: dict[str, Any]) -> None:
|
||||
Runs daily to prevent the audit_log table from growing indefinitely.
|
||||
Iterates per-tenant for RLS compliance.
|
||||
"""
|
||||
from sqlalchemy import text as sa_text, delete as sa_delete
|
||||
from datetime import datetime, timedelta
|
||||
from datetime import UTC, datetime, timedelta
|
||||
|
||||
from sqlalchemy import delete as sa_delete
|
||||
from sqlalchemy import text as sa_text
|
||||
|
||||
from app.core.db import get_worker_session_factory
|
||||
from app.models.audit import AuditLog
|
||||
@@ -362,7 +359,7 @@ async def cleanup_audit_log_job(ctx: dict[str, Any]) -> None:
|
||||
tenant_result = await db.execute(sa_text("SELECT id FROM tenants"))
|
||||
tenant_ids = [row[0] for row in tenant_result]
|
||||
|
||||
cutoff = datetime.utcnow() - timedelta(days=365)
|
||||
cutoff = datetime.now(UTC) - timedelta(days=365)
|
||||
total_deleted = 0
|
||||
for tenant_id in tenant_ids:
|
||||
await db.execute(
|
||||
@@ -393,8 +390,10 @@ async def cleanup_trash_job(ctx: dict[str, Any]) -> None:
|
||||
Runs daily to clean up the trash. Iterates per-tenant for RLS compliance.
|
||||
Default retention: 90 days in trash before permanent deletion.
|
||||
"""
|
||||
from sqlalchemy import text as sa_text, delete as sa_delete
|
||||
from datetime import datetime, timedelta
|
||||
from datetime import UTC, datetime, timedelta
|
||||
|
||||
from sqlalchemy import delete as sa_delete
|
||||
from sqlalchemy import text as sa_text
|
||||
|
||||
from app.core.db import get_worker_session_factory
|
||||
from app.models.contact import Contact
|
||||
@@ -406,7 +405,7 @@ async def cleanup_trash_job(ctx: dict[str, Any]) -> None:
|
||||
tenant_result = await db.execute(sa_text("SELECT id FROM tenants"))
|
||||
tenant_ids = [row[0] for row in tenant_result]
|
||||
|
||||
cutoff = datetime.utcnow() - timedelta(days=90)
|
||||
cutoff = datetime.now(UTC) - timedelta(days=90)
|
||||
total_deleted = 0
|
||||
|
||||
for tenant_id in tenant_ids:
|
||||
|
||||
+28
-3
@@ -17,8 +17,9 @@ from app.core.db import get_db, set_tenant_context, set_user_context
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# Known write-permission modules — used by require_write() to check
|
||||
# specific permissions instead of broad wildcards like *:write
|
||||
# Legacy fallback list — used by require_write() only when the permission
|
||||
# registry is not initialized. The live source of truth is generated from
|
||||
# the registry (see _get_write_permissions, ARCH-022).
|
||||
_WRITE_PERMISSIONS = [
|
||||
"users:write",
|
||||
"roles:write",
|
||||
@@ -35,6 +36,30 @@ _WRITE_PERMISSIONS = [
|
||||
]
|
||||
|
||||
|
||||
def _get_write_permissions() -> list[str]:
|
||||
"""Return all known ``module:write`` permission keys (ARCH-022).
|
||||
|
||||
Generated from the permission registry so plugin write permissions are
|
||||
picked up automatically without touching this file. Falls back to the
|
||||
static legacy list when the registry is unavailable/uninitialized.
|
||||
"""
|
||||
try:
|
||||
from app.core.permission_registry import get_permission_registry
|
||||
|
||||
registry = get_permission_registry()
|
||||
if getattr(registry, "_initialized", False):
|
||||
perms = [
|
||||
entry["key"]
|
||||
for entry in registry.get_all()
|
||||
if entry["key"].endswith(":write")
|
||||
]
|
||||
if perms:
|
||||
return sorted(perms)
|
||||
except Exception:
|
||||
pass
|
||||
return list(_WRITE_PERMISSIONS)
|
||||
|
||||
|
||||
async def get_redis_dep() -> aioredis.Redis:
|
||||
"""FastAPI dependency for Redis client."""
|
||||
return get_redis()
|
||||
@@ -261,7 +286,7 @@ async def require_write(
|
||||
# Check via permission system for specific write permissions
|
||||
from app.core.permissions import check_permission
|
||||
|
||||
for perm in _WRITE_PERMISSIONS:
|
||||
for perm in _get_write_permissions():
|
||||
if check_permission(current_user, perm):
|
||||
return current_user
|
||||
|
||||
|
||||
+36
-20
@@ -22,8 +22,12 @@ logger = logging.getLogger(__name__)
|
||||
|
||||
from app.config import get_settings # noqa: E402
|
||||
from app.core.db import close_engine, get_engine # noqa: E402
|
||||
from app.core.error_codes import ApiError, build_error_response # noqa: E402
|
||||
from app.core.middleware import CSRFMiddleware, SecurityHeadersMiddleware # noqa: E402
|
||||
from app.core.error_codes import ERROR_CODES, ApiError, build_error_response # noqa: E402
|
||||
from app.core.middleware import ( # noqa: E402
|
||||
AuditMiddleware,
|
||||
CSRFMiddleware,
|
||||
SecurityHeadersMiddleware,
|
||||
)
|
||||
from app.core.monitoring import record_error, record_request # noqa: E402
|
||||
from app.core.rate_limit import GeneralRateLimitMiddleware # noqa: E402
|
||||
from app.core.resilience import CircuitBreakerMiddleware # noqa: E402
|
||||
@@ -36,16 +40,14 @@ from app.routes import ( # noqa: E402
|
||||
attachments,
|
||||
audit,
|
||||
auth,
|
||||
compliance,
|
||||
backups,
|
||||
bank_accounts,
|
||||
contact_folder_permissions,
|
||||
contact_folders,
|
||||
contacts,
|
||||
compliance,
|
||||
currencies,
|
||||
custom_field_definitions,
|
||||
custom_fields,
|
||||
dashboard,
|
||||
delegations,
|
||||
entity_history,
|
||||
entity_permissions,
|
||||
errors,
|
||||
@@ -59,7 +61,6 @@ from app.routes import ( # noqa: E402
|
||||
owner_transfer,
|
||||
permission_templates,
|
||||
plugins,
|
||||
delegations,
|
||||
policies,
|
||||
roles,
|
||||
saved_filters,
|
||||
@@ -284,22 +285,23 @@ async def lifespan(app: FastAPI):
|
||||
logger.info(f"Plugin {name} is inactive — skipping activation")
|
||||
continue
|
||||
|
||||
# Activate plugin with a FRESH session per plugin to avoid RLS state leakage
|
||||
# RLS fail-closed requires app.current_tenant_id for tenant-table writes.
|
||||
# Plugin activation may fail on duplicate cron job inserts — this is harmless
|
||||
# since cron jobs already exist from previous startups.
|
||||
# Activate plugin ONCE per process (ARCH-002 fix): a fresh session with
|
||||
# the first tenant's RLS context satisfies fail-closed RLS for any
|
||||
# tenant-table writes during activation. Plugins that need per-tenant
|
||||
# data must seed it themselves (e.g. via the default-tenant mechanism).
|
||||
# Calling on_activate once prevents duplicate event listeners, cron
|
||||
# jobs, mini-apps and other contributions at multi-tenant startups.
|
||||
plugin_activated = False
|
||||
for tenant_id in all_tenant_ids:
|
||||
if all_tenant_ids:
|
||||
try:
|
||||
async with async_session() as plugin_db:
|
||||
await set_tenant_context(plugin_db, tenant_id)
|
||||
await set_tenant_context(plugin_db, all_tenant_ids[0])
|
||||
await plugin.on_activate(plugin_db, container, event_bus)
|
||||
await plugin_db.flush()
|
||||
await plugin_db.commit()
|
||||
plugin_activated = True
|
||||
except Exception as exc:
|
||||
logger.warning(f"[STARTUP] Plugin {name} activation issue for tenant {tenant_id}: {exc}")
|
||||
break
|
||||
logger.warning(f"[STARTUP] Plugin {name} activation issue: {exc}")
|
||||
|
||||
if plugin_activated:
|
||||
plugin_record.status = "active"
|
||||
@@ -474,6 +476,7 @@ def create_app() -> FastAPI:
|
||||
)
|
||||
app.add_middleware(CSRFMiddleware)
|
||||
app.add_middleware(SecurityHeadersMiddleware)
|
||||
app.add_middleware(AuditMiddleware)
|
||||
app.add_middleware(GeneralRateLimitMiddleware)
|
||||
app.add_middleware(RequestLoggingMiddleware)
|
||||
app.add_middleware(CircuitBreakerMiddleware)
|
||||
@@ -516,6 +519,21 @@ def create_app() -> FastAPI:
|
||||
504: "service_timeout",
|
||||
}
|
||||
code = status_to_code.get(exc.status_code, "internal_error" if exc.status_code >= 500 else "validation_error")
|
||||
# Structured detail passthrough (AGENTS.md): when a route raises
|
||||
# HTTPException with a dict detail containing a machine-readable ``code``,
|
||||
# preserve the structured shape instead of stringifying it.
|
||||
raw_detail = exc.detail
|
||||
if isinstance(raw_detail, dict):
|
||||
inner_code = raw_detail.get("code", code)
|
||||
body = build_error_response(
|
||||
code=inner_code if inner_code in ERROR_CODES else code,
|
||||
detail=raw_detail.get("detail") or str(raw_detail),
|
||||
trace_id=trace_id,
|
||||
)
|
||||
# Preserve the full structured detail as a nested object so clients
|
||||
# can read ``resp.json()["detail"]["code"]``.
|
||||
body["detail"] = raw_detail
|
||||
else:
|
||||
body = build_error_response(
|
||||
code=code,
|
||||
detail=str(exc.detail) if exc.detail else None,
|
||||
@@ -544,11 +562,9 @@ def create_app() -> FastAPI:
|
||||
app.include_router(groups.router)
|
||||
app.include_router(tenants.router)
|
||||
app.include_router(notifications.router)
|
||||
from app.routes.companies import router as companies_router
|
||||
app.include_router(companies_router)
|
||||
app.include_router(contacts.router)
|
||||
app.include_router(contact_folders.router)
|
||||
app.include_router(contact_folder_permissions.router)
|
||||
# NOTE: contacts/companies/contact-folders routes are plugin-owned now
|
||||
# (Block B1) and mounted via the manifest.routes mechanism below with
|
||||
# require_active_plugin("contacts") protection.
|
||||
app.include_router(entity_permissions.router)
|
||||
app.include_router(dashboard.router)
|
||||
app.include_router(entity_history.router)
|
||||
|
||||
@@ -6,8 +6,8 @@ from app.models.audit import AuditLog
|
||||
from app.models.auth import ApiToken, PasswordResetToken
|
||||
from app.models.backup import Backup
|
||||
from app.models.bank_account import BankAccount
|
||||
from app.models.consumer_inbox import ConsumerInbox
|
||||
from app.models.compliance import ComplianceIncident
|
||||
from app.models.consumer_inbox import ConsumerInbox
|
||||
from app.models.contact import Contact, ContactPerson
|
||||
from app.models.contact_folder import ContactFolder
|
||||
from app.models.contact_merge import ContactMergeHistory
|
||||
|
||||
+1
-2
@@ -11,13 +11,12 @@ from datetime import datetime
|
||||
from typing import Any
|
||||
|
||||
from sqlalchemy import DateTime, ForeignKey, String, func
|
||||
from sqlalchemy.dialects.postgresql import JSONB
|
||||
from sqlalchemy.dialects.postgresql import JSONB, TSVECTOR
|
||||
from sqlalchemy.dialects.postgresql import UUID as PGUUID
|
||||
from sqlalchemy.orm import Mapped, mapped_column
|
||||
|
||||
from app.core.db import Base, TenantMixin
|
||||
from app.models.owned_mixin import OwnedMixin
|
||||
from sqlalchemy.dialects.postgresql import TSVECTOR
|
||||
|
||||
# Re-export EntityHistory as DeletionLog for backward compatibility.
|
||||
# Tests import DeletionLog from app.models.audit and use entity_snapshot attribute.
|
||||
|
||||
+47
-1
@@ -5,7 +5,7 @@ from __future__ import annotations
|
||||
import uuid
|
||||
from datetime import datetime
|
||||
|
||||
from sqlalchemy import DateTime, Integer, String, Text, func
|
||||
from sqlalchemy import DateTime, ForeignKey, Integer, String, Text, UniqueConstraint, func
|
||||
from sqlalchemy.dialects.postgresql import JSONB
|
||||
from sqlalchemy.dialects.postgresql import UUID as PGUUID
|
||||
from sqlalchemy.orm import Mapped, mapped_column
|
||||
@@ -77,3 +77,49 @@ class EventOutbox(Base):
|
||||
failed_at: Mapped[datetime | None] = mapped_column(
|
||||
DateTime(timezone=True), nullable=True,
|
||||
)
|
||||
|
||||
|
||||
class OutboxDelivery(Base):
|
||||
"""Per-consumer delivery status for an event_outbox row (Migration 0075).
|
||||
|
||||
Tracks whether each consumer successfully processed an event; an event is
|
||||
only 'published' when all mandatory deliveries succeed.
|
||||
"""
|
||||
|
||||
__tablename__ = "outbox_deliveries"
|
||||
__table_args__ = (
|
||||
UniqueConstraint(
|
||||
"event_id", "consumer_name",
|
||||
name="uq_outbox_deliveries_event_consumer",
|
||||
),
|
||||
)
|
||||
|
||||
id: Mapped[uuid.UUID] = mapped_column(
|
||||
PGUUID(as_uuid=True), primary_key=True,
|
||||
server_default=func.gen_random_uuid(),
|
||||
)
|
||||
event_id: Mapped[uuid.UUID] = mapped_column(
|
||||
PGUUID(as_uuid=True),
|
||||
ForeignKey("event_outbox.id", ondelete="CASCADE"),
|
||||
nullable=False,
|
||||
)
|
||||
consumer_name: Mapped[str] = mapped_column(String(150), nullable=False)
|
||||
status: Mapped[str] = mapped_column(
|
||||
String(30), nullable=False, server_default="pending",
|
||||
)
|
||||
attempt_count: Mapped[int] = mapped_column(
|
||||
Integer, nullable=False, server_default="0",
|
||||
)
|
||||
next_attempt_at: Mapped[datetime | None] = mapped_column(
|
||||
DateTime(timezone=True), nullable=True,
|
||||
)
|
||||
last_error: Mapped[str | None] = mapped_column(Text, nullable=True)
|
||||
processed_at: Mapped[datetime | None] = mapped_column(
|
||||
DateTime(timezone=True), nullable=True,
|
||||
)
|
||||
created_at: Mapped[datetime] = mapped_column(
|
||||
DateTime(timezone=True), nullable=False, server_default=func.now(),
|
||||
)
|
||||
updated_at: Mapped[datetime] = mapped_column(
|
||||
DateTime(timezone=True), nullable=False, server_default=func.now(),
|
||||
)
|
||||
|
||||
@@ -53,6 +53,9 @@ class SystemSettings(Base, TenantMixin, OwnedMixin):
|
||||
theme_border_radius: Mapped[str] = mapped_column(String(20), nullable=False, default="0.5rem")
|
||||
# Backup configuration
|
||||
backup_enabled: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False, server_default="false")
|
||||
backup_interval: Mapped[str] = mapped_column(String(20), nullable=False, default="daily", server_default="daily")
|
||||
backup_retention_days: Mapped[int] = mapped_column(Integer, nullable=False, default=7, server_default="7")
|
||||
backup_destination: Mapped[str] = mapped_column(String(20), nullable=False, default="local", server_default="local")
|
||||
# Automation plugin settings (JSONB)
|
||||
automation_config: Mapped[dict | None] = mapped_column(JSONB, nullable=True)
|
||||
# Retention policy overrides (JSONB) — compliance module
|
||||
|
||||
@@ -12,7 +12,6 @@ from sqlalchemy.dialects.postgresql import UUID as PGUUID
|
||||
from sqlalchemy.orm import Mapped, mapped_column
|
||||
|
||||
from app.core.db import Base, SoftDeleteMixin, TimestampMixin
|
||||
from app.models.owned_mixin import OwnedMixin
|
||||
|
||||
|
||||
class User(Base, TimestampMixin, SoftDeleteMixin):
|
||||
|
||||
@@ -97,6 +97,15 @@ class BasePlugin(ABC):
|
||||
"""
|
||||
return []
|
||||
|
||||
async def register_event_handlers(self, event_bus: EventBus) -> None:
|
||||
"""Register event handlers for the background worker (ARCH-038 hook).
|
||||
|
||||
The worker calls this on every active plugin at startup so plugins
|
||||
can subscribe to events even when the web process is separate.
|
||||
Default: no-op. Override to subscribe handlers.
|
||||
"""
|
||||
return None
|
||||
|
||||
# ─── Job Modules ───
|
||||
|
||||
def get_job_modules(self) -> list[str]:
|
||||
|
||||
@@ -12,7 +12,6 @@ from sqlalchemy.orm import Mapped, mapped_column
|
||||
|
||||
from app.core.db import Base, TenantMixin
|
||||
from app.models.owned_mixin import OwnedMixin
|
||||
from pgvector.sqlalchemy import Vector
|
||||
|
||||
|
||||
class AgentMemory(Base, TenantMixin, OwnedMixin):
|
||||
|
||||
@@ -22,6 +22,7 @@ from app.plugins.builtins.ai_assistant.schemas import (
|
||||
ExternalAgentRequest,
|
||||
ExternalAgentResponse,
|
||||
)
|
||||
from app.plugins.builtins.ai_assistant.services import stream_chat_comm as stream_chat
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
@@ -120,13 +121,18 @@ async def run_agent_external(
|
||||
}
|
||||
|
||||
# Run the agent via streaming chat (non-streaming mode)
|
||||
from app.plugins.builtins.ai_assistant.services import stream_chat_comm
|
||||
|
||||
full_response = ""
|
||||
async with get_db() as stream_db:
|
||||
await set_tenant_context(stream_db, tenant_id)
|
||||
async for chunk in stream_chat(
|
||||
stream_db, session, agent, data.message, user_context, tenant_id
|
||||
stream_db,
|
||||
session.id,
|
||||
agent,
|
||||
data.message,
|
||||
user_context,
|
||||
tenant_id,
|
||||
uuid.UUID(current_user["user_id"]),
|
||||
):
|
||||
if chunk.startswith("data: ") and chunk != "data: [DONE]\n\n":
|
||||
try:
|
||||
|
||||
@@ -0,0 +1,10 @@
|
||||
-- Dual-path convergence (Gate B): add compliance columns that Alembic
|
||||
-- migration 0119 adds on the core path. Idempotent so both install paths
|
||||
-- converge to the identical schema.
|
||||
ALTER TABLE ai_providers ADD COLUMN IF NOT EXISTS region VARCHAR(20) NOT NULL DEFAULT 'unknown';
|
||||
ALTER TABLE ai_providers ADD COLUMN IF NOT EXISTS hosting_type VARCHAR(30) NOT NULL DEFAULT 'cloud';
|
||||
ALTER TABLE ai_providers ADD COLUMN IF NOT EXISTS dpa_status VARCHAR(20) NOT NULL DEFAULT 'none';
|
||||
ALTER TABLE ai_providers ADD COLUMN IF NOT EXISTS retention_policy TEXT NOT NULL DEFAULT '';
|
||||
ALTER TABLE ai_providers ADD COLUMN IF NOT EXISTS training_on_customer_data BOOLEAN NOT NULL DEFAULT FALSE;
|
||||
ALTER TABLE ai_providers ADD COLUMN IF NOT EXISTS transfer_notice TEXT NOT NULL DEFAULT '';
|
||||
ALTER TABLE ai_providers ADD COLUMN IF NOT EXISTS allowed_data_classes JSONB NOT NULL DEFAULT '[]'::jsonb;
|
||||
@@ -42,7 +42,7 @@ class AIAssistantPlugin(BasePlugin):
|
||||
),
|
||||
],
|
||||
events=[],
|
||||
migrations=["0001_initial.sql", "0002_folders_attachments.sql"],
|
||||
migrations=["0001_initial.sql", "0002_folders_attachments.sql", "0003_sort_order.sql", "0004_compliance_fields.sql"],
|
||||
permissions=[
|
||||
"ai:read",
|
||||
"ai:write",
|
||||
@@ -52,10 +52,10 @@ class AIAssistantPlugin(BasePlugin):
|
||||
],
|
||||
is_core=True,
|
||||
menu_items=[
|
||||
FrontendMenuItem(label_key='nav.aiAssistant', label='KI Assistent', path='/ai-assistant', icon='Bot', order=90),
|
||||
FrontendMenuItem(label_key='nav.aiAssistant', label='KI Assistent', path='/ai-assistant', icon='Bot', order=90, permission='ai:read'),
|
||||
],
|
||||
page_routes=[
|
||||
FrontendPageRoute(path='/ai-assistant', component='@/pages/AIAssistant', protected=True),
|
||||
FrontendPageRoute(path='/ai-assistant', component='@/pages/AIAssistant', protected=True, permission='ai:read'),
|
||||
],
|
||||
settings_pages=[
|
||||
FrontendSettingsPage(path='ai', label_key='settings.ai', label='AI Settings', component='@/pages/AISettings', icon='Bot', order=60),
|
||||
|
||||
@@ -165,8 +165,8 @@ async def get_open_tasks_handler(arguments: dict[str, Any], context: dict[str, A
|
||||
|
||||
from app.plugins.builtins.calendar.contracts import get_contract as get_calendar_contract
|
||||
_cal = get_calendar_contract()
|
||||
calendar_entry = _cal.calendar_entry
|
||||
calendar_entry_link = _cal.calendar_entry_link
|
||||
calendar_entry = _cal.CalendarEntry
|
||||
calendar_entry_link = _cal.CalendarEntryLink
|
||||
|
||||
db, tenant_id, _ = await _get_db_and_tenant(context)
|
||||
entity_type = arguments["entity_type"]
|
||||
|
||||
@@ -69,10 +69,12 @@ async def push_suggestion(user_id: str, suggestion: dict[str, Any]) -> None:
|
||||
# Post suggestion to Communication (I-WORK-PROACTIVE)
|
||||
try:
|
||||
import uuid as uuid_mod
|
||||
|
||||
from sqlalchemy import select as sa_select
|
||||
|
||||
from app.core.db import get_worker_session_factory
|
||||
from app.plugins.builtins.contracts import get_contract_registry
|
||||
from app.plugins.builtins.kommunikation.models import CommConversation
|
||||
from sqlalchemy import select as sa_select
|
||||
from app.core.db import get_worker_session_factory
|
||||
komm = get_contract_registry().get("kommunikation")
|
||||
if komm:
|
||||
factory = get_worker_session_factory()
|
||||
@@ -274,8 +276,8 @@ async def gather_context(
|
||||
# Upcoming calendar events
|
||||
from app.plugins.builtins.calendar.contracts import get_contract as get_calendar_contract
|
||||
_cal = get_calendar_contract()
|
||||
calendar_entry = _cal.calendar_entry
|
||||
calendar_entry_link = _cal.calendar_entry_link
|
||||
calendar_entry = _cal.CalendarEntry
|
||||
calendar_entry_link = _cal.CalendarEntryLink
|
||||
|
||||
now = datetime.now(UTC)
|
||||
event_result = await db.execute(
|
||||
@@ -389,8 +391,8 @@ async def gather_context(
|
||||
# Upcoming events
|
||||
from app.plugins.builtins.calendar.contracts import get_contract as get_calendar_contract
|
||||
_cal = get_calendar_contract()
|
||||
calendar_entry = _cal.calendar_entry
|
||||
calendar_entry_link = _cal.calendar_entry_link
|
||||
calendar_entry = _cal.CalendarEntry
|
||||
calendar_entry_link = _cal.CalendarEntryLink
|
||||
|
||||
now = datetime.now(UTC)
|
||||
event_result = await db.execute(
|
||||
|
||||
@@ -62,7 +62,10 @@ class AIUIControlPlugin(BasePlugin):
|
||||
from app.plugins.builtins.contracts import get_contract_registry
|
||||
get_contract_registry().unregister(self.manifest.name)
|
||||
|
||||
await super().on_deactivate(db, service_container, event_bus)
|
||||
# Remove the WebSocket manager BEFORE super() so that event handlers
|
||||
# being unsubscribed can no longer reach it (ARCH-044).
|
||||
if service_container.has("ai_ui_control_ws"):
|
||||
service_container.remove("ai_ui_control_ws")
|
||||
logger.info("AI UI Control WebSocket manager removed")
|
||||
|
||||
await super().on_deactivate(db, service_container, event_bus)
|
||||
|
||||
@@ -55,7 +55,8 @@ async def send_agent_message(
|
||||
|
||||
# 2. Create a kommunikation message in a dedicated agent room
|
||||
try:
|
||||
from app.plugins.builtins.kommunikation.contracts import CommConversation as Room, CommMessage as Message
|
||||
from app.plugins.builtins.kommunikation.contracts import CommConversation as Room
|
||||
from app.plugins.builtins.kommunikation.contracts import CommMessage as Message
|
||||
|
||||
# Find or create the agent-to-agent room
|
||||
room_name = f"agent:{from_agent_id}:{target_agent.id}"
|
||||
|
||||
@@ -619,6 +619,7 @@ async def stream_agent_run(
|
||||
in real-time as the agent processes.
|
||||
"""
|
||||
from fastapi.responses import StreamingResponse
|
||||
|
||||
from app.ai.agent_stream import stream_react_loop
|
||||
from app.plugins.builtins.ai_assistant.contracts import get_tool_registry
|
||||
|
||||
|
||||
@@ -11,6 +11,7 @@ Safety features:
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import uuid
|
||||
from datetime import UTC, datetime
|
||||
from typing import Any
|
||||
|
||||
@@ -37,12 +38,12 @@ async def run_agent(
|
||||
3. Infinite loop: same tool 5x consecutively (handled in ReAct loop)
|
||||
4. Budget limit: cumulative cost_usd
|
||||
"""
|
||||
from app.plugins.builtins.ai_assistant.contracts import get_tool_registry
|
||||
from app.plugins.builtins.automation.models import (
|
||||
AgentDefinition,
|
||||
AgentRun,
|
||||
AgentRunStep,
|
||||
)
|
||||
from app.plugins.builtins.ai_assistant.contracts import get_tool_registry
|
||||
|
||||
factory = get_session_factory()
|
||||
|
||||
@@ -110,9 +111,9 @@ async def run_agent(
|
||||
try:
|
||||
from app.plugins.builtins.contracts import get_contract
|
||||
mail_contract = get_contract("mail")
|
||||
if mail_contract and hasattr(mail_contract, "get_recent_mails"):
|
||||
if mail_contract and hasattr(mail_contract, "Mail"):
|
||||
from sqlalchemy import select as _select
|
||||
from app.plugins.builtins.mail.models import Mail
|
||||
Mail = mail_contract.Mail
|
||||
async with factory() as db:
|
||||
mail_q = await db.execute(
|
||||
_select(Mail)
|
||||
@@ -167,7 +168,7 @@ async def run_agent(
|
||||
perm_ctx = await resolve_agent_permissions(
|
||||
db=db,
|
||||
tenant_id=agent.tenant_id,
|
||||
user_id=agent.created_by or uuid_mod.uuid4(),
|
||||
user_id=agent.created_by or uuid.uuid4(),
|
||||
agent_definition=agent,
|
||||
)
|
||||
|
||||
@@ -379,23 +380,16 @@ async def run_agent(
|
||||
komm = get_contract_registry().get("kommunikation")
|
||||
if komm:
|
||||
async with factory() as db:
|
||||
# Find or create agent conversation room
|
||||
from app.plugins.builtins.contracts import get_contract as _get_contract
|
||||
_komm_contract = _get_contract("kommunikation")
|
||||
from app.plugins.builtins.kommunikation.models import CommConversation
|
||||
from sqlalchemy import select as sa_select
|
||||
# Find or create agent conversation room via contract
|
||||
# (find_locked_room_id matches create_plugin_room semantics)
|
||||
room_title = f"Agent: {agent.name}"
|
||||
existing = await db.execute(
|
||||
sa_select(CommConversation).where(
|
||||
CommConversation.tenant_id == agent.tenant_id,
|
||||
CommConversation.title == room_title,
|
||||
CommConversation.is_locked.is_(True),
|
||||
CommConversation.locked_by == "automation",
|
||||
CommConversation.deleted_at.is_(None),
|
||||
conv_id = await komm.find_locked_room_id(
|
||||
db=db,
|
||||
tenant_id=agent.tenant_id,
|
||||
plugin_name="automation",
|
||||
title=room_title,
|
||||
)
|
||||
)
|
||||
conv = existing.scalar_one_or_none()
|
||||
if not conv:
|
||||
if not conv_id:
|
||||
room = await komm.create_plugin_room(
|
||||
db=db,
|
||||
tenant_id=agent.tenant_id,
|
||||
@@ -405,8 +399,6 @@ async def run_agent(
|
||||
participant_type="agent",
|
||||
)
|
||||
conv_id = uuid.UUID(room["conversation_id"])
|
||||
else:
|
||||
conv_id = conv.id
|
||||
|
||||
# Post result as message with action_card block
|
||||
status = result_data.get("status", "unknown")
|
||||
|
||||
@@ -0,0 +1,31 @@
|
||||
-- Dual-path convergence (Gate B): create the ReAct step-tracking table
|
||||
-- that Alembic migration 0121 creates on the core path, add the Phase-F
|
||||
-- columns from 0122, and apply the RLS policy from 0129/0136. Idempotent
|
||||
-- so both install paths converge to the identical schema.
|
||||
CREATE TABLE IF NOT EXISTS automation_agent_run_steps (
|
||||
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
tenant_id UUID NOT NULL,
|
||||
agent_run_id UUID NOT NULL REFERENCES automation_agent_runs(id) ON DELETE CASCADE,
|
||||
step_number INTEGER NOT NULL,
|
||||
thought TEXT,
|
||||
action VARCHAR(255),
|
||||
action_input JSONB,
|
||||
observation TEXT,
|
||||
cost_usd FLOAT NOT NULL DEFAULT 0.0,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS ix_agent_run_steps_run ON automation_agent_run_steps(tenant_id, agent_run_id);
|
||||
|
||||
ALTER TABLE automation_agent_definitions ADD COLUMN IF NOT EXISTS temperature FLOAT NOT NULL DEFAULT 0.3;
|
||||
ALTER TABLE automation_agent_definitions ADD COLUMN IF NOT EXISTS max_tokens INTEGER NOT NULL DEFAULT 1000;
|
||||
ALTER TABLE automation_agent_definitions ADD COLUMN IF NOT EXISTS max_steps INTEGER NOT NULL DEFAULT 20;
|
||||
ALTER TABLE automation_agent_definitions ADD COLUMN IF NOT EXISTS trace_mode VARCHAR(20) NOT NULL DEFAULT 'standard';
|
||||
ALTER TABLE automation_agent_definitions ADD COLUMN IF NOT EXISTS skill_ids JSONB NOT NULL DEFAULT '[]'::jsonb;
|
||||
ALTER TABLE automation_agent_definitions ADD COLUMN IF NOT EXISTS trigger_config JSONB NOT NULL DEFAULT '{}'::jsonb;
|
||||
ALTER TABLE automation_agent_definitions ADD COLUMN IF NOT EXISTS ai_use_case_metadata JSONB NOT NULL DEFAULT '{}'::jsonb;
|
||||
|
||||
-- RLS matching migrations 0129 + 0136 (current_tenant_id variant)
|
||||
ALTER TABLE automation_agent_run_steps ENABLE ROW LEVEL SECURITY;
|
||||
DROP POLICY IF EXISTS tenant_isolation ON automation_agent_run_steps;
|
||||
CREATE POLICY tenant_isolation ON automation_agent_run_steps
|
||||
USING (tenant_id::text = current_setting('app.current_tenant_id', true));
|
||||
@@ -38,7 +38,7 @@ class AutomationPlugin(BasePlugin):
|
||||
"Define AI agents with LLM models and tools, create event/schedule/manual "
|
||||
"automations with conditions and actions, schedule cron jobs, and track execution logs."
|
||||
),
|
||||
dependencies=[],
|
||||
dependencies=["mail"],
|
||||
routes=[
|
||||
PluginRouteDef(
|
||||
path="/api/v1/automation",
|
||||
@@ -62,7 +62,7 @@ class AutomationPlugin(BasePlugin):
|
||||
"mail.received",
|
||||
"workflow.timeout",
|
||||
],
|
||||
migrations=["0001_initial.sql", "0002_agent_subtasks.sql", "0003_skill_definitions.sql"],
|
||||
migrations=["0001_initial.sql", "0002_agent_subtasks.sql", "0003_skill_definitions.sql", "0004_run_steps_phase_f.sql"],
|
||||
permissions=[
|
||||
"automation:read",
|
||||
"automation:write",
|
||||
@@ -82,6 +82,7 @@ class AutomationPlugin(BasePlugin):
|
||||
path="/workflows",
|
||||
icon="Workflow",
|
||||
order=52,
|
||||
permission="automation:read",
|
||||
),
|
||||
FrontendMenuItem(
|
||||
label_key="nav.importExport",
|
||||
@@ -89,6 +90,7 @@ class AutomationPlugin(BasePlugin):
|
||||
path="/import-export",
|
||||
icon="ArrowUpDown",
|
||||
order=53,
|
||||
permission="import_export:read",
|
||||
),
|
||||
FrontendMenuItem(
|
||||
label_key="nav.dedupMerge",
|
||||
@@ -96,6 +98,7 @@ class AutomationPlugin(BasePlugin):
|
||||
path="/contacts/dedup",
|
||||
icon="Copy",
|
||||
order=54,
|
||||
permission="contacts:read",
|
||||
),
|
||||
FrontendMenuItem(
|
||||
label_key="nav.tags",
|
||||
@@ -103,6 +106,7 @@ class AutomationPlugin(BasePlugin):
|
||||
path="/tags",
|
||||
icon="Tag",
|
||||
order=55,
|
||||
permission="tags:read",
|
||||
),
|
||||
FrontendMenuItem(
|
||||
label_key="nav.activity",
|
||||
@@ -110,6 +114,7 @@ class AutomationPlugin(BasePlugin):
|
||||
path="/activity",
|
||||
icon="Activity",
|
||||
order=56,
|
||||
permission="contacts:read",
|
||||
),
|
||||
],
|
||||
page_routes=[
|
||||
@@ -117,21 +122,25 @@ class AutomationPlugin(BasePlugin):
|
||||
path="/automation",
|
||||
component="@/pages/AutomationDashboard",
|
||||
order=50,
|
||||
permission="automation:read",
|
||||
),
|
||||
FrontendPageRoute(
|
||||
path="/agents",
|
||||
component="@/pages/AgentDashboard",
|
||||
order=51,
|
||||
permission="agents:read",
|
||||
),
|
||||
FrontendPageRoute(
|
||||
path="/workflows",
|
||||
component="@/pages/Workflows",
|
||||
order=52,
|
||||
permission="automation:read",
|
||||
),
|
||||
FrontendPageRoute(
|
||||
path="/import-export",
|
||||
component="@/pages/ImportExport",
|
||||
order=53,
|
||||
permission="import_export:read",
|
||||
),
|
||||
],
|
||||
settings_pages=[
|
||||
@@ -239,18 +248,25 @@ class AutomationPlugin(BasePlugin):
|
||||
logger.exception("Failed to register own cron jobs")
|
||||
# Register pre-built agents in DB (if not already present)
|
||||
try:
|
||||
from app.plugins.builtins.automation.models import AgentDefinition
|
||||
from app.plugins.builtins.automation.prebuilt.email_triage_agent import create_email_triage_agent
|
||||
from app.plugins.builtins.automation.prebuilt.contact_enrichment_agent import create_contact_enrichment_agent
|
||||
from app.plugins.builtins.automation.prebuilt.follow_up_agent import create_follow_up_agent
|
||||
from app.plugins.builtins.automation.prebuilt.report_agent import create_report_agent
|
||||
from sqlalchemy import select as sa_select
|
||||
|
||||
# Get first tenant + admin user for seeding
|
||||
from app.models.user import User
|
||||
from app.models.tenant import Tenant
|
||||
tenant_result = await db.execute(sa_select(Tenant).limit(1))
|
||||
tenant = tenant_result.scalar_one_or_none()
|
||||
# Get system tenant + admin user for seeding (ARCH-043:
|
||||
# deterministic slug lookup instead of arbitrary first row)
|
||||
from app.core.db import get_system_tenant
|
||||
from app.models.user import User, UserTenant
|
||||
from app.plugins.builtins.automation.models import AgentDefinition
|
||||
from app.plugins.builtins.automation.prebuilt.contact_enrichment_agent import (
|
||||
create_contact_enrichment_agent,
|
||||
)
|
||||
from app.plugins.builtins.automation.prebuilt.email_triage_agent import (
|
||||
create_email_triage_agent,
|
||||
)
|
||||
from app.plugins.builtins.automation.prebuilt.follow_up_agent import (
|
||||
create_follow_up_agent,
|
||||
)
|
||||
from app.plugins.builtins.automation.prebuilt.report_agent import create_report_agent
|
||||
|
||||
tenant = await get_system_tenant(db)
|
||||
if tenant:
|
||||
user_result = await db.execute(
|
||||
sa_select(User)
|
||||
@@ -288,15 +304,14 @@ class AutomationPlugin(BasePlugin):
|
||||
def _register_workflow_agent_tools(self) -> None:
|
||||
"""Register I-AW agent tools for starting and inspecting workflows."""
|
||||
import uuid
|
||||
from typing import Any
|
||||
|
||||
from app.ai.tool_registry import get_tool_registry
|
||||
registry = get_tool_registry()
|
||||
|
||||
async def _start_workflow_handler(arguments: dict[str, Any], context: dict[str, Any]) -> dict[str, Any]:
|
||||
"""Start a workflow by ID."""
|
||||
from app.services.workflow_service import create_instance
|
||||
from app.core.db import get_worker_session_factory
|
||||
from app.services.workflow_service import create_instance
|
||||
workflow_id = arguments.get("workflow_id", "")
|
||||
tenant_id = context.get("tenant_id")
|
||||
user_id = context.get("user_id")
|
||||
@@ -332,8 +347,9 @@ class AutomationPlugin(BasePlugin):
|
||||
async def _check_workflow_status_handler(arguments: dict[str, Any], context: dict[str, Any]) -> dict[str, Any]:
|
||||
"""Check the status of a workflow instance."""
|
||||
from sqlalchemy import select
|
||||
from app.models.workflow import WorkflowInstance
|
||||
|
||||
from app.core.db import get_worker_session_factory
|
||||
from app.models.workflow import WorkflowInstance
|
||||
instance_id = arguments.get("instance_id", "")
|
||||
tenant_id = context.get("tenant_id")
|
||||
if not instance_id or not tenant_id:
|
||||
@@ -418,16 +434,16 @@ class AutomationPlugin(BasePlugin):
|
||||
from another plugin's manifest. Uses plugin name prefixing for conflict resolution."""
|
||||
from sqlalchemy import select
|
||||
|
||||
# Get default tenant_id from the first tenant in the DB
|
||||
from app.models.tenant import Tenant
|
||||
# Get system tenant for contributions (ARCH-043: deterministic slug
|
||||
# lookup instead of arbitrary first row)
|
||||
from app.core.db import get_system_tenant
|
||||
from app.plugins.builtins.automation.models import AutomationCronJob
|
||||
from app.plugins.builtins.automation.services import (
|
||||
AgentService,
|
||||
AutomationService,
|
||||
CronJobService,
|
||||
)
|
||||
tenant_result = await db.execute(select(Tenant).limit(1))
|
||||
tenant = tenant_result.scalar_one_or_none()
|
||||
tenant = await get_system_tenant(db)
|
||||
default_tenant_id = tenant.id if tenant else None
|
||||
if default_tenant_id is None:
|
||||
logger.warning("No tenant found — skipping plugin contributions registration")
|
||||
|
||||
@@ -3,7 +3,9 @@
|
||||
Enriches contact data by searching for related information.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import uuid
|
||||
|
||||
from app.plugins.builtins.automation.models import AgentDefinition
|
||||
|
||||
CONTACT_ENRICHMENT_SYSTEM_PROMPT = """You are a Contact Enrichment Agent for a CRM system.
|
||||
|
||||
@@ -3,7 +3,9 @@
|
||||
Sorts and prioritizes incoming emails automatically.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import uuid
|
||||
|
||||
from app.plugins.builtins.automation.models import AgentDefinition
|
||||
|
||||
EMAIL_TRIAGE_SYSTEM_PROMPT = """You are an E-Mail Triage Agent for a CRM system.
|
||||
|
||||
@@ -3,7 +3,9 @@
|
||||
Reminds about and creates follow-up tasks for contacts.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import uuid
|
||||
|
||||
from app.plugins.builtins.automation.models import AgentDefinition
|
||||
|
||||
FOLLOW_UP_SYSTEM_PROMPT = """You are a Follow-up Agent for a CRM system.
|
||||
|
||||
@@ -3,7 +3,9 @@
|
||||
Generates reports from CRM data using search and API tools.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import uuid
|
||||
|
||||
from app.plugins.builtins.automation.models import AgentDefinition
|
||||
|
||||
REPORT_SYSTEM_PROMPT = """You are a Report Agent for a CRM system.
|
||||
|
||||
@@ -1,60 +1,132 @@
|
||||
"""Tests for the Automation & Agents plugin.
|
||||
|
||||
Uses pytest with async fixtures. Tests use SQLite in-memory database
|
||||
since PostgreSQL may not be available in the dev container.
|
||||
Uses pytest with async fixtures against an ephemeral PostgreSQL database
|
||||
(SQLITE-001 fix) — matches the project convention and exercises the real
|
||||
PGUUID/JSONB column types.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
# Register ALL plugin models so create_all can resolve cross-plugin FKs
|
||||
# (e.g. entity_attachments.dms_file_id -> files) — same pattern as
|
||||
# scripts/sync_plugin_schema.py.
|
||||
import importlib
|
||||
import os
|
||||
import pkgutil
|
||||
import uuid
|
||||
from collections.abc import AsyncGenerator
|
||||
from datetime import UTC, datetime, timedelta
|
||||
|
||||
import pytest
|
||||
import pytest_asyncio
|
||||
from sqlalchemy import text
|
||||
from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker, create_async_engine
|
||||
|
||||
import app.models # noqa: F401 — registers core models
|
||||
import app.models.outbox # noqa: F401 — event_outbox is NOT re-exported by app.models
|
||||
import app.plugins.builtins as _builtins_pkg
|
||||
from app.core.db import Base
|
||||
from app.plugins.builtins.automation.models import (
|
||||
|
||||
for _importer, _modname, _ispkg in pkgutil.iter_modules(_builtins_pkg.__path__):
|
||||
if not _ispkg:
|
||||
continue
|
||||
try:
|
||||
importlib.import_module(f"app.plugins.builtins.{_modname}.models")
|
||||
except ImportError:
|
||||
pass # plugin without models module
|
||||
except Exception: # pragma: no cover - defensive
|
||||
pass
|
||||
|
||||
from app.plugins.builtins.automation.models import ( # noqa: E402 — after dynamic plugin-model discovery
|
||||
AgentRun,
|
||||
AutomationRun,
|
||||
)
|
||||
from app.plugins.builtins.automation.services import (
|
||||
from app.plugins.builtins.automation.services import ( # noqa: E402 — after dynamic plugin-model discovery
|
||||
AgentService,
|
||||
AutomationService,
|
||||
CronJobService,
|
||||
)
|
||||
|
||||
|
||||
def _ephemeral_db_url() -> str:
|
||||
"""Derive an ephemeral test DB URL from DATABASE_URL/.env.test."""
|
||||
base_url = os.environ.get(
|
||||
"DATABASE_URL",
|
||||
"postgresql+asyncpg://leocrm_test:test123@localhost:5432/leocrm_test",
|
||||
)
|
||||
return f"{base_url.rsplit('/', 1)[0]}/automation_test_{uuid.uuid4().hex[:8]}"
|
||||
|
||||
|
||||
# ─── Fixtures ───
|
||||
|
||||
|
||||
@pytest_asyncio.fixture
|
||||
async def db() -> AsyncGenerator[AsyncSession, None]:
|
||||
"""Create an in-memory SQLite database for testing."""
|
||||
engine = create_async_engine(
|
||||
"sqlite+aiosqlite:///:memory:",
|
||||
echo=False,
|
||||
)
|
||||
"""Create an ephemeral PostgreSQL database for this test run."""
|
||||
db_url = _ephemeral_db_url()
|
||||
admin_url = db_url.rsplit("/", 1)[0] + "/postgres"
|
||||
|
||||
from sqlalchemy.ext.asyncio import create_async_engine as _cae
|
||||
|
||||
admin_engine = _cae(admin_url, isolation_level="AUTOCOMMIT")
|
||||
async with admin_engine.connect() as conn:
|
||||
await conn.execute(text(f'CREATE DATABASE "{db_url.rsplit("/", 1)[1]}"'))
|
||||
await admin_engine.dispose()
|
||||
|
||||
# Plugin models use the pgvector Vector type — enable the extension in
|
||||
# the fresh database before create_all runs (must connect to the target
|
||||
# DB itself; CREATE EXTENSION has no ON DATABASE clause).
|
||||
ext_engine = _cae(db_url, isolation_level="AUTOCOMMIT")
|
||||
async with ext_engine.connect() as conn:
|
||||
await conn.execute(text("CREATE EXTENSION IF NOT EXISTS vector"))
|
||||
await ext_engine.dispose()
|
||||
|
||||
engine = create_async_engine(db_url, echo=False)
|
||||
|
||||
async with engine.begin() as conn:
|
||||
await conn.run_sync(Base.metadata.create_all)
|
||||
|
||||
async_session = async_sessionmaker(engine, class_=AsyncSession, expire_on_commit=False)
|
||||
|
||||
try:
|
||||
async with async_session() as session:
|
||||
yield session
|
||||
|
||||
finally:
|
||||
await engine.dispose()
|
||||
admin_engine2 = _cae(admin_url, isolation_level="AUTOCOMMIT")
|
||||
async with admin_engine2.connect() as conn:
|
||||
await conn.execute(text(f'DROP DATABASE IF EXISTS "{db_url.rsplit("/", 1)[1]}"'))
|
||||
await admin_engine2.dispose()
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def tenant_id() -> uuid.UUID:
|
||||
return uuid.uuid4()
|
||||
@pytest_asyncio.fixture
|
||||
async def tenant_id(db: AsyncSession) -> uuid.UUID:
|
||||
"""Create a real tenant row — PostgreSQL enforces FKs, unlike SQLite."""
|
||||
from app.models.tenant import Tenant
|
||||
|
||||
tid = uuid.uuid4()
|
||||
db.add(Tenant(id=tid, name="Test Org", slug=f"test-{tid.hex[:8]}"))
|
||||
await db.commit()
|
||||
return tid
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def user_id() -> uuid.UUID:
|
||||
return uuid.uuid4()
|
||||
@pytest_asyncio.fixture
|
||||
async def user_id(db: AsyncSession, tenant_id: uuid.UUID) -> uuid.UUID:
|
||||
"""Create a real user row belonging to the test tenant."""
|
||||
from app.models.user import User
|
||||
|
||||
uid = uuid.uuid4()
|
||||
db.add(
|
||||
User(
|
||||
id=uid,
|
||||
email=f"test-{uid.hex[:8]}@example.com",
|
||||
name="Test User",
|
||||
password_hash="not-a-real-hash",
|
||||
is_active=True,
|
||||
)
|
||||
)
|
||||
await db.commit()
|
||||
return uid
|
||||
|
||||
|
||||
# ─── AgentService Tests ───
|
||||
@@ -425,11 +497,17 @@ class TestDryRunMode:
|
||||
assert automation.dry_run is True
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_dry_run_flag_in_run(self, db: AsyncSession, tenant_id: uuid.UUID):
|
||||
async def test_dry_run_flag_in_run(self, db: AsyncSession, tenant_id: uuid.UUID, user_id: uuid.UUID):
|
||||
"""Test that dry_run flag is stored in AutomationRun."""
|
||||
# PostgreSQL enforces the FK to automations — create a real one first
|
||||
data = {"name": "dry-run-flag", "description": "", "trigger_type": "manual",
|
||||
"trigger_config": {}, "conditions": [], "actions": [],
|
||||
"is_active": True, "dry_run": True}
|
||||
automation = await AutomationService.create(db, tenant_id, data, user_id=user_id)
|
||||
|
||||
run = AutomationRun(
|
||||
tenant_id=tenant_id,
|
||||
automation_id=uuid.uuid4(),
|
||||
automation_id=automation.id,
|
||||
status="dry_run",
|
||||
started_at=datetime.now(UTC),
|
||||
dry_run=True,
|
||||
@@ -475,7 +553,9 @@ class TestRateLimiting:
|
||||
)
|
||||
recent_runs = result.scalar() or 0
|
||||
assert recent_runs == 2
|
||||
assert recent_runs < agent.max_executions_per_hour # 2 < 2 is False, so limit would be hit
|
||||
# With max_executions_per_hour=2 and 2 runs in the window, the limit
|
||||
# is reached — the next execution must be blocked.
|
||||
assert recent_runs >= agent.max_executions_per_hour
|
||||
|
||||
|
||||
# ─── Budget Limit Tests ───
|
||||
@@ -512,7 +592,8 @@ class TestBudgetLimit:
|
||||
.where(AgentRun.agent_id == agent.id)
|
||||
)
|
||||
total_cost = float(cost_result.scalar() or 0.0)
|
||||
assert total_cost == 0.6
|
||||
# FLOAT column accumulates binary rounding (0.6000000000000001)
|
||||
assert total_cost == pytest.approx(0.6)
|
||||
assert total_cost >= agent.budget_limit_usd # 0.6 >= 0.5, budget exceeded
|
||||
|
||||
|
||||
|
||||
@@ -6,6 +6,7 @@ import uuid
|
||||
from datetime import datetime
|
||||
from typing import Any
|
||||
|
||||
from pgvector.sqlalchemy import Vector
|
||||
from sqlalchemy import (
|
||||
Boolean,
|
||||
DateTime,
|
||||
@@ -13,14 +14,12 @@ from sqlalchemy import (
|
||||
Index,
|
||||
String,
|
||||
)
|
||||
from sqlalchemy.dialects.postgresql import JSONB
|
||||
from sqlalchemy.dialects.postgresql import JSONB, TSVECTOR
|
||||
from sqlalchemy.dialects.postgresql import UUID as PGUUID
|
||||
from sqlalchemy.orm import Mapped, mapped_column
|
||||
|
||||
from app.core.db import Base, TenantMixin
|
||||
from app.models.owned_mixin import OwnedMixin
|
||||
from sqlalchemy.dialects.postgresql import TSVECTOR
|
||||
from pgvector.sqlalchemy import Vector
|
||||
|
||||
|
||||
class Calendar(Base, TenantMixin, OwnedMixin):
|
||||
|
||||
@@ -4,7 +4,7 @@ from __future__ import annotations
|
||||
|
||||
from app.plugins.base import BasePlugin
|
||||
from app.plugins.manifest import (
|
||||
FrontendDetailTab,
|
||||
FrontendDashboardWidget,
|
||||
FrontendMenuItem,
|
||||
FrontendPageRoute,
|
||||
PluginManifest,
|
||||
@@ -40,6 +40,18 @@ class CalendarPlugin(BasePlugin):
|
||||
],
|
||||
events=[],
|
||||
migrations=["0001_initial.sql", "0002_add_deleted_at.sql"],
|
||||
dashboard_widgets=[
|
||||
FrontendDashboardWidget(
|
||||
id="calendar_upcoming",
|
||||
label_key="dashboard.calendarUpcoming",
|
||||
label="Upcoming Appointments",
|
||||
component="@/components/dashboard/CalendarUpcomingWidget",
|
||||
icon="Calendar",
|
||||
order=30,
|
||||
col_span=1,
|
||||
permission="calendar:read",
|
||||
),
|
||||
],
|
||||
permissions=[
|
||||
"calendar:read",
|
||||
"calendar:write",
|
||||
@@ -48,14 +60,14 @@ class CalendarPlugin(BasePlugin):
|
||||
"calendar:admin",
|
||||
],
|
||||
menu_items=[
|
||||
FrontendMenuItem(label_key='nav.calendar', label='Kalender', path='/calendar', icon='Calendar', order=20),
|
||||
FrontendMenuItem(label_key='nav.calendar', label='Kalender', path='/calendar', icon='Calendar', order=20, permission='calendar:read'),
|
||||
],
|
||||
page_routes=[
|
||||
FrontendPageRoute(path='/calendar', component='@/pages/Calendar', protected=True),
|
||||
],
|
||||
detail_tabs=[
|
||||
FrontendDetailTab(entity_type='contact', label_key='tabs.calendar', label='Calendar', component='@/components/contact/ContactCalendarTab', icon='Calendar', order=30, permission='calendar:read'),
|
||||
FrontendPageRoute(path='/calendar', component='@/pages/Calendar', protected=True, permission='calendar:read'),
|
||||
],
|
||||
# BUG (ghost component): ContactCalendarTab does not exist in the
|
||||
# frontend — tab removed until implemented (Block I-D).
|
||||
detail_tabs=[],
|
||||
author="LeoCRM Team",
|
||||
min_app_version="1.0.0",
|
||||
hooks=["calendar.before_appointment", "calendar.after_appointment"],
|
||||
|
||||
@@ -21,6 +21,7 @@ from fastapi.responses import StreamingResponse
|
||||
from sqlalchemy import select, update
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
|
||||
from app.core.audit import log_audit
|
||||
from app.core.db import get_db
|
||||
from app.deps import get_current_user, require_admin, require_permission
|
||||
from app.plugins.builtins.calendar.ics_utils import (
|
||||
@@ -1023,5 +1024,3 @@ async def book_resource(
|
||||
"start_at": booking.start_at.isoformat(),
|
||||
"end_at": booking.end_at.isoformat(),
|
||||
}
|
||||
|
||||
from app.core.audit import log_audit
|
||||
@@ -0,0 +1,62 @@
|
||||
"""Public contract for the contacts plugin.
|
||||
|
||||
Exposes the symbols that other core modules and plugins need without
|
||||
importing from internal modules directly (Block C7: dashboard counts).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Any
|
||||
|
||||
from sqlalchemy import func, select
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
|
||||
from app.core.visibility import apply_visibility_filter
|
||||
from app.models.contact import Contact
|
||||
from app.plugins.builtins.contracts import get_contract_registry
|
||||
|
||||
|
||||
class ContactsContract:
|
||||
"""Public API surface for the contacts plugin."""
|
||||
|
||||
contract_name = "contacts"
|
||||
|
||||
@staticmethod
|
||||
async def get_counts(
|
||||
db: AsyncSession,
|
||||
tenant_id: Any,
|
||||
user_id: Any,
|
||||
is_system_admin: bool = False,
|
||||
) -> dict[str, int]:
|
||||
"""Return visibility-filtered contact/company/person counts."""
|
||||
queries = []
|
||||
for type_filter in (None, "company", "person"):
|
||||
query = select(func.count(Contact.id)).where(
|
||||
Contact.tenant_id == tenant_id,
|
||||
Contact.deleted_at.is_(None),
|
||||
)
|
||||
if type_filter is not None:
|
||||
query = query.where(Contact.type == type_filter)
|
||||
query = await apply_visibility_filter(
|
||||
db, query, "contact", Contact, user_id, tenant_id, is_system_admin
|
||||
)
|
||||
queries.append(query)
|
||||
|
||||
results = [((await db.execute(q)).scalar() or 0) for q in queries]
|
||||
return {
|
||||
"contacts": results[0],
|
||||
"companies": results[1],
|
||||
"persons": results[2],
|
||||
"total": results[0],
|
||||
}
|
||||
|
||||
@classmethod
|
||||
def get_function(cls, name: str):
|
||||
"""Return a callable exposed by this contract, or None if absent."""
|
||||
return getattr(cls, name, None)
|
||||
|
||||
|
||||
# ─── self-registration ───
|
||||
|
||||
_contract = ContactsContract()
|
||||
get_contract_registry().register("contacts", _contract)
|
||||
@@ -9,27 +9,67 @@ from __future__ import annotations
|
||||
import logging
|
||||
|
||||
from app.plugins.base import BasePlugin
|
||||
from app.plugins.manifest import PluginManifest
|
||||
from app.plugins.manifest import (
|
||||
FrontendDashboardWidget,
|
||||
PluginManifest,
|
||||
PluginRouteDef,
|
||||
)
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class ContactsPlugin(BasePlugin):
|
||||
"""Contacts plugin — manages Contact entity lifecycle (models, permissions, restore, history).
|
||||
"""Contacts plugin — owns the full Contact domain (Block B1).
|
||||
|
||||
Routes remain in app/routes/contacts.py as core routes, but entity lifecycle
|
||||
(permissions, entity models, restore, history) is managed through on_activate/on_deactivate.
|
||||
Routes (contacts, companies, contact folders, folder permissions) live in
|
||||
this plugin and are mounted via manifest.routes with
|
||||
require_active_plugin("contacts") protection. Entity lifecycle
|
||||
(permissions, entity models, restore, history) is managed through
|
||||
on_activate/on_deactivate like every other business plugin.
|
||||
"""
|
||||
|
||||
manifest = PluginManifest(
|
||||
name="contacts",
|
||||
version="1.0.0",
|
||||
version="1.1.0",
|
||||
display_name="Contacts",
|
||||
description="Core CRM contacts — persons and companies.",
|
||||
dependencies=[],
|
||||
routes=[], # Routes are registered as core routes in main.py
|
||||
routes=[
|
||||
PluginRouteDef(
|
||||
path="/api/v1/contacts",
|
||||
module="app.plugins.builtins.contacts.routes",
|
||||
router_attr="router",
|
||||
),
|
||||
PluginRouteDef(
|
||||
path="/api/v1/companies",
|
||||
module="app.plugins.builtins.contacts.company_routes",
|
||||
router_attr="router",
|
||||
),
|
||||
PluginRouteDef(
|
||||
path="/api/v1/contact-folders",
|
||||
module="app.plugins.builtins.contacts.folder_routes",
|
||||
router_attr="router",
|
||||
),
|
||||
PluginRouteDef(
|
||||
path="/api/v1/contact-folders",
|
||||
module="app.plugins.builtins.contacts.folder_permission_routes",
|
||||
router_attr="router",
|
||||
),
|
||||
],
|
||||
events=[],
|
||||
migrations=[],
|
||||
dashboard_widgets=[
|
||||
FrontendDashboardWidget(
|
||||
id="recent_contacts",
|
||||
label_key="dashboard.recentContacts",
|
||||
label="Recent Contacts",
|
||||
component="@/components/dashboard/RecentContactsWidget",
|
||||
icon="Users",
|
||||
order=10,
|
||||
col_span=2,
|
||||
permission="contacts:read",
|
||||
),
|
||||
],
|
||||
permissions=[
|
||||
"contacts:read",
|
||||
"contacts:write",
|
||||
|
||||
@@ -59,20 +59,31 @@ class ContractRegistry:
|
||||
cls._instance = super().__new__(cls)
|
||||
cls._instance._contracts: dict[str, Any] = {}
|
||||
cls._instance._loaded: set[str] = set()
|
||||
cls._instance._unregistered: set[str] = set()
|
||||
return cls._instance
|
||||
|
||||
# ─── registration ───
|
||||
|
||||
def register(self, plugin_name: str, contract: Any) -> None:
|
||||
"""Register or replace a contract for a plugin."""
|
||||
"""Register or replace a contract for a plugin.
|
||||
|
||||
Clears the unregistered marker so a later deactivation can be
|
||||
distinguished from a fresh lazy-load again (ARCH-014).
|
||||
"""
|
||||
self._unregistered.discard(plugin_name)
|
||||
self._contracts[plugin_name] = contract
|
||||
self._loaded.add(plugin_name)
|
||||
logger.debug("Contract registered for plugin '%s'", plugin_name)
|
||||
|
||||
def unregister(self, plugin_name: str) -> None:
|
||||
"""Remove a contract (e.g. when the plugin is deactivated)."""
|
||||
"""Remove a contract (e.g. when the plugin is deactivated).
|
||||
|
||||
Marks the plugin as explicitly unregistered so later ``get_contract``
|
||||
calls cannot resurrect the contract via lazy-loading (ARCH-014).
|
||||
"""
|
||||
self._contracts.pop(plugin_name, None)
|
||||
self._loaded.discard(plugin_name)
|
||||
self._unregistered.add(plugin_name)
|
||||
|
||||
# ─── lookup ───
|
||||
|
||||
@@ -85,6 +96,11 @@ class ContractRegistry:
|
||||
if plugin_name in self._contracts:
|
||||
return self._contracts[plugin_name]
|
||||
|
||||
# Explicitly unregistered (deactivated): never resurrect via
|
||||
# lazy-loading (ARCH-014) — the deactivated contract must stay gone.
|
||||
if plugin_name in self._unregistered:
|
||||
return None
|
||||
|
||||
if plugin_name not in self._loaded:
|
||||
self._try_lazy_load(plugin_name)
|
||||
|
||||
|
||||
@@ -4,16 +4,16 @@ from __future__ import annotations
|
||||
|
||||
import uuid
|
||||
from datetime import datetime
|
||||
from typing import Any
|
||||
|
||||
from sqlalchemy import DateTime, ForeignKey, Index, Integer, String, UniqueConstraint, Text
|
||||
from pgvector.sqlalchemy import Vector
|
||||
from sqlalchemy import DateTime, ForeignKey, Index, Integer, String, Text, UniqueConstraint
|
||||
from sqlalchemy.dialects.postgresql import TSVECTOR
|
||||
from sqlalchemy.dialects.postgresql import UUID as PGUUID
|
||||
from sqlalchemy.orm import Mapped, mapped_column
|
||||
|
||||
from app.core.db import Base, TenantMixin
|
||||
from app.models.owned_mixin import OwnedMixin
|
||||
from pgvector.sqlalchemy import Vector
|
||||
from sqlalchemy.dialects.postgresql import TSVECTOR
|
||||
from typing import Any
|
||||
|
||||
|
||||
class Folder(Base, TenantMixin, OwnedMixin):
|
||||
|
||||
@@ -4,7 +4,6 @@ from __future__ import annotations
|
||||
|
||||
from app.plugins.base import BasePlugin
|
||||
from app.plugins.manifest import (
|
||||
FrontendDetailTab,
|
||||
FrontendMenuItem,
|
||||
FrontendPageRoute,
|
||||
PluginManifest,
|
||||
@@ -38,22 +37,35 @@ class DmsPlugin(BasePlugin):
|
||||
"dms:admin",
|
||||
],
|
||||
menu_items=[
|
||||
FrontendMenuItem(label_key='nav.dms', label='Dateien', path='/dms', icon='FolderOpen', group='Dateien', order=40),
|
||||
FrontendMenuItem(label_key='nav.dms.trash', label='Papierkorb', path='/dms/trash', icon='Trash2', group='Dateien', order=41),
|
||||
FrontendMenuItem(label_key='nav.dms', label='Dateien', path='/dms', icon='FolderOpen', group='Dateien', order=40, permission='dms:read'),
|
||||
FrontendMenuItem(label_key='nav.dms.trash', label='Papierkorb', path='/dms/trash', icon='Trash2', group='Dateien', order=41, permission='dms:read'),
|
||||
],
|
||||
page_routes=[
|
||||
FrontendPageRoute(path='/dms', component='@/pages/Dms', protected=True),
|
||||
FrontendPageRoute(path='/dms/trash', component='@/pages/DmsTrash', protected=True),
|
||||
],
|
||||
detail_tabs=[
|
||||
FrontendDetailTab(entity_type='contact', label_key='tabs.files', label='Dateien', component='@/components/contact/ContactFilesTab', icon='FolderOpen', order=40, permission='dms:read'),
|
||||
FrontendPageRoute(path='/dms', component='@/pages/Dms', protected=True, permission='dms:read'),
|
||||
FrontendPageRoute(path='/dms/trash', component='@/pages/DmsTrash', protected=True, permission='dms:read'),
|
||||
],
|
||||
# BUG (ghost component): ContactFilesTab does not exist in the
|
||||
# frontend — tab removed until implemented (Block I-D).
|
||||
detail_tabs=[],
|
||||
author="LeoCRM Team",
|
||||
min_app_version="1.0.0",
|
||||
hooks=["dms.before_upload"],
|
||||
contract_version="1.0.0",
|
||||
)
|
||||
|
||||
def get_entity_models(self) -> dict[str, type]:
|
||||
"""Entity types owned by DMS for the permission resolver.
|
||||
|
||||
``file`` is a legacy alias still used by the permissions routes.
|
||||
"""
|
||||
from app.plugins.builtins.dms.models import File, Folder
|
||||
|
||||
return {
|
||||
"dms_file": File,
|
||||
"dms_folder": Folder,
|
||||
"file": File,
|
||||
}
|
||||
|
||||
async def on_activate(self, db, service_container, event_bus) -> None:
|
||||
"""Activate plugin: register restore config + history hooks."""
|
||||
await super().on_activate(db, service_container, event_bus)
|
||||
|
||||
@@ -5,7 +5,7 @@ from __future__ import annotations
|
||||
from typing import Any
|
||||
|
||||
from app.plugins.base import BasePlugin
|
||||
from app.plugins.manifest import FrontendDetailTab, PluginManifest, PluginRouteDef
|
||||
from app.plugins.manifest import PluginManifest, PluginRouteDef
|
||||
|
||||
|
||||
class EntityLinksPlugin(BasePlugin):
|
||||
@@ -42,9 +42,9 @@ class EntityLinksPlugin(BasePlugin):
|
||||
"entity_links:delete",
|
||||
],
|
||||
is_core=True,
|
||||
detail_tabs=[
|
||||
FrontendDetailTab(entity_type='contact', label_key='tabs.links', label='Verknüpfungen', component='@/components/contact/ContactLinksTab', icon='Link', order=60, permission='entity_links:read'),
|
||||
],
|
||||
# BUG (ghost component): ContactLinksTab does not exist in the
|
||||
# frontend — tab removed until implemented (Block I-D).
|
||||
detail_tabs=[],
|
||||
|
||||
author="LeoCRM Team",
|
||||
min_app_version="1.0.0",
|
||||
|
||||
@@ -40,6 +40,10 @@ class GraphRAGPlugin(BasePlugin):
|
||||
|
||||
async def on_activate(self, db, service_container, event_bus) -> None:
|
||||
"""Activate plugin: register GraphRAG search provider."""
|
||||
# Base class first (event subscriptions etc.), then own registrations
|
||||
# so a failure in our code cannot leave the base half-initialized (ARCH-037).
|
||||
await super().on_activate(db, service_container, event_bus)
|
||||
|
||||
from app.plugins.builtins.graph_rag.provider import GraphRAGSearchProvider
|
||||
from app.plugins.builtins.unified_search.contracts import get_search_registry
|
||||
|
||||
@@ -50,8 +54,6 @@ class GraphRAGPlugin(BasePlugin):
|
||||
import logging
|
||||
logging.getLogger(__name__).exception("Failed to register GraphRAGSearchProvider")
|
||||
|
||||
await super().on_activate(db, service_container, event_bus)
|
||||
|
||||
async def on_deactivate(self, db, service_container, event_bus) -> None:
|
||||
"""Deactivate plugin: unregister search provider and contract."""
|
||||
from app.plugins.builtins.unified_search.contracts import get_search_registry
|
||||
|
||||
@@ -25,7 +25,8 @@ async def cleanup_knowledge_job(ctx: dict[str, Any]) -> None:
|
||||
Runs daily. Keeps approved extractions indefinitely.
|
||||
Iterates per-tenant for RLS compliance.
|
||||
"""
|
||||
from sqlalchemy import text as sa_text, delete as sa_delete
|
||||
from sqlalchemy import delete as sa_delete
|
||||
from sqlalchemy import text as sa_text
|
||||
|
||||
from app.core.db import get_worker_session_factory
|
||||
|
||||
|
||||
@@ -1,12 +1,17 @@
|
||||
"""Knowledge extraction models — tracks LLM extractions and review queue."""
|
||||
from __future__ import annotations
|
||||
|
||||
import uuid
|
||||
from datetime import datetime
|
||||
from sqlalchemy import DateTime, Float, ForeignKey, Index, Integer, String, Text, func
|
||||
from sqlalchemy.dialects.postgresql import JSONB, UUID as PGUUID
|
||||
|
||||
from sqlalchemy import DateTime, Float, ForeignKey, Index, String, Text, func
|
||||
from sqlalchemy.dialects.postgresql import JSONB
|
||||
from sqlalchemy.dialects.postgresql import UUID as PGUUID
|
||||
from sqlalchemy.orm import Mapped, mapped_column
|
||||
|
||||
from app.core.db import Base, TenantMixin
|
||||
|
||||
|
||||
class KnowledgeExtraction(Base, TenantMixin):
|
||||
"""Tracks a single knowledge extraction run from a source (wiki, dms, mail, comm)."""
|
||||
__tablename__ = "knowledge_extractions"
|
||||
|
||||
@@ -1,8 +1,10 @@
|
||||
"""Knowledge plugin — LLM-based entity/relationship extraction, ask-knowledge, review queue."""
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import uuid
|
||||
from typing import Any
|
||||
|
||||
from app.plugins.base import BasePlugin
|
||||
from app.plugins.manifest import PluginManifest, PluginRouteDef
|
||||
|
||||
@@ -25,7 +27,7 @@ class KnowledgePlugin(BasePlugin):
|
||||
"""Register event-driven extraction hooks on activation."""
|
||||
await super().on_activate(db, service_container, event_bus)
|
||||
try:
|
||||
from app.core.hooks import register_action
|
||||
from app.core.hooks import get_hook_registry
|
||||
from app.plugins.builtins.knowledge.services import extract_knowledge
|
||||
async def on_wiki_create(*args, **kwargs):
|
||||
article_id = kwargs.get("article_id") or kwargs.get("entity_id")
|
||||
@@ -41,7 +43,9 @@ class KnowledgePlugin(BasePlugin):
|
||||
source_type="wiki_article", source_id=uuid.UUID(str(article_id)),
|
||||
source_title=title, source_text=content,
|
||||
)
|
||||
register_action("wiki.article.created", on_wiki_create, priority=20, owner_tag="knowledge")
|
||||
get_hook_registry().register_action(
|
||||
"wiki.article.created", on_wiki_create, priority=20, owner_tag="knowledge"
|
||||
)
|
||||
# H-DATA-LIFE: Re-extract when wiki article is updated
|
||||
async def on_wiki_update(*args, **kwargs):
|
||||
article_id = kwargs.get("article_id") or kwargs.get("entity_id")
|
||||
@@ -57,7 +61,9 @@ class KnowledgePlugin(BasePlugin):
|
||||
source_type="wiki_article", source_id=uuid.UUID(str(article_id)),
|
||||
source_title=title, source_text=content,
|
||||
)
|
||||
register_action("wiki.article.updated", on_wiki_update, priority=20, owner_tag="knowledge")
|
||||
get_hook_registry().register_action(
|
||||
"wiki.article.updated", on_wiki_update, priority=20, owner_tag="knowledge"
|
||||
)
|
||||
logger.info("Registered knowledge extraction hooks")
|
||||
except Exception:
|
||||
logger.exception("Failed to register knowledge hooks")
|
||||
@@ -74,8 +80,8 @@ class KnowledgePlugin(BasePlugin):
|
||||
|
||||
async def _ask_knowledge_handler(arguments: dict[str, Any], context: dict[str, Any]) -> dict[str, Any]:
|
||||
"""Ask a knowledge question."""
|
||||
from app.plugins.builtins.knowledge.services import ask_knowledge
|
||||
from app.core.db import get_worker_session_factory
|
||||
from app.plugins.builtins.knowledge.services import ask_knowledge
|
||||
question = arguments.get("question", "")
|
||||
tenant_id = context.get("tenant_id")
|
||||
if not question or not tenant_id:
|
||||
@@ -103,8 +109,8 @@ class KnowledgePlugin(BasePlugin):
|
||||
|
||||
async def _search_knowledge_handler(arguments: dict[str, Any], context: dict[str, Any]) -> dict[str, Any]:
|
||||
"""Search wiki articles via unified search."""
|
||||
from app.plugins.builtins.unified_search.provider_registry import get_search_registry
|
||||
from app.core.db import get_worker_session_factory
|
||||
from app.plugins.builtins.unified_search.provider_registry import get_search_registry
|
||||
query = arguments.get("query", "")
|
||||
tenant_id = context.get("tenant_id")
|
||||
if not query or not tenant_id:
|
||||
@@ -149,8 +155,8 @@ class KnowledgePlugin(BasePlugin):
|
||||
|
||||
async def on_deactivate(self, db, service_container, event_bus) -> None:
|
||||
"""Clean up on deactivation."""
|
||||
from app.core.hooks import unregister_actions_by_owner
|
||||
unregister_actions_by_owner("knowledge")
|
||||
from app.core.hooks import get_hook_registry
|
||||
get_hook_registry().unregister_all_for_plugin("knowledge")
|
||||
# Unregister knowledge agent tools from the core AI tool registry
|
||||
try:
|
||||
from app.ai.tool_registry import get_tool_registry
|
||||
|
||||
@@ -1,51 +1,70 @@
|
||||
"""Knowledge plugin routes — extraction, ask, review queue."""
|
||||
from __future__ import annotations
|
||||
|
||||
import uuid
|
||||
|
||||
from fastapi import APIRouter, Depends, HTTPException, Query
|
||||
from pydantic import BaseModel, Field
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
|
||||
from app.core.db import get_db
|
||||
from app.deps import require_permission
|
||||
from app.plugins.builtins.knowledge.services import extract_knowledge, ask_knowledge, get_review_queue, review_extraction
|
||||
from app.plugins.builtins.knowledge.services import (
|
||||
ask_knowledge,
|
||||
extract_knowledge,
|
||||
get_review_queue,
|
||||
review_extraction,
|
||||
)
|
||||
|
||||
router = APIRouter(prefix="/api/v1/knowledge", tags=["knowledge"])
|
||||
|
||||
|
||||
class KnowledgeExtractRequest(BaseModel):
|
||||
"""Extract knowledge from a source."""
|
||||
|
||||
source_type: str = Field(..., min_length=1)
|
||||
source_id: uuid.UUID
|
||||
source_title: str | None = None
|
||||
source_text: str = Field(..., min_length=1)
|
||||
|
||||
|
||||
class KnowledgeAskRequest(BaseModel):
|
||||
"""Ask a knowledge question."""
|
||||
|
||||
question: str = Field(..., min_length=1)
|
||||
|
||||
|
||||
class KnowledgeReviewRequest(BaseModel):
|
||||
"""Approve or reject a knowledge extraction."""
|
||||
|
||||
approved: bool = False
|
||||
notes: str | None = None
|
||||
|
||||
|
||||
@router.post("/extract")
|
||||
async def extract(
|
||||
body: dict,
|
||||
body: KnowledgeExtractRequest,
|
||||
db: AsyncSession = Depends(get_db),
|
||||
current_user: dict = Depends(require_permission("wiki:read")),
|
||||
):
|
||||
"""Extract knowledge from a source (wiki article, dms file, mail, communication)."""
|
||||
tenant_id = uuid.UUID(current_user["tenant_id"])
|
||||
source_type = body.get("source_type", "")
|
||||
source_id = body.get("source_id", "")
|
||||
source_title = body.get("source_title")
|
||||
source_text = body.get("source_text", "")
|
||||
if not source_type or not source_id or not source_text:
|
||||
raise HTTPException(400, detail={"detail": "source_type, source_id, source_text required", "code": "missing_fields"})
|
||||
try:
|
||||
sid = uuid.UUID(source_id)
|
||||
except ValueError:
|
||||
raise HTTPException(400, detail={"detail": "Invalid source_id", "code": "invalid_id"}) from None
|
||||
result = await extract_knowledge(
|
||||
db=db, tenant_id=tenant_id, source_type=source_type, source_id=sid,
|
||||
source_title=source_title, source_text=source_text,
|
||||
db=db, tenant_id=tenant_id, source_type=body.source_type, source_id=body.source_id,
|
||||
source_title=body.source_title, source_text=body.source_text,
|
||||
user_id=uuid.UUID(current_user["user_id"]) if current_user.get("user_id") else None,
|
||||
)
|
||||
return result
|
||||
|
||||
@router.post("/ask")
|
||||
async def ask(
|
||||
body: dict,
|
||||
body: KnowledgeAskRequest,
|
||||
db: AsyncSession = Depends(get_db),
|
||||
current_user: dict = Depends(require_permission("wiki:read")),
|
||||
):
|
||||
"""Ask a knowledge question — uses wiki + graph_rag as context."""
|
||||
tenant_id = uuid.UUID(current_user["tenant_id"])
|
||||
question = body.get("question") or body.get("query", "")
|
||||
if not question:
|
||||
raise HTTPException(400, detail={"detail": "question required", "code": "missing_question"})
|
||||
result = await ask_knowledge(db=db, tenant_id=tenant_id, question=question)
|
||||
result = await ask_knowledge(db=db, tenant_id=tenant_id, question=body.question)
|
||||
return result
|
||||
|
||||
@router.get("/review")
|
||||
@@ -62,22 +81,20 @@ async def review_queue(
|
||||
@router.post("/review/{extraction_id}")
|
||||
async def review(
|
||||
extraction_id: str,
|
||||
body: dict,
|
||||
body: KnowledgeReviewRequest,
|
||||
db: AsyncSession = Depends(get_db),
|
||||
current_user: dict = Depends(require_permission("wiki:write")),
|
||||
):
|
||||
"""Approve or reject a knowledge extraction."""
|
||||
tenant_id = uuid.UUID(current_user["tenant_id"])
|
||||
approved = body.get("approved", False)
|
||||
notes = body.get("notes")
|
||||
try:
|
||||
eid = uuid.UUID(extraction_id)
|
||||
except ValueError:
|
||||
raise HTTPException(400, detail={"detail": "Invalid extraction_id", "code": "invalid_id"}) from None
|
||||
result = await review_extraction(
|
||||
db=db, tenant_id=tenant_id, extraction_id=eid, approved=approved,
|
||||
db=db, tenant_id=tenant_id, extraction_id=eid, approved=body.approved,
|
||||
user_id=uuid.UUID(current_user["user_id"]) if current_user.get("user_id") else None,
|
||||
notes=notes,
|
||||
notes=body.notes,
|
||||
)
|
||||
if "error" in result:
|
||||
raise HTTPException(404, detail={"detail": result["error"], "code": "not_found"})
|
||||
|
||||
@@ -1,10 +1,13 @@
|
||||
"""Knowledge extraction services — LLM-based entity/relationship extraction."""
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import uuid
|
||||
from typing import Any
|
||||
from sqlalchemy import select, update
|
||||
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
|
||||
from app.ai.llm_client import llm_complete
|
||||
from app.plugins.builtins.knowledge.models import KnowledgeExtraction
|
||||
|
||||
|
||||
@@ -0,0 +1,7 @@
|
||||
-- Dual-path convergence (Gate B): add columns that Alembic migrations
|
||||
-- 0120 (is_system) and 0140 (folder_id) add on the core path. Idempotent
|
||||
-- so both install paths converge to the identical schema.
|
||||
ALTER TABLE comm_conversations ADD COLUMN IF NOT EXISTS is_system BOOLEAN NOT NULL DEFAULT FALSE;
|
||||
CREATE INDEX IF NOT EXISTS ix_comm_conversations_tenant_system ON comm_conversations(tenant_id, is_system);
|
||||
ALTER TABLE comm_conversations ADD COLUMN IF NOT EXISTS folder_id UUID;
|
||||
CREATE INDEX IF NOT EXISTS ix_comm_conversations_folder ON comm_conversations(folder_id);
|
||||
@@ -39,7 +39,7 @@ class KommunikationPlugin(BasePlugin):
|
||||
"participant.left",
|
||||
"reaction.added",
|
||||
],
|
||||
migrations=["0001_initial.sql"],
|
||||
migrations=["0001_initial.sql", "0002_system_channel_folders.sql"],
|
||||
permissions=[
|
||||
"comm:read",
|
||||
"comm:write",
|
||||
@@ -50,10 +50,10 @@ class KommunikationPlugin(BasePlugin):
|
||||
],
|
||||
is_core=True,
|
||||
menu_items=[
|
||||
FrontendMenuItem(label_key='nav.communication', label='Kommunikation', path='/communication', icon='MessageSquare', order=80),
|
||||
FrontendMenuItem(label_key='nav.communication', label='Kommunikation', path='/communication', icon='MessageSquare', order=80, permission='comm:read'),
|
||||
],
|
||||
page_routes=[
|
||||
FrontendPageRoute(path='/communication', component='@/pages/Communication', protected=True),
|
||||
FrontendPageRoute(path='/communication', component='@/pages/Communication', protected=True, permission='comm:read'),
|
||||
],
|
||||
author="LeoCRM Team",
|
||||
min_app_version="1.0.0",
|
||||
@@ -136,6 +136,13 @@ class KommunikationPlugin(BasePlugin):
|
||||
from app.plugins.builtins.contracts import get_contract_registry
|
||||
get_contract_registry().unregister(self.manifest.name)
|
||||
|
||||
# Remove services registered in on_activate BEFORE super() so that
|
||||
# event handlers being unsubscribed can no longer reach them (ARCH-033).
|
||||
for service_name in ("comm_websocket", "comm_miniapps"):
|
||||
if service_container.has(service_name):
|
||||
service_container.remove(service_name)
|
||||
logger.info("Removed '%s' from service container", service_name)
|
||||
|
||||
await super().on_deactivate(db, service_container, event_bus)
|
||||
logger.info("Kommunikation plugin deactivated")
|
||||
|
||||
|
||||
@@ -0,0 +1,117 @@
|
||||
"""Attachment storage helpers for the Mail plugin.
|
||||
|
||||
Extracted from services.py as part of the God-object split (BUG-018 pilot).
|
||||
Re-exported by ``app.plugins.builtins.mail.services``.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import re
|
||||
import uuid
|
||||
|
||||
import aiofiles
|
||||
|
||||
from app.config import settings
|
||||
from app.plugins.builtins.mail.models import MailAttachment
|
||||
|
||||
# ─── Attachment Storage Helpers ───
|
||||
|
||||
MAX_ATTACHMENT_SIZE = 25 * 1024 * 1024 # 25 MB
|
||||
|
||||
|
||||
def _decode_mime_filename(filename: str) -> str:
|
||||
"""Decode MIME-encoded filename, handling =?charset?Q?...?= and =?charset?B?...?= patterns."""
|
||||
if not filename:
|
||||
return "attachment"
|
||||
# If no MIME encoding pattern, return as-is
|
||||
if "=?" not in filename:
|
||||
return filename
|
||||
try:
|
||||
from email.header import decode_header, make_header
|
||||
|
||||
return str(make_header(decode_header(filename)))
|
||||
except Exception:
|
||||
# Fallback: manually decode Q-encoding if decode_header fails
|
||||
# This handles cases where the email parser partially processed the filename
|
||||
try:
|
||||
import base64
|
||||
|
||||
def decode_q(match):
|
||||
charset, encoding, encoded = (
|
||||
match.group(1),
|
||||
match.group(2).upper(),
|
||||
match.group(3),
|
||||
)
|
||||
if encoding == "B":
|
||||
decoded = base64.b64decode(encoded).decode(
|
||||
charset or "utf-8", errors="replace"
|
||||
)
|
||||
else: # Q encoding
|
||||
decoded = encoded.replace("_", " ")
|
||||
decoded = re.sub(
|
||||
r"=([0-9A-Fa-f]{2})",
|
||||
lambda m: chr(int(m.group(1), 16)),
|
||||
decoded,
|
||||
)
|
||||
decoded = decoded.encode("latin-1").decode(
|
||||
charset or "utf-8", errors="replace"
|
||||
)
|
||||
return decoded
|
||||
|
||||
return re.sub(r"=\?([^?]+)\?([BbQq])\?([^?]*)\?=", decode_q, filename)
|
||||
except Exception:
|
||||
return filename
|
||||
|
||||
|
||||
def _sanitize_filename(filename: str) -> str:
|
||||
"""Sanitize a filename to prevent path traversal attacks."""
|
||||
# Remove any path components — keep only the basename
|
||||
filename = os.path.basename(filename or "attachment")
|
||||
# Replace potentially dangerous characters
|
||||
filename = re.sub(r"[^a-zA-Z0-9._-]", "_", filename)
|
||||
# Ensure non-empty
|
||||
if not filename:
|
||||
filename = "attachment"
|
||||
# Limit length
|
||||
if len(filename) > 200:
|
||||
name, ext = os.path.splitext(filename)
|
||||
filename = name[:200 - len(ext)] + ext
|
||||
return filename
|
||||
|
||||
|
||||
def _attachment_storage_path(mail_id: uuid.UUID, filename: str) -> str:
|
||||
"""Build the on-disk storage path for a mail attachment."""
|
||||
safe_name = _sanitize_filename(filename)
|
||||
return os.path.join(
|
||||
settings.storage_path,
|
||||
"mail_attachments",
|
||||
str(mail_id),
|
||||
safe_name,
|
||||
)
|
||||
|
||||
|
||||
async def _save_attachment_to_storage(
|
||||
mail_id: uuid.UUID, filename: str, content: bytes
|
||||
) -> str:
|
||||
"""Save attachment content to disk and return the storage path."""
|
||||
storage_path = _attachment_storage_path(mail_id, filename)
|
||||
os.makedirs(os.path.dirname(storage_path), exist_ok=True)
|
||||
async with aiofiles.open(storage_path, "wb") as f:
|
||||
await f.write(content)
|
||||
return storage_path
|
||||
|
||||
|
||||
def attachment_to_response(att: MailAttachment) -> dict:
|
||||
"""Convert a MailAttachment ORM object to a response dict."""
|
||||
return {
|
||||
"id": str(att.id),
|
||||
"mail_id": str(att.mail_id),
|
||||
"filename": att.filename,
|
||||
"mime_type": att.mime_type,
|
||||
"size_bytes": att.size_bytes,
|
||||
"size": att.size_bytes, # alias for frontend compatibility
|
||||
"content_id": att.content_id,
|
||||
"is_inline": bool(att.content_id),
|
||||
"dms_file_id": str(att.dms_file_id) if att.dms_file_id else None,
|
||||
}
|
||||
@@ -0,0 +1,74 @@
|
||||
"""AES-256 password encryption for Mail accounts (Fernet-based).
|
||||
|
||||
Extracted from services.py as part of the God-object split (BUG-018 pilot).
|
||||
Re-exported by ``app.plugins.builtins.mail.services`` for backwards
|
||||
compatibility.
|
||||
|
||||
⚠️ The legacy salt and MAIL_ENCRYPTION_KEY env guard are load-bearing:
|
||||
changing them makes existing encrypted passwords unreadable.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import base64
|
||||
import os
|
||||
|
||||
from cryptography.fernet import Fernet
|
||||
from cryptography.hazmat.primitives import hashes
|
||||
from cryptography.hazmat.primitives.kdf.pbkdf2 import PBKDF2HMAC
|
||||
|
||||
MAIL_ENCRYPTION_KEY = os.environ.get("MAIL_ENCRYPTION_KEY")
|
||||
if not MAIL_ENCRYPTION_KEY:
|
||||
raise RuntimeError(
|
||||
"MAIL_ENCRYPTION_KEY environment variable is required. "
|
||||
"Set it to a strong random value."
|
||||
)
|
||||
|
||||
# Legacy salt for backward compatibility with existing encrypted passwords
|
||||
_LEGACY_SALT = b"leocrm-mail-salt"
|
||||
|
||||
|
||||
def _derive_key(password: str, salt: bytes) -> bytes:
|
||||
"""Derive a 32-byte Fernet key from a password using PBKDF2 with the given salt."""
|
||||
kdf = PBKDF2HMAC(
|
||||
algorithm=hashes.SHA256(),
|
||||
length=32,
|
||||
salt=salt,
|
||||
iterations=480000,
|
||||
)
|
||||
return base64.urlsafe_b64encode(kdf.derive(password.encode()))
|
||||
|
||||
|
||||
def generate_salt() -> str:
|
||||
"""Generate a random 32-byte salt and return as base64 string."""
|
||||
salt = os.urandom(32)
|
||||
return base64.urlsafe_b64encode(salt).decode()
|
||||
|
||||
|
||||
def _get_fernet(salt_b64: str | None = None) -> Fernet:
|
||||
"""Get a Fernet instance. If salt_b64 is provided, use it; otherwise use legacy salt."""
|
||||
if salt_b64:
|
||||
salt = base64.urlsafe_b64decode(salt_b64.encode())
|
||||
else:
|
||||
salt = _LEGACY_SALT
|
||||
return Fernet(_derive_key(MAIL_ENCRYPTION_KEY, salt))
|
||||
|
||||
|
||||
def encrypt_password(plaintext: str, salt_b64: str | None = None) -> str:
|
||||
"""Encrypt a password using AES-256 (Fernet). Returns base64 ciphertext.
|
||||
|
||||
If salt_b64 is provided, uses that salt for key derivation.
|
||||
If not, uses the legacy hardcoded salt (for backward compatibility).
|
||||
"""
|
||||
fernet = _get_fernet(salt_b64)
|
||||
return fernet.encrypt(plaintext.encode()).decode()
|
||||
|
||||
|
||||
def decrypt_password(ciphertext: str, salt_b64: str | None = None) -> str:
|
||||
"""Decrypt a password encrypted with encrypt_password.
|
||||
|
||||
If salt_b64 is provided, uses that salt for key derivation.
|
||||
If not, uses the legacy hardcoded salt (for backward compatibility).
|
||||
"""
|
||||
fernet = _get_fernet(salt_b64)
|
||||
return fernet.decrypt(ciphertext.encode()).decode()
|
||||
File diff suppressed because it is too large
Load Diff
@@ -4,7 +4,9 @@ from __future__ import annotations
|
||||
|
||||
import uuid
|
||||
from datetime import UTC, datetime
|
||||
from typing import Any
|
||||
|
||||
from pgvector.sqlalchemy import Vector
|
||||
from sqlalchemy import (
|
||||
JSON,
|
||||
Boolean,
|
||||
@@ -16,14 +18,12 @@ from sqlalchemy import (
|
||||
Text,
|
||||
UniqueConstraint,
|
||||
)
|
||||
from sqlalchemy.dialects.postgresql import TSVECTOR
|
||||
from sqlalchemy.dialects.postgresql import UUID as PGUUID
|
||||
from sqlalchemy.orm import Mapped, mapped_column
|
||||
|
||||
from app.core.db import Base, TenantMixin
|
||||
from app.models.owned_mixin import OwnedMixin
|
||||
from sqlalchemy.dialects.postgresql import TSVECTOR
|
||||
from pgvector.sqlalchemy import Vector
|
||||
from typing import Any
|
||||
|
||||
# --- Mail Accounts (F-MAIL-14, F-MAIL-18) ---
|
||||
|
||||
|
||||
@@ -0,0 +1,48 @@
|
||||
"""PGP encryption/decryption for the Mail plugin using pgpy.
|
||||
|
||||
Extracted from services.py as part of the God-object split (BUG-018 pilot).
|
||||
Re-exported by ``app.plugins.builtins.mail.services``.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import pgpy
|
||||
|
||||
|
||||
def import_pgp_private_key(private_key_armored: str, passphrase: str = "") -> tuple[str, str]:
|
||||
"""Import a PGP private key. Returns (key_id, public_key_armored)."""
|
||||
key, _ = pgpy.PGPKey.from_blob(private_key_armored)
|
||||
if key.is_protected:
|
||||
with key.unlock(passphrase):
|
||||
pub_key = key.pubkey
|
||||
key_id = str(key.fingerprint).upper()[-16:]
|
||||
return key_id, str(pub_key)
|
||||
pub_key = key.pubkey
|
||||
key_id = str(key.fingerprint).upper()[-16:]
|
||||
return key_id, str(pub_key)
|
||||
|
||||
|
||||
def import_pgp_public_key(public_key_armored: str) -> str:
|
||||
"""Import a PGP public key. Returns key_id."""
|
||||
key, _ = pgpy.PGPKey.from_blob(public_key_armored)
|
||||
return str(key.fingerprint).upper()[-16:]
|
||||
|
||||
|
||||
def pgp_encrypt_message(plaintext: str, recipient_public_key_armored: str) -> str:
|
||||
"""Encrypt a message with recipient's public PGP key."""
|
||||
pub_key, _ = pgpy.PGPKey.from_blob(recipient_public_key_armored)
|
||||
msg = pgpy.PGPMessage.new(plaintext)
|
||||
encrypted = pub_key.encrypt(msg)
|
||||
return str(encrypted)
|
||||
|
||||
|
||||
def pgp_decrypt_message(ciphertext: str, private_key_armored: str, passphrase: str = "") -> str:
|
||||
"""Decrypt a PGP-encrypted message."""
|
||||
key, _ = pgpy.PGPKey.from_blob(private_key_armored)
|
||||
enc_msg = pgpy.PGPMessage.from_blob(ciphertext)
|
||||
if key.is_protected:
|
||||
with key.unlock(passphrase):
|
||||
decrypted = key.decrypt(enc_msg)
|
||||
return decrypted.message.decode("utf-8")
|
||||
decrypted = key.decrypt(enc_msg)
|
||||
return decrypted.message.decode("utf-8")
|
||||
@@ -9,7 +9,6 @@ from typing import Any
|
||||
|
||||
from app.plugins.base import BasePlugin
|
||||
from app.plugins.manifest import (
|
||||
FrontendDetailTab,
|
||||
FrontendMenuItem,
|
||||
FrontendPageRoute,
|
||||
FrontendSettingsPage,
|
||||
@@ -115,7 +114,11 @@ async def _auto_sync_loop() -> None:
|
||||
class MailPlugin(BasePlugin):
|
||||
"""Mail plugin for email management: IMAP sync, SMTP send, threading, rules, PGP."""
|
||||
|
||||
_auto_sync_task: asyncio.Task | None = None
|
||||
def __init__(self) -> None:
|
||||
super().__init__()
|
||||
# Instance attribute: multiple plugin instances must not share the
|
||||
# background task state (ARCH-036).
|
||||
self._auto_sync_task: asyncio.Task | None = None
|
||||
|
||||
manifest = PluginManifest(
|
||||
name="mail",
|
||||
@@ -137,24 +140,30 @@ class MailPlugin(BasePlugin):
|
||||
migrations=["0001_initial.sql", "0006_flag_type.sql", "0007_sync_queue.sql", "0008_sync_queue_deleted_at.sql", "0009_remove_mail_soft_delete.sql", "0010_add_deleted_at.sql"],
|
||||
permissions=["mail:read", "mail:send", "mail:config", "mail:share", "mail:write", "mail:delete"],
|
||||
menu_items=[
|
||||
FrontendMenuItem(label_key='nav.mail', label='E-Mail', path='/mail', icon='Mail', order=30),
|
||||
FrontendMenuItem(label_key='nav.mail', label='E-Mail', path='/mail', icon='Mail', order=30, permission='mail:read'),
|
||||
],
|
||||
page_routes=[
|
||||
FrontendPageRoute(path='/mail', component='@/pages/Mail', protected=True),
|
||||
FrontendPageRoute(path='/mail/settings', component='@/pages/MailSettings', protected=True),
|
||||
FrontendPageRoute(path='/mail', component='@/pages/Mail', protected=True, permission='mail:read'),
|
||||
FrontendPageRoute(path='/mail/settings', component='@/pages/MailSettings', protected=True, permission='mail:config'),
|
||||
],
|
||||
settings_pages=[
|
||||
FrontendSettingsPage(path='mail', label_key='settings.mail', label='Mail', component='@/pages/MailSettings', icon='Mail', order=50),
|
||||
],
|
||||
detail_tabs=[
|
||||
FrontendDetailTab(entity_type='contact', label_key='tabs.email', label='E-Mails', component='@/components/contact/ContactMailTab', icon='Mail', order=20, permission='mail:read'),
|
||||
],
|
||||
# BUG (ghost component): ContactMailTab does not exist in the
|
||||
# frontend — tab removed until implemented (Block I-D).
|
||||
detail_tabs=[],
|
||||
author="LeoCRM Team",
|
||||
min_app_version="1.0.0",
|
||||
hooks=["mail.before_send", "mail.after_send"],
|
||||
contract_version="1.0.0",
|
||||
)
|
||||
|
||||
def get_entity_models(self) -> dict[str, type]:
|
||||
"""Entity types owned by Mail for the permission resolver."""
|
||||
from app.plugins.builtins.mail.models import MailAccount
|
||||
|
||||
return {"mail_account": MailAccount}
|
||||
|
||||
async def on_activate(
|
||||
self, db, service_container, event_bus
|
||||
) -> None:
|
||||
|
||||
@@ -56,6 +56,7 @@ from app.plugins.builtins.mail.schemas import (
|
||||
MailRuleCreate,
|
||||
MailSendRequest,
|
||||
MailSignatureCreate,
|
||||
MailSignatureUpdate,
|
||||
MailTemplateCreate,
|
||||
PgpKeyImport,
|
||||
SendPermissionCreate,
|
||||
@@ -892,7 +893,10 @@ async def list_threads(
|
||||
threads[tid]["mail_count"] += 1
|
||||
threads[tid]["mails"].append(mail_to_response(mail))
|
||||
items = list(threads.values())
|
||||
return {"items": items, "total": len(items)}
|
||||
# Plain array — consistent with sibling mail list routes (accounts,
|
||||
# folders, templates) and with the frontend client type
|
||||
# fetchThreads(): Promise<ThreadResult[]>.
|
||||
return items
|
||||
|
||||
|
||||
# ─── Templates (F-MAIL-06) ───
|
||||
@@ -992,6 +996,66 @@ async def list_signatures(
|
||||
return [signature_to_response(s) for s in sigs]
|
||||
|
||||
|
||||
@router.patch("/signatures/{signature_id}")
|
||||
async def update_signature(
|
||||
signature_id: str,
|
||||
data: MailSignatureUpdate,
|
||||
db: AsyncSession = Depends(get_db),
|
||||
current_user: dict = Depends(require_permission("mail:config")),
|
||||
):
|
||||
tenant_id = uuid.UUID(current_user["tenant_id"])
|
||||
user_id = uuid.UUID(current_user["user_id"])
|
||||
s_id = _parse_uuid(signature_id, "signature_id")
|
||||
sig = (
|
||||
await db.execute(
|
||||
select(MailSignature).where(
|
||||
and_(MailSignature.id == s_id, MailSignature.tenant_id == tenant_id)
|
||||
)
|
||||
)
|
||||
).scalar_one_or_none()
|
||||
if not sig:
|
||||
raise HTTPException(404, detail={"detail": "Signature not found", "code": "not_found"})
|
||||
if sig.user_id != user_id:
|
||||
raise HTTPException(403, detail={"detail": "Not your signature", "code": "forbidden"})
|
||||
sig.name = data.name
|
||||
sig.body_html = data.body_html
|
||||
if data.is_default:
|
||||
others = (
|
||||
await db.execute(
|
||||
select(MailSignature).where(MailSignature.user_id == user_id)
|
||||
)
|
||||
).scalars().all()
|
||||
for o in others:
|
||||
o.is_default = False
|
||||
sig.is_default = data.is_default
|
||||
await db.flush()
|
||||
return signature_to_response(sig)
|
||||
|
||||
|
||||
@router.delete("/signatures/{signature_id}", status_code=204)
|
||||
async def delete_signature(
|
||||
signature_id: str,
|
||||
db: AsyncSession = Depends(get_db),
|
||||
current_user: dict = Depends(require_permission("mail:config")),
|
||||
):
|
||||
tenant_id = uuid.UUID(current_user["tenant_id"])
|
||||
user_id = uuid.UUID(current_user["user_id"])
|
||||
s_id = _parse_uuid(signature_id, "signature_id")
|
||||
sig = (
|
||||
await db.execute(
|
||||
select(MailSignature).where(
|
||||
and_(MailSignature.id == s_id, MailSignature.tenant_id == tenant_id)
|
||||
)
|
||||
)
|
||||
).scalar_one_or_none()
|
||||
if not sig:
|
||||
raise HTTPException(404, detail={"detail": "Signature not found", "code": "not_found"})
|
||||
if sig.user_id != user_id:
|
||||
raise HTTPException(403, detail={"detail": "Not your signature", "code": "forbidden"})
|
||||
await db.delete(sig)
|
||||
await db.flush()
|
||||
|
||||
|
||||
# ─── Rules (F-MAIL-07) ───
|
||||
|
||||
|
||||
@@ -1206,6 +1270,28 @@ async def list_labels(
|
||||
return [label_to_response(lbl) for lbl in labels]
|
||||
|
||||
|
||||
@router.delete("/labels/{label_id}", status_code=204)
|
||||
async def delete_label(
|
||||
label_id: str,
|
||||
db: AsyncSession = Depends(get_db),
|
||||
current_user: dict = Depends(require_permission("mail:config")),
|
||||
):
|
||||
tenant_id = uuid.UUID(current_user["tenant_id"])
|
||||
user_id = uuid.UUID(current_user["user_id"])
|
||||
l_id = _parse_uuid(label_id, "label_id")
|
||||
label = (
|
||||
await db.execute(
|
||||
select(MailLabel).where(and_(MailLabel.id == l_id, MailLabel.tenant_id == tenant_id))
|
||||
)
|
||||
).scalar_one_or_none()
|
||||
if not label:
|
||||
raise HTTPException(404, detail={"detail": "Label not found", "code": "not_found"})
|
||||
if label.user_id != user_id:
|
||||
raise HTTPException(403, detail={"detail": "Not your label", "code": "forbidden"})
|
||||
await db.delete(label)
|
||||
await db.flush()
|
||||
|
||||
|
||||
# ─── Contact PGP Keys (F-MAIL-12) ───
|
||||
|
||||
|
||||
@@ -1494,8 +1580,8 @@ async def create_event_from_mail(
|
||||
try:
|
||||
from app.plugins.builtins.calendar.contracts import get_contract as get_calendar_contract
|
||||
_cal = get_calendar_contract()
|
||||
calendar = _cal.calendar
|
||||
calendar_entry = _cal.calendar_entry
|
||||
calendar = _cal.Calendar
|
||||
calendar_entry = _cal.CalendarEntry
|
||||
except ImportError:
|
||||
return {"created": False, "error": "calendar plugin not available"}
|
||||
cal_id = _parse_uuid(data.calendar_id, "calendar_id")
|
||||
@@ -1862,4 +1948,3 @@ async def get_mail(
|
||||
)
|
||||
return mail_to_response(mail, attachments=list(attachments), labels=list(label_assignments))
|
||||
|
||||
from app.core.audit import log_audit
|
||||
@@ -0,0 +1,62 @@
|
||||
"""HTML sanitization for the Mail plugin using nh3.
|
||||
|
||||
Extracted from services.py as part of the God-object split (BUG-018 pilot).
|
||||
Re-exported by ``app.plugins.builtins.mail.services``.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import nh3
|
||||
|
||||
|
||||
def sanitize_html(raw_html: str) -> str:
|
||||
"""Sanitize HTML using nh3 — removes script tags and dangerous attributes."""
|
||||
if not raw_html:
|
||||
return ""
|
||||
return nh3.clean(
|
||||
raw_html,
|
||||
tags={
|
||||
"a",
|
||||
"b",
|
||||
"br",
|
||||
"div",
|
||||
"em",
|
||||
"h1",
|
||||
"h2",
|
||||
"h3",
|
||||
"h4",
|
||||
"h5",
|
||||
"h6",
|
||||
"hr",
|
||||
"i",
|
||||
"img",
|
||||
"li",
|
||||
"ol",
|
||||
"p",
|
||||
"span",
|
||||
"strong",
|
||||
"table",
|
||||
"tbody",
|
||||
"td",
|
||||
"th",
|
||||
"thead",
|
||||
"tr",
|
||||
"u",
|
||||
"ul",
|
||||
"blockquote",
|
||||
"code",
|
||||
"pre",
|
||||
"font",
|
||||
"center",
|
||||
},
|
||||
attributes={
|
||||
"a": {"href", "title", "target"},
|
||||
"img": {"src", "alt", "width", "height"},
|
||||
"span": {"style"},
|
||||
"div": {"style"},
|
||||
"font": {"color", "size", "face"},
|
||||
"p": {"style"},
|
||||
"td": {"style"},
|
||||
"th": {"style"},
|
||||
},
|
||||
)
|
||||
@@ -259,6 +259,12 @@ class MailSignatureResponse(BaseModel):
|
||||
is_default: bool
|
||||
|
||||
|
||||
class MailSignatureUpdate(BaseModel):
|
||||
name: str = Field(..., min_length=1, max_length=255)
|
||||
body_html: str = ""
|
||||
is_default: bool = False
|
||||
|
||||
|
||||
# ─── Rules ───
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,169 @@
|
||||
"""Response serializers for the Mail plugin ORM models.
|
||||
|
||||
Extracted from services.py as part of the God-object split (BUG-018 pilot).
|
||||
Re-exported by ``app.plugins.builtins.mail.services``.
|
||||
|
||||
account_to_response NEVER includes passwords — security-critical contract.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
|
||||
from app.plugins.builtins.mail.models import (
|
||||
Mail,
|
||||
MailAccount,
|
||||
MailAttachment,
|
||||
MailFolder,
|
||||
MailLabel,
|
||||
MailRule,
|
||||
MailSignature,
|
||||
MailTemplate,
|
||||
)
|
||||
from app.plugins.builtins.mail.text_utils import (
|
||||
extract_email_addresses, # noqa: F401 — re-exported for backwards compat
|
||||
)
|
||||
|
||||
|
||||
def attachment_to_response(att: MailAttachment) -> dict:
|
||||
"""Convert a MailAttachment ORM object to a response dict."""
|
||||
return {
|
||||
"id": str(att.id),
|
||||
"mail_id": str(att.mail_id),
|
||||
"filename": att.filename,
|
||||
"mime_type": att.mime_type,
|
||||
"size_bytes": att.size_bytes,
|
||||
"size": att.size_bytes, # alias for frontend compatibility
|
||||
"content_id": att.content_id,
|
||||
"is_inline": bool(att.content_id),
|
||||
"dms_file_id": str(att.dms_file_id) if att.dms_file_id else None,
|
||||
}
|
||||
|
||||
|
||||
def account_to_response(account: MailAccount) -> dict:
|
||||
"""Convert MailAccount to response dict, NEVER including password."""
|
||||
return {
|
||||
"id": str(account.id),
|
||||
"email": account.email_address,
|
||||
"email_address": account.email_address,
|
||||
"display_name": account.display_name,
|
||||
"imap_host": account.imap_host,
|
||||
"imap_port": account.imap_port,
|
||||
"imap_ssl": account.imap_ssl,
|
||||
"smtp_host": account.smtp_host,
|
||||
"smtp_port": account.smtp_port,
|
||||
"smtp_tls": account.smtp_tls,
|
||||
"username": account.username,
|
||||
"is_shared": account.is_shared,
|
||||
"is_active": account.is_active,
|
||||
"sent_folder_imap_name": account.sent_folder_imap_name,
|
||||
"drafts_folder_imap_name": account.drafts_folder_imap_name,
|
||||
"spam_folder_imap_name": account.spam_folder_imap_name,
|
||||
"trash_folder_imap_name": account.trash_folder_imap_name,
|
||||
"created_at": account.created_at,
|
||||
"updated_at": account.updated_at,
|
||||
}
|
||||
|
||||
|
||||
def mail_to_response(
|
||||
mail: Mail,
|
||||
attachments: list[MailAttachment] | None = None,
|
||||
labels: list[MailLabel] | None = None,
|
||||
) -> dict:
|
||||
"""Convert a Mail ORM object to a response dict."""
|
||||
resp = {
|
||||
"id": str(mail.id),
|
||||
"account_id": str(mail.account_id),
|
||||
"folder_id": str(mail.folder_id),
|
||||
"message_id": mail.message_id,
|
||||
"thread_id": mail.thread_id,
|
||||
"in_reply_to": mail.in_reply_to,
|
||||
"subject": mail.subject,
|
||||
"from_address": mail.from_address,
|
||||
"from_name": mail.from_address.split("<")[0].strip().strip('"') if "<" in mail.from_address else mail.from_address,
|
||||
"to_addresses": extract_email_addresses(mail.to_addresses) if mail.to_addresses else [],
|
||||
"cc_addresses": extract_email_addresses(mail.cc_addresses) if mail.cc_addresses else [],
|
||||
"bcc_addresses": extract_email_addresses(mail.bcc_addresses) if mail.bcc_addresses else [],
|
||||
"body_text": mail.body_text,
|
||||
"body_html": mail.body_html if mail.body_html else None,
|
||||
"body_html_sanitized": mail.body_html_sanitized,
|
||||
"sanitized_html": mail.body_html_sanitized,
|
||||
"date": mail.received_at.isoformat() if mail.received_at else (mail.sent_at.isoformat() if mail.sent_at else None),
|
||||
"is_seen": mail.is_seen,
|
||||
"is_flagged": mail.is_flagged,
|
||||
"flag_type": getattr(mail, 'flag_type', None),
|
||||
"is_draft": mail.is_draft,
|
||||
"is_answered": mail.is_answered,
|
||||
"is_forwarded": mail.is_forwarded,
|
||||
"has_attachments": mail.has_attachments,
|
||||
"size_bytes": mail.size_bytes,
|
||||
"received_at": mail.received_at,
|
||||
"sent_at": mail.sent_at,
|
||||
"contact_id": str(mail.contact_id) if mail.contact_id else None,
|
||||
"attachments": [],
|
||||
"labels": [],
|
||||
}
|
||||
if attachments:
|
||||
resp["attachments"] = [attachment_to_response(a) for a in attachments]
|
||||
if labels:
|
||||
resp["labels"] = [
|
||||
{"id": str(lbl.id), "name": lbl.name, "color": lbl.color} for lbl in labels
|
||||
]
|
||||
return resp
|
||||
|
||||
|
||||
def folder_to_response(folder: MailFolder) -> dict:
|
||||
"""Convert MailFolder to response dict."""
|
||||
return {
|
||||
"id": str(folder.id),
|
||||
"account_id": str(folder.account_id),
|
||||
"name": folder.name,
|
||||
"imap_name": folder.imap_name,
|
||||
"parent_id": str(folder.parent_id) if folder.parent_id else None,
|
||||
"is_standard": folder.is_standard,
|
||||
"unread_count": folder.unread_count,
|
||||
"total_count": folder.total_count,
|
||||
}
|
||||
|
||||
|
||||
def rule_to_response(rule: MailRule) -> dict:
|
||||
"""Convert MailRule to response dict."""
|
||||
return {
|
||||
"id": str(rule.id),
|
||||
"name": rule.name,
|
||||
"account_id": str(rule.account_id) if rule.account_id else None,
|
||||
"priority": rule.priority,
|
||||
"is_active": rule.is_active,
|
||||
"conditions": json.loads(rule.conditions) if rule.conditions else {},
|
||||
"actions": json.loads(rule.actions) if rule.actions else {},
|
||||
}
|
||||
|
||||
|
||||
def template_to_response(template: MailTemplate) -> dict:
|
||||
"""Convert MailTemplate to response dict."""
|
||||
return {
|
||||
"id": str(template.id),
|
||||
"name": template.name,
|
||||
"subject": template.subject,
|
||||
"body_html": template.body_html,
|
||||
}
|
||||
|
||||
|
||||
def signature_to_response(sig: MailSignature) -> dict:
|
||||
"""Convert MailSignature to response dict."""
|
||||
return {
|
||||
"id": str(sig.id),
|
||||
"name": sig.name,
|
||||
"body_html": sig.body_html,
|
||||
"account_id": str(sig.account_id) if sig.account_id else None,
|
||||
"is_default": sig.is_default,
|
||||
}
|
||||
|
||||
|
||||
def label_to_response(label: MailLabel) -> dict:
|
||||
"""Convert MailLabel to response dict."""
|
||||
return {
|
||||
"id": str(label.id),
|
||||
"name": label.name,
|
||||
"color": label.color,
|
||||
}
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,358 @@
|
||||
"""SMTP send service for the Mail plugin (F-MAIL-02).
|
||||
|
||||
Extracted from services.py as part of the God-object split (BUG-018 pilot).
|
||||
Re-exported by ``app.plugins.builtins.mail.services``.
|
||||
|
||||
Contains send_mail_via_smtp, reply_to_mail and forward_mail.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import os
|
||||
import uuid
|
||||
from datetime import UTC, datetime
|
||||
from email.message import EmailMessage
|
||||
from email.utils import formataddr, formatdate, make_msgid
|
||||
|
||||
import aiofiles
|
||||
import aioimaplib
|
||||
import aiosmtplib
|
||||
from sqlalchemy import and_, select
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
|
||||
from app.core.notifications import create_notification
|
||||
from app.plugins.builtins.mail.imap_sync import (
|
||||
_compute_thread_id,
|
||||
get_account_password,
|
||||
)
|
||||
from app.plugins.builtins.mail.models import (
|
||||
Mail,
|
||||
MailAccount,
|
||||
MailFolder,
|
||||
MailSignature,
|
||||
)
|
||||
from app.plugins.builtins.mail.sanitize import sanitize_html
|
||||
from app.plugins.builtins.mail.text_utils import _strip_html
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
async def send_mail_via_smtp(
|
||||
db: AsyncSession,
|
||||
*,
|
||||
tenant_id: uuid.UUID,
|
||||
user_id: uuid.UUID,
|
||||
account: MailAccount,
|
||||
to_addrs: list[str],
|
||||
cc_addrs: list[str] = None,
|
||||
bcc_addrs: list[str] = None,
|
||||
subject: str = "",
|
||||
body_html: str = "",
|
||||
body_text: str = "",
|
||||
in_reply_to: str | None = None,
|
||||
references_header: str | None = None,
|
||||
signature: MailSignature | None = None,
|
||||
attachment_paths: list[dict] | None = None,
|
||||
) -> dict:
|
||||
"""Send an email via SMTP using aiosmtplib.
|
||||
|
||||
Args:
|
||||
attachment_paths: list of dicts with keys 'path', 'filename', 'mime_type'
|
||||
pointing to files on disk to attach.
|
||||
"""
|
||||
cc_addrs = cc_addrs or []
|
||||
bcc_addrs = bcc_addrs or []
|
||||
attachment_paths = attachment_paths or []
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# Apply signature if provided
|
||||
if signature and signature.body_html:
|
||||
body_html = body_html + f"<br><br>{signature.body_html}"
|
||||
if body_text:
|
||||
body_text = body_text + "\n\n-- \n" + _strip_html(signature.body_html)
|
||||
|
||||
# Build email message
|
||||
msg = EmailMessage()
|
||||
msg["From"] = formataddr((account.display_name or "", account.email_address))
|
||||
msg["To"] = ", ".join(to_addrs)
|
||||
if cc_addrs:
|
||||
msg["Cc"] = ", ".join(cc_addrs)
|
||||
msg["Subject"] = subject
|
||||
msg["Date"] = formatdate(localtime=True)
|
||||
msg_id = make_msgid(
|
||||
domain=account.email_address.split("@")[-1] if "@" in account.email_address else "localhost"
|
||||
)
|
||||
msg["Message-ID"] = msg_id
|
||||
if in_reply_to:
|
||||
msg["In-Reply-To"] = in_reply_to
|
||||
if references_header:
|
||||
msg["References"] = references_header
|
||||
|
||||
if body_html:
|
||||
msg.set_content(body_text or _strip_html(body_html), subtype="plain")
|
||||
msg.add_alternative(body_html, subtype="html")
|
||||
else:
|
||||
msg.set_content(body_text, subtype="plain")
|
||||
|
||||
# Add attachments to the message
|
||||
for att_info in attachment_paths:
|
||||
file_path = att_info.get("path", "")
|
||||
filename = att_info.get("filename", os.path.basename(file_path))
|
||||
mime_type = att_info.get("mime_type", "application/octet-stream")
|
||||
if not file_path or not os.path.exists(file_path): # noqa: ASYNC240
|
||||
continue
|
||||
async with aiofiles.open(file_path, "rb") as f:
|
||||
content = await f.read()
|
||||
# Determine maintype/subtype from mime_type
|
||||
if "/" in mime_type:
|
||||
maintype, subtype = mime_type.split("/", 1)
|
||||
else:
|
||||
maintype, subtype = "application", "octet-stream"
|
||||
msg.add_attachment(
|
||||
content,
|
||||
maintype=maintype,
|
||||
subtype=subtype,
|
||||
filename=filename,
|
||||
)
|
||||
|
||||
# ── Hook: mail.before_send (Filter) ──
|
||||
from app.core.hooks import apply_filters
|
||||
mail_data = {
|
||||
"subject": subject,
|
||||
"body_html": body_html,
|
||||
"body_text": body_text,
|
||||
"to_addrs": to_addrs,
|
||||
"cc_addrs": cc_addrs,
|
||||
"bcc_addrs": bcc_addrs,
|
||||
"attachment_paths": attachment_paths,
|
||||
}
|
||||
mail_data = await apply_filters("mail.before_send", mail_data)
|
||||
|
||||
# Send via SMTP
|
||||
password = await get_account_password(account)
|
||||
try:
|
||||
smtp = aiosmtplib.SMTP(
|
||||
hostname=account.smtp_host,
|
||||
port=account.smtp_port,
|
||||
use_tls=account.smtp_tls,
|
||||
)
|
||||
await smtp.connect()
|
||||
await smtp.login(account.username, password)
|
||||
recipients = to_addrs + cc_addrs + bcc_addrs
|
||||
await smtp.send_message(msg, recipients=recipients)
|
||||
await smtp.quit()
|
||||
|
||||
# ── Hook: mail.after_send (Action) ──
|
||||
from app.core.hooks import do_action
|
||||
await do_action("mail.after_send", mail_data, db=db, account=account, msg_id=msg_id)
|
||||
await do_action("mail.after_update", mail_data, db=db, tenant_id=str(mail_data.get('tenant_id', '')) if isinstance(mail_data, dict) else None)
|
||||
|
||||
# Store sent mail in Sent folder — use configured mapping if set,
|
||||
# otherwise flexible lookup to handle different IMAP naming conventions
|
||||
if account.sent_folder_imap_name:
|
||||
sent_folder = (
|
||||
await db.execute(
|
||||
select(MailFolder).where(
|
||||
and_(
|
||||
MailFolder.account_id == account.id,
|
||||
MailFolder.imap_name == account.sent_folder_imap_name,
|
||||
)
|
||||
)
|
||||
)
|
||||
).scalar_one_or_none()
|
||||
else:
|
||||
sent_folder = (
|
||||
await db.execute(
|
||||
select(MailFolder).where(
|
||||
and_(
|
||||
MailFolder.account_id == account.id,
|
||||
MailFolder.imap_name.in_(
|
||||
["Sent", "INBOX.Sent", "Sent Items", "Sent Mail"]
|
||||
),
|
||||
)
|
||||
)
|
||||
)
|
||||
).scalar_one_or_none()
|
||||
|
||||
if not sent_folder:
|
||||
sent_folder = (
|
||||
await db.execute(
|
||||
select(MailFolder).where(
|
||||
and_(
|
||||
MailFolder.account_id == account.id,
|
||||
MailFolder.is_standard.is_(True),
|
||||
MailFolder.imap_name.ilike("%sent%"),
|
||||
)
|
||||
)
|
||||
)
|
||||
).scalar_one_or_none()
|
||||
|
||||
if sent_folder:
|
||||
thread_id = _compute_thread_id(msg_id, references_header or "", in_reply_to)
|
||||
sent_mail = Mail(
|
||||
tenant_id=tenant_id,
|
||||
account_id=account.id,
|
||||
folder_id=sent_folder.id,
|
||||
message_id=msg_id,
|
||||
thread_id=thread_id,
|
||||
in_reply_to=in_reply_to,
|
||||
references_header=references_header,
|
||||
subject=subject,
|
||||
from_address=account.email_address,
|
||||
to_addresses=", ".join(to_addrs),
|
||||
cc_addresses=", ".join(cc_addrs),
|
||||
bcc_addresses=", ".join(bcc_addrs),
|
||||
body_text=body_text or _strip_html(body_html),
|
||||
body_html=body_html,
|
||||
body_html_sanitized=sanitize_html(body_html),
|
||||
is_seen=True,
|
||||
is_answered=bool(in_reply_to),
|
||||
sent_at=datetime.now(UTC),
|
||||
)
|
||||
db.add(sent_mail)
|
||||
await db.flush()
|
||||
|
||||
# Upload sent mail to IMAP Sent folder via APPEND
|
||||
try:
|
||||
imap_client = aioimaplib.IMAP4_SSL(host=account.imap_host, port=account.imap_port)
|
||||
await imap_client.wait_hello_from_server()
|
||||
await imap_client.login(account.username, password)
|
||||
# Build raw email bytes for APPEND
|
||||
raw_email_bytes = msg.as_bytes()
|
||||
append_resp = await imap_client.append(
|
||||
sent_folder.imap_name,
|
||||
r'(\Seen)',
|
||||
None,
|
||||
raw_email_bytes,
|
||||
)
|
||||
if append_resp.result == 'OK':
|
||||
logger.info("send_mail: uploaded sent mail to IMAP folder %s", sent_folder.imap_name)
|
||||
# Retrieve the new UID for the appended mail by searching for its Message-ID
|
||||
try:
|
||||
await imap_client.select(sent_folder.imap_name)
|
||||
search_resp = await imap_client.uid_search(f'HEADER Message-ID "{msg_id}"')
|
||||
uid_raw = search_resp[1][0] if search_resp[1] and search_resp[1][0] else b''
|
||||
if isinstance(uid_raw, (bytes, bytearray)) and uid_raw:
|
||||
new_uids = uid_raw.decode().split()
|
||||
if new_uids:
|
||||
sent_mail.imap_uid = new_uids[0]
|
||||
logger.info("send_mail: set imap_uid=%s for sent mail %s", new_uids[0], sent_mail.id)
|
||||
except Exception as uid_exc:
|
||||
logger.warning("send_mail: could not retrieve UID after APPEND: %s", uid_exc)
|
||||
else:
|
||||
logger.warning("send_mail: IMAP APPEND failed for sent folder %s: %s", sent_folder.imap_name, append_resp)
|
||||
await imap_client.logout()
|
||||
except Exception as imap_exc:
|
||||
logger.warning("send_mail: IMAP APPEND failed (non-critical): %s", imap_exc)
|
||||
|
||||
# ── Notification: mail sent ──
|
||||
try:
|
||||
await create_notification(
|
||||
db, tenant_id, user_id,
|
||||
"mail_sent",
|
||||
"E-Mail gesendet",
|
||||
subject,
|
||||
)
|
||||
await db.flush()
|
||||
except Exception:
|
||||
logger.debug("Ignored exception in mail service", exc_info=True)
|
||||
|
||||
return {"status": "sent", "message_id": msg_id}
|
||||
except Exception as e:
|
||||
# ── Notification: send error ──
|
||||
try:
|
||||
await create_notification(
|
||||
db, tenant_id, user_id,
|
||||
"mail_send_error",
|
||||
"E-Mail konnte nicht gesendet werden",
|
||||
str(e),
|
||||
)
|
||||
await db.flush()
|
||||
except Exception:
|
||||
logger.debug("Ignored exception in mail service", exc_info=True)
|
||||
return {"status": "error", "error": str(e)}
|
||||
|
||||
|
||||
async def reply_to_mail(
|
||||
db: AsyncSession,
|
||||
*,
|
||||
tenant_id: uuid.UUID,
|
||||
user_id: uuid.UUID,
|
||||
original_mail: Mail,
|
||||
account: MailAccount,
|
||||
body_html: str,
|
||||
body_text: str = "",
|
||||
reply_to_all: bool = False,
|
||||
signature: MailSignature | None = None,
|
||||
) -> dict:
|
||||
"""Reply to a mail, setting In-Reply-To and References headers (F-MAIL-02)."""
|
||||
to_addrs = [original_mail.from_address]
|
||||
if reply_to_all and original_mail.cc_addresses:
|
||||
to_addrs.extend([a.strip() for a in original_mail.cc_addresses.split(",") if a.strip()])
|
||||
|
||||
refs = original_mail.references_header or ""
|
||||
new_refs = f"{refs} {original_mail.message_id}".strip()
|
||||
|
||||
result = await send_mail_via_smtp(
|
||||
db,
|
||||
tenant_id=tenant_id,
|
||||
user_id=user_id,
|
||||
account=account,
|
||||
to_addrs=to_addrs,
|
||||
subject=f"Re: {original_mail.subject}".replace("Re: Re: ", "Re: "),
|
||||
body_html=body_html,
|
||||
body_text=body_text,
|
||||
in_reply_to=original_mail.message_id,
|
||||
references_header=new_refs,
|
||||
signature=signature,
|
||||
)
|
||||
|
||||
# Mark original as answered
|
||||
original_mail.is_answered = True
|
||||
await db.flush()
|
||||
return result
|
||||
|
||||
|
||||
async def forward_mail(
|
||||
db: AsyncSession,
|
||||
*,
|
||||
tenant_id: uuid.UUID,
|
||||
user_id: uuid.UUID,
|
||||
original_mail: Mail,
|
||||
account: MailAccount,
|
||||
to_addrs: list[str],
|
||||
cc_addrs: list[str] = None,
|
||||
body_html: str = "",
|
||||
body_text: str = "",
|
||||
signature: MailSignature | None = None,
|
||||
) -> dict:
|
||||
"""Forward a mail with original as forwarded content (F-MAIL-02)."""
|
||||
fwd_subject = f"Fwd: {original_mail.subject}".replace("Fwd: Fwd: ", "Fwd: ")
|
||||
fwd_body = (
|
||||
f"<br><br>----- Original Message -----<br>"
|
||||
f"From: {original_mail.from_address}<br>"
|
||||
f"Subject: {original_mail.subject}<br><br>"
|
||||
f"{original_mail.body_html or original_mail.body_text}"
|
||||
)
|
||||
full_html = body_html + fwd_body
|
||||
full_text = (body_text or _strip_html(body_html)) + "\n\n----- Original Message -----\n"
|
||||
|
||||
result = await send_mail_via_smtp(
|
||||
db,
|
||||
tenant_id=tenant_id,
|
||||
user_id=user_id,
|
||||
account=account,
|
||||
to_addrs=to_addrs,
|
||||
cc_addrs=cc_addrs or [],
|
||||
subject=fwd_subject,
|
||||
body_html=full_html,
|
||||
body_text=full_text,
|
||||
signature=signature,
|
||||
)
|
||||
|
||||
original_mail.is_forwarded = True
|
||||
await db.flush()
|
||||
return result
|
||||
|
||||
|
||||
@@ -0,0 +1,35 @@
|
||||
"""Text utilities for the Mail plugin.
|
||||
|
||||
Extracted from services.py as part of the God-object split (BUG-018 pilot).
|
||||
Re-exported by ``app.plugins.builtins.mail.services``.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
|
||||
|
||||
def extract_email_addresses(text: str) -> list[str]:
|
||||
"""Extract email addresses from a text string."""
|
||||
if not text:
|
||||
return []
|
||||
return re.findall(r"[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}", text)
|
||||
|
||||
|
||||
def _strip_html(html: str) -> str:
|
||||
"""Simple HTML to text conversion for plain text fallback."""
|
||||
if not html:
|
||||
return ""
|
||||
# Remove tags
|
||||
text = re.sub(r"<[^>]+>", "", html)
|
||||
# Replace HTML entities
|
||||
text = (
|
||||
text.replace(" ", " ")
|
||||
.replace("&", "&")
|
||||
.replace("<", "<")
|
||||
.replace(">", ">")
|
||||
.replace(""", '"')
|
||||
.replace("'", "'")
|
||||
)
|
||||
# Collapse whitespace
|
||||
return re.sub(r"\s+", " ", text).strip()
|
||||
@@ -4,7 +4,7 @@ from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import uuid
|
||||
from datetime import datetime
|
||||
from datetime import UTC, datetime
|
||||
from typing import Any
|
||||
|
||||
from fastapi import APIRouter, Depends, HTTPException
|
||||
@@ -170,7 +170,7 @@ async def list_server_tools(
|
||||
tools_resp = await client.list_tools()
|
||||
# Update last_connected_at
|
||||
await db.execute(
|
||||
update(McpServerConfigModel).where(McpServerConfigModel.id == sid).values(last_connected_at=datetime.utcnow())
|
||||
update(McpServerConfigModel).where(McpServerConfigModel.id == sid).values(last_connected_at=datetime.now(UTC))
|
||||
)
|
||||
await db.commit()
|
||||
return tools_resp
|
||||
|
||||
@@ -14,7 +14,7 @@ class McpServerPlugin(BasePlugin):
|
||||
version="1.0.0",
|
||||
display_name="MCP Server",
|
||||
description="Exposes LeoCRM data (contacts, calendar, mail, DMS) to external MCP clients via API-token auth.",
|
||||
dependencies=["permissions"],
|
||||
dependencies=["permissions", "unified_search"],
|
||||
routes=[
|
||||
PluginRouteDef(
|
||||
path="/api/v1/mcp",
|
||||
|
||||
@@ -0,0 +1,9 @@
|
||||
"""Minimal Example plugin package.
|
||||
|
||||
Re-export the plugin class so ``discover_builtins`` finds it when scanning
|
||||
the builtins package namespace (see guide chapter 29.1).
|
||||
"""
|
||||
|
||||
from app.plugins.builtins.minimal_example.plugin import MinimalExamplePlugin
|
||||
|
||||
__all__ = ["MinimalExamplePlugin"]
|
||||
@@ -0,0 +1,22 @@
|
||||
from app.plugins.base import BasePlugin
|
||||
from app.plugins.manifest import PluginManifest, PluginRouteDef
|
||||
|
||||
|
||||
class MinimalExamplePlugin(BasePlugin):
|
||||
manifest = PluginManifest(
|
||||
name="minimal_example",
|
||||
version="1.0.0",
|
||||
display_name="Minimal Example",
|
||||
description="A minimal example plugin.",
|
||||
dependencies=[],
|
||||
routes=[
|
||||
PluginRouteDef(
|
||||
path="/api/v1/minimal-example",
|
||||
module="app.plugins.builtins.minimal_example.routes",
|
||||
router_attr="router",
|
||||
),
|
||||
],
|
||||
events=[],
|
||||
migrations=[],
|
||||
permissions=["minimal_example:read"],
|
||||
)
|
||||
@@ -0,0 +1,15 @@
|
||||
from fastapi import APIRouter, Depends
|
||||
|
||||
from app.deps import get_current_user, require_permission
|
||||
|
||||
# NOTE: main.py mounts plugin routers WITHOUT a prefix — the manifest's
|
||||
# route path is documentation only. The full path must live here.
|
||||
router = APIRouter()
|
||||
|
||||
|
||||
@router.get(
|
||||
"/api/v1/minimal-example",
|
||||
dependencies=[Depends(require_permission("minimal_example:read"))],
|
||||
)
|
||||
async def list_items(current_user: dict = Depends(get_current_user)):
|
||||
return {"items": []}
|
||||
@@ -0,0 +1,5 @@
|
||||
-- Dual-path convergence (Gate B): add the folder_id column that Alembic
|
||||
-- migration 0139 adds on the core path. Idempotent so both install paths
|
||||
-- converge to the identical schema.
|
||||
ALTER TABLE report_templates ADD COLUMN IF NOT EXISTS folder_id UUID;
|
||||
CREATE INDEX IF NOT EXISTS ix_report_templates_folder ON report_templates(folder_id);
|
||||
@@ -24,13 +24,13 @@ class ReportGeneratorPlugin(BasePlugin):
|
||||
),
|
||||
],
|
||||
events=["report.requested", "report.generated"],
|
||||
migrations=["0001_initial.sql"],
|
||||
migrations=["0001_initial.sql", "0002_reports_folder_id.sql"],
|
||||
permissions=["reports:read", "reports:generate", "reports:manage_templates"],
|
||||
menu_items=[
|
||||
FrontendMenuItem(label_key='nav.reports', label='Berichte', path='/reports', icon='BarChart3', order=70),
|
||||
FrontendMenuItem(label_key='nav.reports', label='Berichte', path='/reports', icon='BarChart3', order=70, permission='reports:read'),
|
||||
],
|
||||
page_routes=[
|
||||
FrontendPageRoute(path='/reports', component='@/pages/Reports', protected=True),
|
||||
FrontendPageRoute(path='/reports', component='@/pages/Reports', protected=True, permission='reports:read'),
|
||||
],
|
||||
|
||||
author="LeoCRM Team",
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user