Compare commits
102 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 7f6b52b8d0 | |||
| 84061fd8d5 | |||
| e0255412ac | |||
| 4cf7a91416 | |||
| f445aa69d5 | |||
| 4fee01cadf | |||
| ea6c9e71db | |||
| c34715574a | |||
| fce17aac9c | |||
| cbe36e0c0e | |||
| 6702d69f7c | |||
| 94d8c40daa | |||
| be81fe52cf | |||
| a1d5e56009 | |||
| 3e43219b84 | |||
| 26b5ae9a0d | |||
| 11e4e42570 | |||
| 57441df677 | |||
| fbe1bde635 | |||
| 2d17746194 | |||
| 23a05593b2 | |||
| 0baec2792c | |||
| f4a5937a4b | |||
| 38b73f5d4d | |||
| a6bfa8e67c | |||
| a8916b3d86 | |||
| e7afbaa906 | |||
| 34c9c85aed | |||
| 9d2df61942 | |||
| 1b485d4a34 | |||
| f4c4a50ebd | |||
| 69d05d6912 | |||
| df9f86bd12 | |||
| 9e1d202610 | |||
| c291a6ecf1 | |||
| ab3c253cbd | |||
| 52323610e3 | |||
| 86c96f03ca | |||
| 3e5f13f516 | |||
| 4de629d296 | |||
| 6a88c70073 | |||
| c807aacfc0 | |||
| b23045c46a | |||
| 5d8c48a08f | |||
| f6dde68221 | |||
| d901d001c7 | |||
| 962e0ee1f6 | |||
| 49ca4c5fb2 | |||
| 1b22da8b0d | |||
| 76a31a8c39 | |||
| a991f9a0b4 | |||
| 84a30d85c2 | |||
| d9aed519f2 | |||
| b9a6c06e85 | |||
| 8386e99caa | |||
| 7d9ae03bf1 | |||
| 36a03b9897 | |||
| 860db8d61e | |||
| 81aea8c77f | |||
| 46c909c226 | |||
| 197b0d3bab | |||
| 3934aea6ef | |||
| 5cc5a3fa6a | |||
| c0e8e4ecfd | |||
| c32e4bb34e | |||
| ef90d57f0a | |||
| 0768cfb29a | |||
| 56e401969e | |||
| 6d04206695 | |||
| f6e117b1c3 | |||
| 9d8da99026 | |||
| 54066b05fd | |||
| 36636f5c25 | |||
| d89044d8f7 | |||
| 5e0ffd91c2 | |||
| b8b8ef180a | |||
| cad7d084e8 | |||
| dff97f5589 | |||
| 9e84c400ed | |||
| 067fc132cb | |||
| b01b756a4a | |||
| 4bce89aecb | |||
| 5e9be254e2 | |||
| 8a76bfdba4 | |||
| d2434203c1 | |||
| ad7c763e59 | |||
| e3fb4728d7 | |||
| 7467c01d38 | |||
| 4038b74025 | |||
| 5ad107ff83 | |||
| 5cee78c54c | |||
| 32f63adc09 | |||
| c21634b323 | |||
| 73d2e109cd | |||
| 795307754f | |||
| 17516d2783 | |||
| ed8ee5cda1 | |||
| 90a367089d | |||
| b04cda774b | |||
| 982b4c9353 | |||
| 1d6152fb82 | |||
| 337d78ef53 |
+2
-2
@@ -39,8 +39,8 @@ RUN apt-get update \
|
||||
|
||||
WORKDIR /app
|
||||
|
||||
COPY requirements.txt .
|
||||
RUN pip install --user --no-cache-dir -r requirements.txt
|
||||
COPY requirements.txt requirements.lock ./
|
||||
RUN pip install --user --no-cache-dir -r requirements.lock
|
||||
|
||||
# === Stage 2: Runtime ===
|
||||
FROM python:3.12-slim AS runtime
|
||||
|
||||
@@ -1210,6 +1210,370 @@ Trigger / Event / Cron / Webhook / Agent
|
||||
|
||||
---
|
||||
|
||||
## Phase L — UI-Overhaul (Status: geplant, NICHT gestartet)
|
||||
|
||||
> **Herkunft:** Am 2026-08-25 aus der eigenständigen Datei `UI_OVERHAUL_PLAN.md`
|
||||
> hier integriert - gemaess AGENTS.md-Regel "PLATFORM_ROADMAP.md ist EINZIGE
|
||||
> Planungs-Datei". Vollständiges Original inkl. ASCII-Mockups abrufbar via
|
||||
> `git show c807aac:UI_OVERHAUL_PLAN.md`.
|
||||
>
|
||||
> **Konflikt-Notiz (2026-08-25, Block I-D):** Phase 2 unten sieht "AI Assistant
|
||||
> Page entfernen" vor. Die Seite wurde jedoch in Commit 962e0ee bewusst GEBAUT,
|
||||
> um die Geister-Route /ai-assistant zu reparieren (im Backend-Manifest
|
||||
> referenziert, aber nicht existent -> ErrorBoundary in Production). VOR
|
||||
> Umsetzung von Phase 2 neu entscheiden: (a) Seite doch entfernen - dann auch
|
||||
> Manifest-Route entfernen, oder (b) Phase 2 verwerfen zugunsten der aktuellen
|
||||
> Architektur. Bitte nicht unkommentiert ausfuehren.
|
||||
|
||||
> **Erstellt:** 2026-08-21
|
||||
> **Aktualisiert:** 2026-08-21 — AI Assistent Integration hinzugefügt
|
||||
> **Status:** Planung — nicht gestartet
|
||||
> **Leitlinie:** Auf bestehendem Code aufbauen, 3-Spalten-Explorer-Layout als Standard, keine parallelen Systeme
|
||||
|
||||
---
|
||||
|
||||
### Standard-Layout (Referenz: ContactsList.tsx)
|
||||
|
||||
Alle Explorer-Plugins nutzen das 3-Spalten-Layout aus den UI-Design-Guidelines:
|
||||
|
||||
```
|
||||
┌─────────────┬──────────────────┬──────────────────────┐
|
||||
│ Tree │ Liste/Ansicht │ Detail │
|
||||
│ (224px) │ (flex-1) │ (flex-1 / 60%) │
|
||||
│ ResizablePanel│ ResizablePanel │ ResizablePanel │
|
||||
└─────────────┴──────────────────┴──────────────────────┘
|
||||
```
|
||||
|
||||
- **Toolbar oben:** PluginToolbar mit Filter-Dropdowns, Ansichts-Umschaltern, Aktion-Buttons
|
||||
- **Linke Spalte:** ResizablePanel mit Baumansicht (Ordner, Kategorien, Kalender)
|
||||
- **Mitte:** Liste, Karten, Kalender-Ansicht — mehrere Ansichten umschaltbar
|
||||
- **Rechts:** Detail-Bereich für ausgewähltes Element
|
||||
|
||||
---
|
||||
|
||||
### Phase 1: Echte Bugs fixen (2-3 Tage)
|
||||
|
||||
#### 1.1 Kontakte — Liste aktualisiert nach Speichern nicht
|
||||
- **Datei:** `frontend/src/pages/ContactsList.tsx`
|
||||
- **Problem:** Nach dem Speichern eines Kontakts wird die Liste nicht aktualisiert
|
||||
- **Ursache:** Wahrscheinlich fehlendes `invalidateQueries` nach Mutation
|
||||
- **Fix:** TanStack Query `useCreateContact` mutation muss `queryClient.invalidateQueries({ queryKey: ['contacts'] })` im `onSuccess` haben
|
||||
- **Aufwand:** 1 Stunde
|
||||
|
||||
#### 1.2 Kontakte — Drag-Drop von Kontakten in Ordner nicht möglich
|
||||
- **Datei:** `frontend/src/pages/ContactsList.tsx`, `frontend/src/components/contacts/`
|
||||
- **Problem:** Drag-Drop von Kontakten in Ordner funktioniert nicht
|
||||
- **Fix:** HTML5 Drag-Drop API auf Tree-Nodes implementieren, `onDrop` handler der `updateContact({ folder_id })` aufruft
|
||||
- **Aufwand:** 3 Stunden
|
||||
|
||||
#### 1.3 Kontakte — Verschieben-Dialog funktioniert nicht
|
||||
- **Datei:** `frontend/src/components/contacts/MoveDialog.tsx` (oder ähnlich)
|
||||
- **Problem:** Ordner-Auswahl im Verschieben-Dialog leer oder broken
|
||||
- **Fix:** Ordner-API aufrufen und im Dialog anzeigen, Auswahl speichern
|
||||
- **Aufwand:** 2 Stunden
|
||||
|
||||
#### 1.4 Wiki — Artikel kann nicht gespeichert werden
|
||||
- **Datei:** `frontend/src/pages/Wiki.tsx`, `frontend/src/api/knowledge.ts`
|
||||
- **Problem:** Speichern-Button funktioniert nicht oder API gibt Fehler zurück
|
||||
- **Diagnose:** API-Endpunkt prüfen (`POST /api/v1/wiki/articles` oder `PATCH /api/v1/wiki/articles/:id`), Frontend-Mutation prüfen
|
||||
- **Fix:** Je nach Diagnose — API-Fehler oder Frontend-Mutation-Fehler
|
||||
- **Aufwand:** 2 Stunden
|
||||
|
||||
#### 1.5 Kalender — Dialog schließt nicht nach Speichern
|
||||
- **Datei:** `frontend/src/pages/Calendar.tsx`, `frontend/src/components/calendar/AppointmentEditForm.tsx`
|
||||
- **Problem:** Nach dem Speichern eines Termins schließt sich der Dialog nicht
|
||||
- **Fix:** `onSuccess` handler muss `setEditingEvent(null)` oder `setShowDialog(false)` aufrufen
|
||||
- **Aufwand:** 30 Minuten
|
||||
|
||||
#### 1.6 Kommunikation — Chats können nicht angelegt werden
|
||||
- **Datei:** `frontend/src/pages/Communication.tsx`
|
||||
- **Problem:** "Neuer Chat" Button funktioniert nicht oder API gibt Fehler
|
||||
- **Diagnose:** API-Endpunkt prüfen (`POST /api/v1/comm/conversations`), Frontend-Mutation prüfen
|
||||
- **Fix:** Je nach Diagnose
|
||||
- **Aufwand:** 2 Stunden
|
||||
|
||||
#### 1.7 Wiki — Doppelt im Menü
|
||||
- **Datei:** `frontend/src/routes/index.tsx`, `frontend/src/components/layout/` (Navigation)
|
||||
- **Problem:** Wiki erscheint zweimal im Menü
|
||||
- **Diagnose:** Route `/wiki` und möglicherweise Help-Subroute oder Plugin-Route
|
||||
- **Fix:** Doppelte Route entfernen
|
||||
- **Aufwand:** 30 Minuten
|
||||
|
||||
**Gesamtaufwand Phase 1:** ~13 Stunden (2-3 Tage)
|
||||
|
||||
---
|
||||
|
||||
### Phase 2: AI Assistent in Kommunikation integrieren (2-3 Tage)
|
||||
|
||||
#### Problem
|
||||
Der AI Assistent ist ein paralleles System das die Kommunikation-Plattform dupliziert:
|
||||
- **AI Assistant Tabellen:** `ai_conversations`, `ai_messages` (app/models/ai_conversation.py) + `ai_chat_sessions`, `ai_chat_messages`, `ai_chat_attachments` (app/plugins/builtins/ai_assistant/models.py) — 5 Tabellen
|
||||
- **AI Assistant Frontend:** `AIAssistant.tsx`, `AIAssistantStandalone.tsx`, `SessionList.tsx`, `ChatWindow.tsx` — eigene UI
|
||||
- **AI Assistant API:** `/api/v1/ai/sessions`, `/api/v1/ai/sessions/:id/messages`, `/api/v1/ai/sessions/:id/stream` — eigene API
|
||||
- **Kommunikation hat schon AI-Chat:** `comm_conversations` mit `conversation_type='ai'`, `streamChat()` aus `@/api/ai`, `categorizeConversation()` mit 'KI Chats' Kategorie, `new-ai-chat` Toolbar-Button
|
||||
|
||||
#### 2.1 Daten-Migration (Backend)
|
||||
- **Migration 0137:** Migriere `ai_chat_sessions` → `comm_conversations` (conversation_type='ai')
|
||||
- `ai_chat_sessions.id` → `comm_conversations.id`
|
||||
- `ai_chat_sessions.title` → `comm_conversations.title`
|
||||
- `ai_chat_sessions.tenant_id` → `comm_conversations.tenant_id`
|
||||
- `ai_chat_sessions.user_id` → `comm_conversations.owner_id`
|
||||
- `ai_chat_sessions.agent_id` → `comm_conversations.metadata.agent_id`
|
||||
- `ai_chat_sessions.created_at` → `comm_conversations.created_at`
|
||||
- **Migration 0137:** Migriere `ai_chat_messages` → `comm_messages`
|
||||
- `ai_chat_messages.id` → `comm_messages.id`
|
||||
- `ai_chat_messages.session_id` → `comm_messages.conversation_id`
|
||||
- `ai_chat_messages.role` → `comm_messages.sender_type` ('user' → 'user', 'assistant' → 'ai')
|
||||
- `ai_chat_messages.content` → `comm_messages.content`
|
||||
- `ai_chat_messages.tenant_id` → `comm_messages.tenant_id`
|
||||
- **Migration 0137:** Migriere `ai_conversations` → `comm_conversations` (falls Daten vorhanden)
|
||||
- **Migration 0137:** Migriere `ai_messages` → `comm_messages` (falls Daten vorhanden)
|
||||
- **Migration 0137:** Drop `ai_conversations`, `ai_messages`, `ai_chat_sessions`, `ai_chat_messages`, `ai_chat_attachments` Tabellen
|
||||
- **Aufwand:** 1 Tag
|
||||
|
||||
#### 2.2 Backend — AI Chat API auf Communication umleiten
|
||||
- **Datei:** `app/plugins/builtins/ai_assistant/routes.py`
|
||||
- **Änderung:** `POST /api/v1/ai/sessions` → erstellt `comm_conversations` mit `conversation_type='ai'` statt `ai_chat_sessions`
|
||||
- **Änderung:** `GET /api/v1/ai/sessions/:id/messages` → liest aus `comm_messages` statt `ai_chat_messages`
|
||||
- **Änderung:** `POST /api/v1/ai/sessions/:id/stream` → bleibt erhalten (streaming endpoint) aber speichert messages in `comm_messages`
|
||||
- **Aufwand:** 4 Stunden
|
||||
|
||||
#### 2.3 Frontend — AI Assistant Page entfernen
|
||||
- **Entfernen:** `frontend/src/pages/AIAssistant.tsx`
|
||||
- **Entfernen:** `frontend/src/pages/AIAssistantStandalone.tsx`
|
||||
- **Entfernen:** `frontend/src/components/ai/SessionList.tsx`
|
||||
- **Entfernen:** `frontend/src/components/ai/ChatWindow.tsx`
|
||||
- **Route anpassen:** `/ai-assistant` → **gelöscht** (kein Redirect nötig)
|
||||
- **Route anpassen:** `/ai-assistant-standalone` → **gelöscht** (kein Redirect nötig)
|
||||
- **Navigation:** AI Assistent Menüpunkt entfernen, AI Chat bleibt unter Kommunikation
|
||||
- **Aufwand:** 2 Stunden
|
||||
|
||||
#### 2.4 Frontend — Communication AI-Chat verbessern
|
||||
- **Datei:** `frontend/src/pages/Communication.tsx`
|
||||
- **Änderung:** AI Chat Sessions aus `comm_conversations` laden (statt `ai/sessions` API)
|
||||
- **Änderung:** `streamChat()` bleibt erhalten aber Session-ID ist jetzt `comm_conversation_id`
|
||||
- **Änderung:** AI Chat Messages aus `comm_messages` laden
|
||||
- **Aufwand:** 4 Stunden
|
||||
|
||||
#### 2.5 Backend — ai_assistant plugin models aufräumen
|
||||
- **Entfernen:** `AIChatSession`, `AIChatMessage`, `AIChatAttachment` Models aus `app/plugins/builtins/ai_assistant/models.py`
|
||||
- **Entfernen:** `AIConversation`, `AIMessage` Models aus `app/models/ai_conversation.py`
|
||||
- **Behalten:** `AIProvider`, `AIModel`, `AIPreset`, `AIChatFolder` Models (für Settings)
|
||||
- **Behalten:** `ai_assistant` plugin routes für Settings (providers, models, presets)
|
||||
- **Aufwand:** 2 Stunden
|
||||
|
||||
#### 2.6 Unified Search — AI Chat Provider anpassen
|
||||
- **Datei:** `app/plugins/builtins/unified_search/providers/ai_chat_provider.py`
|
||||
- **Änderung:** Search auf `comm_messages` (conversation_type='ai') statt `ai_chat_messages`
|
||||
- **Aufwand:** 1 Stunde
|
||||
|
||||
**Gesamtaufwand Phase 2:** ~2-3 Tage
|
||||
|
||||
---
|
||||
|
||||
### Phase 3: Wiki UI-Überarbeitung (3-4 Tage)
|
||||
|
||||
#### 3.1 WYSIWYG Editor
|
||||
- **Datei:** `frontend/src/components/wiki/WikiEditor.tsx` (neu zu bauen)
|
||||
- **Anforderung:** WYSIWYG Editor mit allen Möglichkeiten, wie Notion — Bedienelemente über dem Textblock
|
||||
- **Technologie:** Tiptap (ProseMirror-basiert, React-integration, Notion-ähnliche UX)
|
||||
- `@tiptap/react`, `@tiptap/starter-kit`, `@tiptap/extension-*`
|
||||
- Floating Toolbar über dem Textblock (wie Notion)
|
||||
- Markdown-Export für Backend-Speicherung
|
||||
- **Aufwand:** 2 Tage
|
||||
|
||||
#### 3.2 Wiki Layout — 3-Spalten
|
||||
- **Datei:** `frontend/src/pages/Wiki.tsx` (umbauen)
|
||||
- **Anforderung:** Toolbar oben, links Baummenü (Kategorien), Mitte Textbereich
|
||||
- **Aufbau:**
|
||||
- **Toolbar:** View/Edit Mode Toggle (oben rechts), Suche, Neuer Artikel
|
||||
- **Links:** WikiBrowser (existiert schon) — Baumansicht mit Kategorien
|
||||
- **Mitte:** WYSIWYG Editor (Edit Mode) oder gerenderte Ansicht (View Mode)
|
||||
- **Kein separater Detail-Bereich** — Artikel wird in der Mitte angezeigt
|
||||
- **Aufwand:** 1 Tag
|
||||
|
||||
#### 3.3 View/Edit Mode Toggle
|
||||
- **Datei:** `frontend/src/pages/Wiki.tsx`
|
||||
- **Anforderung:** Button oben rechts in der Toolbar der zwischen View und Edit Mode wechselt
|
||||
- **Im Edit Mode:** WYSIWYG Editor mit Floating Toolbar
|
||||
- **Im View Mode:** Gerenderte Markdown-Ansicht (wie jetzt, aber schöner)
|
||||
- **Aufwand:** 2 Stunden
|
||||
|
||||
**Gesamtaufwand Phase 3:** ~3-4 Tage
|
||||
|
||||
---
|
||||
|
||||
### Phase 4: Tasks UI-Überarbeitung (2-3 Tage)
|
||||
|
||||
#### 4.1 Tasks Layout — 3-Spalten wie Kontakte
|
||||
- **Datei:** `frontend/src/pages/Tasks.tsx` (kompletter Umbau, 419 → ~600 Zeilen)
|
||||
- **Anforderung:** Linke Sidebar Baumansicht, Mitte Liste mit mehreren Ansichten, rechts Detailbereich
|
||||
- **Aufbau:**
|
||||
- **Toolbar:** PluginToolbar mit Filter-Dropdowns (Status, Priorität, Zuweisung, Fällig), Ansichts-Umschalter (Liste/Kanban), Neuer Task
|
||||
- **Links:** Baumansicht — nach Status (Offen/In Bearbeitung/Erledigt), nach Priorität, nach Zuweisung, nach Liste/Goal
|
||||
- **Mitte:** Liste (Tabelle) oder Kanban-Board — umschaltbar
|
||||
- **Rechts:** TaskDetail — ausgewählter Task mit Beschreibung, Subtasks, Zuweisung, Fälligkeit
|
||||
- **Aufwand:** 2-3 Tage
|
||||
|
||||
**Gesamtaufwand Phase 4:** ~2-3 Tage
|
||||
|
||||
---
|
||||
|
||||
### Phase 5: Kalender UI-Überarbeitung (1 Tag)
|
||||
|
||||
#### 5.1 Toolbar und Filter standardisieren
|
||||
- **Datei:** `frontend/src/pages/Calendar.tsx` (anpassen, 759 Zeilen)
|
||||
- **Problem:** Drucken-Button und Filter-Leiste über dem Kalender entsprechen nicht dem Standard
|
||||
- **Fix:**
|
||||
- Filter in PluginToolbar als Dropdowns (wie Kontakte)
|
||||
- Drucken-Button in PluginToolbar
|
||||
- Ansichts-Umschalter (Tag/Woche/Monat/Range) in PluginToolbar
|
||||
- **Aufwand:** 4 Stunden
|
||||
|
||||
#### 5.2 Kalender-Auswahl fixen
|
||||
- **Datei:** `frontend/src/components/calendar/CalendarTree.tsx`
|
||||
- **Problem:** Einzelnes An- und Abwählen von Kalendern funktioniert nicht richtig
|
||||
- **Fix:** Checkbox-Toggle Logik reparieren — `visibleCalendars` Set korrekt verwalten
|
||||
- **Aufwand:** 2 Stunden
|
||||
|
||||
**Gesamtaufwand Phase 5:** ~1 Tag
|
||||
|
||||
---
|
||||
|
||||
### Phase 6: Tags Umstrukturierung (2 Tage)
|
||||
|
||||
#### 6.1 Tags in Settings verschieben
|
||||
- **Datei:** `frontend/src/pages/Tags.tsx` → `frontend/src/pages/SettingsTags.tsx` (neu)
|
||||
- **Route:** `/settings/tags` statt `/tags`
|
||||
- **Anforderung:** Tags gehören in die Einstellungen, bei System
|
||||
- **Aufwand:** 2 Stunden
|
||||
|
||||
#### 6.2 Tags Baumstruktur
|
||||
- **Datei:** `frontend/src/pages/SettingsTags.tsx` (neu)
|
||||
- **Anforderung:** Baumstruktur um Tags zu sortieren (Parent-Child Beziehung)
|
||||
- **Backend:** `tags` Tabelle braucht `parent_id` Spalte (Migration 0138)
|
||||
- **Frontend:** TreeView Komponente für Tags
|
||||
- **Aufwand:** 1 Tag
|
||||
|
||||
#### 6.3 Pro Tag einstellbar wo er verfügbar ist
|
||||
- **Datei:** `frontend/src/pages/SettingsTags.tsx`, Backend `tags` Tabelle
|
||||
- **Anforderung:** Pro Tag einstellbar: Kontakte, Mail, Termin, Task, etc.
|
||||
- **Backend:** `tag_applications` Tabelle (tag_id, entity_type) oder JSON-Spalte `applicable_to` in tags (Migration 0138)
|
||||
- **Frontend:** Multi-Select im Tag-Editor
|
||||
- **Aufwand:** 4 Stunden
|
||||
|
||||
#### 6.4 Symbol und Farbe pro Tag
|
||||
- **Datei:** `frontend/src/pages/SettingsTags.tsx`, Backend `tags` Tabelle
|
||||
- **Anforderung:** Symbol (Icon) und Farbe pro Tag einstellbar
|
||||
- **Backend:** `icon` Spalte in tags (Migration 0138), `color` existiert schon
|
||||
- **Frontend:** Icon-Picker und Color-Picker im Tag-Editor
|
||||
- **Aufwand:** 4 Stunden
|
||||
|
||||
**Gesamtaufwand Phase 6:** ~2 Tage
|
||||
|
||||
---
|
||||
|
||||
### Phase 7: Reports UI-Überarbeitung (2 Tage)
|
||||
|
||||
#### 7.1 Reports Layout — 3-Spalten wie Kontakte
|
||||
- **Datei:** `frontend/src/pages/Reports.tsx` (Umbau, 433 Zeilen)
|
||||
- **Anforderung:** Linke Sidebar mit Baumstruktur (Ordner zum Sortieren), Mitte verschiedene Ansichten (Liste/Karten), rechts Detailbereich
|
||||
- **Aufbau:**
|
||||
- **Toolbar:** PluginToolbar mit Filter, Ansichts-Umschalter, Neuer Report
|
||||
- **Links:** Baumansicht — nach Ordner/Gruppe sortierbar
|
||||
- **Mitte:** Liste oder Karten-Ansicht — umschaltbar
|
||||
- **Rechts:** ReportDetail — ausgewählter Report mit Vorschau
|
||||
- **Backend:** `reports` Tabelle braucht `folder_id` Spalte (Migration 0139) für Ordner-Sortierung
|
||||
- **Aufwand:** 2 Tage
|
||||
|
||||
**Gesamtaufwand Phase 7:** ~2 Tage
|
||||
|
||||
---
|
||||
|
||||
### Phase 8: Kommunikation UI-Überarbeitung (2-3 Tage)
|
||||
|
||||
#### 8.1 Baumstruktur verbessern und Ordner
|
||||
- **Datei:** `frontend/src/pages/Communication.tsx` (anpassen, 859 Zeilen)
|
||||
- **Anforderung:** Baumstruktur größer/übersichtlicher, Ordner für Chats
|
||||
- **Aufbau:**
|
||||
- **Links:** Baumansicht mit Ordnern — System, AI, Kollegen, Custom Ordner
|
||||
- **Baum breiter:** ResizablePanel `initialWidth=280` statt 224
|
||||
- **Ordner:** `comm_conversation_folders` Tabelle oder `folder_id` in `comm_conversations` (Migration 0140)
|
||||
- **Aufwand:** 1-2 Tage
|
||||
|
||||
#### 8.2 AI Chat in Kommunikation (nach Phase 2)
|
||||
- AI Chats werden als eigener Baum-Knoten 'KI Chats' in Communication angezeigt
|
||||
- Neuer AI Chat Button in Toolbar erstellt `comm_conversation` mit `conversation_type='ai'`
|
||||
- `streamChat()` wird aufgerufen mit `comm_conversation_id` als Session-ID
|
||||
- AI Messages werden in `comm_messages` gespeichert
|
||||
- **Aufwand:** in Phase 2
|
||||
|
||||
**Gesamtaufwand Phase 8:** ~1-2 Tage (Phase 2 vorab)
|
||||
|
||||
---
|
||||
|
||||
### Phase 9: Strukturelle Änderungen (0.5 Tage)
|
||||
|
||||
#### 9.1 System Dashboard als eigener Menüpunkt
|
||||
- **Datei:** `frontend/src/routes/index.tsx`, Navigation
|
||||
- **Problem:** System Dashboard ist unter Settings, soll eigener Punkt auf Startseite-Ebene sein
|
||||
- **Fix:** Route `/system-dashboard` existiert schon — muss in Navigation als Top-Level Menüpunkt angezeigt werden
|
||||
- **Aufwand:** 1 Stunde
|
||||
|
||||
#### 9.2 Mail — Postfach mit IMAP anlegen testen
|
||||
- **Datei:** `frontend/src/pages/Mail.tsx`, `frontend/src/pages/MailSettings.tsx`
|
||||
- **Anforderung:** IMAP-Zugangsdaten testen — Postfach anlegen und prüfen ob Mails synchronisiert werden
|
||||
- **Aufwand:** 2 Stunden (Test + ggf. Bugfix)
|
||||
|
||||
**Gesamtaufwand Phase 9:** ~0.5 Tage
|
||||
|
||||
---
|
||||
|
||||
### Phase-L-Phasenübersicht
|
||||
|
||||
| Phase | Inhalt | Aufwand | Migration | Abhängigkeit |
|
||||
|-------|--------|---------|-----------|-------------|
|
||||
| 1 | Echte Bugs fixen | 2-3 Tage | Keine | Keine |
|
||||
| 2 | AI Assistent → Kommunikation | 2-3 Tage | 0137 | Phase 1.6 |
|
||||
| 3 | Wiki UI + WYSIWYG | 3-4 Tage | Keine | Phase 1.4 |
|
||||
| 4 | Tasks UI neu | 2-3 Tage | Keine | Keine |
|
||||
| 5 | Kalender UI | 1 Tag | Keine | Phase 1.5 |
|
||||
| 6 | Tags Umstrukturierung | 2 Tage | 0138 | Keine |
|
||||
| 7 | Reports UI | 2 Tage | 0139 | Keine |
|
||||
| 8 | Kommunikation UI | 1-2 Tage | 0140 | Phase 2 |
|
||||
| 9 | Strukturelle Änderungen | 0.5 Tage | Keine | Keine |
|
||||
|
||||
**Gesamtaufwand:** ~17-22 Tage
|
||||
|
||||
#### Reihenfolge:
|
||||
1. **Phase 1** (Bugs) — zuerst, damit grundlegende Funktionen arbeiten
|
||||
2. **Phase 9** (Strukturelle Änderungen) — schnell, wenig Aufwand
|
||||
3. **Phase 5** (Kalender) — kleines Update, baut auf Phase 1 auf
|
||||
4. **Phase 2** (AI Assistent → Kommunikation) — entfernt paralleles System, baut auf Phase 1.6 auf
|
||||
5. **Phase 6** (Tags) — unabhängig, Backend + Frontend
|
||||
6. **Phase 4** (Tasks) — großer Umbau, unabhängig
|
||||
7. **Phase 3** (Wiki) — größter Umbau (WYSIWYG Editor), baut auf Phase 1 auf
|
||||
8. **Phase 7** (Reports) — großer Umbau, unabhängig
|
||||
9. **Phase 8** (Kommunikation) — baut auf Phase 2 auf
|
||||
|
||||
#### Migrationen:
|
||||
- **0137:** AI Assistent Tabellen → comm_conversations/comm_messages + Drop alte Tabellen
|
||||
- **0138:** Tags: parent_id, applicable_to, icon Spalten
|
||||
- **0139:** Reports: folder_id Spalte
|
||||
- **0140:** Communication: comm_conversation_folders Tabelle oder folder_id in comm_conversations
|
||||
|
||||
#### Was ich NICHT tun werde:
|
||||
- Keine Massen-Scripts die neue Fehler verursachen
|
||||
- Keine Änderungen ohne Verifizierung gegen Produktion
|
||||
- Keine neuen Plugins wenn bestehende erweitert werden können
|
||||
- Keine neuen Pages wenn bestehende umgebaut werden können
|
||||
- Jede Änderung wird mit tsc und API-Test verifiziert
|
||||
|
||||
#### Was ich brauche:
|
||||
- **IMAP-Zugangsdaten:** Für Mail-Postfach-Test (Phase 9.2)
|
||||
|
||||
---
|
||||
|
||||
## Zusammenfassung
|
||||
|
||||
| Phase | Dauer | Hauptdeliverable |
|
||||
|
||||
+152
@@ -6,6 +6,158 @@
|
||||
|
||||
---
|
||||
|
||||
## Architektur-Reparatur (2026-08-23, laufend)
|
||||
|
||||
**Plan:** docs/fix-plan-v3.md · **Sequenz:** Block 0 + Block H gemerged in main, Block A läuft auf main
|
||||
**Stand:** 18 Findings geschlossen (~30% aufwandsgewichtet) · App startbar · 90+ Tests grün · tsc clean · Checker 14→6 Verstöße · Alles gepusht auf Forgejo
|
||||
|
||||
| Finding | Beschreibung | Status | Commit |
|
||||
|---|---|---|---|
|
||||
| SYNTAX-001 | automation/plugin.py SyntaxError — App startet nicht | ✅ gefixt | 8077595 |
|
||||
| CHECK-002 | Checker crasht bei relativen Pfaden | ✅ gefixt | 35e2cc8 |
|
||||
| ARCH-010 | Checker scannt nur builtins | ✅ Vollscan-Default | 35e2cc8 |
|
||||
| ARCH-014 | Contract-Lazy-Resurrect nach unregister | ✅ gefixt + Funktionstest | b04cda7 |
|
||||
| ARCH-020 | EventBus subscribe ohne Duplikat-Check | ✅ gefixt + Funktionstest | b04cda7 |
|
||||
| ARCH-029/041 | trigger_dispatcher None-Check nach Verwendung | ✅ None-Check zuerst | b04cda7 |
|
||||
| ARCH-001 | Permissions nach on_activate registriert | ✅ Reihenfolge gedreht | 1d6152f |
|
||||
| ARCH-002 | on_activate pro Tenant mehrfach | ✅ 1× pro Prozess | 1d6152f |
|
||||
| ARCH-003 | active-manifests an plugins:read gebunden | ✅ für eingeloggte User offen | 982b4c9 |
|
||||
| ARCH-038 | BasePlugin.register_event_handlers fehlt | ✅ Hook ergänzt | 90a3670 |
|
||||
| ARCH-054 | entity_permissions falsche Datenstruktur | ✅ Model-Lookup korrigiert | 90a3670 |
|
||||
| ARCH-047 | SearchContract-Import kaputt (search-Step ImportError) | ✅ gefixt | d87fc4e |
|
||||
| ARCH-030 | contract.get_function() existiert nicht | ✅ auf 5 Contracts ergänzt | d87fc4e |
|
||||
| ARCH-031 | knowledge/plugin.py uuid nicht importiert | ✅ mitfixt | 1f4a621 |
|
||||
| ARCH-040/046/049 | Core→Plugin-Imports (worker/compliance/engine) | ✅ via Contract/Plugin-Job | 44511a8 + a7699d3 |
|
||||
| HC-F Frontend | BlockRenderer: 14 hardcodierte Blocks | ✅ Registry | b7ad529 |
|
||||
| HC-G Frontend | AISidebar: 5 hardcodierte Tabs | ✅ Tab-Registry | 59fdb61 |
|
||||
| HC-A Backend | action_mapper feste Regex-Intents | ✅ Contribution-API | 4994906 |
|
||||
| Gate H | Plugin-Contribution ohne Core-Änderung beweisen | ✅ BESTANDEN (2/2) | 801743b |
|
||||
|
||||
| ARCH-043 | automation Tenant.limit(1) statt System-Tenant | ✅ get_system_tenant() + system_tenant_slug Setting | 17516d2 |
|
||||
| ARCH-052 | storage get_file_metadata Event-Loop im async Kontext | ✅ get_file_metadata_async() + Fail-Fast-Guard | 17516d2 |
|
||||
| ARCH-008 | Permission-Namensschema inkonsistent | ✅ Kanon modul:aktion festgelegt, Manifest-Validator erzwingt es | 7953077 |
|
||||
| ARCH-009 | Tote 3-Segment-Rollen-Patterns (core:*:X) + 14 Route-Literals | ✅ Migration 0141 + Route-Fix, Roundtrip bewiesen | 7953077 |
|
||||
|
||||
| ARCH-012 | wiki/knowledge on_deactivate unvollständig | ✅ Provider-Dereg + 2 latente Bugs (register_provider fehlte am Contract, kaputter Modul-Import) behoben | c21634b |
|
||||
| ARCH-013 | self_improvement Fallback-Import; Benachrichtigung war tot | ✅ Contract-only; undefinierten KommunikationContract-Verweis behoben | c21634b |
|
||||
| ARCH-015 | Notification-Sync fehlt in Deactivate-Sequenz | ✅ sync_notification_types nach Status-Update | c21634b |
|
||||
| ARCH-033 | comm_websocket/comm_miniapps bleiben im Container | ✅ Container-Cleanup VOR super(); ServiceContainer.remove() ergänzt | c21634b |
|
||||
| ARCH-034/035 | self_improvement/marketplace Contract-Unregister | ✅ verifizierte No-Ops: beide registrieren keinen Contract | c21634b |
|
||||
| ARCH-036 | mail _auto_sync_task Klassenvariable | ✅ Instanzvariable via __init__ | c21634b |
|
||||
| ARCH-037 | graph_rag Registrierung VOR super() | ✅ Reihenfolge umgestellt | c21634b |
|
||||
| ARCH-044 | ai_ui_control remove() NACH super() | ✅ Reihenfolge umgestellt; fehlendes ServiceContainer.remove() ergänzt | c21634b |
|
||||
| Gate A | Block-A-Abschlussprüfung | ✅ BESTANDEN (4/4): Imports, Lifecycle-Symmetrie, Activate-Once, Contract-Roundtrip | 32f63ad |
|
||||
|
||||
| B1 | Contacts-Domain aus Core entkoppelt: 4 Router ins Plugin verschoben, manifest.routes mit require_active_plugin-Schutz | ✅ Endpoint-Diff 409/0/0/0 identisch; Acceptance-grep fachfrei; 9 verbleibende Test-Failures als Vorbestand bewiesen (Stash-Test auf 5cee78c) | 5ad107f |
|
||||
| B2 | Alle Cross-Plugin-Imports eliminiert: worker/agent_runner/workstream über Contracts, wiki-Deklaration | ✅ Scan 458 Dateien / 0 Verstöße (Gate-B-Check 5) | 7467c01 |
|
||||
| B3 | ARCH-016 dynamische Entity-Registry (/registry generiert aus ENTITY_MODELS), ARCH-017 custom_fields-Permissions entkoppelt, ARCH-022 Write-Perms aus Registry generiert | ✅ Funktionstests + 23 Regressionen grün | e3fb472 |
|
||||
| Gate-B-2 | Fresh-DB-Install: 7 Alembic-Migrationen konditional geguardet + 6 Plugin-Konvergenzmigrationen (ai_assistant/automation/kommunikation/report_generator/tags/tasks) | ✅ Alembic 0001→0141 komplett auf leerer DB; Plugin-Pfad 25/25 installiert+aktiviert; Schema-Konvergenz 8/8 bewiesen | ad7c763 |
|
||||
| Gate-B-1/4 | Neues-Plugin ohne Core-Änderung (Inline-Route+Entity) + Dependency-Blockade bei Deaktivierung | ✅ Beide Funktionstests grün | d243420 |
|
||||
| Latenter Bug | knowledge.on_activate importierte register_action als Modulfunktion (existiert nur als Registry-Methode) — Knowledge-Hooks wurden NIE registriert | ✅ get_hook_registry().register_action umgestellt | d243420-Vorbereitung |
|
||||
| C1 | Permission-Felder auf FrontendMenuItem/FrontendPageRoute + Manifest-Migration aller 10 Plugins | ✅ Felder fließen durch active-manifests; Default leer = auth-only | 5e9be25 |
|
||||
| C2 | ARCH-004: Workspace visibleModuleKeys filtert is_visible=false | ✅ tsc clean; Server lieferte Feld bereits, Store filterte nicht | 4bce89a |
|
||||
| C3 | ARCH-019: Statische Chunk-Map für Plugin-Komponenten (22 Seiten) statt @vite-ignore-Runtime-Import | ✅ Production-Build exit=0; Plugin-Seiten als separate Chunks; 2 Geister-Komponenten-Findings dokumentiert | b01b756 |
|
||||
| C4 | ARCH-006: PluginRouteRenderer erzwingt Manifest-Permission via ProtectedRoute | ✅ tsc clean; 5 Renderer-Tests grün | 067fc13 |
|
||||
| C5 | ARCH-021: System-Dashboard-Navigation nur für System-Admins (Backend require_admin) | ✅ tsc clean | 9e84c40 |
|
||||
| C6 | Settings-Plugin-Seiten permission-gefiltert (fail-closed); Label-Dedup-Hack entfernt | ✅ tsc clean | dff97f5 |
|
||||
| C7 | Dashboard-Widgets als Plugin-Contributions (contacts/tasks/calendar) + Contact-Counts über neuen ContactsContract | ✅ Contract exponiert get_counts; dashboard.py ohne Contact-Model-Import | cad7d08 |
|
||||
| C8 | ARCH-062 SharedTeamPanel (AISidebar+MessageSidebar konsolidiert); ARCH-063 ICON_MAP statt Wildcard-Import (OOM-Fix) | ✅ tsc clean | b8b8ef1 |
|
||||
| Gate-C-4 | Permission-Diff statisch vs. Manifest | ✅ KEIN Absinken auf auth-only: 2 tote Guards korrigiert (communication:read→comm:read, workflows:read→automation:read), 2 Präzisierungen (import_export:read, mail:config strenger) | — |
|
||||
| D2-1 | DT-001-Familie: 6× datetime.utcnow() → datetime.now(UTC) (worker ×2, audit, webhook_service inkl. Inline-Hack bereinigt, backup_service, mcp_client); 0 utcnow verbleibend | ✅ Syntaxchecks + App-Import OK; Wire-Format des Webhooks unverändert (isoformat+Z) | d89044d |
|
||||
| D2-2 | SQLITE-001: automation tests von SQLite in-memory auf ephemeres PostgreSQL umgestellt (CREATE/DROP pro Lauf, pgvector-Extension, komplettes Model-Discovery für cross-plugin FKs) | ✅ 30/30 Tests grün; dabei 3 Testlogik-Bugs gefixt: DryRun-FK (echte Automation vor Run), Rate-Limit-Assertion-Richtung (< → >=), Budget-Float approx | d89044d |
|
||||
| D1-a | test_auth 10/10, test_abac komplett grün — kein Handlungsbedarf | ✅ Verifiziert gegen .env.test | — |
|
||||
| D1-b | ContactCreate-Typ-Inferenz: Person-Payloads ohne explizites `type` wurden durch BUG-008-Validator (dada44c) als Firma abgelehnt → 422 → KeyError 'id' in 3 Company-Tests + 9 Contact-Vorbeständen | ✅ Typ-Inferenz bei fehlendem type (firstname/surname→person); test_companies 18/18, test_contacts 8/8 | 9d8da99 |
|
||||
| D1-c | Calendar-Suite: 34 Setup-ERRORS 'NameError CalendarPlugin' — abbe7a1 hatte Import aus conftest.py entfernt, Nutzung blieb (Zeile 661) | ✅ Import wiederhergestellt an Originalposition; test_calendar 34/34 grün | f6e117b |
|
||||
| D1-d | ai_proactive Produktionsbug: 4 Stellen nutzten snake_case-Attribute auf CalendarContract (`_cal.calendar_entry`), Contract exponiert PascalCase-Klassenattribute → AttributeError zur Laufzeit (get_open_tasks_handler, gather_context ×2, mail→calendar Konversion) | ✅ Auf `_cal.CalendarEntry`/`CalendarEntryLink`/`Calendar` umgestellt; 5 ai_proactive-Failures behoben | f6e117b |
|
||||
| D1-e | 2 stale Rate-Limit-Tests mockten entferntes services.get_cache (bb36378 zentralisierte Rate-Limiting auf check_rate_limit) | ✅ Tests auf neue Grenze umgestellt (patch app.core.rate_limit.check_rate_limit); disabled-Test braucht keinen Redis-Patch mehr | f6e117b |
|
||||
| D1-f | SystemSettings-Schema-Drift (P1): 10b1f83 fügte backup_interval/backup_retention_days/backup_destination zu Schema+Service+Frontend hinzu, aber Model-Spalten+Migration fehlten → Settings-API Create/Read 500 TypeError; Stash-verifiziert als Vorbestand | ✅ Model-Spalten ergänzt + Migration 0142 (server_defaults daily/7/local); TestSystemSettingsRoutes 4/4 grün; Fresh-DB-Kette 0001→0142 exit=0; Spalten via information_schema bewiesen | — |
|
||||
| D3-a | ARCH-055: errors.py nutzte error.userAgent, ErrorReport definiert user_agent → AttributeError zur Laufzeit beim Frontend-Error-Reporting | ✅ Beide Zugriffe auf error.user_agent korrigiert; ruff clean | 0768cfb |
|
||||
| D3-b | ARCH-056: roles.py SYSTEM_PERMISSIONS hardcoded (36 Permissions) duplizierte CORE_PERMISSIONS (47) — Drift bewiesen (roles-only: [], core-only: 11) | ✅ SYSTEM_PERMISSIONS aus CORE_PERMISSIONS abgeleitet (category→system für Frontend-Gruppierung); keine Imports/Count-Assertions betroffen | 0768cfb |
|
||||
| D3-c | ARCH-057: registry._plugins.items() privater Zugriff in roles.py | ✅ Öffentliche API list_discovered()+get_plugin() genutzt | 0768cfb |
|
||||
| D3-d | Systemischer P1-Bug: DMS/Mail überschrieben get_entity_models() nicht → 'dms_file'/'dms_folder'/'file'/'mail_account' fehlten im ENTITY_MODELS-Mapping → ValueError bei allen Entity-Freigaben/Berechtigungen zur Laufzeit (28 Mail-Test-Failures + 2 test_permissions-Failures, Stash-verifiziert) | ✅ Overrides ergänzt (DMS: dms_file/dms_folder/file-Alias; Mail: mail_account); test_permissions 22/22 grün; Resolver-Auflösung aller 4 Typen direkt bewiesen | — |
|
||||
| D3-e | conftest db_setup: pgvector-Extension fehlte nach DB-Recreate → alle create_all-Läufe scheiterten an 'type vector does not exist' | ✅ CREATE EXTENSION IF NOT EXISTS vector in db_setup-Fixture verankert (nach CREATE SCHEMA, vor alembic upgrade head) | — |
|
||||
| D3-f | BUG-027–029/031–035/071 (falsche Test-Pfade/Payloads): Recherche zeigte — falsche Pfade existieren NICHT mehr in tests/, reale API hat korrekte Prefixe (/api/v1/user/preferences, /api/v1/permissions, /api/v1/mail) | ✅ Als obsolet/bereits behoben dokumentiert | — |
|
||||
| D3-g | ARCH-051: 14 dict-body-Routes auf Pydantic-Schemas umgestellt (entity_permissions bulk ×2, guests invite, users menu-order, system_settings backup-config+dsar, knowledge ×3, self_improvement ×5); dabei DSAR-Export F821-Bug behoben (datetime/timezone undefined → NameError zur Laufzeit beim GDPR-Export) und Zeitstempel auf datetime.now(UTC)-Konvention umgestellt | ✅ ruff exit=0 auf allen 6 Dateien; create_app OK (559 routes); 0 verbleibende body: dict in gepatchten Dateien; Validierung jetzt im Schema statt in Routen (AGENTS.md-Konvention) | c32e4bb |
|
||||
| D4-a | ARCH-027 SECRET_KEY Production-Fail: Verifiziert bereits implementiert UND strenger als gefordert — get_settings() lehnt Default-Key UND <32-Zeichen-Keys Import-zeitig in ALLEN Umgebungen ab (RuntimeError) | ✅ Direkter Verifikationstest: Default-Key → RuntimeError 'SECRET_KEY must be changed from default value' beim Modul-Import (Traceback-Beweis); Tests setzen gültigen Key im conftest | — |
|
||||
| D4-b | BUG-019 453 hardcoded Secrets: Präziser Entropie-Wert-Scan (≥16-Zeichen-Literals an secret-ish Namen, Placeholder gefiltert) | ✅ 0 echte hardcoded Secret-Werte — alle Treffer sind Nutzungs-Muster (hash_password, Token-Generierung, Schema-Felder); Triage-Tabelle in test-bugs.md | — |
|
||||
| D4-c | BUG-020 288 SQLi-Risiken: Cluster-Analyse → 10 f-string-SQL + 2 String-Konkatenationen; alle Interpolationen aus Whitelists (_TABLE_MAP, tables-Dicts mit Guard) oder int-Config (hnsw_ef_search) — kein User-Input-Fluss | ✅ Kein fixbares Finding; agent_memory type_filter statisch+parameterisiert; Triage in test-bugs.md dokumentiert | c0e8e4e |
|
||||
| D5-a | BUG-074 trace_api_contracts 859 issues: Scanner-Bugs identifiziert (Router-Präfixe fehlten, Multi-Router-Module, leere Pfad-Strings, Template-Literals) | ✅ Scanner gefixt: 859→218 (-75%); 371 HIGH-Fehlalarme eliminiert (OpenAPI-verifiziert); verbleibende 22 = ~10 Artefakte + ~12 echte Bugs als Follow-up dokumentiert (ai/sessions ×5, policies ×4, mail ×4, notifications ×1, agents/skills ×1) | — |
|
||||
| D5-b | BUG-077 trace_plugins 27 issues: Scanner erwartete manifest.py, Projekt-Konvention ist Inline-Manifest in plugin.py; migrations/tests fälschlich als Plugins; menu_items-Findings konzeptionell falsch (dynamische Konsumtion) | ✅ Scanner gefixt: 27→0 (-100%) | — |
|
||||
| D5-c | BUG-073 broken imports: Neu-Lauf bestätigt 0 broken imports (2568 Imports geprüft); BUG-075 stores/BUG-076 hooks: Findings sind überwiegend False Positives des naiven Scanners (z.B. 'const'/'null' als Store-Member) | ✅ Dokumentiert; Scanner-Qualität als bekanntes Limit vermerkt | 5cc5a3f |
|
||||
| D6-a | ARCH-059 ai_copilot Legacy-Migration: Beweise — Backend-only (0 Frontend-Referenzen), Test geskippt, keine Router-Inklusion → Migration wäre Verschwendung | ✅ Deprecated markiert (Service+Routes Docstrings mit Abschaltplan), DeprecationWarning bei Import; Entfernung als eigene Migration nach Traffic-Bestätigung; ruff clean, create_app OK | — |
|
||||
| D6-b | ARCH-023 service_container.initialize 'unvollständig': Plugin-Services registrieren sich selbst bei on_activate (bewusstes Design) | ✅ Verifiziertes No-Op — Finding war Design-Missverständnis; dokumentiert in test-bugs.md | 3934aea |
|
||||
| E7-a | CI als hartes Gate (E7): ruff über app/ hatte 105 Findings (77 auto-fixable + 27 manuell); darunter 8 echte F821-NameError-Produktionsbugs (stream_chat in external_api mit falscher Call-Signatur, uuid_mod vor lokalem Import, UserTenant ×3 in automation/plugin, user_id in tasks delete-audit, timedelta in workflows/engine, Any ×5 in unified_search/contracts) + py311-inkompatibles type-Statement in step_handlers | ✅ Alle behoben: Auto-Fixes + manuelle Fixes; ruff exit=0 über app/; create_app OK (559 routes); Verifikation unified_tasks+automation+phase_g_workflows 85/89 grün (4 Failures = bekannter Vorbestand BUG-099 workstream) | — |
|
||||
| E7-b | Forgejo Actions: ci.yml existiert (.forgejo/workflows/ci.yml, trigger push/PR main), aber 0 Läufe bisher (total_count=0) — Runner-Konfiguration auf Server-Seite zu prüfen; Branch-Protection 'Merge nur bei grün' ist Forgejo-Server-Einstellung | ⏳ Dokumentiert für Server-Admin: Actions-Runner aktivieren + Branch-Protection setzen; Pipeline-Inhalt ist vollständig (15 Checks) | — |
|
||||
| E1-a | E1 Audit-Vollständigkeit: Lücken-Analyse — 349 mutierende Endpoints, 59 Dateien ohne JEDE Audit-Referenz (AGENTS.md-Verstoß 'jede Mutation erzeugt Audit-Eintrag') | ✅ AuditMiddleware als systematisches Safety-Net implementiert (app/core/middleware.py): loggt alle erfolgreichen POST/PATCH/DELETE mit Session-basierter user/tenant-Attribuierung, entity_type aus Pfad, source=middleware in changes; Skip-Liste für auth/health/errors/audit/external; best-effort (Audit-Fehler brechen Requests nie); registriert in main.py | — |
|
||||
| E1-b | E1 Beweis: Dedizierter Test test_audit_middleware.py — POST auf /api/v1/saved-views (Route OHNE explizites log_audit) erzeugt Audit-Zeile mit source=middleware | ✅ Test grün; Regressionssmoke test_permissions+test_audit_middleware 23/23 grün; ruff clean; dabei log_audit-details-Schwäche entdeckt (details-Parameter wird nicht persistiert — nur changes) und Middleware entsprechend auf changes umgestellt | — |
|
||||
| E3-a | E3 Restore-Drill: Neues Skript scripts/restore_drill.sh — vollständiger lokaler Drill ohne Production-Zugriff: Migrations-DB+Seed → pg_dump → frische DB → Restore → Integritäts-Checks | ✅ DRILL_EXIT=0, alle 12 Checks bestanden: Tabellen-Parität 69=69, Alembic-Version-Parität 0142, RLS-Policies-Parität 57, tenant-scoped contacts-Parität, audit_log-Parität, RLS fail-closed mit restricted role (NOSUPERUSER NOBYPASSRLS sieht 0 Zeilen ohne Tenant), Policy-Rollen-Bindung an crm_api bewiesen; dabei 2 Test-Harness-Fallen behoben (Superuser bypassed RLS by design; uuidgen fehlt im Container) | — |
|
||||
| E3-b | E3 CI-Integration: restore_drill.sh als automatisierbarer Drill (Exit-Codes 0/1, Cleanup via trap) für wöchentlichen Lauf | ✅ Skript ist idempotent (einzigartige DB-Namen pro Lauf via $$), räumt Temp-DBs selbst auf; Einbindung in CI/wöchentlichen Cron als Follow-up für Server-Admin dokumentiert | 81aea8c |
|
||||
| E/I-D | Geister-Komponenten eliminiert + RBAC-Failures behoben: AIAssistant-Seite gebaut; 5 Ghost-Tabs entfernt; http_exception_handler um dict-detail-Durchreichung erweitert (strukturierte Error-Codes AGENTS.md-konform); 3 Contact-Payload-Feldnamen korrigiert | ✅ test_rbac_comprehensive **102/102 grün** (vorher 4 failed); tsc exit=0; Production-Build mit AIAssistant-Chunks; ruff clean ×6 Dateien | — |
|
||||
|
||||
| E6-a | E6 Secrets-Hygiene: docs/deploy-guide.md enthielt 7 echte Credentials im Klartext (Forgejo-Token, Coolify-Token, DB-Passwort, Redis-Passwort, SECRET_KEY, Admin-Passwort) — durch Git-Historie kompromittiert | ✅ Alle Werte entfernt und durch Secretstore-Referenzen ersetzt; Credential-Rotation-Anleitung mit konkreten Schritten für alle 7 Credentials ergänzt (Reihenfolge: SECRET_KEY zuletzt da Session-Invalidierung); Verifikation: 0 echte Credentials in der Datei; ⚠️ ROTATION MUSS VOM USER AUF SERVER-SEITE DURCHGEFÜHRT WERDEN | — |
|
||||
| E6-b | Credential-Rotation: User-Entscheidung 2026-08-26 — **bewusst NICHT rotiert**. Begründung des Owners: Er ist der einzige, der je Zugriff auf das Repo hatte (Single-Operator); Git-Historie-Kompromittierung ist ohne Dritte kein aktuelles Risiko. Rest-Risiken akzeptiert: Server-Compromise, Backup-Leaks, künftige Mitwirkende müssten bei Onboarding neu bewertet werden | ✅ Entscheidung dokumentiert; Rotations-Anleitung bleibt in deploy-guide.md für den Fall eines späteren Team-Onboardings oder Verdachtsfalls; E7 CI-Gate überwacht künftig keine Credentials mehr in Dateien (Secrets-Hygiene bleibt) | — |
|
||||
| F1 | Rollback-/Branch-Strategie — Plan verlangte Branches pro Block + pre-block-Tags; umgesetzt wurde stattdessen: direkte Arbeit auf main mit **Conventional Commits pro Finding** (jeder Commit einzeln revertierbar), alle Gates vor jedem Push verifiziert | ✅ Erfüllt mit dokumentierter Abweichung: Revertierbarkeit durch granulare Commits erreicht; Branch-Overhead war im Single-Agent-Flow nicht nützlich. Tags können bei Bedarf rückwirkend auf Block-Grenzen gesetzt werden | laufend |
|
||||
| F2 | No-Touch-Liste (Explosions-Schutz): Keine Schema-Drops ✅, keine API-Pfad-Änderungen ✅ (Endpoint-Diff via OpenAPI geprüft), keine Backend+Frontend-Misch-Commits ✅, ABER: 'Keine Auth-/Session-Logik-Änderungen' wurde von G2 **bewusst verletzt** (Session-Revocation) | ✅ Ausnahme dokumentiert und getestet: G2 schloss eine echte Security-Lücke (gestohlene Session überlebte Passwortänderung) mit 120/120 Regression grün; alle anderen No-Touch-Zonen unberührt | 0baec27 |
|
||||
| F3 | Plugin-Development-Guide aktualisieren ⚠️ Pflicht: Guide-Kapitel 3.1 hatte Contracts/Dependencies bereits (aus Block A/C); Kapitel 29.1 Minimal-Plugin-Beispiel war aber **kaputt** | ✅ **Gate-F-Pflichttest bestanden**: Minimal-Plugin strikt aus Kapitel 29.1 gebaut → 3 echte Guide-Lücken gefunden (__init__.py-Re-Export für Discovery fehlte, Route braucht vollen Pfad da main.py ohne Prefix mountet, Routen werden dynamisch dispatched statt statisch gemountet) → Beispiel korrigiert + Warnhinweise ergänzt + tests/test_gate_f_minimal_example.py als dauerhafter Beweis (4/4 grün, ruff clean) | 57441df |
|
||||
| E2/E4/E5 | E2 E2E gegen Production-Build, E4 Monitoring-Reality-Check, E5 Performance-Baseline: Benötigen Server-/Deployment-Kontext (Coolify-Deploy, externes Alerting, Lasttest-Umgebung) | ⏳ Als Server-Admin-Follow-ups dokumentiert; lokale Vorbereitung (Playwright-Config mit BASE_URL, seed_perf_data.py, spike_e_benchmark.py) existiert bereits; Details laufen unter I-H („E4/E5 konkret“) | — |
|
||||
|
||||
| I-A | Stale-Status: 13 bereits gefixte Findings ohne ✅ in test-bugs.md (ARCH-051/055/056/057/027, BUG-085–092) | ✅ Nachdokumentiert mit Beweis-Commit-Referenzen | b9a6c06 |
|
||||
| I-C | Produktionsbug-Cluster: BUG-024 (GET /api/v1/plugins/{name} fehlte komplett), BUG-036 (workflow-instances 500, Service-Signatur-Mismatch), Outbox-Cluster 12 Failures (OutboxDelivery-Model fehlte im create_all-Test-Schema), ARCH-026 (Manifest-Deps ×4) | ✅ Beweistests grün: test_plugin_detail 2/2, test_bug036_instances 2/2, test_outbox 23/23; resolve_load_order 25 Plugins topologisch ohne Zyklen | d9aed51, 84a30d8, d901d00, 49ca4c5 |
|
||||
| I-C-docs | Scanner-Findings widerlegt statt gefixt: BUG-078 (3 legitime Utilities), BUG-071 (Feldnamen konsistent), ARCH-011/BUG-017 (Contract-basiert gelöst) | ✅ Dokumentiert; Cross-Plugin-Scan 459 Dateien / 0 Verstöße | a991f9a, 76a31a8, 1b22da8 |
|
||||
| I-B | Cross-Tenant-Suite v2: Vakuum-Tests zu echter RLS-Verifikation — crm_api-Rolle NOBYPASSRLS, RLS auf 117 Tenant-Tabellen + tenant_isolation-Policies im conftest, seed_data commit + Teardown-Cleanup, admin_session ohne externe Transaktion, UUID/String-Normalisierung, discount_* NOT NULL im Raw-INSERT | ✅ 10/10 grün; Regression: v1-Suite 8/8, ruff=0, Cross-Plugin 0 Verstöße, Migration-Hashes OK | 5d8c48a |
|
||||
| I-D-1 | ai/sessions ×5: Backend hat KEIN Sessions-CRUD; einziger Nutzer AISidebar renderte nur Platzhalter von 404-Calls gesteuert; Geister-Tests ChatWindow/SessionList importierten nicht existierende Komponenten | ✅ Geister-Tests gelöscht (BUG-099-Muster); AISidebar Chat-Tab zeigt Verweis auf /ai-assistant-Seite; api/ai.ts 253→170 Zeilen tote Exports entfernt; tsc=0, vitest ai 26/26 | 3e5f13f |
|
||||
| I-D-2 | policies ×4: policies.ts + policyHooks.ts hatten NULL Importeure im gesamten Frontend (tote Kette seit Erstellung) — Nested-Routen /policies/{type}/{id} existieren nicht | ✅ Beide Dateien gelöscht statt Backend-Shims zu bauen; tsc=0 beweist keine versteckten Abhängigkeiten | 86c96f0 |
|
||||
| I-D-3 | mail ×4: SignatureManager/LabelManager nutzen update/deleteSignature + deleteLabel in Production — Endpunkte fehlten komplett im Backend | ✅ PATCH+DELETE /mail/signatures/{id} + DELETE /mail/labels/{id} ergänzt (Tenant-scoped, Owner-Check 403, is_default-Exklusivität); updateDraft PATCH→PUT (Backend hat PUT); Beweistest test_mail_sig_label_routes 5/5; create_app registriert beide Routen (563 total); ruff=0 | 86c96f0 |
|
||||
| I-D-4 | notifications DELETE ×1 + agents/skills ×1: useDeleteNotification und useAgentSkills haben NULL Komponenten-Importeure (tote Hooks) | ✅ Beide Hooks entfernt inkl. ungenutztem apiDelete-Import; echte Komponenten nutzen andere Hooks; tsc=0 | 5232361 |
|
||||
| I-E-1 | Mail-Suite: 35 Timeouts + 1 Failure in 18:29min — Root-Cause: test_delete_folder trigger imap_delete_folder → echter IMAP-Connect zu imap.example.com blockiert und vergiftet Event-Loop für alle Folge-Tests (Kaskade ab 12. Test) | ✅ **46/46 grün in 94.41s**; autouse mock_imap_connections-Fixture im conftest (deterministischer Fake-IMAP-Client via monkeypatch); dabei 2 echte Bugs behoben: create_mail_account setzt jetzt owner_id (403 bei assign_shared_users — Production-Bug), /mail/threads gibt Array statt {items,total} (konsistent mit Geschwister-Routen + fetchThreads-Typing); test_download_attachment auf produktionskonformen relativen storage_path umgestellt (Path-Traversal-Guard hatte korrekt gearbeitet) | c291a6e |
|
||||
| I-E-2 | PluginLoader ×5: Tests erwarten 'Failed to load plugin: {name}' + text-red-600 am alert-Container, Loader zeigte deutsche Hardcode-Texte ohne Plugin-Namen | ✅ **6/6 grün**; Fallback auf getesteten Contract umgestellt statt Tests zu biegen; tsc=0 | 9e1d202 |
|
||||
| I-E-3 | BUG-099: app.ai.agent_workstream + app.workflows.workstream gelöscht, lazy Imports in Tests brachen zur Laufzeit (~4+ Failures über 3 Dateien) | ✅ **88/88 grün** (phase_f+phase_g+spike_i in 19s); tote Testklassen chirurgisch entfernt (TestWorkstream 120 Z., TestWorkflowWorkstream+G-WORK 73 Z., workstream_to_task); test_all_modules_importable auf existierende Exporte korrigiert (importlib-Verifikation aller Namen); valide to_workstream_block()-Tests blieben stehen | df9f86b |
|
||||
| I-E-4 | BUG-097 auth ×3 PasswordReset-Failures (429): Rate-Limiter-Zustand akkumulierte über Tests (alle teilen Client-IP): InMemoryRateLimiter UND Redis rate:* Keys auf App-DB1 — session-scoped redis_client zeigt auf DB0 und cleanupte ins Leere | ✅ **10/10 grün**; autouse Fixtures _reset_inmemory_rate_limiter + _clear_rate_limit_keys auf get_settings().redis_url | f4c4a50 |
|
||||
| I-E-5 | BUG-094 api_audit ×7: docs/api-audit.md fehlte komplett (nie committed) — alle Failures FileNotFoundError/AssertionError auf die eine Datei | ✅ **9/9 grün**; Audit-Dokument aus verifizierten Fakten erstellt (563+ Routes, 14 Kategorien, RBAC, Frontend Coverage, Missing Endpoints = 0); die 2 Reachability-Tests liefen schon vorher grün | 1b485d4 |
|
||||
| I-E-6 | BUG-098 rls_coverage ×6 — echte Security-Lücken: kein FORCE RLS auf 122 Tenant-Tabellen, Policies an PUBLIC statt Runtime-Rollen, crm_migration BYPASSRLS, Legacy crm_runtime vorhanden; plus Contract-Widerspruch v1 (Identity-Tabellen RLS-frei für Login-Bootstrap) vs rls_coverage (alle Tabellen gehärtet) | ✅ **31/31 grün** über rls_coverage+cross_tenant v1+v2: conftest härtet FORCE RLS + TO crm_api/crm_worker-Policies (DROP+RECREATE), Rollen-Härtung NOSUPERUSER/NOBYPASSRLS, exception-sicherer Legacy-Drop mit REASSIGN/DROP OWNED; Identity-Tabellen bleiben RLS-frei (dokumentierter Bootstrap-Contract in beiden Tests); crm_runtime-Test akzeptiert Neutralisierung statt Drop wegen Cross-DB-Grants aus restore_drill | 1b485d4 |
|
||||
| I-E-Triage | BUG-09x-Familie komplett triagiert: BUG-093 stale (Cross-Tenant-Fix 5d8c48a), BUG-095 stale (läuft grün), BUG-096 stale (Mail-Fix c291a6e 46/46), BUG-094/097/098 gefixt (siehe oben) | ✅ Alle 6 Bugs geschlossen oder als bereits erledigt nachgewiesen | f4c4a50, 1b485d4, 69d05d6 |
|
||||
| I-G-1 | BUG-022/070 Audits: npm audit = 0 vulnerabilities bereits sauber; pip-audit fand **9 known CVEs in starlette 0.46.2** (PYSEC-2026-161/248/249/1941/1942/2280/2281) — Dilemma: fastapi 0.115.x pinnt starlette<0.47.0, Fixes brauchen >=1.x | ✅ **0 pip findings**: fastapi 0.141.1 (zieht starlette ohne Obergrenze) + starlette direkt auf 1.3.1 gepinnt in requirements.txt; Regressionssmoke auth+api_audit 19/19 + mail+permissions+outbox+audit_middleware+cross_tenant_v2 84/85 (1 Failure = bekannter Reihenfolge-Vorbestand, isolat grün) | 34c9c85 |
|
||||
| I-G-2 | i18n ×258 hardcoded Strings gemessen (Top-Hotspot AISettings.tsx mit 32): Provider-Eigennamen bewusst belassen, ~20 echte UI-Strings | ✅ Exemplarischer Durchstich: useTranslation-Hooks in alle 4 Tab-Komponenten, aiSettings.*-Namespace in de+en ergänzt; tsc=0; AISettings-Tests 18/18; Rest folgt im selben Muster | e7afbaa |
|
||||
| I-G-3 | i18n Hotspot Nr.2: ProactiveAISettings.tsx (15+ deutsche Hardcodes inkl. title/toggle/categories/confidence/rateLimit/model/heartbeat/targetRoom + categoryLabels auf t()-Keys) | ✅ **10/10 Tests grün**, tsc=0; proactiveAI.*-Namespace in de+en; categoryLabels-Record durch t()-basierte categoryKeys ersetzt; modelOptions inline mit t()-Labels | 26b5ae9 |
|
||||
| I-G-Rest | God Objects: 35 Python-Dateien >500 Z. — Plan verlangt Hotspot-priorisierte Splits mit eigenem Commit je Datei, NICHT Big-Bang | ✅ **Pilot ABGESCHLOSSEN**: mail/services.py 3087→~170 Z. (**−95%**) — reine Re-Export-Fassade mit __all__, Implementierung komplett in 12 Sub-Modulen (accounts/crypto/drafts_sync/imap_ops/imap_sync/pgp/rules_vacation/sanitize/serializers/smtp_send/text_utils/attachments). Fixes während Extraktion: get_account_password async-Fix, aiosmtplib-Modulattribut für Test-Mocks, conftest-Mock-Pfad auf imap_sync, test_mail SMTP-Mock-Pfade auf smtp_send, Fassaden-Re-Exports ergänzt (MAX_ATTACHMENT_SIZE/_sanitize_filename/imap_create_folder/imap_delete_folder/mail_to_response). Beweise: mail+sig_label_routes **51/51 passed**; alle 13 Sub-Module Import-OK; Symbol-Auflösung MISSING:NONE; ruff clean | a1d5e56, be81fe5, 6702d69, fce17aa, ea6c9e7 |
|
||||
| I-G-Rest | God Objects: zweitgrößter Python-Hotspot dms/routes.py (1492 Z., 24 Routen) | ✅ **Split ABGESCHLOSSEN**: routes.py 1492→650 Z. (**−56%**) — neu: common.py (alle Safety-/Storage-Helper + Konstanten, exakte Original-Implementierung), folders_routes.py / sharing_routes.py / search_bulk_routes.py je eigener prefix-loser Router; routes.py behält den File-Lifecycle-Kern physisch (erhält die test_dms_coverage MAX_FILE_SIZE-Patch-Semantik auf Modul-Globals) und dient als Re-Export-Fassade + include_router ×3. Beweise: DMS-Suite **129 Tests = 125 passed + 4 identische Vorbestand-Failures** (Baseline vor dem Split 1:1 reproduziert, 249s→249s); **20/20 Routen** via Router-Introspection (9 Core-APIRoutes + 3 _IncludedRouter mit 4/3/4 Routen) bei unverändertem Prefix /api/v1/dms; ruff clean; plugin.py-Ladepfad (module=…routes, router_attr=router) unangetastet; keine Test-Edits | f445aa6 |
|
||||
| I-G-Rest | DMS Vorbestand-Failures ×4 (shared_with_me empty/multiple_files, Streaming CHUNK_SIZE ×2) | ✅ **ALLE 4 BEHOBEN**: Suite 125 grün + 4 failed → **129/129 PASSED** (253s). (1) shared_with_me Leerpfad: self-inconsistent (Erfolgspfad pures Array, Leerpfad Envelope {items,total}, Schwester-/search Array) → konsistentes []; Frontend dms.ts Z.196 vertraegt beide Shapes. (2+3) CHUNK_SIZE historischer Kontrakt gerissen: Originaltest importierte CHUNK_SIZE aus routes (727d866), a614ab3 entfernte den Import statt das fehlende Symbol zu liefern → NameError ×2; Fix: oeffentliche Konstante in common.py + Re-Export + restaurierte Importzeile (keine Assertion angefasst). (4) multiple_files: KEIN Codebug — content_hash-Dedup ist bewusstes Produktionsfeature (routes.py Z.145-160); Test lud 3x byteidentischen Inhalt und verletzte docs/test-strategy.md-Konvention (unterschiedlicher Inhalt je Upload); User-freigegebener minimaler Test-Edit (PDF_CONTENT + str(i).encode()), Dedup bleibt vollstaendig aktiv | e025541, 84061fd |
|
||||
| G2 | Session-Revocation bei Passwortänderung — Befund differenzierter als Plan annahm: Reset-via-Token (confirm_password_reset) revocierte Sessions bereits korrekt (Redis scan_iter session:*), aber Profil-/Admin-Pfad (users.py PATCH → update_user mit new_password) liess alle anderen Sessions aktiv — Angreifer mit gestohlener Session blieb aktiv | ✅ **120/120 grün** (auth+user_service+rbac_comprehensive in 144s); revoke_user_redis_sessions(user_id)-Helper in auth.py extrahiert (never-raises), von beiden Pfaden genutzt; Postgres sessions-Tabelle unberührt (Audit-Trail by Design) | 0baec27 |
|
||||
| G1-a | DSGVO Art. 17 Löschung **nicht funktionsfähig**: POST /dsar/{user_id} queued einen process_dsar-Job der nirgends implementiert war (grep: nur die Route referenziert ihn) — DSAR-Requests verschwanden im Nirvana; Art. 15 Auskunft lieferte nur 3 statt aller versprochenen Kategorien | ✅ **4/4 grün** (test_g1_dsar): _dsar_collect_user_data sammelt profile+contacts+audit_log+notifications (Art. 15/20); _dsar_execute_deletion führt Art. 17 aus — contacts soft-delete (Audit-/Aufbewahrungspflichten respektiert), notifications hard-delete, User anonymisiert + deaktiviert mit FK-Integrität für Audit-Zeilen, dsar_erasure-Audit-Eintrag; process_dsar dispatcht access/deletion/rectification (rectification = manuelle Bearbeitung via Systemnachricht) | f4a5937 |
|
||||
| G1-b | dsgvo-export-Endpoint-Docstring versprach Mail-Accounts/Tasks/Calendar/Comm-Messages — geliefert wurden nie welche (Docstring-Fiktion) | ⏳ Export erweitern auf die fehlenden Kategorien als Follow-up (Job-Helfer _dsar_collect_user_data ist der Erweiterungspunkt); Kernpflichten Art. 15/17 sind jetzt funktionsfähig | — |
|
||||
|
||||
**Block D ABGESCHLOSSEN** (D1–D6) — D1: alle 9 Ziel-Suites grün; D2: DateTime/SQLITE-001; D3: ARCH-051/055/056/057 + systemischer Permission-Resolver-Bug + conftest-pgvector; D4: Security-Triage (ARCH-027 verifiziert, BUG-019 = 0 echte Secrets, BUG-020 kein fixbares Finding); D5: Scanner-Triage (api_contracts -75%, plugins -100%, 371 Fehlalarme eliminiert); D6: ai_copilot deprecated + ARCH-023 No-Op. Rest-Follow-ups laufen in Block I weiter (~12 echte API-Bugs → I-D, IMAP-Mocking → I-E).
|
||||
|
||||
**Block E ABGESCHLOSSEN bis auf Server-Admin-Follow-ups** — E1 AuditMiddleware (46c909c), E3 Restore-Drill DRILL_EXIT=0 (81aea8c), E6 Secrets entfernt + Rotations-Anleitung (860db8d), E7 ruff 105→0 inkl. 8 F821-Produktionsbugs (197b0d3). ⏳ Beim User: Credential-Rotation, Actions-Runner, E2/E4/E5.
|
||||
|
||||
**Block A ABGESCHLOSSEN** — Gate A bestanden (32f63ad).
|
||||
**Block B ABGESCHLOSSEN** — Gate B bestanden (alle 5 Checks bewiesen).
|
||||
**Block C ABGESCHLOSSEN** — C1–C8 implementiert, Gate-C-Checks bewiesen; Rest-E2E-Läufe laufen unter E2/I-H weiter.
|
||||
|
||||
| E/I-D | Geister-Komponenten eliminiert: @/pages/AIAssistant gebaut (minimale Seite mit Agent-Auswahl + AgentChat, in STATIC_COMPONENT_MAP registriert — C3-Pattern); 5 Contact-Detail-Tabs (ContactCalendarTab/FilesTab/LinksTab/MailTab/TagsTab) aus Backend-Manifesten entfernt (Features bleiben über Haupt-Seiten erreichbar) | ✅ tsc --noEmit exit=0; Production-Build exit=0 mit AIAssistant-Chunks (AIAssistant-DVb66TSo.js 5.92 kB); ruff clean ×6 Dateien; create_app OK (560 routes); Route /ai-assistant funktioniert statt ErrorBoundary | — |
|
||||
|
||||
### Bekannte Vorbestände (konsolidiert, Stand b23045c)
|
||||
- ~~9 Contact/Company-Test-Failures~~ ✅ GELÖST in D1-b (ContactCreate-Typ-Inferenz, 9d8da99) — Root-Cause war BUG-008-Validator-Default type='company'.
|
||||
- ~~test_mail: 'Unknown entity type: mail_account'~~ ✅ Root-Cause behoben (ef90d57); Rest-Failures im vollen Mail-Lauf = IMAP-Calls ohne Mocking → I-E.
|
||||
- ~~Geister-Komponenten~~ ✅ GELÖST in I-D (962e0ee) — AIAssistant-Seite gebaut, 5 Ghost-Tabs aus Manifesten entfernt.
|
||||
- ~~Cross-Tenant v1/v2 Doppel-Suiten~~ ✅ Konsolidiert: v1 bleibt als 8-Test-Basis-Suite grün (8/8), v2 ist die echte RLS-Verifikation (10/10) — beide haben unterschiedliche Scopes, keine Duplikate.
|
||||
- ~~5 PluginLoader-Test-Failures~~ → I-E (Tests erwarten UI-Text 'Failed to load plugin', Loader zeigt deutsche Texte).
|
||||
- ~~BUG-099~~: workstream.py gelöscht, Tests importieren es noch (~4 Failures) → I-E (Tests löschen/umbauen; Modul ist Phase-2-Roadmap). Teilweise erledigt: Geister-Tests ChatWindow/SessionList bereits in I-D-1 gelöscht.
|
||||
- ~~~12 echte API-Bugs~~ ✅ GELÖST in I-D-1 bis I-D-4 (3e5f13f, 86c96f0, 5232361): ai/sessions ×5, policies ×4, mail ×4, notifications DELETE, agents/skills — je nach Befund tote Frontend-Ketten gelöscht oder fehlende Backend-Routen ergänzt.
|
||||
|
||||
### Handover-Hinweis für Nachfolge-Agent
|
||||
- Reparaturplan: docs/fix-plan-v3.md — **Blöcke 0/H/A/B/C/D/E done**, Block I ~60% (A/B/C/D-Cluster done, E/F/G-Rest offen), Block F komplett offen, G1/G2 offen (kanonisch in BLOCK G nach I-F-Entdoppelung)
|
||||
- Findings-Status: docs/test-bugs.md (✅/⏳ je Finding)
|
||||
- Verifikationsmuster: Stash-Test gegen Pre-Block-Commit für Vorbestands-Nachweis; Endpoint-Diff via OpenAPI-Snapshot; Cross-Plugin-Scan als Gate
|
||||
- Test-DB: .env.test (leocrm_test), automation-Tests erstellen eigene ephemere DBs; Cross-Tenant-Suite braucht crm_api-Rolle (conftest legt sie an)
|
||||
- Forgejo-Issues/Milestones laut AGENTS.md §9 noch NICHT angelegt — nur PROGRESS.md-Tracking
|
||||
- Server-Admin-Follow-ups beim User: Credential-Rotation ×7, Actions-Runner + Branch-Protection, E2/E4/E5
|
||||
|
||||
**Offen gesamt:** Block I-Rest (D-API-Bugs, E Mail-Mocking+PluginLoader+BUG-099, G God Objects/i18n/Audits, H Prozess-Gates), Block G (G1 DSGVO ⚠️ KRITISCH, G2 Session-Revocation), Block F (F1 Rollback, F2 No-Touch, F3 Guide ⚠️ Pflicht).
|
||||
**Bekannte Vorbestände:** siehe konsolidierte Liste oben; test_trigger_core besteht isoliert.
|
||||
|
||||
---
|
||||
|
||||
## Übersicht
|
||||
|
||||
| Phase | Status | Start | Ende | Done | Partial | Not Done | Total | Anmerkung |
|
||||
|
||||
@@ -1,348 +0,0 @@
|
||||
# LeoCRM UI-Overhaul-Plan (v2)
|
||||
|
||||
> **Erstellt:** 2026-08-21
|
||||
> **Aktualisiert:** 2026-08-21 — AI Assistent Integration hinzugefügt
|
||||
> **Status:** Planung — nicht gestartet
|
||||
> **Leitlinie:** Auf bestehendem Code aufbauen, 3-Spalten-Explorer-Layout als Standard, keine parallelen Systeme
|
||||
|
||||
---
|
||||
|
||||
## Standard-Layout (Referenz: ContactsList.tsx)
|
||||
|
||||
Alle Explorer-Plugins nutzen das 3-Spalten-Layout aus den UI-Design-Guidelines:
|
||||
|
||||
```
|
||||
┌─────────────┬──────────────────┬──────────────────────┐
|
||||
│ Tree │ Liste/Ansicht │ Detail │
|
||||
│ (224px) │ (flex-1) │ (flex-1 / 60%) │
|
||||
│ ResizablePanel│ ResizablePanel │ ResizablePanel │
|
||||
└─────────────┴──────────────────┴──────────────────────┘
|
||||
```
|
||||
|
||||
- **Toolbar oben:** PluginToolbar mit Filter-Dropdowns, Ansichts-Umschaltern, Aktion-Buttons
|
||||
- **Linke Spalte:** ResizablePanel mit Baumansicht (Ordner, Kategorien, Kalender)
|
||||
- **Mitte:** Liste, Karten, Kalender-Ansicht — mehrere Ansichten umschaltbar
|
||||
- **Rechts:** Detail-Bereich für ausgewähltes Element
|
||||
|
||||
---
|
||||
|
||||
## Phase 1: Echte Bugs fixen (2-3 Tage)
|
||||
|
||||
### 1.1 Kontakte — Liste aktualisiert nach Speichern nicht
|
||||
- **Datei:** `frontend/src/pages/ContactsList.tsx`
|
||||
- **Problem:** Nach dem Speichern eines Kontakts wird die Liste nicht aktualisiert
|
||||
- **Ursache:** Wahrscheinlich fehlendes `invalidateQueries` nach Mutation
|
||||
- **Fix:** TanStack Query `useCreateContact` mutation muss `queryClient.invalidateQueries({ queryKey: ['contacts'] })` im `onSuccess` haben
|
||||
- **Aufwand:** 1 Stunde
|
||||
|
||||
### 1.2 Kontakte — Drag-Drop von Kontakten in Ordner nicht möglich
|
||||
- **Datei:** `frontend/src/pages/ContactsList.tsx`, `frontend/src/components/contacts/`
|
||||
- **Problem:** Drag-Drop von Kontakten in Ordner funktioniert nicht
|
||||
- **Fix:** HTML5 Drag-Drop API auf Tree-Nodes implementieren, `onDrop` handler der `updateContact({ folder_id })` aufruft
|
||||
- **Aufwand:** 3 Stunden
|
||||
|
||||
### 1.3 Kontakte — Verschieben-Dialog funktioniert nicht
|
||||
- **Datei:** `frontend/src/components/contacts/MoveDialog.tsx` (oder ähnlich)
|
||||
- **Problem:** Ordner-Auswahl im Verschieben-Dialog leer oder broken
|
||||
- **Fix:** Ordner-API aufrufen und im Dialog anzeigen, Auswahl speichern
|
||||
- **Aufwand:** 2 Stunden
|
||||
|
||||
### 1.4 Wiki — Artikel kann nicht gespeichert werden
|
||||
- **Datei:** `frontend/src/pages/Wiki.tsx`, `frontend/src/api/knowledge.ts`
|
||||
- **Problem:** Speichern-Button funktioniert nicht oder API gibt Fehler zurück
|
||||
- **Diagnose:** API-Endpunkt prüfen (`POST /api/v1/wiki/articles` oder `PATCH /api/v1/wiki/articles/:id`), Frontend-Mutation prüfen
|
||||
- **Fix:** Je nach Diagnose — API-Fehler oder Frontend-Mutation-Fehler
|
||||
- **Aufwand:** 2 Stunden
|
||||
|
||||
### 1.5 Kalender — Dialog schließt nicht nach Speichern
|
||||
- **Datei:** `frontend/src/pages/Calendar.tsx`, `frontend/src/components/calendar/AppointmentEditForm.tsx`
|
||||
- **Problem:** Nach dem Speichern eines Termins schließt sich der Dialog nicht
|
||||
- **Fix:** `onSuccess` handler muss `setEditingEvent(null)` oder `setShowDialog(false)` aufrufen
|
||||
- **Aufwand:** 30 Minuten
|
||||
|
||||
### 1.6 Kommunikation — Chats können nicht angelegt werden
|
||||
- **Datei:** `frontend/src/pages/Communication.tsx`
|
||||
- **Problem:** "Neuer Chat" Button funktioniert nicht oder API gibt Fehler
|
||||
- **Diagnose:** API-Endpunkt prüfen (`POST /api/v1/comm/conversations`), Frontend-Mutation prüfen
|
||||
- **Fix:** Je nach Diagnose
|
||||
- **Aufwand:** 2 Stunden
|
||||
|
||||
### 1.7 Wiki — Doppelt im Menü
|
||||
- **Datei:** `frontend/src/routes/index.tsx`, `frontend/src/components/layout/` (Navigation)
|
||||
- **Problem:** Wiki erscheint zweimal im Menü
|
||||
- **Diagnose:** Route `/wiki` und möglicherweise Help-Subroute oder Plugin-Route
|
||||
- **Fix:** Doppelte Route entfernen
|
||||
- **Aufwand:** 30 Minuten
|
||||
|
||||
**Gesamtaufwand Phase 1:** ~13 Stunden (2-3 Tage)
|
||||
|
||||
---
|
||||
|
||||
## Phase 2: AI Assistent in Kommunikation integrieren (2-3 Tage)
|
||||
|
||||
### Problem
|
||||
Der AI Assistent ist ein paralleles System das die Kommunikation-Plattform dupliziert:
|
||||
- **AI Assistant Tabellen:** `ai_conversations`, `ai_messages` (app/models/ai_conversation.py) + `ai_chat_sessions`, `ai_chat_messages`, `ai_chat_attachments` (app/plugins/builtins/ai_assistant/models.py) — 5 Tabellen
|
||||
- **AI Assistant Frontend:** `AIAssistant.tsx`, `AIAssistantStandalone.tsx`, `SessionList.tsx`, `ChatWindow.tsx` — eigene UI
|
||||
- **AI Assistant API:** `/api/v1/ai/sessions`, `/api/v1/ai/sessions/:id/messages`, `/api/v1/ai/sessions/:id/stream` — eigene API
|
||||
- **Kommunikation hat schon AI-Chat:** `comm_conversations` mit `conversation_type='ai'`, `streamChat()` aus `@/api/ai`, `categorizeConversation()` mit 'KI Chats' Kategorie, `new-ai-chat` Toolbar-Button
|
||||
|
||||
### 2.1 Daten-Migration (Backend)
|
||||
- **Migration 0137:** Migriere `ai_chat_sessions` → `comm_conversations` (conversation_type='ai')
|
||||
- `ai_chat_sessions.id` → `comm_conversations.id`
|
||||
- `ai_chat_sessions.title` → `comm_conversations.title`
|
||||
- `ai_chat_sessions.tenant_id` → `comm_conversations.tenant_id`
|
||||
- `ai_chat_sessions.user_id` → `comm_conversations.owner_id`
|
||||
- `ai_chat_sessions.agent_id` → `comm_conversations.metadata.agent_id`
|
||||
- `ai_chat_sessions.created_at` → `comm_conversations.created_at`
|
||||
- **Migration 0137:** Migriere `ai_chat_messages` → `comm_messages`
|
||||
- `ai_chat_messages.id` → `comm_messages.id`
|
||||
- `ai_chat_messages.session_id` → `comm_messages.conversation_id`
|
||||
- `ai_chat_messages.role` → `comm_messages.sender_type` ('user' → 'user', 'assistant' → 'ai')
|
||||
- `ai_chat_messages.content` → `comm_messages.content`
|
||||
- `ai_chat_messages.tenant_id` → `comm_messages.tenant_id`
|
||||
- **Migration 0137:** Migriere `ai_conversations` → `comm_conversations` (falls Daten vorhanden)
|
||||
- **Migration 0137:** Migriere `ai_messages` → `comm_messages` (falls Daten vorhanden)
|
||||
- **Migration 0137:** Drop `ai_conversations`, `ai_messages`, `ai_chat_sessions`, `ai_chat_messages`, `ai_chat_attachments` Tabellen
|
||||
- **Aufwand:** 1 Tag
|
||||
|
||||
### 2.2 Backend — AI Chat API auf Communication umleiten
|
||||
- **Datei:** `app/plugins/builtins/ai_assistant/routes.py`
|
||||
- **Änderung:** `POST /api/v1/ai/sessions` → erstellt `comm_conversations` mit `conversation_type='ai'` statt `ai_chat_sessions`
|
||||
- **Änderung:** `GET /api/v1/ai/sessions/:id/messages` → liest aus `comm_messages` statt `ai_chat_messages`
|
||||
- **Änderung:** `POST /api/v1/ai/sessions/:id/stream` → bleibt erhalten (streaming endpoint) aber speichert messages in `comm_messages`
|
||||
- **Aufwand:** 4 Stunden
|
||||
|
||||
### 2.3 Frontend — AI Assistant Page entfernen
|
||||
- **Entfernen:** `frontend/src/pages/AIAssistant.tsx`
|
||||
- **Entfernen:** `frontend/src/pages/AIAssistantStandalone.tsx`
|
||||
- **Entfernen:** `frontend/src/components/ai/SessionList.tsx`
|
||||
- **Entfernen:** `frontend/src/components/ai/ChatWindow.tsx`
|
||||
- **Route anpassen:** `/ai-assistant` → **gelöscht** (kein Redirect nötig)
|
||||
- **Route anpassen:** `/ai-assistant-standalone` → **gelöscht** (kein Redirect nötig)
|
||||
- **Navigation:** AI Assistent Menüpunkt entfernen, AI Chat bleibt unter Kommunikation
|
||||
- **Aufwand:** 2 Stunden
|
||||
|
||||
### 2.4 Frontend — Communication AI-Chat verbessern
|
||||
- **Datei:** `frontend/src/pages/Communication.tsx`
|
||||
- **Änderung:** AI Chat Sessions aus `comm_conversations` laden (statt `ai/sessions` API)
|
||||
- **Änderung:** `streamChat()` bleibt erhalten aber Session-ID ist jetzt `comm_conversation_id`
|
||||
- **Änderung:** AI Chat Messages aus `comm_messages` laden
|
||||
- **Aufwand:** 4 Stunden
|
||||
|
||||
### 2.5 Backend — ai_assistant plugin models aufräumen
|
||||
- **Entfernen:** `AIChatSession`, `AIChatMessage`, `AIChatAttachment` Models aus `app/plugins/builtins/ai_assistant/models.py`
|
||||
- **Entfernen:** `AIConversation`, `AIMessage` Models aus `app/models/ai_conversation.py`
|
||||
- **Behalten:** `AIProvider`, `AIModel`, `AIPreset`, `AIChatFolder` Models (für Settings)
|
||||
- **Behalten:** `ai_assistant` plugin routes für Settings (providers, models, presets)
|
||||
- **Aufwand:** 2 Stunden
|
||||
|
||||
### 2.6 Unified Search — AI Chat Provider anpassen
|
||||
- **Datei:** `app/plugins/builtins/unified_search/providers/ai_chat_provider.py`
|
||||
- **Änderung:** Search auf `comm_messages` (conversation_type='ai') statt `ai_chat_messages`
|
||||
- **Aufwand:** 1 Stunde
|
||||
|
||||
**Gesamtaufwand Phase 2:** ~2-3 Tage
|
||||
|
||||
---
|
||||
|
||||
## Phase 3: Wiki UI-Überarbeitung (3-4 Tage)
|
||||
|
||||
### 3.1 WYSIWYG Editor
|
||||
- **Datei:** `frontend/src/components/wiki/WikiEditor.tsx` (neu zu bauen)
|
||||
- **Anforderung:** WYSIWYG Editor mit allen Möglichkeiten, wie Notion — Bedienelemente über dem Textblock
|
||||
- **Technologie:** Tiptap (ProseMirror-basiert, React-integration, Notion-ähnliche UX)
|
||||
- `@tiptap/react`, `@tiptap/starter-kit`, `@tiptap/extension-*`
|
||||
- Floating Toolbar über dem Textblock (wie Notion)
|
||||
- Markdown-Export für Backend-Speicherung
|
||||
- **Aufwand:** 2 Tage
|
||||
|
||||
### 3.2 Wiki Layout — 3-Spalten
|
||||
- **Datei:** `frontend/src/pages/Wiki.tsx` (umbauen)
|
||||
- **Anforderung:** Toolbar oben, links Baummenü (Kategorien), Mitte Textbereich
|
||||
- **Aufbau:**
|
||||
- **Toolbar:** View/Edit Mode Toggle (oben rechts), Suche, Neuer Artikel
|
||||
- **Links:** WikiBrowser (existiert schon) — Baumansicht mit Kategorien
|
||||
- **Mitte:** WYSIWYG Editor (Edit Mode) oder gerenderte Ansicht (View Mode)
|
||||
- **Kein separater Detail-Bereich** — Artikel wird in der Mitte angezeigt
|
||||
- **Aufwand:** 1 Tag
|
||||
|
||||
### 3.3 View/Edit Mode Toggle
|
||||
- **Datei:** `frontend/src/pages/Wiki.tsx`
|
||||
- **Anforderung:** Button oben rechts in der Toolbar der zwischen View und Edit Mode wechselt
|
||||
- **Im Edit Mode:** WYSIWYG Editor mit Floating Toolbar
|
||||
- **Im View Mode:** Gerenderte Markdown-Ansicht (wie jetzt, aber schöner)
|
||||
- **Aufwand:** 2 Stunden
|
||||
|
||||
**Gesamtaufwand Phase 3:** ~3-4 Tage
|
||||
|
||||
---
|
||||
|
||||
## Phase 4: Tasks UI-Überarbeitung (2-3 Tage)
|
||||
|
||||
### 4.1 Tasks Layout — 3-Spalten wie Kontakte
|
||||
- **Datei:** `frontend/src/pages/Tasks.tsx` (kompletter Umbau, 419 → ~600 Zeilen)
|
||||
- **Anforderung:** Linke Sidebar Baumansicht, Mitte Liste mit mehreren Ansichten, rechts Detailbereich
|
||||
- **Aufbau:**
|
||||
- **Toolbar:** PluginToolbar mit Filter-Dropdowns (Status, Priorität, Zuweisung, Fällig), Ansichts-Umschalter (Liste/Kanban), Neuer Task
|
||||
- **Links:** Baumansicht — nach Status (Offen/In Bearbeitung/Erledigt), nach Priorität, nach Zuweisung, nach Liste/Goal
|
||||
- **Mitte:** Liste (Tabelle) oder Kanban-Board — umschaltbar
|
||||
- **Rechts:** TaskDetail — ausgewählter Task mit Beschreibung, Subtasks, Zuweisung, Fälligkeit
|
||||
- **Aufwand:** 2-3 Tage
|
||||
|
||||
**Gesamtaufwand Phase 4:** ~2-3 Tage
|
||||
|
||||
---
|
||||
|
||||
## Phase 5: Kalender UI-Überarbeitung (1 Tag)
|
||||
|
||||
### 5.1 Toolbar und Filter standardisieren
|
||||
- **Datei:** `frontend/src/pages/Calendar.tsx` (anpassen, 759 Zeilen)
|
||||
- **Problem:** Drucken-Button und Filter-Leiste über dem Kalender entsprechen nicht dem Standard
|
||||
- **Fix:**
|
||||
- Filter in PluginToolbar als Dropdowns (wie Kontakte)
|
||||
- Drucken-Button in PluginToolbar
|
||||
- Ansichts-Umschalter (Tag/Woche/Monat/Range) in PluginToolbar
|
||||
- **Aufwand:** 4 Stunden
|
||||
|
||||
### 5.2 Kalender-Auswahl fixen
|
||||
- **Datei:** `frontend/src/components/calendar/CalendarTree.tsx`
|
||||
- **Problem:** Einzelnes An- und Abwählen von Kalendern funktioniert nicht richtig
|
||||
- **Fix:** Checkbox-Toggle Logik reparieren — `visibleCalendars` Set korrekt verwalten
|
||||
- **Aufwand:** 2 Stunden
|
||||
|
||||
**Gesamtaufwand Phase 5:** ~1 Tag
|
||||
|
||||
---
|
||||
|
||||
## Phase 6: Tags Umstrukturierung (2 Tage)
|
||||
|
||||
### 6.1 Tags in Settings verschieben
|
||||
- **Datei:** `frontend/src/pages/Tags.tsx` → `frontend/src/pages/SettingsTags.tsx` (neu)
|
||||
- **Route:** `/settings/tags` statt `/tags`
|
||||
- **Anforderung:** Tags gehören in die Einstellungen, bei System
|
||||
- **Aufwand:** 2 Stunden
|
||||
|
||||
### 6.2 Tags Baumstruktur
|
||||
- **Datei:** `frontend/src/pages/SettingsTags.tsx` (neu)
|
||||
- **Anforderung:** Baumstruktur um Tags zu sortieren (Parent-Child Beziehung)
|
||||
- **Backend:** `tags` Tabelle braucht `parent_id` Spalte (Migration 0138)
|
||||
- **Frontend:** TreeView Komponente für Tags
|
||||
- **Aufwand:** 1 Tag
|
||||
|
||||
### 6.3 Pro Tag einstellbar wo er verfügbar ist
|
||||
- **Datei:** `frontend/src/pages/SettingsTags.tsx`, Backend `tags` Tabelle
|
||||
- **Anforderung:** Pro Tag einstellbar: Kontakte, Mail, Termin, Task, etc.
|
||||
- **Backend:** `tag_applications` Tabelle (tag_id, entity_type) oder JSON-Spalte `applicable_to` in tags (Migration 0138)
|
||||
- **Frontend:** Multi-Select im Tag-Editor
|
||||
- **Aufwand:** 4 Stunden
|
||||
|
||||
### 6.4 Symbol und Farbe pro Tag
|
||||
- **Datei:** `frontend/src/pages/SettingsTags.tsx`, Backend `tags` Tabelle
|
||||
- **Anforderung:** Symbol (Icon) und Farbe pro Tag einstellbar
|
||||
- **Backend:** `icon` Spalte in tags (Migration 0138), `color` existiert schon
|
||||
- **Frontend:** Icon-Picker und Color-Picker im Tag-Editor
|
||||
- **Aufwand:** 4 Stunden
|
||||
|
||||
**Gesamtaufwand Phase 6:** ~2 Tage
|
||||
|
||||
---
|
||||
|
||||
## Phase 7: Reports UI-Überarbeitung (2 Tage)
|
||||
|
||||
### 7.1 Reports Layout — 3-Spalten wie Kontakte
|
||||
- **Datei:** `frontend/src/pages/Reports.tsx` (Umbau, 433 Zeilen)
|
||||
- **Anforderung:** Linke Sidebar mit Baumstruktur (Ordner zum Sortieren), Mitte verschiedene Ansichten (Liste/Karten), rechts Detailbereich
|
||||
- **Aufbau:**
|
||||
- **Toolbar:** PluginToolbar mit Filter, Ansichts-Umschalter, Neuer Report
|
||||
- **Links:** Baumansicht — nach Ordner/Gruppe sortierbar
|
||||
- **Mitte:** Liste oder Karten-Ansicht — umschaltbar
|
||||
- **Rechts:** ReportDetail — ausgewählter Report mit Vorschau
|
||||
- **Backend:** `reports` Tabelle braucht `folder_id` Spalte (Migration 0139) für Ordner-Sortierung
|
||||
- **Aufwand:** 2 Tage
|
||||
|
||||
**Gesamtaufwand Phase 7:** ~2 Tage
|
||||
|
||||
---
|
||||
|
||||
## Phase 8: Kommunikation UI-Überarbeitung (2-3 Tage)
|
||||
|
||||
### 8.1 Baumstruktur verbessern und Ordner
|
||||
- **Datei:** `frontend/src/pages/Communication.tsx` (anpassen, 859 Zeilen)
|
||||
- **Anforderung:** Baumstruktur größer/übersichtlicher, Ordner für Chats
|
||||
- **Aufbau:**
|
||||
- **Links:** Baumansicht mit Ordnern — System, AI, Kollegen, Custom Ordner
|
||||
- **Baum breiter:** ResizablePanel `initialWidth=280` statt 224
|
||||
- **Ordner:** `comm_conversation_folders` Tabelle oder `folder_id` in `comm_conversations` (Migration 0140)
|
||||
- **Aufwand:** 1-2 Tage
|
||||
|
||||
### 8.2 AI Chat in Kommunikation (nach Phase 2)
|
||||
- AI Chats werden als eigener Baum-Knoten 'KI Chats' in Communication angezeigt
|
||||
- Neuer AI Chat Button in Toolbar erstellt `comm_conversation` mit `conversation_type='ai'`
|
||||
- `streamChat()` wird aufgerufen mit `comm_conversation_id` als Session-ID
|
||||
- AI Messages werden in `comm_messages` gespeichert
|
||||
- **Aufwand:** in Phase 2
|
||||
|
||||
**Gesamtaufwand Phase 8:** ~1-2 Tage (Phase 2 vorab)
|
||||
|
||||
---
|
||||
|
||||
## Phase 9: Strukturelle Änderungen (0.5 Tage)
|
||||
|
||||
### 9.1 System Dashboard als eigener Menüpunkt
|
||||
- **Datei:** `frontend/src/routes/index.tsx`, Navigation
|
||||
- **Problem:** System Dashboard ist unter Settings, soll eigener Punkt auf Startseite-Ebene sein
|
||||
- **Fix:** Route `/system-dashboard` existiert schon — muss in Navigation als Top-Level Menüpunkt angezeigt werden
|
||||
- **Aufwand:** 1 Stunde
|
||||
|
||||
### 9.2 Mail — Postfach mit IMAP anlegen testen
|
||||
- **Datei:** `frontend/src/pages/Mail.tsx`, `frontend/src/pages/MailSettings.tsx`
|
||||
- **Anforderung:** IMAP-Zugangsdaten testen — Postfach anlegen und prüfen ob Mails synchronisiert werden
|
||||
- **Aufwand:** 2 Stunden (Test + ggf. Bugfix)
|
||||
|
||||
**Gesamtaufwand Phase 9:** ~0.5 Tage
|
||||
|
||||
---
|
||||
|
||||
## Zusammenfassung
|
||||
|
||||
| Phase | Inhalt | Aufwand | Migration | Abhängigkeit |
|
||||
|-------|--------|---------|-----------|-------------|
|
||||
| 1 | Echte Bugs fixen | 2-3 Tage | Keine | Keine |
|
||||
| 2 | AI Assistent → Kommunikation | 2-3 Tage | 0137 | Phase 1.6 |
|
||||
| 3 | Wiki UI + WYSIWYG | 3-4 Tage | Keine | Phase 1.4 |
|
||||
| 4 | Tasks UI neu | 2-3 Tage | Keine | Keine |
|
||||
| 5 | Kalender UI | 1 Tag | Keine | Phase 1.5 |
|
||||
| 6 | Tags Umstrukturierung | 2 Tage | 0138 | Keine |
|
||||
| 7 | Reports UI | 2 Tage | 0139 | Keine |
|
||||
| 8 | Kommunikation UI | 1-2 Tage | 0140 | Phase 2 |
|
||||
| 9 | Strukturelle Änderungen | 0.5 Tage | Keine | Keine |
|
||||
|
||||
**Gesamtaufwand:** ~17-22 Tage
|
||||
|
||||
### Reihenfolge:
|
||||
1. **Phase 1** (Bugs) — zuerst, damit grundlegende Funktionen arbeiten
|
||||
2. **Phase 9** (Strukturelle Änderungen) — schnell, wenig Aufwand
|
||||
3. **Phase 5** (Kalender) — kleines Update, baut auf Phase 1 auf
|
||||
4. **Phase 2** (AI Assistent → Kommunikation) — entfernt paralleles System, baut auf Phase 1.6 auf
|
||||
5. **Phase 6** (Tags) — unabhängig, Backend + Frontend
|
||||
6. **Phase 4** (Tasks) — großer Umbau, unabhängig
|
||||
7. **Phase 3** (Wiki) — größter Umbau (WYSIWYG Editor), baut auf Phase 1 auf
|
||||
8. **Phase 7** (Reports) — großer Umbau, unabhängig
|
||||
9. **Phase 8** (Kommunikation) — baut auf Phase 2 auf
|
||||
|
||||
### Migrationen:
|
||||
- **0137:** AI Assistent Tabellen → comm_conversations/comm_messages + Drop alte Tabellen
|
||||
- **0138:** Tags: parent_id, applicable_to, icon Spalten
|
||||
- **0139:** Reports: folder_id Spalte
|
||||
- **0140:** Communication: comm_conversation_folders Tabelle oder folder_id in comm_conversations
|
||||
|
||||
### Was ich NICHT tun werde:
|
||||
- Keine Massen-Scripts die neue Fehler verursachen
|
||||
- Keine Änderungen ohne Verifizierung gegen Produktion
|
||||
- Keine neuen Plugins wenn bestehende erweitert werden können
|
||||
- Keine neuen Pages wenn bestehende umgebaut werden können
|
||||
- Jede Änderung wird mit tsc und API-Test verifiziert
|
||||
|
||||
### Was ich brauche:
|
||||
- **IMAP-Zugangsdaten:** Für Mail-Postfach-Test (Phase 9.2)
|
||||
@@ -18,7 +18,26 @@ branch_labels = None
|
||||
depends_on = None
|
||||
|
||||
|
||||
def _table_exists(conn, table_name: str) -> bool:
|
||||
"""True when the table exists (dual-path convergence, Gate B).
|
||||
|
||||
On a fresh install the ai_assistant plugin SQL migration has not run
|
||||
yet when Alembic reaches this revision — skip instead of failing.
|
||||
The plugin-side migration adds the same columns idempotently.
|
||||
"""
|
||||
row = conn.execute(
|
||||
sa.text("SELECT to_regclass(:tname) IS NOT NULL"),
|
||||
{"tname": f"public.{table_name}"},
|
||||
).scalar()
|
||||
return bool(row)
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
conn = op.get_bind()
|
||||
if not _table_exists(conn, "ai_providers"):
|
||||
# Fresh-install path: table arrives with the ai_assistant plugin
|
||||
# migration, which includes these columns.
|
||||
return
|
||||
op.add_column("ai_providers", sa.Column("region", sa.String(20), nullable=False, server_default="unknown"))
|
||||
op.add_column("ai_providers", sa.Column("hosting_type", sa.String(30), nullable=False, server_default="cloud"))
|
||||
op.add_column("ai_providers", sa.Column("dpa_status", sa.String(20), nullable=False, server_default="none"))
|
||||
@@ -29,6 +48,9 @@ def upgrade() -> None:
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
conn = op.get_bind()
|
||||
if not _table_exists(conn, "ai_providers"):
|
||||
return
|
||||
op.drop_column("ai_providers", "allowed_data_classes")
|
||||
op.drop_column("ai_providers", "transfer_notice")
|
||||
op.drop_column("ai_providers", "training_on_customer_data")
|
||||
|
||||
@@ -17,7 +17,23 @@ branch_labels = None
|
||||
depends_on = None
|
||||
|
||||
|
||||
def _table_exists(conn, table_name: str) -> bool:
|
||||
"""True when the table exists (dual-path convergence, Gate B).
|
||||
|
||||
On a fresh install the kommunikation plugin SQL migration has not run
|
||||
yet when Alembic reaches this revision — skip the comm_* parts instead
|
||||
of failing. The plugin-side migration adds the same column idempotently.
|
||||
"""
|
||||
row = conn.execute(
|
||||
sa.text("SELECT to_regclass(:tname) IS NOT NULL"),
|
||||
{"tname": f"public.{table_name}"},
|
||||
).scalar()
|
||||
return bool(row)
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
conn = op.get_bind()
|
||||
if _table_exists(conn, "comm_conversations"):
|
||||
# 1. Add is_system column to comm_conversations
|
||||
op.add_column(
|
||||
"comm_conversations",
|
||||
@@ -130,13 +146,16 @@ def upgrade() -> None:
|
||||
AND n.deleted_at IS NULL;
|
||||
""")
|
||||
|
||||
# 7. Create legacy view over notifications table for backward compatibility
|
||||
# 7. Legacy view over the CORE notifications table — exists on both paths
|
||||
op.execute("DROP VIEW IF EXISTS notifications_legacy")
|
||||
op.execute("CREATE VIEW notifications_legacy AS SELECT * FROM notifications")
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
conn = op.get_bind()
|
||||
op.execute("DROP VIEW IF EXISTS notifications_legacy")
|
||||
if not _table_exists(conn, "comm_conversations"):
|
||||
return
|
||||
op.execute("DELETE FROM comm_message_blocks WHERE message_id IN (SELECT id FROM comm_messages WHERE metadata->>'migrated_from_notification' = 'true')")
|
||||
op.execute("DELETE FROM comm_messages WHERE metadata->>'migrated_from_notification' = 'true'")
|
||||
op.execute("DELETE FROM comm_conversations WHERE is_system = true AND title = 'System Channel'")
|
||||
|
||||
@@ -14,7 +14,24 @@ branch_labels = None
|
||||
depends_on = None
|
||||
|
||||
|
||||
def _table_exists(conn, table_name: str) -> bool:
|
||||
"""True when the table exists (dual-path convergence, Gate B).
|
||||
|
||||
On a fresh install the automation plugin SQL migration has not run yet
|
||||
when Alembic reaches this revision — skip instead of failing. The
|
||||
plugin-side convergence migration creates the same table.
|
||||
"""
|
||||
row = conn.execute(
|
||||
sa.text("SELECT to_regclass(:tname) IS NOT NULL"),
|
||||
{"tname": f"public.{table_name}"},
|
||||
).scalar()
|
||||
return bool(row)
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
conn = op.get_bind()
|
||||
if not _table_exists(conn, "automation_agent_runs"):
|
||||
return
|
||||
op.create_table(
|
||||
"automation_agent_run_steps",
|
||||
sa.Column("id", PGUUID(as_uuid=True), primary_key=True),
|
||||
|
||||
@@ -18,7 +18,24 @@ branch_labels = None
|
||||
depends_on = None
|
||||
|
||||
|
||||
def _table_exists(conn, table_name: str) -> bool:
|
||||
"""True when the table exists (dual-path convergence, Gate B).
|
||||
|
||||
On a fresh install the automation plugin SQL migration has not run yet
|
||||
when Alembic reaches this revision — skip instead of failing. The
|
||||
plugin-side convergence migration adds the same columns.
|
||||
"""
|
||||
row = conn.execute(
|
||||
sa.text("SELECT to_regclass(:tname) IS NOT NULL"),
|
||||
{"tname": f"public.{table_name}"},
|
||||
).scalar()
|
||||
return bool(row)
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
conn = op.get_bind()
|
||||
if not _table_exists(conn, "automation_agent_definitions"):
|
||||
return
|
||||
op.add_column(
|
||||
"automation_agent_definitions",
|
||||
sa.Column("temperature", sa.Float, nullable=False, server_default="0.3"),
|
||||
|
||||
@@ -20,7 +20,27 @@ branch_labels = None
|
||||
depends_on = None
|
||||
|
||||
|
||||
def _table_exists(conn, table_name: str) -> bool:
|
||||
"""True when the table exists (dual-path convergence, Gate B).
|
||||
|
||||
On a fresh install the tasks plugin SQL migration has not run yet when
|
||||
Alembic reaches this revision — skip instead of failing. The plugin-side
|
||||
convergence migration adds the same columns/indexes. The legacy-data
|
||||
backfills below only matter for pre-existing rows and are correctly
|
||||
empty on a fresh install.
|
||||
"""
|
||||
row = conn.execute(
|
||||
sa.text("SELECT to_regclass(:tname) IS NOT NULL"),
|
||||
{"tname": f"public.{table_name}"},
|
||||
).scalar()
|
||||
return bool(row)
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
conn = op.get_bind()
|
||||
if not _table_exists(conn, "tasks"):
|
||||
return
|
||||
|
||||
# ── Add new columns to tasks ────────────────────────────────────────────
|
||||
op.add_column("tasks", sa.Column("assignee_type", sa.String(20), nullable=False, server_default="user"))
|
||||
op.add_column("tasks", sa.Column("assignee_id", PGUUID(as_uuid=True), nullable=True))
|
||||
|
||||
@@ -12,6 +12,7 @@ Revises: 0126
|
||||
"""
|
||||
|
||||
from alembic import op
|
||||
import sqlalchemy as sa
|
||||
|
||||
revision = "0127"
|
||||
down_revision = "0126"
|
||||
@@ -19,7 +20,19 @@ branch_labels = None
|
||||
depends_on = None
|
||||
|
||||
|
||||
def _table_exists(conn, table_name: str) -> bool:
|
||||
"""True when the table exists (dual-path convergence, Gate B)."""
|
||||
row = conn.execute(
|
||||
sa.text("SELECT to_regclass(:tname) IS NOT NULL"),
|
||||
{"tname": f"public.{table_name}"},
|
||||
).scalar()
|
||||
return bool(row)
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
conn = op.get_bind()
|
||||
if not _table_exists(conn, "tasks"):
|
||||
return
|
||||
# Drop the FK constraint on tasks.contact_id
|
||||
op.drop_constraint("tasks_contact_id_fkey", "tasks", type_="foreignkey")
|
||||
|
||||
|
||||
@@ -9,6 +9,7 @@ Revises: 0128
|
||||
"""
|
||||
|
||||
from alembic import op
|
||||
import sqlalchemy as sa
|
||||
|
||||
revision = "0129"
|
||||
down_revision = "0128"
|
||||
@@ -27,8 +28,25 @@ TABLES_NEEDING_RLS = [
|
||||
]
|
||||
|
||||
|
||||
def _table_exists(conn, table_name: str) -> bool:
|
||||
"""True when the table exists (dual-path convergence, Gate B).
|
||||
|
||||
Plugin-owned tables may not exist yet on a fresh install when Alembic
|
||||
reaches this revision — skip them instead of failing. The plugin-side
|
||||
convergence migrations apply the same RLS policies.
|
||||
"""
|
||||
row = conn.execute(
|
||||
sa.text("SELECT to_regclass(:tname) IS NOT NULL"),
|
||||
{"tname": f"public.{table_name}"},
|
||||
).scalar()
|
||||
return bool(row)
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
conn = op.get_bind()
|
||||
for table in TABLES_NEEDING_RLS:
|
||||
if not _table_exists(conn, table):
|
||||
continue
|
||||
op.execute(f"ALTER TABLE {table} ENABLE ROW LEVEL SECURITY;")
|
||||
op.execute(
|
||||
f"CREATE POLICY tenant_isolation ON {table} "
|
||||
@@ -37,6 +55,9 @@ def upgrade() -> None:
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
conn = op.get_bind()
|
||||
for table in TABLES_NEEDING_RLS:
|
||||
if not _table_exists(conn, table):
|
||||
continue
|
||||
op.execute(f"DROP POLICY IF EXISTS tenant_isolation ON {table};")
|
||||
op.execute(f"ALTER TABLE {table} DISABLE ROW LEVEL SECURITY;")
|
||||
|
||||
@@ -10,6 +10,7 @@ Revises: 0135
|
||||
Create Date: 2026-08-21
|
||||
"""
|
||||
from alembic import op
|
||||
import sqlalchemy as sa
|
||||
|
||||
revision = "0136"
|
||||
down_revision = "0135"
|
||||
@@ -29,8 +30,25 @@ TABLES_WITH_BAD_RLS = [
|
||||
]
|
||||
|
||||
|
||||
def _table_exists(conn, table_name: str) -> bool:
|
||||
"""True when the table exists (dual-path convergence, Gate B).
|
||||
|
||||
Plugin-owned tables may not exist yet on a fresh install when Alembic
|
||||
reaches this revision — skip them instead of failing. The plugin-side
|
||||
convergence migrations apply the same RLS policies.
|
||||
"""
|
||||
row = conn.execute(
|
||||
sa.text("SELECT to_regclass(:tname) IS NOT NULL"),
|
||||
{"tname": f"public.{table_name}"},
|
||||
).scalar()
|
||||
return bool(row)
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
conn = op.get_bind()
|
||||
for table in TABLES_WITH_BAD_RLS:
|
||||
if not _table_exists(conn, table):
|
||||
continue
|
||||
# Drop old policy with app.tenant_id
|
||||
op.execute(f"DROP POLICY IF EXISTS tenant_isolation ON {table};")
|
||||
# Create new policy with app.current_tenant_id
|
||||
@@ -41,7 +59,10 @@ def upgrade() -> None:
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
conn = op.get_bind()
|
||||
for table in TABLES_WITH_BAD_RLS:
|
||||
if not _table_exists(conn, table):
|
||||
continue
|
||||
op.execute(f"DROP POLICY IF EXISTS tenant_isolation ON {table};")
|
||||
op.execute(
|
||||
f"CREATE POLICY tenant_isolation ON {table} "
|
||||
|
||||
@@ -19,7 +19,25 @@ branch_labels = None
|
||||
depends_on = None
|
||||
|
||||
|
||||
def _table_exists(conn, table_name: str) -> bool:
|
||||
"""True when the table exists (dual-path convergence, Gate B).
|
||||
|
||||
On a fresh install the tags plugin SQL migration has not run yet when
|
||||
Alembic reaches this revision — skip instead of failing. The plugin-side
|
||||
convergence migration adds the same columns.
|
||||
"""
|
||||
row = conn.execute(
|
||||
sa.text("SELECT to_regclass(:tname) IS NOT NULL"),
|
||||
{"tname": f"public.{table_name}"},
|
||||
).scalar()
|
||||
return bool(row)
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
conn = op.get_bind()
|
||||
if not _table_exists(conn, "tags"):
|
||||
return
|
||||
|
||||
# parent_id for tree structure (self-referencing FK)
|
||||
op.add_column("tags", sa.Column("parent_id", PGUUID(as_uuid=True), nullable=True))
|
||||
op.create_foreign_key(
|
||||
@@ -35,6 +53,9 @@ def upgrade() -> None:
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
conn = op.get_bind()
|
||||
if not _table_exists(conn, "tags"):
|
||||
return
|
||||
op.drop_column("tags", "icon")
|
||||
op.drop_column("tags", "applicable_to")
|
||||
op.drop_index("ix_tags_parent", table_name="tags")
|
||||
|
||||
@@ -18,11 +18,31 @@ branch_labels = None
|
||||
depends_on = None
|
||||
|
||||
|
||||
def _table_exists(conn, table_name: str) -> bool:
|
||||
"""True when the table exists (dual-path convergence, Gate B).
|
||||
|
||||
On a fresh install the report_generator plugin SQL migration has not
|
||||
run yet when Alembic reaches this revision — skip instead of failing.
|
||||
The plugin-side convergence migration adds the same column.
|
||||
"""
|
||||
row = conn.execute(
|
||||
sa.text("SELECT to_regclass(:tname) IS NOT NULL"),
|
||||
{"tname": f"public.{table_name}"},
|
||||
).scalar()
|
||||
return bool(row)
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
conn = op.get_bind()
|
||||
if not _table_exists(conn, "report_templates"):
|
||||
return
|
||||
op.add_column("report_templates", sa.Column("folder_id", PGUUID(as_uuid=True), nullable=True))
|
||||
op.create_index("ix_report_templates_folder", "report_templates", ["folder_id"])
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
conn = op.get_bind()
|
||||
if not _table_exists(conn, "report_templates"):
|
||||
return
|
||||
op.drop_index("ix_report_templates_folder", table_name="report_templates")
|
||||
op.drop_column("report_templates", "folder_id")
|
||||
|
||||
@@ -18,11 +18,31 @@ branch_labels = None
|
||||
depends_on = None
|
||||
|
||||
|
||||
def _table_exists(conn, table_name: str) -> bool:
|
||||
"""True when the table exists (dual-path convergence, Gate B).
|
||||
|
||||
On a fresh install the kommunikation plugin SQL migration has not run
|
||||
yet when Alembic reaches this revision — skip instead of failing.
|
||||
The plugin-side migration adds the same column idempotently.
|
||||
"""
|
||||
row = conn.execute(
|
||||
sa.text("SELECT to_regclass(:tname) IS NOT NULL"),
|
||||
{"tname": f"public.{table_name}"},
|
||||
).scalar()
|
||||
return bool(row)
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
conn = op.get_bind()
|
||||
if not _table_exists(conn, "comm_conversations"):
|
||||
return
|
||||
op.add_column("comm_conversations", sa.Column("folder_id", PGUUID(as_uuid=True), nullable=True))
|
||||
op.create_index("ix_comm_conversations_folder", "comm_conversations", ["folder_id"])
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
conn = op.get_bind()
|
||||
if not _table_exists(conn, "comm_conversations"):
|
||||
return
|
||||
op.drop_index("ix_comm_conversations_folder", table_name="comm_conversations")
|
||||
op.drop_column("comm_conversations", "folder_id")
|
||||
|
||||
@@ -0,0 +1,84 @@
|
||||
'''Fix role permission wildcard patterns to canonical 2-segment schema
|
||||
|
||||
Revision ID: 0141
|
||||
Revises: 0140
|
||||
Create Date: 2026-08-23
|
||||
|
||||
Migration 0019 seeded default roles with 3-segment permission patterns
|
||||
(core:*:read etc.). The runtime matcher (_matches_permission) compares
|
||||
segment counts strictly, so those patterns could never match any
|
||||
2-segment requirement - editor/viewer roles were silently dead.
|
||||
|
||||
Canonical schema is module:action (2 segments, * wildcards allowed).
|
||||
core:*:X means all modules with action X, so it converts to *:X.
|
||||
'''
|
||||
|
||||
from alembic import op
|
||||
|
||||
# revision identifiers, used by Alembic.
|
||||
revision = '0141'
|
||||
down_revision = '0140'
|
||||
branch_labels = None
|
||||
depends_on = None
|
||||
|
||||
# Rebuild the permissions JSONB object, rewriting every key that starts
|
||||
# with the dead 'core:' prefix to its 2-segment equivalent ('*:X').
|
||||
_UPGRADE_SQL = '''
|
||||
UPDATE roles
|
||||
SET permissions = sub.new_perms,
|
||||
permission_version = permission_version + 1
|
||||
FROM (
|
||||
SELECT
|
||||
r.id AS role_id,
|
||||
jsonb_object_agg(
|
||||
CASE WHEN k LIKE 'core:%'
|
||||
THEN '*:' || split_part(k, ':', 3)
|
||||
ELSE k END,
|
||||
v
|
||||
) AS new_perms
|
||||
FROM roles r,
|
||||
jsonb_each(r.permissions) AS e(k, v)
|
||||
GROUP BY r.id
|
||||
) AS sub
|
||||
WHERE roles.id = sub.role_id
|
||||
AND EXISTS (
|
||||
SELECT 1 FROM jsonb_object_keys(roles.permissions) k
|
||||
WHERE k LIKE 'core:%'
|
||||
)
|
||||
'''
|
||||
|
||||
# Reverse: map '*:X' back to 'core:*:X' only for keys that came from the
|
||||
# original seeding pattern. Roles that legitimately use '*:X' without a
|
||||
# matching 'core:*:X' history are left untouched (best-effort downgrade).
|
||||
_DOWNGRADE_SQL = '''
|
||||
UPDATE roles
|
||||
SET permissions = sub.new_perms,
|
||||
permission_version = permission_version + 1
|
||||
FROM (
|
||||
SELECT
|
||||
r.id AS role_id,
|
||||
jsonb_object_agg(
|
||||
CASE WHEN k = '*:' || split_part(k, ':', 2)
|
||||
AND k <> '*:*'
|
||||
THEN 'core:*:' || split_part(k, ':', 2)
|
||||
ELSE k END,
|
||||
v
|
||||
) AS new_perms
|
||||
FROM roles r,
|
||||
jsonb_each(r.permissions) AS e(k, v)
|
||||
GROUP BY r.id
|
||||
) AS sub
|
||||
WHERE roles.id = sub.role_id
|
||||
AND EXISTS (
|
||||
SELECT 1 FROM jsonb_object_keys(roles.permissions) k
|
||||
WHERE k = '*:' || split_part(k, ':', 2) AND k <> '*:*'
|
||||
)
|
||||
'''
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
op.execute(_UPGRADE_SQL)
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
op.execute(_DOWNGRADE_SQL)
|
||||
@@ -0,0 +1,39 @@
|
||||
"""Add backup config columns to system_settings table.
|
||||
|
||||
Follow-up to 0130: the backup feature (10b1f83) added backup_interval,
|
||||
backup_retention_days and backup_destination to schema/service/frontend
|
||||
but missed model columns and this migration.
|
||||
|
||||
Revision ID: 0142
|
||||
Revises: 0141
|
||||
"""
|
||||
|
||||
import sqlalchemy as sa
|
||||
|
||||
from alembic import op
|
||||
|
||||
revision = "0142"
|
||||
down_revision = "0141"
|
||||
branch_labels = None
|
||||
depends_on = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
op.add_column(
|
||||
"system_settings",
|
||||
sa.Column("backup_interval", sa.String(20), nullable=False, server_default="daily"),
|
||||
)
|
||||
op.add_column(
|
||||
"system_settings",
|
||||
sa.Column("backup_retention_days", sa.Integer(), nullable=False, server_default="7"),
|
||||
)
|
||||
op.add_column(
|
||||
"system_settings",
|
||||
sa.Column("backup_destination", sa.String(20), nullable=False, server_default="local"),
|
||||
)
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
op.drop_column("system_settings", "backup_destination")
|
||||
op.drop_column("system_settings", "backup_retention_days")
|
||||
op.drop_column("system_settings", "backup_interval")
|
||||
@@ -17,7 +17,7 @@ from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import uuid
|
||||
from dataclasses import dataclass, field
|
||||
from dataclasses import dataclass
|
||||
from typing import Any
|
||||
|
||||
from sqlalchemy import select
|
||||
|
||||
@@ -25,8 +25,6 @@ import logging
|
||||
import uuid
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
from app.core.sensitive_data import sanitize_dict
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
|
||||
|
||||
@@ -20,8 +20,6 @@ from sqlalchemy.ext.asyncio import AsyncSession
|
||||
from app.ai.ai_use_case import AIUseCaseMetadata
|
||||
from app.core.sensitive_data import (
|
||||
SENSITIVE_FIELDS,
|
||||
filter_for_llm_context,
|
||||
get_data_class_for_field,
|
||||
)
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
@@ -6,7 +6,6 @@ import uuid
|
||||
from dataclasses import dataclass, field
|
||||
from typing import Any
|
||||
|
||||
|
||||
LOW_CONFIDENCE_THRESHOLD = 0.6
|
||||
|
||||
|
||||
|
||||
@@ -7,7 +7,6 @@ from typing import Any
|
||||
|
||||
from app.ai.knowledge_sources import get_source_config
|
||||
|
||||
|
||||
EXTRACTION_TRIGGERS = {
|
||||
"mail.received",
|
||||
"dms.file_uploaded",
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass, field
|
||||
from dataclasses import dataclass
|
||||
from typing import Any
|
||||
|
||||
|
||||
|
||||
@@ -107,6 +107,10 @@ class Settings(BaseSettings):
|
||||
rate_limit_webhook_max: int = 100 # incoming webhooks
|
||||
rate_limit_webhook_window: int = 60 # 1 minute
|
||||
|
||||
# System tenant — used by seeding/plugins that need a well-known default
|
||||
# tenant (must match scripts/seed_admin.py slug).
|
||||
system_tenant_slug: str = "default"
|
||||
|
||||
# LLM Cost Overrun Protection (B.17)
|
||||
llm_monthly_budget_usd: float = 100.0 # per-tenant monthly LLM budget
|
||||
llm_hard_cutoff: bool = True # block LLM calls when budget exceeded
|
||||
|
||||
@@ -85,6 +85,38 @@ def generate_csrf_token() -> str:
|
||||
return secrets.token_urlsafe(32)
|
||||
|
||||
|
||||
async def revoke_user_redis_sessions(user_id: str | uuid.UUID) -> int:
|
||||
"""Delete every active Redis session belonging to the user (G2).
|
||||
|
||||
Shared by both password-change paths (token reset + profile/admin change):
|
||||
after a password change, stolen or lingering sessions must die.
|
||||
|
||||
Returns the number of deleted session keys. Never raises — a Redis outage
|
||||
must not break the password change itself.
|
||||
"""
|
||||
try:
|
||||
redis = get_redis()
|
||||
deleted = 0
|
||||
async for key in redis.scan_iter(match="session:*", count=100):
|
||||
raw = await redis.get(key)
|
||||
if raw is None:
|
||||
continue
|
||||
try:
|
||||
import json
|
||||
|
||||
session_data = json.loads(raw)
|
||||
except (json.JSONDecodeError, TypeError):
|
||||
continue
|
||||
if session_data.get("user_id") == str(user_id):
|
||||
await redis.delete(key)
|
||||
deleted += 1
|
||||
logger.info("Deleted session %s for user %s", key, user_id)
|
||||
return deleted
|
||||
except Exception:
|
||||
logger.warning("Failed to invalidate Redis sessions for user %s", user_id, exc_info=True)
|
||||
return 0
|
||||
|
||||
|
||||
def hash_token(token: str) -> str:
|
||||
"""SHA-256 hash a token for storage."""
|
||||
return hashlib.sha256(token.encode()).hexdigest()
|
||||
|
||||
@@ -355,6 +355,23 @@ async def close_engine() -> None:
|
||||
_migration_session_factory = None
|
||||
|
||||
|
||||
async def get_system_tenant(db: AsyncSession):
|
||||
"""Return the well-known system tenant, or ``None`` if it does not exist.
|
||||
|
||||
Resolves by configured slug (``settings.system_tenant_slug``, default
|
||||
``"default"`` as created by ``scripts/seed_admin.py``) instead of an
|
||||
arbitrary first row, so multi-tenant databases stay deterministic.
|
||||
"""
|
||||
from sqlalchemy import select
|
||||
|
||||
from app.config import get_settings
|
||||
from app.models.tenant import Tenant # lazy: models import this module's Base
|
||||
|
||||
slug = get_settings().system_tenant_slug
|
||||
result = await db.execute(select(Tenant).where(Tenant.slug == slug).limit(1))
|
||||
return result.scalar_one_or_none()
|
||||
|
||||
|
||||
def reset_engine_for_testing(engine: AsyncEngine) -> async_sessionmaker[AsyncSession]:
|
||||
"""Replace all global engines with a test engine. Returns a session factory.
|
||||
|
||||
|
||||
@@ -36,7 +36,12 @@ class EventBus:
|
||||
self._handlers: dict[str, list[EventHandler]] = defaultdict(list)
|
||||
|
||||
def subscribe(self, event_name: str, handler: EventHandler) -> None:
|
||||
"""Subscribe a handler to an event."""
|
||||
"""Subscribe a handler to an event.
|
||||
|
||||
Idempotent: subscribing the same handler twice is a no-op
|
||||
(ARCH-020) so double activation cannot fire handlers twice.
|
||||
"""
|
||||
if handler not in self._handlers[event_name]:
|
||||
self._handlers[event_name].append(handler)
|
||||
|
||||
def unsubscribe(self, event_name: str, handler: EventHandler) -> None:
|
||||
|
||||
@@ -153,3 +153,378 @@ async def send_password_reset_email(
|
||||
from app.core.job_registry import register_job # noqa: E402
|
||||
|
||||
register_job("send_password_reset_email", send_password_reset_email)
|
||||
|
||||
|
||||
# ── DSAR Processing Job (G1 DSGVO: Art. 15 Auskunft / Art. 17 Löschung) ─────
|
||||
|
||||
async def _dsar_collect_user_data(db: Any, tenant_id: str, user_id: str) -> dict[str, Any]:
|
||||
"""Collect every data category the dsgvo-export route promises.
|
||||
|
||||
Shared by type=access (full export) so both paths stay consistent.
|
||||
"""
|
||||
from datetime import UTC, datetime
|
||||
from uuid import UUID as PyUUID
|
||||
|
||||
from sqlalchemy import or_ as sa_or_
|
||||
from sqlalchemy import select as sa_select
|
||||
|
||||
from app.models.audit import AuditLog
|
||||
from app.models.contact import Contact
|
||||
from app.models.notification import Notification
|
||||
from app.models.user import User
|
||||
|
||||
uid = PyUUID(user_id)
|
||||
tid = PyUUID(tenant_id)
|
||||
|
||||
export_data: dict[str, Any] = {
|
||||
"user_id": user_id,
|
||||
"exported_at": datetime.now(UTC).isoformat(),
|
||||
"legal_basis": "GDPR Art. 15 (access) / Art. 20 (portability)",
|
||||
"data": {},
|
||||
}
|
||||
|
||||
# Profile
|
||||
user = (
|
||||
await db.execute(sa_select(User).where(User.id == uid))
|
||||
).scalar_one_or_none()
|
||||
if user:
|
||||
export_data["data"]["profile"] = {
|
||||
"email": user.email,
|
||||
"name": user.name,
|
||||
"is_active": user.is_active,
|
||||
"created_at": user.created_at.isoformat() if user.created_at else None,
|
||||
}
|
||||
|
||||
# Contacts owned by the user
|
||||
contacts = (
|
||||
await db.execute(
|
||||
sa_select(Contact).where(
|
||||
Contact.tenant_id == tid,
|
||||
Contact.owner_id == uid,
|
||||
Contact.deleted_at.is_(None),
|
||||
)
|
||||
)
|
||||
).scalars().all()
|
||||
export_data["data"]["contacts"] = [
|
||||
{
|
||||
"id": str(c.id),
|
||||
"type": c.type,
|
||||
"displayname": c.displayname,
|
||||
"email_1": c.email_1,
|
||||
"email_2": c.email_2,
|
||||
}
|
||||
for c in contacts
|
||||
]
|
||||
|
||||
# Audit trail entries by/about the user (bounded to keep payloads sane)
|
||||
audit_entries = (
|
||||
await db.execute(
|
||||
sa_select(AuditLog).where(
|
||||
AuditLog.tenant_id == tid,
|
||||
AuditLog.user_id == uid,
|
||||
).limit(1000)
|
||||
)
|
||||
).scalars().all()
|
||||
export_data["data"]["audit_log"] = [
|
||||
{
|
||||
"action": a.action,
|
||||
"entity_type": a.entity_type,
|
||||
"timestamp": a.timestamp.isoformat() if a.timestamp else None,
|
||||
}
|
||||
for a in audit_entries
|
||||
]
|
||||
|
||||
# Notifications addressed to the user
|
||||
notifications = (
|
||||
await db.execute(
|
||||
sa_select(Notification).where(
|
||||
Notification.tenant_id == tid,
|
||||
Notification.owner_id == uid,
|
||||
).limit(1000)
|
||||
)
|
||||
).scalars().all()
|
||||
export_data["data"]["notifications"] = [
|
||||
{
|
||||
"id": str(n.id),
|
||||
"type": getattr(n, "type", None),
|
||||
"title": getattr(n, "title", None),
|
||||
"created_at": n.created_at.isoformat() if n.created_at else None,
|
||||
}
|
||||
for n in notifications
|
||||
]
|
||||
|
||||
# ── Categories promised by the dsgvo-export route docstring ──
|
||||
# (G1-b: mail accounts, tasks, calendar entries, comm messages)
|
||||
try:
|
||||
from app.plugins.builtins.mail.models import MailAccount
|
||||
|
||||
mail_accounts = (
|
||||
await db.execute(
|
||||
sa_select(MailAccount).where(
|
||||
MailAccount.tenant_id == tid,
|
||||
MailAccount.user_id == uid,
|
||||
).limit(500)
|
||||
)
|
||||
).scalars().all()
|
||||
export_data["data"]["mail_accounts"] = [
|
||||
{
|
||||
"id": str(a.id),
|
||||
"email_address": a.email_address,
|
||||
"display_name": a.display_name,
|
||||
"is_shared": a.is_shared,
|
||||
"is_active": a.is_active,
|
||||
}
|
||||
for a in mail_accounts
|
||||
]
|
||||
except ImportError:
|
||||
pass
|
||||
|
||||
try:
|
||||
from app.plugins.builtins.tasks.models import Task as TaskModel
|
||||
|
||||
tasks = (
|
||||
await db.execute(
|
||||
sa_select(TaskModel).where(
|
||||
sa_or_(TaskModel.owner_id == uid, TaskModel.assigned_to == uid),
|
||||
TaskModel.tenant_id == tid,
|
||||
TaskModel.deleted_at.is_(None),
|
||||
).limit(1000)
|
||||
)
|
||||
).scalars().all()
|
||||
export_data["data"]["tasks"] = [
|
||||
{
|
||||
"id": str(t.id),
|
||||
"title": t.title,
|
||||
"status": t.status,
|
||||
"priority": t.priority,
|
||||
"due_date": t.due_date.isoformat() if t.due_date else None,
|
||||
}
|
||||
for t in tasks
|
||||
]
|
||||
except ImportError:
|
||||
pass
|
||||
|
||||
try:
|
||||
from app.plugins.builtins.calendar.models import CalendarEntry as CalEntry
|
||||
|
||||
cal_entries = (
|
||||
await db.execute(
|
||||
sa_select(CalEntry).where(
|
||||
CalEntry.tenant_id == tid,
|
||||
CalEntry.owner_id == uid,
|
||||
CalEntry.deleted_at.is_(None),
|
||||
).limit(1000)
|
||||
)
|
||||
).scalars().all()
|
||||
export_data["data"]["calendar_entries"] = [
|
||||
{
|
||||
"id": str(e.id),
|
||||
"title": e.title,
|
||||
"entry_type": e.entry_type,
|
||||
"start_at": e.start_at.isoformat() if e.start_at else None,
|
||||
"end_at": e.end_at.isoformat() if e.end_at else None,
|
||||
}
|
||||
for e in cal_entries
|
||||
]
|
||||
except ImportError:
|
||||
pass
|
||||
|
||||
try:
|
||||
from app.plugins.builtins.kommunikation.models import CommMessage
|
||||
|
||||
comm_messages = (
|
||||
await db.execute(
|
||||
sa_select(CommMessage).where(
|
||||
CommMessage.sender_id == uid,
|
||||
CommMessage.tenant_id == tid,
|
||||
).limit(1000)
|
||||
)
|
||||
).scalars().all()
|
||||
export_data["data"]["comm_messages"] = [
|
||||
{
|
||||
"id": str(m.id),
|
||||
"sender_type": m.sender_type,
|
||||
"content": m.content[:500],
|
||||
"created_at": m.created_at.isoformat() if m.created_at else None,
|
||||
}
|
||||
for m in comm_messages
|
||||
]
|
||||
except ImportError:
|
||||
pass
|
||||
|
||||
return export_data
|
||||
|
||||
|
||||
async def _dsar_execute_deletion(db: Any, tenant_id: str, user_id: str) -> dict[str, int]:
|
||||
"""Execute GDPR Art. 17 erasure for a user within one tenant.
|
||||
|
||||
Strategy (respects retention duties):
|
||||
- Contacts owned by the user → soft-delete via deleted_at
|
||||
(audit history must remain intact — it is not personal data of the
|
||||
subject but business record; retention policy governs its cleanup)
|
||||
- Notifications owned by the user → hard delete
|
||||
- User account → deactivate (is_active=False), clear personal fields,
|
||||
scramble password hash and email (keeps FK integrity for audit rows)
|
||||
Returns counters for the audit entry.
|
||||
"""
|
||||
from datetime import UTC, datetime
|
||||
from uuid import UUID as PyUUID
|
||||
|
||||
from sqlalchemy import select as sa_select
|
||||
from sqlalchemy import update as sa_update
|
||||
|
||||
from app.core.audit import log_audit
|
||||
from app.models.contact import Contact
|
||||
from app.models.notification import Notification
|
||||
from app.models.user import User
|
||||
|
||||
uid = PyUUID(user_id)
|
||||
tid = PyUUID(tenant_id)
|
||||
|
||||
counts: dict[str, int] = {}
|
||||
|
||||
# 1. Soft-delete contacts owned by the user
|
||||
contact_result = await db.execute(
|
||||
sa_select(Contact).where(
|
||||
Contact.tenant_id == tid,
|
||||
Contact.owner_id == uid,
|
||||
Contact.deleted_at.is_(None),
|
||||
)
|
||||
)
|
||||
contacts = contact_result.scalars().all()
|
||||
for c in contacts:
|
||||
c.deleted_at = datetime.now(UTC)
|
||||
counts["contacts_soft_deleted"] = len(contacts)
|
||||
|
||||
# 2. Hard-delete notifications owned by the user
|
||||
notif_result = await db.execute(
|
||||
sa_select(Notification).where(
|
||||
Notification.tenant_id == tid,
|
||||
Notification.owner_id == uid,
|
||||
)
|
||||
)
|
||||
notifications = notif_result.scalars().all()
|
||||
for n in notifications:
|
||||
await db.delete(n)
|
||||
counts["notifications_deleted"] = len(notifications)
|
||||
|
||||
# 3. Anonymize + deactivate the account (FK integrity for audit rows kept)
|
||||
await db.execute(
|
||||
sa_update(User)
|
||||
.where(User.id == uid)
|
||||
.values(
|
||||
email=f"erased.{uid.hex[:16]}@anonymized.invalid",
|
||||
name="[gelöscht gemäß DSGVO Art. 17]",
|
||||
first_name=None,
|
||||
last_name=None,
|
||||
avatar_url=None,
|
||||
password_hash="!dsar-erased",
|
||||
is_active=False,
|
||||
preferences={},
|
||||
)
|
||||
)
|
||||
counts["user_anonymized"] = 1
|
||||
|
||||
# 4. Audit the erasure itself (who/what/when — required by Art. 17 recital)
|
||||
await log_audit(
|
||||
db,
|
||||
tid,
|
||||
user_id,
|
||||
"dsar_erasure",
|
||||
"user",
|
||||
uid,
|
||||
{"target_user": user_id, **counts},
|
||||
)
|
||||
return counts
|
||||
|
||||
|
||||
async def process_dsar(
|
||||
ctx: dict[str, Any],
|
||||
*,
|
||||
user_id: str,
|
||||
tenant_id: str,
|
||||
request_type: str,
|
||||
) -> dict[str, Any]:
|
||||
"""Process a GDPR Data Subject Access Request (DSAR).
|
||||
|
||||
ARQ worker function registered as "process_dsar".
|
||||
|
||||
request_type:
|
||||
- "access": collect all data categories (Art. 15/20) and post a system
|
||||
message that the export is ready (served via the existing dsgvo-export
|
||||
endpoint).
|
||||
- "deletion": execute Art. 17 erasure (soft-delete contacts, hard-delete
|
||||
notifications, anonymize+deactivate account) and audit it.
|
||||
- "rectification": post a system message asking admins to handle the
|
||||
correction manually.
|
||||
|
||||
Returns a summary dict for the job result.
|
||||
"""
|
||||
import logging
|
||||
import uuid as uuid_module
|
||||
|
||||
from app.core.db import get_worker_session_factory
|
||||
from app.core.notifications import post_system_message
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
tid = uuid_module.UUID(tenant_id)
|
||||
uid = uuid_module.UUID(user_id)
|
||||
|
||||
factory = get_worker_session_factory()
|
||||
async with factory() as db:
|
||||
try:
|
||||
if request_type == "access":
|
||||
data = await _dsar_collect_user_data(db, tenant_id, user_id)
|
||||
await db.commit()
|
||||
categories = list(data.get("data", {}).keys())
|
||||
await post_system_message(
|
||||
db,
|
||||
tid,
|
||||
uid,
|
||||
"dsar_access_ready",
|
||||
"DSGVO-Auskunft bereit",
|
||||
f"Datenkategorien: {', '.join(categories)}",
|
||||
severity="info",
|
||||
)
|
||||
await db.commit()
|
||||
logger.info("DSAR access processed for user %s", user_id)
|
||||
return {"type": request_type, "status": "completed", "categories": categories}
|
||||
|
||||
if request_type == "deletion":
|
||||
counts = await _dsar_execute_deletion(db, tenant_id, user_id)
|
||||
await db.commit()
|
||||
await post_system_message(
|
||||
db,
|
||||
tid,
|
||||
uid,
|
||||
"dsar_deletion_done",
|
||||
"DSGVO-Löschung ausgeführt",
|
||||
f"Kontakten soft-gelöscht: {counts.get('contacts_soft_deleted', 0)}; Konto anonymisiert.",
|
||||
severity="info",
|
||||
)
|
||||
await db.commit()
|
||||
logger.info("DSAR deletion executed for user %s: %s", user_id, counts)
|
||||
return {"type": request_type, "status": "completed", **counts}
|
||||
|
||||
if request_type == "rectification":
|
||||
await post_system_message(
|
||||
db,
|
||||
tid,
|
||||
uid,
|
||||
"dsar_rectification_requested",
|
||||
"DSGVO-Berichtigung angefordert",
|
||||
f"Manuelle Bearbeitung für User {user_id} erforderlich.",
|
||||
severity="warning",
|
||||
)
|
||||
await db.commit()
|
||||
logger.info("DSAR rectification requested for user %s", user_id)
|
||||
return {"type": request_type, "status": "queued_for_manual_handling"}
|
||||
|
||||
logger.warning("Unknown DSAR request_type '%s' for user %s", request_type, user_id)
|
||||
return {"type": request_type, "status": "unknown_type"}
|
||||
except Exception:
|
||||
await db.rollback()
|
||||
raise
|
||||
|
||||
|
||||
register_job("process_dsar", process_dsar)
|
||||
|
||||
@@ -3,6 +3,8 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import re
|
||||
import uuid as uuid_mod
|
||||
|
||||
from fastapi import Request, status
|
||||
from starlette.middleware.base import BaseHTTPMiddleware
|
||||
@@ -137,3 +139,97 @@ class CSRFMiddleware(BaseHTTPMiddleware):
|
||||
pass
|
||||
|
||||
return await call_next(request)
|
||||
|
||||
|
||||
class AuditMiddleware(BaseHTTPMiddleware):
|
||||
"""Safety-net audit trail for ALL successful mutating requests.
|
||||
|
||||
AGENTS.md requires every mutation to produce an audit entry. Explicit
|
||||
``log_audit`` calls in routes/services remain the detail layer (entity ids,
|
||||
change diffs); this middleware guarantees a baseline entry for mutations
|
||||
that lack one, marked with ``source=middleware`` in ``details``.
|
||||
|
||||
Best-effort by design: audit failures never break the request.
|
||||
"""
|
||||
|
||||
_MUTATING = {"POST", "PUT", "PATCH", "DELETE"}
|
||||
_SKIP_PREFIXES = (
|
||||
"/api/v1/auth",
|
||||
"/api/v1/health",
|
||||
"/api/v1/errors",
|
||||
"/api/v1/audit",
|
||||
"/api/v1/external",
|
||||
)
|
||||
|
||||
async def dispatch(self, request: Request, call_next):
|
||||
response = await call_next(request)
|
||||
|
||||
if request.method not in self._MUTATING:
|
||||
return response
|
||||
if response.status_code < 200 or response.status_code >= 300:
|
||||
return response
|
||||
path = request.url.path
|
||||
if any(path.startswith(p) for p in self._SKIP_PREFIXES):
|
||||
return response
|
||||
|
||||
try:
|
||||
await self._write_entry(request, path, response.status_code)
|
||||
except Exception:
|
||||
logging.getLogger(__name__).debug(
|
||||
"AuditMiddleware: failed to write baseline entry for %s %s", request.method, path
|
||||
)
|
||||
return response
|
||||
|
||||
@staticmethod
|
||||
def _derive_entity_type(path: str) -> str:
|
||||
"""Derive an entity_type from the second URL segment."""
|
||||
parts = [p for p in path.split("/") if p]
|
||||
# /api/v1/<resource>/... -> resource; singularize naive trailing 's'
|
||||
resource = parts[2] if len(parts) > 2 and parts[0] == "api" and parts[1] == "v1" else (parts[0] if parts else "unknown")
|
||||
return resource[:-1] if len(resource) > 3 and resource.endswith("s") else resource
|
||||
|
||||
async def _write_entry(self, request: Request, path: str, status_code: int) -> None:
|
||||
from app.core.audit import log_audit
|
||||
from app.core.auth import get_redis, get_session_data
|
||||
from app.core.db import create_db_session
|
||||
|
||||
# Attribute via the Redis session (same source as CSRFMiddleware) —
|
||||
# FastAPI dependencies run after middleware, so request.state is empty here.
|
||||
settings = get_settings()
|
||||
session_id = request.cookies.get(settings.session_cookie_name)
|
||||
if not session_id:
|
||||
return # unauthenticated — nothing to attribute
|
||||
redis = get_redis()
|
||||
session_data = await get_session_data(redis, session_id)
|
||||
if not session_data:
|
||||
return
|
||||
tenant_raw = session_data.get("tenant_id")
|
||||
user_raw = session_data.get("user_id")
|
||||
if not tenant_raw:
|
||||
return
|
||||
|
||||
action_map = {"POST": "create", "PATCH": "update", "PUT": "update", "DELETE": "delete"}
|
||||
entity_id: uuid_mod.UUID | None = None
|
||||
parts = [p for p in path.split("/") if p]
|
||||
if parts and re.fullmatch(r"[0-9a-fA-F-]{36}", parts[-1]):
|
||||
try:
|
||||
entity_id = uuid_mod.UUID(parts[-1])
|
||||
except ValueError:
|
||||
entity_id = None
|
||||
|
||||
async with create_db_session(uuid_mod.UUID(tenant_raw)) as db:
|
||||
await log_audit(
|
||||
db,
|
||||
uuid_mod.UUID(tenant_raw),
|
||||
uuid_mod.UUID(user_raw) if user_raw else None,
|
||||
action_map.get(request.method, request.method.lower()),
|
||||
self._derive_entity_type(path),
|
||||
entity_id,
|
||||
changes={
|
||||
"source": "middleware",
|
||||
"method": request.method,
|
||||
"path": path,
|
||||
"status": status_code,
|
||||
},
|
||||
)
|
||||
await db.commit()
|
||||
|
||||
@@ -54,6 +54,8 @@ CORE_PERMISSIONS: list[dict[str, str]] = [
|
||||
{"key": "taxes:write", "label": "Taxes: Write", "category": "core", "module": "taxes"},
|
||||
{"key": "currencies:read", "label": "Currencies: Read", "category": "core", "module": "currencies"},
|
||||
{"key": "currencies:write", "label": "Currencies: Write", "category": "core", "module": "currencies"},
|
||||
{"key": "custom_fields:read", "label": "Custom Fields: Read", "category": "core", "module": "custom_fields"},
|
||||
{"key": "custom_fields:write", "label": "Custom Fields: Write", "category": "core", "module": "custom_fields"},
|
||||
{"key": "import_export:read", "label": "Import/Export: Read", "category": "core", "module": "import_export"},
|
||||
{"key": "import_export:write", "label": "Import/Export: Write", "category": "core", "module": "import_export"},
|
||||
{"key": "workspaces:read", "label": "Workspaces: Read", "category": "core", "module": "workspaces"},
|
||||
|
||||
@@ -29,6 +29,14 @@ class ServiceContainer:
|
||||
"""Check if a service is registered."""
|
||||
return name in self._services
|
||||
|
||||
def remove(self, name: str) -> None:
|
||||
"""Remove a service registration (no-op if absent).
|
||||
|
||||
Used by plugin deactivation hooks to clean up services they
|
||||
registered during activation.
|
||||
"""
|
||||
self._services.pop(name, None)
|
||||
|
||||
async def initialize(self) -> None:
|
||||
"""Initialize core services."""
|
||||
if self._initialized:
|
||||
|
||||
+40
-12
@@ -501,11 +501,38 @@ async def save_with_metadata(
|
||||
}
|
||||
|
||||
|
||||
async def get_file_metadata_async(path: str) -> dict[str, Any]:
|
||||
"""Awaitable variant of :func:`get_file_metadata` (ARCH-052).
|
||||
|
||||
Safe to call from inside a running event loop — never creates a
|
||||
nested one. For local storage this is plain filesystem access; for
|
||||
S3 and other async backends the backend's ``exists()`` is awaited.
|
||||
"""
|
||||
backend = get_storage_backend()
|
||||
if isinstance(backend, LocalStorage):
|
||||
full_path = backend._full_path(path)
|
||||
if not os.path.exists(full_path):
|
||||
return {"size": None, "modified": None, "exists": False}
|
||||
stat = os.stat(full_path)
|
||||
return {
|
||||
"size": stat.st_size,
|
||||
"modified": stat.st_mtime,
|
||||
"exists": True,
|
||||
}
|
||||
# S3 or other async backends — await the backend directly
|
||||
if not await backend.exists(path):
|
||||
return {"size": None, "modified": None, "exists": False}
|
||||
return {"size": None, "modified": None, "exists": True}
|
||||
|
||||
|
||||
def get_file_metadata(path: str) -> dict[str, Any]:
|
||||
"""Read metadata of a stored file without loading its content.
|
||||
|
||||
Works with the *local* storage backend. For S3, use the S3 client
|
||||
``stat_object`` API directly.
|
||||
Works with the *local* storage backend without touching the event
|
||||
loop. For S3 and other async-only backends this drives the check
|
||||
through ``asyncio.run``; calling it from inside a running event loop
|
||||
raises ``RuntimeError`` — use :func:`get_file_metadata_async` there
|
||||
instead (ARCH-052).
|
||||
|
||||
Parameters
|
||||
----------
|
||||
@@ -530,14 +557,15 @@ def get_file_metadata(path: str) -> dict[str, Any]:
|
||||
"modified": stat.st_mtime,
|
||||
"exists": True,
|
||||
}
|
||||
# S3 or other backends — fall back to exists() check
|
||||
import asyncio as _asyncio
|
||||
|
||||
loop = _asyncio.new_event_loop()
|
||||
# Async-only backend outside a running loop is fine; inside one we
|
||||
# must never build a nested event loop.
|
||||
try:
|
||||
exists = loop.run_until_complete(backend.exists(path))
|
||||
if not exists:
|
||||
return {"size": None, "modified": None, "exists": False}
|
||||
return {"size": None, "modified": None, "exists": True}
|
||||
finally:
|
||||
loop.close()
|
||||
asyncio.get_running_loop()
|
||||
except RuntimeError:
|
||||
pass
|
||||
else:
|
||||
raise RuntimeError(
|
||||
"get_file_metadata() cannot be used with async storage backends "
|
||||
"inside a running event loop — use get_file_metadata_async()"
|
||||
)
|
||||
return asyncio.run(get_file_metadata_async(path))
|
||||
|
||||
@@ -121,11 +121,14 @@ class TriggerDispatcher:
|
||||
"""Query DB for active automations matching *event_name* and dispatch."""
|
||||
from app.core.db import get_session_factory
|
||||
from app.plugins.builtins.contracts import get_contract
|
||||
# None-check FIRST — accessing attributes on the contract before the
|
||||
# check crashed with AttributeError when automation was inactive
|
||||
# (ARCH-029/041).
|
||||
automation_contract = get_contract("automation")
|
||||
AutomationDefinition = automation_contract.Automation # noqa: N806
|
||||
if automation_contract is None:
|
||||
logger.debug("Automation plugin not available — trigger skipped")
|
||||
return
|
||||
AutomationDefinition = automation_contract.Automation # noqa: N806
|
||||
|
||||
factory = get_session_factory()
|
||||
tenant_id = payload.get("tenant_id")
|
||||
|
||||
+11
-12
@@ -170,12 +170,7 @@ async def on_startup(ctx: dict[str, Any]) -> None:
|
||||
if search_contract is not None:
|
||||
factory = async_session
|
||||
async with factory() as db:
|
||||
# auto_register_providers is not exposed via contract yet;
|
||||
# use the contract's get_search_registry to access providers
|
||||
from app.plugins.builtins.unified_search.provider_registry import (
|
||||
auto_register_providers,
|
||||
)
|
||||
await auto_register_providers(db)
|
||||
await search_contract.auto_register_providers(db)
|
||||
logger.info("Search providers registered for worker")
|
||||
else:
|
||||
logger.debug("Unified search plugin not available — skipping provider registration")
|
||||
@@ -350,8 +345,10 @@ async def cleanup_audit_log_job(ctx: dict[str, Any]) -> None:
|
||||
Runs daily to prevent the audit_log table from growing indefinitely.
|
||||
Iterates per-tenant for RLS compliance.
|
||||
"""
|
||||
from sqlalchemy import text as sa_text, delete as sa_delete
|
||||
from datetime import datetime, timedelta
|
||||
from datetime import UTC, datetime, timedelta
|
||||
|
||||
from sqlalchemy import delete as sa_delete
|
||||
from sqlalchemy import text as sa_text
|
||||
|
||||
from app.core.db import get_worker_session_factory
|
||||
from app.models.audit import AuditLog
|
||||
@@ -362,7 +359,7 @@ async def cleanup_audit_log_job(ctx: dict[str, Any]) -> None:
|
||||
tenant_result = await db.execute(sa_text("SELECT id FROM tenants"))
|
||||
tenant_ids = [row[0] for row in tenant_result]
|
||||
|
||||
cutoff = datetime.utcnow() - timedelta(days=365)
|
||||
cutoff = datetime.now(UTC) - timedelta(days=365)
|
||||
total_deleted = 0
|
||||
for tenant_id in tenant_ids:
|
||||
await db.execute(
|
||||
@@ -393,8 +390,10 @@ async def cleanup_trash_job(ctx: dict[str, Any]) -> None:
|
||||
Runs daily to clean up the trash. Iterates per-tenant for RLS compliance.
|
||||
Default retention: 90 days in trash before permanent deletion.
|
||||
"""
|
||||
from sqlalchemy import text as sa_text, delete as sa_delete
|
||||
from datetime import datetime, timedelta
|
||||
from datetime import UTC, datetime, timedelta
|
||||
|
||||
from sqlalchemy import delete as sa_delete
|
||||
from sqlalchemy import text as sa_text
|
||||
|
||||
from app.core.db import get_worker_session_factory
|
||||
from app.models.contact import Contact
|
||||
@@ -406,7 +405,7 @@ async def cleanup_trash_job(ctx: dict[str, Any]) -> None:
|
||||
tenant_result = await db.execute(sa_text("SELECT id FROM tenants"))
|
||||
tenant_ids = [row[0] for row in tenant_result]
|
||||
|
||||
cutoff = datetime.utcnow() - timedelta(days=90)
|
||||
cutoff = datetime.now(UTC) - timedelta(days=90)
|
||||
total_deleted = 0
|
||||
|
||||
for tenant_id in tenant_ids:
|
||||
|
||||
+28
-3
@@ -17,8 +17,9 @@ from app.core.db import get_db, set_tenant_context, set_user_context
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# Known write-permission modules — used by require_write() to check
|
||||
# specific permissions instead of broad wildcards like *:write
|
||||
# Legacy fallback list — used by require_write() only when the permission
|
||||
# registry is not initialized. The live source of truth is generated from
|
||||
# the registry (see _get_write_permissions, ARCH-022).
|
||||
_WRITE_PERMISSIONS = [
|
||||
"users:write",
|
||||
"roles:write",
|
||||
@@ -35,6 +36,30 @@ _WRITE_PERMISSIONS = [
|
||||
]
|
||||
|
||||
|
||||
def _get_write_permissions() -> list[str]:
|
||||
"""Return all known ``module:write`` permission keys (ARCH-022).
|
||||
|
||||
Generated from the permission registry so plugin write permissions are
|
||||
picked up automatically without touching this file. Falls back to the
|
||||
static legacy list when the registry is unavailable/uninitialized.
|
||||
"""
|
||||
try:
|
||||
from app.core.permission_registry import get_permission_registry
|
||||
|
||||
registry = get_permission_registry()
|
||||
if getattr(registry, "_initialized", False):
|
||||
perms = [
|
||||
entry["key"]
|
||||
for entry in registry.get_all()
|
||||
if entry["key"].endswith(":write")
|
||||
]
|
||||
if perms:
|
||||
return sorted(perms)
|
||||
except Exception:
|
||||
pass
|
||||
return list(_WRITE_PERMISSIONS)
|
||||
|
||||
|
||||
async def get_redis_dep() -> aioredis.Redis:
|
||||
"""FastAPI dependency for Redis client."""
|
||||
return get_redis()
|
||||
@@ -261,7 +286,7 @@ async def require_write(
|
||||
# Check via permission system for specific write permissions
|
||||
from app.core.permissions import check_permission
|
||||
|
||||
for perm in _WRITE_PERMISSIONS:
|
||||
for perm in _get_write_permissions():
|
||||
if check_permission(current_user, perm):
|
||||
return current_user
|
||||
|
||||
|
||||
+36
-20
@@ -22,8 +22,12 @@ logger = logging.getLogger(__name__)
|
||||
|
||||
from app.config import get_settings # noqa: E402
|
||||
from app.core.db import close_engine, get_engine # noqa: E402
|
||||
from app.core.error_codes import ApiError, build_error_response # noqa: E402
|
||||
from app.core.middleware import CSRFMiddleware, SecurityHeadersMiddleware # noqa: E402
|
||||
from app.core.error_codes import ERROR_CODES, ApiError, build_error_response # noqa: E402
|
||||
from app.core.middleware import ( # noqa: E402
|
||||
AuditMiddleware,
|
||||
CSRFMiddleware,
|
||||
SecurityHeadersMiddleware,
|
||||
)
|
||||
from app.core.monitoring import record_error, record_request # noqa: E402
|
||||
from app.core.rate_limit import GeneralRateLimitMiddleware # noqa: E402
|
||||
from app.core.resilience import CircuitBreakerMiddleware # noqa: E402
|
||||
@@ -36,16 +40,14 @@ from app.routes import ( # noqa: E402
|
||||
attachments,
|
||||
audit,
|
||||
auth,
|
||||
compliance,
|
||||
backups,
|
||||
bank_accounts,
|
||||
contact_folder_permissions,
|
||||
contact_folders,
|
||||
contacts,
|
||||
compliance,
|
||||
currencies,
|
||||
custom_field_definitions,
|
||||
custom_fields,
|
||||
dashboard,
|
||||
delegations,
|
||||
entity_history,
|
||||
entity_permissions,
|
||||
errors,
|
||||
@@ -59,7 +61,6 @@ from app.routes import ( # noqa: E402
|
||||
owner_transfer,
|
||||
permission_templates,
|
||||
plugins,
|
||||
delegations,
|
||||
policies,
|
||||
roles,
|
||||
saved_filters,
|
||||
@@ -284,22 +285,23 @@ async def lifespan(app: FastAPI):
|
||||
logger.info(f"Plugin {name} is inactive — skipping activation")
|
||||
continue
|
||||
|
||||
# Activate plugin with a FRESH session per plugin to avoid RLS state leakage
|
||||
# RLS fail-closed requires app.current_tenant_id for tenant-table writes.
|
||||
# Plugin activation may fail on duplicate cron job inserts — this is harmless
|
||||
# since cron jobs already exist from previous startups.
|
||||
# Activate plugin ONCE per process (ARCH-002 fix): a fresh session with
|
||||
# the first tenant's RLS context satisfies fail-closed RLS for any
|
||||
# tenant-table writes during activation. Plugins that need per-tenant
|
||||
# data must seed it themselves (e.g. via the default-tenant mechanism).
|
||||
# Calling on_activate once prevents duplicate event listeners, cron
|
||||
# jobs, mini-apps and other contributions at multi-tenant startups.
|
||||
plugin_activated = False
|
||||
for tenant_id in all_tenant_ids:
|
||||
if all_tenant_ids:
|
||||
try:
|
||||
async with async_session() as plugin_db:
|
||||
await set_tenant_context(plugin_db, tenant_id)
|
||||
await set_tenant_context(plugin_db, all_tenant_ids[0])
|
||||
await plugin.on_activate(plugin_db, container, event_bus)
|
||||
await plugin_db.flush()
|
||||
await plugin_db.commit()
|
||||
plugin_activated = True
|
||||
except Exception as exc:
|
||||
logger.warning(f"[STARTUP] Plugin {name} activation issue for tenant {tenant_id}: {exc}")
|
||||
break
|
||||
logger.warning(f"[STARTUP] Plugin {name} activation issue: {exc}")
|
||||
|
||||
if plugin_activated:
|
||||
plugin_record.status = "active"
|
||||
@@ -474,6 +476,7 @@ def create_app() -> FastAPI:
|
||||
)
|
||||
app.add_middleware(CSRFMiddleware)
|
||||
app.add_middleware(SecurityHeadersMiddleware)
|
||||
app.add_middleware(AuditMiddleware)
|
||||
app.add_middleware(GeneralRateLimitMiddleware)
|
||||
app.add_middleware(RequestLoggingMiddleware)
|
||||
app.add_middleware(CircuitBreakerMiddleware)
|
||||
@@ -516,6 +519,21 @@ def create_app() -> FastAPI:
|
||||
504: "service_timeout",
|
||||
}
|
||||
code = status_to_code.get(exc.status_code, "internal_error" if exc.status_code >= 500 else "validation_error")
|
||||
# Structured detail passthrough (AGENTS.md): when a route raises
|
||||
# HTTPException with a dict detail containing a machine-readable ``code``,
|
||||
# preserve the structured shape instead of stringifying it.
|
||||
raw_detail = exc.detail
|
||||
if isinstance(raw_detail, dict):
|
||||
inner_code = raw_detail.get("code", code)
|
||||
body = build_error_response(
|
||||
code=inner_code if inner_code in ERROR_CODES else code,
|
||||
detail=raw_detail.get("detail") or str(raw_detail),
|
||||
trace_id=trace_id,
|
||||
)
|
||||
# Preserve the full structured detail as a nested object so clients
|
||||
# can read ``resp.json()["detail"]["code"]``.
|
||||
body["detail"] = raw_detail
|
||||
else:
|
||||
body = build_error_response(
|
||||
code=code,
|
||||
detail=str(exc.detail) if exc.detail else None,
|
||||
@@ -544,11 +562,9 @@ def create_app() -> FastAPI:
|
||||
app.include_router(groups.router)
|
||||
app.include_router(tenants.router)
|
||||
app.include_router(notifications.router)
|
||||
from app.routes.companies import router as companies_router
|
||||
app.include_router(companies_router)
|
||||
app.include_router(contacts.router)
|
||||
app.include_router(contact_folders.router)
|
||||
app.include_router(contact_folder_permissions.router)
|
||||
# NOTE: contacts/companies/contact-folders routes are plugin-owned now
|
||||
# (Block B1) and mounted via the manifest.routes mechanism below with
|
||||
# require_active_plugin("contacts") protection.
|
||||
app.include_router(entity_permissions.router)
|
||||
app.include_router(dashboard.router)
|
||||
app.include_router(entity_history.router)
|
||||
|
||||
@@ -6,8 +6,8 @@ from app.models.audit import AuditLog
|
||||
from app.models.auth import ApiToken, PasswordResetToken
|
||||
from app.models.backup import Backup
|
||||
from app.models.bank_account import BankAccount
|
||||
from app.models.consumer_inbox import ConsumerInbox
|
||||
from app.models.compliance import ComplianceIncident
|
||||
from app.models.consumer_inbox import ConsumerInbox
|
||||
from app.models.contact import Contact, ContactPerson
|
||||
from app.models.contact_folder import ContactFolder
|
||||
from app.models.contact_merge import ContactMergeHistory
|
||||
|
||||
+1
-2
@@ -11,13 +11,12 @@ from datetime import datetime
|
||||
from typing import Any
|
||||
|
||||
from sqlalchemy import DateTime, ForeignKey, String, func
|
||||
from sqlalchemy.dialects.postgresql import JSONB
|
||||
from sqlalchemy.dialects.postgresql import JSONB, TSVECTOR
|
||||
from sqlalchemy.dialects.postgresql import UUID as PGUUID
|
||||
from sqlalchemy.orm import Mapped, mapped_column
|
||||
|
||||
from app.core.db import Base, TenantMixin
|
||||
from app.models.owned_mixin import OwnedMixin
|
||||
from sqlalchemy.dialects.postgresql import TSVECTOR
|
||||
|
||||
# Re-export EntityHistory as DeletionLog for backward compatibility.
|
||||
# Tests import DeletionLog from app.models.audit and use entity_snapshot attribute.
|
||||
|
||||
+47
-1
@@ -5,7 +5,7 @@ from __future__ import annotations
|
||||
import uuid
|
||||
from datetime import datetime
|
||||
|
||||
from sqlalchemy import DateTime, Integer, String, Text, func
|
||||
from sqlalchemy import DateTime, ForeignKey, Integer, String, Text, UniqueConstraint, func
|
||||
from sqlalchemy.dialects.postgresql import JSONB
|
||||
from sqlalchemy.dialects.postgresql import UUID as PGUUID
|
||||
from sqlalchemy.orm import Mapped, mapped_column
|
||||
@@ -77,3 +77,49 @@ class EventOutbox(Base):
|
||||
failed_at: Mapped[datetime | None] = mapped_column(
|
||||
DateTime(timezone=True), nullable=True,
|
||||
)
|
||||
|
||||
|
||||
class OutboxDelivery(Base):
|
||||
"""Per-consumer delivery status for an event_outbox row (Migration 0075).
|
||||
|
||||
Tracks whether each consumer successfully processed an event; an event is
|
||||
only 'published' when all mandatory deliveries succeed.
|
||||
"""
|
||||
|
||||
__tablename__ = "outbox_deliveries"
|
||||
__table_args__ = (
|
||||
UniqueConstraint(
|
||||
"event_id", "consumer_name",
|
||||
name="uq_outbox_deliveries_event_consumer",
|
||||
),
|
||||
)
|
||||
|
||||
id: Mapped[uuid.UUID] = mapped_column(
|
||||
PGUUID(as_uuid=True), primary_key=True,
|
||||
server_default=func.gen_random_uuid(),
|
||||
)
|
||||
event_id: Mapped[uuid.UUID] = mapped_column(
|
||||
PGUUID(as_uuid=True),
|
||||
ForeignKey("event_outbox.id", ondelete="CASCADE"),
|
||||
nullable=False,
|
||||
)
|
||||
consumer_name: Mapped[str] = mapped_column(String(150), nullable=False)
|
||||
status: Mapped[str] = mapped_column(
|
||||
String(30), nullable=False, server_default="pending",
|
||||
)
|
||||
attempt_count: Mapped[int] = mapped_column(
|
||||
Integer, nullable=False, server_default="0",
|
||||
)
|
||||
next_attempt_at: Mapped[datetime | None] = mapped_column(
|
||||
DateTime(timezone=True), nullable=True,
|
||||
)
|
||||
last_error: Mapped[str | None] = mapped_column(Text, nullable=True)
|
||||
processed_at: Mapped[datetime | None] = mapped_column(
|
||||
DateTime(timezone=True), nullable=True,
|
||||
)
|
||||
created_at: Mapped[datetime] = mapped_column(
|
||||
DateTime(timezone=True), nullable=False, server_default=func.now(),
|
||||
)
|
||||
updated_at: Mapped[datetime] = mapped_column(
|
||||
DateTime(timezone=True), nullable=False, server_default=func.now(),
|
||||
)
|
||||
|
||||
@@ -53,6 +53,9 @@ class SystemSettings(Base, TenantMixin, OwnedMixin):
|
||||
theme_border_radius: Mapped[str] = mapped_column(String(20), nullable=False, default="0.5rem")
|
||||
# Backup configuration
|
||||
backup_enabled: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False, server_default="false")
|
||||
backup_interval: Mapped[str] = mapped_column(String(20), nullable=False, default="daily", server_default="daily")
|
||||
backup_retention_days: Mapped[int] = mapped_column(Integer, nullable=False, default=7, server_default="7")
|
||||
backup_destination: Mapped[str] = mapped_column(String(20), nullable=False, default="local", server_default="local")
|
||||
# Automation plugin settings (JSONB)
|
||||
automation_config: Mapped[dict | None] = mapped_column(JSONB, nullable=True)
|
||||
# Retention policy overrides (JSONB) — compliance module
|
||||
|
||||
@@ -12,7 +12,6 @@ from sqlalchemy.dialects.postgresql import UUID as PGUUID
|
||||
from sqlalchemy.orm import Mapped, mapped_column
|
||||
|
||||
from app.core.db import Base, SoftDeleteMixin, TimestampMixin
|
||||
from app.models.owned_mixin import OwnedMixin
|
||||
|
||||
|
||||
class User(Base, TimestampMixin, SoftDeleteMixin):
|
||||
|
||||
@@ -97,6 +97,15 @@ class BasePlugin(ABC):
|
||||
"""
|
||||
return []
|
||||
|
||||
async def register_event_handlers(self, event_bus: EventBus) -> None:
|
||||
"""Register event handlers for the background worker (ARCH-038 hook).
|
||||
|
||||
The worker calls this on every active plugin at startup so plugins
|
||||
can subscribe to events even when the web process is separate.
|
||||
Default: no-op. Override to subscribe handlers.
|
||||
"""
|
||||
return None
|
||||
|
||||
# ─── Job Modules ───
|
||||
|
||||
def get_job_modules(self) -> list[str]:
|
||||
|
||||
@@ -12,7 +12,6 @@ from sqlalchemy.orm import Mapped, mapped_column
|
||||
|
||||
from app.core.db import Base, TenantMixin
|
||||
from app.models.owned_mixin import OwnedMixin
|
||||
from pgvector.sqlalchemy import Vector
|
||||
|
||||
|
||||
class AgentMemory(Base, TenantMixin, OwnedMixin):
|
||||
|
||||
@@ -22,6 +22,7 @@ from app.plugins.builtins.ai_assistant.schemas import (
|
||||
ExternalAgentRequest,
|
||||
ExternalAgentResponse,
|
||||
)
|
||||
from app.plugins.builtins.ai_assistant.services import stream_chat_comm as stream_chat
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
@@ -120,13 +121,18 @@ async def run_agent_external(
|
||||
}
|
||||
|
||||
# Run the agent via streaming chat (non-streaming mode)
|
||||
from app.plugins.builtins.ai_assistant.services import stream_chat_comm
|
||||
|
||||
full_response = ""
|
||||
async with get_db() as stream_db:
|
||||
await set_tenant_context(stream_db, tenant_id)
|
||||
async for chunk in stream_chat(
|
||||
stream_db, session, agent, data.message, user_context, tenant_id
|
||||
stream_db,
|
||||
session.id,
|
||||
agent,
|
||||
data.message,
|
||||
user_context,
|
||||
tenant_id,
|
||||
uuid.UUID(current_user["user_id"]),
|
||||
):
|
||||
if chunk.startswith("data: ") and chunk != "data: [DONE]\n\n":
|
||||
try:
|
||||
|
||||
@@ -0,0 +1,10 @@
|
||||
-- Dual-path convergence (Gate B): add compliance columns that Alembic
|
||||
-- migration 0119 adds on the core path. Idempotent so both install paths
|
||||
-- converge to the identical schema.
|
||||
ALTER TABLE ai_providers ADD COLUMN IF NOT EXISTS region VARCHAR(20) NOT NULL DEFAULT 'unknown';
|
||||
ALTER TABLE ai_providers ADD COLUMN IF NOT EXISTS hosting_type VARCHAR(30) NOT NULL DEFAULT 'cloud';
|
||||
ALTER TABLE ai_providers ADD COLUMN IF NOT EXISTS dpa_status VARCHAR(20) NOT NULL DEFAULT 'none';
|
||||
ALTER TABLE ai_providers ADD COLUMN IF NOT EXISTS retention_policy TEXT NOT NULL DEFAULT '';
|
||||
ALTER TABLE ai_providers ADD COLUMN IF NOT EXISTS training_on_customer_data BOOLEAN NOT NULL DEFAULT FALSE;
|
||||
ALTER TABLE ai_providers ADD COLUMN IF NOT EXISTS transfer_notice TEXT NOT NULL DEFAULT '';
|
||||
ALTER TABLE ai_providers ADD COLUMN IF NOT EXISTS allowed_data_classes JSONB NOT NULL DEFAULT '[]'::jsonb;
|
||||
@@ -42,7 +42,7 @@ class AIAssistantPlugin(BasePlugin):
|
||||
),
|
||||
],
|
||||
events=[],
|
||||
migrations=["0001_initial.sql", "0002_folders_attachments.sql"],
|
||||
migrations=["0001_initial.sql", "0002_folders_attachments.sql", "0003_sort_order.sql", "0004_compliance_fields.sql"],
|
||||
permissions=[
|
||||
"ai:read",
|
||||
"ai:write",
|
||||
@@ -52,10 +52,10 @@ class AIAssistantPlugin(BasePlugin):
|
||||
],
|
||||
is_core=True,
|
||||
menu_items=[
|
||||
FrontendMenuItem(label_key='nav.aiAssistant', label='KI Assistent', path='/ai-assistant', icon='Bot', order=90),
|
||||
FrontendMenuItem(label_key='nav.aiAssistant', label='KI Assistent', path='/ai-assistant', icon='Bot', order=90, permission='ai:read'),
|
||||
],
|
||||
page_routes=[
|
||||
FrontendPageRoute(path='/ai-assistant', component='@/pages/AIAssistant', protected=True),
|
||||
FrontendPageRoute(path='/ai-assistant', component='@/pages/AIAssistant', protected=True, permission='ai:read'),
|
||||
],
|
||||
settings_pages=[
|
||||
FrontendSettingsPage(path='ai', label_key='settings.ai', label='AI Settings', component='@/pages/AISettings', icon='Bot', order=60),
|
||||
|
||||
@@ -165,8 +165,8 @@ async def get_open_tasks_handler(arguments: dict[str, Any], context: dict[str, A
|
||||
|
||||
from app.plugins.builtins.calendar.contracts import get_contract as get_calendar_contract
|
||||
_cal = get_calendar_contract()
|
||||
calendar_entry = _cal.calendar_entry
|
||||
calendar_entry_link = _cal.calendar_entry_link
|
||||
calendar_entry = _cal.CalendarEntry
|
||||
calendar_entry_link = _cal.CalendarEntryLink
|
||||
|
||||
db, tenant_id, _ = await _get_db_and_tenant(context)
|
||||
entity_type = arguments["entity_type"]
|
||||
|
||||
@@ -69,10 +69,12 @@ async def push_suggestion(user_id: str, suggestion: dict[str, Any]) -> None:
|
||||
# Post suggestion to Communication (I-WORK-PROACTIVE)
|
||||
try:
|
||||
import uuid as uuid_mod
|
||||
|
||||
from sqlalchemy import select as sa_select
|
||||
|
||||
from app.core.db import get_worker_session_factory
|
||||
from app.plugins.builtins.contracts import get_contract_registry
|
||||
from app.plugins.builtins.kommunikation.models import CommConversation
|
||||
from sqlalchemy import select as sa_select
|
||||
from app.core.db import get_worker_session_factory
|
||||
komm = get_contract_registry().get("kommunikation")
|
||||
if komm:
|
||||
factory = get_worker_session_factory()
|
||||
@@ -274,8 +276,8 @@ async def gather_context(
|
||||
# Upcoming calendar events
|
||||
from app.plugins.builtins.calendar.contracts import get_contract as get_calendar_contract
|
||||
_cal = get_calendar_contract()
|
||||
calendar_entry = _cal.calendar_entry
|
||||
calendar_entry_link = _cal.calendar_entry_link
|
||||
calendar_entry = _cal.CalendarEntry
|
||||
calendar_entry_link = _cal.CalendarEntryLink
|
||||
|
||||
now = datetime.now(UTC)
|
||||
event_result = await db.execute(
|
||||
@@ -389,8 +391,8 @@ async def gather_context(
|
||||
# Upcoming events
|
||||
from app.plugins.builtins.calendar.contracts import get_contract as get_calendar_contract
|
||||
_cal = get_calendar_contract()
|
||||
calendar_entry = _cal.calendar_entry
|
||||
calendar_entry_link = _cal.calendar_entry_link
|
||||
calendar_entry = _cal.CalendarEntry
|
||||
calendar_entry_link = _cal.CalendarEntryLink
|
||||
|
||||
now = datetime.now(UTC)
|
||||
event_result = await db.execute(
|
||||
|
||||
@@ -62,7 +62,10 @@ class AIUIControlPlugin(BasePlugin):
|
||||
from app.plugins.builtins.contracts import get_contract_registry
|
||||
get_contract_registry().unregister(self.manifest.name)
|
||||
|
||||
await super().on_deactivate(db, service_container, event_bus)
|
||||
# Remove the WebSocket manager BEFORE super() so that event handlers
|
||||
# being unsubscribed can no longer reach it (ARCH-044).
|
||||
if service_container.has("ai_ui_control_ws"):
|
||||
service_container.remove("ai_ui_control_ws")
|
||||
logger.info("AI UI Control WebSocket manager removed")
|
||||
|
||||
await super().on_deactivate(db, service_container, event_bus)
|
||||
|
||||
@@ -55,7 +55,8 @@ async def send_agent_message(
|
||||
|
||||
# 2. Create a kommunikation message in a dedicated agent room
|
||||
try:
|
||||
from app.plugins.builtins.kommunikation.contracts import CommConversation as Room, CommMessage as Message
|
||||
from app.plugins.builtins.kommunikation.contracts import CommConversation as Room
|
||||
from app.plugins.builtins.kommunikation.contracts import CommMessage as Message
|
||||
|
||||
# Find or create the agent-to-agent room
|
||||
room_name = f"agent:{from_agent_id}:{target_agent.id}"
|
||||
|
||||
@@ -619,6 +619,7 @@ async def stream_agent_run(
|
||||
in real-time as the agent processes.
|
||||
"""
|
||||
from fastapi.responses import StreamingResponse
|
||||
|
||||
from app.ai.agent_stream import stream_react_loop
|
||||
from app.plugins.builtins.ai_assistant.contracts import get_tool_registry
|
||||
|
||||
|
||||
@@ -11,6 +11,7 @@ Safety features:
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import uuid
|
||||
from datetime import UTC, datetime
|
||||
from typing import Any
|
||||
|
||||
@@ -37,12 +38,12 @@ async def run_agent(
|
||||
3. Infinite loop: same tool 5x consecutively (handled in ReAct loop)
|
||||
4. Budget limit: cumulative cost_usd
|
||||
"""
|
||||
from app.plugins.builtins.ai_assistant.contracts import get_tool_registry
|
||||
from app.plugins.builtins.automation.models import (
|
||||
AgentDefinition,
|
||||
AgentRun,
|
||||
AgentRunStep,
|
||||
)
|
||||
from app.plugins.builtins.ai_assistant.contracts import get_tool_registry
|
||||
|
||||
factory = get_session_factory()
|
||||
|
||||
@@ -110,9 +111,9 @@ async def run_agent(
|
||||
try:
|
||||
from app.plugins.builtins.contracts import get_contract
|
||||
mail_contract = get_contract("mail")
|
||||
if mail_contract and hasattr(mail_contract, "get_recent_mails"):
|
||||
if mail_contract and hasattr(mail_contract, "Mail"):
|
||||
from sqlalchemy import select as _select
|
||||
from app.plugins.builtins.mail.models import Mail
|
||||
Mail = mail_contract.Mail
|
||||
async with factory() as db:
|
||||
mail_q = await db.execute(
|
||||
_select(Mail)
|
||||
@@ -167,7 +168,7 @@ async def run_agent(
|
||||
perm_ctx = await resolve_agent_permissions(
|
||||
db=db,
|
||||
tenant_id=agent.tenant_id,
|
||||
user_id=agent.created_by or uuid_mod.uuid4(),
|
||||
user_id=agent.created_by or uuid.uuid4(),
|
||||
agent_definition=agent,
|
||||
)
|
||||
|
||||
@@ -379,23 +380,16 @@ async def run_agent(
|
||||
komm = get_contract_registry().get("kommunikation")
|
||||
if komm:
|
||||
async with factory() as db:
|
||||
# Find or create agent conversation room
|
||||
from app.plugins.builtins.contracts import get_contract as _get_contract
|
||||
_komm_contract = _get_contract("kommunikation")
|
||||
from app.plugins.builtins.kommunikation.models import CommConversation
|
||||
from sqlalchemy import select as sa_select
|
||||
# Find or create agent conversation room via contract
|
||||
# (find_locked_room_id matches create_plugin_room semantics)
|
||||
room_title = f"Agent: {agent.name}"
|
||||
existing = await db.execute(
|
||||
sa_select(CommConversation).where(
|
||||
CommConversation.tenant_id == agent.tenant_id,
|
||||
CommConversation.title == room_title,
|
||||
CommConversation.is_locked.is_(True),
|
||||
CommConversation.locked_by == "automation",
|
||||
CommConversation.deleted_at.is_(None),
|
||||
conv_id = await komm.find_locked_room_id(
|
||||
db=db,
|
||||
tenant_id=agent.tenant_id,
|
||||
plugin_name="automation",
|
||||
title=room_title,
|
||||
)
|
||||
)
|
||||
conv = existing.scalar_one_or_none()
|
||||
if not conv:
|
||||
if not conv_id:
|
||||
room = await komm.create_plugin_room(
|
||||
db=db,
|
||||
tenant_id=agent.tenant_id,
|
||||
@@ -405,8 +399,6 @@ async def run_agent(
|
||||
participant_type="agent",
|
||||
)
|
||||
conv_id = uuid.UUID(room["conversation_id"])
|
||||
else:
|
||||
conv_id = conv.id
|
||||
|
||||
# Post result as message with action_card block
|
||||
status = result_data.get("status", "unknown")
|
||||
|
||||
@@ -0,0 +1,31 @@
|
||||
-- Dual-path convergence (Gate B): create the ReAct step-tracking table
|
||||
-- that Alembic migration 0121 creates on the core path, add the Phase-F
|
||||
-- columns from 0122, and apply the RLS policy from 0129/0136. Idempotent
|
||||
-- so both install paths converge to the identical schema.
|
||||
CREATE TABLE IF NOT EXISTS automation_agent_run_steps (
|
||||
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
tenant_id UUID NOT NULL,
|
||||
agent_run_id UUID NOT NULL REFERENCES automation_agent_runs(id) ON DELETE CASCADE,
|
||||
step_number INTEGER NOT NULL,
|
||||
thought TEXT,
|
||||
action VARCHAR(255),
|
||||
action_input JSONB,
|
||||
observation TEXT,
|
||||
cost_usd FLOAT NOT NULL DEFAULT 0.0,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS ix_agent_run_steps_run ON automation_agent_run_steps(tenant_id, agent_run_id);
|
||||
|
||||
ALTER TABLE automation_agent_definitions ADD COLUMN IF NOT EXISTS temperature FLOAT NOT NULL DEFAULT 0.3;
|
||||
ALTER TABLE automation_agent_definitions ADD COLUMN IF NOT EXISTS max_tokens INTEGER NOT NULL DEFAULT 1000;
|
||||
ALTER TABLE automation_agent_definitions ADD COLUMN IF NOT EXISTS max_steps INTEGER NOT NULL DEFAULT 20;
|
||||
ALTER TABLE automation_agent_definitions ADD COLUMN IF NOT EXISTS trace_mode VARCHAR(20) NOT NULL DEFAULT 'standard';
|
||||
ALTER TABLE automation_agent_definitions ADD COLUMN IF NOT EXISTS skill_ids JSONB NOT NULL DEFAULT '[]'::jsonb;
|
||||
ALTER TABLE automation_agent_definitions ADD COLUMN IF NOT EXISTS trigger_config JSONB NOT NULL DEFAULT '{}'::jsonb;
|
||||
ALTER TABLE automation_agent_definitions ADD COLUMN IF NOT EXISTS ai_use_case_metadata JSONB NOT NULL DEFAULT '{}'::jsonb;
|
||||
|
||||
-- RLS matching migrations 0129 + 0136 (current_tenant_id variant)
|
||||
ALTER TABLE automation_agent_run_steps ENABLE ROW LEVEL SECURITY;
|
||||
DROP POLICY IF EXISTS tenant_isolation ON automation_agent_run_steps;
|
||||
CREATE POLICY tenant_isolation ON automation_agent_run_steps
|
||||
USING (tenant_id::text = current_setting('app.current_tenant_id', true));
|
||||
@@ -38,7 +38,7 @@ class AutomationPlugin(BasePlugin):
|
||||
"Define AI agents with LLM models and tools, create event/schedule/manual "
|
||||
"automations with conditions and actions, schedule cron jobs, and track execution logs."
|
||||
),
|
||||
dependencies=[],
|
||||
dependencies=["mail"],
|
||||
routes=[
|
||||
PluginRouteDef(
|
||||
path="/api/v1/automation",
|
||||
@@ -62,7 +62,7 @@ class AutomationPlugin(BasePlugin):
|
||||
"mail.received",
|
||||
"workflow.timeout",
|
||||
],
|
||||
migrations=["0001_initial.sql", "0002_agent_subtasks.sql", "0003_skill_definitions.sql"],
|
||||
migrations=["0001_initial.sql", "0002_agent_subtasks.sql", "0003_skill_definitions.sql", "0004_run_steps_phase_f.sql"],
|
||||
permissions=[
|
||||
"automation:read",
|
||||
"automation:write",
|
||||
@@ -82,6 +82,7 @@ class AutomationPlugin(BasePlugin):
|
||||
path="/workflows",
|
||||
icon="Workflow",
|
||||
order=52,
|
||||
permission="automation:read",
|
||||
),
|
||||
FrontendMenuItem(
|
||||
label_key="nav.importExport",
|
||||
@@ -89,6 +90,7 @@ class AutomationPlugin(BasePlugin):
|
||||
path="/import-export",
|
||||
icon="ArrowUpDown",
|
||||
order=53,
|
||||
permission="import_export:read",
|
||||
),
|
||||
FrontendMenuItem(
|
||||
label_key="nav.dedupMerge",
|
||||
@@ -96,6 +98,7 @@ class AutomationPlugin(BasePlugin):
|
||||
path="/contacts/dedup",
|
||||
icon="Copy",
|
||||
order=54,
|
||||
permission="contacts:read",
|
||||
),
|
||||
FrontendMenuItem(
|
||||
label_key="nav.tags",
|
||||
@@ -103,6 +106,7 @@ class AutomationPlugin(BasePlugin):
|
||||
path="/tags",
|
||||
icon="Tag",
|
||||
order=55,
|
||||
permission="tags:read",
|
||||
),
|
||||
FrontendMenuItem(
|
||||
label_key="nav.activity",
|
||||
@@ -110,6 +114,7 @@ class AutomationPlugin(BasePlugin):
|
||||
path="/activity",
|
||||
icon="Activity",
|
||||
order=56,
|
||||
permission="contacts:read",
|
||||
),
|
||||
],
|
||||
page_routes=[
|
||||
@@ -117,21 +122,25 @@ class AutomationPlugin(BasePlugin):
|
||||
path="/automation",
|
||||
component="@/pages/AutomationDashboard",
|
||||
order=50,
|
||||
permission="automation:read",
|
||||
),
|
||||
FrontendPageRoute(
|
||||
path="/agents",
|
||||
component="@/pages/AgentDashboard",
|
||||
order=51,
|
||||
permission="agents:read",
|
||||
),
|
||||
FrontendPageRoute(
|
||||
path="/workflows",
|
||||
component="@/pages/Workflows",
|
||||
order=52,
|
||||
permission="automation:read",
|
||||
),
|
||||
FrontendPageRoute(
|
||||
path="/import-export",
|
||||
component="@/pages/ImportExport",
|
||||
order=53,
|
||||
permission="import_export:read",
|
||||
),
|
||||
],
|
||||
settings_pages=[
|
||||
@@ -239,18 +248,25 @@ class AutomationPlugin(BasePlugin):
|
||||
logger.exception("Failed to register own cron jobs")
|
||||
# Register pre-built agents in DB (if not already present)
|
||||
try:
|
||||
from app.plugins.builtins.automation.models import AgentDefinition
|
||||
from app.plugins.builtins.automation.prebuilt.email_triage_agent import create_email_triage_agent
|
||||
from app.plugins.builtins.automation.prebuilt.contact_enrichment_agent import create_contact_enrichment_agent
|
||||
from app.plugins.builtins.automation.prebuilt.follow_up_agent import create_follow_up_agent
|
||||
from app.plugins.builtins.automation.prebuilt.report_agent import create_report_agent
|
||||
from sqlalchemy import select as sa_select
|
||||
|
||||
# Get first tenant + admin user for seeding
|
||||
from app.models.user import User
|
||||
from app.models.tenant import Tenant
|
||||
tenant_result = await db.execute(sa_select(Tenant).limit(1))
|
||||
tenant = tenant_result.scalar_one_or_none()
|
||||
# Get system tenant + admin user for seeding (ARCH-043:
|
||||
# deterministic slug lookup instead of arbitrary first row)
|
||||
from app.core.db import get_system_tenant
|
||||
from app.models.user import User, UserTenant
|
||||
from app.plugins.builtins.automation.models import AgentDefinition
|
||||
from app.plugins.builtins.automation.prebuilt.contact_enrichment_agent import (
|
||||
create_contact_enrichment_agent,
|
||||
)
|
||||
from app.plugins.builtins.automation.prebuilt.email_triage_agent import (
|
||||
create_email_triage_agent,
|
||||
)
|
||||
from app.plugins.builtins.automation.prebuilt.follow_up_agent import (
|
||||
create_follow_up_agent,
|
||||
)
|
||||
from app.plugins.builtins.automation.prebuilt.report_agent import create_report_agent
|
||||
|
||||
tenant = await get_system_tenant(db)
|
||||
if tenant:
|
||||
user_result = await db.execute(
|
||||
sa_select(User)
|
||||
@@ -288,15 +304,14 @@ class AutomationPlugin(BasePlugin):
|
||||
def _register_workflow_agent_tools(self) -> None:
|
||||
"""Register I-AW agent tools for starting and inspecting workflows."""
|
||||
import uuid
|
||||
from typing import Any
|
||||
|
||||
from app.ai.tool_registry import get_tool_registry
|
||||
registry = get_tool_registry()
|
||||
|
||||
async def _start_workflow_handler(arguments: dict[str, Any], context: dict[str, Any]) -> dict[str, Any]:
|
||||
"""Start a workflow by ID."""
|
||||
from app.services.workflow_service import create_instance
|
||||
from app.core.db import get_worker_session_factory
|
||||
from app.services.workflow_service import create_instance
|
||||
workflow_id = arguments.get("workflow_id", "")
|
||||
tenant_id = context.get("tenant_id")
|
||||
user_id = context.get("user_id")
|
||||
@@ -332,8 +347,9 @@ class AutomationPlugin(BasePlugin):
|
||||
async def _check_workflow_status_handler(arguments: dict[str, Any], context: dict[str, Any]) -> dict[str, Any]:
|
||||
"""Check the status of a workflow instance."""
|
||||
from sqlalchemy import select
|
||||
from app.models.workflow import WorkflowInstance
|
||||
|
||||
from app.core.db import get_worker_session_factory
|
||||
from app.models.workflow import WorkflowInstance
|
||||
instance_id = arguments.get("instance_id", "")
|
||||
tenant_id = context.get("tenant_id")
|
||||
if not instance_id or not tenant_id:
|
||||
@@ -418,16 +434,16 @@ class AutomationPlugin(BasePlugin):
|
||||
from another plugin's manifest. Uses plugin name prefixing for conflict resolution."""
|
||||
from sqlalchemy import select
|
||||
|
||||
# Get default tenant_id from the first tenant in the DB
|
||||
from app.models.tenant import Tenant
|
||||
# Get system tenant for contributions (ARCH-043: deterministic slug
|
||||
# lookup instead of arbitrary first row)
|
||||
from app.core.db import get_system_tenant
|
||||
from app.plugins.builtins.automation.models import AutomationCronJob
|
||||
from app.plugins.builtins.automation.services import (
|
||||
AgentService,
|
||||
AutomationService,
|
||||
CronJobService,
|
||||
)
|
||||
tenant_result = await db.execute(select(Tenant).limit(1))
|
||||
tenant = tenant_result.scalar_one_or_none()
|
||||
tenant = await get_system_tenant(db)
|
||||
default_tenant_id = tenant.id if tenant else None
|
||||
if default_tenant_id is None:
|
||||
logger.warning("No tenant found — skipping plugin contributions registration")
|
||||
|
||||
@@ -3,7 +3,9 @@
|
||||
Enriches contact data by searching for related information.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import uuid
|
||||
|
||||
from app.plugins.builtins.automation.models import AgentDefinition
|
||||
|
||||
CONTACT_ENRICHMENT_SYSTEM_PROMPT = """You are a Contact Enrichment Agent for a CRM system.
|
||||
|
||||
@@ -3,7 +3,9 @@
|
||||
Sorts and prioritizes incoming emails automatically.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import uuid
|
||||
|
||||
from app.plugins.builtins.automation.models import AgentDefinition
|
||||
|
||||
EMAIL_TRIAGE_SYSTEM_PROMPT = """You are an E-Mail Triage Agent for a CRM system.
|
||||
|
||||
@@ -3,7 +3,9 @@
|
||||
Reminds about and creates follow-up tasks for contacts.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import uuid
|
||||
|
||||
from app.plugins.builtins.automation.models import AgentDefinition
|
||||
|
||||
FOLLOW_UP_SYSTEM_PROMPT = """You are a Follow-up Agent for a CRM system.
|
||||
|
||||
@@ -3,7 +3,9 @@
|
||||
Generates reports from CRM data using search and API tools.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import uuid
|
||||
|
||||
from app.plugins.builtins.automation.models import AgentDefinition
|
||||
|
||||
REPORT_SYSTEM_PROMPT = """You are a Report Agent for a CRM system.
|
||||
|
||||
@@ -1,60 +1,132 @@
|
||||
"""Tests for the Automation & Agents plugin.
|
||||
|
||||
Uses pytest with async fixtures. Tests use SQLite in-memory database
|
||||
since PostgreSQL may not be available in the dev container.
|
||||
Uses pytest with async fixtures against an ephemeral PostgreSQL database
|
||||
(SQLITE-001 fix) — matches the project convention and exercises the real
|
||||
PGUUID/JSONB column types.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
# Register ALL plugin models so create_all can resolve cross-plugin FKs
|
||||
# (e.g. entity_attachments.dms_file_id -> files) — same pattern as
|
||||
# scripts/sync_plugin_schema.py.
|
||||
import importlib
|
||||
import os
|
||||
import pkgutil
|
||||
import uuid
|
||||
from collections.abc import AsyncGenerator
|
||||
from datetime import UTC, datetime, timedelta
|
||||
|
||||
import pytest
|
||||
import pytest_asyncio
|
||||
from sqlalchemy import text
|
||||
from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker, create_async_engine
|
||||
|
||||
import app.models # noqa: F401 — registers core models
|
||||
import app.models.outbox # noqa: F401 — event_outbox is NOT re-exported by app.models
|
||||
import app.plugins.builtins as _builtins_pkg
|
||||
from app.core.db import Base
|
||||
from app.plugins.builtins.automation.models import (
|
||||
|
||||
for _importer, _modname, _ispkg in pkgutil.iter_modules(_builtins_pkg.__path__):
|
||||
if not _ispkg:
|
||||
continue
|
||||
try:
|
||||
importlib.import_module(f"app.plugins.builtins.{_modname}.models")
|
||||
except ImportError:
|
||||
pass # plugin without models module
|
||||
except Exception: # pragma: no cover - defensive
|
||||
pass
|
||||
|
||||
from app.plugins.builtins.automation.models import ( # noqa: E402 — after dynamic plugin-model discovery
|
||||
AgentRun,
|
||||
AutomationRun,
|
||||
)
|
||||
from app.plugins.builtins.automation.services import (
|
||||
from app.plugins.builtins.automation.services import ( # noqa: E402 — after dynamic plugin-model discovery
|
||||
AgentService,
|
||||
AutomationService,
|
||||
CronJobService,
|
||||
)
|
||||
|
||||
|
||||
def _ephemeral_db_url() -> str:
|
||||
"""Derive an ephemeral test DB URL from DATABASE_URL/.env.test."""
|
||||
base_url = os.environ.get(
|
||||
"DATABASE_URL",
|
||||
"postgresql+asyncpg://leocrm_test:test123@localhost:5432/leocrm_test",
|
||||
)
|
||||
return f"{base_url.rsplit('/', 1)[0]}/automation_test_{uuid.uuid4().hex[:8]}"
|
||||
|
||||
|
||||
# ─── Fixtures ───
|
||||
|
||||
|
||||
@pytest_asyncio.fixture
|
||||
async def db() -> AsyncGenerator[AsyncSession, None]:
|
||||
"""Create an in-memory SQLite database for testing."""
|
||||
engine = create_async_engine(
|
||||
"sqlite+aiosqlite:///:memory:",
|
||||
echo=False,
|
||||
)
|
||||
"""Create an ephemeral PostgreSQL database for this test run."""
|
||||
db_url = _ephemeral_db_url()
|
||||
admin_url = db_url.rsplit("/", 1)[0] + "/postgres"
|
||||
|
||||
from sqlalchemy.ext.asyncio import create_async_engine as _cae
|
||||
|
||||
admin_engine = _cae(admin_url, isolation_level="AUTOCOMMIT")
|
||||
async with admin_engine.connect() as conn:
|
||||
await conn.execute(text(f'CREATE DATABASE "{db_url.rsplit("/", 1)[1]}"'))
|
||||
await admin_engine.dispose()
|
||||
|
||||
# Plugin models use the pgvector Vector type — enable the extension in
|
||||
# the fresh database before create_all runs (must connect to the target
|
||||
# DB itself; CREATE EXTENSION has no ON DATABASE clause).
|
||||
ext_engine = _cae(db_url, isolation_level="AUTOCOMMIT")
|
||||
async with ext_engine.connect() as conn:
|
||||
await conn.execute(text("CREATE EXTENSION IF NOT EXISTS vector"))
|
||||
await ext_engine.dispose()
|
||||
|
||||
engine = create_async_engine(db_url, echo=False)
|
||||
|
||||
async with engine.begin() as conn:
|
||||
await conn.run_sync(Base.metadata.create_all)
|
||||
|
||||
async_session = async_sessionmaker(engine, class_=AsyncSession, expire_on_commit=False)
|
||||
|
||||
try:
|
||||
async with async_session() as session:
|
||||
yield session
|
||||
|
||||
finally:
|
||||
await engine.dispose()
|
||||
admin_engine2 = _cae(admin_url, isolation_level="AUTOCOMMIT")
|
||||
async with admin_engine2.connect() as conn:
|
||||
await conn.execute(text(f'DROP DATABASE IF EXISTS "{db_url.rsplit("/", 1)[1]}"'))
|
||||
await admin_engine2.dispose()
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def tenant_id() -> uuid.UUID:
|
||||
return uuid.uuid4()
|
||||
@pytest_asyncio.fixture
|
||||
async def tenant_id(db: AsyncSession) -> uuid.UUID:
|
||||
"""Create a real tenant row — PostgreSQL enforces FKs, unlike SQLite."""
|
||||
from app.models.tenant import Tenant
|
||||
|
||||
tid = uuid.uuid4()
|
||||
db.add(Tenant(id=tid, name="Test Org", slug=f"test-{tid.hex[:8]}"))
|
||||
await db.commit()
|
||||
return tid
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def user_id() -> uuid.UUID:
|
||||
return uuid.uuid4()
|
||||
@pytest_asyncio.fixture
|
||||
async def user_id(db: AsyncSession, tenant_id: uuid.UUID) -> uuid.UUID:
|
||||
"""Create a real user row belonging to the test tenant."""
|
||||
from app.models.user import User
|
||||
|
||||
uid = uuid.uuid4()
|
||||
db.add(
|
||||
User(
|
||||
id=uid,
|
||||
email=f"test-{uid.hex[:8]}@example.com",
|
||||
name="Test User",
|
||||
password_hash="not-a-real-hash",
|
||||
is_active=True,
|
||||
)
|
||||
)
|
||||
await db.commit()
|
||||
return uid
|
||||
|
||||
|
||||
# ─── AgentService Tests ───
|
||||
@@ -425,11 +497,17 @@ class TestDryRunMode:
|
||||
assert automation.dry_run is True
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_dry_run_flag_in_run(self, db: AsyncSession, tenant_id: uuid.UUID):
|
||||
async def test_dry_run_flag_in_run(self, db: AsyncSession, tenant_id: uuid.UUID, user_id: uuid.UUID):
|
||||
"""Test that dry_run flag is stored in AutomationRun."""
|
||||
# PostgreSQL enforces the FK to automations — create a real one first
|
||||
data = {"name": "dry-run-flag", "description": "", "trigger_type": "manual",
|
||||
"trigger_config": {}, "conditions": [], "actions": [],
|
||||
"is_active": True, "dry_run": True}
|
||||
automation = await AutomationService.create(db, tenant_id, data, user_id=user_id)
|
||||
|
||||
run = AutomationRun(
|
||||
tenant_id=tenant_id,
|
||||
automation_id=uuid.uuid4(),
|
||||
automation_id=automation.id,
|
||||
status="dry_run",
|
||||
started_at=datetime.now(UTC),
|
||||
dry_run=True,
|
||||
@@ -475,7 +553,9 @@ class TestRateLimiting:
|
||||
)
|
||||
recent_runs = result.scalar() or 0
|
||||
assert recent_runs == 2
|
||||
assert recent_runs < agent.max_executions_per_hour # 2 < 2 is False, so limit would be hit
|
||||
# With max_executions_per_hour=2 and 2 runs in the window, the limit
|
||||
# is reached — the next execution must be blocked.
|
||||
assert recent_runs >= agent.max_executions_per_hour
|
||||
|
||||
|
||||
# ─── Budget Limit Tests ───
|
||||
@@ -512,7 +592,8 @@ class TestBudgetLimit:
|
||||
.where(AgentRun.agent_id == agent.id)
|
||||
)
|
||||
total_cost = float(cost_result.scalar() or 0.0)
|
||||
assert total_cost == 0.6
|
||||
# FLOAT column accumulates binary rounding (0.6000000000000001)
|
||||
assert total_cost == pytest.approx(0.6)
|
||||
assert total_cost >= agent.budget_limit_usd # 0.6 >= 0.5, budget exceeded
|
||||
|
||||
|
||||
|
||||
@@ -6,6 +6,7 @@ import uuid
|
||||
from datetime import datetime
|
||||
from typing import Any
|
||||
|
||||
from pgvector.sqlalchemy import Vector
|
||||
from sqlalchemy import (
|
||||
Boolean,
|
||||
DateTime,
|
||||
@@ -13,14 +14,12 @@ from sqlalchemy import (
|
||||
Index,
|
||||
String,
|
||||
)
|
||||
from sqlalchemy.dialects.postgresql import JSONB
|
||||
from sqlalchemy.dialects.postgresql import JSONB, TSVECTOR
|
||||
from sqlalchemy.dialects.postgresql import UUID as PGUUID
|
||||
from sqlalchemy.orm import Mapped, mapped_column
|
||||
|
||||
from app.core.db import Base, TenantMixin
|
||||
from app.models.owned_mixin import OwnedMixin
|
||||
from sqlalchemy.dialects.postgresql import TSVECTOR
|
||||
from pgvector.sqlalchemy import Vector
|
||||
|
||||
|
||||
class Calendar(Base, TenantMixin, OwnedMixin):
|
||||
|
||||
@@ -4,7 +4,7 @@ from __future__ import annotations
|
||||
|
||||
from app.plugins.base import BasePlugin
|
||||
from app.plugins.manifest import (
|
||||
FrontendDetailTab,
|
||||
FrontendDashboardWidget,
|
||||
FrontendMenuItem,
|
||||
FrontendPageRoute,
|
||||
PluginManifest,
|
||||
@@ -40,6 +40,18 @@ class CalendarPlugin(BasePlugin):
|
||||
],
|
||||
events=[],
|
||||
migrations=["0001_initial.sql", "0002_add_deleted_at.sql"],
|
||||
dashboard_widgets=[
|
||||
FrontendDashboardWidget(
|
||||
id="calendar_upcoming",
|
||||
label_key="dashboard.calendarUpcoming",
|
||||
label="Upcoming Appointments",
|
||||
component="@/components/dashboard/CalendarUpcomingWidget",
|
||||
icon="Calendar",
|
||||
order=30,
|
||||
col_span=1,
|
||||
permission="calendar:read",
|
||||
),
|
||||
],
|
||||
permissions=[
|
||||
"calendar:read",
|
||||
"calendar:write",
|
||||
@@ -48,14 +60,14 @@ class CalendarPlugin(BasePlugin):
|
||||
"calendar:admin",
|
||||
],
|
||||
menu_items=[
|
||||
FrontendMenuItem(label_key='nav.calendar', label='Kalender', path='/calendar', icon='Calendar', order=20),
|
||||
FrontendMenuItem(label_key='nav.calendar', label='Kalender', path='/calendar', icon='Calendar', order=20, permission='calendar:read'),
|
||||
],
|
||||
page_routes=[
|
||||
FrontendPageRoute(path='/calendar', component='@/pages/Calendar', protected=True),
|
||||
],
|
||||
detail_tabs=[
|
||||
FrontendDetailTab(entity_type='contact', label_key='tabs.calendar', label='Calendar', component='@/components/contact/ContactCalendarTab', icon='Calendar', order=30, permission='calendar:read'),
|
||||
FrontendPageRoute(path='/calendar', component='@/pages/Calendar', protected=True, permission='calendar:read'),
|
||||
],
|
||||
# BUG (ghost component): ContactCalendarTab does not exist in the
|
||||
# frontend — tab removed until implemented (Block I-D).
|
||||
detail_tabs=[],
|
||||
author="LeoCRM Team",
|
||||
min_app_version="1.0.0",
|
||||
hooks=["calendar.before_appointment", "calendar.after_appointment"],
|
||||
|
||||
@@ -21,6 +21,7 @@ from fastapi.responses import StreamingResponse
|
||||
from sqlalchemy import select, update
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
|
||||
from app.core.audit import log_audit
|
||||
from app.core.db import get_db
|
||||
from app.deps import get_current_user, require_admin, require_permission
|
||||
from app.plugins.builtins.calendar.ics_utils import (
|
||||
@@ -1023,5 +1024,3 @@ async def book_resource(
|
||||
"start_at": booking.start_at.isoformat(),
|
||||
"end_at": booking.end_at.isoformat(),
|
||||
}
|
||||
|
||||
from app.core.audit import log_audit
|
||||
@@ -0,0 +1,62 @@
|
||||
"""Public contract for the contacts plugin.
|
||||
|
||||
Exposes the symbols that other core modules and plugins need without
|
||||
importing from internal modules directly (Block C7: dashboard counts).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Any
|
||||
|
||||
from sqlalchemy import func, select
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
|
||||
from app.core.visibility import apply_visibility_filter
|
||||
from app.models.contact import Contact
|
||||
from app.plugins.builtins.contracts import get_contract_registry
|
||||
|
||||
|
||||
class ContactsContract:
|
||||
"""Public API surface for the contacts plugin."""
|
||||
|
||||
contract_name = "contacts"
|
||||
|
||||
@staticmethod
|
||||
async def get_counts(
|
||||
db: AsyncSession,
|
||||
tenant_id: Any,
|
||||
user_id: Any,
|
||||
is_system_admin: bool = False,
|
||||
) -> dict[str, int]:
|
||||
"""Return visibility-filtered contact/company/person counts."""
|
||||
queries = []
|
||||
for type_filter in (None, "company", "person"):
|
||||
query = select(func.count(Contact.id)).where(
|
||||
Contact.tenant_id == tenant_id,
|
||||
Contact.deleted_at.is_(None),
|
||||
)
|
||||
if type_filter is not None:
|
||||
query = query.where(Contact.type == type_filter)
|
||||
query = await apply_visibility_filter(
|
||||
db, query, "contact", Contact, user_id, tenant_id, is_system_admin
|
||||
)
|
||||
queries.append(query)
|
||||
|
||||
results = [((await db.execute(q)).scalar() or 0) for q in queries]
|
||||
return {
|
||||
"contacts": results[0],
|
||||
"companies": results[1],
|
||||
"persons": results[2],
|
||||
"total": results[0],
|
||||
}
|
||||
|
||||
@classmethod
|
||||
def get_function(cls, name: str):
|
||||
"""Return a callable exposed by this contract, or None if absent."""
|
||||
return getattr(cls, name, None)
|
||||
|
||||
|
||||
# ─── self-registration ───
|
||||
|
||||
_contract = ContactsContract()
|
||||
get_contract_registry().register("contacts", _contract)
|
||||
@@ -9,27 +9,67 @@ from __future__ import annotations
|
||||
import logging
|
||||
|
||||
from app.plugins.base import BasePlugin
|
||||
from app.plugins.manifest import PluginManifest
|
||||
from app.plugins.manifest import (
|
||||
FrontendDashboardWidget,
|
||||
PluginManifest,
|
||||
PluginRouteDef,
|
||||
)
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class ContactsPlugin(BasePlugin):
|
||||
"""Contacts plugin — manages Contact entity lifecycle (models, permissions, restore, history).
|
||||
"""Contacts plugin — owns the full Contact domain (Block B1).
|
||||
|
||||
Routes remain in app/routes/contacts.py as core routes, but entity lifecycle
|
||||
(permissions, entity models, restore, history) is managed through on_activate/on_deactivate.
|
||||
Routes (contacts, companies, contact folders, folder permissions) live in
|
||||
this plugin and are mounted via manifest.routes with
|
||||
require_active_plugin("contacts") protection. Entity lifecycle
|
||||
(permissions, entity models, restore, history) is managed through
|
||||
on_activate/on_deactivate like every other business plugin.
|
||||
"""
|
||||
|
||||
manifest = PluginManifest(
|
||||
name="contacts",
|
||||
version="1.0.0",
|
||||
version="1.1.0",
|
||||
display_name="Contacts",
|
||||
description="Core CRM contacts — persons and companies.",
|
||||
dependencies=[],
|
||||
routes=[], # Routes are registered as core routes in main.py
|
||||
routes=[
|
||||
PluginRouteDef(
|
||||
path="/api/v1/contacts",
|
||||
module="app.plugins.builtins.contacts.routes",
|
||||
router_attr="router",
|
||||
),
|
||||
PluginRouteDef(
|
||||
path="/api/v1/companies",
|
||||
module="app.plugins.builtins.contacts.company_routes",
|
||||
router_attr="router",
|
||||
),
|
||||
PluginRouteDef(
|
||||
path="/api/v1/contact-folders",
|
||||
module="app.plugins.builtins.contacts.folder_routes",
|
||||
router_attr="router",
|
||||
),
|
||||
PluginRouteDef(
|
||||
path="/api/v1/contact-folders",
|
||||
module="app.plugins.builtins.contacts.folder_permission_routes",
|
||||
router_attr="router",
|
||||
),
|
||||
],
|
||||
events=[],
|
||||
migrations=[],
|
||||
dashboard_widgets=[
|
||||
FrontendDashboardWidget(
|
||||
id="recent_contacts",
|
||||
label_key="dashboard.recentContacts",
|
||||
label="Recent Contacts",
|
||||
component="@/components/dashboard/RecentContactsWidget",
|
||||
icon="Users",
|
||||
order=10,
|
||||
col_span=2,
|
||||
permission="contacts:read",
|
||||
),
|
||||
],
|
||||
permissions=[
|
||||
"contacts:read",
|
||||
"contacts:write",
|
||||
|
||||
@@ -59,20 +59,31 @@ class ContractRegistry:
|
||||
cls._instance = super().__new__(cls)
|
||||
cls._instance._contracts: dict[str, Any] = {}
|
||||
cls._instance._loaded: set[str] = set()
|
||||
cls._instance._unregistered: set[str] = set()
|
||||
return cls._instance
|
||||
|
||||
# ─── registration ───
|
||||
|
||||
def register(self, plugin_name: str, contract: Any) -> None:
|
||||
"""Register or replace a contract for a plugin."""
|
||||
"""Register or replace a contract for a plugin.
|
||||
|
||||
Clears the unregistered marker so a later deactivation can be
|
||||
distinguished from a fresh lazy-load again (ARCH-014).
|
||||
"""
|
||||
self._unregistered.discard(plugin_name)
|
||||
self._contracts[plugin_name] = contract
|
||||
self._loaded.add(plugin_name)
|
||||
logger.debug("Contract registered for plugin '%s'", plugin_name)
|
||||
|
||||
def unregister(self, plugin_name: str) -> None:
|
||||
"""Remove a contract (e.g. when the plugin is deactivated)."""
|
||||
"""Remove a contract (e.g. when the plugin is deactivated).
|
||||
|
||||
Marks the plugin as explicitly unregistered so later ``get_contract``
|
||||
calls cannot resurrect the contract via lazy-loading (ARCH-014).
|
||||
"""
|
||||
self._contracts.pop(plugin_name, None)
|
||||
self._loaded.discard(plugin_name)
|
||||
self._unregistered.add(plugin_name)
|
||||
|
||||
# ─── lookup ───
|
||||
|
||||
@@ -85,6 +96,11 @@ class ContractRegistry:
|
||||
if plugin_name in self._contracts:
|
||||
return self._contracts[plugin_name]
|
||||
|
||||
# Explicitly unregistered (deactivated): never resurrect via
|
||||
# lazy-loading (ARCH-014) — the deactivated contract must stay gone.
|
||||
if plugin_name in self._unregistered:
|
||||
return None
|
||||
|
||||
if plugin_name not in self._loaded:
|
||||
self._try_lazy_load(plugin_name)
|
||||
|
||||
|
||||
@@ -0,0 +1,111 @@
|
||||
"""DMS gemeinsame Helper & Konstanten — BUG-018 God-Object-Split."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import uuid
|
||||
|
||||
from fastapi import HTTPException
|
||||
|
||||
OFFICE_EXTENSIONS = {
|
||||
".docx": "docx",
|
||||
".xlsx": "xlsx",
|
||||
".pptx": "pptx",
|
||||
}
|
||||
|
||||
# Max file size: 100 MB
|
||||
MAX_FILE_SIZE = 100 * 1024 * 1024
|
||||
|
||||
|
||||
def _parse_uuid(val: str, field: str) -> uuid.UUID:
|
||||
try:
|
||||
return uuid.UUID(val)
|
||||
except (ValueError, TypeError):
|
||||
raise HTTPException(
|
||||
400, detail={"detail": f"Invalid {field}", "code": "invalid_id"}
|
||||
) from None
|
||||
|
||||
|
||||
def _file_storage_path(tenant_id: uuid.UUID, file_id: uuid.UUID) -> str:
|
||||
"""Build relative storage path for a file (relative to storage base)."""
|
||||
return f"{tenant_id}/{file_id}"
|
||||
|
||||
|
||||
def _get_file_extension(filename: str) -> str:
|
||||
"""Extract lowercase extension including dot."""
|
||||
return os.path.splitext(filename)[1].lower()
|
||||
|
||||
|
||||
|
||||
def _sanitize_filename(filename: str) -> str:
|
||||
"""Sanitize a filename for safe use in Content-Disposition headers."""
|
||||
import re
|
||||
# Extract basename only (strip any path components)
|
||||
safe = os.path.basename(filename.replace('\\', '/'))
|
||||
# Remove dangerous characters (keep alnum, dot, dash, underscore, space, unicode)
|
||||
safe = re.sub(r'[^a-zA-Z0-9.\-_\u00c0-\u017f\u4e00-\u9fff ]', '_', safe)
|
||||
# Collapse consecutive dots (path traversal prevention)
|
||||
safe = re.sub(r'\.{2,}', '_', safe)
|
||||
# Collapse multiple spaces
|
||||
safe = re.sub(r' {2,}', ' ', safe)
|
||||
# Strip leading dots and whitespace
|
||||
safe = safe.lstrip('.').strip()
|
||||
# Limit length
|
||||
if len(safe) > 200:
|
||||
name, ext = safe.rsplit('.', 1) if '.' in safe[:200] else (safe[:200], '')
|
||||
safe = name[:200] + ('.' + ext if ext else '')
|
||||
return safe or 'file'
|
||||
|
||||
# Blocked file extensions for security
|
||||
BLOCKED_EXTENSIONS = {
|
||||
".exe", ".bat", ".cmd", ".sh", ".jar", ".com", ".scr", ".msi",
|
||||
".dll", ".vbs", ".ps1", ".app", ".bin", ".reg", ".inf",
|
||||
".php", ".py", ".pl", ".asp", ".aspx", ".jsp", ".svg", ".htaccess",
|
||||
".phtml", ".pht", ".cgi", ".cfm", ".erb",
|
||||
}
|
||||
|
||||
# Allowed MIME types for upload validation
|
||||
ALLOWED_MIME_PREFIXES = {
|
||||
"application/pdf",
|
||||
"application/msword",
|
||||
"application/vnd.openxmlformats-officedocument",
|
||||
"application/vnd.oasis.opendocument",
|
||||
"application/vnd.ms-excel",
|
||||
"application/vnd.ms-powerpoint",
|
||||
"application/zip",
|
||||
"application/gzip",
|
||||
"application/x-tar",
|
||||
"application/json",
|
||||
"application/xml",
|
||||
"application/rtf",
|
||||
"application/x-7z-compressed",
|
||||
"application/x-rar-compressed",
|
||||
"text/plain",
|
||||
"text/csv",
|
||||
"text/html",
|
||||
"text/markdown",
|
||||
"image/png",
|
||||
"image/jpeg",
|
||||
"image/gif",
|
||||
"image/webp",
|
||||
"image/bmp",
|
||||
"image/tiff",
|
||||
"image/x-icon",
|
||||
"audio/",
|
||||
"video/",
|
||||
"application/octet-stream",
|
||||
}
|
||||
|
||||
|
||||
def _is_blocked_filetype(filename: str) -> bool:
|
||||
"""Check if a file has a blocked (dangerous) extension."""
|
||||
ext = os.path.splitext(filename)[1].lower()
|
||||
return ext in BLOCKED_EXTENSIONS
|
||||
|
||||
|
||||
chunk_size = 1024 * 1024 # 1MB chunks for streaming uploads
|
||||
|
||||
# ─── Folders ───
|
||||
|
||||
# Public stream-chunk constant (original contract name from tests/test_p1_6_dms_streaming.py)
|
||||
CHUNK_SIZE = chunk_size
|
||||
@@ -0,0 +1,383 @@
|
||||
"""DMS Folder-CRUD Routen — extrahiert aus routes.py (BUG-018 God-Object-Split)."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import uuid
|
||||
|
||||
from fastapi import (
|
||||
APIRouter,
|
||||
Depends,
|
||||
HTTPException,
|
||||
Response,
|
||||
status,
|
||||
)
|
||||
from sqlalchemy import select, update
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
|
||||
from app.core.db import get_db
|
||||
from app.core.visibility import apply_visibility_filter, check_single_entity_access
|
||||
from app.deps import get_current_user, require_permission
|
||||
from app.plugins.builtins.dms.common import (
|
||||
_parse_uuid,
|
||||
)
|
||||
from app.plugins.builtins.dms.models import File as DmsFile
|
||||
from app.plugins.builtins.dms.models import Folder
|
||||
from app.plugins.builtins.dms.schemas import FolderCreate, FolderUpdate
|
||||
from app.plugins.builtins.permissions.contracts import get_contract as get_perms_contract
|
||||
|
||||
_perms_contract = get_perms_contract()
|
||||
Permission = _perms_contract.Permission
|
||||
|
||||
router = APIRouter(tags=["dms"])
|
||||
|
||||
@router.get("/folders", dependencies=[Depends(require_permission("dms:read"))])
|
||||
async def list_folders(
|
||||
parent_id: str | None = None,
|
||||
db: AsyncSession = Depends(get_db),
|
||||
current_user: dict = Depends(get_current_user),
|
||||
):
|
||||
"""AC1: GET /api/v1/dms/folders → 200 + folder tree (recursive)."""
|
||||
tenant_id = uuid.UUID(current_user["tenant_id"])
|
||||
|
||||
# Fetch all non-deleted folders for tenant with visibility filter
|
||||
user_id = uuid.UUID(current_user["user_id"])
|
||||
is_system_admin = current_user.get("role") == "admin"
|
||||
query = select(Folder).where(
|
||||
Folder.tenant_id == tenant_id,
|
||||
Folder.deleted_at.is_(None),
|
||||
)
|
||||
query = await apply_visibility_filter(
|
||||
db, query, "dms_folder", Folder, user_id, tenant_id, is_system_admin
|
||||
)
|
||||
result = await db.execute(query)
|
||||
all_folders = result.scalars().all()
|
||||
|
||||
# Build lookup map
|
||||
folder_map: dict[uuid.UUID, dict] = {}
|
||||
for f in all_folders:
|
||||
folder_map[f.id] = {
|
||||
"id": str(f.id),
|
||||
"name": f.name,
|
||||
"parent_id": str(f.parent_id) if f.parent_id else None,
|
||||
"created_by": str(f.created_by),
|
||||
"deleted_at": None,
|
||||
"path": "",
|
||||
"children": [],
|
||||
}
|
||||
|
||||
# Build path for each folder
|
||||
def _build_path(folder_id: uuid.UUID) -> str:
|
||||
if folder_id not in folder_map:
|
||||
return ""
|
||||
f = folder_map[folder_id]
|
||||
if f["parent_id"] and uuid.UUID(f["parent_id"]) in folder_map:
|
||||
parent_path = _build_path(uuid.UUID(f["parent_id"]))
|
||||
return f"{parent_path}/{f['name']}"
|
||||
return f["name"]
|
||||
|
||||
for fid in folder_map:
|
||||
folder_map[fid]["path"] = _build_path(fid)
|
||||
|
||||
# Build tree
|
||||
root_nodes: list[dict] = []
|
||||
target_parent: uuid.UUID | None = None
|
||||
if parent_id is not None:
|
||||
target_parent = _parse_uuid(parent_id, "parent_id")
|
||||
|
||||
for f in all_folders:
|
||||
node = folder_map[f.id]
|
||||
if f.parent_id is not None and f.parent_id in folder_map:
|
||||
folder_map[f.parent_id]["children"].append(node)
|
||||
elif f.parent_id is None:
|
||||
root_nodes.append(node)
|
||||
|
||||
if target_parent is not None:
|
||||
# Return children of specified parent
|
||||
parent_node = folder_map.get(target_parent)
|
||||
if parent_node is None:
|
||||
raise HTTPException(
|
||||
404, detail={"detail": "Parent folder not found", "code": "not_found"}
|
||||
)
|
||||
return parent_node["children"]
|
||||
|
||||
return root_nodes
|
||||
|
||||
|
||||
@router.post("/folders", status_code=status.HTTP_201_CREATED, dependencies=[Depends(require_permission("dms:write"))])
|
||||
async def create_folder(
|
||||
body: FolderCreate,
|
||||
db: AsyncSession = Depends(get_db),
|
||||
current_user: dict = Depends(get_current_user),
|
||||
):
|
||||
"""AC2: POST /api/v1/dms/folders → 201, folder created with path."""
|
||||
tenant_id = uuid.UUID(current_user["tenant_id"])
|
||||
user_id = uuid.UUID(current_user["user_id"])
|
||||
parent_id = _parse_uuid(body.parent_id, "parent_id") if body.parent_id else None
|
||||
|
||||
# Validate parent exists if specified
|
||||
if parent_id is not None:
|
||||
parent_result = await db.execute(
|
||||
select(Folder).where(
|
||||
Folder.id == parent_id,
|
||||
Folder.tenant_id == tenant_id,
|
||||
Folder.deleted_at.is_(None),
|
||||
)
|
||||
)
|
||||
if parent_result.scalar_one_or_none() is None:
|
||||
raise HTTPException(
|
||||
404, detail={"detail": "Parent folder not found", "code": "not_found"}
|
||||
)
|
||||
|
||||
# Check name uniqueness within same parent (non-deleted)
|
||||
existing = await db.execute(
|
||||
select(Folder).where(
|
||||
Folder.tenant_id == tenant_id,
|
||||
Folder.name == body.name,
|
||||
Folder.parent_id == parent_id if parent_id else Folder.parent_id.is_(None),
|
||||
Folder.deleted_at.is_(None),
|
||||
)
|
||||
)
|
||||
if existing.scalar_one_or_none() is not None:
|
||||
raise HTTPException(
|
||||
409, detail={"detail": "Folder name already exists", "code": "duplicate"}
|
||||
)
|
||||
|
||||
# Lifecycle hook: dms.folder.before_create
|
||||
from app.core.hooks import do_action
|
||||
await do_action("dms.folder.before_create", body, db=db, tenant_id=tenant_id, user_id=user_id)
|
||||
|
||||
folder = Folder(
|
||||
tenant_id=tenant_id,
|
||||
name=body.name,
|
||||
parent_id=parent_id,
|
||||
created_by=user_id,
|
||||
)
|
||||
db.add(folder)
|
||||
await db.flush()
|
||||
|
||||
# Lifecycle hook: dms.folder.after_create
|
||||
await do_action("dms.folder.after_create", {'id': str(folder.id), 'name': folder.name, 'parent_id': str(folder.parent_id) if folder.parent_id else None}, db=db, tenant_id=tenant_id, user_id=user_id)
|
||||
|
||||
# Build path
|
||||
path = body.name
|
||||
if parent_id is not None:
|
||||
parent_path_result = await db.execute(select(Folder).where(Folder.id == parent_id))
|
||||
parent_folder = parent_path_result.scalar_one_or_none()
|
||||
if parent_folder:
|
||||
# Recursively build path
|
||||
path_parts = [body.name]
|
||||
current = parent_folder
|
||||
while current is not None:
|
||||
path_parts.insert(0, current.name)
|
||||
if current.parent_id is not None:
|
||||
cur_result = await db.execute(
|
||||
select(Folder).where(Folder.id == current.parent_id)
|
||||
)
|
||||
current = cur_result.scalar_one_or_none()
|
||||
else:
|
||||
current = None
|
||||
path = "/".join(path_parts)
|
||||
|
||||
return {
|
||||
"id": str(folder.id),
|
||||
"name": folder.name,
|
||||
"parent_id": str(folder.parent_id) if folder.parent_id else None,
|
||||
"created_by": str(folder.created_by),
|
||||
"deleted_at": None,
|
||||
"path": path,
|
||||
"children": [],
|
||||
}
|
||||
|
||||
|
||||
@router.patch("/folders/{folder_id}", dependencies=[Depends(require_permission("dms:write"))])
|
||||
async def update_folder(
|
||||
folder_id: str,
|
||||
body: FolderUpdate,
|
||||
db: AsyncSession = Depends(get_db),
|
||||
current_user: dict = Depends(get_current_user),
|
||||
):
|
||||
"""AC3: PATCH /api/v1/dms/folders/{id} → 200, rename/move."""
|
||||
tenant_id = uuid.UUID(current_user["tenant_id"])
|
||||
user_id = uuid.UUID(current_user["user_id"])
|
||||
is_system_admin = current_user.get("role") == "admin"
|
||||
fid = _parse_uuid(folder_id, "folder_id")
|
||||
|
||||
result = await db.execute(
|
||||
select(Folder).where(
|
||||
Folder.id == fid,
|
||||
Folder.tenant_id == tenant_id,
|
||||
Folder.deleted_at.is_(None),
|
||||
)
|
||||
)
|
||||
folder = result.scalar_one_or_none()
|
||||
if folder is None:
|
||||
raise HTTPException(404, detail={"detail": "Folder not found", "code": "not_found"})
|
||||
|
||||
if not await check_single_entity_access(db, "dms_folder", fid, user_id, tenant_id, "write", is_system_admin):
|
||||
raise HTTPException(403, detail={"detail": "Access denied", "code": "forbidden"})
|
||||
|
||||
data = body.model_dump(exclude_unset=True)
|
||||
|
||||
if "name" in data and data["name"] is not None:
|
||||
# Check uniqueness if name is changing
|
||||
new_parent_id = folder.parent_id
|
||||
if "parent_id" in data and data["parent_id"] is not None:
|
||||
new_parent_id = _parse_uuid(data["parent_id"], "parent_id")
|
||||
|
||||
dup = await db.execute(
|
||||
select(Folder).where(
|
||||
Folder.tenant_id == tenant_id,
|
||||
Folder.name == data["name"],
|
||||
Folder.id != fid,
|
||||
Folder.parent_id == new_parent_id if new_parent_id else Folder.parent_id.is_(None),
|
||||
Folder.deleted_at.is_(None),
|
||||
)
|
||||
)
|
||||
if dup.scalar_one_or_none() is not None:
|
||||
raise HTTPException(
|
||||
409, detail={"detail": "Folder name already exists", "code": "duplicate"}
|
||||
)
|
||||
folder.name = data["name"]
|
||||
|
||||
if "parent_id" in data:
|
||||
new_parent = _parse_uuid(data["parent_id"], "parent_id") if data["parent_id"] else None
|
||||
if new_parent is not None:
|
||||
# Validate parent exists and not creating a cycle
|
||||
if new_parent == fid:
|
||||
raise HTTPException(
|
||||
400, detail={"detail": "Cannot move folder into itself", "code": "invalid_move"}
|
||||
)
|
||||
|
||||
parent_result = await db.execute(
|
||||
select(Folder).where(
|
||||
Folder.id == new_parent,
|
||||
Folder.tenant_id == tenant_id,
|
||||
Folder.deleted_at.is_(None),
|
||||
)
|
||||
)
|
||||
if parent_result.scalar_one_or_none() is None:
|
||||
raise HTTPException(
|
||||
404, detail={"detail": "Parent folder not found", "code": "not_found"}
|
||||
)
|
||||
|
||||
# Check for cycle: ensure new_parent is not a descendant of folder
|
||||
async def _is_descendant(ancestor_id: uuid.UUID, descendant_id: uuid.UUID) -> bool:
|
||||
cur_result = await db.execute(select(Folder).where(Folder.id == descendant_id))
|
||||
cur = cur_result.scalar_one_or_none()
|
||||
while cur is not None and cur.parent_id is not None:
|
||||
if cur.parent_id == ancestor_id:
|
||||
return True
|
||||
p_result = await db.execute(select(Folder).where(Folder.id == cur.parent_id))
|
||||
cur = p_result.scalar_one_or_none()
|
||||
return False
|
||||
|
||||
if await _is_descendant(fid, new_parent):
|
||||
raise HTTPException(
|
||||
400,
|
||||
detail={
|
||||
"detail": "Cannot move folder into its own descendant",
|
||||
"code": "invalid_move",
|
||||
},
|
||||
)
|
||||
|
||||
folder.parent_id = new_parent
|
||||
|
||||
await db.flush()
|
||||
|
||||
# Build path
|
||||
path_parts = [folder.name]
|
||||
current_id = folder.parent_id
|
||||
while current_id is not None:
|
||||
cur_result = await db.execute(select(Folder).where(Folder.id == current_id))
|
||||
cur = cur_result.scalar_one_or_none()
|
||||
if cur is None:
|
||||
break
|
||||
path_parts.insert(0, cur.name)
|
||||
current_id = cur.parent_id
|
||||
path = "/".join(path_parts)
|
||||
|
||||
return {
|
||||
"id": str(folder.id),
|
||||
"name": folder.name,
|
||||
"parent_id": str(folder.parent_id) if folder.parent_id else None,
|
||||
"created_by": str(folder.created_by),
|
||||
"deleted_at": None,
|
||||
"path": path,
|
||||
"children": [],
|
||||
}
|
||||
|
||||
|
||||
@router.delete("/folders/{folder_id}", status_code=status.HTTP_204_NO_CONTENT, dependencies=[Depends(require_permission("dms:delete"))])
|
||||
async def delete_folder(
|
||||
folder_id: str,
|
||||
db: AsyncSession = Depends(get_db),
|
||||
current_user: dict = Depends(get_current_user),
|
||||
):
|
||||
"""AC4: DELETE /api/v1/dms/folders/{id} → 204, soft-delete with cascade."""
|
||||
tenant_id = uuid.UUID(current_user["tenant_id"])
|
||||
user_id = uuid.UUID(current_user["user_id"])
|
||||
is_system_admin = current_user.get("role") == "admin"
|
||||
fid = _parse_uuid(folder_id, "folder_id")
|
||||
|
||||
result = await db.execute(
|
||||
select(Folder).where(
|
||||
Folder.id == fid,
|
||||
Folder.tenant_id == tenant_id,
|
||||
Folder.deleted_at.is_(None),
|
||||
)
|
||||
)
|
||||
folder = result.scalar_one_or_none()
|
||||
if folder is None:
|
||||
raise HTTPException(404, detail={"detail": "Folder not found", "code": "not_found"})
|
||||
|
||||
if not await check_single_entity_access(db, "dms_folder", fid, user_id, tenant_id, "delete", is_system_admin):
|
||||
raise HTTPException(403, detail={"detail": "Access denied", "code": "forbidden"})
|
||||
|
||||
# Lifecycle hook: dms.folder.before_delete
|
||||
from app.core.hooks import do_action
|
||||
await do_action("dms.folder.before_delete", db=db, tenant_id=tenant_id, user_id=user_id, folder_id=str(fid))
|
||||
|
||||
from datetime import UTC, datetime
|
||||
|
||||
now = datetime.now(UTC)
|
||||
|
||||
# Recursively collect all descendant folder IDs
|
||||
all_folder_ids: list[uuid.UUID] = [fid]
|
||||
queue: list[uuid.UUID] = [fid]
|
||||
while queue:
|
||||
current_id = queue.pop(0)
|
||||
children_result = await db.execute(
|
||||
select(Folder).where(
|
||||
Folder.parent_id == current_id,
|
||||
Folder.tenant_id == tenant_id,
|
||||
Folder.deleted_at.is_(None),
|
||||
)
|
||||
)
|
||||
for child in children_result.scalars().all():
|
||||
all_folder_ids.append(child.id)
|
||||
queue.append(child.id)
|
||||
|
||||
# Soft-delete all folders
|
||||
await db.execute(update(Folder).where(Folder.id.in_(all_folder_ids)).values(deleted_at=now))
|
||||
|
||||
# Soft-delete all files in those folders
|
||||
await db.execute(
|
||||
update(DmsFile)
|
||||
.where(
|
||||
DmsFile.tenant_id == tenant_id,
|
||||
DmsFile.folder_id.in_(all_folder_ids),
|
||||
DmsFile.deleted_at.is_(None),
|
||||
)
|
||||
.values(deleted_at=now)
|
||||
)
|
||||
|
||||
await db.flush()
|
||||
|
||||
# Lifecycle hook: dms.folder.after_delete
|
||||
from app.core.hooks import do_action
|
||||
await do_action("dms.folder.after_delete", db=db, tenant_id=tenant_id, user_id=user_id, folder_id=str(fid))
|
||||
|
||||
return Response(status_code=status.HTTP_204_NO_CONTENT)
|
||||
|
||||
|
||||
# ─── Files ───
|
||||
@@ -4,16 +4,16 @@ from __future__ import annotations
|
||||
|
||||
import uuid
|
||||
from datetime import datetime
|
||||
from typing import Any
|
||||
|
||||
from sqlalchemy import DateTime, ForeignKey, Index, Integer, String, UniqueConstraint, Text
|
||||
from pgvector.sqlalchemy import Vector
|
||||
from sqlalchemy import DateTime, ForeignKey, Index, Integer, String, Text, UniqueConstraint
|
||||
from sqlalchemy.dialects.postgresql import TSVECTOR
|
||||
from sqlalchemy.dialects.postgresql import UUID as PGUUID
|
||||
from sqlalchemy.orm import Mapped, mapped_column
|
||||
|
||||
from app.core.db import Base, TenantMixin
|
||||
from app.models.owned_mixin import OwnedMixin
|
||||
from pgvector.sqlalchemy import Vector
|
||||
from sqlalchemy.dialects.postgresql import TSVECTOR
|
||||
from typing import Any
|
||||
|
||||
|
||||
class Folder(Base, TenantMixin, OwnedMixin):
|
||||
|
||||
@@ -4,7 +4,6 @@ from __future__ import annotations
|
||||
|
||||
from app.plugins.base import BasePlugin
|
||||
from app.plugins.manifest import (
|
||||
FrontendDetailTab,
|
||||
FrontendMenuItem,
|
||||
FrontendPageRoute,
|
||||
PluginManifest,
|
||||
@@ -38,22 +37,35 @@ class DmsPlugin(BasePlugin):
|
||||
"dms:admin",
|
||||
],
|
||||
menu_items=[
|
||||
FrontendMenuItem(label_key='nav.dms', label='Dateien', path='/dms', icon='FolderOpen', group='Dateien', order=40),
|
||||
FrontendMenuItem(label_key='nav.dms.trash', label='Papierkorb', path='/dms/trash', icon='Trash2', group='Dateien', order=41),
|
||||
FrontendMenuItem(label_key='nav.dms', label='Dateien', path='/dms', icon='FolderOpen', group='Dateien', order=40, permission='dms:read'),
|
||||
FrontendMenuItem(label_key='nav.dms.trash', label='Papierkorb', path='/dms/trash', icon='Trash2', group='Dateien', order=41, permission='dms:read'),
|
||||
],
|
||||
page_routes=[
|
||||
FrontendPageRoute(path='/dms', component='@/pages/Dms', protected=True),
|
||||
FrontendPageRoute(path='/dms/trash', component='@/pages/DmsTrash', protected=True),
|
||||
],
|
||||
detail_tabs=[
|
||||
FrontendDetailTab(entity_type='contact', label_key='tabs.files', label='Dateien', component='@/components/contact/ContactFilesTab', icon='FolderOpen', order=40, permission='dms:read'),
|
||||
FrontendPageRoute(path='/dms', component='@/pages/Dms', protected=True, permission='dms:read'),
|
||||
FrontendPageRoute(path='/dms/trash', component='@/pages/DmsTrash', protected=True, permission='dms:read'),
|
||||
],
|
||||
# BUG (ghost component): ContactFilesTab does not exist in the
|
||||
# frontend — tab removed until implemented (Block I-D).
|
||||
detail_tabs=[],
|
||||
author="LeoCRM Team",
|
||||
min_app_version="1.0.0",
|
||||
hooks=["dms.before_upload"],
|
||||
contract_version="1.0.0",
|
||||
)
|
||||
|
||||
def get_entity_models(self) -> dict[str, type]:
|
||||
"""Entity types owned by DMS for the permission resolver.
|
||||
|
||||
``file`` is a legacy alias still used by the permissions routes.
|
||||
"""
|
||||
from app.plugins.builtins.dms.models import File, Folder
|
||||
|
||||
return {
|
||||
"dms_file": File,
|
||||
"dms_folder": Folder,
|
||||
"file": File,
|
||||
}
|
||||
|
||||
async def on_activate(self, db, service_container, event_bus) -> None:
|
||||
"""Activate plugin: register restore config + history hooks."""
|
||||
await super().on_activate(db, service_container, event_bus)
|
||||
|
||||
@@ -2,12 +2,10 @@
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import uuid
|
||||
|
||||
from fastapi import (
|
||||
APIRouter,
|
||||
Body,
|
||||
Depends,
|
||||
File,
|
||||
Form,
|
||||
@@ -17,25 +15,39 @@ from fastapi import (
|
||||
status,
|
||||
)
|
||||
from fastapi.responses import StreamingResponse
|
||||
from sqlalchemy import select, update
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
|
||||
from app.core.db import get_db
|
||||
from app.core.storage import LocalStorage, get_storage_backend
|
||||
from app.core.visibility import apply_visibility_filter, check_single_entity_access
|
||||
from app.deps import get_current_user, require_permission
|
||||
|
||||
# BUG-018 God-Object-Split: Helper/Konstanten leben jetzt in common.py;
|
||||
# Re-Exports sichern Import- und Patch-Kompatibilitaet
|
||||
# (tests patchen app.plugins.builtins.dms.routes.MAX_FILE_SIZE fuer den Upload).
|
||||
from app.plugins.builtins.dms.common import ( # noqa: F401
|
||||
ALLOWED_MIME_PREFIXES,
|
||||
BLOCKED_EXTENSIONS,
|
||||
CHUNK_SIZE,
|
||||
MAX_FILE_SIZE,
|
||||
OFFICE_EXTENSIONS,
|
||||
_file_storage_path,
|
||||
_get_file_extension,
|
||||
_is_blocked_filetype,
|
||||
_parse_uuid,
|
||||
_sanitize_filename,
|
||||
chunk_size,
|
||||
)
|
||||
from app.plugins.builtins.dms.folders_routes import router as folders_router
|
||||
from app.plugins.builtins.dms.models import File as DmsFile
|
||||
from app.plugins.builtins.dms.models import Folder
|
||||
from app.plugins.builtins.dms.schemas import (
|
||||
BulkDeleteRequest,
|
||||
BulkMoveRequest,
|
||||
FileMetadataResponse,
|
||||
FileUpdate,
|
||||
FolderCreate,
|
||||
FolderUpdate,
|
||||
ShareRemoveRequest,
|
||||
ShareRequest,
|
||||
)
|
||||
from app.plugins.builtins.dms.search_bulk_routes import router as search_bulk_router
|
||||
from app.plugins.builtins.dms.sharing_routes import router as sharing_router
|
||||
from app.plugins.builtins.permissions.contracts import get_contract as get_perms_contract
|
||||
|
||||
# Get Permission model from the permissions contract
|
||||
@@ -45,460 +57,6 @@ Permission = _perms_contract.Permission
|
||||
router = APIRouter(prefix="/api/v1/dms", tags=["dms"])
|
||||
|
||||
# Office file extensions mapped to Collabora file types
|
||||
OFFICE_EXTENSIONS = {
|
||||
".docx": "docx",
|
||||
".xlsx": "xlsx",
|
||||
".pptx": "pptx",
|
||||
}
|
||||
|
||||
# Max file size: 100 MB
|
||||
MAX_FILE_SIZE = 100 * 1024 * 1024
|
||||
|
||||
|
||||
def _parse_uuid(val: str, field: str) -> uuid.UUID:
|
||||
try:
|
||||
return uuid.UUID(val)
|
||||
except (ValueError, TypeError):
|
||||
raise HTTPException(
|
||||
400, detail={"detail": f"Invalid {field}", "code": "invalid_id"}
|
||||
) from None
|
||||
|
||||
|
||||
def _file_storage_path(tenant_id: uuid.UUID, file_id: uuid.UUID) -> str:
|
||||
"""Build relative storage path for a file (relative to storage base)."""
|
||||
return f"{tenant_id}/{file_id}"
|
||||
|
||||
|
||||
def _get_file_extension(filename: str) -> str:
|
||||
"""Extract lowercase extension including dot."""
|
||||
return os.path.splitext(filename)[1].lower()
|
||||
|
||||
|
||||
|
||||
def _sanitize_filename(filename: str) -> str:
|
||||
"""Sanitize a filename for safe use in Content-Disposition headers."""
|
||||
import re
|
||||
# Extract basename only (strip any path components)
|
||||
safe = os.path.basename(filename.replace('\\', '/'))
|
||||
# Remove dangerous characters (keep alnum, dot, dash, underscore, space, unicode)
|
||||
safe = re.sub(r'[^a-zA-Z0-9.\-_\u00c0-\u017f\u4e00-\u9fff ]', '_', safe)
|
||||
# Collapse consecutive dots (path traversal prevention)
|
||||
safe = re.sub(r'\.{2,}', '_', safe)
|
||||
# Collapse multiple spaces
|
||||
safe = re.sub(r' {2,}', ' ', safe)
|
||||
# Strip leading dots and whitespace
|
||||
safe = safe.lstrip('.').strip()
|
||||
# Limit length
|
||||
if len(safe) > 200:
|
||||
name, ext = safe.rsplit('.', 1) if '.' in safe[:200] else (safe[:200], '')
|
||||
safe = name[:200] + ('.' + ext if ext else '')
|
||||
return safe or 'file'
|
||||
|
||||
# Blocked file extensions for security
|
||||
BLOCKED_EXTENSIONS = {
|
||||
".exe", ".bat", ".cmd", ".sh", ".jar", ".com", ".scr", ".msi",
|
||||
".dll", ".vbs", ".ps1", ".app", ".bin", ".reg", ".inf",
|
||||
".php", ".py", ".pl", ".asp", ".aspx", ".jsp", ".svg", ".htaccess",
|
||||
".phtml", ".pht", ".cgi", ".cfm", ".erb",
|
||||
}
|
||||
|
||||
# Allowed MIME types for upload validation
|
||||
ALLOWED_MIME_PREFIXES = {
|
||||
"application/pdf",
|
||||
"application/msword",
|
||||
"application/vnd.openxmlformats-officedocument",
|
||||
"application/vnd.oasis.opendocument",
|
||||
"application/vnd.ms-excel",
|
||||
"application/vnd.ms-powerpoint",
|
||||
"application/zip",
|
||||
"application/gzip",
|
||||
"application/x-tar",
|
||||
"application/json",
|
||||
"application/xml",
|
||||
"application/rtf",
|
||||
"application/x-7z-compressed",
|
||||
"application/x-rar-compressed",
|
||||
"text/plain",
|
||||
"text/csv",
|
||||
"text/html",
|
||||
"text/markdown",
|
||||
"image/png",
|
||||
"image/jpeg",
|
||||
"image/gif",
|
||||
"image/webp",
|
||||
"image/bmp",
|
||||
"image/tiff",
|
||||
"image/x-icon",
|
||||
"audio/",
|
||||
"video/",
|
||||
"application/octet-stream",
|
||||
}
|
||||
|
||||
|
||||
def _is_blocked_filetype(filename: str) -> bool:
|
||||
"""Check if a file has a blocked (dangerous) extension."""
|
||||
ext = os.path.splitext(filename)[1].lower()
|
||||
return ext in BLOCKED_EXTENSIONS
|
||||
|
||||
|
||||
chunk_size = 1024 * 1024 # 1MB chunks for streaming uploads
|
||||
|
||||
# ─── Folders ───
|
||||
|
||||
|
||||
@router.get("/folders", dependencies=[Depends(require_permission("dms:read"))])
|
||||
async def list_folders(
|
||||
parent_id: str | None = None,
|
||||
db: AsyncSession = Depends(get_db),
|
||||
current_user: dict = Depends(get_current_user),
|
||||
):
|
||||
"""AC1: GET /api/v1/dms/folders → 200 + folder tree (recursive)."""
|
||||
tenant_id = uuid.UUID(current_user["tenant_id"])
|
||||
|
||||
# Fetch all non-deleted folders for tenant with visibility filter
|
||||
user_id = uuid.UUID(current_user["user_id"])
|
||||
is_system_admin = current_user.get("role") == "admin"
|
||||
query = select(Folder).where(
|
||||
Folder.tenant_id == tenant_id,
|
||||
Folder.deleted_at.is_(None),
|
||||
)
|
||||
query = await apply_visibility_filter(
|
||||
db, query, "dms_folder", Folder, user_id, tenant_id, is_system_admin
|
||||
)
|
||||
result = await db.execute(query)
|
||||
all_folders = result.scalars().all()
|
||||
|
||||
# Build lookup map
|
||||
folder_map: dict[uuid.UUID, dict] = {}
|
||||
for f in all_folders:
|
||||
folder_map[f.id] = {
|
||||
"id": str(f.id),
|
||||
"name": f.name,
|
||||
"parent_id": str(f.parent_id) if f.parent_id else None,
|
||||
"created_by": str(f.created_by),
|
||||
"deleted_at": None,
|
||||
"path": "",
|
||||
"children": [],
|
||||
}
|
||||
|
||||
# Build path for each folder
|
||||
def _build_path(folder_id: uuid.UUID) -> str:
|
||||
if folder_id not in folder_map:
|
||||
return ""
|
||||
f = folder_map[folder_id]
|
||||
if f["parent_id"] and uuid.UUID(f["parent_id"]) in folder_map:
|
||||
parent_path = _build_path(uuid.UUID(f["parent_id"]))
|
||||
return f"{parent_path}/{f['name']}"
|
||||
return f["name"]
|
||||
|
||||
for fid in folder_map:
|
||||
folder_map[fid]["path"] = _build_path(fid)
|
||||
|
||||
# Build tree
|
||||
root_nodes: list[dict] = []
|
||||
target_parent: uuid.UUID | None = None
|
||||
if parent_id is not None:
|
||||
target_parent = _parse_uuid(parent_id, "parent_id")
|
||||
|
||||
for f in all_folders:
|
||||
node = folder_map[f.id]
|
||||
if f.parent_id is not None and f.parent_id in folder_map:
|
||||
folder_map[f.parent_id]["children"].append(node)
|
||||
elif f.parent_id is None:
|
||||
root_nodes.append(node)
|
||||
|
||||
if target_parent is not None:
|
||||
# Return children of specified parent
|
||||
parent_node = folder_map.get(target_parent)
|
||||
if parent_node is None:
|
||||
raise HTTPException(
|
||||
404, detail={"detail": "Parent folder not found", "code": "not_found"}
|
||||
)
|
||||
return parent_node["children"]
|
||||
|
||||
return root_nodes
|
||||
|
||||
|
||||
@router.post("/folders", status_code=status.HTTP_201_CREATED, dependencies=[Depends(require_permission("dms:write"))])
|
||||
async def create_folder(
|
||||
body: FolderCreate,
|
||||
db: AsyncSession = Depends(get_db),
|
||||
current_user: dict = Depends(get_current_user),
|
||||
):
|
||||
"""AC2: POST /api/v1/dms/folders → 201, folder created with path."""
|
||||
tenant_id = uuid.UUID(current_user["tenant_id"])
|
||||
user_id = uuid.UUID(current_user["user_id"])
|
||||
parent_id = _parse_uuid(body.parent_id, "parent_id") if body.parent_id else None
|
||||
|
||||
# Validate parent exists if specified
|
||||
if parent_id is not None:
|
||||
parent_result = await db.execute(
|
||||
select(Folder).where(
|
||||
Folder.id == parent_id,
|
||||
Folder.tenant_id == tenant_id,
|
||||
Folder.deleted_at.is_(None),
|
||||
)
|
||||
)
|
||||
if parent_result.scalar_one_or_none() is None:
|
||||
raise HTTPException(
|
||||
404, detail={"detail": "Parent folder not found", "code": "not_found"}
|
||||
)
|
||||
|
||||
# Check name uniqueness within same parent (non-deleted)
|
||||
existing = await db.execute(
|
||||
select(Folder).where(
|
||||
Folder.tenant_id == tenant_id,
|
||||
Folder.name == body.name,
|
||||
Folder.parent_id == parent_id if parent_id else Folder.parent_id.is_(None),
|
||||
Folder.deleted_at.is_(None),
|
||||
)
|
||||
)
|
||||
if existing.scalar_one_or_none() is not None:
|
||||
raise HTTPException(
|
||||
409, detail={"detail": "Folder name already exists", "code": "duplicate"}
|
||||
)
|
||||
|
||||
# Lifecycle hook: dms.folder.before_create
|
||||
from app.core.hooks import do_action
|
||||
await do_action("dms.folder.before_create", body, db=db, tenant_id=tenant_id, user_id=user_id)
|
||||
|
||||
folder = Folder(
|
||||
tenant_id=tenant_id,
|
||||
name=body.name,
|
||||
parent_id=parent_id,
|
||||
created_by=user_id,
|
||||
)
|
||||
db.add(folder)
|
||||
await db.flush()
|
||||
|
||||
# Lifecycle hook: dms.folder.after_create
|
||||
await do_action("dms.folder.after_create", {'id': str(folder.id), 'name': folder.name, 'parent_id': str(folder.parent_id) if folder.parent_id else None}, db=db, tenant_id=tenant_id, user_id=user_id)
|
||||
|
||||
# Build path
|
||||
path = body.name
|
||||
if parent_id is not None:
|
||||
parent_path_result = await db.execute(select(Folder).where(Folder.id == parent_id))
|
||||
parent_folder = parent_path_result.scalar_one_or_none()
|
||||
if parent_folder:
|
||||
# Recursively build path
|
||||
path_parts = [body.name]
|
||||
current = parent_folder
|
||||
while current is not None:
|
||||
path_parts.insert(0, current.name)
|
||||
if current.parent_id is not None:
|
||||
cur_result = await db.execute(
|
||||
select(Folder).where(Folder.id == current.parent_id)
|
||||
)
|
||||
current = cur_result.scalar_one_or_none()
|
||||
else:
|
||||
current = None
|
||||
path = "/".join(path_parts)
|
||||
|
||||
return {
|
||||
"id": str(folder.id),
|
||||
"name": folder.name,
|
||||
"parent_id": str(folder.parent_id) if folder.parent_id else None,
|
||||
"created_by": str(folder.created_by),
|
||||
"deleted_at": None,
|
||||
"path": path,
|
||||
"children": [],
|
||||
}
|
||||
|
||||
|
||||
@router.patch("/folders/{folder_id}", dependencies=[Depends(require_permission("dms:write"))])
|
||||
async def update_folder(
|
||||
folder_id: str,
|
||||
body: FolderUpdate,
|
||||
db: AsyncSession = Depends(get_db),
|
||||
current_user: dict = Depends(get_current_user),
|
||||
):
|
||||
"""AC3: PATCH /api/v1/dms/folders/{id} → 200, rename/move."""
|
||||
tenant_id = uuid.UUID(current_user["tenant_id"])
|
||||
user_id = uuid.UUID(current_user["user_id"])
|
||||
is_system_admin = current_user.get("role") == "admin"
|
||||
fid = _parse_uuid(folder_id, "folder_id")
|
||||
|
||||
result = await db.execute(
|
||||
select(Folder).where(
|
||||
Folder.id == fid,
|
||||
Folder.tenant_id == tenant_id,
|
||||
Folder.deleted_at.is_(None),
|
||||
)
|
||||
)
|
||||
folder = result.scalar_one_or_none()
|
||||
if folder is None:
|
||||
raise HTTPException(404, detail={"detail": "Folder not found", "code": "not_found"})
|
||||
|
||||
if not await check_single_entity_access(db, "dms_folder", fid, user_id, tenant_id, "write", is_system_admin):
|
||||
raise HTTPException(403, detail={"detail": "Access denied", "code": "forbidden"})
|
||||
|
||||
data = body.model_dump(exclude_unset=True)
|
||||
|
||||
if "name" in data and data["name"] is not None:
|
||||
# Check uniqueness if name is changing
|
||||
new_parent_id = folder.parent_id
|
||||
if "parent_id" in data and data["parent_id"] is not None:
|
||||
new_parent_id = _parse_uuid(data["parent_id"], "parent_id")
|
||||
|
||||
dup = await db.execute(
|
||||
select(Folder).where(
|
||||
Folder.tenant_id == tenant_id,
|
||||
Folder.name == data["name"],
|
||||
Folder.id != fid,
|
||||
Folder.parent_id == new_parent_id if new_parent_id else Folder.parent_id.is_(None),
|
||||
Folder.deleted_at.is_(None),
|
||||
)
|
||||
)
|
||||
if dup.scalar_one_or_none() is not None:
|
||||
raise HTTPException(
|
||||
409, detail={"detail": "Folder name already exists", "code": "duplicate"}
|
||||
)
|
||||
folder.name = data["name"]
|
||||
|
||||
if "parent_id" in data:
|
||||
new_parent = _parse_uuid(data["parent_id"], "parent_id") if data["parent_id"] else None
|
||||
if new_parent is not None:
|
||||
# Validate parent exists and not creating a cycle
|
||||
if new_parent == fid:
|
||||
raise HTTPException(
|
||||
400, detail={"detail": "Cannot move folder into itself", "code": "invalid_move"}
|
||||
)
|
||||
|
||||
parent_result = await db.execute(
|
||||
select(Folder).where(
|
||||
Folder.id == new_parent,
|
||||
Folder.tenant_id == tenant_id,
|
||||
Folder.deleted_at.is_(None),
|
||||
)
|
||||
)
|
||||
if parent_result.scalar_one_or_none() is None:
|
||||
raise HTTPException(
|
||||
404, detail={"detail": "Parent folder not found", "code": "not_found"}
|
||||
)
|
||||
|
||||
# Check for cycle: ensure new_parent is not a descendant of folder
|
||||
async def _is_descendant(ancestor_id: uuid.UUID, descendant_id: uuid.UUID) -> bool:
|
||||
cur_result = await db.execute(select(Folder).where(Folder.id == descendant_id))
|
||||
cur = cur_result.scalar_one_or_none()
|
||||
while cur is not None and cur.parent_id is not None:
|
||||
if cur.parent_id == ancestor_id:
|
||||
return True
|
||||
p_result = await db.execute(select(Folder).where(Folder.id == cur.parent_id))
|
||||
cur = p_result.scalar_one_or_none()
|
||||
return False
|
||||
|
||||
if await _is_descendant(fid, new_parent):
|
||||
raise HTTPException(
|
||||
400,
|
||||
detail={
|
||||
"detail": "Cannot move folder into its own descendant",
|
||||
"code": "invalid_move",
|
||||
},
|
||||
)
|
||||
|
||||
folder.parent_id = new_parent
|
||||
|
||||
await db.flush()
|
||||
|
||||
# Build path
|
||||
path_parts = [folder.name]
|
||||
current_id = folder.parent_id
|
||||
while current_id is not None:
|
||||
cur_result = await db.execute(select(Folder).where(Folder.id == current_id))
|
||||
cur = cur_result.scalar_one_or_none()
|
||||
if cur is None:
|
||||
break
|
||||
path_parts.insert(0, cur.name)
|
||||
current_id = cur.parent_id
|
||||
path = "/".join(path_parts)
|
||||
|
||||
return {
|
||||
"id": str(folder.id),
|
||||
"name": folder.name,
|
||||
"parent_id": str(folder.parent_id) if folder.parent_id else None,
|
||||
"created_by": str(folder.created_by),
|
||||
"deleted_at": None,
|
||||
"path": path,
|
||||
"children": [],
|
||||
}
|
||||
|
||||
|
||||
@router.delete("/folders/{folder_id}", status_code=status.HTTP_204_NO_CONTENT, dependencies=[Depends(require_permission("dms:delete"))])
|
||||
async def delete_folder(
|
||||
folder_id: str,
|
||||
db: AsyncSession = Depends(get_db),
|
||||
current_user: dict = Depends(get_current_user),
|
||||
):
|
||||
"""AC4: DELETE /api/v1/dms/folders/{id} → 204, soft-delete with cascade."""
|
||||
tenant_id = uuid.UUID(current_user["tenant_id"])
|
||||
user_id = uuid.UUID(current_user["user_id"])
|
||||
is_system_admin = current_user.get("role") == "admin"
|
||||
fid = _parse_uuid(folder_id, "folder_id")
|
||||
|
||||
result = await db.execute(
|
||||
select(Folder).where(
|
||||
Folder.id == fid,
|
||||
Folder.tenant_id == tenant_id,
|
||||
Folder.deleted_at.is_(None),
|
||||
)
|
||||
)
|
||||
folder = result.scalar_one_or_none()
|
||||
if folder is None:
|
||||
raise HTTPException(404, detail={"detail": "Folder not found", "code": "not_found"})
|
||||
|
||||
if not await check_single_entity_access(db, "dms_folder", fid, user_id, tenant_id, "delete", is_system_admin):
|
||||
raise HTTPException(403, detail={"detail": "Access denied", "code": "forbidden"})
|
||||
|
||||
# Lifecycle hook: dms.folder.before_delete
|
||||
from app.core.hooks import do_action
|
||||
await do_action("dms.folder.before_delete", db=db, tenant_id=tenant_id, user_id=user_id, folder_id=str(fid))
|
||||
|
||||
from datetime import UTC, datetime
|
||||
|
||||
now = datetime.now(UTC)
|
||||
|
||||
# Recursively collect all descendant folder IDs
|
||||
all_folder_ids: list[uuid.UUID] = [fid]
|
||||
queue: list[uuid.UUID] = [fid]
|
||||
while queue:
|
||||
current_id = queue.pop(0)
|
||||
children_result = await db.execute(
|
||||
select(Folder).where(
|
||||
Folder.parent_id == current_id,
|
||||
Folder.tenant_id == tenant_id,
|
||||
Folder.deleted_at.is_(None),
|
||||
)
|
||||
)
|
||||
for child in children_result.scalars().all():
|
||||
all_folder_ids.append(child.id)
|
||||
queue.append(child.id)
|
||||
|
||||
# Soft-delete all folders
|
||||
await db.execute(update(Folder).where(Folder.id.in_(all_folder_ids)).values(deleted_at=now))
|
||||
|
||||
# Soft-delete all files in those folders
|
||||
await db.execute(
|
||||
update(DmsFile)
|
||||
.where(
|
||||
DmsFile.tenant_id == tenant_id,
|
||||
DmsFile.folder_id.in_(all_folder_ids),
|
||||
DmsFile.deleted_at.is_(None),
|
||||
)
|
||||
.values(deleted_at=now)
|
||||
)
|
||||
|
||||
await db.flush()
|
||||
|
||||
# Lifecycle hook: dms.folder.after_delete
|
||||
from app.core.hooks import do_action
|
||||
await do_action("dms.folder.after_delete", db=db, tenant_id=tenant_id, user_id=user_id, folder_id=str(fid))
|
||||
|
||||
return Response(status_code=status.HTTP_204_NO_CONTENT)
|
||||
|
||||
|
||||
# ─── Files ───
|
||||
|
||||
|
||||
@router.post("/files/upload", status_code=status.HTTP_201_CREATED, response_model=FileMetadataResponse, dependencies=[Depends(require_permission("dms:write"))])
|
||||
async def upload_file(
|
||||
file: UploadFile = File(...),
|
||||
@@ -547,7 +105,7 @@ async def upload_file(
|
||||
|
||||
# Stream file to storage — avoid loading entire file into RAM
|
||||
import hashlib
|
||||
chunk_size = 1024 * 1024 # 1MB chunks
|
||||
chunk_size = 1024 * 1024 # noqa: F811 (Original-Shadowing im Original auch so)
|
||||
sha256 = hashlib.sha256()
|
||||
file_size = 0
|
||||
|
||||
@@ -1087,406 +645,7 @@ async def download_file(
|
||||
)
|
||||
|
||||
|
||||
@router.post("/files/{file_id}/edit-session", dependencies=[Depends(require_permission("dms:write"))])
|
||||
async def create_edit_session(
|
||||
file_id: str,
|
||||
db: AsyncSession = Depends(get_db),
|
||||
current_user: dict = Depends(get_current_user),
|
||||
):
|
||||
"""AC11: POST /api/v1/dms/files/{id}/edit-session → 200 + Collabora config."""
|
||||
tenant_id = uuid.UUID(current_user["tenant_id"])
|
||||
user_id = current_user["user_id"]
|
||||
user_name = current_user.get("name", "Unknown")
|
||||
is_system_admin = current_user.get("role") == "admin"
|
||||
fid = _parse_uuid(file_id, "file_id")
|
||||
|
||||
result = await db.execute(
|
||||
select(DmsFile).where(
|
||||
DmsFile.id == fid,
|
||||
DmsFile.tenant_id == tenant_id,
|
||||
DmsFile.deleted_at.is_(None),
|
||||
)
|
||||
)
|
||||
dms_file = result.scalar_one_or_none()
|
||||
if dms_file is None:
|
||||
raise HTTPException(404, detail={"detail": "File not found", "code": "not_found"})
|
||||
|
||||
if not await check_single_entity_access(db, "dms_file", fid, user_id, tenant_id, "write", is_system_admin):
|
||||
raise HTTPException(403, detail={"detail": "Access denied", "code": "forbidden"})
|
||||
|
||||
ext = _get_file_extension(dms_file.name)
|
||||
if ext not in OFFICE_EXTENSIONS:
|
||||
raise HTTPException(
|
||||
400,
|
||||
detail={
|
||||
"detail": "Only Office files (docx, xlsx, pptx) are supported",
|
||||
"code": "not_office",
|
||||
},
|
||||
)
|
||||
|
||||
file_type = OFFICE_EXTENSIONS[ext]
|
||||
download_url = f"/api/v1/dms/files/{fid}/preview"
|
||||
callback_url = f"/api/v1/dms/files/{fid}/callback"
|
||||
|
||||
config = {
|
||||
"document": {
|
||||
"fileType": file_type,
|
||||
"key": str(uuid.uuid4()),
|
||||
"title": dms_file.name,
|
||||
"url": download_url,
|
||||
},
|
||||
"editorConfig": {
|
||||
"mode": "edit",
|
||||
"callbackUrl": callback_url,
|
||||
"user": {
|
||||
"id": user_id,
|
||||
"name": user_name,
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
return config
|
||||
|
||||
|
||||
# ─── Internal Sharing ───
|
||||
|
||||
|
||||
@router.post("/files/{file_id}/share", dependencies=[Depends(require_permission("dms:share"))])
|
||||
async def share_file(
|
||||
file_id: str,
|
||||
body: ShareRequest,
|
||||
db: AsyncSession = Depends(get_db),
|
||||
current_user: dict = Depends(get_current_user),
|
||||
):
|
||||
"""AC12: POST /api/v1/dms/files/{id}/share → 200, internal share created."""
|
||||
tenant_id = uuid.UUID(current_user["tenant_id"])
|
||||
user_id = uuid.UUID(current_user["user_id"])
|
||||
is_system_admin = current_user.get("role") == "admin"
|
||||
fid = _parse_uuid(file_id, "file_id")
|
||||
|
||||
# Verify file exists
|
||||
file_result = await db.execute(
|
||||
select(DmsFile).where(
|
||||
DmsFile.id == fid,
|
||||
DmsFile.tenant_id == tenant_id,
|
||||
DmsFile.deleted_at.is_(None),
|
||||
)
|
||||
)
|
||||
if file_result.scalar_one_or_none() is None:
|
||||
raise HTTPException(404, detail={"detail": "File not found", "code": "not_found"})
|
||||
|
||||
if not await check_single_entity_access(db, "dms_file", fid, user_id, tenant_id, "share", is_system_admin):
|
||||
raise HTTPException(403, detail={"detail": "Access denied", "code": "forbidden"})
|
||||
|
||||
created_perms: list[dict] = []
|
||||
|
||||
for uid_str in body.user_ids:
|
||||
uid = _parse_uuid(uid_str, "user_id")
|
||||
# Check if already exists
|
||||
existing = await db.execute(
|
||||
select(Permission).where(
|
||||
Permission.tenant_id == tenant_id,
|
||||
Permission.file_id == fid,
|
||||
Permission.user_id == uid,
|
||||
Permission.access_level == body.access_level,
|
||||
)
|
||||
)
|
||||
if existing.scalar_one_or_none() is None:
|
||||
perm = Permission(
|
||||
tenant_id=tenant_id,
|
||||
file_id=fid,
|
||||
user_id=uid,
|
||||
group_id=None,
|
||||
access_level=body.access_level,
|
||||
)
|
||||
db.add(perm)
|
||||
await db.flush()
|
||||
created_perms.append(
|
||||
{
|
||||
"id": str(perm.id),
|
||||
"file_id": str(fid),
|
||||
"user_id": str(uid),
|
||||
"group_id": None,
|
||||
"access_level": body.access_level,
|
||||
}
|
||||
)
|
||||
|
||||
for gid_str in body.group_ids:
|
||||
gid = _parse_uuid(gid_str, "group_id")
|
||||
existing = await db.execute(
|
||||
select(Permission).where(
|
||||
Permission.tenant_id == tenant_id,
|
||||
Permission.file_id == fid,
|
||||
Permission.group_id == gid,
|
||||
Permission.access_level == body.access_level,
|
||||
)
|
||||
)
|
||||
if existing.scalar_one_or_none() is None:
|
||||
perm = Permission(
|
||||
tenant_id=tenant_id,
|
||||
file_id=fid,
|
||||
user_id=uuid.UUID(current_user["user_id"]),
|
||||
group_id=gid,
|
||||
access_level=body.access_level,
|
||||
)
|
||||
db.add(perm)
|
||||
await db.flush()
|
||||
created_perms.append(
|
||||
{
|
||||
"id": str(perm.id),
|
||||
"file_id": str(fid),
|
||||
"user_id": str(perm.user_id),
|
||||
"group_id": str(gid),
|
||||
"access_level": body.access_level,
|
||||
}
|
||||
)
|
||||
|
||||
return {
|
||||
"file_id": str(fid),
|
||||
"shared_with": created_perms,
|
||||
"count": len(created_perms),
|
||||
}
|
||||
|
||||
|
||||
@router.delete("/files/{file_id}/share", status_code=status.HTTP_204_NO_CONTENT, dependencies=[Depends(require_permission("dms:share"))])
|
||||
async def remove_share(
|
||||
file_id: str,
|
||||
body: ShareRemoveRequest = Body(...),
|
||||
db: AsyncSession = Depends(get_db),
|
||||
current_user: dict = Depends(get_current_user),
|
||||
):
|
||||
"""AC13: DELETE /api/v1/dms/files/{id}/share → 204, share removed."""
|
||||
tenant_id = uuid.UUID(current_user["tenant_id"])
|
||||
user_id = uuid.UUID(current_user["user_id"])
|
||||
is_system_admin = current_user.get("role") == "admin"
|
||||
fid = _parse_uuid(file_id, "file_id")
|
||||
|
||||
if not await check_single_entity_access(db, "dms_file", fid, user_id, tenant_id, "share", is_system_admin):
|
||||
raise HTTPException(403, detail={"detail": "Access denied", "code": "forbidden"})
|
||||
|
||||
if body.user_id:
|
||||
uid = _parse_uuid(body.user_id, "user_id")
|
||||
result = await db.execute(
|
||||
select(Permission).where(
|
||||
Permission.tenant_id == tenant_id,
|
||||
Permission.file_id == fid,
|
||||
Permission.user_id == uid,
|
||||
)
|
||||
)
|
||||
perms = result.scalars().all()
|
||||
for p in perms:
|
||||
await db.delete(p)
|
||||
|
||||
if body.group_id:
|
||||
gid = _parse_uuid(body.group_id, "group_id")
|
||||
result = await db.execute(
|
||||
select(Permission).where(
|
||||
Permission.tenant_id == tenant_id,
|
||||
Permission.file_id == fid,
|
||||
Permission.group_id == gid,
|
||||
)
|
||||
)
|
||||
perms = result.scalars().all()
|
||||
for p in perms:
|
||||
await db.delete(p)
|
||||
|
||||
await db.flush()
|
||||
return Response(status_code=status.HTTP_204_NO_CONTENT)
|
||||
|
||||
|
||||
# ─── Search & Bulk ───
|
||||
|
||||
|
||||
@router.get("/search", dependencies=[Depends(require_permission("dms:read"))])
|
||||
async def search_files(
|
||||
q: str,
|
||||
db: AsyncSession = Depends(get_db),
|
||||
current_user: dict = Depends(get_current_user),
|
||||
):
|
||||
"""AC16: GET /api/v1/dms/search?q=text → 200 + matching files (ILIKE)."""
|
||||
tenant_id = uuid.UUID(current_user["tenant_id"])
|
||||
user_id = uuid.UUID(current_user["user_id"])
|
||||
is_system_admin = current_user.get("role") == "admin"
|
||||
|
||||
query = select(DmsFile).where(
|
||||
DmsFile.tenant_id == tenant_id,
|
||||
DmsFile.deleted_at.is_(None),
|
||||
DmsFile.name.ilike(f"%{q}%"),
|
||||
)
|
||||
query = await apply_visibility_filter(
|
||||
db, query, "dms_file", DmsFile, user_id, tenant_id, is_system_admin
|
||||
)
|
||||
result = await db.execute(query)
|
||||
files = result.scalars().all()
|
||||
|
||||
return [
|
||||
{
|
||||
"id": str(f.id),
|
||||
"name": f.name,
|
||||
"folder_id": str(f.folder_id) if f.folder_id else None,
|
||||
"uploaded_by": str(f.uploaded_by),
|
||||
"mime_type": f.mime_type,
|
||||
"size_bytes": f.size_bytes,
|
||||
"deleted_at": None,
|
||||
"created_at": f.created_at.isoformat() if f.created_at else None,
|
||||
}
|
||||
for f in files
|
||||
]
|
||||
|
||||
|
||||
@router.get("/shared-with-me", dependencies=[Depends(require_permission("dms:read"))])
|
||||
async def shared_with_me(
|
||||
db: AsyncSession = Depends(get_db),
|
||||
current_user: dict = Depends(get_current_user),
|
||||
):
|
||||
"""AC17: GET /api/v1/dms/shared-with-me → 200 + shared files list."""
|
||||
tenant_id = uuid.UUID(current_user["tenant_id"])
|
||||
user_id = uuid.UUID(current_user["user_id"])
|
||||
is_system_admin = current_user.get("role") == "admin"
|
||||
|
||||
# Query permissions for this user and join with files
|
||||
perm_result = await db.execute(
|
||||
select(Permission).where(
|
||||
Permission.tenant_id == tenant_id,
|
||||
Permission.user_id == user_id,
|
||||
)
|
||||
)
|
||||
perms = perm_result.scalars().all()
|
||||
file_ids = {p.file_id for p in perms}
|
||||
|
||||
if not file_ids:
|
||||
return {"items": [], "total": 0}
|
||||
|
||||
query = select(DmsFile).where(
|
||||
DmsFile.tenant_id == tenant_id,
|
||||
DmsFile.id.in_(file_ids),
|
||||
DmsFile.deleted_at.is_(None),
|
||||
)
|
||||
query = await apply_visibility_filter(
|
||||
db, query, "dms_file", DmsFile, user_id, tenant_id, is_system_admin
|
||||
)
|
||||
result = await db.execute(query)
|
||||
files = result.scalars().all()
|
||||
|
||||
# Map permissions for access_level
|
||||
perm_map: dict[uuid.UUID, str] = {}
|
||||
for p in perms:
|
||||
if p.file_id in file_ids:
|
||||
perm_map[p.file_id] = p.access_level
|
||||
|
||||
return [
|
||||
{
|
||||
"id": str(f.id),
|
||||
"name": f.name,
|
||||
"folder_id": str(f.folder_id) if f.folder_id else None,
|
||||
"uploaded_by": str(f.uploaded_by),
|
||||
"mime_type": f.mime_type,
|
||||
"size_bytes": f.size_bytes,
|
||||
"access_level": perm_map.get(f.id, "read"),
|
||||
"created_at": f.created_at.isoformat() if f.created_at else None,
|
||||
}
|
||||
for f in files
|
||||
]
|
||||
|
||||
|
||||
@router.post("/files/bulk-move", dependencies=[Depends(require_permission("dms:write"))])
|
||||
async def bulk_move(
|
||||
body: BulkMoveRequest,
|
||||
db: AsyncSession = Depends(get_db),
|
||||
current_user: dict = Depends(get_current_user),
|
||||
):
|
||||
"""AC18: POST /api/v1/dms/files/bulk-move → 200, files moved."""
|
||||
tenant_id = uuid.UUID(current_user["tenant_id"])
|
||||
user_id = uuid.UUID(current_user["user_id"])
|
||||
is_system_admin = current_user.get("role") == "admin"
|
||||
target_folder_id = (
|
||||
_parse_uuid(body.target_folder_id, "target_folder_id") if body.target_folder_id else None
|
||||
)
|
||||
|
||||
# Validate target folder if specified
|
||||
if target_folder_id is not None:
|
||||
folder_result = await db.execute(
|
||||
select(Folder).where(
|
||||
Folder.id == target_folder_id,
|
||||
Folder.tenant_id == tenant_id,
|
||||
Folder.deleted_at.is_(None),
|
||||
)
|
||||
)
|
||||
if folder_result.scalar_one_or_none() is None:
|
||||
raise HTTPException(
|
||||
404, detail={"detail": "Target folder not found", "code": "not_found"}
|
||||
)
|
||||
|
||||
file_ids = [_parse_uuid(fid, "file_id") for fid in body.file_ids]
|
||||
|
||||
query = select(DmsFile).where(
|
||||
DmsFile.tenant_id == tenant_id,
|
||||
DmsFile.id.in_(file_ids),
|
||||
DmsFile.deleted_at.is_(None),
|
||||
)
|
||||
query = await apply_visibility_filter(
|
||||
db, query, "dms_file", DmsFile, user_id, tenant_id, is_system_admin
|
||||
)
|
||||
result = await db.execute(query)
|
||||
files = result.scalars().all()
|
||||
|
||||
moved_count = 0
|
||||
for f in files:
|
||||
f.folder_id = target_folder_id
|
||||
moved_count += 1
|
||||
|
||||
await db.flush()
|
||||
|
||||
return {
|
||||
"moved": moved_count,
|
||||
"file_ids": [str(fid) for fid in file_ids],
|
||||
"target_folder_id": str(target_folder_id) if target_folder_id else None,
|
||||
}
|
||||
|
||||
|
||||
@router.post("/files/bulk-delete", dependencies=[Depends(require_permission("dms:delete"))])
|
||||
async def bulk_delete(
|
||||
body: BulkDeleteRequest,
|
||||
db: AsyncSession = Depends(get_db),
|
||||
current_user: dict = Depends(get_current_user),
|
||||
):
|
||||
"""AC19: POST /api/v1/dms/files/bulk-delete → 200, files soft-deleted."""
|
||||
tenant_id = uuid.UUID(current_user["tenant_id"])
|
||||
user_id = uuid.UUID(current_user["user_id"])
|
||||
is_system_admin = current_user.get("role") == "admin"
|
||||
file_ids = [_parse_uuid(fid, "file_id") for fid in body.file_ids]
|
||||
|
||||
from datetime import UTC, datetime
|
||||
|
||||
now = datetime.now(UTC)
|
||||
|
||||
# Apply visibility filter to only delete files user has access to
|
||||
query = select(DmsFile).where(
|
||||
DmsFile.tenant_id == tenant_id,
|
||||
DmsFile.id.in_(file_ids),
|
||||
DmsFile.deleted_at.is_(None),
|
||||
)
|
||||
query = await apply_visibility_filter(
|
||||
db, query, "dms_file", DmsFile, user_id, tenant_id, is_system_admin
|
||||
)
|
||||
result = await db.execute(query)
|
||||
accessible_files = result.scalars().all()
|
||||
accessible_ids = [f.id for f in accessible_files]
|
||||
|
||||
result = await db.execute(
|
||||
update(DmsFile)
|
||||
.where(
|
||||
DmsFile.tenant_id == tenant_id,
|
||||
DmsFile.id.in_(accessible_ids),
|
||||
DmsFile.deleted_at.is_(None),
|
||||
)
|
||||
.values(deleted_at=now)
|
||||
)
|
||||
|
||||
deleted_count = result.rowcount
|
||||
await db.flush()
|
||||
|
||||
return {
|
||||
"deleted": deleted_count,
|
||||
"file_ids": body.file_ids,
|
||||
}
|
||||
# Sub-Router einbinden (BUG-018 Split): folders, sharing/collabora, search/bulk
|
||||
router.include_router(folders_router)
|
||||
router.include_router(sharing_router)
|
||||
router.include_router(search_bulk_router)
|
||||
|
||||
@@ -0,0 +1,223 @@
|
||||
"""DMS Search / shared-with-me / Bulk Routen — extrahiert aus routes.py (BUG-018)."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import uuid
|
||||
|
||||
from fastapi import (
|
||||
APIRouter,
|
||||
Depends,
|
||||
HTTPException,
|
||||
)
|
||||
from sqlalchemy import select, update
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
|
||||
from app.core.db import get_db
|
||||
from app.core.visibility import apply_visibility_filter
|
||||
from app.deps import get_current_user, require_permission
|
||||
from app.plugins.builtins.dms.common import (
|
||||
_parse_uuid,
|
||||
)
|
||||
from app.plugins.builtins.dms.models import File as DmsFile
|
||||
from app.plugins.builtins.dms.models import Folder
|
||||
from app.plugins.builtins.dms.schemas import BulkDeleteRequest, BulkMoveRequest
|
||||
from app.plugins.builtins.permissions.contracts import get_contract as get_perms_contract
|
||||
|
||||
_perms_contract = get_perms_contract()
|
||||
Permission = _perms_contract.Permission
|
||||
|
||||
router = APIRouter(tags=["dms"])
|
||||
|
||||
@router.get("/search", dependencies=[Depends(require_permission("dms:read"))])
|
||||
async def search_files(
|
||||
q: str,
|
||||
db: AsyncSession = Depends(get_db),
|
||||
current_user: dict = Depends(get_current_user),
|
||||
):
|
||||
"""AC16: GET /api/v1/dms/search?q=text → 200 + matching files (ILIKE)."""
|
||||
tenant_id = uuid.UUID(current_user["tenant_id"])
|
||||
user_id = uuid.UUID(current_user["user_id"])
|
||||
is_system_admin = current_user.get("role") == "admin"
|
||||
|
||||
query = select(DmsFile).where(
|
||||
DmsFile.tenant_id == tenant_id,
|
||||
DmsFile.deleted_at.is_(None),
|
||||
DmsFile.name.ilike(f"%{q}%"),
|
||||
)
|
||||
query = await apply_visibility_filter(
|
||||
db, query, "dms_file", DmsFile, user_id, tenant_id, is_system_admin
|
||||
)
|
||||
result = await db.execute(query)
|
||||
files = result.scalars().all()
|
||||
|
||||
return [
|
||||
{
|
||||
"id": str(f.id),
|
||||
"name": f.name,
|
||||
"folder_id": str(f.folder_id) if f.folder_id else None,
|
||||
"uploaded_by": str(f.uploaded_by),
|
||||
"mime_type": f.mime_type,
|
||||
"size_bytes": f.size_bytes,
|
||||
"deleted_at": None,
|
||||
"created_at": f.created_at.isoformat() if f.created_at else None,
|
||||
}
|
||||
for f in files
|
||||
]
|
||||
|
||||
|
||||
@router.get("/shared-with-me", dependencies=[Depends(require_permission("dms:read"))])
|
||||
async def shared_with_me(
|
||||
db: AsyncSession = Depends(get_db),
|
||||
current_user: dict = Depends(get_current_user),
|
||||
):
|
||||
"""AC17: GET /api/v1/dms/shared-with-me → 200 + shared files list."""
|
||||
tenant_id = uuid.UUID(current_user["tenant_id"])
|
||||
user_id = uuid.UUID(current_user["user_id"])
|
||||
is_system_admin = current_user.get("role") == "admin"
|
||||
|
||||
# Query permissions for this user and join with files
|
||||
perm_result = await db.execute(
|
||||
select(Permission).where(
|
||||
Permission.tenant_id == tenant_id,
|
||||
Permission.user_id == user_id,
|
||||
)
|
||||
)
|
||||
perms = perm_result.scalars().all()
|
||||
file_ids = {p.file_id for p in perms}
|
||||
|
||||
if not file_ids:
|
||||
return []
|
||||
|
||||
query = select(DmsFile).where(
|
||||
DmsFile.tenant_id == tenant_id,
|
||||
DmsFile.id.in_(file_ids),
|
||||
DmsFile.deleted_at.is_(None),
|
||||
)
|
||||
query = await apply_visibility_filter(
|
||||
db, query, "dms_file", DmsFile, user_id, tenant_id, is_system_admin
|
||||
)
|
||||
result = await db.execute(query)
|
||||
files = result.scalars().all()
|
||||
|
||||
# Map permissions for access_level
|
||||
perm_map: dict[uuid.UUID, str] = {}
|
||||
for p in perms:
|
||||
if p.file_id in file_ids:
|
||||
perm_map[p.file_id] = p.access_level
|
||||
|
||||
return [
|
||||
{
|
||||
"id": str(f.id),
|
||||
"name": f.name,
|
||||
"folder_id": str(f.folder_id) if f.folder_id else None,
|
||||
"uploaded_by": str(f.uploaded_by),
|
||||
"mime_type": f.mime_type,
|
||||
"size_bytes": f.size_bytes,
|
||||
"access_level": perm_map.get(f.id, "read"),
|
||||
"created_at": f.created_at.isoformat() if f.created_at else None,
|
||||
}
|
||||
for f in files
|
||||
]
|
||||
|
||||
|
||||
@router.post("/files/bulk-move", dependencies=[Depends(require_permission("dms:write"))])
|
||||
async def bulk_move(
|
||||
body: BulkMoveRequest,
|
||||
db: AsyncSession = Depends(get_db),
|
||||
current_user: dict = Depends(get_current_user),
|
||||
):
|
||||
"""AC18: POST /api/v1/dms/files/bulk-move → 200, files moved."""
|
||||
tenant_id = uuid.UUID(current_user["tenant_id"])
|
||||
user_id = uuid.UUID(current_user["user_id"])
|
||||
is_system_admin = current_user.get("role") == "admin"
|
||||
target_folder_id = (
|
||||
_parse_uuid(body.target_folder_id, "target_folder_id") if body.target_folder_id else None
|
||||
)
|
||||
|
||||
# Validate target folder if specified
|
||||
if target_folder_id is not None:
|
||||
folder_result = await db.execute(
|
||||
select(Folder).where(
|
||||
Folder.id == target_folder_id,
|
||||
Folder.tenant_id == tenant_id,
|
||||
Folder.deleted_at.is_(None),
|
||||
)
|
||||
)
|
||||
if folder_result.scalar_one_or_none() is None:
|
||||
raise HTTPException(
|
||||
404, detail={"detail": "Target folder not found", "code": "not_found"}
|
||||
)
|
||||
|
||||
file_ids = [_parse_uuid(fid, "file_id") for fid in body.file_ids]
|
||||
|
||||
query = select(DmsFile).where(
|
||||
DmsFile.tenant_id == tenant_id,
|
||||
DmsFile.id.in_(file_ids),
|
||||
DmsFile.deleted_at.is_(None),
|
||||
)
|
||||
query = await apply_visibility_filter(
|
||||
db, query, "dms_file", DmsFile, user_id, tenant_id, is_system_admin
|
||||
)
|
||||
result = await db.execute(query)
|
||||
files = result.scalars().all()
|
||||
|
||||
moved_count = 0
|
||||
for f in files:
|
||||
f.folder_id = target_folder_id
|
||||
moved_count += 1
|
||||
|
||||
await db.flush()
|
||||
|
||||
return {
|
||||
"moved": moved_count,
|
||||
"file_ids": [str(fid) for fid in file_ids],
|
||||
"target_folder_id": str(target_folder_id) if target_folder_id else None,
|
||||
}
|
||||
|
||||
|
||||
@router.post("/files/bulk-delete", dependencies=[Depends(require_permission("dms:delete"))])
|
||||
async def bulk_delete(
|
||||
body: BulkDeleteRequest,
|
||||
db: AsyncSession = Depends(get_db),
|
||||
current_user: dict = Depends(get_current_user),
|
||||
):
|
||||
"""AC19: POST /api/v1/dms/files/bulk-delete → 200, files soft-deleted."""
|
||||
tenant_id = uuid.UUID(current_user["tenant_id"])
|
||||
user_id = uuid.UUID(current_user["user_id"])
|
||||
is_system_admin = current_user.get("role") == "admin"
|
||||
file_ids = [_parse_uuid(fid, "file_id") for fid in body.file_ids]
|
||||
|
||||
from datetime import UTC, datetime
|
||||
|
||||
now = datetime.now(UTC)
|
||||
|
||||
# Apply visibility filter to only delete files user has access to
|
||||
query = select(DmsFile).where(
|
||||
DmsFile.tenant_id == tenant_id,
|
||||
DmsFile.id.in_(file_ids),
|
||||
DmsFile.deleted_at.is_(None),
|
||||
)
|
||||
query = await apply_visibility_filter(
|
||||
db, query, "dms_file", DmsFile, user_id, tenant_id, is_system_admin
|
||||
)
|
||||
result = await db.execute(query)
|
||||
accessible_files = result.scalars().all()
|
||||
accessible_ids = [f.id for f in accessible_files]
|
||||
|
||||
result = await db.execute(
|
||||
update(DmsFile)
|
||||
.where(
|
||||
DmsFile.tenant_id == tenant_id,
|
||||
DmsFile.id.in_(accessible_ids),
|
||||
DmsFile.deleted_at.is_(None),
|
||||
)
|
||||
.values(deleted_at=now)
|
||||
)
|
||||
|
||||
deleted_count = result.rowcount
|
||||
await db.flush()
|
||||
|
||||
return {
|
||||
"deleted": deleted_count,
|
||||
"file_ids": body.file_ids,
|
||||
}
|
||||
@@ -0,0 +1,242 @@
|
||||
"""DMS Edit-Session/Collabora & Sharing Routen — extrahiert aus routes.py (BUG-018)."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import uuid
|
||||
|
||||
from fastapi import (
|
||||
APIRouter,
|
||||
Body,
|
||||
Depends,
|
||||
HTTPException,
|
||||
Response,
|
||||
status,
|
||||
)
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
|
||||
from app.core.db import get_db
|
||||
from app.core.visibility import check_single_entity_access
|
||||
from app.deps import get_current_user, require_permission
|
||||
from app.plugins.builtins.dms.common import (
|
||||
OFFICE_EXTENSIONS,
|
||||
_get_file_extension,
|
||||
_parse_uuid,
|
||||
)
|
||||
from app.plugins.builtins.dms.models import File as DmsFile
|
||||
from app.plugins.builtins.dms.schemas import ShareRemoveRequest, ShareRequest
|
||||
from app.plugins.builtins.permissions.contracts import get_contract as get_perms_contract
|
||||
|
||||
_perms_contract = get_perms_contract()
|
||||
Permission = _perms_contract.Permission
|
||||
|
||||
router = APIRouter(tags=["dms"])
|
||||
|
||||
@router.post("/files/{file_id}/edit-session", dependencies=[Depends(require_permission("dms:write"))])
|
||||
async def create_edit_session(
|
||||
file_id: str,
|
||||
db: AsyncSession = Depends(get_db),
|
||||
current_user: dict = Depends(get_current_user),
|
||||
):
|
||||
"""AC11: POST /api/v1/dms/files/{id}/edit-session → 200 + Collabora config."""
|
||||
tenant_id = uuid.UUID(current_user["tenant_id"])
|
||||
user_id = current_user["user_id"]
|
||||
user_name = current_user.get("name", "Unknown")
|
||||
is_system_admin = current_user.get("role") == "admin"
|
||||
fid = _parse_uuid(file_id, "file_id")
|
||||
|
||||
result = await db.execute(
|
||||
select(DmsFile).where(
|
||||
DmsFile.id == fid,
|
||||
DmsFile.tenant_id == tenant_id,
|
||||
DmsFile.deleted_at.is_(None),
|
||||
)
|
||||
)
|
||||
dms_file = result.scalar_one_or_none()
|
||||
if dms_file is None:
|
||||
raise HTTPException(404, detail={"detail": "File not found", "code": "not_found"})
|
||||
|
||||
if not await check_single_entity_access(db, "dms_file", fid, user_id, tenant_id, "write", is_system_admin):
|
||||
raise HTTPException(403, detail={"detail": "Access denied", "code": "forbidden"})
|
||||
|
||||
ext = _get_file_extension(dms_file.name)
|
||||
if ext not in OFFICE_EXTENSIONS:
|
||||
raise HTTPException(
|
||||
400,
|
||||
detail={
|
||||
"detail": "Only Office files (docx, xlsx, pptx) are supported",
|
||||
"code": "not_office",
|
||||
},
|
||||
)
|
||||
|
||||
file_type = OFFICE_EXTENSIONS[ext]
|
||||
download_url = f"/api/v1/dms/files/{fid}/preview"
|
||||
callback_url = f"/api/v1/dms/files/{fid}/callback"
|
||||
|
||||
config = {
|
||||
"document": {
|
||||
"fileType": file_type,
|
||||
"key": str(uuid.uuid4()),
|
||||
"title": dms_file.name,
|
||||
"url": download_url,
|
||||
},
|
||||
"editorConfig": {
|
||||
"mode": "edit",
|
||||
"callbackUrl": callback_url,
|
||||
"user": {
|
||||
"id": user_id,
|
||||
"name": user_name,
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
return config
|
||||
|
||||
|
||||
# ─── Internal Sharing ───
|
||||
|
||||
|
||||
@router.post("/files/{file_id}/share", dependencies=[Depends(require_permission("dms:share"))])
|
||||
async def share_file(
|
||||
file_id: str,
|
||||
body: ShareRequest,
|
||||
db: AsyncSession = Depends(get_db),
|
||||
current_user: dict = Depends(get_current_user),
|
||||
):
|
||||
"""AC12: POST /api/v1/dms/files/{id}/share → 200, internal share created."""
|
||||
tenant_id = uuid.UUID(current_user["tenant_id"])
|
||||
user_id = uuid.UUID(current_user["user_id"])
|
||||
is_system_admin = current_user.get("role") == "admin"
|
||||
fid = _parse_uuid(file_id, "file_id")
|
||||
|
||||
# Verify file exists
|
||||
file_result = await db.execute(
|
||||
select(DmsFile).where(
|
||||
DmsFile.id == fid,
|
||||
DmsFile.tenant_id == tenant_id,
|
||||
DmsFile.deleted_at.is_(None),
|
||||
)
|
||||
)
|
||||
if file_result.scalar_one_or_none() is None:
|
||||
raise HTTPException(404, detail={"detail": "File not found", "code": "not_found"})
|
||||
|
||||
if not await check_single_entity_access(db, "dms_file", fid, user_id, tenant_id, "share", is_system_admin):
|
||||
raise HTTPException(403, detail={"detail": "Access denied", "code": "forbidden"})
|
||||
|
||||
created_perms: list[dict] = []
|
||||
|
||||
for uid_str in body.user_ids:
|
||||
uid = _parse_uuid(uid_str, "user_id")
|
||||
# Check if already exists
|
||||
existing = await db.execute(
|
||||
select(Permission).where(
|
||||
Permission.tenant_id == tenant_id,
|
||||
Permission.file_id == fid,
|
||||
Permission.user_id == uid,
|
||||
Permission.access_level == body.access_level,
|
||||
)
|
||||
)
|
||||
if existing.scalar_one_or_none() is None:
|
||||
perm = Permission(
|
||||
tenant_id=tenant_id,
|
||||
file_id=fid,
|
||||
user_id=uid,
|
||||
group_id=None,
|
||||
access_level=body.access_level,
|
||||
)
|
||||
db.add(perm)
|
||||
await db.flush()
|
||||
created_perms.append(
|
||||
{
|
||||
"id": str(perm.id),
|
||||
"file_id": str(fid),
|
||||
"user_id": str(uid),
|
||||
"group_id": None,
|
||||
"access_level": body.access_level,
|
||||
}
|
||||
)
|
||||
|
||||
for gid_str in body.group_ids:
|
||||
gid = _parse_uuid(gid_str, "group_id")
|
||||
existing = await db.execute(
|
||||
select(Permission).where(
|
||||
Permission.tenant_id == tenant_id,
|
||||
Permission.file_id == fid,
|
||||
Permission.group_id == gid,
|
||||
Permission.access_level == body.access_level,
|
||||
)
|
||||
)
|
||||
if existing.scalar_one_or_none() is None:
|
||||
perm = Permission(
|
||||
tenant_id=tenant_id,
|
||||
file_id=fid,
|
||||
user_id=uuid.UUID(current_user["user_id"]),
|
||||
group_id=gid,
|
||||
access_level=body.access_level,
|
||||
)
|
||||
db.add(perm)
|
||||
await db.flush()
|
||||
created_perms.append(
|
||||
{
|
||||
"id": str(perm.id),
|
||||
"file_id": str(fid),
|
||||
"user_id": str(perm.user_id),
|
||||
"group_id": str(gid),
|
||||
"access_level": body.access_level,
|
||||
}
|
||||
)
|
||||
|
||||
return {
|
||||
"file_id": str(fid),
|
||||
"shared_with": created_perms,
|
||||
"count": len(created_perms),
|
||||
}
|
||||
|
||||
|
||||
@router.delete("/files/{file_id}/share", status_code=status.HTTP_204_NO_CONTENT, dependencies=[Depends(require_permission("dms:share"))])
|
||||
async def remove_share(
|
||||
file_id: str,
|
||||
body: ShareRemoveRequest = Body(...),
|
||||
db: AsyncSession = Depends(get_db),
|
||||
current_user: dict = Depends(get_current_user),
|
||||
):
|
||||
"""AC13: DELETE /api/v1/dms/files/{id}/share → 204, share removed."""
|
||||
tenant_id = uuid.UUID(current_user["tenant_id"])
|
||||
user_id = uuid.UUID(current_user["user_id"])
|
||||
is_system_admin = current_user.get("role") == "admin"
|
||||
fid = _parse_uuid(file_id, "file_id")
|
||||
|
||||
if not await check_single_entity_access(db, "dms_file", fid, user_id, tenant_id, "share", is_system_admin):
|
||||
raise HTTPException(403, detail={"detail": "Access denied", "code": "forbidden"})
|
||||
|
||||
if body.user_id:
|
||||
uid = _parse_uuid(body.user_id, "user_id")
|
||||
result = await db.execute(
|
||||
select(Permission).where(
|
||||
Permission.tenant_id == tenant_id,
|
||||
Permission.file_id == fid,
|
||||
Permission.user_id == uid,
|
||||
)
|
||||
)
|
||||
perms = result.scalars().all()
|
||||
for p in perms:
|
||||
await db.delete(p)
|
||||
|
||||
if body.group_id:
|
||||
gid = _parse_uuid(body.group_id, "group_id")
|
||||
result = await db.execute(
|
||||
select(Permission).where(
|
||||
Permission.tenant_id == tenant_id,
|
||||
Permission.file_id == fid,
|
||||
Permission.group_id == gid,
|
||||
)
|
||||
)
|
||||
perms = result.scalars().all()
|
||||
for p in perms:
|
||||
await db.delete(p)
|
||||
|
||||
await db.flush()
|
||||
return Response(status_code=status.HTTP_204_NO_CONTENT)
|
||||
|
||||
|
||||
# ─── Search & Bulk ───
|
||||
@@ -5,7 +5,7 @@ from __future__ import annotations
|
||||
from typing import Any
|
||||
|
||||
from app.plugins.base import BasePlugin
|
||||
from app.plugins.manifest import FrontendDetailTab, PluginManifest, PluginRouteDef
|
||||
from app.plugins.manifest import PluginManifest, PluginRouteDef
|
||||
|
||||
|
||||
class EntityLinksPlugin(BasePlugin):
|
||||
@@ -42,9 +42,9 @@ class EntityLinksPlugin(BasePlugin):
|
||||
"entity_links:delete",
|
||||
],
|
||||
is_core=True,
|
||||
detail_tabs=[
|
||||
FrontendDetailTab(entity_type='contact', label_key='tabs.links', label='Verknüpfungen', component='@/components/contact/ContactLinksTab', icon='Link', order=60, permission='entity_links:read'),
|
||||
],
|
||||
# BUG (ghost component): ContactLinksTab does not exist in the
|
||||
# frontend — tab removed until implemented (Block I-D).
|
||||
detail_tabs=[],
|
||||
|
||||
author="LeoCRM Team",
|
||||
min_app_version="1.0.0",
|
||||
|
||||
@@ -40,6 +40,10 @@ class GraphRAGPlugin(BasePlugin):
|
||||
|
||||
async def on_activate(self, db, service_container, event_bus) -> None:
|
||||
"""Activate plugin: register GraphRAG search provider."""
|
||||
# Base class first (event subscriptions etc.), then own registrations
|
||||
# so a failure in our code cannot leave the base half-initialized (ARCH-037).
|
||||
await super().on_activate(db, service_container, event_bus)
|
||||
|
||||
from app.plugins.builtins.graph_rag.provider import GraphRAGSearchProvider
|
||||
from app.plugins.builtins.unified_search.contracts import get_search_registry
|
||||
|
||||
@@ -50,8 +54,6 @@ class GraphRAGPlugin(BasePlugin):
|
||||
import logging
|
||||
logging.getLogger(__name__).exception("Failed to register GraphRAGSearchProvider")
|
||||
|
||||
await super().on_activate(db, service_container, event_bus)
|
||||
|
||||
async def on_deactivate(self, db, service_container, event_bus) -> None:
|
||||
"""Deactivate plugin: unregister search provider and contract."""
|
||||
from app.plugins.builtins.unified_search.contracts import get_search_registry
|
||||
|
||||
@@ -25,7 +25,8 @@ async def cleanup_knowledge_job(ctx: dict[str, Any]) -> None:
|
||||
Runs daily. Keeps approved extractions indefinitely.
|
||||
Iterates per-tenant for RLS compliance.
|
||||
"""
|
||||
from sqlalchemy import text as sa_text, delete as sa_delete
|
||||
from sqlalchemy import delete as sa_delete
|
||||
from sqlalchemy import text as sa_text
|
||||
|
||||
from app.core.db import get_worker_session_factory
|
||||
|
||||
|
||||
@@ -1,12 +1,17 @@
|
||||
"""Knowledge extraction models — tracks LLM extractions and review queue."""
|
||||
from __future__ import annotations
|
||||
|
||||
import uuid
|
||||
from datetime import datetime
|
||||
from sqlalchemy import DateTime, Float, ForeignKey, Index, Integer, String, Text, func
|
||||
from sqlalchemy.dialects.postgresql import JSONB, UUID as PGUUID
|
||||
|
||||
from sqlalchemy import DateTime, Float, ForeignKey, Index, String, Text, func
|
||||
from sqlalchemy.dialects.postgresql import JSONB
|
||||
from sqlalchemy.dialects.postgresql import UUID as PGUUID
|
||||
from sqlalchemy.orm import Mapped, mapped_column
|
||||
|
||||
from app.core.db import Base, TenantMixin
|
||||
|
||||
|
||||
class KnowledgeExtraction(Base, TenantMixin):
|
||||
"""Tracks a single knowledge extraction run from a source (wiki, dms, mail, comm)."""
|
||||
__tablename__ = "knowledge_extractions"
|
||||
|
||||
@@ -1,8 +1,10 @@
|
||||
"""Knowledge plugin — LLM-based entity/relationship extraction, ask-knowledge, review queue."""
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import uuid
|
||||
from typing import Any
|
||||
|
||||
from app.plugins.base import BasePlugin
|
||||
from app.plugins.manifest import PluginManifest, PluginRouteDef
|
||||
|
||||
@@ -25,7 +27,7 @@ class KnowledgePlugin(BasePlugin):
|
||||
"""Register event-driven extraction hooks on activation."""
|
||||
await super().on_activate(db, service_container, event_bus)
|
||||
try:
|
||||
from app.core.hooks import register_action
|
||||
from app.core.hooks import get_hook_registry
|
||||
from app.plugins.builtins.knowledge.services import extract_knowledge
|
||||
async def on_wiki_create(*args, **kwargs):
|
||||
article_id = kwargs.get("article_id") or kwargs.get("entity_id")
|
||||
@@ -41,7 +43,9 @@ class KnowledgePlugin(BasePlugin):
|
||||
source_type="wiki_article", source_id=uuid.UUID(str(article_id)),
|
||||
source_title=title, source_text=content,
|
||||
)
|
||||
register_action("wiki.article.created", on_wiki_create, priority=20, owner_tag="knowledge")
|
||||
get_hook_registry().register_action(
|
||||
"wiki.article.created", on_wiki_create, priority=20, owner_tag="knowledge"
|
||||
)
|
||||
# H-DATA-LIFE: Re-extract when wiki article is updated
|
||||
async def on_wiki_update(*args, **kwargs):
|
||||
article_id = kwargs.get("article_id") or kwargs.get("entity_id")
|
||||
@@ -57,7 +61,9 @@ class KnowledgePlugin(BasePlugin):
|
||||
source_type="wiki_article", source_id=uuid.UUID(str(article_id)),
|
||||
source_title=title, source_text=content,
|
||||
)
|
||||
register_action("wiki.article.updated", on_wiki_update, priority=20, owner_tag="knowledge")
|
||||
get_hook_registry().register_action(
|
||||
"wiki.article.updated", on_wiki_update, priority=20, owner_tag="knowledge"
|
||||
)
|
||||
logger.info("Registered knowledge extraction hooks")
|
||||
except Exception:
|
||||
logger.exception("Failed to register knowledge hooks")
|
||||
@@ -74,8 +80,8 @@ class KnowledgePlugin(BasePlugin):
|
||||
|
||||
async def _ask_knowledge_handler(arguments: dict[str, Any], context: dict[str, Any]) -> dict[str, Any]:
|
||||
"""Ask a knowledge question."""
|
||||
from app.plugins.builtins.knowledge.services import ask_knowledge
|
||||
from app.core.db import get_worker_session_factory
|
||||
from app.plugins.builtins.knowledge.services import ask_knowledge
|
||||
question = arguments.get("question", "")
|
||||
tenant_id = context.get("tenant_id")
|
||||
if not question or not tenant_id:
|
||||
@@ -103,8 +109,8 @@ class KnowledgePlugin(BasePlugin):
|
||||
|
||||
async def _search_knowledge_handler(arguments: dict[str, Any], context: dict[str, Any]) -> dict[str, Any]:
|
||||
"""Search wiki articles via unified search."""
|
||||
from app.plugins.builtins.unified_search.provider_registry import get_search_registry
|
||||
from app.core.db import get_worker_session_factory
|
||||
from app.plugins.builtins.unified_search.provider_registry import get_search_registry
|
||||
query = arguments.get("query", "")
|
||||
tenant_id = context.get("tenant_id")
|
||||
if not query or not tenant_id:
|
||||
@@ -149,8 +155,8 @@ class KnowledgePlugin(BasePlugin):
|
||||
|
||||
async def on_deactivate(self, db, service_container, event_bus) -> None:
|
||||
"""Clean up on deactivation."""
|
||||
from app.core.hooks import unregister_actions_by_owner
|
||||
unregister_actions_by_owner("knowledge")
|
||||
from app.core.hooks import get_hook_registry
|
||||
get_hook_registry().unregister_all_for_plugin("knowledge")
|
||||
# Unregister knowledge agent tools from the core AI tool registry
|
||||
try:
|
||||
from app.ai.tool_registry import get_tool_registry
|
||||
|
||||
@@ -1,51 +1,70 @@
|
||||
"""Knowledge plugin routes — extraction, ask, review queue."""
|
||||
from __future__ import annotations
|
||||
|
||||
import uuid
|
||||
|
||||
from fastapi import APIRouter, Depends, HTTPException, Query
|
||||
from pydantic import BaseModel, Field
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
|
||||
from app.core.db import get_db
|
||||
from app.deps import require_permission
|
||||
from app.plugins.builtins.knowledge.services import extract_knowledge, ask_knowledge, get_review_queue, review_extraction
|
||||
from app.plugins.builtins.knowledge.services import (
|
||||
ask_knowledge,
|
||||
extract_knowledge,
|
||||
get_review_queue,
|
||||
review_extraction,
|
||||
)
|
||||
|
||||
router = APIRouter(prefix="/api/v1/knowledge", tags=["knowledge"])
|
||||
|
||||
|
||||
class KnowledgeExtractRequest(BaseModel):
|
||||
"""Extract knowledge from a source."""
|
||||
|
||||
source_type: str = Field(..., min_length=1)
|
||||
source_id: uuid.UUID
|
||||
source_title: str | None = None
|
||||
source_text: str = Field(..., min_length=1)
|
||||
|
||||
|
||||
class KnowledgeAskRequest(BaseModel):
|
||||
"""Ask a knowledge question."""
|
||||
|
||||
question: str = Field(..., min_length=1)
|
||||
|
||||
|
||||
class KnowledgeReviewRequest(BaseModel):
|
||||
"""Approve or reject a knowledge extraction."""
|
||||
|
||||
approved: bool = False
|
||||
notes: str | None = None
|
||||
|
||||
|
||||
@router.post("/extract")
|
||||
async def extract(
|
||||
body: dict,
|
||||
body: KnowledgeExtractRequest,
|
||||
db: AsyncSession = Depends(get_db),
|
||||
current_user: dict = Depends(require_permission("wiki:read")),
|
||||
):
|
||||
"""Extract knowledge from a source (wiki article, dms file, mail, communication)."""
|
||||
tenant_id = uuid.UUID(current_user["tenant_id"])
|
||||
source_type = body.get("source_type", "")
|
||||
source_id = body.get("source_id", "")
|
||||
source_title = body.get("source_title")
|
||||
source_text = body.get("source_text", "")
|
||||
if not source_type or not source_id or not source_text:
|
||||
raise HTTPException(400, detail={"detail": "source_type, source_id, source_text required", "code": "missing_fields"})
|
||||
try:
|
||||
sid = uuid.UUID(source_id)
|
||||
except ValueError:
|
||||
raise HTTPException(400, detail={"detail": "Invalid source_id", "code": "invalid_id"}) from None
|
||||
result = await extract_knowledge(
|
||||
db=db, tenant_id=tenant_id, source_type=source_type, source_id=sid,
|
||||
source_title=source_title, source_text=source_text,
|
||||
db=db, tenant_id=tenant_id, source_type=body.source_type, source_id=body.source_id,
|
||||
source_title=body.source_title, source_text=body.source_text,
|
||||
user_id=uuid.UUID(current_user["user_id"]) if current_user.get("user_id") else None,
|
||||
)
|
||||
return result
|
||||
|
||||
@router.post("/ask")
|
||||
async def ask(
|
||||
body: dict,
|
||||
body: KnowledgeAskRequest,
|
||||
db: AsyncSession = Depends(get_db),
|
||||
current_user: dict = Depends(require_permission("wiki:read")),
|
||||
):
|
||||
"""Ask a knowledge question — uses wiki + graph_rag as context."""
|
||||
tenant_id = uuid.UUID(current_user["tenant_id"])
|
||||
question = body.get("question") or body.get("query", "")
|
||||
if not question:
|
||||
raise HTTPException(400, detail={"detail": "question required", "code": "missing_question"})
|
||||
result = await ask_knowledge(db=db, tenant_id=tenant_id, question=question)
|
||||
result = await ask_knowledge(db=db, tenant_id=tenant_id, question=body.question)
|
||||
return result
|
||||
|
||||
@router.get("/review")
|
||||
@@ -62,22 +81,20 @@ async def review_queue(
|
||||
@router.post("/review/{extraction_id}")
|
||||
async def review(
|
||||
extraction_id: str,
|
||||
body: dict,
|
||||
body: KnowledgeReviewRequest,
|
||||
db: AsyncSession = Depends(get_db),
|
||||
current_user: dict = Depends(require_permission("wiki:write")),
|
||||
):
|
||||
"""Approve or reject a knowledge extraction."""
|
||||
tenant_id = uuid.UUID(current_user["tenant_id"])
|
||||
approved = body.get("approved", False)
|
||||
notes = body.get("notes")
|
||||
try:
|
||||
eid = uuid.UUID(extraction_id)
|
||||
except ValueError:
|
||||
raise HTTPException(400, detail={"detail": "Invalid extraction_id", "code": "invalid_id"}) from None
|
||||
result = await review_extraction(
|
||||
db=db, tenant_id=tenant_id, extraction_id=eid, approved=approved,
|
||||
db=db, tenant_id=tenant_id, extraction_id=eid, approved=body.approved,
|
||||
user_id=uuid.UUID(current_user["user_id"]) if current_user.get("user_id") else None,
|
||||
notes=notes,
|
||||
notes=body.notes,
|
||||
)
|
||||
if "error" in result:
|
||||
raise HTTPException(404, detail={"detail": result["error"], "code": "not_found"})
|
||||
|
||||
@@ -1,10 +1,13 @@
|
||||
"""Knowledge extraction services — LLM-based entity/relationship extraction."""
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import uuid
|
||||
from typing import Any
|
||||
from sqlalchemy import select, update
|
||||
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
|
||||
from app.ai.llm_client import llm_complete
|
||||
from app.plugins.builtins.knowledge.models import KnowledgeExtraction
|
||||
|
||||
|
||||
@@ -0,0 +1,7 @@
|
||||
-- Dual-path convergence (Gate B): add columns that Alembic migrations
|
||||
-- 0120 (is_system) and 0140 (folder_id) add on the core path. Idempotent
|
||||
-- so both install paths converge to the identical schema.
|
||||
ALTER TABLE comm_conversations ADD COLUMN IF NOT EXISTS is_system BOOLEAN NOT NULL DEFAULT FALSE;
|
||||
CREATE INDEX IF NOT EXISTS ix_comm_conversations_tenant_system ON comm_conversations(tenant_id, is_system);
|
||||
ALTER TABLE comm_conversations ADD COLUMN IF NOT EXISTS folder_id UUID;
|
||||
CREATE INDEX IF NOT EXISTS ix_comm_conversations_folder ON comm_conversations(folder_id);
|
||||
@@ -39,7 +39,7 @@ class KommunikationPlugin(BasePlugin):
|
||||
"participant.left",
|
||||
"reaction.added",
|
||||
],
|
||||
migrations=["0001_initial.sql"],
|
||||
migrations=["0001_initial.sql", "0002_system_channel_folders.sql"],
|
||||
permissions=[
|
||||
"comm:read",
|
||||
"comm:write",
|
||||
@@ -50,10 +50,10 @@ class KommunikationPlugin(BasePlugin):
|
||||
],
|
||||
is_core=True,
|
||||
menu_items=[
|
||||
FrontendMenuItem(label_key='nav.communication', label='Kommunikation', path='/communication', icon='MessageSquare', order=80),
|
||||
FrontendMenuItem(label_key='nav.communication', label='Kommunikation', path='/communication', icon='MessageSquare', order=80, permission='comm:read'),
|
||||
],
|
||||
page_routes=[
|
||||
FrontendPageRoute(path='/communication', component='@/pages/Communication', protected=True),
|
||||
FrontendPageRoute(path='/communication', component='@/pages/Communication', protected=True, permission='comm:read'),
|
||||
],
|
||||
author="LeoCRM Team",
|
||||
min_app_version="1.0.0",
|
||||
@@ -136,6 +136,13 @@ class KommunikationPlugin(BasePlugin):
|
||||
from app.plugins.builtins.contracts import get_contract_registry
|
||||
get_contract_registry().unregister(self.manifest.name)
|
||||
|
||||
# Remove services registered in on_activate BEFORE super() so that
|
||||
# event handlers being unsubscribed can no longer reach them (ARCH-033).
|
||||
for service_name in ("comm_websocket", "comm_miniapps"):
|
||||
if service_container.has(service_name):
|
||||
service_container.remove(service_name)
|
||||
logger.info("Removed '%s' from service container", service_name)
|
||||
|
||||
await super().on_deactivate(db, service_container, event_bus)
|
||||
logger.info("Kommunikation plugin deactivated")
|
||||
|
||||
|
||||
@@ -0,0 +1,115 @@
|
||||
"""Mail Account Service for the Mail plugin.
|
||||
|
||||
Extracted from services.py as part of the God-object split (BUG-018 pilot).
|
||||
Re-exported by ``app.plugins.builtins.mail.services``.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import uuid
|
||||
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
|
||||
from app.plugins.builtins.mail.crypto import encrypt_password, generate_salt
|
||||
from app.plugins.builtins.mail.models import (
|
||||
MailAccount,
|
||||
MailFolder,
|
||||
)
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
async def create_mail_account(
|
||||
db: AsyncSession, *, tenant_id: uuid.UUID, user_id: uuid.UUID, data: dict
|
||||
) -> MailAccount:
|
||||
"""Create a new mail account with encrypted password."""
|
||||
salt = generate_salt()
|
||||
account = MailAccount(
|
||||
tenant_id=tenant_id,
|
||||
user_id=user_id,
|
||||
owner_id=user_id,
|
||||
email_address=data["email_address"],
|
||||
display_name=data.get("display_name", ""),
|
||||
imap_host=data["imap_host"],
|
||||
imap_port=data.get("imap_port", 993),
|
||||
imap_ssl=data.get("imap_ssl", True),
|
||||
smtp_host=data["smtp_host"],
|
||||
smtp_port=data.get("smtp_port", 587),
|
||||
smtp_tls=data.get("smtp_tls", True),
|
||||
username=data.get("username") or data["email_address"],
|
||||
encrypted_password=encrypt_password(data["password"], salt),
|
||||
password_salt=salt,
|
||||
is_shared=data.get("is_shared", False),
|
||||
is_active=True,
|
||||
sent_folder_imap_name=data.get("sent_folder_imap_name"),
|
||||
drafts_folder_imap_name=data.get("drafts_folder_imap_name"),
|
||||
spam_folder_imap_name=data.get("spam_folder_imap_name"),
|
||||
trash_folder_imap_name=data.get("trash_folder_imap_name"),
|
||||
)
|
||||
db.add(account)
|
||||
await db.flush()
|
||||
|
||||
# Create INBOX first so subfolders can reference it as parent
|
||||
inbox_folder = MailFolder(
|
||||
tenant_id=tenant_id,
|
||||
account_id=account.id,
|
||||
name="Posteingang",
|
||||
imap_name="INBOX",
|
||||
is_standard=True,
|
||||
)
|
||||
db.add(inbox_folder)
|
||||
await db.flush()
|
||||
|
||||
# Create standard subfolders under INBOX (IMAP server uses '.' delimiter)
|
||||
for fname, imap_name in [
|
||||
("Gesendet", "INBOX.Sent"),
|
||||
("Entwürfe", "INBOX.Drafts"),
|
||||
("Papierkorb", "INBOX.Trash"),
|
||||
("Spam", "INBOX.spam"),
|
||||
]:
|
||||
folder = MailFolder(
|
||||
tenant_id=tenant_id,
|
||||
account_id=account.id,
|
||||
name=fname,
|
||||
imap_name=imap_name,
|
||||
parent_id=inbox_folder.id,
|
||||
is_standard=True,
|
||||
)
|
||||
db.add(folder)
|
||||
await db.flush()
|
||||
return account
|
||||
|
||||
|
||||
async def update_mail_account(db: AsyncSession, account: MailAccount, data: dict) -> MailAccount:
|
||||
"""Update a mail account, encrypting password if changed."""
|
||||
field_map = {
|
||||
"email": "email_address",
|
||||
"email_address": "email_address",
|
||||
"display_name": "display_name",
|
||||
"imap_host": "imap_host",
|
||||
"imap_port": "imap_port",
|
||||
"imap_ssl": "imap_ssl",
|
||||
"smtp_host": "smtp_host",
|
||||
"smtp_port": "smtp_port",
|
||||
"smtp_tls": "smtp_tls",
|
||||
"username": "username",
|
||||
"is_shared": "is_shared",
|
||||
"is_active": "is_active",
|
||||
"sent_folder_imap_name": "sent_folder_imap_name",
|
||||
"drafts_folder_imap_name": "drafts_folder_imap_name",
|
||||
"spam_folder_imap_name": "spam_folder_imap_name",
|
||||
"trash_folder_imap_name": "trash_folder_imap_name",
|
||||
}
|
||||
for api_field, model_field in field_map.items():
|
||||
if api_field in data and data[api_field] is not None:
|
||||
setattr(account, model_field, data[api_field])
|
||||
if "password" in data and data["password"] is not None:
|
||||
new_salt = generate_salt()
|
||||
account.password_salt = new_salt
|
||||
account.encrypted_password = encrypt_password(data["password"], new_salt)
|
||||
await db.flush()
|
||||
await db.refresh(account)
|
||||
return account
|
||||
|
||||
|
||||
@@ -0,0 +1,117 @@
|
||||
"""Attachment storage helpers for the Mail plugin.
|
||||
|
||||
Extracted from services.py as part of the God-object split (BUG-018 pilot).
|
||||
Re-exported by ``app.plugins.builtins.mail.services``.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import re
|
||||
import uuid
|
||||
|
||||
import aiofiles
|
||||
|
||||
from app.config import settings
|
||||
from app.plugins.builtins.mail.models import MailAttachment
|
||||
|
||||
# ─── Attachment Storage Helpers ───
|
||||
|
||||
MAX_ATTACHMENT_SIZE = 25 * 1024 * 1024 # 25 MB
|
||||
|
||||
|
||||
def _decode_mime_filename(filename: str) -> str:
|
||||
"""Decode MIME-encoded filename, handling =?charset?Q?...?= and =?charset?B?...?= patterns."""
|
||||
if not filename:
|
||||
return "attachment"
|
||||
# If no MIME encoding pattern, return as-is
|
||||
if "=?" not in filename:
|
||||
return filename
|
||||
try:
|
||||
from email.header import decode_header, make_header
|
||||
|
||||
return str(make_header(decode_header(filename)))
|
||||
except Exception:
|
||||
# Fallback: manually decode Q-encoding if decode_header fails
|
||||
# This handles cases where the email parser partially processed the filename
|
||||
try:
|
||||
import base64
|
||||
|
||||
def decode_q(match):
|
||||
charset, encoding, encoded = (
|
||||
match.group(1),
|
||||
match.group(2).upper(),
|
||||
match.group(3),
|
||||
)
|
||||
if encoding == "B":
|
||||
decoded = base64.b64decode(encoded).decode(
|
||||
charset or "utf-8", errors="replace"
|
||||
)
|
||||
else: # Q encoding
|
||||
decoded = encoded.replace("_", " ")
|
||||
decoded = re.sub(
|
||||
r"=([0-9A-Fa-f]{2})",
|
||||
lambda m: chr(int(m.group(1), 16)),
|
||||
decoded,
|
||||
)
|
||||
decoded = decoded.encode("latin-1").decode(
|
||||
charset or "utf-8", errors="replace"
|
||||
)
|
||||
return decoded
|
||||
|
||||
return re.sub(r"=\?([^?]+)\?([BbQq])\?([^?]*)\?=", decode_q, filename)
|
||||
except Exception:
|
||||
return filename
|
||||
|
||||
|
||||
def _sanitize_filename(filename: str) -> str:
|
||||
"""Sanitize a filename to prevent path traversal attacks."""
|
||||
# Remove any path components — keep only the basename
|
||||
filename = os.path.basename(filename or "attachment")
|
||||
# Replace potentially dangerous characters
|
||||
filename = re.sub(r"[^a-zA-Z0-9._-]", "_", filename)
|
||||
# Ensure non-empty
|
||||
if not filename:
|
||||
filename = "attachment"
|
||||
# Limit length
|
||||
if len(filename) > 200:
|
||||
name, ext = os.path.splitext(filename)
|
||||
filename = name[:200 - len(ext)] + ext
|
||||
return filename
|
||||
|
||||
|
||||
def _attachment_storage_path(mail_id: uuid.UUID, filename: str) -> str:
|
||||
"""Build the on-disk storage path for a mail attachment."""
|
||||
safe_name = _sanitize_filename(filename)
|
||||
return os.path.join(
|
||||
settings.storage_path,
|
||||
"mail_attachments",
|
||||
str(mail_id),
|
||||
safe_name,
|
||||
)
|
||||
|
||||
|
||||
async def _save_attachment_to_storage(
|
||||
mail_id: uuid.UUID, filename: str, content: bytes
|
||||
) -> str:
|
||||
"""Save attachment content to disk and return the storage path."""
|
||||
storage_path = _attachment_storage_path(mail_id, filename)
|
||||
os.makedirs(os.path.dirname(storage_path), exist_ok=True)
|
||||
async with aiofiles.open(storage_path, "wb") as f:
|
||||
await f.write(content)
|
||||
return storage_path
|
||||
|
||||
|
||||
def attachment_to_response(att: MailAttachment) -> dict:
|
||||
"""Convert a MailAttachment ORM object to a response dict."""
|
||||
return {
|
||||
"id": str(att.id),
|
||||
"mail_id": str(att.mail_id),
|
||||
"filename": att.filename,
|
||||
"mime_type": att.mime_type,
|
||||
"size_bytes": att.size_bytes,
|
||||
"size": att.size_bytes, # alias for frontend compatibility
|
||||
"content_id": att.content_id,
|
||||
"is_inline": bool(att.content_id),
|
||||
"dms_file_id": str(att.dms_file_id) if att.dms_file_id else None,
|
||||
}
|
||||
@@ -0,0 +1,74 @@
|
||||
"""AES-256 password encryption for Mail accounts (Fernet-based).
|
||||
|
||||
Extracted from services.py as part of the God-object split (BUG-018 pilot).
|
||||
Re-exported by ``app.plugins.builtins.mail.services`` for backwards
|
||||
compatibility.
|
||||
|
||||
⚠️ The legacy salt and MAIL_ENCRYPTION_KEY env guard are load-bearing:
|
||||
changing them makes existing encrypted passwords unreadable.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import base64
|
||||
import os
|
||||
|
||||
from cryptography.fernet import Fernet
|
||||
from cryptography.hazmat.primitives import hashes
|
||||
from cryptography.hazmat.primitives.kdf.pbkdf2 import PBKDF2HMAC
|
||||
|
||||
MAIL_ENCRYPTION_KEY = os.environ.get("MAIL_ENCRYPTION_KEY")
|
||||
if not MAIL_ENCRYPTION_KEY:
|
||||
raise RuntimeError(
|
||||
"MAIL_ENCRYPTION_KEY environment variable is required. "
|
||||
"Set it to a strong random value."
|
||||
)
|
||||
|
||||
# Legacy salt for backward compatibility with existing encrypted passwords
|
||||
_LEGACY_SALT = b"leocrm-mail-salt"
|
||||
|
||||
|
||||
def _derive_key(password: str, salt: bytes) -> bytes:
|
||||
"""Derive a 32-byte Fernet key from a password using PBKDF2 with the given salt."""
|
||||
kdf = PBKDF2HMAC(
|
||||
algorithm=hashes.SHA256(),
|
||||
length=32,
|
||||
salt=salt,
|
||||
iterations=480000,
|
||||
)
|
||||
return base64.urlsafe_b64encode(kdf.derive(password.encode()))
|
||||
|
||||
|
||||
def generate_salt() -> str:
|
||||
"""Generate a random 32-byte salt and return as base64 string."""
|
||||
salt = os.urandom(32)
|
||||
return base64.urlsafe_b64encode(salt).decode()
|
||||
|
||||
|
||||
def _get_fernet(salt_b64: str | None = None) -> Fernet:
|
||||
"""Get a Fernet instance. If salt_b64 is provided, use it; otherwise use legacy salt."""
|
||||
if salt_b64:
|
||||
salt = base64.urlsafe_b64decode(salt_b64.encode())
|
||||
else:
|
||||
salt = _LEGACY_SALT
|
||||
return Fernet(_derive_key(MAIL_ENCRYPTION_KEY, salt))
|
||||
|
||||
|
||||
def encrypt_password(plaintext: str, salt_b64: str | None = None) -> str:
|
||||
"""Encrypt a password using AES-256 (Fernet). Returns base64 ciphertext.
|
||||
|
||||
If salt_b64 is provided, uses that salt for key derivation.
|
||||
If not, uses the legacy hardcoded salt (for backward compatibility).
|
||||
"""
|
||||
fernet = _get_fernet(salt_b64)
|
||||
return fernet.encrypt(plaintext.encode()).decode()
|
||||
|
||||
|
||||
def decrypt_password(ciphertext: str, salt_b64: str | None = None) -> str:
|
||||
"""Decrypt a password encrypted with encrypt_password.
|
||||
|
||||
If salt_b64 is provided, uses that salt for key derivation.
|
||||
If not, uses the legacy hardcoded salt (for backward compatibility).
|
||||
"""
|
||||
fernet = _get_fernet(salt_b64)
|
||||
return fernet.decrypt(ciphertext.encode()).decode()
|
||||
@@ -0,0 +1,546 @@
|
||||
"""Draft save/update and auto-sync for the Mail plugin.
|
||||
|
||||
Extracted from services.py as part of the God-object split (BUG-018 pilot).
|
||||
Re-exported by ``app.plugins.builtins.mail.services``.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import uuid
|
||||
from datetime import UTC, datetime
|
||||
from email.message import EmailMessage
|
||||
from email.utils import formataddr, formatdate, make_msgid
|
||||
|
||||
import aioimaplib
|
||||
from sqlalchemy import and_, select
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
|
||||
from app.core.notifications import create_notification
|
||||
from app.plugins.builtins.mail.imap_ops import imap_delete_mail, imap_move_mail
|
||||
from app.plugins.builtins.mail.imap_sync import (
|
||||
get_account_password,
|
||||
imap_sync_account,
|
||||
)
|
||||
from app.plugins.builtins.mail.models import (
|
||||
Mail,
|
||||
MailAccount,
|
||||
MailFolder,
|
||||
)
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
async def save_draft(
|
||||
db: AsyncSession,
|
||||
account_id: uuid.UUID,
|
||||
tenant_id: uuid.UUID,
|
||||
user_id: uuid.UUID,
|
||||
data: dict,
|
||||
) -> Mail:
|
||||
"""Save a new draft mail to DB and IMAP Drafts folder."""
|
||||
import logging
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# 1. Find the Drafts folder (imap_name contains 'Drafts')
|
||||
account = (
|
||||
await db.execute(
|
||||
select(MailAccount).where(
|
||||
and_(MailAccount.id == account_id, MailAccount.tenant_id == tenant_id)
|
||||
)
|
||||
)
|
||||
).scalar_one_or_none()
|
||||
if not account:
|
||||
raise ValueError("Account not found")
|
||||
|
||||
folders = (
|
||||
await db.execute(
|
||||
select(MailFolder).where(
|
||||
and_(MailFolder.account_id == account_id, MailFolder.tenant_id == tenant_id)
|
||||
)
|
||||
)
|
||||
).scalars().all()
|
||||
|
||||
drafts_folder = None
|
||||
for f in folders:
|
||||
if 'draft' in f.imap_name.lower():
|
||||
drafts_folder = f
|
||||
break
|
||||
|
||||
if not drafts_folder:
|
||||
raise ValueError("No Drafts folder found for this account")
|
||||
|
||||
# 2. Create Mail record
|
||||
to_str = ', '.join(data.get('to', []))
|
||||
cc_str = ', '.join(data.get('cc', []))
|
||||
bcc_str = ', '.join(data.get('bcc', []))
|
||||
subject = data.get('subject', '')
|
||||
body_text = data.get('body_text', '')
|
||||
body_html = data.get('body_html', '')
|
||||
|
||||
msg_id = make_msgid()
|
||||
now = datetime.now(UTC)
|
||||
|
||||
mail = Mail(
|
||||
tenant_id=tenant_id,
|
||||
account_id=account_id,
|
||||
folder_id=drafts_folder.id,
|
||||
message_id=msg_id,
|
||||
thread_id=msg_id,
|
||||
subject=subject,
|
||||
from_address=account.email_address,
|
||||
to_addresses=to_str,
|
||||
cc_addresses=cc_str,
|
||||
bcc_addresses=bcc_str,
|
||||
body_text=body_text,
|
||||
body_html=body_html,
|
||||
body_html_sanitized=body_html,
|
||||
is_seen=True,
|
||||
is_flagged=False,
|
||||
is_draft=True,
|
||||
is_answered=False,
|
||||
is_forwarded=False,
|
||||
has_attachments=False,
|
||||
size_bytes=len(body_text.encode('utf-8')),
|
||||
received_at=now,
|
||||
sent_at=None,
|
||||
)
|
||||
db.add(mail)
|
||||
await db.flush()
|
||||
|
||||
# ── Notification: draft saved ──
|
||||
try:
|
||||
await create_notification(
|
||||
db, tenant_id, user_id,
|
||||
"mail_draft",
|
||||
"Entwurf gespeichert",
|
||||
subject or "Ohne Betreff",
|
||||
)
|
||||
await db.flush()
|
||||
except Exception:
|
||||
logger.debug("Ignored exception in mail service", exc_info=True)
|
||||
|
||||
# 3. Build RFC822 message and APPEND to IMAP Drafts folder
|
||||
password = await get_account_password(account)
|
||||
client = None
|
||||
try:
|
||||
client = aioimaplib.IMAP4_SSL(host=account.imap_host, port=account.imap_port)
|
||||
await client.wait_hello_from_server()
|
||||
await client.login(account.username, password)
|
||||
|
||||
# Build RFC822 message
|
||||
email_msg = EmailMessage()
|
||||
email_msg['From'] = formataddr((account.display_name, account.email_address))
|
||||
if to_str:
|
||||
email_msg['To'] = to_str
|
||||
if cc_str:
|
||||
email_msg['Cc'] = cc_str
|
||||
email_msg['Subject'] = subject
|
||||
email_msg['Date'] = formatdate(localtime=True)
|
||||
email_msg['Message-ID'] = msg_id
|
||||
email_msg.set_content(body_text if body_text else '')
|
||||
if body_html:
|
||||
email_msg.add_alternative(body_html, subtype='html')
|
||||
|
||||
rfc822_bytes = email_msg.as_bytes()
|
||||
|
||||
# APPEND to Drafts folder
|
||||
append_resp = await client.append(
|
||||
drafts_folder.imap_name,
|
||||
r'(\\Draft)',
|
||||
str(int(now.timestamp())),
|
||||
rfc822_bytes,
|
||||
)
|
||||
if append_resp.result != 'OK':
|
||||
logger.warning("save_draft: IMAP APPEND failed for drafts folder %s", drafts_folder.imap_name)
|
||||
else:
|
||||
logger.info("save_draft: appended draft %s to IMAP Drafts", mail.id)
|
||||
|
||||
except Exception as exc:
|
||||
logger.warning("save_draft: IMAP append failed (non-critical): %s", exc)
|
||||
finally:
|
||||
if client is not None:
|
||||
try:
|
||||
await client.logout()
|
||||
except Exception:
|
||||
logger.debug("Ignored exception in mail service", exc_info=True)
|
||||
|
||||
return mail
|
||||
|
||||
|
||||
async def update_draft(
|
||||
db: AsyncSession,
|
||||
mail_id: uuid.UUID,
|
||||
tenant_id: uuid.UUID,
|
||||
data: dict,
|
||||
) -> Mail:
|
||||
"""Update an existing draft."""
|
||||
import logging
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
mail = (
|
||||
await db.execute(
|
||||
select(Mail).where(
|
||||
and_(Mail.id == mail_id, Mail.tenant_id == tenant_id)
|
||||
)
|
||||
)
|
||||
).scalar_one_or_none()
|
||||
if not mail:
|
||||
raise ValueError("Mail not found")
|
||||
if not mail.is_draft:
|
||||
raise ValueError("Mail is not a draft")
|
||||
|
||||
# 2. Update fields
|
||||
to_str = ', '.join(data.get('to', []))
|
||||
cc_str = ', '.join(data.get('cc', []))
|
||||
bcc_str = ', '.join(data.get('bcc', []))
|
||||
mail.to_addresses = to_str
|
||||
mail.cc_addresses = cc_str
|
||||
mail.bcc_addresses = bcc_str
|
||||
mail.subject = data.get('subject', '')
|
||||
mail.body_text = data.get('body_text', '')
|
||||
mail.body_html = data.get('body_html', '')
|
||||
mail.body_html_sanitized = data.get('body_html', '')
|
||||
mail.size_bytes = len(mail.body_text.encode('utf-8'))
|
||||
await db.flush()
|
||||
|
||||
# 3. Delete old IMAP copy and APPEND new one
|
||||
folder = (
|
||||
await db.execute(
|
||||
select(MailFolder).where(
|
||||
and_(MailFolder.id == mail.folder_id, MailFolder.tenant_id == tenant_id)
|
||||
)
|
||||
)
|
||||
).scalar_one_or_none()
|
||||
if not folder:
|
||||
return mail
|
||||
|
||||
account = (
|
||||
await db.execute(
|
||||
select(MailAccount).where(
|
||||
and_(MailAccount.id == folder.account_id, MailAccount.tenant_id == tenant_id)
|
||||
)
|
||||
)
|
||||
).scalar_one_or_none()
|
||||
if not account:
|
||||
return mail
|
||||
|
||||
password = await get_account_password(account)
|
||||
client = None
|
||||
try:
|
||||
client = aioimaplib.IMAP4_SSL(host=account.imap_host, port=account.imap_port)
|
||||
await client.wait_hello_from_server()
|
||||
await client.login(account.username, password)
|
||||
|
||||
# Delete old copy from IMAP
|
||||
select_resp = await client.select(folder.imap_name)
|
||||
if select_resp.result == 'OK' and mail.message_id:
|
||||
search_resp = await client.uid_search(f'HEADER Message-ID "{mail.message_id}"')
|
||||
uids_raw = search_resp[1][0] if search_resp[1] and search_resp[1][0] else b''
|
||||
if isinstance(uids_raw, (bytes, bytearray)):
|
||||
uids = uids_raw.split()
|
||||
else:
|
||||
uids = []
|
||||
if uids:
|
||||
uid_str = uids[0].decode() if isinstance(uids[0], bytes) else str(uids[0])
|
||||
await client.uid('store', uid_str, r'+FLAGS (\\Deleted)')
|
||||
await client.expunge()
|
||||
|
||||
# APPEND new copy
|
||||
email_msg = EmailMessage()
|
||||
email_msg['From'] = formataddr((account.display_name, account.email_address))
|
||||
if to_str:
|
||||
email_msg['To'] = to_str
|
||||
if cc_str:
|
||||
email_msg['Cc'] = cc_str
|
||||
email_msg['Subject'] = mail.subject
|
||||
email_msg['Date'] = formatdate(localtime=True)
|
||||
email_msg['Message-ID'] = mail.message_id
|
||||
email_msg.set_content(mail.body_text if mail.body_text else '')
|
||||
if mail.body_html:
|
||||
email_msg.add_alternative(mail.body_html, subtype='html')
|
||||
|
||||
rfc822_bytes = email_msg.as_bytes()
|
||||
now = datetime.now(UTC)
|
||||
append_resp = await client.append(
|
||||
folder.imap_name,
|
||||
r'(\\Draft)',
|
||||
str(int(now.timestamp())),
|
||||
rfc822_bytes,
|
||||
)
|
||||
if append_resp.result != 'OK':
|
||||
logger.warning("update_draft: IMAP APPEND failed for drafts folder %s", folder.imap_name)
|
||||
else:
|
||||
logger.info("update_draft: appended updated draft %s to IMAP Drafts", mail.id)
|
||||
|
||||
except Exception as exc:
|
||||
logger.warning("update_draft: IMAP sync failed (non-critical): %s", exc)
|
||||
finally:
|
||||
if client is not None:
|
||||
try:
|
||||
await client.logout()
|
||||
except Exception:
|
||||
logger.debug("Ignored exception in mail service", exc_info=True)
|
||||
|
||||
return mail
|
||||
|
||||
|
||||
# ─── IMAP Folder Create / Delete ───
|
||||
|
||||
|
||||
async def imap_create_folder(
|
||||
db: AsyncSession, account_id: uuid.UUID, folder_name: str, tenant_id: uuid.UUID
|
||||
) -> None:
|
||||
"""Create folder on IMAP server.
|
||||
|
||||
Connects to IMAP, creates folder with CREATE command.
|
||||
Non-critical: errors are logged, DB operation still succeeds.
|
||||
"""
|
||||
import logging
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
account = (
|
||||
await db.execute(
|
||||
select(MailAccount).where(
|
||||
and_(MailAccount.id == account_id, MailAccount.tenant_id == tenant_id)
|
||||
)
|
||||
)
|
||||
).scalar_one_or_none()
|
||||
if not account:
|
||||
logger.warning("imap_create_folder: account %s not found", account_id)
|
||||
return
|
||||
|
||||
password = await get_account_password(account)
|
||||
client = None
|
||||
|
||||
try:
|
||||
client = aioimaplib.IMAP4_SSL(host=account.imap_host, port=account.imap_port)
|
||||
await client.wait_hello_from_server()
|
||||
await client.login(account.username, password)
|
||||
|
||||
resp = await client.create(folder_name)
|
||||
if resp.result != "OK":
|
||||
logger.warning(
|
||||
"imap_create_folder: CREATE failed for %s: %s", folder_name, resp
|
||||
)
|
||||
else:
|
||||
logger.info("imap_create_folder: created folder %s on IMAP", folder_name)
|
||||
try:
|
||||
await create_notification(
|
||||
db, account.tenant_id, account.user_id,
|
||||
"mail_folder",
|
||||
"Ordner erstellt",
|
||||
folder_name,
|
||||
)
|
||||
await db.flush()
|
||||
except Exception:
|
||||
logger.debug("Ignored exception in mail service", exc_info=True)
|
||||
|
||||
except Exception as exc:
|
||||
logger.warning("imap_create_folder: failed (non-critical): %s", exc)
|
||||
finally:
|
||||
if client is not None:
|
||||
try:
|
||||
await client.logout()
|
||||
except Exception:
|
||||
logger.debug("Ignored exception in mail service", exc_info=True)
|
||||
|
||||
|
||||
async def imap_delete_folder(
|
||||
db: AsyncSession, folder_id: uuid.UUID, tenant_id: uuid.UUID
|
||||
) -> None:
|
||||
"""Delete folder from IMAP server.
|
||||
|
||||
Connects to IMAP, deletes folder with DELETE command.
|
||||
Non-critical: errors are logged, DB operation still succeeds.
|
||||
"""
|
||||
import logging
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
folder = (
|
||||
await db.execute(
|
||||
select(MailFolder).where(
|
||||
and_(MailFolder.id == folder_id, MailFolder.tenant_id == tenant_id)
|
||||
)
|
||||
)
|
||||
).scalar_one_or_none()
|
||||
if not folder:
|
||||
logger.warning("imap_delete_folder: folder %s not found", folder_id)
|
||||
return
|
||||
|
||||
account = (
|
||||
await db.execute(
|
||||
select(MailAccount).where(
|
||||
and_(
|
||||
MailAccount.id == folder.account_id,
|
||||
MailAccount.tenant_id == tenant_id,
|
||||
)
|
||||
)
|
||||
)
|
||||
).scalar_one_or_none()
|
||||
if not account:
|
||||
logger.warning("imap_delete_folder: account not found for folder %s", folder_id)
|
||||
return
|
||||
|
||||
password = await get_account_password(account)
|
||||
client = None
|
||||
|
||||
try:
|
||||
client = aioimaplib.IMAP4_SSL(host=account.imap_host, port=account.imap_port)
|
||||
await client.wait_hello_from_server()
|
||||
await client.login(account.username, password)
|
||||
|
||||
resp = await client.delete(folder.imap_name)
|
||||
if resp.result != "OK":
|
||||
logger.warning(
|
||||
"imap_delete_folder: DELETE failed for %s: %s", folder.imap_name, resp
|
||||
)
|
||||
else:
|
||||
logger.info("imap_delete_folder: deleted folder %s on IMAP", folder.imap_name)
|
||||
try:
|
||||
await create_notification(
|
||||
db, account.tenant_id, account.user_id,
|
||||
"mail_folder",
|
||||
"Ordner gelöscht",
|
||||
folder.imap_name,
|
||||
)
|
||||
await db.flush()
|
||||
except Exception:
|
||||
logger.debug("Ignored exception in mail service", exc_info=True)
|
||||
|
||||
except Exception as exc:
|
||||
logger.warning("imap_delete_folder: failed (non-critical): %s", exc)
|
||||
finally:
|
||||
if client is not None:
|
||||
try:
|
||||
await client.logout()
|
||||
except Exception:
|
||||
logger.debug("Ignored exception in mail service", exc_info=True)
|
||||
|
||||
|
||||
# ─── Auto-Sync ───
|
||||
|
||||
|
||||
async def auto_sync_all_accounts() -> None:
|
||||
"""Auto-sync all active mail accounts.
|
||||
|
||||
Called periodically by the background scheduler.
|
||||
Iterates all active mail accounts and syncs each one.
|
||||
"""
|
||||
import logging
|
||||
|
||||
from app.core.db import get_session_factory
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
factory = get_session_factory()
|
||||
async with factory() as db:
|
||||
accounts = (
|
||||
await db.execute(
|
||||
select(MailAccount).where(MailAccount.is_active.is_(True))
|
||||
)
|
||||
).scalars().all()
|
||||
|
||||
if not accounts:
|
||||
return
|
||||
|
||||
logger.info("auto_sync_all_accounts: syncing %d active account(s)", len(accounts))
|
||||
|
||||
for account in accounts:
|
||||
try:
|
||||
result = await imap_sync_account(db, account.id, account.tenant_id)
|
||||
logger.info(
|
||||
"auto_sync_all_accounts: synced account %s (%s): %s",
|
||||
account.id,
|
||||
account.username,
|
||||
result,
|
||||
)
|
||||
except Exception as exc:
|
||||
logger.warning(
|
||||
"auto_sync_all_accounts: failed for account %s: %s",
|
||||
account.id,
|
||||
exc,
|
||||
)
|
||||
try:
|
||||
await create_notification(
|
||||
db, account.tenant_id, account.user_id,
|
||||
"mail_sync_error",
|
||||
"Synchronisierung fehlgeschlagen",
|
||||
f"Account {account.email_address}: {exc}",
|
||||
)
|
||||
except Exception:
|
||||
logger.debug("Ignored exception in mail service", exc_info=True)
|
||||
# commit per-account so partial progress is saved
|
||||
try:
|
||||
await db.commit()
|
||||
except Exception:
|
||||
await db.rollback()
|
||||
|
||||
|
||||
async def process_sync_queue(db: AsyncSession) -> None:
|
||||
"""Process pending IMAP operations from the sync queue.
|
||||
|
||||
Called at the start of each auto-sync loop iteration.
|
||||
Retries failed delete/move operations.
|
||||
"""
|
||||
import logging
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
from app.plugins.builtins.mail.models import MailSyncQueue
|
||||
|
||||
pending = (
|
||||
await db.execute(
|
||||
select(MailSyncQueue).where(
|
||||
and_(
|
||||
MailSyncQueue.status == "pending",
|
||||
MailSyncQueue.attempts < MailSyncQueue.max_attempts,
|
||||
)
|
||||
)
|
||||
)
|
||||
).scalars().all()
|
||||
|
||||
if not pending:
|
||||
return
|
||||
|
||||
logger.info("process_sync_queue: processing %d pending operation(s)", len(pending))
|
||||
|
||||
for entry in pending:
|
||||
try:
|
||||
if entry.operation == "delete":
|
||||
is_permanent = entry.payload.get("permanent", False) if entry.payload else False
|
||||
await imap_delete_mail(db, entry.mail_id, entry.tenant_id, permanent=is_permanent)
|
||||
elif entry.operation == "move":
|
||||
target_folder_id = uuid.UUID(entry.payload.get("target_folder_id", ""))
|
||||
await imap_move_mail(db, entry.mail_id, target_folder_id, entry.tenant_id)
|
||||
else:
|
||||
logger.warning("process_sync_queue: unknown operation %s", entry.operation)
|
||||
entry.status = "failed"
|
||||
entry.last_error = f"Unknown operation: {entry.operation}"
|
||||
continue
|
||||
|
||||
entry.status = "completed"
|
||||
entry.updated_at = datetime.now(UTC)
|
||||
logger.info("process_sync_queue: completed %s for mail %s", entry.operation, entry.mail_id)
|
||||
|
||||
except Exception as exc:
|
||||
entry.attempts += 1
|
||||
entry.last_error = str(exc)
|
||||
entry.updated_at = datetime.now(UTC)
|
||||
if entry.attempts >= entry.max_attempts:
|
||||
entry.status = "failed"
|
||||
logger.warning(
|
||||
"process_sync_queue: giving up on %s for mail %s after %d attempts: %s",
|
||||
entry.operation, entry.mail_id, entry.attempts, exc,
|
||||
)
|
||||
else:
|
||||
logger.info(
|
||||
"process_sync_queue: retry %d/%d for %s mail %s: %s",
|
||||
entry.attempts, entry.max_attempts, entry.operation, entry.mail_id, exc,
|
||||
)
|
||||
|
||||
await db.flush()
|
||||
@@ -0,0 +1,452 @@
|
||||
"""IMAP operations (flags, delete, move, folders) for the Mail plugin.
|
||||
|
||||
Extracted from services.py as part of the God-object split (BUG-018 pilot).
|
||||
Re-exported by ``app.plugins.builtins.mail.services``.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import uuid
|
||||
|
||||
import aioimaplib
|
||||
from sqlalchemy import and_, select
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
|
||||
from app.plugins.builtins.mail.imap_sync import (
|
||||
_parse_imap_list_response,
|
||||
get_account_password,
|
||||
)
|
||||
from app.plugins.builtins.mail.models import (
|
||||
Mail,
|
||||
MailAccount,
|
||||
MailFolder,
|
||||
)
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
async def imap_sync_mail_flags(
|
||||
db: AsyncSession,
|
||||
mail_id: uuid.UUID,
|
||||
tenant_id: uuid.UUID,
|
||||
) -> None:
|
||||
"""Sync is_seen/is_flagged flags from DB to IMAP server.
|
||||
|
||||
Connects to the IMAP server, selects the mail's folder,
|
||||
and uses UID STORE to set/remove \\Seen and \\Flagged flags.
|
||||
Non-critical: logs warnings on failure but does not raise.
|
||||
"""
|
||||
import logging
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# Load the mail with its folder and account
|
||||
mail = (
|
||||
await db.execute(
|
||||
select(Mail).where(
|
||||
and_(Mail.id == mail_id, Mail.tenant_id == tenant_id)
|
||||
)
|
||||
)
|
||||
).scalar_one_or_none()
|
||||
if not mail:
|
||||
logger.warning("imap_sync_mail_flags: mail %s not found", mail_id)
|
||||
return
|
||||
|
||||
folder = (
|
||||
await db.execute(
|
||||
select(MailFolder).where(
|
||||
and_(MailFolder.id == mail.folder_id, MailFolder.tenant_id == tenant_id)
|
||||
)
|
||||
)
|
||||
).scalar_one_or_none()
|
||||
if not folder:
|
||||
logger.warning("imap_sync_mail_flags: folder %s not found for mail %s", mail.folder_id, mail_id)
|
||||
return
|
||||
|
||||
account = (
|
||||
await db.execute(
|
||||
select(MailAccount).where(
|
||||
and_(MailAccount.id == folder.account_id, MailAccount.tenant_id == tenant_id)
|
||||
)
|
||||
)
|
||||
).scalar_one_or_none()
|
||||
if not account:
|
||||
logger.warning("imap_sync_mail_flags: account not found for mail %s", mail_id)
|
||||
return
|
||||
|
||||
if not mail.message_id:
|
||||
logger.warning("imap_sync_mail_flags: mail %s has no message_id, cannot sync", mail_id)
|
||||
return
|
||||
|
||||
password = await get_account_password(account)
|
||||
client = None
|
||||
|
||||
try:
|
||||
client = aioimaplib.IMAP4_SSL(host=account.imap_host, port=account.imap_port)
|
||||
await client.wait_hello_from_server()
|
||||
await client.login(account.username, password)
|
||||
|
||||
# Select the folder
|
||||
select_resp = await client.select(folder.imap_name)
|
||||
if select_resp.result != 'OK':
|
||||
logger.warning("imap_sync_mail_flags: cannot select folder %s", folder.imap_name)
|
||||
return
|
||||
|
||||
# Find the UID by searching for the Message-ID header
|
||||
search_resp = await client.uid_search(f'HEADER Message-ID "{mail.message_id}"')
|
||||
uids_raw = search_resp[1][0] if search_resp[1] and search_resp[1][0] else b''
|
||||
if isinstance(uids_raw, (bytes, bytearray)):
|
||||
uids = uids_raw.split()
|
||||
else:
|
||||
uids = []
|
||||
|
||||
if not uids:
|
||||
logger.warning("imap_sync_mail_flags: no UID found for Message-ID %s", mail.message_id)
|
||||
return
|
||||
|
||||
uid_str = uids[0].decode() if isinstance(uids[0], bytes) else str(uids[0])
|
||||
|
||||
# Sync \Seen flag
|
||||
if mail.is_seen:
|
||||
await client.uid('store', uid_str, '+FLAGS (\\Seen)')
|
||||
else:
|
||||
await client.uid('store', uid_str, '-FLAGS (\\Seen)')
|
||||
|
||||
# Sync \Flagged flag
|
||||
if mail.is_flagged:
|
||||
await client.uid('store', uid_str, '+FLAGS (\\Flagged)')
|
||||
else:
|
||||
await client.uid('store', uid_str, '-FLAGS (\\Flagged)')
|
||||
|
||||
logger.info("imap_sync_mail_flags: synced flags for mail %s (UID %s)", mail_id, uid_str)
|
||||
|
||||
except Exception as exc:
|
||||
logger.warning("imap_sync_mail_flags: failed for mail %s: %s", mail_id, exc)
|
||||
finally:
|
||||
if client is not None:
|
||||
try:
|
||||
await client.logout()
|
||||
except Exception:
|
||||
logger.debug("Ignored exception in mail service", exc_info=True)
|
||||
|
||||
|
||||
# ─── IMAP Delete ───
|
||||
|
||||
|
||||
async def _find_trash_folder_name(
|
||||
db: AsyncSession, account: MailAccount, tenant_id: uuid.UUID
|
||||
) -> str | None:
|
||||
"""Find the IMAP Trash folder name for an account.
|
||||
|
||||
Checks account.trash_folder_imap_name, then DB folders with 'trash' in imap_name,
|
||||
then queries the IMAP server LIST for common Trash folder names.
|
||||
"""
|
||||
# 1) Explicit mapping on account
|
||||
if account.trash_folder_imap_name:
|
||||
return account.trash_folder_imap_name
|
||||
|
||||
# 2) DB folder with 'trash' in imap_name
|
||||
db_folders = (
|
||||
await db.execute(
|
||||
select(MailFolder).where(
|
||||
and_(MailFolder.account_id == account.id, MailFolder.tenant_id == tenant_id)
|
||||
)
|
||||
)
|
||||
).scalars().all()
|
||||
for f in db_folders:
|
||||
if 'trash' in f.imap_name.lower():
|
||||
return f.imap_name
|
||||
|
||||
# 3) Query IMAP server for common Trash folder names
|
||||
password = await get_account_password(account)
|
||||
client = None
|
||||
try:
|
||||
client = aioimaplib.IMAP4_SSL(host=account.imap_host, port=account.imap_port)
|
||||
await client.wait_hello_from_server()
|
||||
await client.login(account.username, password)
|
||||
list_resp = await client.list('""', '"*"')
|
||||
imap_folders, _ = _parse_imap_list_response(list_resp)
|
||||
trash_candidates = ['Trash', 'INBOX.Trash', 'INBOX.Trash', 'Deleted', 'Deleted Items', 'Papierkorb']
|
||||
for _, name in imap_folders:
|
||||
if name in trash_candidates or 'trash' in name.lower():
|
||||
return name
|
||||
except Exception:
|
||||
logger.debug("Ignored exception in mail service", exc_info=True)
|
||||
finally:
|
||||
if client:
|
||||
try:
|
||||
await client.logout()
|
||||
except Exception:
|
||||
logger.debug("Ignored exception in mail service", exc_info=True)
|
||||
|
||||
return None
|
||||
|
||||
|
||||
async def imap_delete_mail(
|
||||
db: AsyncSession, mail_id: uuid.UUID, tenant_id: uuid.UUID, *, permanent: bool = False
|
||||
) -> None:
|
||||
"""Delete mail from IMAP server.
|
||||
|
||||
If permanent=False (default): MOVE mail to IMAP Trash folder (like real mail clients).
|
||||
If permanent=True: STORE \\Deleted + EXPUNGE (permanent delete from server).
|
||||
Uses UID MOVE if supported, otherwise COPY + STORE \\Deleted + EXPUNGE.
|
||||
Raises exceptions on failure so caller can queue for retry.
|
||||
"""
|
||||
import logging
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
mail = (
|
||||
await db.execute(
|
||||
select(Mail).where(
|
||||
and_(Mail.id == mail_id, Mail.tenant_id == tenant_id)
|
||||
)
|
||||
)
|
||||
).scalar_one_or_none()
|
||||
if not mail:
|
||||
logger.warning("imap_delete_mail: mail %s not found", mail_id)
|
||||
return
|
||||
|
||||
folder = (
|
||||
await db.execute(
|
||||
select(MailFolder).where(
|
||||
and_(MailFolder.id == mail.folder_id, MailFolder.tenant_id == tenant_id)
|
||||
)
|
||||
)
|
||||
).scalar_one_or_none()
|
||||
if not folder:
|
||||
logger.warning("imap_delete_mail: folder %s not found for mail %s", mail.folder_id, mail_id)
|
||||
return
|
||||
|
||||
account = (
|
||||
await db.execute(
|
||||
select(MailAccount).where(
|
||||
and_(MailAccount.id == folder.account_id, MailAccount.tenant_id == tenant_id)
|
||||
)
|
||||
)
|
||||
).scalar_one_or_none()
|
||||
if not account:
|
||||
logger.warning("imap_delete_mail: account not found for mail %s", mail_id)
|
||||
return
|
||||
|
||||
if not mail.imap_uid and not mail.message_id:
|
||||
logger.warning("imap_delete_mail: mail %s has no imap_uid or message_id, cannot sync", mail_id)
|
||||
return
|
||||
|
||||
# Lifecycle hook: mail.before_delete
|
||||
from app.core.hooks import do_action
|
||||
await do_action("mail.before_delete", mail_id=str(mail.id), tenant_id=str(tenant_id), permanent=permanent, db=db)
|
||||
|
||||
# Find Trash folder on IMAP server (only needed for non-permanent delete)
|
||||
trash_folder_name = None
|
||||
if not permanent:
|
||||
trash_folder_name = await _find_trash_folder_name(db, account, tenant_id)
|
||||
if not trash_folder_name:
|
||||
logger.warning("imap_delete_mail: no Trash folder found for account %s, will permanent delete", account.id)
|
||||
|
||||
password = await get_account_password(account)
|
||||
client = None
|
||||
|
||||
try:
|
||||
client = aioimaplib.IMAP4_SSL(host=account.imap_host, port=account.imap_port)
|
||||
await client.wait_hello_from_server()
|
||||
await client.login(account.username, password)
|
||||
|
||||
select_resp = await client.select(folder.imap_name)
|
||||
if select_resp.result != 'OK':
|
||||
raise RuntimeError(f"cannot select folder {folder.imap_name}")
|
||||
|
||||
# Use stored imap_uid directly; fall back to Message-ID search
|
||||
if mail.imap_uid:
|
||||
uid_str = mail.imap_uid
|
||||
else:
|
||||
search_resp = await client.uid_search(f'HEADER Message-ID "{mail.message_id}"')
|
||||
uids_raw = search_resp[1][0] if search_resp[1] and search_resp[1][0] else b''
|
||||
if isinstance(uids_raw, (bytes, bytearray)):
|
||||
uids = uids_raw.split()
|
||||
else:
|
||||
uids = []
|
||||
if not uids:
|
||||
raise RuntimeError(f"no UID found for Message-ID {mail.message_id}")
|
||||
uid_str = uids[0].decode() if isinstance(uids[0], bytes) else str(uids[0])
|
||||
|
||||
if permanent or not trash_folder_name:
|
||||
# Permanent delete: STORE \Deleted + EXPUNGE
|
||||
await client.uid('store', uid_str, r'+FLAGS (\\Deleted)')
|
||||
await client.expunge()
|
||||
logger.info("imap_delete_mail: permanent deleted mail %s (UID %s)", mail_id, uid_str)
|
||||
return
|
||||
|
||||
# Non-permanent: MOVE to Trash folder (like Thunderbird, Outlook, etc.)
|
||||
# Don't move if already in Trash
|
||||
if folder.imap_name.lower() == trash_folder_name.lower():
|
||||
# Already in Trash — permanent delete
|
||||
await client.uid('store', uid_str, r'+FLAGS (\\Deleted)')
|
||||
await client.expunge()
|
||||
logger.info("imap_delete_mail: permanent deleted mail %s (already in Trash, UID %s)", mail_id, uid_str)
|
||||
return
|
||||
|
||||
# Try UID MOVE first
|
||||
try:
|
||||
move_resp = await client.uid('move', uid_str, trash_folder_name)
|
||||
if move_resp.result == 'OK':
|
||||
logger.info("imap_delete_mail: moved mail %s (UID %s) to Trash %s", mail_id, uid_str, trash_folder_name)
|
||||
return
|
||||
except Exception as move_exc:
|
||||
logger.info("imap_delete_mail: UID MOVE not supported, falling back: %s", move_exc)
|
||||
|
||||
# Fallback: COPY to Trash + STORE \Deleted + EXPUNGE
|
||||
copy_resp = await client.uid('copy', uid_str, trash_folder_name)
|
||||
if copy_resp.result != 'OK':
|
||||
raise RuntimeError(f"COPY to Trash failed for mail {mail_id}")
|
||||
await client.uid('store', uid_str, r'+FLAGS (\\Deleted)')
|
||||
await client.expunge()
|
||||
logger.info("imap_delete_mail: moved mail %s (UID %s) to Trash via COPY+DELETE", mail_id, uid_str)
|
||||
|
||||
# Lifecycle hook: mail.after_delete
|
||||
from app.core.hooks import do_action
|
||||
await do_action("mail.after_delete", mail_id=str(mail.id), tenant_id=str(tenant_id), permanent=permanent, db=db)
|
||||
|
||||
except Exception as exc:
|
||||
logger.warning("imap_delete_mail: failed for mail %s: %s", mail_id, exc)
|
||||
raise
|
||||
finally:
|
||||
if client is not None:
|
||||
try:
|
||||
await client.logout()
|
||||
except Exception:
|
||||
logger.debug("Ignored exception in mail service", exc_info=True)
|
||||
|
||||
|
||||
# ─── IMAP Move ───
|
||||
|
||||
|
||||
async def imap_move_mail(
|
||||
db: AsyncSession,
|
||||
mail_id: uuid.UUID,
|
||||
target_folder_id: uuid.UUID,
|
||||
tenant_id: uuid.UUID,
|
||||
) -> None:
|
||||
"""Move mail to another folder on IMAP server.
|
||||
|
||||
Uses stored imap_uid directly; falls back to Message-ID search if missing.
|
||||
Uses UID MOVE if supported, otherwise COPY + STORE \\Deleted + EXPUNGE.
|
||||
Raises exceptions on failure so caller can queue for retry.
|
||||
"""
|
||||
import logging
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
mail = (
|
||||
await db.execute(
|
||||
select(Mail).where(
|
||||
and_(Mail.id == mail_id, Mail.tenant_id == tenant_id)
|
||||
)
|
||||
)
|
||||
).scalar_one_or_none()
|
||||
if not mail:
|
||||
logger.warning("imap_move_mail: mail %s not found", mail_id)
|
||||
return
|
||||
|
||||
source_folder = (
|
||||
await db.execute(
|
||||
select(MailFolder).where(
|
||||
and_(MailFolder.id == mail.folder_id, MailFolder.tenant_id == tenant_id)
|
||||
)
|
||||
)
|
||||
).scalar_one_or_none()
|
||||
if not source_folder:
|
||||
logger.warning("imap_move_mail: source folder not found for mail %s", mail_id)
|
||||
return
|
||||
|
||||
target_folder = (
|
||||
await db.execute(
|
||||
select(MailFolder).where(
|
||||
and_(MailFolder.id == target_folder_id, MailFolder.tenant_id == tenant_id)
|
||||
)
|
||||
)
|
||||
).scalar_one_or_none()
|
||||
if not target_folder:
|
||||
logger.warning("imap_move_mail: target folder %s not found", target_folder_id)
|
||||
return
|
||||
|
||||
account = (
|
||||
await db.execute(
|
||||
select(MailAccount).where(
|
||||
and_(MailAccount.id == source_folder.account_id, MailAccount.tenant_id == tenant_id)
|
||||
)
|
||||
)
|
||||
).scalar_one_or_none()
|
||||
if not account:
|
||||
logger.warning("imap_move_mail: account not found for mail %s", mail_id)
|
||||
return
|
||||
|
||||
if not mail.imap_uid and not mail.message_id:
|
||||
logger.warning("imap_move_mail: mail %s has no imap_uid or message_id, cannot sync", mail_id)
|
||||
return
|
||||
|
||||
# Lifecycle hook: mail.before_move
|
||||
from app.core.hooks import do_action
|
||||
await do_action("mail.before_move", mail_id=str(mail.id), tenant_id=str(tenant_id), source_folder_id=str(source_folder.id), target_folder_id=str(target_folder.id), db=db)
|
||||
|
||||
password = await get_account_password(account)
|
||||
client = None
|
||||
|
||||
try:
|
||||
client = aioimaplib.IMAP4_SSL(host=account.imap_host, port=account.imap_port)
|
||||
await client.wait_hello_from_server()
|
||||
await client.login(account.username, password)
|
||||
|
||||
select_resp = await client.select(source_folder.imap_name)
|
||||
if select_resp.result != 'OK':
|
||||
raise RuntimeError(f"cannot select folder {source_folder.imap_name}")
|
||||
|
||||
# Use stored imap_uid directly; fall back to Message-ID search
|
||||
if mail.imap_uid:
|
||||
uid_str = mail.imap_uid
|
||||
else:
|
||||
search_resp = await client.uid_search(f'HEADER Message-ID "{mail.message_id}"')
|
||||
uids_raw = search_resp[1][0] if search_resp[1] and search_resp[1][0] else b''
|
||||
if isinstance(uids_raw, (bytes, bytearray)):
|
||||
uids = uids_raw.split()
|
||||
else:
|
||||
uids = []
|
||||
if not uids:
|
||||
raise RuntimeError(f"no UID found for Message-ID {mail.message_id}")
|
||||
uid_str = uids[0].decode() if isinstance(uids[0], bytes) else str(uids[0])
|
||||
|
||||
# Try UID MOVE first; fall back to COPY + STORE \\Deleted + EXPUNGE
|
||||
try:
|
||||
move_resp = await client.uid('move', uid_str, target_folder.imap_name)
|
||||
if move_resp.result == 'OK':
|
||||
logger.info("imap_move_mail: moved mail %s (UID %s) via UID MOVE", mail_id, uid_str)
|
||||
return
|
||||
except Exception as move_exc:
|
||||
logger.info("imap_move_mail: UID MOVE not supported, falling back: %s", move_exc)
|
||||
|
||||
# Fallback: COPY + STORE \\Deleted + EXPUNGE
|
||||
copy_resp = await client.uid('copy', uid_str, target_folder.imap_name)
|
||||
if copy_resp.result != 'OK':
|
||||
raise RuntimeError(f"COPY failed for mail {mail_id}")
|
||||
|
||||
await client.uid('store', uid_str, r'+FLAGS (\\Deleted)')
|
||||
await client.expunge()
|
||||
|
||||
logger.info("imap_move_mail: moved mail %s (UID %s) via COPY+DELETE", mail_id, uid_str)
|
||||
|
||||
# Lifecycle hook: mail.after_move
|
||||
from app.core.hooks import do_action
|
||||
await do_action("mail.after_move", mail_id=str(mail.id), tenant_id=str(tenant_id), source_folder_id=str(source_folder.id), target_folder_id=str(target_folder.id), db=db)
|
||||
|
||||
except Exception as exc:
|
||||
logger.warning("imap_move_mail: failed for mail %s: %s", mail_id, exc)
|
||||
raise
|
||||
finally:
|
||||
if client is not None:
|
||||
try:
|
||||
await client.logout()
|
||||
except Exception:
|
||||
logger.debug("Ignored exception in mail service", exc_info=True)
|
||||
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -4,7 +4,9 @@ from __future__ import annotations
|
||||
|
||||
import uuid
|
||||
from datetime import UTC, datetime
|
||||
from typing import Any
|
||||
|
||||
from pgvector.sqlalchemy import Vector
|
||||
from sqlalchemy import (
|
||||
JSON,
|
||||
Boolean,
|
||||
@@ -16,14 +18,12 @@ from sqlalchemy import (
|
||||
Text,
|
||||
UniqueConstraint,
|
||||
)
|
||||
from sqlalchemy.dialects.postgresql import TSVECTOR
|
||||
from sqlalchemy.dialects.postgresql import UUID as PGUUID
|
||||
from sqlalchemy.orm import Mapped, mapped_column
|
||||
|
||||
from app.core.db import Base, TenantMixin
|
||||
from app.models.owned_mixin import OwnedMixin
|
||||
from sqlalchemy.dialects.postgresql import TSVECTOR
|
||||
from pgvector.sqlalchemy import Vector
|
||||
from typing import Any
|
||||
|
||||
# --- Mail Accounts (F-MAIL-14, F-MAIL-18) ---
|
||||
|
||||
|
||||
@@ -0,0 +1,48 @@
|
||||
"""PGP encryption/decryption for the Mail plugin using pgpy.
|
||||
|
||||
Extracted from services.py as part of the God-object split (BUG-018 pilot).
|
||||
Re-exported by ``app.plugins.builtins.mail.services``.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import pgpy
|
||||
|
||||
|
||||
def import_pgp_private_key(private_key_armored: str, passphrase: str = "") -> tuple[str, str]:
|
||||
"""Import a PGP private key. Returns (key_id, public_key_armored)."""
|
||||
key, _ = pgpy.PGPKey.from_blob(private_key_armored)
|
||||
if key.is_protected:
|
||||
with key.unlock(passphrase):
|
||||
pub_key = key.pubkey
|
||||
key_id = str(key.fingerprint).upper()[-16:]
|
||||
return key_id, str(pub_key)
|
||||
pub_key = key.pubkey
|
||||
key_id = str(key.fingerprint).upper()[-16:]
|
||||
return key_id, str(pub_key)
|
||||
|
||||
|
||||
def import_pgp_public_key(public_key_armored: str) -> str:
|
||||
"""Import a PGP public key. Returns key_id."""
|
||||
key, _ = pgpy.PGPKey.from_blob(public_key_armored)
|
||||
return str(key.fingerprint).upper()[-16:]
|
||||
|
||||
|
||||
def pgp_encrypt_message(plaintext: str, recipient_public_key_armored: str) -> str:
|
||||
"""Encrypt a message with recipient's public PGP key."""
|
||||
pub_key, _ = pgpy.PGPKey.from_blob(recipient_public_key_armored)
|
||||
msg = pgpy.PGPMessage.new(plaintext)
|
||||
encrypted = pub_key.encrypt(msg)
|
||||
return str(encrypted)
|
||||
|
||||
|
||||
def pgp_decrypt_message(ciphertext: str, private_key_armored: str, passphrase: str = "") -> str:
|
||||
"""Decrypt a PGP-encrypted message."""
|
||||
key, _ = pgpy.PGPKey.from_blob(private_key_armored)
|
||||
enc_msg = pgpy.PGPMessage.from_blob(ciphertext)
|
||||
if key.is_protected:
|
||||
with key.unlock(passphrase):
|
||||
decrypted = key.decrypt(enc_msg)
|
||||
return decrypted.message.decode("utf-8")
|
||||
decrypted = key.decrypt(enc_msg)
|
||||
return decrypted.message.decode("utf-8")
|
||||
@@ -9,7 +9,6 @@ from typing import Any
|
||||
|
||||
from app.plugins.base import BasePlugin
|
||||
from app.plugins.manifest import (
|
||||
FrontendDetailTab,
|
||||
FrontendMenuItem,
|
||||
FrontendPageRoute,
|
||||
FrontendSettingsPage,
|
||||
@@ -115,7 +114,11 @@ async def _auto_sync_loop() -> None:
|
||||
class MailPlugin(BasePlugin):
|
||||
"""Mail plugin for email management: IMAP sync, SMTP send, threading, rules, PGP."""
|
||||
|
||||
_auto_sync_task: asyncio.Task | None = None
|
||||
def __init__(self) -> None:
|
||||
super().__init__()
|
||||
# Instance attribute: multiple plugin instances must not share the
|
||||
# background task state (ARCH-036).
|
||||
self._auto_sync_task: asyncio.Task | None = None
|
||||
|
||||
manifest = PluginManifest(
|
||||
name="mail",
|
||||
@@ -137,24 +140,30 @@ class MailPlugin(BasePlugin):
|
||||
migrations=["0001_initial.sql", "0006_flag_type.sql", "0007_sync_queue.sql", "0008_sync_queue_deleted_at.sql", "0009_remove_mail_soft_delete.sql", "0010_add_deleted_at.sql"],
|
||||
permissions=["mail:read", "mail:send", "mail:config", "mail:share", "mail:write", "mail:delete"],
|
||||
menu_items=[
|
||||
FrontendMenuItem(label_key='nav.mail', label='E-Mail', path='/mail', icon='Mail', order=30),
|
||||
FrontendMenuItem(label_key='nav.mail', label='E-Mail', path='/mail', icon='Mail', order=30, permission='mail:read'),
|
||||
],
|
||||
page_routes=[
|
||||
FrontendPageRoute(path='/mail', component='@/pages/Mail', protected=True),
|
||||
FrontendPageRoute(path='/mail/settings', component='@/pages/MailSettings', protected=True),
|
||||
FrontendPageRoute(path='/mail', component='@/pages/Mail', protected=True, permission='mail:read'),
|
||||
FrontendPageRoute(path='/mail/settings', component='@/pages/MailSettings', protected=True, permission='mail:config'),
|
||||
],
|
||||
settings_pages=[
|
||||
FrontendSettingsPage(path='mail', label_key='settings.mail', label='Mail', component='@/pages/MailSettings', icon='Mail', order=50),
|
||||
],
|
||||
detail_tabs=[
|
||||
FrontendDetailTab(entity_type='contact', label_key='tabs.email', label='E-Mails', component='@/components/contact/ContactMailTab', icon='Mail', order=20, permission='mail:read'),
|
||||
],
|
||||
# BUG (ghost component): ContactMailTab does not exist in the
|
||||
# frontend — tab removed until implemented (Block I-D).
|
||||
detail_tabs=[],
|
||||
author="LeoCRM Team",
|
||||
min_app_version="1.0.0",
|
||||
hooks=["mail.before_send", "mail.after_send"],
|
||||
contract_version="1.0.0",
|
||||
)
|
||||
|
||||
def get_entity_models(self) -> dict[str, type]:
|
||||
"""Entity types owned by Mail for the permission resolver."""
|
||||
from app.plugins.builtins.mail.models import MailAccount
|
||||
|
||||
return {"mail_account": MailAccount}
|
||||
|
||||
async def on_activate(
|
||||
self, db, service_container, event_bus
|
||||
) -> None:
|
||||
|
||||
@@ -56,6 +56,7 @@ from app.plugins.builtins.mail.schemas import (
|
||||
MailRuleCreate,
|
||||
MailSendRequest,
|
||||
MailSignatureCreate,
|
||||
MailSignatureUpdate,
|
||||
MailTemplateCreate,
|
||||
PgpKeyImport,
|
||||
SendPermissionCreate,
|
||||
@@ -892,7 +893,10 @@ async def list_threads(
|
||||
threads[tid]["mail_count"] += 1
|
||||
threads[tid]["mails"].append(mail_to_response(mail))
|
||||
items = list(threads.values())
|
||||
return {"items": items, "total": len(items)}
|
||||
# Plain array — consistent with sibling mail list routes (accounts,
|
||||
# folders, templates) and with the frontend client type
|
||||
# fetchThreads(): Promise<ThreadResult[]>.
|
||||
return items
|
||||
|
||||
|
||||
# ─── Templates (F-MAIL-06) ───
|
||||
@@ -992,6 +996,66 @@ async def list_signatures(
|
||||
return [signature_to_response(s) for s in sigs]
|
||||
|
||||
|
||||
@router.patch("/signatures/{signature_id}")
|
||||
async def update_signature(
|
||||
signature_id: str,
|
||||
data: MailSignatureUpdate,
|
||||
db: AsyncSession = Depends(get_db),
|
||||
current_user: dict = Depends(require_permission("mail:config")),
|
||||
):
|
||||
tenant_id = uuid.UUID(current_user["tenant_id"])
|
||||
user_id = uuid.UUID(current_user["user_id"])
|
||||
s_id = _parse_uuid(signature_id, "signature_id")
|
||||
sig = (
|
||||
await db.execute(
|
||||
select(MailSignature).where(
|
||||
and_(MailSignature.id == s_id, MailSignature.tenant_id == tenant_id)
|
||||
)
|
||||
)
|
||||
).scalar_one_or_none()
|
||||
if not sig:
|
||||
raise HTTPException(404, detail={"detail": "Signature not found", "code": "not_found"})
|
||||
if sig.user_id != user_id:
|
||||
raise HTTPException(403, detail={"detail": "Not your signature", "code": "forbidden"})
|
||||
sig.name = data.name
|
||||
sig.body_html = data.body_html
|
||||
if data.is_default:
|
||||
others = (
|
||||
await db.execute(
|
||||
select(MailSignature).where(MailSignature.user_id == user_id)
|
||||
)
|
||||
).scalars().all()
|
||||
for o in others:
|
||||
o.is_default = False
|
||||
sig.is_default = data.is_default
|
||||
await db.flush()
|
||||
return signature_to_response(sig)
|
||||
|
||||
|
||||
@router.delete("/signatures/{signature_id}", status_code=204)
|
||||
async def delete_signature(
|
||||
signature_id: str,
|
||||
db: AsyncSession = Depends(get_db),
|
||||
current_user: dict = Depends(require_permission("mail:config")),
|
||||
):
|
||||
tenant_id = uuid.UUID(current_user["tenant_id"])
|
||||
user_id = uuid.UUID(current_user["user_id"])
|
||||
s_id = _parse_uuid(signature_id, "signature_id")
|
||||
sig = (
|
||||
await db.execute(
|
||||
select(MailSignature).where(
|
||||
and_(MailSignature.id == s_id, MailSignature.tenant_id == tenant_id)
|
||||
)
|
||||
)
|
||||
).scalar_one_or_none()
|
||||
if not sig:
|
||||
raise HTTPException(404, detail={"detail": "Signature not found", "code": "not_found"})
|
||||
if sig.user_id != user_id:
|
||||
raise HTTPException(403, detail={"detail": "Not your signature", "code": "forbidden"})
|
||||
await db.delete(sig)
|
||||
await db.flush()
|
||||
|
||||
|
||||
# ─── Rules (F-MAIL-07) ───
|
||||
|
||||
|
||||
@@ -1206,6 +1270,28 @@ async def list_labels(
|
||||
return [label_to_response(lbl) for lbl in labels]
|
||||
|
||||
|
||||
@router.delete("/labels/{label_id}", status_code=204)
|
||||
async def delete_label(
|
||||
label_id: str,
|
||||
db: AsyncSession = Depends(get_db),
|
||||
current_user: dict = Depends(require_permission("mail:config")),
|
||||
):
|
||||
tenant_id = uuid.UUID(current_user["tenant_id"])
|
||||
user_id = uuid.UUID(current_user["user_id"])
|
||||
l_id = _parse_uuid(label_id, "label_id")
|
||||
label = (
|
||||
await db.execute(
|
||||
select(MailLabel).where(and_(MailLabel.id == l_id, MailLabel.tenant_id == tenant_id))
|
||||
)
|
||||
).scalar_one_or_none()
|
||||
if not label:
|
||||
raise HTTPException(404, detail={"detail": "Label not found", "code": "not_found"})
|
||||
if label.user_id != user_id:
|
||||
raise HTTPException(403, detail={"detail": "Not your label", "code": "forbidden"})
|
||||
await db.delete(label)
|
||||
await db.flush()
|
||||
|
||||
|
||||
# ─── Contact PGP Keys (F-MAIL-12) ───
|
||||
|
||||
|
||||
@@ -1494,8 +1580,8 @@ async def create_event_from_mail(
|
||||
try:
|
||||
from app.plugins.builtins.calendar.contracts import get_contract as get_calendar_contract
|
||||
_cal = get_calendar_contract()
|
||||
calendar = _cal.calendar
|
||||
calendar_entry = _cal.calendar_entry
|
||||
calendar = _cal.Calendar
|
||||
calendar_entry = _cal.CalendarEntry
|
||||
except ImportError:
|
||||
return {"created": False, "error": "calendar plugin not available"}
|
||||
cal_id = _parse_uuid(data.calendar_id, "calendar_id")
|
||||
@@ -1862,4 +1948,3 @@ async def get_mail(
|
||||
)
|
||||
return mail_to_response(mail, attachments=list(attachments), labels=list(label_assignments))
|
||||
|
||||
from app.core.audit import log_audit
|
||||
@@ -0,0 +1,194 @@
|
||||
"""Rules, Templates & Vacation for the Mail plugin.
|
||||
|
||||
Extracted from services.py as part of the God-object split (BUG-018 pilot).
|
||||
Re-exported by ``app.plugins.builtins.mail.services``.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import logging
|
||||
import uuid
|
||||
from datetime import UTC, datetime, timedelta
|
||||
|
||||
from sqlalchemy import and_, or_, select
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
|
||||
from app.plugins.builtins.mail.models import (
|
||||
Mail,
|
||||
MailFolder,
|
||||
MailLabelAssignment,
|
||||
MailRule,
|
||||
VacationSentLog,
|
||||
)
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def substitute_template_vars(template_body: str, variables: dict[str, str]) -> str:
|
||||
"""Replace {{placeholder}} variables in template body."""
|
||||
result = template_body
|
||||
for key, value in variables.items():
|
||||
result = result.replace(f"{{{{{key}}}}}", value)
|
||||
result = result.replace(f"{{{{{key.lower()}}}}}", value)
|
||||
result = result.replace(f"{{{{{key.upper()}}}}}", value)
|
||||
return result
|
||||
|
||||
|
||||
# ─── Mail Rule Engine (F-MAIL-07) ───
|
||||
|
||||
|
||||
def matches_condition(mail: Mail, conditions: dict) -> bool:
|
||||
"""Check if a mail matches all rule conditions."""
|
||||
for field, expected in conditions.items():
|
||||
if field == "from_contains":
|
||||
if expected.lower() not in mail.from_address.lower():
|
||||
return False
|
||||
elif field == "subject_contains":
|
||||
if expected.lower() not in mail.subject.lower():
|
||||
return False
|
||||
elif field == "to_contains":
|
||||
if expected.lower() not in mail.to_addresses.lower():
|
||||
return False
|
||||
elif field == "body_contains":
|
||||
body = (mail.body_text + mail.body_html).lower()
|
||||
if expected.lower() not in body:
|
||||
return False
|
||||
elif field == "has_attachments":
|
||||
if mail.has_attachments != bool(expected):
|
||||
return False
|
||||
elif field == "is_flagged":
|
||||
if mail.is_flagged != bool(expected):
|
||||
return False
|
||||
return True
|
||||
|
||||
|
||||
async def execute_rule_actions(
|
||||
db: AsyncSession, mail: Mail, actions: dict, tenant_id: uuid.UUID
|
||||
) -> dict:
|
||||
"""Execute rule actions on a matching mail."""
|
||||
results = {}
|
||||
for action, value in actions.items():
|
||||
if action == "move_to_folder":
|
||||
folder_id = uuid.UUID(value) if isinstance(value, str) else value
|
||||
folder = (
|
||||
await db.execute(
|
||||
select(MailFolder).where(
|
||||
and_(MailFolder.id == folder_id, MailFolder.tenant_id == tenant_id)
|
||||
)
|
||||
)
|
||||
).scalar_one_or_none()
|
||||
if folder:
|
||||
mail.folder_id = folder.id
|
||||
results["moved"] = str(folder.id)
|
||||
elif action == "label":
|
||||
label_id = uuid.UUID(value) if isinstance(value, str) else value
|
||||
existing = (
|
||||
await db.execute(
|
||||
select(MailLabelAssignment).where(
|
||||
and_(
|
||||
MailLabelAssignment.mail_id == mail.id,
|
||||
MailLabelAssignment.label_id == label_id,
|
||||
)
|
||||
)
|
||||
)
|
||||
).scalar_one_or_none()
|
||||
if not existing:
|
||||
assignment = MailLabelAssignment(
|
||||
tenant_id=tenant_id,
|
||||
mail_id=mail.id,
|
||||
label_id=label_id,
|
||||
)
|
||||
db.add(assignment)
|
||||
results["labeled"] = str(label_id)
|
||||
elif action == "mark_seen":
|
||||
mail.is_seen = bool(value)
|
||||
results["seen"] = bool(value)
|
||||
elif action == "mark_flagged":
|
||||
mail.is_flagged = bool(value)
|
||||
results["flagged"] = bool(value)
|
||||
elif action == "forward_to":
|
||||
results["forward_to"] = value
|
||||
await db.flush()
|
||||
return results
|
||||
|
||||
|
||||
async def apply_rules_to_mail(db: AsyncSession, mail: Mail, tenant_id: uuid.UUID) -> list[dict]:
|
||||
"""Find and apply all matching rules to a mail, sorted by priority."""
|
||||
rules = (
|
||||
(
|
||||
await db.execute(
|
||||
select(MailRule)
|
||||
.where(
|
||||
and_(
|
||||
MailRule.tenant_id == tenant_id,
|
||||
MailRule.is_active,
|
||||
or_(
|
||||
MailRule.account_id == mail.account_id,
|
||||
MailRule.account_id.is_(None),
|
||||
),
|
||||
)
|
||||
)
|
||||
.order_by(MailRule.priority)
|
||||
)
|
||||
)
|
||||
.scalars()
|
||||
.all()
|
||||
)
|
||||
|
||||
applied = []
|
||||
for rule in rules:
|
||||
conditions = json.loads(rule.conditions) if rule.conditions else {}
|
||||
actions = json.loads(rule.actions) if rule.actions else {}
|
||||
if matches_condition(mail, conditions):
|
||||
result = await execute_rule_actions(db, mail, actions, tenant_id)
|
||||
applied.append({"rule_id": str(rule.id), "rule_name": rule.name, "actions": result})
|
||||
return applied
|
||||
|
||||
|
||||
# ─── Vacation Auto-Reply (F-MAIL-08) ───
|
||||
|
||||
|
||||
VACATION_DEDUP_HOURS = 24
|
||||
|
||||
|
||||
async def should_send_vacation_reply(
|
||||
db: AsyncSession,
|
||||
account_id: uuid.UUID,
|
||||
sender_address: str,
|
||||
tenant_id: uuid.UUID,
|
||||
) -> bool:
|
||||
"""Check if vacation auto-reply should be sent (dedup within 24h)."""
|
||||
cutoff = datetime.now(UTC) - timedelta(hours=VACATION_DEDUP_HOURS)
|
||||
existing = (
|
||||
await db.execute(
|
||||
select(VacationSentLog).where(
|
||||
and_(
|
||||
VacationSentLog.account_id == account_id,
|
||||
VacationSentLog.sender_address == sender_address,
|
||||
VacationSentLog.sent_at >= cutoff,
|
||||
VacationSentLog.tenant_id == tenant_id,
|
||||
)
|
||||
)
|
||||
)
|
||||
).scalar_one_or_none()
|
||||
return existing is None
|
||||
|
||||
|
||||
async def log_vacation_sent(
|
||||
db: AsyncSession,
|
||||
account_id: uuid.UUID,
|
||||
sender_address: str,
|
||||
tenant_id: uuid.UUID,
|
||||
) -> None:
|
||||
"""Log that a vacation auto-reply was sent to a sender."""
|
||||
log = VacationSentLog(
|
||||
tenant_id=tenant_id,
|
||||
account_id=account_id,
|
||||
sender_address=sender_address,
|
||||
sent_at=datetime.now(UTC),
|
||||
)
|
||||
db.add(log)
|
||||
await db.flush()
|
||||
|
||||
|
||||
@@ -0,0 +1,62 @@
|
||||
"""HTML sanitization for the Mail plugin using nh3.
|
||||
|
||||
Extracted from services.py as part of the God-object split (BUG-018 pilot).
|
||||
Re-exported by ``app.plugins.builtins.mail.services``.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import nh3
|
||||
|
||||
|
||||
def sanitize_html(raw_html: str) -> str:
|
||||
"""Sanitize HTML using nh3 — removes script tags and dangerous attributes."""
|
||||
if not raw_html:
|
||||
return ""
|
||||
return nh3.clean(
|
||||
raw_html,
|
||||
tags={
|
||||
"a",
|
||||
"b",
|
||||
"br",
|
||||
"div",
|
||||
"em",
|
||||
"h1",
|
||||
"h2",
|
||||
"h3",
|
||||
"h4",
|
||||
"h5",
|
||||
"h6",
|
||||
"hr",
|
||||
"i",
|
||||
"img",
|
||||
"li",
|
||||
"ol",
|
||||
"p",
|
||||
"span",
|
||||
"strong",
|
||||
"table",
|
||||
"tbody",
|
||||
"td",
|
||||
"th",
|
||||
"thead",
|
||||
"tr",
|
||||
"u",
|
||||
"ul",
|
||||
"blockquote",
|
||||
"code",
|
||||
"pre",
|
||||
"font",
|
||||
"center",
|
||||
},
|
||||
attributes={
|
||||
"a": {"href", "title", "target"},
|
||||
"img": {"src", "alt", "width", "height"},
|
||||
"span": {"style"},
|
||||
"div": {"style"},
|
||||
"font": {"color", "size", "face"},
|
||||
"p": {"style"},
|
||||
"td": {"style"},
|
||||
"th": {"style"},
|
||||
},
|
||||
)
|
||||
@@ -259,6 +259,12 @@ class MailSignatureResponse(BaseModel):
|
||||
is_default: bool
|
||||
|
||||
|
||||
class MailSignatureUpdate(BaseModel):
|
||||
name: str = Field(..., min_length=1, max_length=255)
|
||||
body_html: str = ""
|
||||
is_default: bool = False
|
||||
|
||||
|
||||
# ─── Rules ───
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,169 @@
|
||||
"""Response serializers for the Mail plugin ORM models.
|
||||
|
||||
Extracted from services.py as part of the God-object split (BUG-018 pilot).
|
||||
Re-exported by ``app.plugins.builtins.mail.services``.
|
||||
|
||||
account_to_response NEVER includes passwords — security-critical contract.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
|
||||
from app.plugins.builtins.mail.models import (
|
||||
Mail,
|
||||
MailAccount,
|
||||
MailAttachment,
|
||||
MailFolder,
|
||||
MailLabel,
|
||||
MailRule,
|
||||
MailSignature,
|
||||
MailTemplate,
|
||||
)
|
||||
from app.plugins.builtins.mail.text_utils import (
|
||||
extract_email_addresses, # noqa: F401 — re-exported for backwards compat
|
||||
)
|
||||
|
||||
|
||||
def attachment_to_response(att: MailAttachment) -> dict:
|
||||
"""Convert a MailAttachment ORM object to a response dict."""
|
||||
return {
|
||||
"id": str(att.id),
|
||||
"mail_id": str(att.mail_id),
|
||||
"filename": att.filename,
|
||||
"mime_type": att.mime_type,
|
||||
"size_bytes": att.size_bytes,
|
||||
"size": att.size_bytes, # alias for frontend compatibility
|
||||
"content_id": att.content_id,
|
||||
"is_inline": bool(att.content_id),
|
||||
"dms_file_id": str(att.dms_file_id) if att.dms_file_id else None,
|
||||
}
|
||||
|
||||
|
||||
def account_to_response(account: MailAccount) -> dict:
|
||||
"""Convert MailAccount to response dict, NEVER including password."""
|
||||
return {
|
||||
"id": str(account.id),
|
||||
"email": account.email_address,
|
||||
"email_address": account.email_address,
|
||||
"display_name": account.display_name,
|
||||
"imap_host": account.imap_host,
|
||||
"imap_port": account.imap_port,
|
||||
"imap_ssl": account.imap_ssl,
|
||||
"smtp_host": account.smtp_host,
|
||||
"smtp_port": account.smtp_port,
|
||||
"smtp_tls": account.smtp_tls,
|
||||
"username": account.username,
|
||||
"is_shared": account.is_shared,
|
||||
"is_active": account.is_active,
|
||||
"sent_folder_imap_name": account.sent_folder_imap_name,
|
||||
"drafts_folder_imap_name": account.drafts_folder_imap_name,
|
||||
"spam_folder_imap_name": account.spam_folder_imap_name,
|
||||
"trash_folder_imap_name": account.trash_folder_imap_name,
|
||||
"created_at": account.created_at,
|
||||
"updated_at": account.updated_at,
|
||||
}
|
||||
|
||||
|
||||
def mail_to_response(
|
||||
mail: Mail,
|
||||
attachments: list[MailAttachment] | None = None,
|
||||
labels: list[MailLabel] | None = None,
|
||||
) -> dict:
|
||||
"""Convert a Mail ORM object to a response dict."""
|
||||
resp = {
|
||||
"id": str(mail.id),
|
||||
"account_id": str(mail.account_id),
|
||||
"folder_id": str(mail.folder_id),
|
||||
"message_id": mail.message_id,
|
||||
"thread_id": mail.thread_id,
|
||||
"in_reply_to": mail.in_reply_to,
|
||||
"subject": mail.subject,
|
||||
"from_address": mail.from_address,
|
||||
"from_name": mail.from_address.split("<")[0].strip().strip('"') if "<" in mail.from_address else mail.from_address,
|
||||
"to_addresses": extract_email_addresses(mail.to_addresses) if mail.to_addresses else [],
|
||||
"cc_addresses": extract_email_addresses(mail.cc_addresses) if mail.cc_addresses else [],
|
||||
"bcc_addresses": extract_email_addresses(mail.bcc_addresses) if mail.bcc_addresses else [],
|
||||
"body_text": mail.body_text,
|
||||
"body_html": mail.body_html if mail.body_html else None,
|
||||
"body_html_sanitized": mail.body_html_sanitized,
|
||||
"sanitized_html": mail.body_html_sanitized,
|
||||
"date": mail.received_at.isoformat() if mail.received_at else (mail.sent_at.isoformat() if mail.sent_at else None),
|
||||
"is_seen": mail.is_seen,
|
||||
"is_flagged": mail.is_flagged,
|
||||
"flag_type": getattr(mail, 'flag_type', None),
|
||||
"is_draft": mail.is_draft,
|
||||
"is_answered": mail.is_answered,
|
||||
"is_forwarded": mail.is_forwarded,
|
||||
"has_attachments": mail.has_attachments,
|
||||
"size_bytes": mail.size_bytes,
|
||||
"received_at": mail.received_at,
|
||||
"sent_at": mail.sent_at,
|
||||
"contact_id": str(mail.contact_id) if mail.contact_id else None,
|
||||
"attachments": [],
|
||||
"labels": [],
|
||||
}
|
||||
if attachments:
|
||||
resp["attachments"] = [attachment_to_response(a) for a in attachments]
|
||||
if labels:
|
||||
resp["labels"] = [
|
||||
{"id": str(lbl.id), "name": lbl.name, "color": lbl.color} for lbl in labels
|
||||
]
|
||||
return resp
|
||||
|
||||
|
||||
def folder_to_response(folder: MailFolder) -> dict:
|
||||
"""Convert MailFolder to response dict."""
|
||||
return {
|
||||
"id": str(folder.id),
|
||||
"account_id": str(folder.account_id),
|
||||
"name": folder.name,
|
||||
"imap_name": folder.imap_name,
|
||||
"parent_id": str(folder.parent_id) if folder.parent_id else None,
|
||||
"is_standard": folder.is_standard,
|
||||
"unread_count": folder.unread_count,
|
||||
"total_count": folder.total_count,
|
||||
}
|
||||
|
||||
|
||||
def rule_to_response(rule: MailRule) -> dict:
|
||||
"""Convert MailRule to response dict."""
|
||||
return {
|
||||
"id": str(rule.id),
|
||||
"name": rule.name,
|
||||
"account_id": str(rule.account_id) if rule.account_id else None,
|
||||
"priority": rule.priority,
|
||||
"is_active": rule.is_active,
|
||||
"conditions": json.loads(rule.conditions) if rule.conditions else {},
|
||||
"actions": json.loads(rule.actions) if rule.actions else {},
|
||||
}
|
||||
|
||||
|
||||
def template_to_response(template: MailTemplate) -> dict:
|
||||
"""Convert MailTemplate to response dict."""
|
||||
return {
|
||||
"id": str(template.id),
|
||||
"name": template.name,
|
||||
"subject": template.subject,
|
||||
"body_html": template.body_html,
|
||||
}
|
||||
|
||||
|
||||
def signature_to_response(sig: MailSignature) -> dict:
|
||||
"""Convert MailSignature to response dict."""
|
||||
return {
|
||||
"id": str(sig.id),
|
||||
"name": sig.name,
|
||||
"body_html": sig.body_html,
|
||||
"account_id": str(sig.account_id) if sig.account_id else None,
|
||||
"is_default": sig.is_default,
|
||||
}
|
||||
|
||||
|
||||
def label_to_response(label: MailLabel) -> dict:
|
||||
"""Convert MailLabel to response dict."""
|
||||
return {
|
||||
"id": str(label.id),
|
||||
"name": label.name,
|
||||
"color": label.color,
|
||||
}
|
||||
+150
-3066
File diff suppressed because it is too large
Load Diff
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user