Compare commits
97 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 4fee01cadf | |||
| ea6c9e71db | |||
| c34715574a | |||
| fce17aac9c | |||
| cbe36e0c0e | |||
| 6702d69f7c | |||
| 94d8c40daa | |||
| be81fe52cf | |||
| a1d5e56009 | |||
| 3e43219b84 | |||
| 26b5ae9a0d | |||
| 11e4e42570 | |||
| 57441df677 | |||
| fbe1bde635 | |||
| 2d17746194 | |||
| 23a05593b2 | |||
| 0baec2792c | |||
| f4a5937a4b | |||
| 38b73f5d4d | |||
| a6bfa8e67c | |||
| a8916b3d86 | |||
| e7afbaa906 | |||
| 34c9c85aed | |||
| 9d2df61942 | |||
| 1b485d4a34 | |||
| f4c4a50ebd | |||
| 69d05d6912 | |||
| df9f86bd12 | |||
| 9e1d202610 | |||
| c291a6ecf1 | |||
| ab3c253cbd | |||
| 52323610e3 | |||
| 86c96f03ca | |||
| 3e5f13f516 | |||
| 4de629d296 | |||
| 6a88c70073 | |||
| c807aacfc0 | |||
| b23045c46a | |||
| 5d8c48a08f | |||
| f6dde68221 | |||
| d901d001c7 | |||
| 962e0ee1f6 | |||
| 49ca4c5fb2 | |||
| 1b22da8b0d | |||
| 76a31a8c39 | |||
| a991f9a0b4 | |||
| 84a30d85c2 | |||
| d9aed519f2 | |||
| b9a6c06e85 | |||
| 8386e99caa | |||
| 7d9ae03bf1 | |||
| 36a03b9897 | |||
| 860db8d61e | |||
| 81aea8c77f | |||
| 46c909c226 | |||
| 197b0d3bab | |||
| 3934aea6ef | |||
| 5cc5a3fa6a | |||
| c0e8e4ecfd | |||
| c32e4bb34e | |||
| ef90d57f0a | |||
| 0768cfb29a | |||
| 56e401969e | |||
| 6d04206695 | |||
| f6e117b1c3 | |||
| 9d8da99026 | |||
| 54066b05fd | |||
| 36636f5c25 | |||
| d89044d8f7 | |||
| 5e0ffd91c2 | |||
| b8b8ef180a | |||
| cad7d084e8 | |||
| dff97f5589 | |||
| 9e84c400ed | |||
| 067fc132cb | |||
| b01b756a4a | |||
| 4bce89aecb | |||
| 5e9be254e2 | |||
| 8a76bfdba4 | |||
| d2434203c1 | |||
| ad7c763e59 | |||
| e3fb4728d7 | |||
| 7467c01d38 | |||
| 4038b74025 | |||
| 5ad107ff83 | |||
| 5cee78c54c | |||
| 32f63adc09 | |||
| c21634b323 | |||
| 73d2e109cd | |||
| 795307754f | |||
| 17516d2783 | |||
| ed8ee5cda1 | |||
| 90a367089d | |||
| b04cda774b | |||
| 982b4c9353 | |||
| 1d6152fb82 | |||
| 337d78ef53 |
+2
-2
@@ -39,8 +39,8 @@ RUN apt-get update \
|
|||||||
|
|
||||||
WORKDIR /app
|
WORKDIR /app
|
||||||
|
|
||||||
COPY requirements.txt .
|
COPY requirements.txt requirements.lock ./
|
||||||
RUN pip install --user --no-cache-dir -r requirements.txt
|
RUN pip install --user --no-cache-dir -r requirements.lock
|
||||||
|
|
||||||
# === Stage 2: Runtime ===
|
# === Stage 2: Runtime ===
|
||||||
FROM python:3.12-slim AS runtime
|
FROM python:3.12-slim AS runtime
|
||||||
|
|||||||
@@ -1210,6 +1210,370 @@ Trigger / Event / Cron / Webhook / Agent
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
## Phase L — UI-Overhaul (Status: geplant, NICHT gestartet)
|
||||||
|
|
||||||
|
> **Herkunft:** Am 2026-08-25 aus der eigenständigen Datei `UI_OVERHAUL_PLAN.md`
|
||||||
|
> hier integriert - gemaess AGENTS.md-Regel "PLATFORM_ROADMAP.md ist EINZIGE
|
||||||
|
> Planungs-Datei". Vollständiges Original inkl. ASCII-Mockups abrufbar via
|
||||||
|
> `git show c807aac:UI_OVERHAUL_PLAN.md`.
|
||||||
|
>
|
||||||
|
> **Konflikt-Notiz (2026-08-25, Block I-D):** Phase 2 unten sieht "AI Assistant
|
||||||
|
> Page entfernen" vor. Die Seite wurde jedoch in Commit 962e0ee bewusst GEBAUT,
|
||||||
|
> um die Geister-Route /ai-assistant zu reparieren (im Backend-Manifest
|
||||||
|
> referenziert, aber nicht existent -> ErrorBoundary in Production). VOR
|
||||||
|
> Umsetzung von Phase 2 neu entscheiden: (a) Seite doch entfernen - dann auch
|
||||||
|
> Manifest-Route entfernen, oder (b) Phase 2 verwerfen zugunsten der aktuellen
|
||||||
|
> Architektur. Bitte nicht unkommentiert ausfuehren.
|
||||||
|
|
||||||
|
> **Erstellt:** 2026-08-21
|
||||||
|
> **Aktualisiert:** 2026-08-21 — AI Assistent Integration hinzugefügt
|
||||||
|
> **Status:** Planung — nicht gestartet
|
||||||
|
> **Leitlinie:** Auf bestehendem Code aufbauen, 3-Spalten-Explorer-Layout als Standard, keine parallelen Systeme
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Standard-Layout (Referenz: ContactsList.tsx)
|
||||||
|
|
||||||
|
Alle Explorer-Plugins nutzen das 3-Spalten-Layout aus den UI-Design-Guidelines:
|
||||||
|
|
||||||
|
```
|
||||||
|
┌─────────────┬──────────────────┬──────────────────────┐
|
||||||
|
│ Tree │ Liste/Ansicht │ Detail │
|
||||||
|
│ (224px) │ (flex-1) │ (flex-1 / 60%) │
|
||||||
|
│ ResizablePanel│ ResizablePanel │ ResizablePanel │
|
||||||
|
└─────────────┴──────────────────┴──────────────────────┘
|
||||||
|
```
|
||||||
|
|
||||||
|
- **Toolbar oben:** PluginToolbar mit Filter-Dropdowns, Ansichts-Umschaltern, Aktion-Buttons
|
||||||
|
- **Linke Spalte:** ResizablePanel mit Baumansicht (Ordner, Kategorien, Kalender)
|
||||||
|
- **Mitte:** Liste, Karten, Kalender-Ansicht — mehrere Ansichten umschaltbar
|
||||||
|
- **Rechts:** Detail-Bereich für ausgewähltes Element
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Phase 1: Echte Bugs fixen (2-3 Tage)
|
||||||
|
|
||||||
|
#### 1.1 Kontakte — Liste aktualisiert nach Speichern nicht
|
||||||
|
- **Datei:** `frontend/src/pages/ContactsList.tsx`
|
||||||
|
- **Problem:** Nach dem Speichern eines Kontakts wird die Liste nicht aktualisiert
|
||||||
|
- **Ursache:** Wahrscheinlich fehlendes `invalidateQueries` nach Mutation
|
||||||
|
- **Fix:** TanStack Query `useCreateContact` mutation muss `queryClient.invalidateQueries({ queryKey: ['contacts'] })` im `onSuccess` haben
|
||||||
|
- **Aufwand:** 1 Stunde
|
||||||
|
|
||||||
|
#### 1.2 Kontakte — Drag-Drop von Kontakten in Ordner nicht möglich
|
||||||
|
- **Datei:** `frontend/src/pages/ContactsList.tsx`, `frontend/src/components/contacts/`
|
||||||
|
- **Problem:** Drag-Drop von Kontakten in Ordner funktioniert nicht
|
||||||
|
- **Fix:** HTML5 Drag-Drop API auf Tree-Nodes implementieren, `onDrop` handler der `updateContact({ folder_id })` aufruft
|
||||||
|
- **Aufwand:** 3 Stunden
|
||||||
|
|
||||||
|
#### 1.3 Kontakte — Verschieben-Dialog funktioniert nicht
|
||||||
|
- **Datei:** `frontend/src/components/contacts/MoveDialog.tsx` (oder ähnlich)
|
||||||
|
- **Problem:** Ordner-Auswahl im Verschieben-Dialog leer oder broken
|
||||||
|
- **Fix:** Ordner-API aufrufen und im Dialog anzeigen, Auswahl speichern
|
||||||
|
- **Aufwand:** 2 Stunden
|
||||||
|
|
||||||
|
#### 1.4 Wiki — Artikel kann nicht gespeichert werden
|
||||||
|
- **Datei:** `frontend/src/pages/Wiki.tsx`, `frontend/src/api/knowledge.ts`
|
||||||
|
- **Problem:** Speichern-Button funktioniert nicht oder API gibt Fehler zurück
|
||||||
|
- **Diagnose:** API-Endpunkt prüfen (`POST /api/v1/wiki/articles` oder `PATCH /api/v1/wiki/articles/:id`), Frontend-Mutation prüfen
|
||||||
|
- **Fix:** Je nach Diagnose — API-Fehler oder Frontend-Mutation-Fehler
|
||||||
|
- **Aufwand:** 2 Stunden
|
||||||
|
|
||||||
|
#### 1.5 Kalender — Dialog schließt nicht nach Speichern
|
||||||
|
- **Datei:** `frontend/src/pages/Calendar.tsx`, `frontend/src/components/calendar/AppointmentEditForm.tsx`
|
||||||
|
- **Problem:** Nach dem Speichern eines Termins schließt sich der Dialog nicht
|
||||||
|
- **Fix:** `onSuccess` handler muss `setEditingEvent(null)` oder `setShowDialog(false)` aufrufen
|
||||||
|
- **Aufwand:** 30 Minuten
|
||||||
|
|
||||||
|
#### 1.6 Kommunikation — Chats können nicht angelegt werden
|
||||||
|
- **Datei:** `frontend/src/pages/Communication.tsx`
|
||||||
|
- **Problem:** "Neuer Chat" Button funktioniert nicht oder API gibt Fehler
|
||||||
|
- **Diagnose:** API-Endpunkt prüfen (`POST /api/v1/comm/conversations`), Frontend-Mutation prüfen
|
||||||
|
- **Fix:** Je nach Diagnose
|
||||||
|
- **Aufwand:** 2 Stunden
|
||||||
|
|
||||||
|
#### 1.7 Wiki — Doppelt im Menü
|
||||||
|
- **Datei:** `frontend/src/routes/index.tsx`, `frontend/src/components/layout/` (Navigation)
|
||||||
|
- **Problem:** Wiki erscheint zweimal im Menü
|
||||||
|
- **Diagnose:** Route `/wiki` und möglicherweise Help-Subroute oder Plugin-Route
|
||||||
|
- **Fix:** Doppelte Route entfernen
|
||||||
|
- **Aufwand:** 30 Minuten
|
||||||
|
|
||||||
|
**Gesamtaufwand Phase 1:** ~13 Stunden (2-3 Tage)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Phase 2: AI Assistent in Kommunikation integrieren (2-3 Tage)
|
||||||
|
|
||||||
|
#### Problem
|
||||||
|
Der AI Assistent ist ein paralleles System das die Kommunikation-Plattform dupliziert:
|
||||||
|
- **AI Assistant Tabellen:** `ai_conversations`, `ai_messages` (app/models/ai_conversation.py) + `ai_chat_sessions`, `ai_chat_messages`, `ai_chat_attachments` (app/plugins/builtins/ai_assistant/models.py) — 5 Tabellen
|
||||||
|
- **AI Assistant Frontend:** `AIAssistant.tsx`, `AIAssistantStandalone.tsx`, `SessionList.tsx`, `ChatWindow.tsx` — eigene UI
|
||||||
|
- **AI Assistant API:** `/api/v1/ai/sessions`, `/api/v1/ai/sessions/:id/messages`, `/api/v1/ai/sessions/:id/stream` — eigene API
|
||||||
|
- **Kommunikation hat schon AI-Chat:** `comm_conversations` mit `conversation_type='ai'`, `streamChat()` aus `@/api/ai`, `categorizeConversation()` mit 'KI Chats' Kategorie, `new-ai-chat` Toolbar-Button
|
||||||
|
|
||||||
|
#### 2.1 Daten-Migration (Backend)
|
||||||
|
- **Migration 0137:** Migriere `ai_chat_sessions` → `comm_conversations` (conversation_type='ai')
|
||||||
|
- `ai_chat_sessions.id` → `comm_conversations.id`
|
||||||
|
- `ai_chat_sessions.title` → `comm_conversations.title`
|
||||||
|
- `ai_chat_sessions.tenant_id` → `comm_conversations.tenant_id`
|
||||||
|
- `ai_chat_sessions.user_id` → `comm_conversations.owner_id`
|
||||||
|
- `ai_chat_sessions.agent_id` → `comm_conversations.metadata.agent_id`
|
||||||
|
- `ai_chat_sessions.created_at` → `comm_conversations.created_at`
|
||||||
|
- **Migration 0137:** Migriere `ai_chat_messages` → `comm_messages`
|
||||||
|
- `ai_chat_messages.id` → `comm_messages.id`
|
||||||
|
- `ai_chat_messages.session_id` → `comm_messages.conversation_id`
|
||||||
|
- `ai_chat_messages.role` → `comm_messages.sender_type` ('user' → 'user', 'assistant' → 'ai')
|
||||||
|
- `ai_chat_messages.content` → `comm_messages.content`
|
||||||
|
- `ai_chat_messages.tenant_id` → `comm_messages.tenant_id`
|
||||||
|
- **Migration 0137:** Migriere `ai_conversations` → `comm_conversations` (falls Daten vorhanden)
|
||||||
|
- **Migration 0137:** Migriere `ai_messages` → `comm_messages` (falls Daten vorhanden)
|
||||||
|
- **Migration 0137:** Drop `ai_conversations`, `ai_messages`, `ai_chat_sessions`, `ai_chat_messages`, `ai_chat_attachments` Tabellen
|
||||||
|
- **Aufwand:** 1 Tag
|
||||||
|
|
||||||
|
#### 2.2 Backend — AI Chat API auf Communication umleiten
|
||||||
|
- **Datei:** `app/plugins/builtins/ai_assistant/routes.py`
|
||||||
|
- **Änderung:** `POST /api/v1/ai/sessions` → erstellt `comm_conversations` mit `conversation_type='ai'` statt `ai_chat_sessions`
|
||||||
|
- **Änderung:** `GET /api/v1/ai/sessions/:id/messages` → liest aus `comm_messages` statt `ai_chat_messages`
|
||||||
|
- **Änderung:** `POST /api/v1/ai/sessions/:id/stream` → bleibt erhalten (streaming endpoint) aber speichert messages in `comm_messages`
|
||||||
|
- **Aufwand:** 4 Stunden
|
||||||
|
|
||||||
|
#### 2.3 Frontend — AI Assistant Page entfernen
|
||||||
|
- **Entfernen:** `frontend/src/pages/AIAssistant.tsx`
|
||||||
|
- **Entfernen:** `frontend/src/pages/AIAssistantStandalone.tsx`
|
||||||
|
- **Entfernen:** `frontend/src/components/ai/SessionList.tsx`
|
||||||
|
- **Entfernen:** `frontend/src/components/ai/ChatWindow.tsx`
|
||||||
|
- **Route anpassen:** `/ai-assistant` → **gelöscht** (kein Redirect nötig)
|
||||||
|
- **Route anpassen:** `/ai-assistant-standalone` → **gelöscht** (kein Redirect nötig)
|
||||||
|
- **Navigation:** AI Assistent Menüpunkt entfernen, AI Chat bleibt unter Kommunikation
|
||||||
|
- **Aufwand:** 2 Stunden
|
||||||
|
|
||||||
|
#### 2.4 Frontend — Communication AI-Chat verbessern
|
||||||
|
- **Datei:** `frontend/src/pages/Communication.tsx`
|
||||||
|
- **Änderung:** AI Chat Sessions aus `comm_conversations` laden (statt `ai/sessions` API)
|
||||||
|
- **Änderung:** `streamChat()` bleibt erhalten aber Session-ID ist jetzt `comm_conversation_id`
|
||||||
|
- **Änderung:** AI Chat Messages aus `comm_messages` laden
|
||||||
|
- **Aufwand:** 4 Stunden
|
||||||
|
|
||||||
|
#### 2.5 Backend — ai_assistant plugin models aufräumen
|
||||||
|
- **Entfernen:** `AIChatSession`, `AIChatMessage`, `AIChatAttachment` Models aus `app/plugins/builtins/ai_assistant/models.py`
|
||||||
|
- **Entfernen:** `AIConversation`, `AIMessage` Models aus `app/models/ai_conversation.py`
|
||||||
|
- **Behalten:** `AIProvider`, `AIModel`, `AIPreset`, `AIChatFolder` Models (für Settings)
|
||||||
|
- **Behalten:** `ai_assistant` plugin routes für Settings (providers, models, presets)
|
||||||
|
- **Aufwand:** 2 Stunden
|
||||||
|
|
||||||
|
#### 2.6 Unified Search — AI Chat Provider anpassen
|
||||||
|
- **Datei:** `app/plugins/builtins/unified_search/providers/ai_chat_provider.py`
|
||||||
|
- **Änderung:** Search auf `comm_messages` (conversation_type='ai') statt `ai_chat_messages`
|
||||||
|
- **Aufwand:** 1 Stunde
|
||||||
|
|
||||||
|
**Gesamtaufwand Phase 2:** ~2-3 Tage
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Phase 3: Wiki UI-Überarbeitung (3-4 Tage)
|
||||||
|
|
||||||
|
#### 3.1 WYSIWYG Editor
|
||||||
|
- **Datei:** `frontend/src/components/wiki/WikiEditor.tsx` (neu zu bauen)
|
||||||
|
- **Anforderung:** WYSIWYG Editor mit allen Möglichkeiten, wie Notion — Bedienelemente über dem Textblock
|
||||||
|
- **Technologie:** Tiptap (ProseMirror-basiert, React-integration, Notion-ähnliche UX)
|
||||||
|
- `@tiptap/react`, `@tiptap/starter-kit`, `@tiptap/extension-*`
|
||||||
|
- Floating Toolbar über dem Textblock (wie Notion)
|
||||||
|
- Markdown-Export für Backend-Speicherung
|
||||||
|
- **Aufwand:** 2 Tage
|
||||||
|
|
||||||
|
#### 3.2 Wiki Layout — 3-Spalten
|
||||||
|
- **Datei:** `frontend/src/pages/Wiki.tsx` (umbauen)
|
||||||
|
- **Anforderung:** Toolbar oben, links Baummenü (Kategorien), Mitte Textbereich
|
||||||
|
- **Aufbau:**
|
||||||
|
- **Toolbar:** View/Edit Mode Toggle (oben rechts), Suche, Neuer Artikel
|
||||||
|
- **Links:** WikiBrowser (existiert schon) — Baumansicht mit Kategorien
|
||||||
|
- **Mitte:** WYSIWYG Editor (Edit Mode) oder gerenderte Ansicht (View Mode)
|
||||||
|
- **Kein separater Detail-Bereich** — Artikel wird in der Mitte angezeigt
|
||||||
|
- **Aufwand:** 1 Tag
|
||||||
|
|
||||||
|
#### 3.3 View/Edit Mode Toggle
|
||||||
|
- **Datei:** `frontend/src/pages/Wiki.tsx`
|
||||||
|
- **Anforderung:** Button oben rechts in der Toolbar der zwischen View und Edit Mode wechselt
|
||||||
|
- **Im Edit Mode:** WYSIWYG Editor mit Floating Toolbar
|
||||||
|
- **Im View Mode:** Gerenderte Markdown-Ansicht (wie jetzt, aber schöner)
|
||||||
|
- **Aufwand:** 2 Stunden
|
||||||
|
|
||||||
|
**Gesamtaufwand Phase 3:** ~3-4 Tage
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Phase 4: Tasks UI-Überarbeitung (2-3 Tage)
|
||||||
|
|
||||||
|
#### 4.1 Tasks Layout — 3-Spalten wie Kontakte
|
||||||
|
- **Datei:** `frontend/src/pages/Tasks.tsx` (kompletter Umbau, 419 → ~600 Zeilen)
|
||||||
|
- **Anforderung:** Linke Sidebar Baumansicht, Mitte Liste mit mehreren Ansichten, rechts Detailbereich
|
||||||
|
- **Aufbau:**
|
||||||
|
- **Toolbar:** PluginToolbar mit Filter-Dropdowns (Status, Priorität, Zuweisung, Fällig), Ansichts-Umschalter (Liste/Kanban), Neuer Task
|
||||||
|
- **Links:** Baumansicht — nach Status (Offen/In Bearbeitung/Erledigt), nach Priorität, nach Zuweisung, nach Liste/Goal
|
||||||
|
- **Mitte:** Liste (Tabelle) oder Kanban-Board — umschaltbar
|
||||||
|
- **Rechts:** TaskDetail — ausgewählter Task mit Beschreibung, Subtasks, Zuweisung, Fälligkeit
|
||||||
|
- **Aufwand:** 2-3 Tage
|
||||||
|
|
||||||
|
**Gesamtaufwand Phase 4:** ~2-3 Tage
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Phase 5: Kalender UI-Überarbeitung (1 Tag)
|
||||||
|
|
||||||
|
#### 5.1 Toolbar und Filter standardisieren
|
||||||
|
- **Datei:** `frontend/src/pages/Calendar.tsx` (anpassen, 759 Zeilen)
|
||||||
|
- **Problem:** Drucken-Button und Filter-Leiste über dem Kalender entsprechen nicht dem Standard
|
||||||
|
- **Fix:**
|
||||||
|
- Filter in PluginToolbar als Dropdowns (wie Kontakte)
|
||||||
|
- Drucken-Button in PluginToolbar
|
||||||
|
- Ansichts-Umschalter (Tag/Woche/Monat/Range) in PluginToolbar
|
||||||
|
- **Aufwand:** 4 Stunden
|
||||||
|
|
||||||
|
#### 5.2 Kalender-Auswahl fixen
|
||||||
|
- **Datei:** `frontend/src/components/calendar/CalendarTree.tsx`
|
||||||
|
- **Problem:** Einzelnes An- und Abwählen von Kalendern funktioniert nicht richtig
|
||||||
|
- **Fix:** Checkbox-Toggle Logik reparieren — `visibleCalendars` Set korrekt verwalten
|
||||||
|
- **Aufwand:** 2 Stunden
|
||||||
|
|
||||||
|
**Gesamtaufwand Phase 5:** ~1 Tag
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Phase 6: Tags Umstrukturierung (2 Tage)
|
||||||
|
|
||||||
|
#### 6.1 Tags in Settings verschieben
|
||||||
|
- **Datei:** `frontend/src/pages/Tags.tsx` → `frontend/src/pages/SettingsTags.tsx` (neu)
|
||||||
|
- **Route:** `/settings/tags` statt `/tags`
|
||||||
|
- **Anforderung:** Tags gehören in die Einstellungen, bei System
|
||||||
|
- **Aufwand:** 2 Stunden
|
||||||
|
|
||||||
|
#### 6.2 Tags Baumstruktur
|
||||||
|
- **Datei:** `frontend/src/pages/SettingsTags.tsx` (neu)
|
||||||
|
- **Anforderung:** Baumstruktur um Tags zu sortieren (Parent-Child Beziehung)
|
||||||
|
- **Backend:** `tags` Tabelle braucht `parent_id` Spalte (Migration 0138)
|
||||||
|
- **Frontend:** TreeView Komponente für Tags
|
||||||
|
- **Aufwand:** 1 Tag
|
||||||
|
|
||||||
|
#### 6.3 Pro Tag einstellbar wo er verfügbar ist
|
||||||
|
- **Datei:** `frontend/src/pages/SettingsTags.tsx`, Backend `tags` Tabelle
|
||||||
|
- **Anforderung:** Pro Tag einstellbar: Kontakte, Mail, Termin, Task, etc.
|
||||||
|
- **Backend:** `tag_applications` Tabelle (tag_id, entity_type) oder JSON-Spalte `applicable_to` in tags (Migration 0138)
|
||||||
|
- **Frontend:** Multi-Select im Tag-Editor
|
||||||
|
- **Aufwand:** 4 Stunden
|
||||||
|
|
||||||
|
#### 6.4 Symbol und Farbe pro Tag
|
||||||
|
- **Datei:** `frontend/src/pages/SettingsTags.tsx`, Backend `tags` Tabelle
|
||||||
|
- **Anforderung:** Symbol (Icon) und Farbe pro Tag einstellbar
|
||||||
|
- **Backend:** `icon` Spalte in tags (Migration 0138), `color` existiert schon
|
||||||
|
- **Frontend:** Icon-Picker und Color-Picker im Tag-Editor
|
||||||
|
- **Aufwand:** 4 Stunden
|
||||||
|
|
||||||
|
**Gesamtaufwand Phase 6:** ~2 Tage
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Phase 7: Reports UI-Überarbeitung (2 Tage)
|
||||||
|
|
||||||
|
#### 7.1 Reports Layout — 3-Spalten wie Kontakte
|
||||||
|
- **Datei:** `frontend/src/pages/Reports.tsx` (Umbau, 433 Zeilen)
|
||||||
|
- **Anforderung:** Linke Sidebar mit Baumstruktur (Ordner zum Sortieren), Mitte verschiedene Ansichten (Liste/Karten), rechts Detailbereich
|
||||||
|
- **Aufbau:**
|
||||||
|
- **Toolbar:** PluginToolbar mit Filter, Ansichts-Umschalter, Neuer Report
|
||||||
|
- **Links:** Baumansicht — nach Ordner/Gruppe sortierbar
|
||||||
|
- **Mitte:** Liste oder Karten-Ansicht — umschaltbar
|
||||||
|
- **Rechts:** ReportDetail — ausgewählter Report mit Vorschau
|
||||||
|
- **Backend:** `reports` Tabelle braucht `folder_id` Spalte (Migration 0139) für Ordner-Sortierung
|
||||||
|
- **Aufwand:** 2 Tage
|
||||||
|
|
||||||
|
**Gesamtaufwand Phase 7:** ~2 Tage
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Phase 8: Kommunikation UI-Überarbeitung (2-3 Tage)
|
||||||
|
|
||||||
|
#### 8.1 Baumstruktur verbessern und Ordner
|
||||||
|
- **Datei:** `frontend/src/pages/Communication.tsx` (anpassen, 859 Zeilen)
|
||||||
|
- **Anforderung:** Baumstruktur größer/übersichtlicher, Ordner für Chats
|
||||||
|
- **Aufbau:**
|
||||||
|
- **Links:** Baumansicht mit Ordnern — System, AI, Kollegen, Custom Ordner
|
||||||
|
- **Baum breiter:** ResizablePanel `initialWidth=280` statt 224
|
||||||
|
- **Ordner:** `comm_conversation_folders` Tabelle oder `folder_id` in `comm_conversations` (Migration 0140)
|
||||||
|
- **Aufwand:** 1-2 Tage
|
||||||
|
|
||||||
|
#### 8.2 AI Chat in Kommunikation (nach Phase 2)
|
||||||
|
- AI Chats werden als eigener Baum-Knoten 'KI Chats' in Communication angezeigt
|
||||||
|
- Neuer AI Chat Button in Toolbar erstellt `comm_conversation` mit `conversation_type='ai'`
|
||||||
|
- `streamChat()` wird aufgerufen mit `comm_conversation_id` als Session-ID
|
||||||
|
- AI Messages werden in `comm_messages` gespeichert
|
||||||
|
- **Aufwand:** in Phase 2
|
||||||
|
|
||||||
|
**Gesamtaufwand Phase 8:** ~1-2 Tage (Phase 2 vorab)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Phase 9: Strukturelle Änderungen (0.5 Tage)
|
||||||
|
|
||||||
|
#### 9.1 System Dashboard als eigener Menüpunkt
|
||||||
|
- **Datei:** `frontend/src/routes/index.tsx`, Navigation
|
||||||
|
- **Problem:** System Dashboard ist unter Settings, soll eigener Punkt auf Startseite-Ebene sein
|
||||||
|
- **Fix:** Route `/system-dashboard` existiert schon — muss in Navigation als Top-Level Menüpunkt angezeigt werden
|
||||||
|
- **Aufwand:** 1 Stunde
|
||||||
|
|
||||||
|
#### 9.2 Mail — Postfach mit IMAP anlegen testen
|
||||||
|
- **Datei:** `frontend/src/pages/Mail.tsx`, `frontend/src/pages/MailSettings.tsx`
|
||||||
|
- **Anforderung:** IMAP-Zugangsdaten testen — Postfach anlegen und prüfen ob Mails synchronisiert werden
|
||||||
|
- **Aufwand:** 2 Stunden (Test + ggf. Bugfix)
|
||||||
|
|
||||||
|
**Gesamtaufwand Phase 9:** ~0.5 Tage
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Phase-L-Phasenübersicht
|
||||||
|
|
||||||
|
| Phase | Inhalt | Aufwand | Migration | Abhängigkeit |
|
||||||
|
|-------|--------|---------|-----------|-------------|
|
||||||
|
| 1 | Echte Bugs fixen | 2-3 Tage | Keine | Keine |
|
||||||
|
| 2 | AI Assistent → Kommunikation | 2-3 Tage | 0137 | Phase 1.6 |
|
||||||
|
| 3 | Wiki UI + WYSIWYG | 3-4 Tage | Keine | Phase 1.4 |
|
||||||
|
| 4 | Tasks UI neu | 2-3 Tage | Keine | Keine |
|
||||||
|
| 5 | Kalender UI | 1 Tag | Keine | Phase 1.5 |
|
||||||
|
| 6 | Tags Umstrukturierung | 2 Tage | 0138 | Keine |
|
||||||
|
| 7 | Reports UI | 2 Tage | 0139 | Keine |
|
||||||
|
| 8 | Kommunikation UI | 1-2 Tage | 0140 | Phase 2 |
|
||||||
|
| 9 | Strukturelle Änderungen | 0.5 Tage | Keine | Keine |
|
||||||
|
|
||||||
|
**Gesamtaufwand:** ~17-22 Tage
|
||||||
|
|
||||||
|
#### Reihenfolge:
|
||||||
|
1. **Phase 1** (Bugs) — zuerst, damit grundlegende Funktionen arbeiten
|
||||||
|
2. **Phase 9** (Strukturelle Änderungen) — schnell, wenig Aufwand
|
||||||
|
3. **Phase 5** (Kalender) — kleines Update, baut auf Phase 1 auf
|
||||||
|
4. **Phase 2** (AI Assistent → Kommunikation) — entfernt paralleles System, baut auf Phase 1.6 auf
|
||||||
|
5. **Phase 6** (Tags) — unabhängig, Backend + Frontend
|
||||||
|
6. **Phase 4** (Tasks) — großer Umbau, unabhängig
|
||||||
|
7. **Phase 3** (Wiki) — größter Umbau (WYSIWYG Editor), baut auf Phase 1 auf
|
||||||
|
8. **Phase 7** (Reports) — großer Umbau, unabhängig
|
||||||
|
9. **Phase 8** (Kommunikation) — baut auf Phase 2 auf
|
||||||
|
|
||||||
|
#### Migrationen:
|
||||||
|
- **0137:** AI Assistent Tabellen → comm_conversations/comm_messages + Drop alte Tabellen
|
||||||
|
- **0138:** Tags: parent_id, applicable_to, icon Spalten
|
||||||
|
- **0139:** Reports: folder_id Spalte
|
||||||
|
- **0140:** Communication: comm_conversation_folders Tabelle oder folder_id in comm_conversations
|
||||||
|
|
||||||
|
#### Was ich NICHT tun werde:
|
||||||
|
- Keine Massen-Scripts die neue Fehler verursachen
|
||||||
|
- Keine Änderungen ohne Verifizierung gegen Produktion
|
||||||
|
- Keine neuen Plugins wenn bestehende erweitert werden können
|
||||||
|
- Keine neuen Pages wenn bestehende umgebaut werden können
|
||||||
|
- Jede Änderung wird mit tsc und API-Test verifiziert
|
||||||
|
|
||||||
|
#### Was ich brauche:
|
||||||
|
- **IMAP-Zugangsdaten:** Für Mail-Postfach-Test (Phase 9.2)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
## Zusammenfassung
|
## Zusammenfassung
|
||||||
|
|
||||||
| Phase | Dauer | Hauptdeliverable |
|
| Phase | Dauer | Hauptdeliverable |
|
||||||
|
|||||||
+150
@@ -6,6 +6,156 @@
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
## Architektur-Reparatur (2026-08-23, laufend)
|
||||||
|
|
||||||
|
**Plan:** docs/fix-plan-v3.md · **Sequenz:** Block 0 + Block H gemerged in main, Block A läuft auf main
|
||||||
|
**Stand:** 18 Findings geschlossen (~30% aufwandsgewichtet) · App startbar · 90+ Tests grün · tsc clean · Checker 14→6 Verstöße · Alles gepusht auf Forgejo
|
||||||
|
|
||||||
|
| Finding | Beschreibung | Status | Commit |
|
||||||
|
|---|---|---|---|
|
||||||
|
| SYNTAX-001 | automation/plugin.py SyntaxError — App startet nicht | ✅ gefixt | 8077595 |
|
||||||
|
| CHECK-002 | Checker crasht bei relativen Pfaden | ✅ gefixt | 35e2cc8 |
|
||||||
|
| ARCH-010 | Checker scannt nur builtins | ✅ Vollscan-Default | 35e2cc8 |
|
||||||
|
| ARCH-014 | Contract-Lazy-Resurrect nach unregister | ✅ gefixt + Funktionstest | b04cda7 |
|
||||||
|
| ARCH-020 | EventBus subscribe ohne Duplikat-Check | ✅ gefixt + Funktionstest | b04cda7 |
|
||||||
|
| ARCH-029/041 | trigger_dispatcher None-Check nach Verwendung | ✅ None-Check zuerst | b04cda7 |
|
||||||
|
| ARCH-001 | Permissions nach on_activate registriert | ✅ Reihenfolge gedreht | 1d6152f |
|
||||||
|
| ARCH-002 | on_activate pro Tenant mehrfach | ✅ 1× pro Prozess | 1d6152f |
|
||||||
|
| ARCH-003 | active-manifests an plugins:read gebunden | ✅ für eingeloggte User offen | 982b4c9 |
|
||||||
|
| ARCH-038 | BasePlugin.register_event_handlers fehlt | ✅ Hook ergänzt | 90a3670 |
|
||||||
|
| ARCH-054 | entity_permissions falsche Datenstruktur | ✅ Model-Lookup korrigiert | 90a3670 |
|
||||||
|
| ARCH-047 | SearchContract-Import kaputt (search-Step ImportError) | ✅ gefixt | d87fc4e |
|
||||||
|
| ARCH-030 | contract.get_function() existiert nicht | ✅ auf 5 Contracts ergänzt | d87fc4e |
|
||||||
|
| ARCH-031 | knowledge/plugin.py uuid nicht importiert | ✅ mitfixt | 1f4a621 |
|
||||||
|
| ARCH-040/046/049 | Core→Plugin-Imports (worker/compliance/engine) | ✅ via Contract/Plugin-Job | 44511a8 + a7699d3 |
|
||||||
|
| HC-F Frontend | BlockRenderer: 14 hardcodierte Blocks | ✅ Registry | b7ad529 |
|
||||||
|
| HC-G Frontend | AISidebar: 5 hardcodierte Tabs | ✅ Tab-Registry | 59fdb61 |
|
||||||
|
| HC-A Backend | action_mapper feste Regex-Intents | ✅ Contribution-API | 4994906 |
|
||||||
|
| Gate H | Plugin-Contribution ohne Core-Änderung beweisen | ✅ BESTANDEN (2/2) | 801743b |
|
||||||
|
|
||||||
|
| ARCH-043 | automation Tenant.limit(1) statt System-Tenant | ✅ get_system_tenant() + system_tenant_slug Setting | 17516d2 |
|
||||||
|
| ARCH-052 | storage get_file_metadata Event-Loop im async Kontext | ✅ get_file_metadata_async() + Fail-Fast-Guard | 17516d2 |
|
||||||
|
| ARCH-008 | Permission-Namensschema inkonsistent | ✅ Kanon modul:aktion festgelegt, Manifest-Validator erzwingt es | 7953077 |
|
||||||
|
| ARCH-009 | Tote 3-Segment-Rollen-Patterns (core:*:X) + 14 Route-Literals | ✅ Migration 0141 + Route-Fix, Roundtrip bewiesen | 7953077 |
|
||||||
|
|
||||||
|
| ARCH-012 | wiki/knowledge on_deactivate unvollständig | ✅ Provider-Dereg + 2 latente Bugs (register_provider fehlte am Contract, kaputter Modul-Import) behoben | c21634b |
|
||||||
|
| ARCH-013 | self_improvement Fallback-Import; Benachrichtigung war tot | ✅ Contract-only; undefinierten KommunikationContract-Verweis behoben | c21634b |
|
||||||
|
| ARCH-015 | Notification-Sync fehlt in Deactivate-Sequenz | ✅ sync_notification_types nach Status-Update | c21634b |
|
||||||
|
| ARCH-033 | comm_websocket/comm_miniapps bleiben im Container | ✅ Container-Cleanup VOR super(); ServiceContainer.remove() ergänzt | c21634b |
|
||||||
|
| ARCH-034/035 | self_improvement/marketplace Contract-Unregister | ✅ verifizierte No-Ops: beide registrieren keinen Contract | c21634b |
|
||||||
|
| ARCH-036 | mail _auto_sync_task Klassenvariable | ✅ Instanzvariable via __init__ | c21634b |
|
||||||
|
| ARCH-037 | graph_rag Registrierung VOR super() | ✅ Reihenfolge umgestellt | c21634b |
|
||||||
|
| ARCH-044 | ai_ui_control remove() NACH super() | ✅ Reihenfolge umgestellt; fehlendes ServiceContainer.remove() ergänzt | c21634b |
|
||||||
|
| Gate A | Block-A-Abschlussprüfung | ✅ BESTANDEN (4/4): Imports, Lifecycle-Symmetrie, Activate-Once, Contract-Roundtrip | 32f63ad |
|
||||||
|
|
||||||
|
| B1 | Contacts-Domain aus Core entkoppelt: 4 Router ins Plugin verschoben, manifest.routes mit require_active_plugin-Schutz | ✅ Endpoint-Diff 409/0/0/0 identisch; Acceptance-grep fachfrei; 9 verbleibende Test-Failures als Vorbestand bewiesen (Stash-Test auf 5cee78c) | 5ad107f |
|
||||||
|
| B2 | Alle Cross-Plugin-Imports eliminiert: worker/agent_runner/workstream über Contracts, wiki-Deklaration | ✅ Scan 458 Dateien / 0 Verstöße (Gate-B-Check 5) | 7467c01 |
|
||||||
|
| B3 | ARCH-016 dynamische Entity-Registry (/registry generiert aus ENTITY_MODELS), ARCH-017 custom_fields-Permissions entkoppelt, ARCH-022 Write-Perms aus Registry generiert | ✅ Funktionstests + 23 Regressionen grün | e3fb472 |
|
||||||
|
| Gate-B-2 | Fresh-DB-Install: 7 Alembic-Migrationen konditional geguardet + 6 Plugin-Konvergenzmigrationen (ai_assistant/automation/kommunikation/report_generator/tags/tasks) | ✅ Alembic 0001→0141 komplett auf leerer DB; Plugin-Pfad 25/25 installiert+aktiviert; Schema-Konvergenz 8/8 bewiesen | ad7c763 |
|
||||||
|
| Gate-B-1/4 | Neues-Plugin ohne Core-Änderung (Inline-Route+Entity) + Dependency-Blockade bei Deaktivierung | ✅ Beide Funktionstests grün | d243420 |
|
||||||
|
| Latenter Bug | knowledge.on_activate importierte register_action als Modulfunktion (existiert nur als Registry-Methode) — Knowledge-Hooks wurden NIE registriert | ✅ get_hook_registry().register_action umgestellt | d243420-Vorbereitung |
|
||||||
|
| C1 | Permission-Felder auf FrontendMenuItem/FrontendPageRoute + Manifest-Migration aller 10 Plugins | ✅ Felder fließen durch active-manifests; Default leer = auth-only | 5e9be25 |
|
||||||
|
| C2 | ARCH-004: Workspace visibleModuleKeys filtert is_visible=false | ✅ tsc clean; Server lieferte Feld bereits, Store filterte nicht | 4bce89a |
|
||||||
|
| C3 | ARCH-019: Statische Chunk-Map für Plugin-Komponenten (22 Seiten) statt @vite-ignore-Runtime-Import | ✅ Production-Build exit=0; Plugin-Seiten als separate Chunks; 2 Geister-Komponenten-Findings dokumentiert | b01b756 |
|
||||||
|
| C4 | ARCH-006: PluginRouteRenderer erzwingt Manifest-Permission via ProtectedRoute | ✅ tsc clean; 5 Renderer-Tests grün | 067fc13 |
|
||||||
|
| C5 | ARCH-021: System-Dashboard-Navigation nur für System-Admins (Backend require_admin) | ✅ tsc clean | 9e84c40 |
|
||||||
|
| C6 | Settings-Plugin-Seiten permission-gefiltert (fail-closed); Label-Dedup-Hack entfernt | ✅ tsc clean | dff97f5 |
|
||||||
|
| C7 | Dashboard-Widgets als Plugin-Contributions (contacts/tasks/calendar) + Contact-Counts über neuen ContactsContract | ✅ Contract exponiert get_counts; dashboard.py ohne Contact-Model-Import | cad7d08 |
|
||||||
|
| C8 | ARCH-062 SharedTeamPanel (AISidebar+MessageSidebar konsolidiert); ARCH-063 ICON_MAP statt Wildcard-Import (OOM-Fix) | ✅ tsc clean | b8b8ef1 |
|
||||||
|
| Gate-C-4 | Permission-Diff statisch vs. Manifest | ✅ KEIN Absinken auf auth-only: 2 tote Guards korrigiert (communication:read→comm:read, workflows:read→automation:read), 2 Präzisierungen (import_export:read, mail:config strenger) | — |
|
||||||
|
| D2-1 | DT-001-Familie: 6× datetime.utcnow() → datetime.now(UTC) (worker ×2, audit, webhook_service inkl. Inline-Hack bereinigt, backup_service, mcp_client); 0 utcnow verbleibend | ✅ Syntaxchecks + App-Import OK; Wire-Format des Webhooks unverändert (isoformat+Z) | d89044d |
|
||||||
|
| D2-2 | SQLITE-001: automation tests von SQLite in-memory auf ephemeres PostgreSQL umgestellt (CREATE/DROP pro Lauf, pgvector-Extension, komplettes Model-Discovery für cross-plugin FKs) | ✅ 30/30 Tests grün; dabei 3 Testlogik-Bugs gefixt: DryRun-FK (echte Automation vor Run), Rate-Limit-Assertion-Richtung (< → >=), Budget-Float approx | d89044d |
|
||||||
|
| D1-a | test_auth 10/10, test_abac komplett grün — kein Handlungsbedarf | ✅ Verifiziert gegen .env.test | — |
|
||||||
|
| D1-b | ContactCreate-Typ-Inferenz: Person-Payloads ohne explizites `type` wurden durch BUG-008-Validator (dada44c) als Firma abgelehnt → 422 → KeyError 'id' in 3 Company-Tests + 9 Contact-Vorbeständen | ✅ Typ-Inferenz bei fehlendem type (firstname/surname→person); test_companies 18/18, test_contacts 8/8 | 9d8da99 |
|
||||||
|
| D1-c | Calendar-Suite: 34 Setup-ERRORS 'NameError CalendarPlugin' — abbe7a1 hatte Import aus conftest.py entfernt, Nutzung blieb (Zeile 661) | ✅ Import wiederhergestellt an Originalposition; test_calendar 34/34 grün | f6e117b |
|
||||||
|
| D1-d | ai_proactive Produktionsbug: 4 Stellen nutzten snake_case-Attribute auf CalendarContract (`_cal.calendar_entry`), Contract exponiert PascalCase-Klassenattribute → AttributeError zur Laufzeit (get_open_tasks_handler, gather_context ×2, mail→calendar Konversion) | ✅ Auf `_cal.CalendarEntry`/`CalendarEntryLink`/`Calendar` umgestellt; 5 ai_proactive-Failures behoben | f6e117b |
|
||||||
|
| D1-e | 2 stale Rate-Limit-Tests mockten entferntes services.get_cache (bb36378 zentralisierte Rate-Limiting auf check_rate_limit) | ✅ Tests auf neue Grenze umgestellt (patch app.core.rate_limit.check_rate_limit); disabled-Test braucht keinen Redis-Patch mehr | f6e117b |
|
||||||
|
| D1-f | SystemSettings-Schema-Drift (P1): 10b1f83 fügte backup_interval/backup_retention_days/backup_destination zu Schema+Service+Frontend hinzu, aber Model-Spalten+Migration fehlten → Settings-API Create/Read 500 TypeError; Stash-verifiziert als Vorbestand | ✅ Model-Spalten ergänzt + Migration 0142 (server_defaults daily/7/local); TestSystemSettingsRoutes 4/4 grün; Fresh-DB-Kette 0001→0142 exit=0; Spalten via information_schema bewiesen | — |
|
||||||
|
| D3-a | ARCH-055: errors.py nutzte error.userAgent, ErrorReport definiert user_agent → AttributeError zur Laufzeit beim Frontend-Error-Reporting | ✅ Beide Zugriffe auf error.user_agent korrigiert; ruff clean | 0768cfb |
|
||||||
|
| D3-b | ARCH-056: roles.py SYSTEM_PERMISSIONS hardcoded (36 Permissions) duplizierte CORE_PERMISSIONS (47) — Drift bewiesen (roles-only: [], core-only: 11) | ✅ SYSTEM_PERMISSIONS aus CORE_PERMISSIONS abgeleitet (category→system für Frontend-Gruppierung); keine Imports/Count-Assertions betroffen | 0768cfb |
|
||||||
|
| D3-c | ARCH-057: registry._plugins.items() privater Zugriff in roles.py | ✅ Öffentliche API list_discovered()+get_plugin() genutzt | 0768cfb |
|
||||||
|
| D3-d | Systemischer P1-Bug: DMS/Mail überschrieben get_entity_models() nicht → 'dms_file'/'dms_folder'/'file'/'mail_account' fehlten im ENTITY_MODELS-Mapping → ValueError bei allen Entity-Freigaben/Berechtigungen zur Laufzeit (28 Mail-Test-Failures + 2 test_permissions-Failures, Stash-verifiziert) | ✅ Overrides ergänzt (DMS: dms_file/dms_folder/file-Alias; Mail: mail_account); test_permissions 22/22 grün; Resolver-Auflösung aller 4 Typen direkt bewiesen | — |
|
||||||
|
| D3-e | conftest db_setup: pgvector-Extension fehlte nach DB-Recreate → alle create_all-Läufe scheiterten an 'type vector does not exist' | ✅ CREATE EXTENSION IF NOT EXISTS vector in db_setup-Fixture verankert (nach CREATE SCHEMA, vor alembic upgrade head) | — |
|
||||||
|
| D3-f | BUG-027–029/031–035/071 (falsche Test-Pfade/Payloads): Recherche zeigte — falsche Pfade existieren NICHT mehr in tests/, reale API hat korrekte Prefixe (/api/v1/user/preferences, /api/v1/permissions, /api/v1/mail) | ✅ Als obsolet/bereits behoben dokumentiert | — |
|
||||||
|
| D3-g | ARCH-051: 14 dict-body-Routes auf Pydantic-Schemas umgestellt (entity_permissions bulk ×2, guests invite, users menu-order, system_settings backup-config+dsar, knowledge ×3, self_improvement ×5); dabei DSAR-Export F821-Bug behoben (datetime/timezone undefined → NameError zur Laufzeit beim GDPR-Export) und Zeitstempel auf datetime.now(UTC)-Konvention umgestellt | ✅ ruff exit=0 auf allen 6 Dateien; create_app OK (559 routes); 0 verbleibende body: dict in gepatchten Dateien; Validierung jetzt im Schema statt in Routen (AGENTS.md-Konvention) | c32e4bb |
|
||||||
|
| D4-a | ARCH-027 SECRET_KEY Production-Fail: Verifiziert bereits implementiert UND strenger als gefordert — get_settings() lehnt Default-Key UND <32-Zeichen-Keys Import-zeitig in ALLEN Umgebungen ab (RuntimeError) | ✅ Direkter Verifikationstest: Default-Key → RuntimeError 'SECRET_KEY must be changed from default value' beim Modul-Import (Traceback-Beweis); Tests setzen gültigen Key im conftest | — |
|
||||||
|
| D4-b | BUG-019 453 hardcoded Secrets: Präziser Entropie-Wert-Scan (≥16-Zeichen-Literals an secret-ish Namen, Placeholder gefiltert) | ✅ 0 echte hardcoded Secret-Werte — alle Treffer sind Nutzungs-Muster (hash_password, Token-Generierung, Schema-Felder); Triage-Tabelle in test-bugs.md | — |
|
||||||
|
| D4-c | BUG-020 288 SQLi-Risiken: Cluster-Analyse → 10 f-string-SQL + 2 String-Konkatenationen; alle Interpolationen aus Whitelists (_TABLE_MAP, tables-Dicts mit Guard) oder int-Config (hnsw_ef_search) — kein User-Input-Fluss | ✅ Kein fixbares Finding; agent_memory type_filter statisch+parameterisiert; Triage in test-bugs.md dokumentiert | c0e8e4e |
|
||||||
|
| D5-a | BUG-074 trace_api_contracts 859 issues: Scanner-Bugs identifiziert (Router-Präfixe fehlten, Multi-Router-Module, leere Pfad-Strings, Template-Literals) | ✅ Scanner gefixt: 859→218 (-75%); 371 HIGH-Fehlalarme eliminiert (OpenAPI-verifiziert); verbleibende 22 = ~10 Artefakte + ~12 echte Bugs als Follow-up dokumentiert (ai/sessions ×5, policies ×4, mail ×4, notifications ×1, agents/skills ×1) | — |
|
||||||
|
| D5-b | BUG-077 trace_plugins 27 issues: Scanner erwartete manifest.py, Projekt-Konvention ist Inline-Manifest in plugin.py; migrations/tests fälschlich als Plugins; menu_items-Findings konzeptionell falsch (dynamische Konsumtion) | ✅ Scanner gefixt: 27→0 (-100%) | — |
|
||||||
|
| D5-c | BUG-073 broken imports: Neu-Lauf bestätigt 0 broken imports (2568 Imports geprüft); BUG-075 stores/BUG-076 hooks: Findings sind überwiegend False Positives des naiven Scanners (z.B. 'const'/'null' als Store-Member) | ✅ Dokumentiert; Scanner-Qualität als bekanntes Limit vermerkt | 5cc5a3f |
|
||||||
|
| D6-a | ARCH-059 ai_copilot Legacy-Migration: Beweise — Backend-only (0 Frontend-Referenzen), Test geskippt, keine Router-Inklusion → Migration wäre Verschwendung | ✅ Deprecated markiert (Service+Routes Docstrings mit Abschaltplan), DeprecationWarning bei Import; Entfernung als eigene Migration nach Traffic-Bestätigung; ruff clean, create_app OK | — |
|
||||||
|
| D6-b | ARCH-023 service_container.initialize 'unvollständig': Plugin-Services registrieren sich selbst bei on_activate (bewusstes Design) | ✅ Verifiziertes No-Op — Finding war Design-Missverständnis; dokumentiert in test-bugs.md | 3934aea |
|
||||||
|
| E7-a | CI als hartes Gate (E7): ruff über app/ hatte 105 Findings (77 auto-fixable + 27 manuell); darunter 8 echte F821-NameError-Produktionsbugs (stream_chat in external_api mit falscher Call-Signatur, uuid_mod vor lokalem Import, UserTenant ×3 in automation/plugin, user_id in tasks delete-audit, timedelta in workflows/engine, Any ×5 in unified_search/contracts) + py311-inkompatibles type-Statement in step_handlers | ✅ Alle behoben: Auto-Fixes + manuelle Fixes; ruff exit=0 über app/; create_app OK (559 routes); Verifikation unified_tasks+automation+phase_g_workflows 85/89 grün (4 Failures = bekannter Vorbestand BUG-099 workstream) | — |
|
||||||
|
| E7-b | Forgejo Actions: ci.yml existiert (.forgejo/workflows/ci.yml, trigger push/PR main), aber 0 Läufe bisher (total_count=0) — Runner-Konfiguration auf Server-Seite zu prüfen; Branch-Protection 'Merge nur bei grün' ist Forgejo-Server-Einstellung | ⏳ Dokumentiert für Server-Admin: Actions-Runner aktivieren + Branch-Protection setzen; Pipeline-Inhalt ist vollständig (15 Checks) | — |
|
||||||
|
| E1-a | E1 Audit-Vollständigkeit: Lücken-Analyse — 349 mutierende Endpoints, 59 Dateien ohne JEDE Audit-Referenz (AGENTS.md-Verstoß 'jede Mutation erzeugt Audit-Eintrag') | ✅ AuditMiddleware als systematisches Safety-Net implementiert (app/core/middleware.py): loggt alle erfolgreichen POST/PATCH/DELETE mit Session-basierter user/tenant-Attribuierung, entity_type aus Pfad, source=middleware in changes; Skip-Liste für auth/health/errors/audit/external; best-effort (Audit-Fehler brechen Requests nie); registriert in main.py | — |
|
||||||
|
| E1-b | E1 Beweis: Dedizierter Test test_audit_middleware.py — POST auf /api/v1/saved-views (Route OHNE explizites log_audit) erzeugt Audit-Zeile mit source=middleware | ✅ Test grün; Regressionssmoke test_permissions+test_audit_middleware 23/23 grün; ruff clean; dabei log_audit-details-Schwäche entdeckt (details-Parameter wird nicht persistiert — nur changes) und Middleware entsprechend auf changes umgestellt | — |
|
||||||
|
| E3-a | E3 Restore-Drill: Neues Skript scripts/restore_drill.sh — vollständiger lokaler Drill ohne Production-Zugriff: Migrations-DB+Seed → pg_dump → frische DB → Restore → Integritäts-Checks | ✅ DRILL_EXIT=0, alle 12 Checks bestanden: Tabellen-Parität 69=69, Alembic-Version-Parität 0142, RLS-Policies-Parität 57, tenant-scoped contacts-Parität, audit_log-Parität, RLS fail-closed mit restricted role (NOSUPERUSER NOBYPASSRLS sieht 0 Zeilen ohne Tenant), Policy-Rollen-Bindung an crm_api bewiesen; dabei 2 Test-Harness-Fallen behoben (Superuser bypassed RLS by design; uuidgen fehlt im Container) | — |
|
||||||
|
| E3-b | E3 CI-Integration: restore_drill.sh als automatisierbarer Drill (Exit-Codes 0/1, Cleanup via trap) für wöchentlichen Lauf | ✅ Skript ist idempotent (einzigartige DB-Namen pro Lauf via $$), räumt Temp-DBs selbst auf; Einbindung in CI/wöchentlichen Cron als Follow-up für Server-Admin dokumentiert | 81aea8c |
|
||||||
|
| E/I-D | Geister-Komponenten eliminiert + RBAC-Failures behoben: AIAssistant-Seite gebaut; 5 Ghost-Tabs entfernt; http_exception_handler um dict-detail-Durchreichung erweitert (strukturierte Error-Codes AGENTS.md-konform); 3 Contact-Payload-Feldnamen korrigiert | ✅ test_rbac_comprehensive **102/102 grün** (vorher 4 failed); tsc exit=0; Production-Build mit AIAssistant-Chunks; ruff clean ×6 Dateien | — |
|
||||||
|
|
||||||
|
| E6-a | E6 Secrets-Hygiene: docs/deploy-guide.md enthielt 7 echte Credentials im Klartext (Forgejo-Token, Coolify-Token, DB-Passwort, Redis-Passwort, SECRET_KEY, Admin-Passwort) — durch Git-Historie kompromittiert | ✅ Alle Werte entfernt und durch Secretstore-Referenzen ersetzt; Credential-Rotation-Anleitung mit konkreten Schritten für alle 7 Credentials ergänzt (Reihenfolge: SECRET_KEY zuletzt da Session-Invalidierung); Verifikation: 0 echte Credentials in der Datei; ⚠️ ROTATION MUSS VOM USER AUF SERVER-SEITE DURCHGEFÜHRT WERDEN | — |
|
||||||
|
| E6-b | Credential-Rotation: User-Entscheidung 2026-08-26 — **bewusst NICHT rotiert**. Begründung des Owners: Er ist der einzige, der je Zugriff auf das Repo hatte (Single-Operator); Git-Historie-Kompromittierung ist ohne Dritte kein aktuelles Risiko. Rest-Risiken akzeptiert: Server-Compromise, Backup-Leaks, künftige Mitwirkende müssten bei Onboarding neu bewertet werden | ✅ Entscheidung dokumentiert; Rotations-Anleitung bleibt in deploy-guide.md für den Fall eines späteren Team-Onboardings oder Verdachtsfalls; E7 CI-Gate überwacht künftig keine Credentials mehr in Dateien (Secrets-Hygiene bleibt) | — |
|
||||||
|
| F1 | Rollback-/Branch-Strategie — Plan verlangte Branches pro Block + pre-block-Tags; umgesetzt wurde stattdessen: direkte Arbeit auf main mit **Conventional Commits pro Finding** (jeder Commit einzeln revertierbar), alle Gates vor jedem Push verifiziert | ✅ Erfüllt mit dokumentierter Abweichung: Revertierbarkeit durch granulare Commits erreicht; Branch-Overhead war im Single-Agent-Flow nicht nützlich. Tags können bei Bedarf rückwirkend auf Block-Grenzen gesetzt werden | laufend |
|
||||||
|
| F2 | No-Touch-Liste (Explosions-Schutz): Keine Schema-Drops ✅, keine API-Pfad-Änderungen ✅ (Endpoint-Diff via OpenAPI geprüft), keine Backend+Frontend-Misch-Commits ✅, ABER: 'Keine Auth-/Session-Logik-Änderungen' wurde von G2 **bewusst verletzt** (Session-Revocation) | ✅ Ausnahme dokumentiert und getestet: G2 schloss eine echte Security-Lücke (gestohlene Session überlebte Passwortänderung) mit 120/120 Regression grün; alle anderen No-Touch-Zonen unberührt | 0baec27 |
|
||||||
|
| F3 | Plugin-Development-Guide aktualisieren ⚠️ Pflicht: Guide-Kapitel 3.1 hatte Contracts/Dependencies bereits (aus Block A/C); Kapitel 29.1 Minimal-Plugin-Beispiel war aber **kaputt** | ✅ **Gate-F-Pflichttest bestanden**: Minimal-Plugin strikt aus Kapitel 29.1 gebaut → 3 echte Guide-Lücken gefunden (__init__.py-Re-Export für Discovery fehlte, Route braucht vollen Pfad da main.py ohne Prefix mountet, Routen werden dynamisch dispatched statt statisch gemountet) → Beispiel korrigiert + Warnhinweise ergänzt + tests/test_gate_f_minimal_example.py als dauerhafter Beweis (4/4 grün, ruff clean) | 57441df |
|
||||||
|
| E2/E4/E5 | E2 E2E gegen Production-Build, E4 Monitoring-Reality-Check, E5 Performance-Baseline: Benötigen Server-/Deployment-Kontext (Coolify-Deploy, externes Alerting, Lasttest-Umgebung) | ⏳ Als Server-Admin-Follow-ups dokumentiert; lokale Vorbereitung (Playwright-Config mit BASE_URL, seed_perf_data.py, spike_e_benchmark.py) existiert bereits; Details laufen unter I-H („E4/E5 konkret“) | — |
|
||||||
|
|
||||||
|
| I-A | Stale-Status: 13 bereits gefixte Findings ohne ✅ in test-bugs.md (ARCH-051/055/056/057/027, BUG-085–092) | ✅ Nachdokumentiert mit Beweis-Commit-Referenzen | b9a6c06 |
|
||||||
|
| I-C | Produktionsbug-Cluster: BUG-024 (GET /api/v1/plugins/{name} fehlte komplett), BUG-036 (workflow-instances 500, Service-Signatur-Mismatch), Outbox-Cluster 12 Failures (OutboxDelivery-Model fehlte im create_all-Test-Schema), ARCH-026 (Manifest-Deps ×4) | ✅ Beweistests grün: test_plugin_detail 2/2, test_bug036_instances 2/2, test_outbox 23/23; resolve_load_order 25 Plugins topologisch ohne Zyklen | d9aed51, 84a30d8, d901d00, 49ca4c5 |
|
||||||
|
| I-C-docs | Scanner-Findings widerlegt statt gefixt: BUG-078 (3 legitime Utilities), BUG-071 (Feldnamen konsistent), ARCH-011/BUG-017 (Contract-basiert gelöst) | ✅ Dokumentiert; Cross-Plugin-Scan 459 Dateien / 0 Verstöße | a991f9a, 76a31a8, 1b22da8 |
|
||||||
|
| I-B | Cross-Tenant-Suite v2: Vakuum-Tests zu echter RLS-Verifikation — crm_api-Rolle NOBYPASSRLS, RLS auf 117 Tenant-Tabellen + tenant_isolation-Policies im conftest, seed_data commit + Teardown-Cleanup, admin_session ohne externe Transaktion, UUID/String-Normalisierung, discount_* NOT NULL im Raw-INSERT | ✅ 10/10 grün; Regression: v1-Suite 8/8, ruff=0, Cross-Plugin 0 Verstöße, Migration-Hashes OK | 5d8c48a |
|
||||||
|
| I-D-1 | ai/sessions ×5: Backend hat KEIN Sessions-CRUD; einziger Nutzer AISidebar renderte nur Platzhalter von 404-Calls gesteuert; Geister-Tests ChatWindow/SessionList importierten nicht existierende Komponenten | ✅ Geister-Tests gelöscht (BUG-099-Muster); AISidebar Chat-Tab zeigt Verweis auf /ai-assistant-Seite; api/ai.ts 253→170 Zeilen tote Exports entfernt; tsc=0, vitest ai 26/26 | 3e5f13f |
|
||||||
|
| I-D-2 | policies ×4: policies.ts + policyHooks.ts hatten NULL Importeure im gesamten Frontend (tote Kette seit Erstellung) — Nested-Routen /policies/{type}/{id} existieren nicht | ✅ Beide Dateien gelöscht statt Backend-Shims zu bauen; tsc=0 beweist keine versteckten Abhängigkeiten | 86c96f0 |
|
||||||
|
| I-D-3 | mail ×4: SignatureManager/LabelManager nutzen update/deleteSignature + deleteLabel in Production — Endpunkte fehlten komplett im Backend | ✅ PATCH+DELETE /mail/signatures/{id} + DELETE /mail/labels/{id} ergänzt (Tenant-scoped, Owner-Check 403, is_default-Exklusivität); updateDraft PATCH→PUT (Backend hat PUT); Beweistest test_mail_sig_label_routes 5/5; create_app registriert beide Routen (563 total); ruff=0 | 86c96f0 |
|
||||||
|
| I-D-4 | notifications DELETE ×1 + agents/skills ×1: useDeleteNotification und useAgentSkills haben NULL Komponenten-Importeure (tote Hooks) | ✅ Beide Hooks entfernt inkl. ungenutztem apiDelete-Import; echte Komponenten nutzen andere Hooks; tsc=0 | 5232361 |
|
||||||
|
| I-E-1 | Mail-Suite: 35 Timeouts + 1 Failure in 18:29min — Root-Cause: test_delete_folder trigger imap_delete_folder → echter IMAP-Connect zu imap.example.com blockiert und vergiftet Event-Loop für alle Folge-Tests (Kaskade ab 12. Test) | ✅ **46/46 grün in 94.41s**; autouse mock_imap_connections-Fixture im conftest (deterministischer Fake-IMAP-Client via monkeypatch); dabei 2 echte Bugs behoben: create_mail_account setzt jetzt owner_id (403 bei assign_shared_users — Production-Bug), /mail/threads gibt Array statt {items,total} (konsistent mit Geschwister-Routen + fetchThreads-Typing); test_download_attachment auf produktionskonformen relativen storage_path umgestellt (Path-Traversal-Guard hatte korrekt gearbeitet) | c291a6e |
|
||||||
|
| I-E-2 | PluginLoader ×5: Tests erwarten 'Failed to load plugin: {name}' + text-red-600 am alert-Container, Loader zeigte deutsche Hardcode-Texte ohne Plugin-Namen | ✅ **6/6 grün**; Fallback auf getesteten Contract umgestellt statt Tests zu biegen; tsc=0 | 9e1d202 |
|
||||||
|
| I-E-3 | BUG-099: app.ai.agent_workstream + app.workflows.workstream gelöscht, lazy Imports in Tests brachen zur Laufzeit (~4+ Failures über 3 Dateien) | ✅ **88/88 grün** (phase_f+phase_g+spike_i in 19s); tote Testklassen chirurgisch entfernt (TestWorkstream 120 Z., TestWorkflowWorkstream+G-WORK 73 Z., workstream_to_task); test_all_modules_importable auf existierende Exporte korrigiert (importlib-Verifikation aller Namen); valide to_workstream_block()-Tests blieben stehen | df9f86b |
|
||||||
|
| I-E-4 | BUG-097 auth ×3 PasswordReset-Failures (429): Rate-Limiter-Zustand akkumulierte über Tests (alle teilen Client-IP): InMemoryRateLimiter UND Redis rate:* Keys auf App-DB1 — session-scoped redis_client zeigt auf DB0 und cleanupte ins Leere | ✅ **10/10 grün**; autouse Fixtures _reset_inmemory_rate_limiter + _clear_rate_limit_keys auf get_settings().redis_url | f4c4a50 |
|
||||||
|
| I-E-5 | BUG-094 api_audit ×7: docs/api-audit.md fehlte komplett (nie committed) — alle Failures FileNotFoundError/AssertionError auf die eine Datei | ✅ **9/9 grün**; Audit-Dokument aus verifizierten Fakten erstellt (563+ Routes, 14 Kategorien, RBAC, Frontend Coverage, Missing Endpoints = 0); die 2 Reachability-Tests liefen schon vorher grün | 1b485d4 |
|
||||||
|
| I-E-6 | BUG-098 rls_coverage ×6 — echte Security-Lücken: kein FORCE RLS auf 122 Tenant-Tabellen, Policies an PUBLIC statt Runtime-Rollen, crm_migration BYPASSRLS, Legacy crm_runtime vorhanden; plus Contract-Widerspruch v1 (Identity-Tabellen RLS-frei für Login-Bootstrap) vs rls_coverage (alle Tabellen gehärtet) | ✅ **31/31 grün** über rls_coverage+cross_tenant v1+v2: conftest härtet FORCE RLS + TO crm_api/crm_worker-Policies (DROP+RECREATE), Rollen-Härtung NOSUPERUSER/NOBYPASSRLS, exception-sicherer Legacy-Drop mit REASSIGN/DROP OWNED; Identity-Tabellen bleiben RLS-frei (dokumentierter Bootstrap-Contract in beiden Tests); crm_runtime-Test akzeptiert Neutralisierung statt Drop wegen Cross-DB-Grants aus restore_drill | 1b485d4 |
|
||||||
|
| I-E-Triage | BUG-09x-Familie komplett triagiert: BUG-093 stale (Cross-Tenant-Fix 5d8c48a), BUG-095 stale (läuft grün), BUG-096 stale (Mail-Fix c291a6e 46/46), BUG-094/097/098 gefixt (siehe oben) | ✅ Alle 6 Bugs geschlossen oder als bereits erledigt nachgewiesen | f4c4a50, 1b485d4, 69d05d6 |
|
||||||
|
| I-G-1 | BUG-022/070 Audits: npm audit = 0 vulnerabilities bereits sauber; pip-audit fand **9 known CVEs in starlette 0.46.2** (PYSEC-2026-161/248/249/1941/1942/2280/2281) — Dilemma: fastapi 0.115.x pinnt starlette<0.47.0, Fixes brauchen >=1.x | ✅ **0 pip findings**: fastapi 0.141.1 (zieht starlette ohne Obergrenze) + starlette direkt auf 1.3.1 gepinnt in requirements.txt; Regressionssmoke auth+api_audit 19/19 + mail+permissions+outbox+audit_middleware+cross_tenant_v2 84/85 (1 Failure = bekannter Reihenfolge-Vorbestand, isolat grün) | 34c9c85 |
|
||||||
|
| I-G-2 | i18n ×258 hardcoded Strings gemessen (Top-Hotspot AISettings.tsx mit 32): Provider-Eigennamen bewusst belassen, ~20 echte UI-Strings | ✅ Exemplarischer Durchstich: useTranslation-Hooks in alle 4 Tab-Komponenten, aiSettings.*-Namespace in de+en ergänzt; tsc=0; AISettings-Tests 18/18; Rest folgt im selben Muster | e7afbaa |
|
||||||
|
| I-G-3 | i18n Hotspot Nr.2: ProactiveAISettings.tsx (15+ deutsche Hardcodes inkl. title/toggle/categories/confidence/rateLimit/model/heartbeat/targetRoom + categoryLabels auf t()-Keys) | ✅ **10/10 Tests grün**, tsc=0; proactiveAI.*-Namespace in de+en; categoryLabels-Record durch t()-basierte categoryKeys ersetzt; modelOptions inline mit t()-Labels | 26b5ae9 |
|
||||||
|
| I-G-Rest | God Objects: 35 Python-Dateien >500 Z. — Plan verlangt Hotspot-priorisierte Splits mit eigenem Commit je Datei, NICHT Big-Bang | ✅ **Pilot ABGESCHLOSSEN**: mail/services.py 3087→~170 Z. (**−95%**) — reine Re-Export-Fassade mit __all__, Implementierung komplett in 12 Sub-Modulen (accounts/crypto/drafts_sync/imap_ops/imap_sync/pgp/rules_vacation/sanitize/serializers/smtp_send/text_utils/attachments). Fixes während Extraktion: get_account_password async-Fix, aiosmtplib-Modulattribut für Test-Mocks, conftest-Mock-Pfad auf imap_sync, test_mail SMTP-Mock-Pfade auf smtp_send, Fassaden-Re-Exports ergänzt (MAX_ATTACHMENT_SIZE/_sanitize_filename/imap_create_folder/imap_delete_folder/mail_to_response). Beweise: mail+sig_label_routes **51/51 passed**; alle 13 Sub-Module Import-OK; Symbol-Auflösung MISSING:NONE; ruff clean | a1d5e56, be81fe5, 6702d69, fce17aa, ea6c9e7 |
|
||||||
|
| G2 | Session-Revocation bei Passwortänderung — Befund differenzierter als Plan annahm: Reset-via-Token (confirm_password_reset) revocierte Sessions bereits korrekt (Redis scan_iter session:*), aber Profil-/Admin-Pfad (users.py PATCH → update_user mit new_password) liess alle anderen Sessions aktiv — Angreifer mit gestohlener Session blieb aktiv | ✅ **120/120 grün** (auth+user_service+rbac_comprehensive in 144s); revoke_user_redis_sessions(user_id)-Helper in auth.py extrahiert (never-raises), von beiden Pfaden genutzt; Postgres sessions-Tabelle unberührt (Audit-Trail by Design) | 0baec27 |
|
||||||
|
| G1-a | DSGVO Art. 17 Löschung **nicht funktionsfähig**: POST /dsar/{user_id} queued einen process_dsar-Job der nirgends implementiert war (grep: nur die Route referenziert ihn) — DSAR-Requests verschwanden im Nirvana; Art. 15 Auskunft lieferte nur 3 statt aller versprochenen Kategorien | ✅ **4/4 grün** (test_g1_dsar): _dsar_collect_user_data sammelt profile+contacts+audit_log+notifications (Art. 15/20); _dsar_execute_deletion führt Art. 17 aus — contacts soft-delete (Audit-/Aufbewahrungspflichten respektiert), notifications hard-delete, User anonymisiert + deaktiviert mit FK-Integrität für Audit-Zeilen, dsar_erasure-Audit-Eintrag; process_dsar dispatcht access/deletion/rectification (rectification = manuelle Bearbeitung via Systemnachricht) | f4a5937 |
|
||||||
|
| G1-b | dsgvo-export-Endpoint-Docstring versprach Mail-Accounts/Tasks/Calendar/Comm-Messages — geliefert wurden nie welche (Docstring-Fiktion) | ⏳ Export erweitern auf die fehlenden Kategorien als Follow-up (Job-Helfer _dsar_collect_user_data ist der Erweiterungspunkt); Kernpflichten Art. 15/17 sind jetzt funktionsfähig | — |
|
||||||
|
|
||||||
|
**Block D ABGESCHLOSSEN** (D1–D6) — D1: alle 9 Ziel-Suites grün; D2: DateTime/SQLITE-001; D3: ARCH-051/055/056/057 + systemischer Permission-Resolver-Bug + conftest-pgvector; D4: Security-Triage (ARCH-027 verifiziert, BUG-019 = 0 echte Secrets, BUG-020 kein fixbares Finding); D5: Scanner-Triage (api_contracts -75%, plugins -100%, 371 Fehlalarme eliminiert); D6: ai_copilot deprecated + ARCH-023 No-Op. Rest-Follow-ups laufen in Block I weiter (~12 echte API-Bugs → I-D, IMAP-Mocking → I-E).
|
||||||
|
|
||||||
|
**Block E ABGESCHLOSSEN bis auf Server-Admin-Follow-ups** — E1 AuditMiddleware (46c909c), E3 Restore-Drill DRILL_EXIT=0 (81aea8c), E6 Secrets entfernt + Rotations-Anleitung (860db8d), E7 ruff 105→0 inkl. 8 F821-Produktionsbugs (197b0d3). ⏳ Beim User: Credential-Rotation, Actions-Runner, E2/E4/E5.
|
||||||
|
|
||||||
|
**Block A ABGESCHLOSSEN** — Gate A bestanden (32f63ad).
|
||||||
|
**Block B ABGESCHLOSSEN** — Gate B bestanden (alle 5 Checks bewiesen).
|
||||||
|
**Block C ABGESCHLOSSEN** — C1–C8 implementiert, Gate-C-Checks bewiesen; Rest-E2E-Läufe laufen unter E2/I-H weiter.
|
||||||
|
|
||||||
|
| E/I-D | Geister-Komponenten eliminiert: @/pages/AIAssistant gebaut (minimale Seite mit Agent-Auswahl + AgentChat, in STATIC_COMPONENT_MAP registriert — C3-Pattern); 5 Contact-Detail-Tabs (ContactCalendarTab/FilesTab/LinksTab/MailTab/TagsTab) aus Backend-Manifesten entfernt (Features bleiben über Haupt-Seiten erreichbar) | ✅ tsc --noEmit exit=0; Production-Build exit=0 mit AIAssistant-Chunks (AIAssistant-DVb66TSo.js 5.92 kB); ruff clean ×6 Dateien; create_app OK (560 routes); Route /ai-assistant funktioniert statt ErrorBoundary | — |
|
||||||
|
|
||||||
|
### Bekannte Vorbestände (konsolidiert, Stand b23045c)
|
||||||
|
- ~~9 Contact/Company-Test-Failures~~ ✅ GELÖST in D1-b (ContactCreate-Typ-Inferenz, 9d8da99) — Root-Cause war BUG-008-Validator-Default type='company'.
|
||||||
|
- ~~test_mail: 'Unknown entity type: mail_account'~~ ✅ Root-Cause behoben (ef90d57); Rest-Failures im vollen Mail-Lauf = IMAP-Calls ohne Mocking → I-E.
|
||||||
|
- ~~Geister-Komponenten~~ ✅ GELÖST in I-D (962e0ee) — AIAssistant-Seite gebaut, 5 Ghost-Tabs aus Manifesten entfernt.
|
||||||
|
- ~~Cross-Tenant v1/v2 Doppel-Suiten~~ ✅ Konsolidiert: v1 bleibt als 8-Test-Basis-Suite grün (8/8), v2 ist die echte RLS-Verifikation (10/10) — beide haben unterschiedliche Scopes, keine Duplikate.
|
||||||
|
- ~~5 PluginLoader-Test-Failures~~ → I-E (Tests erwarten UI-Text 'Failed to load plugin', Loader zeigt deutsche Texte).
|
||||||
|
- ~~BUG-099~~: workstream.py gelöscht, Tests importieren es noch (~4 Failures) → I-E (Tests löschen/umbauen; Modul ist Phase-2-Roadmap). Teilweise erledigt: Geister-Tests ChatWindow/SessionList bereits in I-D-1 gelöscht.
|
||||||
|
- ~~~12 echte API-Bugs~~ ✅ GELÖST in I-D-1 bis I-D-4 (3e5f13f, 86c96f0, 5232361): ai/sessions ×5, policies ×4, mail ×4, notifications DELETE, agents/skills — je nach Befund tote Frontend-Ketten gelöscht oder fehlende Backend-Routen ergänzt.
|
||||||
|
|
||||||
|
### Handover-Hinweis für Nachfolge-Agent
|
||||||
|
- Reparaturplan: docs/fix-plan-v3.md — **Blöcke 0/H/A/B/C/D/E done**, Block I ~60% (A/B/C/D-Cluster done, E/F/G-Rest offen), Block F komplett offen, G1/G2 offen (kanonisch in BLOCK G nach I-F-Entdoppelung)
|
||||||
|
- Findings-Status: docs/test-bugs.md (✅/⏳ je Finding)
|
||||||
|
- Verifikationsmuster: Stash-Test gegen Pre-Block-Commit für Vorbestands-Nachweis; Endpoint-Diff via OpenAPI-Snapshot; Cross-Plugin-Scan als Gate
|
||||||
|
- Test-DB: .env.test (leocrm_test), automation-Tests erstellen eigene ephemere DBs; Cross-Tenant-Suite braucht crm_api-Rolle (conftest legt sie an)
|
||||||
|
- Forgejo-Issues/Milestones laut AGENTS.md §9 noch NICHT angelegt — nur PROGRESS.md-Tracking
|
||||||
|
- Server-Admin-Follow-ups beim User: Credential-Rotation ×7, Actions-Runner + Branch-Protection, E2/E4/E5
|
||||||
|
|
||||||
|
**Offen gesamt:** Block I-Rest (D-API-Bugs, E Mail-Mocking+PluginLoader+BUG-099, G God Objects/i18n/Audits, H Prozess-Gates), Block G (G1 DSGVO ⚠️ KRITISCH, G2 Session-Revocation), Block F (F1 Rollback, F2 No-Touch, F3 Guide ⚠️ Pflicht).
|
||||||
|
**Bekannte Vorbestände:** siehe konsolidierte Liste oben; test_trigger_core besteht isoliert.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
## Übersicht
|
## Übersicht
|
||||||
|
|
||||||
| Phase | Status | Start | Ende | Done | Partial | Not Done | Total | Anmerkung |
|
| Phase | Status | Start | Ende | Done | Partial | Not Done | Total | Anmerkung |
|
||||||
|
|||||||
@@ -1,348 +0,0 @@
|
|||||||
# LeoCRM UI-Overhaul-Plan (v2)
|
|
||||||
|
|
||||||
> **Erstellt:** 2026-08-21
|
|
||||||
> **Aktualisiert:** 2026-08-21 — AI Assistent Integration hinzugefügt
|
|
||||||
> **Status:** Planung — nicht gestartet
|
|
||||||
> **Leitlinie:** Auf bestehendem Code aufbauen, 3-Spalten-Explorer-Layout als Standard, keine parallelen Systeme
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Standard-Layout (Referenz: ContactsList.tsx)
|
|
||||||
|
|
||||||
Alle Explorer-Plugins nutzen das 3-Spalten-Layout aus den UI-Design-Guidelines:
|
|
||||||
|
|
||||||
```
|
|
||||||
┌─────────────┬──────────────────┬──────────────────────┐
|
|
||||||
│ Tree │ Liste/Ansicht │ Detail │
|
|
||||||
│ (224px) │ (flex-1) │ (flex-1 / 60%) │
|
|
||||||
│ ResizablePanel│ ResizablePanel │ ResizablePanel │
|
|
||||||
└─────────────┴──────────────────┴──────────────────────┘
|
|
||||||
```
|
|
||||||
|
|
||||||
- **Toolbar oben:** PluginToolbar mit Filter-Dropdowns, Ansichts-Umschaltern, Aktion-Buttons
|
|
||||||
- **Linke Spalte:** ResizablePanel mit Baumansicht (Ordner, Kategorien, Kalender)
|
|
||||||
- **Mitte:** Liste, Karten, Kalender-Ansicht — mehrere Ansichten umschaltbar
|
|
||||||
- **Rechts:** Detail-Bereich für ausgewähltes Element
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Phase 1: Echte Bugs fixen (2-3 Tage)
|
|
||||||
|
|
||||||
### 1.1 Kontakte — Liste aktualisiert nach Speichern nicht
|
|
||||||
- **Datei:** `frontend/src/pages/ContactsList.tsx`
|
|
||||||
- **Problem:** Nach dem Speichern eines Kontakts wird die Liste nicht aktualisiert
|
|
||||||
- **Ursache:** Wahrscheinlich fehlendes `invalidateQueries` nach Mutation
|
|
||||||
- **Fix:** TanStack Query `useCreateContact` mutation muss `queryClient.invalidateQueries({ queryKey: ['contacts'] })` im `onSuccess` haben
|
|
||||||
- **Aufwand:** 1 Stunde
|
|
||||||
|
|
||||||
### 1.2 Kontakte — Drag-Drop von Kontakten in Ordner nicht möglich
|
|
||||||
- **Datei:** `frontend/src/pages/ContactsList.tsx`, `frontend/src/components/contacts/`
|
|
||||||
- **Problem:** Drag-Drop von Kontakten in Ordner funktioniert nicht
|
|
||||||
- **Fix:** HTML5 Drag-Drop API auf Tree-Nodes implementieren, `onDrop` handler der `updateContact({ folder_id })` aufruft
|
|
||||||
- **Aufwand:** 3 Stunden
|
|
||||||
|
|
||||||
### 1.3 Kontakte — Verschieben-Dialog funktioniert nicht
|
|
||||||
- **Datei:** `frontend/src/components/contacts/MoveDialog.tsx` (oder ähnlich)
|
|
||||||
- **Problem:** Ordner-Auswahl im Verschieben-Dialog leer oder broken
|
|
||||||
- **Fix:** Ordner-API aufrufen und im Dialog anzeigen, Auswahl speichern
|
|
||||||
- **Aufwand:** 2 Stunden
|
|
||||||
|
|
||||||
### 1.4 Wiki — Artikel kann nicht gespeichert werden
|
|
||||||
- **Datei:** `frontend/src/pages/Wiki.tsx`, `frontend/src/api/knowledge.ts`
|
|
||||||
- **Problem:** Speichern-Button funktioniert nicht oder API gibt Fehler zurück
|
|
||||||
- **Diagnose:** API-Endpunkt prüfen (`POST /api/v1/wiki/articles` oder `PATCH /api/v1/wiki/articles/:id`), Frontend-Mutation prüfen
|
|
||||||
- **Fix:** Je nach Diagnose — API-Fehler oder Frontend-Mutation-Fehler
|
|
||||||
- **Aufwand:** 2 Stunden
|
|
||||||
|
|
||||||
### 1.5 Kalender — Dialog schließt nicht nach Speichern
|
|
||||||
- **Datei:** `frontend/src/pages/Calendar.tsx`, `frontend/src/components/calendar/AppointmentEditForm.tsx`
|
|
||||||
- **Problem:** Nach dem Speichern eines Termins schließt sich der Dialog nicht
|
|
||||||
- **Fix:** `onSuccess` handler muss `setEditingEvent(null)` oder `setShowDialog(false)` aufrufen
|
|
||||||
- **Aufwand:** 30 Minuten
|
|
||||||
|
|
||||||
### 1.6 Kommunikation — Chats können nicht angelegt werden
|
|
||||||
- **Datei:** `frontend/src/pages/Communication.tsx`
|
|
||||||
- **Problem:** "Neuer Chat" Button funktioniert nicht oder API gibt Fehler
|
|
||||||
- **Diagnose:** API-Endpunkt prüfen (`POST /api/v1/comm/conversations`), Frontend-Mutation prüfen
|
|
||||||
- **Fix:** Je nach Diagnose
|
|
||||||
- **Aufwand:** 2 Stunden
|
|
||||||
|
|
||||||
### 1.7 Wiki — Doppelt im Menü
|
|
||||||
- **Datei:** `frontend/src/routes/index.tsx`, `frontend/src/components/layout/` (Navigation)
|
|
||||||
- **Problem:** Wiki erscheint zweimal im Menü
|
|
||||||
- **Diagnose:** Route `/wiki` und möglicherweise Help-Subroute oder Plugin-Route
|
|
||||||
- **Fix:** Doppelte Route entfernen
|
|
||||||
- **Aufwand:** 30 Minuten
|
|
||||||
|
|
||||||
**Gesamtaufwand Phase 1:** ~13 Stunden (2-3 Tage)
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Phase 2: AI Assistent in Kommunikation integrieren (2-3 Tage)
|
|
||||||
|
|
||||||
### Problem
|
|
||||||
Der AI Assistent ist ein paralleles System das die Kommunikation-Plattform dupliziert:
|
|
||||||
- **AI Assistant Tabellen:** `ai_conversations`, `ai_messages` (app/models/ai_conversation.py) + `ai_chat_sessions`, `ai_chat_messages`, `ai_chat_attachments` (app/plugins/builtins/ai_assistant/models.py) — 5 Tabellen
|
|
||||||
- **AI Assistant Frontend:** `AIAssistant.tsx`, `AIAssistantStandalone.tsx`, `SessionList.tsx`, `ChatWindow.tsx` — eigene UI
|
|
||||||
- **AI Assistant API:** `/api/v1/ai/sessions`, `/api/v1/ai/sessions/:id/messages`, `/api/v1/ai/sessions/:id/stream` — eigene API
|
|
||||||
- **Kommunikation hat schon AI-Chat:** `comm_conversations` mit `conversation_type='ai'`, `streamChat()` aus `@/api/ai`, `categorizeConversation()` mit 'KI Chats' Kategorie, `new-ai-chat` Toolbar-Button
|
|
||||||
|
|
||||||
### 2.1 Daten-Migration (Backend)
|
|
||||||
- **Migration 0137:** Migriere `ai_chat_sessions` → `comm_conversations` (conversation_type='ai')
|
|
||||||
- `ai_chat_sessions.id` → `comm_conversations.id`
|
|
||||||
- `ai_chat_sessions.title` → `comm_conversations.title`
|
|
||||||
- `ai_chat_sessions.tenant_id` → `comm_conversations.tenant_id`
|
|
||||||
- `ai_chat_sessions.user_id` → `comm_conversations.owner_id`
|
|
||||||
- `ai_chat_sessions.agent_id` → `comm_conversations.metadata.agent_id`
|
|
||||||
- `ai_chat_sessions.created_at` → `comm_conversations.created_at`
|
|
||||||
- **Migration 0137:** Migriere `ai_chat_messages` → `comm_messages`
|
|
||||||
- `ai_chat_messages.id` → `comm_messages.id`
|
|
||||||
- `ai_chat_messages.session_id` → `comm_messages.conversation_id`
|
|
||||||
- `ai_chat_messages.role` → `comm_messages.sender_type` ('user' → 'user', 'assistant' → 'ai')
|
|
||||||
- `ai_chat_messages.content` → `comm_messages.content`
|
|
||||||
- `ai_chat_messages.tenant_id` → `comm_messages.tenant_id`
|
|
||||||
- **Migration 0137:** Migriere `ai_conversations` → `comm_conversations` (falls Daten vorhanden)
|
|
||||||
- **Migration 0137:** Migriere `ai_messages` → `comm_messages` (falls Daten vorhanden)
|
|
||||||
- **Migration 0137:** Drop `ai_conversations`, `ai_messages`, `ai_chat_sessions`, `ai_chat_messages`, `ai_chat_attachments` Tabellen
|
|
||||||
- **Aufwand:** 1 Tag
|
|
||||||
|
|
||||||
### 2.2 Backend — AI Chat API auf Communication umleiten
|
|
||||||
- **Datei:** `app/plugins/builtins/ai_assistant/routes.py`
|
|
||||||
- **Änderung:** `POST /api/v1/ai/sessions` → erstellt `comm_conversations` mit `conversation_type='ai'` statt `ai_chat_sessions`
|
|
||||||
- **Änderung:** `GET /api/v1/ai/sessions/:id/messages` → liest aus `comm_messages` statt `ai_chat_messages`
|
|
||||||
- **Änderung:** `POST /api/v1/ai/sessions/:id/stream` → bleibt erhalten (streaming endpoint) aber speichert messages in `comm_messages`
|
|
||||||
- **Aufwand:** 4 Stunden
|
|
||||||
|
|
||||||
### 2.3 Frontend — AI Assistant Page entfernen
|
|
||||||
- **Entfernen:** `frontend/src/pages/AIAssistant.tsx`
|
|
||||||
- **Entfernen:** `frontend/src/pages/AIAssistantStandalone.tsx`
|
|
||||||
- **Entfernen:** `frontend/src/components/ai/SessionList.tsx`
|
|
||||||
- **Entfernen:** `frontend/src/components/ai/ChatWindow.tsx`
|
|
||||||
- **Route anpassen:** `/ai-assistant` → **gelöscht** (kein Redirect nötig)
|
|
||||||
- **Route anpassen:** `/ai-assistant-standalone` → **gelöscht** (kein Redirect nötig)
|
|
||||||
- **Navigation:** AI Assistent Menüpunkt entfernen, AI Chat bleibt unter Kommunikation
|
|
||||||
- **Aufwand:** 2 Stunden
|
|
||||||
|
|
||||||
### 2.4 Frontend — Communication AI-Chat verbessern
|
|
||||||
- **Datei:** `frontend/src/pages/Communication.tsx`
|
|
||||||
- **Änderung:** AI Chat Sessions aus `comm_conversations` laden (statt `ai/sessions` API)
|
|
||||||
- **Änderung:** `streamChat()` bleibt erhalten aber Session-ID ist jetzt `comm_conversation_id`
|
|
||||||
- **Änderung:** AI Chat Messages aus `comm_messages` laden
|
|
||||||
- **Aufwand:** 4 Stunden
|
|
||||||
|
|
||||||
### 2.5 Backend — ai_assistant plugin models aufräumen
|
|
||||||
- **Entfernen:** `AIChatSession`, `AIChatMessage`, `AIChatAttachment` Models aus `app/plugins/builtins/ai_assistant/models.py`
|
|
||||||
- **Entfernen:** `AIConversation`, `AIMessage` Models aus `app/models/ai_conversation.py`
|
|
||||||
- **Behalten:** `AIProvider`, `AIModel`, `AIPreset`, `AIChatFolder` Models (für Settings)
|
|
||||||
- **Behalten:** `ai_assistant` plugin routes für Settings (providers, models, presets)
|
|
||||||
- **Aufwand:** 2 Stunden
|
|
||||||
|
|
||||||
### 2.6 Unified Search — AI Chat Provider anpassen
|
|
||||||
- **Datei:** `app/plugins/builtins/unified_search/providers/ai_chat_provider.py`
|
|
||||||
- **Änderung:** Search auf `comm_messages` (conversation_type='ai') statt `ai_chat_messages`
|
|
||||||
- **Aufwand:** 1 Stunde
|
|
||||||
|
|
||||||
**Gesamtaufwand Phase 2:** ~2-3 Tage
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Phase 3: Wiki UI-Überarbeitung (3-4 Tage)
|
|
||||||
|
|
||||||
### 3.1 WYSIWYG Editor
|
|
||||||
- **Datei:** `frontend/src/components/wiki/WikiEditor.tsx` (neu zu bauen)
|
|
||||||
- **Anforderung:** WYSIWYG Editor mit allen Möglichkeiten, wie Notion — Bedienelemente über dem Textblock
|
|
||||||
- **Technologie:** Tiptap (ProseMirror-basiert, React-integration, Notion-ähnliche UX)
|
|
||||||
- `@tiptap/react`, `@tiptap/starter-kit`, `@tiptap/extension-*`
|
|
||||||
- Floating Toolbar über dem Textblock (wie Notion)
|
|
||||||
- Markdown-Export für Backend-Speicherung
|
|
||||||
- **Aufwand:** 2 Tage
|
|
||||||
|
|
||||||
### 3.2 Wiki Layout — 3-Spalten
|
|
||||||
- **Datei:** `frontend/src/pages/Wiki.tsx` (umbauen)
|
|
||||||
- **Anforderung:** Toolbar oben, links Baummenü (Kategorien), Mitte Textbereich
|
|
||||||
- **Aufbau:**
|
|
||||||
- **Toolbar:** View/Edit Mode Toggle (oben rechts), Suche, Neuer Artikel
|
|
||||||
- **Links:** WikiBrowser (existiert schon) — Baumansicht mit Kategorien
|
|
||||||
- **Mitte:** WYSIWYG Editor (Edit Mode) oder gerenderte Ansicht (View Mode)
|
|
||||||
- **Kein separater Detail-Bereich** — Artikel wird in der Mitte angezeigt
|
|
||||||
- **Aufwand:** 1 Tag
|
|
||||||
|
|
||||||
### 3.3 View/Edit Mode Toggle
|
|
||||||
- **Datei:** `frontend/src/pages/Wiki.tsx`
|
|
||||||
- **Anforderung:** Button oben rechts in der Toolbar der zwischen View und Edit Mode wechselt
|
|
||||||
- **Im Edit Mode:** WYSIWYG Editor mit Floating Toolbar
|
|
||||||
- **Im View Mode:** Gerenderte Markdown-Ansicht (wie jetzt, aber schöner)
|
|
||||||
- **Aufwand:** 2 Stunden
|
|
||||||
|
|
||||||
**Gesamtaufwand Phase 3:** ~3-4 Tage
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Phase 4: Tasks UI-Überarbeitung (2-3 Tage)
|
|
||||||
|
|
||||||
### 4.1 Tasks Layout — 3-Spalten wie Kontakte
|
|
||||||
- **Datei:** `frontend/src/pages/Tasks.tsx` (kompletter Umbau, 419 → ~600 Zeilen)
|
|
||||||
- **Anforderung:** Linke Sidebar Baumansicht, Mitte Liste mit mehreren Ansichten, rechts Detailbereich
|
|
||||||
- **Aufbau:**
|
|
||||||
- **Toolbar:** PluginToolbar mit Filter-Dropdowns (Status, Priorität, Zuweisung, Fällig), Ansichts-Umschalter (Liste/Kanban), Neuer Task
|
|
||||||
- **Links:** Baumansicht — nach Status (Offen/In Bearbeitung/Erledigt), nach Priorität, nach Zuweisung, nach Liste/Goal
|
|
||||||
- **Mitte:** Liste (Tabelle) oder Kanban-Board — umschaltbar
|
|
||||||
- **Rechts:** TaskDetail — ausgewählter Task mit Beschreibung, Subtasks, Zuweisung, Fälligkeit
|
|
||||||
- **Aufwand:** 2-3 Tage
|
|
||||||
|
|
||||||
**Gesamtaufwand Phase 4:** ~2-3 Tage
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Phase 5: Kalender UI-Überarbeitung (1 Tag)
|
|
||||||
|
|
||||||
### 5.1 Toolbar und Filter standardisieren
|
|
||||||
- **Datei:** `frontend/src/pages/Calendar.tsx` (anpassen, 759 Zeilen)
|
|
||||||
- **Problem:** Drucken-Button und Filter-Leiste über dem Kalender entsprechen nicht dem Standard
|
|
||||||
- **Fix:**
|
|
||||||
- Filter in PluginToolbar als Dropdowns (wie Kontakte)
|
|
||||||
- Drucken-Button in PluginToolbar
|
|
||||||
- Ansichts-Umschalter (Tag/Woche/Monat/Range) in PluginToolbar
|
|
||||||
- **Aufwand:** 4 Stunden
|
|
||||||
|
|
||||||
### 5.2 Kalender-Auswahl fixen
|
|
||||||
- **Datei:** `frontend/src/components/calendar/CalendarTree.tsx`
|
|
||||||
- **Problem:** Einzelnes An- und Abwählen von Kalendern funktioniert nicht richtig
|
|
||||||
- **Fix:** Checkbox-Toggle Logik reparieren — `visibleCalendars` Set korrekt verwalten
|
|
||||||
- **Aufwand:** 2 Stunden
|
|
||||||
|
|
||||||
**Gesamtaufwand Phase 5:** ~1 Tag
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Phase 6: Tags Umstrukturierung (2 Tage)
|
|
||||||
|
|
||||||
### 6.1 Tags in Settings verschieben
|
|
||||||
- **Datei:** `frontend/src/pages/Tags.tsx` → `frontend/src/pages/SettingsTags.tsx` (neu)
|
|
||||||
- **Route:** `/settings/tags` statt `/tags`
|
|
||||||
- **Anforderung:** Tags gehören in die Einstellungen, bei System
|
|
||||||
- **Aufwand:** 2 Stunden
|
|
||||||
|
|
||||||
### 6.2 Tags Baumstruktur
|
|
||||||
- **Datei:** `frontend/src/pages/SettingsTags.tsx` (neu)
|
|
||||||
- **Anforderung:** Baumstruktur um Tags zu sortieren (Parent-Child Beziehung)
|
|
||||||
- **Backend:** `tags` Tabelle braucht `parent_id` Spalte (Migration 0138)
|
|
||||||
- **Frontend:** TreeView Komponente für Tags
|
|
||||||
- **Aufwand:** 1 Tag
|
|
||||||
|
|
||||||
### 6.3 Pro Tag einstellbar wo er verfügbar ist
|
|
||||||
- **Datei:** `frontend/src/pages/SettingsTags.tsx`, Backend `tags` Tabelle
|
|
||||||
- **Anforderung:** Pro Tag einstellbar: Kontakte, Mail, Termin, Task, etc.
|
|
||||||
- **Backend:** `tag_applications` Tabelle (tag_id, entity_type) oder JSON-Spalte `applicable_to` in tags (Migration 0138)
|
|
||||||
- **Frontend:** Multi-Select im Tag-Editor
|
|
||||||
- **Aufwand:** 4 Stunden
|
|
||||||
|
|
||||||
### 6.4 Symbol und Farbe pro Tag
|
|
||||||
- **Datei:** `frontend/src/pages/SettingsTags.tsx`, Backend `tags` Tabelle
|
|
||||||
- **Anforderung:** Symbol (Icon) und Farbe pro Tag einstellbar
|
|
||||||
- **Backend:** `icon` Spalte in tags (Migration 0138), `color` existiert schon
|
|
||||||
- **Frontend:** Icon-Picker und Color-Picker im Tag-Editor
|
|
||||||
- **Aufwand:** 4 Stunden
|
|
||||||
|
|
||||||
**Gesamtaufwand Phase 6:** ~2 Tage
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Phase 7: Reports UI-Überarbeitung (2 Tage)
|
|
||||||
|
|
||||||
### 7.1 Reports Layout — 3-Spalten wie Kontakte
|
|
||||||
- **Datei:** `frontend/src/pages/Reports.tsx` (Umbau, 433 Zeilen)
|
|
||||||
- **Anforderung:** Linke Sidebar mit Baumstruktur (Ordner zum Sortieren), Mitte verschiedene Ansichten (Liste/Karten), rechts Detailbereich
|
|
||||||
- **Aufbau:**
|
|
||||||
- **Toolbar:** PluginToolbar mit Filter, Ansichts-Umschalter, Neuer Report
|
|
||||||
- **Links:** Baumansicht — nach Ordner/Gruppe sortierbar
|
|
||||||
- **Mitte:** Liste oder Karten-Ansicht — umschaltbar
|
|
||||||
- **Rechts:** ReportDetail — ausgewählter Report mit Vorschau
|
|
||||||
- **Backend:** `reports` Tabelle braucht `folder_id` Spalte (Migration 0139) für Ordner-Sortierung
|
|
||||||
- **Aufwand:** 2 Tage
|
|
||||||
|
|
||||||
**Gesamtaufwand Phase 7:** ~2 Tage
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Phase 8: Kommunikation UI-Überarbeitung (2-3 Tage)
|
|
||||||
|
|
||||||
### 8.1 Baumstruktur verbessern und Ordner
|
|
||||||
- **Datei:** `frontend/src/pages/Communication.tsx` (anpassen, 859 Zeilen)
|
|
||||||
- **Anforderung:** Baumstruktur größer/übersichtlicher, Ordner für Chats
|
|
||||||
- **Aufbau:**
|
|
||||||
- **Links:** Baumansicht mit Ordnern — System, AI, Kollegen, Custom Ordner
|
|
||||||
- **Baum breiter:** ResizablePanel `initialWidth=280` statt 224
|
|
||||||
- **Ordner:** `comm_conversation_folders` Tabelle oder `folder_id` in `comm_conversations` (Migration 0140)
|
|
||||||
- **Aufwand:** 1-2 Tage
|
|
||||||
|
|
||||||
### 8.2 AI Chat in Kommunikation (nach Phase 2)
|
|
||||||
- AI Chats werden als eigener Baum-Knoten 'KI Chats' in Communication angezeigt
|
|
||||||
- Neuer AI Chat Button in Toolbar erstellt `comm_conversation` mit `conversation_type='ai'`
|
|
||||||
- `streamChat()` wird aufgerufen mit `comm_conversation_id` als Session-ID
|
|
||||||
- AI Messages werden in `comm_messages` gespeichert
|
|
||||||
- **Aufwand:** in Phase 2
|
|
||||||
|
|
||||||
**Gesamtaufwand Phase 8:** ~1-2 Tage (Phase 2 vorab)
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Phase 9: Strukturelle Änderungen (0.5 Tage)
|
|
||||||
|
|
||||||
### 9.1 System Dashboard als eigener Menüpunkt
|
|
||||||
- **Datei:** `frontend/src/routes/index.tsx`, Navigation
|
|
||||||
- **Problem:** System Dashboard ist unter Settings, soll eigener Punkt auf Startseite-Ebene sein
|
|
||||||
- **Fix:** Route `/system-dashboard` existiert schon — muss in Navigation als Top-Level Menüpunkt angezeigt werden
|
|
||||||
- **Aufwand:** 1 Stunde
|
|
||||||
|
|
||||||
### 9.2 Mail — Postfach mit IMAP anlegen testen
|
|
||||||
- **Datei:** `frontend/src/pages/Mail.tsx`, `frontend/src/pages/MailSettings.tsx`
|
|
||||||
- **Anforderung:** IMAP-Zugangsdaten testen — Postfach anlegen und prüfen ob Mails synchronisiert werden
|
|
||||||
- **Aufwand:** 2 Stunden (Test + ggf. Bugfix)
|
|
||||||
|
|
||||||
**Gesamtaufwand Phase 9:** ~0.5 Tage
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Zusammenfassung
|
|
||||||
|
|
||||||
| Phase | Inhalt | Aufwand | Migration | Abhängigkeit |
|
|
||||||
|-------|--------|---------|-----------|-------------|
|
|
||||||
| 1 | Echte Bugs fixen | 2-3 Tage | Keine | Keine |
|
|
||||||
| 2 | AI Assistent → Kommunikation | 2-3 Tage | 0137 | Phase 1.6 |
|
|
||||||
| 3 | Wiki UI + WYSIWYG | 3-4 Tage | Keine | Phase 1.4 |
|
|
||||||
| 4 | Tasks UI neu | 2-3 Tage | Keine | Keine |
|
|
||||||
| 5 | Kalender UI | 1 Tag | Keine | Phase 1.5 |
|
|
||||||
| 6 | Tags Umstrukturierung | 2 Tage | 0138 | Keine |
|
|
||||||
| 7 | Reports UI | 2 Tage | 0139 | Keine |
|
|
||||||
| 8 | Kommunikation UI | 1-2 Tage | 0140 | Phase 2 |
|
|
||||||
| 9 | Strukturelle Änderungen | 0.5 Tage | Keine | Keine |
|
|
||||||
|
|
||||||
**Gesamtaufwand:** ~17-22 Tage
|
|
||||||
|
|
||||||
### Reihenfolge:
|
|
||||||
1. **Phase 1** (Bugs) — zuerst, damit grundlegende Funktionen arbeiten
|
|
||||||
2. **Phase 9** (Strukturelle Änderungen) — schnell, wenig Aufwand
|
|
||||||
3. **Phase 5** (Kalender) — kleines Update, baut auf Phase 1 auf
|
|
||||||
4. **Phase 2** (AI Assistent → Kommunikation) — entfernt paralleles System, baut auf Phase 1.6 auf
|
|
||||||
5. **Phase 6** (Tags) — unabhängig, Backend + Frontend
|
|
||||||
6. **Phase 4** (Tasks) — großer Umbau, unabhängig
|
|
||||||
7. **Phase 3** (Wiki) — größter Umbau (WYSIWYG Editor), baut auf Phase 1 auf
|
|
||||||
8. **Phase 7** (Reports) — großer Umbau, unabhängig
|
|
||||||
9. **Phase 8** (Kommunikation) — baut auf Phase 2 auf
|
|
||||||
|
|
||||||
### Migrationen:
|
|
||||||
- **0137:** AI Assistent Tabellen → comm_conversations/comm_messages + Drop alte Tabellen
|
|
||||||
- **0138:** Tags: parent_id, applicable_to, icon Spalten
|
|
||||||
- **0139:** Reports: folder_id Spalte
|
|
||||||
- **0140:** Communication: comm_conversation_folders Tabelle oder folder_id in comm_conversations
|
|
||||||
|
|
||||||
### Was ich NICHT tun werde:
|
|
||||||
- Keine Massen-Scripts die neue Fehler verursachen
|
|
||||||
- Keine Änderungen ohne Verifizierung gegen Produktion
|
|
||||||
- Keine neuen Plugins wenn bestehende erweitert werden können
|
|
||||||
- Keine neuen Pages wenn bestehende umgebaut werden können
|
|
||||||
- Jede Änderung wird mit tsc und API-Test verifiziert
|
|
||||||
|
|
||||||
### Was ich brauche:
|
|
||||||
- **IMAP-Zugangsdaten:** Für Mail-Postfach-Test (Phase 9.2)
|
|
||||||
@@ -18,7 +18,26 @@ branch_labels = None
|
|||||||
depends_on = None
|
depends_on = None
|
||||||
|
|
||||||
|
|
||||||
|
def _table_exists(conn, table_name: str) -> bool:
|
||||||
|
"""True when the table exists (dual-path convergence, Gate B).
|
||||||
|
|
||||||
|
On a fresh install the ai_assistant plugin SQL migration has not run
|
||||||
|
yet when Alembic reaches this revision — skip instead of failing.
|
||||||
|
The plugin-side migration adds the same columns idempotently.
|
||||||
|
"""
|
||||||
|
row = conn.execute(
|
||||||
|
sa.text("SELECT to_regclass(:tname) IS NOT NULL"),
|
||||||
|
{"tname": f"public.{table_name}"},
|
||||||
|
).scalar()
|
||||||
|
return bool(row)
|
||||||
|
|
||||||
|
|
||||||
def upgrade() -> None:
|
def upgrade() -> None:
|
||||||
|
conn = op.get_bind()
|
||||||
|
if not _table_exists(conn, "ai_providers"):
|
||||||
|
# Fresh-install path: table arrives with the ai_assistant plugin
|
||||||
|
# migration, which includes these columns.
|
||||||
|
return
|
||||||
op.add_column("ai_providers", sa.Column("region", sa.String(20), nullable=False, server_default="unknown"))
|
op.add_column("ai_providers", sa.Column("region", sa.String(20), nullable=False, server_default="unknown"))
|
||||||
op.add_column("ai_providers", sa.Column("hosting_type", sa.String(30), nullable=False, server_default="cloud"))
|
op.add_column("ai_providers", sa.Column("hosting_type", sa.String(30), nullable=False, server_default="cloud"))
|
||||||
op.add_column("ai_providers", sa.Column("dpa_status", sa.String(20), nullable=False, server_default="none"))
|
op.add_column("ai_providers", sa.Column("dpa_status", sa.String(20), nullable=False, server_default="none"))
|
||||||
@@ -29,6 +48,9 @@ def upgrade() -> None:
|
|||||||
|
|
||||||
|
|
||||||
def downgrade() -> None:
|
def downgrade() -> None:
|
||||||
|
conn = op.get_bind()
|
||||||
|
if not _table_exists(conn, "ai_providers"):
|
||||||
|
return
|
||||||
op.drop_column("ai_providers", "allowed_data_classes")
|
op.drop_column("ai_providers", "allowed_data_classes")
|
||||||
op.drop_column("ai_providers", "transfer_notice")
|
op.drop_column("ai_providers", "transfer_notice")
|
||||||
op.drop_column("ai_providers", "training_on_customer_data")
|
op.drop_column("ai_providers", "training_on_customer_data")
|
||||||
|
|||||||
@@ -17,7 +17,23 @@ branch_labels = None
|
|||||||
depends_on = None
|
depends_on = None
|
||||||
|
|
||||||
|
|
||||||
|
def _table_exists(conn, table_name: str) -> bool:
|
||||||
|
"""True when the table exists (dual-path convergence, Gate B).
|
||||||
|
|
||||||
|
On a fresh install the kommunikation plugin SQL migration has not run
|
||||||
|
yet when Alembic reaches this revision — skip the comm_* parts instead
|
||||||
|
of failing. The plugin-side migration adds the same column idempotently.
|
||||||
|
"""
|
||||||
|
row = conn.execute(
|
||||||
|
sa.text("SELECT to_regclass(:tname) IS NOT NULL"),
|
||||||
|
{"tname": f"public.{table_name}"},
|
||||||
|
).scalar()
|
||||||
|
return bool(row)
|
||||||
|
|
||||||
|
|
||||||
def upgrade() -> None:
|
def upgrade() -> None:
|
||||||
|
conn = op.get_bind()
|
||||||
|
if _table_exists(conn, "comm_conversations"):
|
||||||
# 1. Add is_system column to comm_conversations
|
# 1. Add is_system column to comm_conversations
|
||||||
op.add_column(
|
op.add_column(
|
||||||
"comm_conversations",
|
"comm_conversations",
|
||||||
@@ -130,13 +146,16 @@ def upgrade() -> None:
|
|||||||
AND n.deleted_at IS NULL;
|
AND n.deleted_at IS NULL;
|
||||||
""")
|
""")
|
||||||
|
|
||||||
# 7. Create legacy view over notifications table for backward compatibility
|
# 7. Legacy view over the CORE notifications table — exists on both paths
|
||||||
op.execute("DROP VIEW IF EXISTS notifications_legacy")
|
op.execute("DROP VIEW IF EXISTS notifications_legacy")
|
||||||
op.execute("CREATE VIEW notifications_legacy AS SELECT * FROM notifications")
|
op.execute("CREATE VIEW notifications_legacy AS SELECT * FROM notifications")
|
||||||
|
|
||||||
|
|
||||||
def downgrade() -> None:
|
def downgrade() -> None:
|
||||||
|
conn = op.get_bind()
|
||||||
op.execute("DROP VIEW IF EXISTS notifications_legacy")
|
op.execute("DROP VIEW IF EXISTS notifications_legacy")
|
||||||
|
if not _table_exists(conn, "comm_conversations"):
|
||||||
|
return
|
||||||
op.execute("DELETE FROM comm_message_blocks WHERE message_id IN (SELECT id FROM comm_messages WHERE metadata->>'migrated_from_notification' = 'true')")
|
op.execute("DELETE FROM comm_message_blocks WHERE message_id IN (SELECT id FROM comm_messages WHERE metadata->>'migrated_from_notification' = 'true')")
|
||||||
op.execute("DELETE FROM comm_messages WHERE metadata->>'migrated_from_notification' = 'true'")
|
op.execute("DELETE FROM comm_messages WHERE metadata->>'migrated_from_notification' = 'true'")
|
||||||
op.execute("DELETE FROM comm_conversations WHERE is_system = true AND title = 'System Channel'")
|
op.execute("DELETE FROM comm_conversations WHERE is_system = true AND title = 'System Channel'")
|
||||||
|
|||||||
@@ -14,7 +14,24 @@ branch_labels = None
|
|||||||
depends_on = None
|
depends_on = None
|
||||||
|
|
||||||
|
|
||||||
|
def _table_exists(conn, table_name: str) -> bool:
|
||||||
|
"""True when the table exists (dual-path convergence, Gate B).
|
||||||
|
|
||||||
|
On a fresh install the automation plugin SQL migration has not run yet
|
||||||
|
when Alembic reaches this revision — skip instead of failing. The
|
||||||
|
plugin-side convergence migration creates the same table.
|
||||||
|
"""
|
||||||
|
row = conn.execute(
|
||||||
|
sa.text("SELECT to_regclass(:tname) IS NOT NULL"),
|
||||||
|
{"tname": f"public.{table_name}"},
|
||||||
|
).scalar()
|
||||||
|
return bool(row)
|
||||||
|
|
||||||
|
|
||||||
def upgrade() -> None:
|
def upgrade() -> None:
|
||||||
|
conn = op.get_bind()
|
||||||
|
if not _table_exists(conn, "automation_agent_runs"):
|
||||||
|
return
|
||||||
op.create_table(
|
op.create_table(
|
||||||
"automation_agent_run_steps",
|
"automation_agent_run_steps",
|
||||||
sa.Column("id", PGUUID(as_uuid=True), primary_key=True),
|
sa.Column("id", PGUUID(as_uuid=True), primary_key=True),
|
||||||
|
|||||||
@@ -18,7 +18,24 @@ branch_labels = None
|
|||||||
depends_on = None
|
depends_on = None
|
||||||
|
|
||||||
|
|
||||||
|
def _table_exists(conn, table_name: str) -> bool:
|
||||||
|
"""True when the table exists (dual-path convergence, Gate B).
|
||||||
|
|
||||||
|
On a fresh install the automation plugin SQL migration has not run yet
|
||||||
|
when Alembic reaches this revision — skip instead of failing. The
|
||||||
|
plugin-side convergence migration adds the same columns.
|
||||||
|
"""
|
||||||
|
row = conn.execute(
|
||||||
|
sa.text("SELECT to_regclass(:tname) IS NOT NULL"),
|
||||||
|
{"tname": f"public.{table_name}"},
|
||||||
|
).scalar()
|
||||||
|
return bool(row)
|
||||||
|
|
||||||
|
|
||||||
def upgrade() -> None:
|
def upgrade() -> None:
|
||||||
|
conn = op.get_bind()
|
||||||
|
if not _table_exists(conn, "automation_agent_definitions"):
|
||||||
|
return
|
||||||
op.add_column(
|
op.add_column(
|
||||||
"automation_agent_definitions",
|
"automation_agent_definitions",
|
||||||
sa.Column("temperature", sa.Float, nullable=False, server_default="0.3"),
|
sa.Column("temperature", sa.Float, nullable=False, server_default="0.3"),
|
||||||
|
|||||||
@@ -20,7 +20,27 @@ branch_labels = None
|
|||||||
depends_on = None
|
depends_on = None
|
||||||
|
|
||||||
|
|
||||||
|
def _table_exists(conn, table_name: str) -> bool:
|
||||||
|
"""True when the table exists (dual-path convergence, Gate B).
|
||||||
|
|
||||||
|
On a fresh install the tasks plugin SQL migration has not run yet when
|
||||||
|
Alembic reaches this revision — skip instead of failing. The plugin-side
|
||||||
|
convergence migration adds the same columns/indexes. The legacy-data
|
||||||
|
backfills below only matter for pre-existing rows and are correctly
|
||||||
|
empty on a fresh install.
|
||||||
|
"""
|
||||||
|
row = conn.execute(
|
||||||
|
sa.text("SELECT to_regclass(:tname) IS NOT NULL"),
|
||||||
|
{"tname": f"public.{table_name}"},
|
||||||
|
).scalar()
|
||||||
|
return bool(row)
|
||||||
|
|
||||||
|
|
||||||
def upgrade() -> None:
|
def upgrade() -> None:
|
||||||
|
conn = op.get_bind()
|
||||||
|
if not _table_exists(conn, "tasks"):
|
||||||
|
return
|
||||||
|
|
||||||
# ── Add new columns to tasks ────────────────────────────────────────────
|
# ── Add new columns to tasks ────────────────────────────────────────────
|
||||||
op.add_column("tasks", sa.Column("assignee_type", sa.String(20), nullable=False, server_default="user"))
|
op.add_column("tasks", sa.Column("assignee_type", sa.String(20), nullable=False, server_default="user"))
|
||||||
op.add_column("tasks", sa.Column("assignee_id", PGUUID(as_uuid=True), nullable=True))
|
op.add_column("tasks", sa.Column("assignee_id", PGUUID(as_uuid=True), nullable=True))
|
||||||
|
|||||||
@@ -12,6 +12,7 @@ Revises: 0126
|
|||||||
"""
|
"""
|
||||||
|
|
||||||
from alembic import op
|
from alembic import op
|
||||||
|
import sqlalchemy as sa
|
||||||
|
|
||||||
revision = "0127"
|
revision = "0127"
|
||||||
down_revision = "0126"
|
down_revision = "0126"
|
||||||
@@ -19,7 +20,19 @@ branch_labels = None
|
|||||||
depends_on = None
|
depends_on = None
|
||||||
|
|
||||||
|
|
||||||
|
def _table_exists(conn, table_name: str) -> bool:
|
||||||
|
"""True when the table exists (dual-path convergence, Gate B)."""
|
||||||
|
row = conn.execute(
|
||||||
|
sa.text("SELECT to_regclass(:tname) IS NOT NULL"),
|
||||||
|
{"tname": f"public.{table_name}"},
|
||||||
|
).scalar()
|
||||||
|
return bool(row)
|
||||||
|
|
||||||
|
|
||||||
def upgrade() -> None:
|
def upgrade() -> None:
|
||||||
|
conn = op.get_bind()
|
||||||
|
if not _table_exists(conn, "tasks"):
|
||||||
|
return
|
||||||
# Drop the FK constraint on tasks.contact_id
|
# Drop the FK constraint on tasks.contact_id
|
||||||
op.drop_constraint("tasks_contact_id_fkey", "tasks", type_="foreignkey")
|
op.drop_constraint("tasks_contact_id_fkey", "tasks", type_="foreignkey")
|
||||||
|
|
||||||
|
|||||||
@@ -9,6 +9,7 @@ Revises: 0128
|
|||||||
"""
|
"""
|
||||||
|
|
||||||
from alembic import op
|
from alembic import op
|
||||||
|
import sqlalchemy as sa
|
||||||
|
|
||||||
revision = "0129"
|
revision = "0129"
|
||||||
down_revision = "0128"
|
down_revision = "0128"
|
||||||
@@ -27,8 +28,25 @@ TABLES_NEEDING_RLS = [
|
|||||||
]
|
]
|
||||||
|
|
||||||
|
|
||||||
|
def _table_exists(conn, table_name: str) -> bool:
|
||||||
|
"""True when the table exists (dual-path convergence, Gate B).
|
||||||
|
|
||||||
|
Plugin-owned tables may not exist yet on a fresh install when Alembic
|
||||||
|
reaches this revision — skip them instead of failing. The plugin-side
|
||||||
|
convergence migrations apply the same RLS policies.
|
||||||
|
"""
|
||||||
|
row = conn.execute(
|
||||||
|
sa.text("SELECT to_regclass(:tname) IS NOT NULL"),
|
||||||
|
{"tname": f"public.{table_name}"},
|
||||||
|
).scalar()
|
||||||
|
return bool(row)
|
||||||
|
|
||||||
|
|
||||||
def upgrade() -> None:
|
def upgrade() -> None:
|
||||||
|
conn = op.get_bind()
|
||||||
for table in TABLES_NEEDING_RLS:
|
for table in TABLES_NEEDING_RLS:
|
||||||
|
if not _table_exists(conn, table):
|
||||||
|
continue
|
||||||
op.execute(f"ALTER TABLE {table} ENABLE ROW LEVEL SECURITY;")
|
op.execute(f"ALTER TABLE {table} ENABLE ROW LEVEL SECURITY;")
|
||||||
op.execute(
|
op.execute(
|
||||||
f"CREATE POLICY tenant_isolation ON {table} "
|
f"CREATE POLICY tenant_isolation ON {table} "
|
||||||
@@ -37,6 +55,9 @@ def upgrade() -> None:
|
|||||||
|
|
||||||
|
|
||||||
def downgrade() -> None:
|
def downgrade() -> None:
|
||||||
|
conn = op.get_bind()
|
||||||
for table in TABLES_NEEDING_RLS:
|
for table in TABLES_NEEDING_RLS:
|
||||||
|
if not _table_exists(conn, table):
|
||||||
|
continue
|
||||||
op.execute(f"DROP POLICY IF EXISTS tenant_isolation ON {table};")
|
op.execute(f"DROP POLICY IF EXISTS tenant_isolation ON {table};")
|
||||||
op.execute(f"ALTER TABLE {table} DISABLE ROW LEVEL SECURITY;")
|
op.execute(f"ALTER TABLE {table} DISABLE ROW LEVEL SECURITY;")
|
||||||
|
|||||||
@@ -10,6 +10,7 @@ Revises: 0135
|
|||||||
Create Date: 2026-08-21
|
Create Date: 2026-08-21
|
||||||
"""
|
"""
|
||||||
from alembic import op
|
from alembic import op
|
||||||
|
import sqlalchemy as sa
|
||||||
|
|
||||||
revision = "0136"
|
revision = "0136"
|
||||||
down_revision = "0135"
|
down_revision = "0135"
|
||||||
@@ -29,8 +30,25 @@ TABLES_WITH_BAD_RLS = [
|
|||||||
]
|
]
|
||||||
|
|
||||||
|
|
||||||
|
def _table_exists(conn, table_name: str) -> bool:
|
||||||
|
"""True when the table exists (dual-path convergence, Gate B).
|
||||||
|
|
||||||
|
Plugin-owned tables may not exist yet on a fresh install when Alembic
|
||||||
|
reaches this revision — skip them instead of failing. The plugin-side
|
||||||
|
convergence migrations apply the same RLS policies.
|
||||||
|
"""
|
||||||
|
row = conn.execute(
|
||||||
|
sa.text("SELECT to_regclass(:tname) IS NOT NULL"),
|
||||||
|
{"tname": f"public.{table_name}"},
|
||||||
|
).scalar()
|
||||||
|
return bool(row)
|
||||||
|
|
||||||
|
|
||||||
def upgrade() -> None:
|
def upgrade() -> None:
|
||||||
|
conn = op.get_bind()
|
||||||
for table in TABLES_WITH_BAD_RLS:
|
for table in TABLES_WITH_BAD_RLS:
|
||||||
|
if not _table_exists(conn, table):
|
||||||
|
continue
|
||||||
# Drop old policy with app.tenant_id
|
# Drop old policy with app.tenant_id
|
||||||
op.execute(f"DROP POLICY IF EXISTS tenant_isolation ON {table};")
|
op.execute(f"DROP POLICY IF EXISTS tenant_isolation ON {table};")
|
||||||
# Create new policy with app.current_tenant_id
|
# Create new policy with app.current_tenant_id
|
||||||
@@ -41,7 +59,10 @@ def upgrade() -> None:
|
|||||||
|
|
||||||
|
|
||||||
def downgrade() -> None:
|
def downgrade() -> None:
|
||||||
|
conn = op.get_bind()
|
||||||
for table in TABLES_WITH_BAD_RLS:
|
for table in TABLES_WITH_BAD_RLS:
|
||||||
|
if not _table_exists(conn, table):
|
||||||
|
continue
|
||||||
op.execute(f"DROP POLICY IF EXISTS tenant_isolation ON {table};")
|
op.execute(f"DROP POLICY IF EXISTS tenant_isolation ON {table};")
|
||||||
op.execute(
|
op.execute(
|
||||||
f"CREATE POLICY tenant_isolation ON {table} "
|
f"CREATE POLICY tenant_isolation ON {table} "
|
||||||
|
|||||||
@@ -19,7 +19,25 @@ branch_labels = None
|
|||||||
depends_on = None
|
depends_on = None
|
||||||
|
|
||||||
|
|
||||||
|
def _table_exists(conn, table_name: str) -> bool:
|
||||||
|
"""True when the table exists (dual-path convergence, Gate B).
|
||||||
|
|
||||||
|
On a fresh install the tags plugin SQL migration has not run yet when
|
||||||
|
Alembic reaches this revision — skip instead of failing. The plugin-side
|
||||||
|
convergence migration adds the same columns.
|
||||||
|
"""
|
||||||
|
row = conn.execute(
|
||||||
|
sa.text("SELECT to_regclass(:tname) IS NOT NULL"),
|
||||||
|
{"tname": f"public.{table_name}"},
|
||||||
|
).scalar()
|
||||||
|
return bool(row)
|
||||||
|
|
||||||
|
|
||||||
def upgrade() -> None:
|
def upgrade() -> None:
|
||||||
|
conn = op.get_bind()
|
||||||
|
if not _table_exists(conn, "tags"):
|
||||||
|
return
|
||||||
|
|
||||||
# parent_id for tree structure (self-referencing FK)
|
# parent_id for tree structure (self-referencing FK)
|
||||||
op.add_column("tags", sa.Column("parent_id", PGUUID(as_uuid=True), nullable=True))
|
op.add_column("tags", sa.Column("parent_id", PGUUID(as_uuid=True), nullable=True))
|
||||||
op.create_foreign_key(
|
op.create_foreign_key(
|
||||||
@@ -35,6 +53,9 @@ def upgrade() -> None:
|
|||||||
|
|
||||||
|
|
||||||
def downgrade() -> None:
|
def downgrade() -> None:
|
||||||
|
conn = op.get_bind()
|
||||||
|
if not _table_exists(conn, "tags"):
|
||||||
|
return
|
||||||
op.drop_column("tags", "icon")
|
op.drop_column("tags", "icon")
|
||||||
op.drop_column("tags", "applicable_to")
|
op.drop_column("tags", "applicable_to")
|
||||||
op.drop_index("ix_tags_parent", table_name="tags")
|
op.drop_index("ix_tags_parent", table_name="tags")
|
||||||
|
|||||||
@@ -18,11 +18,31 @@ branch_labels = None
|
|||||||
depends_on = None
|
depends_on = None
|
||||||
|
|
||||||
|
|
||||||
|
def _table_exists(conn, table_name: str) -> bool:
|
||||||
|
"""True when the table exists (dual-path convergence, Gate B).
|
||||||
|
|
||||||
|
On a fresh install the report_generator plugin SQL migration has not
|
||||||
|
run yet when Alembic reaches this revision — skip instead of failing.
|
||||||
|
The plugin-side convergence migration adds the same column.
|
||||||
|
"""
|
||||||
|
row = conn.execute(
|
||||||
|
sa.text("SELECT to_regclass(:tname) IS NOT NULL"),
|
||||||
|
{"tname": f"public.{table_name}"},
|
||||||
|
).scalar()
|
||||||
|
return bool(row)
|
||||||
|
|
||||||
|
|
||||||
def upgrade() -> None:
|
def upgrade() -> None:
|
||||||
|
conn = op.get_bind()
|
||||||
|
if not _table_exists(conn, "report_templates"):
|
||||||
|
return
|
||||||
op.add_column("report_templates", sa.Column("folder_id", PGUUID(as_uuid=True), nullable=True))
|
op.add_column("report_templates", sa.Column("folder_id", PGUUID(as_uuid=True), nullable=True))
|
||||||
op.create_index("ix_report_templates_folder", "report_templates", ["folder_id"])
|
op.create_index("ix_report_templates_folder", "report_templates", ["folder_id"])
|
||||||
|
|
||||||
|
|
||||||
def downgrade() -> None:
|
def downgrade() -> None:
|
||||||
|
conn = op.get_bind()
|
||||||
|
if not _table_exists(conn, "report_templates"):
|
||||||
|
return
|
||||||
op.drop_index("ix_report_templates_folder", table_name="report_templates")
|
op.drop_index("ix_report_templates_folder", table_name="report_templates")
|
||||||
op.drop_column("report_templates", "folder_id")
|
op.drop_column("report_templates", "folder_id")
|
||||||
|
|||||||
@@ -18,11 +18,31 @@ branch_labels = None
|
|||||||
depends_on = None
|
depends_on = None
|
||||||
|
|
||||||
|
|
||||||
|
def _table_exists(conn, table_name: str) -> bool:
|
||||||
|
"""True when the table exists (dual-path convergence, Gate B).
|
||||||
|
|
||||||
|
On a fresh install the kommunikation plugin SQL migration has not run
|
||||||
|
yet when Alembic reaches this revision — skip instead of failing.
|
||||||
|
The plugin-side migration adds the same column idempotently.
|
||||||
|
"""
|
||||||
|
row = conn.execute(
|
||||||
|
sa.text("SELECT to_regclass(:tname) IS NOT NULL"),
|
||||||
|
{"tname": f"public.{table_name}"},
|
||||||
|
).scalar()
|
||||||
|
return bool(row)
|
||||||
|
|
||||||
|
|
||||||
def upgrade() -> None:
|
def upgrade() -> None:
|
||||||
|
conn = op.get_bind()
|
||||||
|
if not _table_exists(conn, "comm_conversations"):
|
||||||
|
return
|
||||||
op.add_column("comm_conversations", sa.Column("folder_id", PGUUID(as_uuid=True), nullable=True))
|
op.add_column("comm_conversations", sa.Column("folder_id", PGUUID(as_uuid=True), nullable=True))
|
||||||
op.create_index("ix_comm_conversations_folder", "comm_conversations", ["folder_id"])
|
op.create_index("ix_comm_conversations_folder", "comm_conversations", ["folder_id"])
|
||||||
|
|
||||||
|
|
||||||
def downgrade() -> None:
|
def downgrade() -> None:
|
||||||
|
conn = op.get_bind()
|
||||||
|
if not _table_exists(conn, "comm_conversations"):
|
||||||
|
return
|
||||||
op.drop_index("ix_comm_conversations_folder", table_name="comm_conversations")
|
op.drop_index("ix_comm_conversations_folder", table_name="comm_conversations")
|
||||||
op.drop_column("comm_conversations", "folder_id")
|
op.drop_column("comm_conversations", "folder_id")
|
||||||
|
|||||||
@@ -0,0 +1,84 @@
|
|||||||
|
'''Fix role permission wildcard patterns to canonical 2-segment schema
|
||||||
|
|
||||||
|
Revision ID: 0141
|
||||||
|
Revises: 0140
|
||||||
|
Create Date: 2026-08-23
|
||||||
|
|
||||||
|
Migration 0019 seeded default roles with 3-segment permission patterns
|
||||||
|
(core:*:read etc.). The runtime matcher (_matches_permission) compares
|
||||||
|
segment counts strictly, so those patterns could never match any
|
||||||
|
2-segment requirement - editor/viewer roles were silently dead.
|
||||||
|
|
||||||
|
Canonical schema is module:action (2 segments, * wildcards allowed).
|
||||||
|
core:*:X means all modules with action X, so it converts to *:X.
|
||||||
|
'''
|
||||||
|
|
||||||
|
from alembic import op
|
||||||
|
|
||||||
|
# revision identifiers, used by Alembic.
|
||||||
|
revision = '0141'
|
||||||
|
down_revision = '0140'
|
||||||
|
branch_labels = None
|
||||||
|
depends_on = None
|
||||||
|
|
||||||
|
# Rebuild the permissions JSONB object, rewriting every key that starts
|
||||||
|
# with the dead 'core:' prefix to its 2-segment equivalent ('*:X').
|
||||||
|
_UPGRADE_SQL = '''
|
||||||
|
UPDATE roles
|
||||||
|
SET permissions = sub.new_perms,
|
||||||
|
permission_version = permission_version + 1
|
||||||
|
FROM (
|
||||||
|
SELECT
|
||||||
|
r.id AS role_id,
|
||||||
|
jsonb_object_agg(
|
||||||
|
CASE WHEN k LIKE 'core:%'
|
||||||
|
THEN '*:' || split_part(k, ':', 3)
|
||||||
|
ELSE k END,
|
||||||
|
v
|
||||||
|
) AS new_perms
|
||||||
|
FROM roles r,
|
||||||
|
jsonb_each(r.permissions) AS e(k, v)
|
||||||
|
GROUP BY r.id
|
||||||
|
) AS sub
|
||||||
|
WHERE roles.id = sub.role_id
|
||||||
|
AND EXISTS (
|
||||||
|
SELECT 1 FROM jsonb_object_keys(roles.permissions) k
|
||||||
|
WHERE k LIKE 'core:%'
|
||||||
|
)
|
||||||
|
'''
|
||||||
|
|
||||||
|
# Reverse: map '*:X' back to 'core:*:X' only for keys that came from the
|
||||||
|
# original seeding pattern. Roles that legitimately use '*:X' without a
|
||||||
|
# matching 'core:*:X' history are left untouched (best-effort downgrade).
|
||||||
|
_DOWNGRADE_SQL = '''
|
||||||
|
UPDATE roles
|
||||||
|
SET permissions = sub.new_perms,
|
||||||
|
permission_version = permission_version + 1
|
||||||
|
FROM (
|
||||||
|
SELECT
|
||||||
|
r.id AS role_id,
|
||||||
|
jsonb_object_agg(
|
||||||
|
CASE WHEN k = '*:' || split_part(k, ':', 2)
|
||||||
|
AND k <> '*:*'
|
||||||
|
THEN 'core:*:' || split_part(k, ':', 2)
|
||||||
|
ELSE k END,
|
||||||
|
v
|
||||||
|
) AS new_perms
|
||||||
|
FROM roles r,
|
||||||
|
jsonb_each(r.permissions) AS e(k, v)
|
||||||
|
GROUP BY r.id
|
||||||
|
) AS sub
|
||||||
|
WHERE roles.id = sub.role_id
|
||||||
|
AND EXISTS (
|
||||||
|
SELECT 1 FROM jsonb_object_keys(roles.permissions) k
|
||||||
|
WHERE k = '*:' || split_part(k, ':', 2) AND k <> '*:*'
|
||||||
|
)
|
||||||
|
'''
|
||||||
|
|
||||||
|
|
||||||
|
def upgrade() -> None:
|
||||||
|
op.execute(_UPGRADE_SQL)
|
||||||
|
|
||||||
|
|
||||||
|
def downgrade() -> None:
|
||||||
|
op.execute(_DOWNGRADE_SQL)
|
||||||
@@ -0,0 +1,39 @@
|
|||||||
|
"""Add backup config columns to system_settings table.
|
||||||
|
|
||||||
|
Follow-up to 0130: the backup feature (10b1f83) added backup_interval,
|
||||||
|
backup_retention_days and backup_destination to schema/service/frontend
|
||||||
|
but missed model columns and this migration.
|
||||||
|
|
||||||
|
Revision ID: 0142
|
||||||
|
Revises: 0141
|
||||||
|
"""
|
||||||
|
|
||||||
|
import sqlalchemy as sa
|
||||||
|
|
||||||
|
from alembic import op
|
||||||
|
|
||||||
|
revision = "0142"
|
||||||
|
down_revision = "0141"
|
||||||
|
branch_labels = None
|
||||||
|
depends_on = None
|
||||||
|
|
||||||
|
|
||||||
|
def upgrade() -> None:
|
||||||
|
op.add_column(
|
||||||
|
"system_settings",
|
||||||
|
sa.Column("backup_interval", sa.String(20), nullable=False, server_default="daily"),
|
||||||
|
)
|
||||||
|
op.add_column(
|
||||||
|
"system_settings",
|
||||||
|
sa.Column("backup_retention_days", sa.Integer(), nullable=False, server_default="7"),
|
||||||
|
)
|
||||||
|
op.add_column(
|
||||||
|
"system_settings",
|
||||||
|
sa.Column("backup_destination", sa.String(20), nullable=False, server_default="local"),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def downgrade() -> None:
|
||||||
|
op.drop_column("system_settings", "backup_destination")
|
||||||
|
op.drop_column("system_settings", "backup_retention_days")
|
||||||
|
op.drop_column("system_settings", "backup_interval")
|
||||||
@@ -17,7 +17,7 @@ from __future__ import annotations
|
|||||||
|
|
||||||
import logging
|
import logging
|
||||||
import uuid
|
import uuid
|
||||||
from dataclasses import dataclass, field
|
from dataclasses import dataclass
|
||||||
from typing import Any
|
from typing import Any
|
||||||
|
|
||||||
from sqlalchemy import select
|
from sqlalchemy import select
|
||||||
|
|||||||
@@ -25,8 +25,6 @@ import logging
|
|||||||
import uuid
|
import uuid
|
||||||
from typing import TYPE_CHECKING, Any
|
from typing import TYPE_CHECKING, Any
|
||||||
|
|
||||||
from app.core.sensitive_data import sanitize_dict
|
|
||||||
|
|
||||||
if TYPE_CHECKING:
|
if TYPE_CHECKING:
|
||||||
from sqlalchemy.ext.asyncio import AsyncSession
|
from sqlalchemy.ext.asyncio import AsyncSession
|
||||||
|
|
||||||
|
|||||||
@@ -20,8 +20,6 @@ from sqlalchemy.ext.asyncio import AsyncSession
|
|||||||
from app.ai.ai_use_case import AIUseCaseMetadata
|
from app.ai.ai_use_case import AIUseCaseMetadata
|
||||||
from app.core.sensitive_data import (
|
from app.core.sensitive_data import (
|
||||||
SENSITIVE_FIELDS,
|
SENSITIVE_FIELDS,
|
||||||
filter_for_llm_context,
|
|
||||||
get_data_class_for_field,
|
|
||||||
)
|
)
|
||||||
|
|
||||||
logger = logging.getLogger(__name__)
|
logger = logging.getLogger(__name__)
|
||||||
|
|||||||
@@ -6,7 +6,6 @@ import uuid
|
|||||||
from dataclasses import dataclass, field
|
from dataclasses import dataclass, field
|
||||||
from typing import Any
|
from typing import Any
|
||||||
|
|
||||||
|
|
||||||
LOW_CONFIDENCE_THRESHOLD = 0.6
|
LOW_CONFIDENCE_THRESHOLD = 0.6
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -7,7 +7,6 @@ from typing import Any
|
|||||||
|
|
||||||
from app.ai.knowledge_sources import get_source_config
|
from app.ai.knowledge_sources import get_source_config
|
||||||
|
|
||||||
|
|
||||||
EXTRACTION_TRIGGERS = {
|
EXTRACTION_TRIGGERS = {
|
||||||
"mail.received",
|
"mail.received",
|
||||||
"dms.file_uploaded",
|
"dms.file_uploaded",
|
||||||
|
|||||||
@@ -2,7 +2,7 @@
|
|||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
from dataclasses import dataclass, field
|
from dataclasses import dataclass
|
||||||
from typing import Any
|
from typing import Any
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -107,6 +107,10 @@ class Settings(BaseSettings):
|
|||||||
rate_limit_webhook_max: int = 100 # incoming webhooks
|
rate_limit_webhook_max: int = 100 # incoming webhooks
|
||||||
rate_limit_webhook_window: int = 60 # 1 minute
|
rate_limit_webhook_window: int = 60 # 1 minute
|
||||||
|
|
||||||
|
# System tenant — used by seeding/plugins that need a well-known default
|
||||||
|
# tenant (must match scripts/seed_admin.py slug).
|
||||||
|
system_tenant_slug: str = "default"
|
||||||
|
|
||||||
# LLM Cost Overrun Protection (B.17)
|
# LLM Cost Overrun Protection (B.17)
|
||||||
llm_monthly_budget_usd: float = 100.0 # per-tenant monthly LLM budget
|
llm_monthly_budget_usd: float = 100.0 # per-tenant monthly LLM budget
|
||||||
llm_hard_cutoff: bool = True # block LLM calls when budget exceeded
|
llm_hard_cutoff: bool = True # block LLM calls when budget exceeded
|
||||||
|
|||||||
@@ -85,6 +85,38 @@ def generate_csrf_token() -> str:
|
|||||||
return secrets.token_urlsafe(32)
|
return secrets.token_urlsafe(32)
|
||||||
|
|
||||||
|
|
||||||
|
async def revoke_user_redis_sessions(user_id: str | uuid.UUID) -> int:
|
||||||
|
"""Delete every active Redis session belonging to the user (G2).
|
||||||
|
|
||||||
|
Shared by both password-change paths (token reset + profile/admin change):
|
||||||
|
after a password change, stolen or lingering sessions must die.
|
||||||
|
|
||||||
|
Returns the number of deleted session keys. Never raises — a Redis outage
|
||||||
|
must not break the password change itself.
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
redis = get_redis()
|
||||||
|
deleted = 0
|
||||||
|
async for key in redis.scan_iter(match="session:*", count=100):
|
||||||
|
raw = await redis.get(key)
|
||||||
|
if raw is None:
|
||||||
|
continue
|
||||||
|
try:
|
||||||
|
import json
|
||||||
|
|
||||||
|
session_data = json.loads(raw)
|
||||||
|
except (json.JSONDecodeError, TypeError):
|
||||||
|
continue
|
||||||
|
if session_data.get("user_id") == str(user_id):
|
||||||
|
await redis.delete(key)
|
||||||
|
deleted += 1
|
||||||
|
logger.info("Deleted session %s for user %s", key, user_id)
|
||||||
|
return deleted
|
||||||
|
except Exception:
|
||||||
|
logger.warning("Failed to invalidate Redis sessions for user %s", user_id, exc_info=True)
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
def hash_token(token: str) -> str:
|
def hash_token(token: str) -> str:
|
||||||
"""SHA-256 hash a token for storage."""
|
"""SHA-256 hash a token for storage."""
|
||||||
return hashlib.sha256(token.encode()).hexdigest()
|
return hashlib.sha256(token.encode()).hexdigest()
|
||||||
|
|||||||
@@ -355,6 +355,23 @@ async def close_engine() -> None:
|
|||||||
_migration_session_factory = None
|
_migration_session_factory = None
|
||||||
|
|
||||||
|
|
||||||
|
async def get_system_tenant(db: AsyncSession):
|
||||||
|
"""Return the well-known system tenant, or ``None`` if it does not exist.
|
||||||
|
|
||||||
|
Resolves by configured slug (``settings.system_tenant_slug``, default
|
||||||
|
``"default"`` as created by ``scripts/seed_admin.py``) instead of an
|
||||||
|
arbitrary first row, so multi-tenant databases stay deterministic.
|
||||||
|
"""
|
||||||
|
from sqlalchemy import select
|
||||||
|
|
||||||
|
from app.config import get_settings
|
||||||
|
from app.models.tenant import Tenant # lazy: models import this module's Base
|
||||||
|
|
||||||
|
slug = get_settings().system_tenant_slug
|
||||||
|
result = await db.execute(select(Tenant).where(Tenant.slug == slug).limit(1))
|
||||||
|
return result.scalar_one_or_none()
|
||||||
|
|
||||||
|
|
||||||
def reset_engine_for_testing(engine: AsyncEngine) -> async_sessionmaker[AsyncSession]:
|
def reset_engine_for_testing(engine: AsyncEngine) -> async_sessionmaker[AsyncSession]:
|
||||||
"""Replace all global engines with a test engine. Returns a session factory.
|
"""Replace all global engines with a test engine. Returns a session factory.
|
||||||
|
|
||||||
|
|||||||
@@ -36,7 +36,12 @@ class EventBus:
|
|||||||
self._handlers: dict[str, list[EventHandler]] = defaultdict(list)
|
self._handlers: dict[str, list[EventHandler]] = defaultdict(list)
|
||||||
|
|
||||||
def subscribe(self, event_name: str, handler: EventHandler) -> None:
|
def subscribe(self, event_name: str, handler: EventHandler) -> None:
|
||||||
"""Subscribe a handler to an event."""
|
"""Subscribe a handler to an event.
|
||||||
|
|
||||||
|
Idempotent: subscribing the same handler twice is a no-op
|
||||||
|
(ARCH-020) so double activation cannot fire handlers twice.
|
||||||
|
"""
|
||||||
|
if handler not in self._handlers[event_name]:
|
||||||
self._handlers[event_name].append(handler)
|
self._handlers[event_name].append(handler)
|
||||||
|
|
||||||
def unsubscribe(self, event_name: str, handler: EventHandler) -> None:
|
def unsubscribe(self, event_name: str, handler: EventHandler) -> None:
|
||||||
|
|||||||
@@ -153,3 +153,378 @@ async def send_password_reset_email(
|
|||||||
from app.core.job_registry import register_job # noqa: E402
|
from app.core.job_registry import register_job # noqa: E402
|
||||||
|
|
||||||
register_job("send_password_reset_email", send_password_reset_email)
|
register_job("send_password_reset_email", send_password_reset_email)
|
||||||
|
|
||||||
|
|
||||||
|
# ── DSAR Processing Job (G1 DSGVO: Art. 15 Auskunft / Art. 17 Löschung) ─────
|
||||||
|
|
||||||
|
async def _dsar_collect_user_data(db: Any, tenant_id: str, user_id: str) -> dict[str, Any]:
|
||||||
|
"""Collect every data category the dsgvo-export route promises.
|
||||||
|
|
||||||
|
Shared by type=access (full export) so both paths stay consistent.
|
||||||
|
"""
|
||||||
|
from datetime import UTC, datetime
|
||||||
|
from uuid import UUID as PyUUID
|
||||||
|
|
||||||
|
from sqlalchemy import or_ as sa_or_
|
||||||
|
from sqlalchemy import select as sa_select
|
||||||
|
|
||||||
|
from app.models.audit import AuditLog
|
||||||
|
from app.models.contact import Contact
|
||||||
|
from app.models.notification import Notification
|
||||||
|
from app.models.user import User
|
||||||
|
|
||||||
|
uid = PyUUID(user_id)
|
||||||
|
tid = PyUUID(tenant_id)
|
||||||
|
|
||||||
|
export_data: dict[str, Any] = {
|
||||||
|
"user_id": user_id,
|
||||||
|
"exported_at": datetime.now(UTC).isoformat(),
|
||||||
|
"legal_basis": "GDPR Art. 15 (access) / Art. 20 (portability)",
|
||||||
|
"data": {},
|
||||||
|
}
|
||||||
|
|
||||||
|
# Profile
|
||||||
|
user = (
|
||||||
|
await db.execute(sa_select(User).where(User.id == uid))
|
||||||
|
).scalar_one_or_none()
|
||||||
|
if user:
|
||||||
|
export_data["data"]["profile"] = {
|
||||||
|
"email": user.email,
|
||||||
|
"name": user.name,
|
||||||
|
"is_active": user.is_active,
|
||||||
|
"created_at": user.created_at.isoformat() if user.created_at else None,
|
||||||
|
}
|
||||||
|
|
||||||
|
# Contacts owned by the user
|
||||||
|
contacts = (
|
||||||
|
await db.execute(
|
||||||
|
sa_select(Contact).where(
|
||||||
|
Contact.tenant_id == tid,
|
||||||
|
Contact.owner_id == uid,
|
||||||
|
Contact.deleted_at.is_(None),
|
||||||
|
)
|
||||||
|
)
|
||||||
|
).scalars().all()
|
||||||
|
export_data["data"]["contacts"] = [
|
||||||
|
{
|
||||||
|
"id": str(c.id),
|
||||||
|
"type": c.type,
|
||||||
|
"displayname": c.displayname,
|
||||||
|
"email_1": c.email_1,
|
||||||
|
"email_2": c.email_2,
|
||||||
|
}
|
||||||
|
for c in contacts
|
||||||
|
]
|
||||||
|
|
||||||
|
# Audit trail entries by/about the user (bounded to keep payloads sane)
|
||||||
|
audit_entries = (
|
||||||
|
await db.execute(
|
||||||
|
sa_select(AuditLog).where(
|
||||||
|
AuditLog.tenant_id == tid,
|
||||||
|
AuditLog.user_id == uid,
|
||||||
|
).limit(1000)
|
||||||
|
)
|
||||||
|
).scalars().all()
|
||||||
|
export_data["data"]["audit_log"] = [
|
||||||
|
{
|
||||||
|
"action": a.action,
|
||||||
|
"entity_type": a.entity_type,
|
||||||
|
"timestamp": a.timestamp.isoformat() if a.timestamp else None,
|
||||||
|
}
|
||||||
|
for a in audit_entries
|
||||||
|
]
|
||||||
|
|
||||||
|
# Notifications addressed to the user
|
||||||
|
notifications = (
|
||||||
|
await db.execute(
|
||||||
|
sa_select(Notification).where(
|
||||||
|
Notification.tenant_id == tid,
|
||||||
|
Notification.owner_id == uid,
|
||||||
|
).limit(1000)
|
||||||
|
)
|
||||||
|
).scalars().all()
|
||||||
|
export_data["data"]["notifications"] = [
|
||||||
|
{
|
||||||
|
"id": str(n.id),
|
||||||
|
"type": getattr(n, "type", None),
|
||||||
|
"title": getattr(n, "title", None),
|
||||||
|
"created_at": n.created_at.isoformat() if n.created_at else None,
|
||||||
|
}
|
||||||
|
for n in notifications
|
||||||
|
]
|
||||||
|
|
||||||
|
# ── Categories promised by the dsgvo-export route docstring ──
|
||||||
|
# (G1-b: mail accounts, tasks, calendar entries, comm messages)
|
||||||
|
try:
|
||||||
|
from app.plugins.builtins.mail.models import MailAccount
|
||||||
|
|
||||||
|
mail_accounts = (
|
||||||
|
await db.execute(
|
||||||
|
sa_select(MailAccount).where(
|
||||||
|
MailAccount.tenant_id == tid,
|
||||||
|
MailAccount.user_id == uid,
|
||||||
|
).limit(500)
|
||||||
|
)
|
||||||
|
).scalars().all()
|
||||||
|
export_data["data"]["mail_accounts"] = [
|
||||||
|
{
|
||||||
|
"id": str(a.id),
|
||||||
|
"email_address": a.email_address,
|
||||||
|
"display_name": a.display_name,
|
||||||
|
"is_shared": a.is_shared,
|
||||||
|
"is_active": a.is_active,
|
||||||
|
}
|
||||||
|
for a in mail_accounts
|
||||||
|
]
|
||||||
|
except ImportError:
|
||||||
|
pass
|
||||||
|
|
||||||
|
try:
|
||||||
|
from app.plugins.builtins.tasks.models import Task as TaskModel
|
||||||
|
|
||||||
|
tasks = (
|
||||||
|
await db.execute(
|
||||||
|
sa_select(TaskModel).where(
|
||||||
|
sa_or_(TaskModel.owner_id == uid, TaskModel.assigned_to == uid),
|
||||||
|
TaskModel.tenant_id == tid,
|
||||||
|
TaskModel.deleted_at.is_(None),
|
||||||
|
).limit(1000)
|
||||||
|
)
|
||||||
|
).scalars().all()
|
||||||
|
export_data["data"]["tasks"] = [
|
||||||
|
{
|
||||||
|
"id": str(t.id),
|
||||||
|
"title": t.title,
|
||||||
|
"status": t.status,
|
||||||
|
"priority": t.priority,
|
||||||
|
"due_date": t.due_date.isoformat() if t.due_date else None,
|
||||||
|
}
|
||||||
|
for t in tasks
|
||||||
|
]
|
||||||
|
except ImportError:
|
||||||
|
pass
|
||||||
|
|
||||||
|
try:
|
||||||
|
from app.plugins.builtins.calendar.models import CalendarEntry as CalEntry
|
||||||
|
|
||||||
|
cal_entries = (
|
||||||
|
await db.execute(
|
||||||
|
sa_select(CalEntry).where(
|
||||||
|
CalEntry.tenant_id == tid,
|
||||||
|
CalEntry.owner_id == uid,
|
||||||
|
CalEntry.deleted_at.is_(None),
|
||||||
|
).limit(1000)
|
||||||
|
)
|
||||||
|
).scalars().all()
|
||||||
|
export_data["data"]["calendar_entries"] = [
|
||||||
|
{
|
||||||
|
"id": str(e.id),
|
||||||
|
"title": e.title,
|
||||||
|
"entry_type": e.entry_type,
|
||||||
|
"start_at": e.start_at.isoformat() if e.start_at else None,
|
||||||
|
"end_at": e.end_at.isoformat() if e.end_at else None,
|
||||||
|
}
|
||||||
|
for e in cal_entries
|
||||||
|
]
|
||||||
|
except ImportError:
|
||||||
|
pass
|
||||||
|
|
||||||
|
try:
|
||||||
|
from app.plugins.builtins.kommunikation.models import CommMessage
|
||||||
|
|
||||||
|
comm_messages = (
|
||||||
|
await db.execute(
|
||||||
|
sa_select(CommMessage).where(
|
||||||
|
CommMessage.sender_id == uid,
|
||||||
|
CommMessage.tenant_id == tid,
|
||||||
|
).limit(1000)
|
||||||
|
)
|
||||||
|
).scalars().all()
|
||||||
|
export_data["data"]["comm_messages"] = [
|
||||||
|
{
|
||||||
|
"id": str(m.id),
|
||||||
|
"sender_type": m.sender_type,
|
||||||
|
"content": m.content[:500],
|
||||||
|
"created_at": m.created_at.isoformat() if m.created_at else None,
|
||||||
|
}
|
||||||
|
for m in comm_messages
|
||||||
|
]
|
||||||
|
except ImportError:
|
||||||
|
pass
|
||||||
|
|
||||||
|
return export_data
|
||||||
|
|
||||||
|
|
||||||
|
async def _dsar_execute_deletion(db: Any, tenant_id: str, user_id: str) -> dict[str, int]:
|
||||||
|
"""Execute GDPR Art. 17 erasure for a user within one tenant.
|
||||||
|
|
||||||
|
Strategy (respects retention duties):
|
||||||
|
- Contacts owned by the user → soft-delete via deleted_at
|
||||||
|
(audit history must remain intact — it is not personal data of the
|
||||||
|
subject but business record; retention policy governs its cleanup)
|
||||||
|
- Notifications owned by the user → hard delete
|
||||||
|
- User account → deactivate (is_active=False), clear personal fields,
|
||||||
|
scramble password hash and email (keeps FK integrity for audit rows)
|
||||||
|
Returns counters for the audit entry.
|
||||||
|
"""
|
||||||
|
from datetime import UTC, datetime
|
||||||
|
from uuid import UUID as PyUUID
|
||||||
|
|
||||||
|
from sqlalchemy import select as sa_select
|
||||||
|
from sqlalchemy import update as sa_update
|
||||||
|
|
||||||
|
from app.core.audit import log_audit
|
||||||
|
from app.models.contact import Contact
|
||||||
|
from app.models.notification import Notification
|
||||||
|
from app.models.user import User
|
||||||
|
|
||||||
|
uid = PyUUID(user_id)
|
||||||
|
tid = PyUUID(tenant_id)
|
||||||
|
|
||||||
|
counts: dict[str, int] = {}
|
||||||
|
|
||||||
|
# 1. Soft-delete contacts owned by the user
|
||||||
|
contact_result = await db.execute(
|
||||||
|
sa_select(Contact).where(
|
||||||
|
Contact.tenant_id == tid,
|
||||||
|
Contact.owner_id == uid,
|
||||||
|
Contact.deleted_at.is_(None),
|
||||||
|
)
|
||||||
|
)
|
||||||
|
contacts = contact_result.scalars().all()
|
||||||
|
for c in contacts:
|
||||||
|
c.deleted_at = datetime.now(UTC)
|
||||||
|
counts["contacts_soft_deleted"] = len(contacts)
|
||||||
|
|
||||||
|
# 2. Hard-delete notifications owned by the user
|
||||||
|
notif_result = await db.execute(
|
||||||
|
sa_select(Notification).where(
|
||||||
|
Notification.tenant_id == tid,
|
||||||
|
Notification.owner_id == uid,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
notifications = notif_result.scalars().all()
|
||||||
|
for n in notifications:
|
||||||
|
await db.delete(n)
|
||||||
|
counts["notifications_deleted"] = len(notifications)
|
||||||
|
|
||||||
|
# 3. Anonymize + deactivate the account (FK integrity for audit rows kept)
|
||||||
|
await db.execute(
|
||||||
|
sa_update(User)
|
||||||
|
.where(User.id == uid)
|
||||||
|
.values(
|
||||||
|
email=f"erased.{uid.hex[:16]}@anonymized.invalid",
|
||||||
|
name="[gelöscht gemäß DSGVO Art. 17]",
|
||||||
|
first_name=None,
|
||||||
|
last_name=None,
|
||||||
|
avatar_url=None,
|
||||||
|
password_hash="!dsar-erased",
|
||||||
|
is_active=False,
|
||||||
|
preferences={},
|
||||||
|
)
|
||||||
|
)
|
||||||
|
counts["user_anonymized"] = 1
|
||||||
|
|
||||||
|
# 4. Audit the erasure itself (who/what/when — required by Art. 17 recital)
|
||||||
|
await log_audit(
|
||||||
|
db,
|
||||||
|
tid,
|
||||||
|
user_id,
|
||||||
|
"dsar_erasure",
|
||||||
|
"user",
|
||||||
|
uid,
|
||||||
|
{"target_user": user_id, **counts},
|
||||||
|
)
|
||||||
|
return counts
|
||||||
|
|
||||||
|
|
||||||
|
async def process_dsar(
|
||||||
|
ctx: dict[str, Any],
|
||||||
|
*,
|
||||||
|
user_id: str,
|
||||||
|
tenant_id: str,
|
||||||
|
request_type: str,
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
"""Process a GDPR Data Subject Access Request (DSAR).
|
||||||
|
|
||||||
|
ARQ worker function registered as "process_dsar".
|
||||||
|
|
||||||
|
request_type:
|
||||||
|
- "access": collect all data categories (Art. 15/20) and post a system
|
||||||
|
message that the export is ready (served via the existing dsgvo-export
|
||||||
|
endpoint).
|
||||||
|
- "deletion": execute Art. 17 erasure (soft-delete contacts, hard-delete
|
||||||
|
notifications, anonymize+deactivate account) and audit it.
|
||||||
|
- "rectification": post a system message asking admins to handle the
|
||||||
|
correction manually.
|
||||||
|
|
||||||
|
Returns a summary dict for the job result.
|
||||||
|
"""
|
||||||
|
import logging
|
||||||
|
import uuid as uuid_module
|
||||||
|
|
||||||
|
from app.core.db import get_worker_session_factory
|
||||||
|
from app.core.notifications import post_system_message
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
tid = uuid_module.UUID(tenant_id)
|
||||||
|
uid = uuid_module.UUID(user_id)
|
||||||
|
|
||||||
|
factory = get_worker_session_factory()
|
||||||
|
async with factory() as db:
|
||||||
|
try:
|
||||||
|
if request_type == "access":
|
||||||
|
data = await _dsar_collect_user_data(db, tenant_id, user_id)
|
||||||
|
await db.commit()
|
||||||
|
categories = list(data.get("data", {}).keys())
|
||||||
|
await post_system_message(
|
||||||
|
db,
|
||||||
|
tid,
|
||||||
|
uid,
|
||||||
|
"dsar_access_ready",
|
||||||
|
"DSGVO-Auskunft bereit",
|
||||||
|
f"Datenkategorien: {', '.join(categories)}",
|
||||||
|
severity="info",
|
||||||
|
)
|
||||||
|
await db.commit()
|
||||||
|
logger.info("DSAR access processed for user %s", user_id)
|
||||||
|
return {"type": request_type, "status": "completed", "categories": categories}
|
||||||
|
|
||||||
|
if request_type == "deletion":
|
||||||
|
counts = await _dsar_execute_deletion(db, tenant_id, user_id)
|
||||||
|
await db.commit()
|
||||||
|
await post_system_message(
|
||||||
|
db,
|
||||||
|
tid,
|
||||||
|
uid,
|
||||||
|
"dsar_deletion_done",
|
||||||
|
"DSGVO-Löschung ausgeführt",
|
||||||
|
f"Kontakten soft-gelöscht: {counts.get('contacts_soft_deleted', 0)}; Konto anonymisiert.",
|
||||||
|
severity="info",
|
||||||
|
)
|
||||||
|
await db.commit()
|
||||||
|
logger.info("DSAR deletion executed for user %s: %s", user_id, counts)
|
||||||
|
return {"type": request_type, "status": "completed", **counts}
|
||||||
|
|
||||||
|
if request_type == "rectification":
|
||||||
|
await post_system_message(
|
||||||
|
db,
|
||||||
|
tid,
|
||||||
|
uid,
|
||||||
|
"dsar_rectification_requested",
|
||||||
|
"DSGVO-Berichtigung angefordert",
|
||||||
|
f"Manuelle Bearbeitung für User {user_id} erforderlich.",
|
||||||
|
severity="warning",
|
||||||
|
)
|
||||||
|
await db.commit()
|
||||||
|
logger.info("DSAR rectification requested for user %s", user_id)
|
||||||
|
return {"type": request_type, "status": "queued_for_manual_handling"}
|
||||||
|
|
||||||
|
logger.warning("Unknown DSAR request_type '%s' for user %s", request_type, user_id)
|
||||||
|
return {"type": request_type, "status": "unknown_type"}
|
||||||
|
except Exception:
|
||||||
|
await db.rollback()
|
||||||
|
raise
|
||||||
|
|
||||||
|
|
||||||
|
register_job("process_dsar", process_dsar)
|
||||||
|
|||||||
@@ -3,6 +3,8 @@
|
|||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import logging
|
import logging
|
||||||
|
import re
|
||||||
|
import uuid as uuid_mod
|
||||||
|
|
||||||
from fastapi import Request, status
|
from fastapi import Request, status
|
||||||
from starlette.middleware.base import BaseHTTPMiddleware
|
from starlette.middleware.base import BaseHTTPMiddleware
|
||||||
@@ -137,3 +139,97 @@ class CSRFMiddleware(BaseHTTPMiddleware):
|
|||||||
pass
|
pass
|
||||||
|
|
||||||
return await call_next(request)
|
return await call_next(request)
|
||||||
|
|
||||||
|
|
||||||
|
class AuditMiddleware(BaseHTTPMiddleware):
|
||||||
|
"""Safety-net audit trail for ALL successful mutating requests.
|
||||||
|
|
||||||
|
AGENTS.md requires every mutation to produce an audit entry. Explicit
|
||||||
|
``log_audit`` calls in routes/services remain the detail layer (entity ids,
|
||||||
|
change diffs); this middleware guarantees a baseline entry for mutations
|
||||||
|
that lack one, marked with ``source=middleware`` in ``details``.
|
||||||
|
|
||||||
|
Best-effort by design: audit failures never break the request.
|
||||||
|
"""
|
||||||
|
|
||||||
|
_MUTATING = {"POST", "PUT", "PATCH", "DELETE"}
|
||||||
|
_SKIP_PREFIXES = (
|
||||||
|
"/api/v1/auth",
|
||||||
|
"/api/v1/health",
|
||||||
|
"/api/v1/errors",
|
||||||
|
"/api/v1/audit",
|
||||||
|
"/api/v1/external",
|
||||||
|
)
|
||||||
|
|
||||||
|
async def dispatch(self, request: Request, call_next):
|
||||||
|
response = await call_next(request)
|
||||||
|
|
||||||
|
if request.method not in self._MUTATING:
|
||||||
|
return response
|
||||||
|
if response.status_code < 200 or response.status_code >= 300:
|
||||||
|
return response
|
||||||
|
path = request.url.path
|
||||||
|
if any(path.startswith(p) for p in self._SKIP_PREFIXES):
|
||||||
|
return response
|
||||||
|
|
||||||
|
try:
|
||||||
|
await self._write_entry(request, path, response.status_code)
|
||||||
|
except Exception:
|
||||||
|
logging.getLogger(__name__).debug(
|
||||||
|
"AuditMiddleware: failed to write baseline entry for %s %s", request.method, path
|
||||||
|
)
|
||||||
|
return response
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _derive_entity_type(path: str) -> str:
|
||||||
|
"""Derive an entity_type from the second URL segment."""
|
||||||
|
parts = [p for p in path.split("/") if p]
|
||||||
|
# /api/v1/<resource>/... -> resource; singularize naive trailing 's'
|
||||||
|
resource = parts[2] if len(parts) > 2 and parts[0] == "api" and parts[1] == "v1" else (parts[0] if parts else "unknown")
|
||||||
|
return resource[:-1] if len(resource) > 3 and resource.endswith("s") else resource
|
||||||
|
|
||||||
|
async def _write_entry(self, request: Request, path: str, status_code: int) -> None:
|
||||||
|
from app.core.audit import log_audit
|
||||||
|
from app.core.auth import get_redis, get_session_data
|
||||||
|
from app.core.db import create_db_session
|
||||||
|
|
||||||
|
# Attribute via the Redis session (same source as CSRFMiddleware) —
|
||||||
|
# FastAPI dependencies run after middleware, so request.state is empty here.
|
||||||
|
settings = get_settings()
|
||||||
|
session_id = request.cookies.get(settings.session_cookie_name)
|
||||||
|
if not session_id:
|
||||||
|
return # unauthenticated — nothing to attribute
|
||||||
|
redis = get_redis()
|
||||||
|
session_data = await get_session_data(redis, session_id)
|
||||||
|
if not session_data:
|
||||||
|
return
|
||||||
|
tenant_raw = session_data.get("tenant_id")
|
||||||
|
user_raw = session_data.get("user_id")
|
||||||
|
if not tenant_raw:
|
||||||
|
return
|
||||||
|
|
||||||
|
action_map = {"POST": "create", "PATCH": "update", "PUT": "update", "DELETE": "delete"}
|
||||||
|
entity_id: uuid_mod.UUID | None = None
|
||||||
|
parts = [p for p in path.split("/") if p]
|
||||||
|
if parts and re.fullmatch(r"[0-9a-fA-F-]{36}", parts[-1]):
|
||||||
|
try:
|
||||||
|
entity_id = uuid_mod.UUID(parts[-1])
|
||||||
|
except ValueError:
|
||||||
|
entity_id = None
|
||||||
|
|
||||||
|
async with create_db_session(uuid_mod.UUID(tenant_raw)) as db:
|
||||||
|
await log_audit(
|
||||||
|
db,
|
||||||
|
uuid_mod.UUID(tenant_raw),
|
||||||
|
uuid_mod.UUID(user_raw) if user_raw else None,
|
||||||
|
action_map.get(request.method, request.method.lower()),
|
||||||
|
self._derive_entity_type(path),
|
||||||
|
entity_id,
|
||||||
|
changes={
|
||||||
|
"source": "middleware",
|
||||||
|
"method": request.method,
|
||||||
|
"path": path,
|
||||||
|
"status": status_code,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
await db.commit()
|
||||||
|
|||||||
@@ -54,6 +54,8 @@ CORE_PERMISSIONS: list[dict[str, str]] = [
|
|||||||
{"key": "taxes:write", "label": "Taxes: Write", "category": "core", "module": "taxes"},
|
{"key": "taxes:write", "label": "Taxes: Write", "category": "core", "module": "taxes"},
|
||||||
{"key": "currencies:read", "label": "Currencies: Read", "category": "core", "module": "currencies"},
|
{"key": "currencies:read", "label": "Currencies: Read", "category": "core", "module": "currencies"},
|
||||||
{"key": "currencies:write", "label": "Currencies: Write", "category": "core", "module": "currencies"},
|
{"key": "currencies:write", "label": "Currencies: Write", "category": "core", "module": "currencies"},
|
||||||
|
{"key": "custom_fields:read", "label": "Custom Fields: Read", "category": "core", "module": "custom_fields"},
|
||||||
|
{"key": "custom_fields:write", "label": "Custom Fields: Write", "category": "core", "module": "custom_fields"},
|
||||||
{"key": "import_export:read", "label": "Import/Export: Read", "category": "core", "module": "import_export"},
|
{"key": "import_export:read", "label": "Import/Export: Read", "category": "core", "module": "import_export"},
|
||||||
{"key": "import_export:write", "label": "Import/Export: Write", "category": "core", "module": "import_export"},
|
{"key": "import_export:write", "label": "Import/Export: Write", "category": "core", "module": "import_export"},
|
||||||
{"key": "workspaces:read", "label": "Workspaces: Read", "category": "core", "module": "workspaces"},
|
{"key": "workspaces:read", "label": "Workspaces: Read", "category": "core", "module": "workspaces"},
|
||||||
|
|||||||
@@ -29,6 +29,14 @@ class ServiceContainer:
|
|||||||
"""Check if a service is registered."""
|
"""Check if a service is registered."""
|
||||||
return name in self._services
|
return name in self._services
|
||||||
|
|
||||||
|
def remove(self, name: str) -> None:
|
||||||
|
"""Remove a service registration (no-op if absent).
|
||||||
|
|
||||||
|
Used by plugin deactivation hooks to clean up services they
|
||||||
|
registered during activation.
|
||||||
|
"""
|
||||||
|
self._services.pop(name, None)
|
||||||
|
|
||||||
async def initialize(self) -> None:
|
async def initialize(self) -> None:
|
||||||
"""Initialize core services."""
|
"""Initialize core services."""
|
||||||
if self._initialized:
|
if self._initialized:
|
||||||
|
|||||||
+40
-12
@@ -501,11 +501,38 @@ async def save_with_metadata(
|
|||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
|
async def get_file_metadata_async(path: str) -> dict[str, Any]:
|
||||||
|
"""Awaitable variant of :func:`get_file_metadata` (ARCH-052).
|
||||||
|
|
||||||
|
Safe to call from inside a running event loop — never creates a
|
||||||
|
nested one. For local storage this is plain filesystem access; for
|
||||||
|
S3 and other async backends the backend's ``exists()`` is awaited.
|
||||||
|
"""
|
||||||
|
backend = get_storage_backend()
|
||||||
|
if isinstance(backend, LocalStorage):
|
||||||
|
full_path = backend._full_path(path)
|
||||||
|
if not os.path.exists(full_path):
|
||||||
|
return {"size": None, "modified": None, "exists": False}
|
||||||
|
stat = os.stat(full_path)
|
||||||
|
return {
|
||||||
|
"size": stat.st_size,
|
||||||
|
"modified": stat.st_mtime,
|
||||||
|
"exists": True,
|
||||||
|
}
|
||||||
|
# S3 or other async backends — await the backend directly
|
||||||
|
if not await backend.exists(path):
|
||||||
|
return {"size": None, "modified": None, "exists": False}
|
||||||
|
return {"size": None, "modified": None, "exists": True}
|
||||||
|
|
||||||
|
|
||||||
def get_file_metadata(path: str) -> dict[str, Any]:
|
def get_file_metadata(path: str) -> dict[str, Any]:
|
||||||
"""Read metadata of a stored file without loading its content.
|
"""Read metadata of a stored file without loading its content.
|
||||||
|
|
||||||
Works with the *local* storage backend. For S3, use the S3 client
|
Works with the *local* storage backend without touching the event
|
||||||
``stat_object`` API directly.
|
loop. For S3 and other async-only backends this drives the check
|
||||||
|
through ``asyncio.run``; calling it from inside a running event loop
|
||||||
|
raises ``RuntimeError`` — use :func:`get_file_metadata_async` there
|
||||||
|
instead (ARCH-052).
|
||||||
|
|
||||||
Parameters
|
Parameters
|
||||||
----------
|
----------
|
||||||
@@ -530,14 +557,15 @@ def get_file_metadata(path: str) -> dict[str, Any]:
|
|||||||
"modified": stat.st_mtime,
|
"modified": stat.st_mtime,
|
||||||
"exists": True,
|
"exists": True,
|
||||||
}
|
}
|
||||||
# S3 or other backends — fall back to exists() check
|
# Async-only backend outside a running loop is fine; inside one we
|
||||||
import asyncio as _asyncio
|
# must never build a nested event loop.
|
||||||
|
|
||||||
loop = _asyncio.new_event_loop()
|
|
||||||
try:
|
try:
|
||||||
exists = loop.run_until_complete(backend.exists(path))
|
asyncio.get_running_loop()
|
||||||
if not exists:
|
except RuntimeError:
|
||||||
return {"size": None, "modified": None, "exists": False}
|
pass
|
||||||
return {"size": None, "modified": None, "exists": True}
|
else:
|
||||||
finally:
|
raise RuntimeError(
|
||||||
loop.close()
|
"get_file_metadata() cannot be used with async storage backends "
|
||||||
|
"inside a running event loop — use get_file_metadata_async()"
|
||||||
|
)
|
||||||
|
return asyncio.run(get_file_metadata_async(path))
|
||||||
|
|||||||
@@ -121,11 +121,14 @@ class TriggerDispatcher:
|
|||||||
"""Query DB for active automations matching *event_name* and dispatch."""
|
"""Query DB for active automations matching *event_name* and dispatch."""
|
||||||
from app.core.db import get_session_factory
|
from app.core.db import get_session_factory
|
||||||
from app.plugins.builtins.contracts import get_contract
|
from app.plugins.builtins.contracts import get_contract
|
||||||
|
# None-check FIRST — accessing attributes on the contract before the
|
||||||
|
# check crashed with AttributeError when automation was inactive
|
||||||
|
# (ARCH-029/041).
|
||||||
automation_contract = get_contract("automation")
|
automation_contract = get_contract("automation")
|
||||||
AutomationDefinition = automation_contract.Automation # noqa: N806
|
|
||||||
if automation_contract is None:
|
if automation_contract is None:
|
||||||
logger.debug("Automation plugin not available — trigger skipped")
|
logger.debug("Automation plugin not available — trigger skipped")
|
||||||
return
|
return
|
||||||
|
AutomationDefinition = automation_contract.Automation # noqa: N806
|
||||||
|
|
||||||
factory = get_session_factory()
|
factory = get_session_factory()
|
||||||
tenant_id = payload.get("tenant_id")
|
tenant_id = payload.get("tenant_id")
|
||||||
|
|||||||
+11
-12
@@ -170,12 +170,7 @@ async def on_startup(ctx: dict[str, Any]) -> None:
|
|||||||
if search_contract is not None:
|
if search_contract is not None:
|
||||||
factory = async_session
|
factory = async_session
|
||||||
async with factory() as db:
|
async with factory() as db:
|
||||||
# auto_register_providers is not exposed via contract yet;
|
await search_contract.auto_register_providers(db)
|
||||||
# use the contract's get_search_registry to access providers
|
|
||||||
from app.plugins.builtins.unified_search.provider_registry import (
|
|
||||||
auto_register_providers,
|
|
||||||
)
|
|
||||||
await auto_register_providers(db)
|
|
||||||
logger.info("Search providers registered for worker")
|
logger.info("Search providers registered for worker")
|
||||||
else:
|
else:
|
||||||
logger.debug("Unified search plugin not available — skipping provider registration")
|
logger.debug("Unified search plugin not available — skipping provider registration")
|
||||||
@@ -350,8 +345,10 @@ async def cleanup_audit_log_job(ctx: dict[str, Any]) -> None:
|
|||||||
Runs daily to prevent the audit_log table from growing indefinitely.
|
Runs daily to prevent the audit_log table from growing indefinitely.
|
||||||
Iterates per-tenant for RLS compliance.
|
Iterates per-tenant for RLS compliance.
|
||||||
"""
|
"""
|
||||||
from sqlalchemy import text as sa_text, delete as sa_delete
|
from datetime import UTC, datetime, timedelta
|
||||||
from datetime import datetime, timedelta
|
|
||||||
|
from sqlalchemy import delete as sa_delete
|
||||||
|
from sqlalchemy import text as sa_text
|
||||||
|
|
||||||
from app.core.db import get_worker_session_factory
|
from app.core.db import get_worker_session_factory
|
||||||
from app.models.audit import AuditLog
|
from app.models.audit import AuditLog
|
||||||
@@ -362,7 +359,7 @@ async def cleanup_audit_log_job(ctx: dict[str, Any]) -> None:
|
|||||||
tenant_result = await db.execute(sa_text("SELECT id FROM tenants"))
|
tenant_result = await db.execute(sa_text("SELECT id FROM tenants"))
|
||||||
tenant_ids = [row[0] for row in tenant_result]
|
tenant_ids = [row[0] for row in tenant_result]
|
||||||
|
|
||||||
cutoff = datetime.utcnow() - timedelta(days=365)
|
cutoff = datetime.now(UTC) - timedelta(days=365)
|
||||||
total_deleted = 0
|
total_deleted = 0
|
||||||
for tenant_id in tenant_ids:
|
for tenant_id in tenant_ids:
|
||||||
await db.execute(
|
await db.execute(
|
||||||
@@ -393,8 +390,10 @@ async def cleanup_trash_job(ctx: dict[str, Any]) -> None:
|
|||||||
Runs daily to clean up the trash. Iterates per-tenant for RLS compliance.
|
Runs daily to clean up the trash. Iterates per-tenant for RLS compliance.
|
||||||
Default retention: 90 days in trash before permanent deletion.
|
Default retention: 90 days in trash before permanent deletion.
|
||||||
"""
|
"""
|
||||||
from sqlalchemy import text as sa_text, delete as sa_delete
|
from datetime import UTC, datetime, timedelta
|
||||||
from datetime import datetime, timedelta
|
|
||||||
|
from sqlalchemy import delete as sa_delete
|
||||||
|
from sqlalchemy import text as sa_text
|
||||||
|
|
||||||
from app.core.db import get_worker_session_factory
|
from app.core.db import get_worker_session_factory
|
||||||
from app.models.contact import Contact
|
from app.models.contact import Contact
|
||||||
@@ -406,7 +405,7 @@ async def cleanup_trash_job(ctx: dict[str, Any]) -> None:
|
|||||||
tenant_result = await db.execute(sa_text("SELECT id FROM tenants"))
|
tenant_result = await db.execute(sa_text("SELECT id FROM tenants"))
|
||||||
tenant_ids = [row[0] for row in tenant_result]
|
tenant_ids = [row[0] for row in tenant_result]
|
||||||
|
|
||||||
cutoff = datetime.utcnow() - timedelta(days=90)
|
cutoff = datetime.now(UTC) - timedelta(days=90)
|
||||||
total_deleted = 0
|
total_deleted = 0
|
||||||
|
|
||||||
for tenant_id in tenant_ids:
|
for tenant_id in tenant_ids:
|
||||||
|
|||||||
+28
-3
@@ -17,8 +17,9 @@ from app.core.db import get_db, set_tenant_context, set_user_context
|
|||||||
|
|
||||||
logger = logging.getLogger(__name__)
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
# Known write-permission modules — used by require_write() to check
|
# Legacy fallback list — used by require_write() only when the permission
|
||||||
# specific permissions instead of broad wildcards like *:write
|
# registry is not initialized. The live source of truth is generated from
|
||||||
|
# the registry (see _get_write_permissions, ARCH-022).
|
||||||
_WRITE_PERMISSIONS = [
|
_WRITE_PERMISSIONS = [
|
||||||
"users:write",
|
"users:write",
|
||||||
"roles:write",
|
"roles:write",
|
||||||
@@ -35,6 +36,30 @@ _WRITE_PERMISSIONS = [
|
|||||||
]
|
]
|
||||||
|
|
||||||
|
|
||||||
|
def _get_write_permissions() -> list[str]:
|
||||||
|
"""Return all known ``module:write`` permission keys (ARCH-022).
|
||||||
|
|
||||||
|
Generated from the permission registry so plugin write permissions are
|
||||||
|
picked up automatically without touching this file. Falls back to the
|
||||||
|
static legacy list when the registry is unavailable/uninitialized.
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
from app.core.permission_registry import get_permission_registry
|
||||||
|
|
||||||
|
registry = get_permission_registry()
|
||||||
|
if getattr(registry, "_initialized", False):
|
||||||
|
perms = [
|
||||||
|
entry["key"]
|
||||||
|
for entry in registry.get_all()
|
||||||
|
if entry["key"].endswith(":write")
|
||||||
|
]
|
||||||
|
if perms:
|
||||||
|
return sorted(perms)
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
return list(_WRITE_PERMISSIONS)
|
||||||
|
|
||||||
|
|
||||||
async def get_redis_dep() -> aioredis.Redis:
|
async def get_redis_dep() -> aioredis.Redis:
|
||||||
"""FastAPI dependency for Redis client."""
|
"""FastAPI dependency for Redis client."""
|
||||||
return get_redis()
|
return get_redis()
|
||||||
@@ -261,7 +286,7 @@ async def require_write(
|
|||||||
# Check via permission system for specific write permissions
|
# Check via permission system for specific write permissions
|
||||||
from app.core.permissions import check_permission
|
from app.core.permissions import check_permission
|
||||||
|
|
||||||
for perm in _WRITE_PERMISSIONS:
|
for perm in _get_write_permissions():
|
||||||
if check_permission(current_user, perm):
|
if check_permission(current_user, perm):
|
||||||
return current_user
|
return current_user
|
||||||
|
|
||||||
|
|||||||
+36
-20
@@ -22,8 +22,12 @@ logger = logging.getLogger(__name__)
|
|||||||
|
|
||||||
from app.config import get_settings # noqa: E402
|
from app.config import get_settings # noqa: E402
|
||||||
from app.core.db import close_engine, get_engine # noqa: E402
|
from app.core.db import close_engine, get_engine # noqa: E402
|
||||||
from app.core.error_codes import ApiError, build_error_response # noqa: E402
|
from app.core.error_codes import ERROR_CODES, ApiError, build_error_response # noqa: E402
|
||||||
from app.core.middleware import CSRFMiddleware, SecurityHeadersMiddleware # noqa: E402
|
from app.core.middleware import ( # noqa: E402
|
||||||
|
AuditMiddleware,
|
||||||
|
CSRFMiddleware,
|
||||||
|
SecurityHeadersMiddleware,
|
||||||
|
)
|
||||||
from app.core.monitoring import record_error, record_request # noqa: E402
|
from app.core.monitoring import record_error, record_request # noqa: E402
|
||||||
from app.core.rate_limit import GeneralRateLimitMiddleware # noqa: E402
|
from app.core.rate_limit import GeneralRateLimitMiddleware # noqa: E402
|
||||||
from app.core.resilience import CircuitBreakerMiddleware # noqa: E402
|
from app.core.resilience import CircuitBreakerMiddleware # noqa: E402
|
||||||
@@ -36,16 +40,14 @@ from app.routes import ( # noqa: E402
|
|||||||
attachments,
|
attachments,
|
||||||
audit,
|
audit,
|
||||||
auth,
|
auth,
|
||||||
compliance,
|
|
||||||
backups,
|
backups,
|
||||||
bank_accounts,
|
bank_accounts,
|
||||||
contact_folder_permissions,
|
compliance,
|
||||||
contact_folders,
|
|
||||||
contacts,
|
|
||||||
currencies,
|
currencies,
|
||||||
custom_field_definitions,
|
custom_field_definitions,
|
||||||
custom_fields,
|
custom_fields,
|
||||||
dashboard,
|
dashboard,
|
||||||
|
delegations,
|
||||||
entity_history,
|
entity_history,
|
||||||
entity_permissions,
|
entity_permissions,
|
||||||
errors,
|
errors,
|
||||||
@@ -59,7 +61,6 @@ from app.routes import ( # noqa: E402
|
|||||||
owner_transfer,
|
owner_transfer,
|
||||||
permission_templates,
|
permission_templates,
|
||||||
plugins,
|
plugins,
|
||||||
delegations,
|
|
||||||
policies,
|
policies,
|
||||||
roles,
|
roles,
|
||||||
saved_filters,
|
saved_filters,
|
||||||
@@ -284,22 +285,23 @@ async def lifespan(app: FastAPI):
|
|||||||
logger.info(f"Plugin {name} is inactive — skipping activation")
|
logger.info(f"Plugin {name} is inactive — skipping activation")
|
||||||
continue
|
continue
|
||||||
|
|
||||||
# Activate plugin with a FRESH session per plugin to avoid RLS state leakage
|
# Activate plugin ONCE per process (ARCH-002 fix): a fresh session with
|
||||||
# RLS fail-closed requires app.current_tenant_id for tenant-table writes.
|
# the first tenant's RLS context satisfies fail-closed RLS for any
|
||||||
# Plugin activation may fail on duplicate cron job inserts — this is harmless
|
# tenant-table writes during activation. Plugins that need per-tenant
|
||||||
# since cron jobs already exist from previous startups.
|
# data must seed it themselves (e.g. via the default-tenant mechanism).
|
||||||
|
# Calling on_activate once prevents duplicate event listeners, cron
|
||||||
|
# jobs, mini-apps and other contributions at multi-tenant startups.
|
||||||
plugin_activated = False
|
plugin_activated = False
|
||||||
for tenant_id in all_tenant_ids:
|
if all_tenant_ids:
|
||||||
try:
|
try:
|
||||||
async with async_session() as plugin_db:
|
async with async_session() as plugin_db:
|
||||||
await set_tenant_context(plugin_db, tenant_id)
|
await set_tenant_context(plugin_db, all_tenant_ids[0])
|
||||||
await plugin.on_activate(plugin_db, container, event_bus)
|
await plugin.on_activate(plugin_db, container, event_bus)
|
||||||
await plugin_db.flush()
|
await plugin_db.flush()
|
||||||
await plugin_db.commit()
|
await plugin_db.commit()
|
||||||
plugin_activated = True
|
plugin_activated = True
|
||||||
except Exception as exc:
|
except Exception as exc:
|
||||||
logger.warning(f"[STARTUP] Plugin {name} activation issue for tenant {tenant_id}: {exc}")
|
logger.warning(f"[STARTUP] Plugin {name} activation issue: {exc}")
|
||||||
break
|
|
||||||
|
|
||||||
if plugin_activated:
|
if plugin_activated:
|
||||||
plugin_record.status = "active"
|
plugin_record.status = "active"
|
||||||
@@ -474,6 +476,7 @@ def create_app() -> FastAPI:
|
|||||||
)
|
)
|
||||||
app.add_middleware(CSRFMiddleware)
|
app.add_middleware(CSRFMiddleware)
|
||||||
app.add_middleware(SecurityHeadersMiddleware)
|
app.add_middleware(SecurityHeadersMiddleware)
|
||||||
|
app.add_middleware(AuditMiddleware)
|
||||||
app.add_middleware(GeneralRateLimitMiddleware)
|
app.add_middleware(GeneralRateLimitMiddleware)
|
||||||
app.add_middleware(RequestLoggingMiddleware)
|
app.add_middleware(RequestLoggingMiddleware)
|
||||||
app.add_middleware(CircuitBreakerMiddleware)
|
app.add_middleware(CircuitBreakerMiddleware)
|
||||||
@@ -516,6 +519,21 @@ def create_app() -> FastAPI:
|
|||||||
504: "service_timeout",
|
504: "service_timeout",
|
||||||
}
|
}
|
||||||
code = status_to_code.get(exc.status_code, "internal_error" if exc.status_code >= 500 else "validation_error")
|
code = status_to_code.get(exc.status_code, "internal_error" if exc.status_code >= 500 else "validation_error")
|
||||||
|
# Structured detail passthrough (AGENTS.md): when a route raises
|
||||||
|
# HTTPException with a dict detail containing a machine-readable ``code``,
|
||||||
|
# preserve the structured shape instead of stringifying it.
|
||||||
|
raw_detail = exc.detail
|
||||||
|
if isinstance(raw_detail, dict):
|
||||||
|
inner_code = raw_detail.get("code", code)
|
||||||
|
body = build_error_response(
|
||||||
|
code=inner_code if inner_code in ERROR_CODES else code,
|
||||||
|
detail=raw_detail.get("detail") or str(raw_detail),
|
||||||
|
trace_id=trace_id,
|
||||||
|
)
|
||||||
|
# Preserve the full structured detail as a nested object so clients
|
||||||
|
# can read ``resp.json()["detail"]["code"]``.
|
||||||
|
body["detail"] = raw_detail
|
||||||
|
else:
|
||||||
body = build_error_response(
|
body = build_error_response(
|
||||||
code=code,
|
code=code,
|
||||||
detail=str(exc.detail) if exc.detail else None,
|
detail=str(exc.detail) if exc.detail else None,
|
||||||
@@ -544,11 +562,9 @@ def create_app() -> FastAPI:
|
|||||||
app.include_router(groups.router)
|
app.include_router(groups.router)
|
||||||
app.include_router(tenants.router)
|
app.include_router(tenants.router)
|
||||||
app.include_router(notifications.router)
|
app.include_router(notifications.router)
|
||||||
from app.routes.companies import router as companies_router
|
# NOTE: contacts/companies/contact-folders routes are plugin-owned now
|
||||||
app.include_router(companies_router)
|
# (Block B1) and mounted via the manifest.routes mechanism below with
|
||||||
app.include_router(contacts.router)
|
# require_active_plugin("contacts") protection.
|
||||||
app.include_router(contact_folders.router)
|
|
||||||
app.include_router(contact_folder_permissions.router)
|
|
||||||
app.include_router(entity_permissions.router)
|
app.include_router(entity_permissions.router)
|
||||||
app.include_router(dashboard.router)
|
app.include_router(dashboard.router)
|
||||||
app.include_router(entity_history.router)
|
app.include_router(entity_history.router)
|
||||||
|
|||||||
@@ -6,8 +6,8 @@ from app.models.audit import AuditLog
|
|||||||
from app.models.auth import ApiToken, PasswordResetToken
|
from app.models.auth import ApiToken, PasswordResetToken
|
||||||
from app.models.backup import Backup
|
from app.models.backup import Backup
|
||||||
from app.models.bank_account import BankAccount
|
from app.models.bank_account import BankAccount
|
||||||
from app.models.consumer_inbox import ConsumerInbox
|
|
||||||
from app.models.compliance import ComplianceIncident
|
from app.models.compliance import ComplianceIncident
|
||||||
|
from app.models.consumer_inbox import ConsumerInbox
|
||||||
from app.models.contact import Contact, ContactPerson
|
from app.models.contact import Contact, ContactPerson
|
||||||
from app.models.contact_folder import ContactFolder
|
from app.models.contact_folder import ContactFolder
|
||||||
from app.models.contact_merge import ContactMergeHistory
|
from app.models.contact_merge import ContactMergeHistory
|
||||||
|
|||||||
+1
-2
@@ -11,13 +11,12 @@ from datetime import datetime
|
|||||||
from typing import Any
|
from typing import Any
|
||||||
|
|
||||||
from sqlalchemy import DateTime, ForeignKey, String, func
|
from sqlalchemy import DateTime, ForeignKey, String, func
|
||||||
from sqlalchemy.dialects.postgresql import JSONB
|
from sqlalchemy.dialects.postgresql import JSONB, TSVECTOR
|
||||||
from sqlalchemy.dialects.postgresql import UUID as PGUUID
|
from sqlalchemy.dialects.postgresql import UUID as PGUUID
|
||||||
from sqlalchemy.orm import Mapped, mapped_column
|
from sqlalchemy.orm import Mapped, mapped_column
|
||||||
|
|
||||||
from app.core.db import Base, TenantMixin
|
from app.core.db import Base, TenantMixin
|
||||||
from app.models.owned_mixin import OwnedMixin
|
from app.models.owned_mixin import OwnedMixin
|
||||||
from sqlalchemy.dialects.postgresql import TSVECTOR
|
|
||||||
|
|
||||||
# Re-export EntityHistory as DeletionLog for backward compatibility.
|
# Re-export EntityHistory as DeletionLog for backward compatibility.
|
||||||
# Tests import DeletionLog from app.models.audit and use entity_snapshot attribute.
|
# Tests import DeletionLog from app.models.audit and use entity_snapshot attribute.
|
||||||
|
|||||||
+47
-1
@@ -5,7 +5,7 @@ from __future__ import annotations
|
|||||||
import uuid
|
import uuid
|
||||||
from datetime import datetime
|
from datetime import datetime
|
||||||
|
|
||||||
from sqlalchemy import DateTime, Integer, String, Text, func
|
from sqlalchemy import DateTime, ForeignKey, Integer, String, Text, UniqueConstraint, func
|
||||||
from sqlalchemy.dialects.postgresql import JSONB
|
from sqlalchemy.dialects.postgresql import JSONB
|
||||||
from sqlalchemy.dialects.postgresql import UUID as PGUUID
|
from sqlalchemy.dialects.postgresql import UUID as PGUUID
|
||||||
from sqlalchemy.orm import Mapped, mapped_column
|
from sqlalchemy.orm import Mapped, mapped_column
|
||||||
@@ -77,3 +77,49 @@ class EventOutbox(Base):
|
|||||||
failed_at: Mapped[datetime | None] = mapped_column(
|
failed_at: Mapped[datetime | None] = mapped_column(
|
||||||
DateTime(timezone=True), nullable=True,
|
DateTime(timezone=True), nullable=True,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class OutboxDelivery(Base):
|
||||||
|
"""Per-consumer delivery status for an event_outbox row (Migration 0075).
|
||||||
|
|
||||||
|
Tracks whether each consumer successfully processed an event; an event is
|
||||||
|
only 'published' when all mandatory deliveries succeed.
|
||||||
|
"""
|
||||||
|
|
||||||
|
__tablename__ = "outbox_deliveries"
|
||||||
|
__table_args__ = (
|
||||||
|
UniqueConstraint(
|
||||||
|
"event_id", "consumer_name",
|
||||||
|
name="uq_outbox_deliveries_event_consumer",
|
||||||
|
),
|
||||||
|
)
|
||||||
|
|
||||||
|
id: Mapped[uuid.UUID] = mapped_column(
|
||||||
|
PGUUID(as_uuid=True), primary_key=True,
|
||||||
|
server_default=func.gen_random_uuid(),
|
||||||
|
)
|
||||||
|
event_id: Mapped[uuid.UUID] = mapped_column(
|
||||||
|
PGUUID(as_uuid=True),
|
||||||
|
ForeignKey("event_outbox.id", ondelete="CASCADE"),
|
||||||
|
nullable=False,
|
||||||
|
)
|
||||||
|
consumer_name: Mapped[str] = mapped_column(String(150), nullable=False)
|
||||||
|
status: Mapped[str] = mapped_column(
|
||||||
|
String(30), nullable=False, server_default="pending",
|
||||||
|
)
|
||||||
|
attempt_count: Mapped[int] = mapped_column(
|
||||||
|
Integer, nullable=False, server_default="0",
|
||||||
|
)
|
||||||
|
next_attempt_at: Mapped[datetime | None] = mapped_column(
|
||||||
|
DateTime(timezone=True), nullable=True,
|
||||||
|
)
|
||||||
|
last_error: Mapped[str | None] = mapped_column(Text, nullable=True)
|
||||||
|
processed_at: Mapped[datetime | None] = mapped_column(
|
||||||
|
DateTime(timezone=True), nullable=True,
|
||||||
|
)
|
||||||
|
created_at: Mapped[datetime] = mapped_column(
|
||||||
|
DateTime(timezone=True), nullable=False, server_default=func.now(),
|
||||||
|
)
|
||||||
|
updated_at: Mapped[datetime] = mapped_column(
|
||||||
|
DateTime(timezone=True), nullable=False, server_default=func.now(),
|
||||||
|
)
|
||||||
|
|||||||
@@ -53,6 +53,9 @@ class SystemSettings(Base, TenantMixin, OwnedMixin):
|
|||||||
theme_border_radius: Mapped[str] = mapped_column(String(20), nullable=False, default="0.5rem")
|
theme_border_radius: Mapped[str] = mapped_column(String(20), nullable=False, default="0.5rem")
|
||||||
# Backup configuration
|
# Backup configuration
|
||||||
backup_enabled: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False, server_default="false")
|
backup_enabled: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False, server_default="false")
|
||||||
|
backup_interval: Mapped[str] = mapped_column(String(20), nullable=False, default="daily", server_default="daily")
|
||||||
|
backup_retention_days: Mapped[int] = mapped_column(Integer, nullable=False, default=7, server_default="7")
|
||||||
|
backup_destination: Mapped[str] = mapped_column(String(20), nullable=False, default="local", server_default="local")
|
||||||
# Automation plugin settings (JSONB)
|
# Automation plugin settings (JSONB)
|
||||||
automation_config: Mapped[dict | None] = mapped_column(JSONB, nullable=True)
|
automation_config: Mapped[dict | None] = mapped_column(JSONB, nullable=True)
|
||||||
# Retention policy overrides (JSONB) — compliance module
|
# Retention policy overrides (JSONB) — compliance module
|
||||||
|
|||||||
@@ -12,7 +12,6 @@ from sqlalchemy.dialects.postgresql import UUID as PGUUID
|
|||||||
from sqlalchemy.orm import Mapped, mapped_column
|
from sqlalchemy.orm import Mapped, mapped_column
|
||||||
|
|
||||||
from app.core.db import Base, SoftDeleteMixin, TimestampMixin
|
from app.core.db import Base, SoftDeleteMixin, TimestampMixin
|
||||||
from app.models.owned_mixin import OwnedMixin
|
|
||||||
|
|
||||||
|
|
||||||
class User(Base, TimestampMixin, SoftDeleteMixin):
|
class User(Base, TimestampMixin, SoftDeleteMixin):
|
||||||
|
|||||||
@@ -97,6 +97,15 @@ class BasePlugin(ABC):
|
|||||||
"""
|
"""
|
||||||
return []
|
return []
|
||||||
|
|
||||||
|
async def register_event_handlers(self, event_bus: EventBus) -> None:
|
||||||
|
"""Register event handlers for the background worker (ARCH-038 hook).
|
||||||
|
|
||||||
|
The worker calls this on every active plugin at startup so plugins
|
||||||
|
can subscribe to events even when the web process is separate.
|
||||||
|
Default: no-op. Override to subscribe handlers.
|
||||||
|
"""
|
||||||
|
return None
|
||||||
|
|
||||||
# ─── Job Modules ───
|
# ─── Job Modules ───
|
||||||
|
|
||||||
def get_job_modules(self) -> list[str]:
|
def get_job_modules(self) -> list[str]:
|
||||||
|
|||||||
@@ -12,7 +12,6 @@ from sqlalchemy.orm import Mapped, mapped_column
|
|||||||
|
|
||||||
from app.core.db import Base, TenantMixin
|
from app.core.db import Base, TenantMixin
|
||||||
from app.models.owned_mixin import OwnedMixin
|
from app.models.owned_mixin import OwnedMixin
|
||||||
from pgvector.sqlalchemy import Vector
|
|
||||||
|
|
||||||
|
|
||||||
class AgentMemory(Base, TenantMixin, OwnedMixin):
|
class AgentMemory(Base, TenantMixin, OwnedMixin):
|
||||||
|
|||||||
@@ -22,6 +22,7 @@ from app.plugins.builtins.ai_assistant.schemas import (
|
|||||||
ExternalAgentRequest,
|
ExternalAgentRequest,
|
||||||
ExternalAgentResponse,
|
ExternalAgentResponse,
|
||||||
)
|
)
|
||||||
|
from app.plugins.builtins.ai_assistant.services import stream_chat_comm as stream_chat
|
||||||
|
|
||||||
logger = logging.getLogger(__name__)
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
@@ -120,13 +121,18 @@ async def run_agent_external(
|
|||||||
}
|
}
|
||||||
|
|
||||||
# Run the agent via streaming chat (non-streaming mode)
|
# Run the agent via streaming chat (non-streaming mode)
|
||||||
from app.plugins.builtins.ai_assistant.services import stream_chat_comm
|
|
||||||
|
|
||||||
full_response = ""
|
full_response = ""
|
||||||
async with get_db() as stream_db:
|
async with get_db() as stream_db:
|
||||||
await set_tenant_context(stream_db, tenant_id)
|
await set_tenant_context(stream_db, tenant_id)
|
||||||
async for chunk in stream_chat(
|
async for chunk in stream_chat(
|
||||||
stream_db, session, agent, data.message, user_context, tenant_id
|
stream_db,
|
||||||
|
session.id,
|
||||||
|
agent,
|
||||||
|
data.message,
|
||||||
|
user_context,
|
||||||
|
tenant_id,
|
||||||
|
uuid.UUID(current_user["user_id"]),
|
||||||
):
|
):
|
||||||
if chunk.startswith("data: ") and chunk != "data: [DONE]\n\n":
|
if chunk.startswith("data: ") and chunk != "data: [DONE]\n\n":
|
||||||
try:
|
try:
|
||||||
|
|||||||
@@ -0,0 +1,10 @@
|
|||||||
|
-- Dual-path convergence (Gate B): add compliance columns that Alembic
|
||||||
|
-- migration 0119 adds on the core path. Idempotent so both install paths
|
||||||
|
-- converge to the identical schema.
|
||||||
|
ALTER TABLE ai_providers ADD COLUMN IF NOT EXISTS region VARCHAR(20) NOT NULL DEFAULT 'unknown';
|
||||||
|
ALTER TABLE ai_providers ADD COLUMN IF NOT EXISTS hosting_type VARCHAR(30) NOT NULL DEFAULT 'cloud';
|
||||||
|
ALTER TABLE ai_providers ADD COLUMN IF NOT EXISTS dpa_status VARCHAR(20) NOT NULL DEFAULT 'none';
|
||||||
|
ALTER TABLE ai_providers ADD COLUMN IF NOT EXISTS retention_policy TEXT NOT NULL DEFAULT '';
|
||||||
|
ALTER TABLE ai_providers ADD COLUMN IF NOT EXISTS training_on_customer_data BOOLEAN NOT NULL DEFAULT FALSE;
|
||||||
|
ALTER TABLE ai_providers ADD COLUMN IF NOT EXISTS transfer_notice TEXT NOT NULL DEFAULT '';
|
||||||
|
ALTER TABLE ai_providers ADD COLUMN IF NOT EXISTS allowed_data_classes JSONB NOT NULL DEFAULT '[]'::jsonb;
|
||||||
@@ -42,7 +42,7 @@ class AIAssistantPlugin(BasePlugin):
|
|||||||
),
|
),
|
||||||
],
|
],
|
||||||
events=[],
|
events=[],
|
||||||
migrations=["0001_initial.sql", "0002_folders_attachments.sql"],
|
migrations=["0001_initial.sql", "0002_folders_attachments.sql", "0003_sort_order.sql", "0004_compliance_fields.sql"],
|
||||||
permissions=[
|
permissions=[
|
||||||
"ai:read",
|
"ai:read",
|
||||||
"ai:write",
|
"ai:write",
|
||||||
@@ -52,10 +52,10 @@ class AIAssistantPlugin(BasePlugin):
|
|||||||
],
|
],
|
||||||
is_core=True,
|
is_core=True,
|
||||||
menu_items=[
|
menu_items=[
|
||||||
FrontendMenuItem(label_key='nav.aiAssistant', label='KI Assistent', path='/ai-assistant', icon='Bot', order=90),
|
FrontendMenuItem(label_key='nav.aiAssistant', label='KI Assistent', path='/ai-assistant', icon='Bot', order=90, permission='ai:read'),
|
||||||
],
|
],
|
||||||
page_routes=[
|
page_routes=[
|
||||||
FrontendPageRoute(path='/ai-assistant', component='@/pages/AIAssistant', protected=True),
|
FrontendPageRoute(path='/ai-assistant', component='@/pages/AIAssistant', protected=True, permission='ai:read'),
|
||||||
],
|
],
|
||||||
settings_pages=[
|
settings_pages=[
|
||||||
FrontendSettingsPage(path='ai', label_key='settings.ai', label='AI Settings', component='@/pages/AISettings', icon='Bot', order=60),
|
FrontendSettingsPage(path='ai', label_key='settings.ai', label='AI Settings', component='@/pages/AISettings', icon='Bot', order=60),
|
||||||
|
|||||||
@@ -165,8 +165,8 @@ async def get_open_tasks_handler(arguments: dict[str, Any], context: dict[str, A
|
|||||||
|
|
||||||
from app.plugins.builtins.calendar.contracts import get_contract as get_calendar_contract
|
from app.plugins.builtins.calendar.contracts import get_contract as get_calendar_contract
|
||||||
_cal = get_calendar_contract()
|
_cal = get_calendar_contract()
|
||||||
calendar_entry = _cal.calendar_entry
|
calendar_entry = _cal.CalendarEntry
|
||||||
calendar_entry_link = _cal.calendar_entry_link
|
calendar_entry_link = _cal.CalendarEntryLink
|
||||||
|
|
||||||
db, tenant_id, _ = await _get_db_and_tenant(context)
|
db, tenant_id, _ = await _get_db_and_tenant(context)
|
||||||
entity_type = arguments["entity_type"]
|
entity_type = arguments["entity_type"]
|
||||||
|
|||||||
@@ -69,10 +69,12 @@ async def push_suggestion(user_id: str, suggestion: dict[str, Any]) -> None:
|
|||||||
# Post suggestion to Communication (I-WORK-PROACTIVE)
|
# Post suggestion to Communication (I-WORK-PROACTIVE)
|
||||||
try:
|
try:
|
||||||
import uuid as uuid_mod
|
import uuid as uuid_mod
|
||||||
|
|
||||||
|
from sqlalchemy import select as sa_select
|
||||||
|
|
||||||
|
from app.core.db import get_worker_session_factory
|
||||||
from app.plugins.builtins.contracts import get_contract_registry
|
from app.plugins.builtins.contracts import get_contract_registry
|
||||||
from app.plugins.builtins.kommunikation.models import CommConversation
|
from app.plugins.builtins.kommunikation.models import CommConversation
|
||||||
from sqlalchemy import select as sa_select
|
|
||||||
from app.core.db import get_worker_session_factory
|
|
||||||
komm = get_contract_registry().get("kommunikation")
|
komm = get_contract_registry().get("kommunikation")
|
||||||
if komm:
|
if komm:
|
||||||
factory = get_worker_session_factory()
|
factory = get_worker_session_factory()
|
||||||
@@ -274,8 +276,8 @@ async def gather_context(
|
|||||||
# Upcoming calendar events
|
# Upcoming calendar events
|
||||||
from app.plugins.builtins.calendar.contracts import get_contract as get_calendar_contract
|
from app.plugins.builtins.calendar.contracts import get_contract as get_calendar_contract
|
||||||
_cal = get_calendar_contract()
|
_cal = get_calendar_contract()
|
||||||
calendar_entry = _cal.calendar_entry
|
calendar_entry = _cal.CalendarEntry
|
||||||
calendar_entry_link = _cal.calendar_entry_link
|
calendar_entry_link = _cal.CalendarEntryLink
|
||||||
|
|
||||||
now = datetime.now(UTC)
|
now = datetime.now(UTC)
|
||||||
event_result = await db.execute(
|
event_result = await db.execute(
|
||||||
@@ -389,8 +391,8 @@ async def gather_context(
|
|||||||
# Upcoming events
|
# Upcoming events
|
||||||
from app.plugins.builtins.calendar.contracts import get_contract as get_calendar_contract
|
from app.plugins.builtins.calendar.contracts import get_contract as get_calendar_contract
|
||||||
_cal = get_calendar_contract()
|
_cal = get_calendar_contract()
|
||||||
calendar_entry = _cal.calendar_entry
|
calendar_entry = _cal.CalendarEntry
|
||||||
calendar_entry_link = _cal.calendar_entry_link
|
calendar_entry_link = _cal.CalendarEntryLink
|
||||||
|
|
||||||
now = datetime.now(UTC)
|
now = datetime.now(UTC)
|
||||||
event_result = await db.execute(
|
event_result = await db.execute(
|
||||||
|
|||||||
@@ -62,7 +62,10 @@ class AIUIControlPlugin(BasePlugin):
|
|||||||
from app.plugins.builtins.contracts import get_contract_registry
|
from app.plugins.builtins.contracts import get_contract_registry
|
||||||
get_contract_registry().unregister(self.manifest.name)
|
get_contract_registry().unregister(self.manifest.name)
|
||||||
|
|
||||||
await super().on_deactivate(db, service_container, event_bus)
|
# Remove the WebSocket manager BEFORE super() so that event handlers
|
||||||
|
# being unsubscribed can no longer reach it (ARCH-044).
|
||||||
if service_container.has("ai_ui_control_ws"):
|
if service_container.has("ai_ui_control_ws"):
|
||||||
service_container.remove("ai_ui_control_ws")
|
service_container.remove("ai_ui_control_ws")
|
||||||
logger.info("AI UI Control WebSocket manager removed")
|
logger.info("AI UI Control WebSocket manager removed")
|
||||||
|
|
||||||
|
await super().on_deactivate(db, service_container, event_bus)
|
||||||
|
|||||||
@@ -55,7 +55,8 @@ async def send_agent_message(
|
|||||||
|
|
||||||
# 2. Create a kommunikation message in a dedicated agent room
|
# 2. Create a kommunikation message in a dedicated agent room
|
||||||
try:
|
try:
|
||||||
from app.plugins.builtins.kommunikation.contracts import CommConversation as Room, CommMessage as Message
|
from app.plugins.builtins.kommunikation.contracts import CommConversation as Room
|
||||||
|
from app.plugins.builtins.kommunikation.contracts import CommMessage as Message
|
||||||
|
|
||||||
# Find or create the agent-to-agent room
|
# Find or create the agent-to-agent room
|
||||||
room_name = f"agent:{from_agent_id}:{target_agent.id}"
|
room_name = f"agent:{from_agent_id}:{target_agent.id}"
|
||||||
|
|||||||
@@ -619,6 +619,7 @@ async def stream_agent_run(
|
|||||||
in real-time as the agent processes.
|
in real-time as the agent processes.
|
||||||
"""
|
"""
|
||||||
from fastapi.responses import StreamingResponse
|
from fastapi.responses import StreamingResponse
|
||||||
|
|
||||||
from app.ai.agent_stream import stream_react_loop
|
from app.ai.agent_stream import stream_react_loop
|
||||||
from app.plugins.builtins.ai_assistant.contracts import get_tool_registry
|
from app.plugins.builtins.ai_assistant.contracts import get_tool_registry
|
||||||
|
|
||||||
|
|||||||
@@ -11,6 +11,7 @@ Safety features:
|
|||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import logging
|
import logging
|
||||||
|
import uuid
|
||||||
from datetime import UTC, datetime
|
from datetime import UTC, datetime
|
||||||
from typing import Any
|
from typing import Any
|
||||||
|
|
||||||
@@ -37,12 +38,12 @@ async def run_agent(
|
|||||||
3. Infinite loop: same tool 5x consecutively (handled in ReAct loop)
|
3. Infinite loop: same tool 5x consecutively (handled in ReAct loop)
|
||||||
4. Budget limit: cumulative cost_usd
|
4. Budget limit: cumulative cost_usd
|
||||||
"""
|
"""
|
||||||
|
from app.plugins.builtins.ai_assistant.contracts import get_tool_registry
|
||||||
from app.plugins.builtins.automation.models import (
|
from app.plugins.builtins.automation.models import (
|
||||||
AgentDefinition,
|
AgentDefinition,
|
||||||
AgentRun,
|
AgentRun,
|
||||||
AgentRunStep,
|
AgentRunStep,
|
||||||
)
|
)
|
||||||
from app.plugins.builtins.ai_assistant.contracts import get_tool_registry
|
|
||||||
|
|
||||||
factory = get_session_factory()
|
factory = get_session_factory()
|
||||||
|
|
||||||
@@ -110,9 +111,9 @@ async def run_agent(
|
|||||||
try:
|
try:
|
||||||
from app.plugins.builtins.contracts import get_contract
|
from app.plugins.builtins.contracts import get_contract
|
||||||
mail_contract = get_contract("mail")
|
mail_contract = get_contract("mail")
|
||||||
if mail_contract and hasattr(mail_contract, "get_recent_mails"):
|
if mail_contract and hasattr(mail_contract, "Mail"):
|
||||||
from sqlalchemy import select as _select
|
from sqlalchemy import select as _select
|
||||||
from app.plugins.builtins.mail.models import Mail
|
Mail = mail_contract.Mail
|
||||||
async with factory() as db:
|
async with factory() as db:
|
||||||
mail_q = await db.execute(
|
mail_q = await db.execute(
|
||||||
_select(Mail)
|
_select(Mail)
|
||||||
@@ -167,7 +168,7 @@ async def run_agent(
|
|||||||
perm_ctx = await resolve_agent_permissions(
|
perm_ctx = await resolve_agent_permissions(
|
||||||
db=db,
|
db=db,
|
||||||
tenant_id=agent.tenant_id,
|
tenant_id=agent.tenant_id,
|
||||||
user_id=agent.created_by or uuid_mod.uuid4(),
|
user_id=agent.created_by or uuid.uuid4(),
|
||||||
agent_definition=agent,
|
agent_definition=agent,
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -379,23 +380,16 @@ async def run_agent(
|
|||||||
komm = get_contract_registry().get("kommunikation")
|
komm = get_contract_registry().get("kommunikation")
|
||||||
if komm:
|
if komm:
|
||||||
async with factory() as db:
|
async with factory() as db:
|
||||||
# Find or create agent conversation room
|
# Find or create agent conversation room via contract
|
||||||
from app.plugins.builtins.contracts import get_contract as _get_contract
|
# (find_locked_room_id matches create_plugin_room semantics)
|
||||||
_komm_contract = _get_contract("kommunikation")
|
|
||||||
from app.plugins.builtins.kommunikation.models import CommConversation
|
|
||||||
from sqlalchemy import select as sa_select
|
|
||||||
room_title = f"Agent: {agent.name}"
|
room_title = f"Agent: {agent.name}"
|
||||||
existing = await db.execute(
|
conv_id = await komm.find_locked_room_id(
|
||||||
sa_select(CommConversation).where(
|
db=db,
|
||||||
CommConversation.tenant_id == agent.tenant_id,
|
tenant_id=agent.tenant_id,
|
||||||
CommConversation.title == room_title,
|
plugin_name="automation",
|
||||||
CommConversation.is_locked.is_(True),
|
title=room_title,
|
||||||
CommConversation.locked_by == "automation",
|
|
||||||
CommConversation.deleted_at.is_(None),
|
|
||||||
)
|
)
|
||||||
)
|
if not conv_id:
|
||||||
conv = existing.scalar_one_or_none()
|
|
||||||
if not conv:
|
|
||||||
room = await komm.create_plugin_room(
|
room = await komm.create_plugin_room(
|
||||||
db=db,
|
db=db,
|
||||||
tenant_id=agent.tenant_id,
|
tenant_id=agent.tenant_id,
|
||||||
@@ -405,8 +399,6 @@ async def run_agent(
|
|||||||
participant_type="agent",
|
participant_type="agent",
|
||||||
)
|
)
|
||||||
conv_id = uuid.UUID(room["conversation_id"])
|
conv_id = uuid.UUID(room["conversation_id"])
|
||||||
else:
|
|
||||||
conv_id = conv.id
|
|
||||||
|
|
||||||
# Post result as message with action_card block
|
# Post result as message with action_card block
|
||||||
status = result_data.get("status", "unknown")
|
status = result_data.get("status", "unknown")
|
||||||
|
|||||||
@@ -0,0 +1,31 @@
|
|||||||
|
-- Dual-path convergence (Gate B): create the ReAct step-tracking table
|
||||||
|
-- that Alembic migration 0121 creates on the core path, add the Phase-F
|
||||||
|
-- columns from 0122, and apply the RLS policy from 0129/0136. Idempotent
|
||||||
|
-- so both install paths converge to the identical schema.
|
||||||
|
CREATE TABLE IF NOT EXISTS automation_agent_run_steps (
|
||||||
|
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||||
|
tenant_id UUID NOT NULL,
|
||||||
|
agent_run_id UUID NOT NULL REFERENCES automation_agent_runs(id) ON DELETE CASCADE,
|
||||||
|
step_number INTEGER NOT NULL,
|
||||||
|
thought TEXT,
|
||||||
|
action VARCHAR(255),
|
||||||
|
action_input JSONB,
|
||||||
|
observation TEXT,
|
||||||
|
cost_usd FLOAT NOT NULL DEFAULT 0.0,
|
||||||
|
created_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||||
|
);
|
||||||
|
CREATE INDEX IF NOT EXISTS ix_agent_run_steps_run ON automation_agent_run_steps(tenant_id, agent_run_id);
|
||||||
|
|
||||||
|
ALTER TABLE automation_agent_definitions ADD COLUMN IF NOT EXISTS temperature FLOAT NOT NULL DEFAULT 0.3;
|
||||||
|
ALTER TABLE automation_agent_definitions ADD COLUMN IF NOT EXISTS max_tokens INTEGER NOT NULL DEFAULT 1000;
|
||||||
|
ALTER TABLE automation_agent_definitions ADD COLUMN IF NOT EXISTS max_steps INTEGER NOT NULL DEFAULT 20;
|
||||||
|
ALTER TABLE automation_agent_definitions ADD COLUMN IF NOT EXISTS trace_mode VARCHAR(20) NOT NULL DEFAULT 'standard';
|
||||||
|
ALTER TABLE automation_agent_definitions ADD COLUMN IF NOT EXISTS skill_ids JSONB NOT NULL DEFAULT '[]'::jsonb;
|
||||||
|
ALTER TABLE automation_agent_definitions ADD COLUMN IF NOT EXISTS trigger_config JSONB NOT NULL DEFAULT '{}'::jsonb;
|
||||||
|
ALTER TABLE automation_agent_definitions ADD COLUMN IF NOT EXISTS ai_use_case_metadata JSONB NOT NULL DEFAULT '{}'::jsonb;
|
||||||
|
|
||||||
|
-- RLS matching migrations 0129 + 0136 (current_tenant_id variant)
|
||||||
|
ALTER TABLE automation_agent_run_steps ENABLE ROW LEVEL SECURITY;
|
||||||
|
DROP POLICY IF EXISTS tenant_isolation ON automation_agent_run_steps;
|
||||||
|
CREATE POLICY tenant_isolation ON automation_agent_run_steps
|
||||||
|
USING (tenant_id::text = current_setting('app.current_tenant_id', true));
|
||||||
@@ -38,7 +38,7 @@ class AutomationPlugin(BasePlugin):
|
|||||||
"Define AI agents with LLM models and tools, create event/schedule/manual "
|
"Define AI agents with LLM models and tools, create event/schedule/manual "
|
||||||
"automations with conditions and actions, schedule cron jobs, and track execution logs."
|
"automations with conditions and actions, schedule cron jobs, and track execution logs."
|
||||||
),
|
),
|
||||||
dependencies=[],
|
dependencies=["mail"],
|
||||||
routes=[
|
routes=[
|
||||||
PluginRouteDef(
|
PluginRouteDef(
|
||||||
path="/api/v1/automation",
|
path="/api/v1/automation",
|
||||||
@@ -62,7 +62,7 @@ class AutomationPlugin(BasePlugin):
|
|||||||
"mail.received",
|
"mail.received",
|
||||||
"workflow.timeout",
|
"workflow.timeout",
|
||||||
],
|
],
|
||||||
migrations=["0001_initial.sql", "0002_agent_subtasks.sql", "0003_skill_definitions.sql"],
|
migrations=["0001_initial.sql", "0002_agent_subtasks.sql", "0003_skill_definitions.sql", "0004_run_steps_phase_f.sql"],
|
||||||
permissions=[
|
permissions=[
|
||||||
"automation:read",
|
"automation:read",
|
||||||
"automation:write",
|
"automation:write",
|
||||||
@@ -82,6 +82,7 @@ class AutomationPlugin(BasePlugin):
|
|||||||
path="/workflows",
|
path="/workflows",
|
||||||
icon="Workflow",
|
icon="Workflow",
|
||||||
order=52,
|
order=52,
|
||||||
|
permission="automation:read",
|
||||||
),
|
),
|
||||||
FrontendMenuItem(
|
FrontendMenuItem(
|
||||||
label_key="nav.importExport",
|
label_key="nav.importExport",
|
||||||
@@ -89,6 +90,7 @@ class AutomationPlugin(BasePlugin):
|
|||||||
path="/import-export",
|
path="/import-export",
|
||||||
icon="ArrowUpDown",
|
icon="ArrowUpDown",
|
||||||
order=53,
|
order=53,
|
||||||
|
permission="import_export:read",
|
||||||
),
|
),
|
||||||
FrontendMenuItem(
|
FrontendMenuItem(
|
||||||
label_key="nav.dedupMerge",
|
label_key="nav.dedupMerge",
|
||||||
@@ -96,6 +98,7 @@ class AutomationPlugin(BasePlugin):
|
|||||||
path="/contacts/dedup",
|
path="/contacts/dedup",
|
||||||
icon="Copy",
|
icon="Copy",
|
||||||
order=54,
|
order=54,
|
||||||
|
permission="contacts:read",
|
||||||
),
|
),
|
||||||
FrontendMenuItem(
|
FrontendMenuItem(
|
||||||
label_key="nav.tags",
|
label_key="nav.tags",
|
||||||
@@ -103,6 +106,7 @@ class AutomationPlugin(BasePlugin):
|
|||||||
path="/tags",
|
path="/tags",
|
||||||
icon="Tag",
|
icon="Tag",
|
||||||
order=55,
|
order=55,
|
||||||
|
permission="tags:read",
|
||||||
),
|
),
|
||||||
FrontendMenuItem(
|
FrontendMenuItem(
|
||||||
label_key="nav.activity",
|
label_key="nav.activity",
|
||||||
@@ -110,6 +114,7 @@ class AutomationPlugin(BasePlugin):
|
|||||||
path="/activity",
|
path="/activity",
|
||||||
icon="Activity",
|
icon="Activity",
|
||||||
order=56,
|
order=56,
|
||||||
|
permission="contacts:read",
|
||||||
),
|
),
|
||||||
],
|
],
|
||||||
page_routes=[
|
page_routes=[
|
||||||
@@ -117,21 +122,25 @@ class AutomationPlugin(BasePlugin):
|
|||||||
path="/automation",
|
path="/automation",
|
||||||
component="@/pages/AutomationDashboard",
|
component="@/pages/AutomationDashboard",
|
||||||
order=50,
|
order=50,
|
||||||
|
permission="automation:read",
|
||||||
),
|
),
|
||||||
FrontendPageRoute(
|
FrontendPageRoute(
|
||||||
path="/agents",
|
path="/agents",
|
||||||
component="@/pages/AgentDashboard",
|
component="@/pages/AgentDashboard",
|
||||||
order=51,
|
order=51,
|
||||||
|
permission="agents:read",
|
||||||
),
|
),
|
||||||
FrontendPageRoute(
|
FrontendPageRoute(
|
||||||
path="/workflows",
|
path="/workflows",
|
||||||
component="@/pages/Workflows",
|
component="@/pages/Workflows",
|
||||||
order=52,
|
order=52,
|
||||||
|
permission="automation:read",
|
||||||
),
|
),
|
||||||
FrontendPageRoute(
|
FrontendPageRoute(
|
||||||
path="/import-export",
|
path="/import-export",
|
||||||
component="@/pages/ImportExport",
|
component="@/pages/ImportExport",
|
||||||
order=53,
|
order=53,
|
||||||
|
permission="import_export:read",
|
||||||
),
|
),
|
||||||
],
|
],
|
||||||
settings_pages=[
|
settings_pages=[
|
||||||
@@ -239,18 +248,25 @@ class AutomationPlugin(BasePlugin):
|
|||||||
logger.exception("Failed to register own cron jobs")
|
logger.exception("Failed to register own cron jobs")
|
||||||
# Register pre-built agents in DB (if not already present)
|
# Register pre-built agents in DB (if not already present)
|
||||||
try:
|
try:
|
||||||
from app.plugins.builtins.automation.models import AgentDefinition
|
|
||||||
from app.plugins.builtins.automation.prebuilt.email_triage_agent import create_email_triage_agent
|
|
||||||
from app.plugins.builtins.automation.prebuilt.contact_enrichment_agent import create_contact_enrichment_agent
|
|
||||||
from app.plugins.builtins.automation.prebuilt.follow_up_agent import create_follow_up_agent
|
|
||||||
from app.plugins.builtins.automation.prebuilt.report_agent import create_report_agent
|
|
||||||
from sqlalchemy import select as sa_select
|
from sqlalchemy import select as sa_select
|
||||||
|
|
||||||
# Get first tenant + admin user for seeding
|
# Get system tenant + admin user for seeding (ARCH-043:
|
||||||
from app.models.user import User
|
# deterministic slug lookup instead of arbitrary first row)
|
||||||
from app.models.tenant import Tenant
|
from app.core.db import get_system_tenant
|
||||||
tenant_result = await db.execute(sa_select(Tenant).limit(1))
|
from app.models.user import User, UserTenant
|
||||||
tenant = tenant_result.scalar_one_or_none()
|
from app.plugins.builtins.automation.models import AgentDefinition
|
||||||
|
from app.plugins.builtins.automation.prebuilt.contact_enrichment_agent import (
|
||||||
|
create_contact_enrichment_agent,
|
||||||
|
)
|
||||||
|
from app.plugins.builtins.automation.prebuilt.email_triage_agent import (
|
||||||
|
create_email_triage_agent,
|
||||||
|
)
|
||||||
|
from app.plugins.builtins.automation.prebuilt.follow_up_agent import (
|
||||||
|
create_follow_up_agent,
|
||||||
|
)
|
||||||
|
from app.plugins.builtins.automation.prebuilt.report_agent import create_report_agent
|
||||||
|
|
||||||
|
tenant = await get_system_tenant(db)
|
||||||
if tenant:
|
if tenant:
|
||||||
user_result = await db.execute(
|
user_result = await db.execute(
|
||||||
sa_select(User)
|
sa_select(User)
|
||||||
@@ -288,15 +304,14 @@ class AutomationPlugin(BasePlugin):
|
|||||||
def _register_workflow_agent_tools(self) -> None:
|
def _register_workflow_agent_tools(self) -> None:
|
||||||
"""Register I-AW agent tools for starting and inspecting workflows."""
|
"""Register I-AW agent tools for starting and inspecting workflows."""
|
||||||
import uuid
|
import uuid
|
||||||
from typing import Any
|
|
||||||
|
|
||||||
from app.ai.tool_registry import get_tool_registry
|
from app.ai.tool_registry import get_tool_registry
|
||||||
registry = get_tool_registry()
|
registry = get_tool_registry()
|
||||||
|
|
||||||
async def _start_workflow_handler(arguments: dict[str, Any], context: dict[str, Any]) -> dict[str, Any]:
|
async def _start_workflow_handler(arguments: dict[str, Any], context: dict[str, Any]) -> dict[str, Any]:
|
||||||
"""Start a workflow by ID."""
|
"""Start a workflow by ID."""
|
||||||
from app.services.workflow_service import create_instance
|
|
||||||
from app.core.db import get_worker_session_factory
|
from app.core.db import get_worker_session_factory
|
||||||
|
from app.services.workflow_service import create_instance
|
||||||
workflow_id = arguments.get("workflow_id", "")
|
workflow_id = arguments.get("workflow_id", "")
|
||||||
tenant_id = context.get("tenant_id")
|
tenant_id = context.get("tenant_id")
|
||||||
user_id = context.get("user_id")
|
user_id = context.get("user_id")
|
||||||
@@ -332,8 +347,9 @@ class AutomationPlugin(BasePlugin):
|
|||||||
async def _check_workflow_status_handler(arguments: dict[str, Any], context: dict[str, Any]) -> dict[str, Any]:
|
async def _check_workflow_status_handler(arguments: dict[str, Any], context: dict[str, Any]) -> dict[str, Any]:
|
||||||
"""Check the status of a workflow instance."""
|
"""Check the status of a workflow instance."""
|
||||||
from sqlalchemy import select
|
from sqlalchemy import select
|
||||||
from app.models.workflow import WorkflowInstance
|
|
||||||
from app.core.db import get_worker_session_factory
|
from app.core.db import get_worker_session_factory
|
||||||
|
from app.models.workflow import WorkflowInstance
|
||||||
instance_id = arguments.get("instance_id", "")
|
instance_id = arguments.get("instance_id", "")
|
||||||
tenant_id = context.get("tenant_id")
|
tenant_id = context.get("tenant_id")
|
||||||
if not instance_id or not tenant_id:
|
if not instance_id or not tenant_id:
|
||||||
@@ -418,16 +434,16 @@ class AutomationPlugin(BasePlugin):
|
|||||||
from another plugin's manifest. Uses plugin name prefixing for conflict resolution."""
|
from another plugin's manifest. Uses plugin name prefixing for conflict resolution."""
|
||||||
from sqlalchemy import select
|
from sqlalchemy import select
|
||||||
|
|
||||||
# Get default tenant_id from the first tenant in the DB
|
# Get system tenant for contributions (ARCH-043: deterministic slug
|
||||||
from app.models.tenant import Tenant
|
# lookup instead of arbitrary first row)
|
||||||
|
from app.core.db import get_system_tenant
|
||||||
from app.plugins.builtins.automation.models import AutomationCronJob
|
from app.plugins.builtins.automation.models import AutomationCronJob
|
||||||
from app.plugins.builtins.automation.services import (
|
from app.plugins.builtins.automation.services import (
|
||||||
AgentService,
|
AgentService,
|
||||||
AutomationService,
|
AutomationService,
|
||||||
CronJobService,
|
CronJobService,
|
||||||
)
|
)
|
||||||
tenant_result = await db.execute(select(Tenant).limit(1))
|
tenant = await get_system_tenant(db)
|
||||||
tenant = tenant_result.scalar_one_or_none()
|
|
||||||
default_tenant_id = tenant.id if tenant else None
|
default_tenant_id = tenant.id if tenant else None
|
||||||
if default_tenant_id is None:
|
if default_tenant_id is None:
|
||||||
logger.warning("No tenant found — skipping plugin contributions registration")
|
logger.warning("No tenant found — skipping plugin contributions registration")
|
||||||
|
|||||||
@@ -3,7 +3,9 @@
|
|||||||
Enriches contact data by searching for related information.
|
Enriches contact data by searching for related information.
|
||||||
"""
|
"""
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import uuid
|
import uuid
|
||||||
|
|
||||||
from app.plugins.builtins.automation.models import AgentDefinition
|
from app.plugins.builtins.automation.models import AgentDefinition
|
||||||
|
|
||||||
CONTACT_ENRICHMENT_SYSTEM_PROMPT = """You are a Contact Enrichment Agent for a CRM system.
|
CONTACT_ENRICHMENT_SYSTEM_PROMPT = """You are a Contact Enrichment Agent for a CRM system.
|
||||||
|
|||||||
@@ -3,7 +3,9 @@
|
|||||||
Sorts and prioritizes incoming emails automatically.
|
Sorts and prioritizes incoming emails automatically.
|
||||||
"""
|
"""
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import uuid
|
import uuid
|
||||||
|
|
||||||
from app.plugins.builtins.automation.models import AgentDefinition
|
from app.plugins.builtins.automation.models import AgentDefinition
|
||||||
|
|
||||||
EMAIL_TRIAGE_SYSTEM_PROMPT = """You are an E-Mail Triage Agent for a CRM system.
|
EMAIL_TRIAGE_SYSTEM_PROMPT = """You are an E-Mail Triage Agent for a CRM system.
|
||||||
|
|||||||
@@ -3,7 +3,9 @@
|
|||||||
Reminds about and creates follow-up tasks for contacts.
|
Reminds about and creates follow-up tasks for contacts.
|
||||||
"""
|
"""
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import uuid
|
import uuid
|
||||||
|
|
||||||
from app.plugins.builtins.automation.models import AgentDefinition
|
from app.plugins.builtins.automation.models import AgentDefinition
|
||||||
|
|
||||||
FOLLOW_UP_SYSTEM_PROMPT = """You are a Follow-up Agent for a CRM system.
|
FOLLOW_UP_SYSTEM_PROMPT = """You are a Follow-up Agent for a CRM system.
|
||||||
|
|||||||
@@ -3,7 +3,9 @@
|
|||||||
Generates reports from CRM data using search and API tools.
|
Generates reports from CRM data using search and API tools.
|
||||||
"""
|
"""
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import uuid
|
import uuid
|
||||||
|
|
||||||
from app.plugins.builtins.automation.models import AgentDefinition
|
from app.plugins.builtins.automation.models import AgentDefinition
|
||||||
|
|
||||||
REPORT_SYSTEM_PROMPT = """You are a Report Agent for a CRM system.
|
REPORT_SYSTEM_PROMPT = """You are a Report Agent for a CRM system.
|
||||||
|
|||||||
@@ -1,60 +1,132 @@
|
|||||||
"""Tests for the Automation & Agents plugin.
|
"""Tests for the Automation & Agents plugin.
|
||||||
|
|
||||||
Uses pytest with async fixtures. Tests use SQLite in-memory database
|
Uses pytest with async fixtures against an ephemeral PostgreSQL database
|
||||||
since PostgreSQL may not be available in the dev container.
|
(SQLITE-001 fix) — matches the project convention and exercises the real
|
||||||
|
PGUUID/JSONB column types.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
|
# Register ALL plugin models so create_all can resolve cross-plugin FKs
|
||||||
|
# (e.g. entity_attachments.dms_file_id -> files) — same pattern as
|
||||||
|
# scripts/sync_plugin_schema.py.
|
||||||
|
import importlib
|
||||||
|
import os
|
||||||
|
import pkgutil
|
||||||
import uuid
|
import uuid
|
||||||
from collections.abc import AsyncGenerator
|
from collections.abc import AsyncGenerator
|
||||||
from datetime import UTC, datetime, timedelta
|
from datetime import UTC, datetime, timedelta
|
||||||
|
|
||||||
import pytest
|
import pytest
|
||||||
import pytest_asyncio
|
import pytest_asyncio
|
||||||
|
from sqlalchemy import text
|
||||||
from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker, create_async_engine
|
from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker, create_async_engine
|
||||||
|
|
||||||
|
import app.models # noqa: F401 — registers core models
|
||||||
|
import app.models.outbox # noqa: F401 — event_outbox is NOT re-exported by app.models
|
||||||
|
import app.plugins.builtins as _builtins_pkg
|
||||||
from app.core.db import Base
|
from app.core.db import Base
|
||||||
from app.plugins.builtins.automation.models import (
|
|
||||||
|
for _importer, _modname, _ispkg in pkgutil.iter_modules(_builtins_pkg.__path__):
|
||||||
|
if not _ispkg:
|
||||||
|
continue
|
||||||
|
try:
|
||||||
|
importlib.import_module(f"app.plugins.builtins.{_modname}.models")
|
||||||
|
except ImportError:
|
||||||
|
pass # plugin without models module
|
||||||
|
except Exception: # pragma: no cover - defensive
|
||||||
|
pass
|
||||||
|
|
||||||
|
from app.plugins.builtins.automation.models import ( # noqa: E402 — after dynamic plugin-model discovery
|
||||||
AgentRun,
|
AgentRun,
|
||||||
AutomationRun,
|
AutomationRun,
|
||||||
)
|
)
|
||||||
from app.plugins.builtins.automation.services import (
|
from app.plugins.builtins.automation.services import ( # noqa: E402 — after dynamic plugin-model discovery
|
||||||
AgentService,
|
AgentService,
|
||||||
AutomationService,
|
AutomationService,
|
||||||
CronJobService,
|
CronJobService,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _ephemeral_db_url() -> str:
|
||||||
|
"""Derive an ephemeral test DB URL from DATABASE_URL/.env.test."""
|
||||||
|
base_url = os.environ.get(
|
||||||
|
"DATABASE_URL",
|
||||||
|
"postgresql+asyncpg://leocrm_test:test123@localhost:5432/leocrm_test",
|
||||||
|
)
|
||||||
|
return f"{base_url.rsplit('/', 1)[0]}/automation_test_{uuid.uuid4().hex[:8]}"
|
||||||
|
|
||||||
|
|
||||||
# ─── Fixtures ───
|
# ─── Fixtures ───
|
||||||
|
|
||||||
|
|
||||||
@pytest_asyncio.fixture
|
@pytest_asyncio.fixture
|
||||||
async def db() -> AsyncGenerator[AsyncSession, None]:
|
async def db() -> AsyncGenerator[AsyncSession, None]:
|
||||||
"""Create an in-memory SQLite database for testing."""
|
"""Create an ephemeral PostgreSQL database for this test run."""
|
||||||
engine = create_async_engine(
|
db_url = _ephemeral_db_url()
|
||||||
"sqlite+aiosqlite:///:memory:",
|
admin_url = db_url.rsplit("/", 1)[0] + "/postgres"
|
||||||
echo=False,
|
|
||||||
)
|
from sqlalchemy.ext.asyncio import create_async_engine as _cae
|
||||||
|
|
||||||
|
admin_engine = _cae(admin_url, isolation_level="AUTOCOMMIT")
|
||||||
|
async with admin_engine.connect() as conn:
|
||||||
|
await conn.execute(text(f'CREATE DATABASE "{db_url.rsplit("/", 1)[1]}"'))
|
||||||
|
await admin_engine.dispose()
|
||||||
|
|
||||||
|
# Plugin models use the pgvector Vector type — enable the extension in
|
||||||
|
# the fresh database before create_all runs (must connect to the target
|
||||||
|
# DB itself; CREATE EXTENSION has no ON DATABASE clause).
|
||||||
|
ext_engine = _cae(db_url, isolation_level="AUTOCOMMIT")
|
||||||
|
async with ext_engine.connect() as conn:
|
||||||
|
await conn.execute(text("CREATE EXTENSION IF NOT EXISTS vector"))
|
||||||
|
await ext_engine.dispose()
|
||||||
|
|
||||||
|
engine = create_async_engine(db_url, echo=False)
|
||||||
|
|
||||||
async with engine.begin() as conn:
|
async with engine.begin() as conn:
|
||||||
await conn.run_sync(Base.metadata.create_all)
|
await conn.run_sync(Base.metadata.create_all)
|
||||||
|
|
||||||
async_session = async_sessionmaker(engine, class_=AsyncSession, expire_on_commit=False)
|
async_session = async_sessionmaker(engine, class_=AsyncSession, expire_on_commit=False)
|
||||||
|
|
||||||
|
try:
|
||||||
async with async_session() as session:
|
async with async_session() as session:
|
||||||
yield session
|
yield session
|
||||||
|
finally:
|
||||||
await engine.dispose()
|
await engine.dispose()
|
||||||
|
admin_engine2 = _cae(admin_url, isolation_level="AUTOCOMMIT")
|
||||||
|
async with admin_engine2.connect() as conn:
|
||||||
|
await conn.execute(text(f'DROP DATABASE IF EXISTS "{db_url.rsplit("/", 1)[1]}"'))
|
||||||
|
await admin_engine2.dispose()
|
||||||
|
|
||||||
|
|
||||||
@pytest.fixture
|
@pytest_asyncio.fixture
|
||||||
def tenant_id() -> uuid.UUID:
|
async def tenant_id(db: AsyncSession) -> uuid.UUID:
|
||||||
return uuid.uuid4()
|
"""Create a real tenant row — PostgreSQL enforces FKs, unlike SQLite."""
|
||||||
|
from app.models.tenant import Tenant
|
||||||
|
|
||||||
|
tid = uuid.uuid4()
|
||||||
|
db.add(Tenant(id=tid, name="Test Org", slug=f"test-{tid.hex[:8]}"))
|
||||||
|
await db.commit()
|
||||||
|
return tid
|
||||||
|
|
||||||
|
|
||||||
@pytest.fixture
|
@pytest_asyncio.fixture
|
||||||
def user_id() -> uuid.UUID:
|
async def user_id(db: AsyncSession, tenant_id: uuid.UUID) -> uuid.UUID:
|
||||||
return uuid.uuid4()
|
"""Create a real user row belonging to the test tenant."""
|
||||||
|
from app.models.user import User
|
||||||
|
|
||||||
|
uid = uuid.uuid4()
|
||||||
|
db.add(
|
||||||
|
User(
|
||||||
|
id=uid,
|
||||||
|
email=f"test-{uid.hex[:8]}@example.com",
|
||||||
|
name="Test User",
|
||||||
|
password_hash="not-a-real-hash",
|
||||||
|
is_active=True,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
await db.commit()
|
||||||
|
return uid
|
||||||
|
|
||||||
|
|
||||||
# ─── AgentService Tests ───
|
# ─── AgentService Tests ───
|
||||||
@@ -425,11 +497,17 @@ class TestDryRunMode:
|
|||||||
assert automation.dry_run is True
|
assert automation.dry_run is True
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
@pytest.mark.asyncio
|
||||||
async def test_dry_run_flag_in_run(self, db: AsyncSession, tenant_id: uuid.UUID):
|
async def test_dry_run_flag_in_run(self, db: AsyncSession, tenant_id: uuid.UUID, user_id: uuid.UUID):
|
||||||
"""Test that dry_run flag is stored in AutomationRun."""
|
"""Test that dry_run flag is stored in AutomationRun."""
|
||||||
|
# PostgreSQL enforces the FK to automations — create a real one first
|
||||||
|
data = {"name": "dry-run-flag", "description": "", "trigger_type": "manual",
|
||||||
|
"trigger_config": {}, "conditions": [], "actions": [],
|
||||||
|
"is_active": True, "dry_run": True}
|
||||||
|
automation = await AutomationService.create(db, tenant_id, data, user_id=user_id)
|
||||||
|
|
||||||
run = AutomationRun(
|
run = AutomationRun(
|
||||||
tenant_id=tenant_id,
|
tenant_id=tenant_id,
|
||||||
automation_id=uuid.uuid4(),
|
automation_id=automation.id,
|
||||||
status="dry_run",
|
status="dry_run",
|
||||||
started_at=datetime.now(UTC),
|
started_at=datetime.now(UTC),
|
||||||
dry_run=True,
|
dry_run=True,
|
||||||
@@ -475,7 +553,9 @@ class TestRateLimiting:
|
|||||||
)
|
)
|
||||||
recent_runs = result.scalar() or 0
|
recent_runs = result.scalar() or 0
|
||||||
assert recent_runs == 2
|
assert recent_runs == 2
|
||||||
assert recent_runs < agent.max_executions_per_hour # 2 < 2 is False, so limit would be hit
|
# With max_executions_per_hour=2 and 2 runs in the window, the limit
|
||||||
|
# is reached — the next execution must be blocked.
|
||||||
|
assert recent_runs >= agent.max_executions_per_hour
|
||||||
|
|
||||||
|
|
||||||
# ─── Budget Limit Tests ───
|
# ─── Budget Limit Tests ───
|
||||||
@@ -512,7 +592,8 @@ class TestBudgetLimit:
|
|||||||
.where(AgentRun.agent_id == agent.id)
|
.where(AgentRun.agent_id == agent.id)
|
||||||
)
|
)
|
||||||
total_cost = float(cost_result.scalar() or 0.0)
|
total_cost = float(cost_result.scalar() or 0.0)
|
||||||
assert total_cost == 0.6
|
# FLOAT column accumulates binary rounding (0.6000000000000001)
|
||||||
|
assert total_cost == pytest.approx(0.6)
|
||||||
assert total_cost >= agent.budget_limit_usd # 0.6 >= 0.5, budget exceeded
|
assert total_cost >= agent.budget_limit_usd # 0.6 >= 0.5, budget exceeded
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -6,6 +6,7 @@ import uuid
|
|||||||
from datetime import datetime
|
from datetime import datetime
|
||||||
from typing import Any
|
from typing import Any
|
||||||
|
|
||||||
|
from pgvector.sqlalchemy import Vector
|
||||||
from sqlalchemy import (
|
from sqlalchemy import (
|
||||||
Boolean,
|
Boolean,
|
||||||
DateTime,
|
DateTime,
|
||||||
@@ -13,14 +14,12 @@ from sqlalchemy import (
|
|||||||
Index,
|
Index,
|
||||||
String,
|
String,
|
||||||
)
|
)
|
||||||
from sqlalchemy.dialects.postgresql import JSONB
|
from sqlalchemy.dialects.postgresql import JSONB, TSVECTOR
|
||||||
from sqlalchemy.dialects.postgresql import UUID as PGUUID
|
from sqlalchemy.dialects.postgresql import UUID as PGUUID
|
||||||
from sqlalchemy.orm import Mapped, mapped_column
|
from sqlalchemy.orm import Mapped, mapped_column
|
||||||
|
|
||||||
from app.core.db import Base, TenantMixin
|
from app.core.db import Base, TenantMixin
|
||||||
from app.models.owned_mixin import OwnedMixin
|
from app.models.owned_mixin import OwnedMixin
|
||||||
from sqlalchemy.dialects.postgresql import TSVECTOR
|
|
||||||
from pgvector.sqlalchemy import Vector
|
|
||||||
|
|
||||||
|
|
||||||
class Calendar(Base, TenantMixin, OwnedMixin):
|
class Calendar(Base, TenantMixin, OwnedMixin):
|
||||||
|
|||||||
@@ -4,7 +4,7 @@ from __future__ import annotations
|
|||||||
|
|
||||||
from app.plugins.base import BasePlugin
|
from app.plugins.base import BasePlugin
|
||||||
from app.plugins.manifest import (
|
from app.plugins.manifest import (
|
||||||
FrontendDetailTab,
|
FrontendDashboardWidget,
|
||||||
FrontendMenuItem,
|
FrontendMenuItem,
|
||||||
FrontendPageRoute,
|
FrontendPageRoute,
|
||||||
PluginManifest,
|
PluginManifest,
|
||||||
@@ -40,6 +40,18 @@ class CalendarPlugin(BasePlugin):
|
|||||||
],
|
],
|
||||||
events=[],
|
events=[],
|
||||||
migrations=["0001_initial.sql", "0002_add_deleted_at.sql"],
|
migrations=["0001_initial.sql", "0002_add_deleted_at.sql"],
|
||||||
|
dashboard_widgets=[
|
||||||
|
FrontendDashboardWidget(
|
||||||
|
id="calendar_upcoming",
|
||||||
|
label_key="dashboard.calendarUpcoming",
|
||||||
|
label="Upcoming Appointments",
|
||||||
|
component="@/components/dashboard/CalendarUpcomingWidget",
|
||||||
|
icon="Calendar",
|
||||||
|
order=30,
|
||||||
|
col_span=1,
|
||||||
|
permission="calendar:read",
|
||||||
|
),
|
||||||
|
],
|
||||||
permissions=[
|
permissions=[
|
||||||
"calendar:read",
|
"calendar:read",
|
||||||
"calendar:write",
|
"calendar:write",
|
||||||
@@ -48,14 +60,14 @@ class CalendarPlugin(BasePlugin):
|
|||||||
"calendar:admin",
|
"calendar:admin",
|
||||||
],
|
],
|
||||||
menu_items=[
|
menu_items=[
|
||||||
FrontendMenuItem(label_key='nav.calendar', label='Kalender', path='/calendar', icon='Calendar', order=20),
|
FrontendMenuItem(label_key='nav.calendar', label='Kalender', path='/calendar', icon='Calendar', order=20, permission='calendar:read'),
|
||||||
],
|
],
|
||||||
page_routes=[
|
page_routes=[
|
||||||
FrontendPageRoute(path='/calendar', component='@/pages/Calendar', protected=True),
|
FrontendPageRoute(path='/calendar', component='@/pages/Calendar', protected=True, permission='calendar:read'),
|
||||||
],
|
|
||||||
detail_tabs=[
|
|
||||||
FrontendDetailTab(entity_type='contact', label_key='tabs.calendar', label='Calendar', component='@/components/contact/ContactCalendarTab', icon='Calendar', order=30, permission='calendar:read'),
|
|
||||||
],
|
],
|
||||||
|
# BUG (ghost component): ContactCalendarTab does not exist in the
|
||||||
|
# frontend — tab removed until implemented (Block I-D).
|
||||||
|
detail_tabs=[],
|
||||||
author="LeoCRM Team",
|
author="LeoCRM Team",
|
||||||
min_app_version="1.0.0",
|
min_app_version="1.0.0",
|
||||||
hooks=["calendar.before_appointment", "calendar.after_appointment"],
|
hooks=["calendar.before_appointment", "calendar.after_appointment"],
|
||||||
|
|||||||
@@ -21,6 +21,7 @@ from fastapi.responses import StreamingResponse
|
|||||||
from sqlalchemy import select, update
|
from sqlalchemy import select, update
|
||||||
from sqlalchemy.ext.asyncio import AsyncSession
|
from sqlalchemy.ext.asyncio import AsyncSession
|
||||||
|
|
||||||
|
from app.core.audit import log_audit
|
||||||
from app.core.db import get_db
|
from app.core.db import get_db
|
||||||
from app.deps import get_current_user, require_admin, require_permission
|
from app.deps import get_current_user, require_admin, require_permission
|
||||||
from app.plugins.builtins.calendar.ics_utils import (
|
from app.plugins.builtins.calendar.ics_utils import (
|
||||||
@@ -1023,5 +1024,3 @@ async def book_resource(
|
|||||||
"start_at": booking.start_at.isoformat(),
|
"start_at": booking.start_at.isoformat(),
|
||||||
"end_at": booking.end_at.isoformat(),
|
"end_at": booking.end_at.isoformat(),
|
||||||
}
|
}
|
||||||
|
|
||||||
from app.core.audit import log_audit
|
|
||||||
@@ -0,0 +1,62 @@
|
|||||||
|
"""Public contract for the contacts plugin.
|
||||||
|
|
||||||
|
Exposes the symbols that other core modules and plugins need without
|
||||||
|
importing from internal modules directly (Block C7: dashboard counts).
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
from sqlalchemy import func, select
|
||||||
|
from sqlalchemy.ext.asyncio import AsyncSession
|
||||||
|
|
||||||
|
from app.core.visibility import apply_visibility_filter
|
||||||
|
from app.models.contact import Contact
|
||||||
|
from app.plugins.builtins.contracts import get_contract_registry
|
||||||
|
|
||||||
|
|
||||||
|
class ContactsContract:
|
||||||
|
"""Public API surface for the contacts plugin."""
|
||||||
|
|
||||||
|
contract_name = "contacts"
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
async def get_counts(
|
||||||
|
db: AsyncSession,
|
||||||
|
tenant_id: Any,
|
||||||
|
user_id: Any,
|
||||||
|
is_system_admin: bool = False,
|
||||||
|
) -> dict[str, int]:
|
||||||
|
"""Return visibility-filtered contact/company/person counts."""
|
||||||
|
queries = []
|
||||||
|
for type_filter in (None, "company", "person"):
|
||||||
|
query = select(func.count(Contact.id)).where(
|
||||||
|
Contact.tenant_id == tenant_id,
|
||||||
|
Contact.deleted_at.is_(None),
|
||||||
|
)
|
||||||
|
if type_filter is not None:
|
||||||
|
query = query.where(Contact.type == type_filter)
|
||||||
|
query = await apply_visibility_filter(
|
||||||
|
db, query, "contact", Contact, user_id, tenant_id, is_system_admin
|
||||||
|
)
|
||||||
|
queries.append(query)
|
||||||
|
|
||||||
|
results = [((await db.execute(q)).scalar() or 0) for q in queries]
|
||||||
|
return {
|
||||||
|
"contacts": results[0],
|
||||||
|
"companies": results[1],
|
||||||
|
"persons": results[2],
|
||||||
|
"total": results[0],
|
||||||
|
}
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def get_function(cls, name: str):
|
||||||
|
"""Return a callable exposed by this contract, or None if absent."""
|
||||||
|
return getattr(cls, name, None)
|
||||||
|
|
||||||
|
|
||||||
|
# ─── self-registration ───
|
||||||
|
|
||||||
|
_contract = ContactsContract()
|
||||||
|
get_contract_registry().register("contacts", _contract)
|
||||||
@@ -9,27 +9,67 @@ from __future__ import annotations
|
|||||||
import logging
|
import logging
|
||||||
|
|
||||||
from app.plugins.base import BasePlugin
|
from app.plugins.base import BasePlugin
|
||||||
from app.plugins.manifest import PluginManifest
|
from app.plugins.manifest import (
|
||||||
|
FrontendDashboardWidget,
|
||||||
|
PluginManifest,
|
||||||
|
PluginRouteDef,
|
||||||
|
)
|
||||||
|
|
||||||
logger = logging.getLogger(__name__)
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
|
||||||
class ContactsPlugin(BasePlugin):
|
class ContactsPlugin(BasePlugin):
|
||||||
"""Contacts plugin — manages Contact entity lifecycle (models, permissions, restore, history).
|
"""Contacts plugin — owns the full Contact domain (Block B1).
|
||||||
|
|
||||||
Routes remain in app/routes/contacts.py as core routes, but entity lifecycle
|
Routes (contacts, companies, contact folders, folder permissions) live in
|
||||||
(permissions, entity models, restore, history) is managed through on_activate/on_deactivate.
|
this plugin and are mounted via manifest.routes with
|
||||||
|
require_active_plugin("contacts") protection. Entity lifecycle
|
||||||
|
(permissions, entity models, restore, history) is managed through
|
||||||
|
on_activate/on_deactivate like every other business plugin.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
manifest = PluginManifest(
|
manifest = PluginManifest(
|
||||||
name="contacts",
|
name="contacts",
|
||||||
version="1.0.0",
|
version="1.1.0",
|
||||||
display_name="Contacts",
|
display_name="Contacts",
|
||||||
description="Core CRM contacts — persons and companies.",
|
description="Core CRM contacts — persons and companies.",
|
||||||
dependencies=[],
|
dependencies=[],
|
||||||
routes=[], # Routes are registered as core routes in main.py
|
routes=[
|
||||||
|
PluginRouteDef(
|
||||||
|
path="/api/v1/contacts",
|
||||||
|
module="app.plugins.builtins.contacts.routes",
|
||||||
|
router_attr="router",
|
||||||
|
),
|
||||||
|
PluginRouteDef(
|
||||||
|
path="/api/v1/companies",
|
||||||
|
module="app.plugins.builtins.contacts.company_routes",
|
||||||
|
router_attr="router",
|
||||||
|
),
|
||||||
|
PluginRouteDef(
|
||||||
|
path="/api/v1/contact-folders",
|
||||||
|
module="app.plugins.builtins.contacts.folder_routes",
|
||||||
|
router_attr="router",
|
||||||
|
),
|
||||||
|
PluginRouteDef(
|
||||||
|
path="/api/v1/contact-folders",
|
||||||
|
module="app.plugins.builtins.contacts.folder_permission_routes",
|
||||||
|
router_attr="router",
|
||||||
|
),
|
||||||
|
],
|
||||||
events=[],
|
events=[],
|
||||||
migrations=[],
|
migrations=[],
|
||||||
|
dashboard_widgets=[
|
||||||
|
FrontendDashboardWidget(
|
||||||
|
id="recent_contacts",
|
||||||
|
label_key="dashboard.recentContacts",
|
||||||
|
label="Recent Contacts",
|
||||||
|
component="@/components/dashboard/RecentContactsWidget",
|
||||||
|
icon="Users",
|
||||||
|
order=10,
|
||||||
|
col_span=2,
|
||||||
|
permission="contacts:read",
|
||||||
|
),
|
||||||
|
],
|
||||||
permissions=[
|
permissions=[
|
||||||
"contacts:read",
|
"contacts:read",
|
||||||
"contacts:write",
|
"contacts:write",
|
||||||
|
|||||||
@@ -59,20 +59,31 @@ class ContractRegistry:
|
|||||||
cls._instance = super().__new__(cls)
|
cls._instance = super().__new__(cls)
|
||||||
cls._instance._contracts: dict[str, Any] = {}
|
cls._instance._contracts: dict[str, Any] = {}
|
||||||
cls._instance._loaded: set[str] = set()
|
cls._instance._loaded: set[str] = set()
|
||||||
|
cls._instance._unregistered: set[str] = set()
|
||||||
return cls._instance
|
return cls._instance
|
||||||
|
|
||||||
# ─── registration ───
|
# ─── registration ───
|
||||||
|
|
||||||
def register(self, plugin_name: str, contract: Any) -> None:
|
def register(self, plugin_name: str, contract: Any) -> None:
|
||||||
"""Register or replace a contract for a plugin."""
|
"""Register or replace a contract for a plugin.
|
||||||
|
|
||||||
|
Clears the unregistered marker so a later deactivation can be
|
||||||
|
distinguished from a fresh lazy-load again (ARCH-014).
|
||||||
|
"""
|
||||||
|
self._unregistered.discard(plugin_name)
|
||||||
self._contracts[plugin_name] = contract
|
self._contracts[plugin_name] = contract
|
||||||
self._loaded.add(plugin_name)
|
self._loaded.add(plugin_name)
|
||||||
logger.debug("Contract registered for plugin '%s'", plugin_name)
|
logger.debug("Contract registered for plugin '%s'", plugin_name)
|
||||||
|
|
||||||
def unregister(self, plugin_name: str) -> None:
|
def unregister(self, plugin_name: str) -> None:
|
||||||
"""Remove a contract (e.g. when the plugin is deactivated)."""
|
"""Remove a contract (e.g. when the plugin is deactivated).
|
||||||
|
|
||||||
|
Marks the plugin as explicitly unregistered so later ``get_contract``
|
||||||
|
calls cannot resurrect the contract via lazy-loading (ARCH-014).
|
||||||
|
"""
|
||||||
self._contracts.pop(plugin_name, None)
|
self._contracts.pop(plugin_name, None)
|
||||||
self._loaded.discard(plugin_name)
|
self._loaded.discard(plugin_name)
|
||||||
|
self._unregistered.add(plugin_name)
|
||||||
|
|
||||||
# ─── lookup ───
|
# ─── lookup ───
|
||||||
|
|
||||||
@@ -85,6 +96,11 @@ class ContractRegistry:
|
|||||||
if plugin_name in self._contracts:
|
if plugin_name in self._contracts:
|
||||||
return self._contracts[plugin_name]
|
return self._contracts[plugin_name]
|
||||||
|
|
||||||
|
# Explicitly unregistered (deactivated): never resurrect via
|
||||||
|
# lazy-loading (ARCH-014) — the deactivated contract must stay gone.
|
||||||
|
if plugin_name in self._unregistered:
|
||||||
|
return None
|
||||||
|
|
||||||
if plugin_name not in self._loaded:
|
if plugin_name not in self._loaded:
|
||||||
self._try_lazy_load(plugin_name)
|
self._try_lazy_load(plugin_name)
|
||||||
|
|
||||||
|
|||||||
@@ -4,16 +4,16 @@ from __future__ import annotations
|
|||||||
|
|
||||||
import uuid
|
import uuid
|
||||||
from datetime import datetime
|
from datetime import datetime
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
from sqlalchemy import DateTime, ForeignKey, Index, Integer, String, UniqueConstraint, Text
|
from pgvector.sqlalchemy import Vector
|
||||||
|
from sqlalchemy import DateTime, ForeignKey, Index, Integer, String, Text, UniqueConstraint
|
||||||
|
from sqlalchemy.dialects.postgresql import TSVECTOR
|
||||||
from sqlalchemy.dialects.postgresql import UUID as PGUUID
|
from sqlalchemy.dialects.postgresql import UUID as PGUUID
|
||||||
from sqlalchemy.orm import Mapped, mapped_column
|
from sqlalchemy.orm import Mapped, mapped_column
|
||||||
|
|
||||||
from app.core.db import Base, TenantMixin
|
from app.core.db import Base, TenantMixin
|
||||||
from app.models.owned_mixin import OwnedMixin
|
from app.models.owned_mixin import OwnedMixin
|
||||||
from pgvector.sqlalchemy import Vector
|
|
||||||
from sqlalchemy.dialects.postgresql import TSVECTOR
|
|
||||||
from typing import Any
|
|
||||||
|
|
||||||
|
|
||||||
class Folder(Base, TenantMixin, OwnedMixin):
|
class Folder(Base, TenantMixin, OwnedMixin):
|
||||||
|
|||||||
@@ -4,7 +4,6 @@ from __future__ import annotations
|
|||||||
|
|
||||||
from app.plugins.base import BasePlugin
|
from app.plugins.base import BasePlugin
|
||||||
from app.plugins.manifest import (
|
from app.plugins.manifest import (
|
||||||
FrontendDetailTab,
|
|
||||||
FrontendMenuItem,
|
FrontendMenuItem,
|
||||||
FrontendPageRoute,
|
FrontendPageRoute,
|
||||||
PluginManifest,
|
PluginManifest,
|
||||||
@@ -38,22 +37,35 @@ class DmsPlugin(BasePlugin):
|
|||||||
"dms:admin",
|
"dms:admin",
|
||||||
],
|
],
|
||||||
menu_items=[
|
menu_items=[
|
||||||
FrontendMenuItem(label_key='nav.dms', label='Dateien', path='/dms', icon='FolderOpen', group='Dateien', order=40),
|
FrontendMenuItem(label_key='nav.dms', label='Dateien', path='/dms', icon='FolderOpen', group='Dateien', order=40, permission='dms:read'),
|
||||||
FrontendMenuItem(label_key='nav.dms.trash', label='Papierkorb', path='/dms/trash', icon='Trash2', group='Dateien', order=41),
|
FrontendMenuItem(label_key='nav.dms.trash', label='Papierkorb', path='/dms/trash', icon='Trash2', group='Dateien', order=41, permission='dms:read'),
|
||||||
],
|
],
|
||||||
page_routes=[
|
page_routes=[
|
||||||
FrontendPageRoute(path='/dms', component='@/pages/Dms', protected=True),
|
FrontendPageRoute(path='/dms', component='@/pages/Dms', protected=True, permission='dms:read'),
|
||||||
FrontendPageRoute(path='/dms/trash', component='@/pages/DmsTrash', protected=True),
|
FrontendPageRoute(path='/dms/trash', component='@/pages/DmsTrash', protected=True, permission='dms:read'),
|
||||||
],
|
|
||||||
detail_tabs=[
|
|
||||||
FrontendDetailTab(entity_type='contact', label_key='tabs.files', label='Dateien', component='@/components/contact/ContactFilesTab', icon='FolderOpen', order=40, permission='dms:read'),
|
|
||||||
],
|
],
|
||||||
|
# BUG (ghost component): ContactFilesTab does not exist in the
|
||||||
|
# frontend — tab removed until implemented (Block I-D).
|
||||||
|
detail_tabs=[],
|
||||||
author="LeoCRM Team",
|
author="LeoCRM Team",
|
||||||
min_app_version="1.0.0",
|
min_app_version="1.0.0",
|
||||||
hooks=["dms.before_upload"],
|
hooks=["dms.before_upload"],
|
||||||
contract_version="1.0.0",
|
contract_version="1.0.0",
|
||||||
)
|
)
|
||||||
|
|
||||||
|
def get_entity_models(self) -> dict[str, type]:
|
||||||
|
"""Entity types owned by DMS for the permission resolver.
|
||||||
|
|
||||||
|
``file`` is a legacy alias still used by the permissions routes.
|
||||||
|
"""
|
||||||
|
from app.plugins.builtins.dms.models import File, Folder
|
||||||
|
|
||||||
|
return {
|
||||||
|
"dms_file": File,
|
||||||
|
"dms_folder": Folder,
|
||||||
|
"file": File,
|
||||||
|
}
|
||||||
|
|
||||||
async def on_activate(self, db, service_container, event_bus) -> None:
|
async def on_activate(self, db, service_container, event_bus) -> None:
|
||||||
"""Activate plugin: register restore config + history hooks."""
|
"""Activate plugin: register restore config + history hooks."""
|
||||||
await super().on_activate(db, service_container, event_bus)
|
await super().on_activate(db, service_container, event_bus)
|
||||||
|
|||||||
@@ -5,7 +5,7 @@ from __future__ import annotations
|
|||||||
from typing import Any
|
from typing import Any
|
||||||
|
|
||||||
from app.plugins.base import BasePlugin
|
from app.plugins.base import BasePlugin
|
||||||
from app.plugins.manifest import FrontendDetailTab, PluginManifest, PluginRouteDef
|
from app.plugins.manifest import PluginManifest, PluginRouteDef
|
||||||
|
|
||||||
|
|
||||||
class EntityLinksPlugin(BasePlugin):
|
class EntityLinksPlugin(BasePlugin):
|
||||||
@@ -42,9 +42,9 @@ class EntityLinksPlugin(BasePlugin):
|
|||||||
"entity_links:delete",
|
"entity_links:delete",
|
||||||
],
|
],
|
||||||
is_core=True,
|
is_core=True,
|
||||||
detail_tabs=[
|
# BUG (ghost component): ContactLinksTab does not exist in the
|
||||||
FrontendDetailTab(entity_type='contact', label_key='tabs.links', label='Verknüpfungen', component='@/components/contact/ContactLinksTab', icon='Link', order=60, permission='entity_links:read'),
|
# frontend — tab removed until implemented (Block I-D).
|
||||||
],
|
detail_tabs=[],
|
||||||
|
|
||||||
author="LeoCRM Team",
|
author="LeoCRM Team",
|
||||||
min_app_version="1.0.0",
|
min_app_version="1.0.0",
|
||||||
|
|||||||
@@ -40,6 +40,10 @@ class GraphRAGPlugin(BasePlugin):
|
|||||||
|
|
||||||
async def on_activate(self, db, service_container, event_bus) -> None:
|
async def on_activate(self, db, service_container, event_bus) -> None:
|
||||||
"""Activate plugin: register GraphRAG search provider."""
|
"""Activate plugin: register GraphRAG search provider."""
|
||||||
|
# Base class first (event subscriptions etc.), then own registrations
|
||||||
|
# so a failure in our code cannot leave the base half-initialized (ARCH-037).
|
||||||
|
await super().on_activate(db, service_container, event_bus)
|
||||||
|
|
||||||
from app.plugins.builtins.graph_rag.provider import GraphRAGSearchProvider
|
from app.plugins.builtins.graph_rag.provider import GraphRAGSearchProvider
|
||||||
from app.plugins.builtins.unified_search.contracts import get_search_registry
|
from app.plugins.builtins.unified_search.contracts import get_search_registry
|
||||||
|
|
||||||
@@ -50,8 +54,6 @@ class GraphRAGPlugin(BasePlugin):
|
|||||||
import logging
|
import logging
|
||||||
logging.getLogger(__name__).exception("Failed to register GraphRAGSearchProvider")
|
logging.getLogger(__name__).exception("Failed to register GraphRAGSearchProvider")
|
||||||
|
|
||||||
await super().on_activate(db, service_container, event_bus)
|
|
||||||
|
|
||||||
async def on_deactivate(self, db, service_container, event_bus) -> None:
|
async def on_deactivate(self, db, service_container, event_bus) -> None:
|
||||||
"""Deactivate plugin: unregister search provider and contract."""
|
"""Deactivate plugin: unregister search provider and contract."""
|
||||||
from app.plugins.builtins.unified_search.contracts import get_search_registry
|
from app.plugins.builtins.unified_search.contracts import get_search_registry
|
||||||
|
|||||||
@@ -25,7 +25,8 @@ async def cleanup_knowledge_job(ctx: dict[str, Any]) -> None:
|
|||||||
Runs daily. Keeps approved extractions indefinitely.
|
Runs daily. Keeps approved extractions indefinitely.
|
||||||
Iterates per-tenant for RLS compliance.
|
Iterates per-tenant for RLS compliance.
|
||||||
"""
|
"""
|
||||||
from sqlalchemy import text as sa_text, delete as sa_delete
|
from sqlalchemy import delete as sa_delete
|
||||||
|
from sqlalchemy import text as sa_text
|
||||||
|
|
||||||
from app.core.db import get_worker_session_factory
|
from app.core.db import get_worker_session_factory
|
||||||
|
|
||||||
|
|||||||
@@ -1,12 +1,17 @@
|
|||||||
"""Knowledge extraction models — tracks LLM extractions and review queue."""
|
"""Knowledge extraction models — tracks LLM extractions and review queue."""
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import uuid
|
import uuid
|
||||||
from datetime import datetime
|
from datetime import datetime
|
||||||
from sqlalchemy import DateTime, Float, ForeignKey, Index, Integer, String, Text, func
|
|
||||||
from sqlalchemy.dialects.postgresql import JSONB, UUID as PGUUID
|
from sqlalchemy import DateTime, Float, ForeignKey, Index, String, Text, func
|
||||||
|
from sqlalchemy.dialects.postgresql import JSONB
|
||||||
|
from sqlalchemy.dialects.postgresql import UUID as PGUUID
|
||||||
from sqlalchemy.orm import Mapped, mapped_column
|
from sqlalchemy.orm import Mapped, mapped_column
|
||||||
|
|
||||||
from app.core.db import Base, TenantMixin
|
from app.core.db import Base, TenantMixin
|
||||||
|
|
||||||
|
|
||||||
class KnowledgeExtraction(Base, TenantMixin):
|
class KnowledgeExtraction(Base, TenantMixin):
|
||||||
"""Tracks a single knowledge extraction run from a source (wiki, dms, mail, comm)."""
|
"""Tracks a single knowledge extraction run from a source (wiki, dms, mail, comm)."""
|
||||||
__tablename__ = "knowledge_extractions"
|
__tablename__ = "knowledge_extractions"
|
||||||
|
|||||||
@@ -1,8 +1,10 @@
|
|||||||
"""Knowledge plugin — LLM-based entity/relationship extraction, ask-knowledge, review queue."""
|
"""Knowledge plugin — LLM-based entity/relationship extraction, ask-knowledge, review queue."""
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import logging
|
import logging
|
||||||
import uuid
|
import uuid
|
||||||
from typing import Any
|
from typing import Any
|
||||||
|
|
||||||
from app.plugins.base import BasePlugin
|
from app.plugins.base import BasePlugin
|
||||||
from app.plugins.manifest import PluginManifest, PluginRouteDef
|
from app.plugins.manifest import PluginManifest, PluginRouteDef
|
||||||
|
|
||||||
@@ -25,7 +27,7 @@ class KnowledgePlugin(BasePlugin):
|
|||||||
"""Register event-driven extraction hooks on activation."""
|
"""Register event-driven extraction hooks on activation."""
|
||||||
await super().on_activate(db, service_container, event_bus)
|
await super().on_activate(db, service_container, event_bus)
|
||||||
try:
|
try:
|
||||||
from app.core.hooks import register_action
|
from app.core.hooks import get_hook_registry
|
||||||
from app.plugins.builtins.knowledge.services import extract_knowledge
|
from app.plugins.builtins.knowledge.services import extract_knowledge
|
||||||
async def on_wiki_create(*args, **kwargs):
|
async def on_wiki_create(*args, **kwargs):
|
||||||
article_id = kwargs.get("article_id") or kwargs.get("entity_id")
|
article_id = kwargs.get("article_id") or kwargs.get("entity_id")
|
||||||
@@ -41,7 +43,9 @@ class KnowledgePlugin(BasePlugin):
|
|||||||
source_type="wiki_article", source_id=uuid.UUID(str(article_id)),
|
source_type="wiki_article", source_id=uuid.UUID(str(article_id)),
|
||||||
source_title=title, source_text=content,
|
source_title=title, source_text=content,
|
||||||
)
|
)
|
||||||
register_action("wiki.article.created", on_wiki_create, priority=20, owner_tag="knowledge")
|
get_hook_registry().register_action(
|
||||||
|
"wiki.article.created", on_wiki_create, priority=20, owner_tag="knowledge"
|
||||||
|
)
|
||||||
# H-DATA-LIFE: Re-extract when wiki article is updated
|
# H-DATA-LIFE: Re-extract when wiki article is updated
|
||||||
async def on_wiki_update(*args, **kwargs):
|
async def on_wiki_update(*args, **kwargs):
|
||||||
article_id = kwargs.get("article_id") or kwargs.get("entity_id")
|
article_id = kwargs.get("article_id") or kwargs.get("entity_id")
|
||||||
@@ -57,7 +61,9 @@ class KnowledgePlugin(BasePlugin):
|
|||||||
source_type="wiki_article", source_id=uuid.UUID(str(article_id)),
|
source_type="wiki_article", source_id=uuid.UUID(str(article_id)),
|
||||||
source_title=title, source_text=content,
|
source_title=title, source_text=content,
|
||||||
)
|
)
|
||||||
register_action("wiki.article.updated", on_wiki_update, priority=20, owner_tag="knowledge")
|
get_hook_registry().register_action(
|
||||||
|
"wiki.article.updated", on_wiki_update, priority=20, owner_tag="knowledge"
|
||||||
|
)
|
||||||
logger.info("Registered knowledge extraction hooks")
|
logger.info("Registered knowledge extraction hooks")
|
||||||
except Exception:
|
except Exception:
|
||||||
logger.exception("Failed to register knowledge hooks")
|
logger.exception("Failed to register knowledge hooks")
|
||||||
@@ -74,8 +80,8 @@ class KnowledgePlugin(BasePlugin):
|
|||||||
|
|
||||||
async def _ask_knowledge_handler(arguments: dict[str, Any], context: dict[str, Any]) -> dict[str, Any]:
|
async def _ask_knowledge_handler(arguments: dict[str, Any], context: dict[str, Any]) -> dict[str, Any]:
|
||||||
"""Ask a knowledge question."""
|
"""Ask a knowledge question."""
|
||||||
from app.plugins.builtins.knowledge.services import ask_knowledge
|
|
||||||
from app.core.db import get_worker_session_factory
|
from app.core.db import get_worker_session_factory
|
||||||
|
from app.plugins.builtins.knowledge.services import ask_knowledge
|
||||||
question = arguments.get("question", "")
|
question = arguments.get("question", "")
|
||||||
tenant_id = context.get("tenant_id")
|
tenant_id = context.get("tenant_id")
|
||||||
if not question or not tenant_id:
|
if not question or not tenant_id:
|
||||||
@@ -103,8 +109,8 @@ class KnowledgePlugin(BasePlugin):
|
|||||||
|
|
||||||
async def _search_knowledge_handler(arguments: dict[str, Any], context: dict[str, Any]) -> dict[str, Any]:
|
async def _search_knowledge_handler(arguments: dict[str, Any], context: dict[str, Any]) -> dict[str, Any]:
|
||||||
"""Search wiki articles via unified search."""
|
"""Search wiki articles via unified search."""
|
||||||
from app.plugins.builtins.unified_search.provider_registry import get_search_registry
|
|
||||||
from app.core.db import get_worker_session_factory
|
from app.core.db import get_worker_session_factory
|
||||||
|
from app.plugins.builtins.unified_search.provider_registry import get_search_registry
|
||||||
query = arguments.get("query", "")
|
query = arguments.get("query", "")
|
||||||
tenant_id = context.get("tenant_id")
|
tenant_id = context.get("tenant_id")
|
||||||
if not query or not tenant_id:
|
if not query or not tenant_id:
|
||||||
@@ -149,8 +155,8 @@ class KnowledgePlugin(BasePlugin):
|
|||||||
|
|
||||||
async def on_deactivate(self, db, service_container, event_bus) -> None:
|
async def on_deactivate(self, db, service_container, event_bus) -> None:
|
||||||
"""Clean up on deactivation."""
|
"""Clean up on deactivation."""
|
||||||
from app.core.hooks import unregister_actions_by_owner
|
from app.core.hooks import get_hook_registry
|
||||||
unregister_actions_by_owner("knowledge")
|
get_hook_registry().unregister_all_for_plugin("knowledge")
|
||||||
# Unregister knowledge agent tools from the core AI tool registry
|
# Unregister knowledge agent tools from the core AI tool registry
|
||||||
try:
|
try:
|
||||||
from app.ai.tool_registry import get_tool_registry
|
from app.ai.tool_registry import get_tool_registry
|
||||||
|
|||||||
@@ -1,51 +1,70 @@
|
|||||||
"""Knowledge plugin routes — extraction, ask, review queue."""
|
"""Knowledge plugin routes — extraction, ask, review queue."""
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import uuid
|
import uuid
|
||||||
|
|
||||||
from fastapi import APIRouter, Depends, HTTPException, Query
|
from fastapi import APIRouter, Depends, HTTPException, Query
|
||||||
|
from pydantic import BaseModel, Field
|
||||||
from sqlalchemy.ext.asyncio import AsyncSession
|
from sqlalchemy.ext.asyncio import AsyncSession
|
||||||
|
|
||||||
from app.core.db import get_db
|
from app.core.db import get_db
|
||||||
from app.deps import require_permission
|
from app.deps import require_permission
|
||||||
from app.plugins.builtins.knowledge.services import extract_knowledge, ask_knowledge, get_review_queue, review_extraction
|
from app.plugins.builtins.knowledge.services import (
|
||||||
|
ask_knowledge,
|
||||||
|
extract_knowledge,
|
||||||
|
get_review_queue,
|
||||||
|
review_extraction,
|
||||||
|
)
|
||||||
|
|
||||||
router = APIRouter(prefix="/api/v1/knowledge", tags=["knowledge"])
|
router = APIRouter(prefix="/api/v1/knowledge", tags=["knowledge"])
|
||||||
|
|
||||||
|
|
||||||
|
class KnowledgeExtractRequest(BaseModel):
|
||||||
|
"""Extract knowledge from a source."""
|
||||||
|
|
||||||
|
source_type: str = Field(..., min_length=1)
|
||||||
|
source_id: uuid.UUID
|
||||||
|
source_title: str | None = None
|
||||||
|
source_text: str = Field(..., min_length=1)
|
||||||
|
|
||||||
|
|
||||||
|
class KnowledgeAskRequest(BaseModel):
|
||||||
|
"""Ask a knowledge question."""
|
||||||
|
|
||||||
|
question: str = Field(..., min_length=1)
|
||||||
|
|
||||||
|
|
||||||
|
class KnowledgeReviewRequest(BaseModel):
|
||||||
|
"""Approve or reject a knowledge extraction."""
|
||||||
|
|
||||||
|
approved: bool = False
|
||||||
|
notes: str | None = None
|
||||||
|
|
||||||
|
|
||||||
@router.post("/extract")
|
@router.post("/extract")
|
||||||
async def extract(
|
async def extract(
|
||||||
body: dict,
|
body: KnowledgeExtractRequest,
|
||||||
db: AsyncSession = Depends(get_db),
|
db: AsyncSession = Depends(get_db),
|
||||||
current_user: dict = Depends(require_permission("wiki:read")),
|
current_user: dict = Depends(require_permission("wiki:read")),
|
||||||
):
|
):
|
||||||
"""Extract knowledge from a source (wiki article, dms file, mail, communication)."""
|
"""Extract knowledge from a source (wiki article, dms file, mail, communication)."""
|
||||||
tenant_id = uuid.UUID(current_user["tenant_id"])
|
tenant_id = uuid.UUID(current_user["tenant_id"])
|
||||||
source_type = body.get("source_type", "")
|
|
||||||
source_id = body.get("source_id", "")
|
|
||||||
source_title = body.get("source_title")
|
|
||||||
source_text = body.get("source_text", "")
|
|
||||||
if not source_type or not source_id or not source_text:
|
|
||||||
raise HTTPException(400, detail={"detail": "source_type, source_id, source_text required", "code": "missing_fields"})
|
|
||||||
try:
|
|
||||||
sid = uuid.UUID(source_id)
|
|
||||||
except ValueError:
|
|
||||||
raise HTTPException(400, detail={"detail": "Invalid source_id", "code": "invalid_id"}) from None
|
|
||||||
result = await extract_knowledge(
|
result = await extract_knowledge(
|
||||||
db=db, tenant_id=tenant_id, source_type=source_type, source_id=sid,
|
db=db, tenant_id=tenant_id, source_type=body.source_type, source_id=body.source_id,
|
||||||
source_title=source_title, source_text=source_text,
|
source_title=body.source_title, source_text=body.source_text,
|
||||||
user_id=uuid.UUID(current_user["user_id"]) if current_user.get("user_id") else None,
|
user_id=uuid.UUID(current_user["user_id"]) if current_user.get("user_id") else None,
|
||||||
)
|
)
|
||||||
return result
|
return result
|
||||||
|
|
||||||
@router.post("/ask")
|
@router.post("/ask")
|
||||||
async def ask(
|
async def ask(
|
||||||
body: dict,
|
body: KnowledgeAskRequest,
|
||||||
db: AsyncSession = Depends(get_db),
|
db: AsyncSession = Depends(get_db),
|
||||||
current_user: dict = Depends(require_permission("wiki:read")),
|
current_user: dict = Depends(require_permission("wiki:read")),
|
||||||
):
|
):
|
||||||
"""Ask a knowledge question — uses wiki + graph_rag as context."""
|
"""Ask a knowledge question — uses wiki + graph_rag as context."""
|
||||||
tenant_id = uuid.UUID(current_user["tenant_id"])
|
tenant_id = uuid.UUID(current_user["tenant_id"])
|
||||||
question = body.get("question") or body.get("query", "")
|
result = await ask_knowledge(db=db, tenant_id=tenant_id, question=body.question)
|
||||||
if not question:
|
|
||||||
raise HTTPException(400, detail={"detail": "question required", "code": "missing_question"})
|
|
||||||
result = await ask_knowledge(db=db, tenant_id=tenant_id, question=question)
|
|
||||||
return result
|
return result
|
||||||
|
|
||||||
@router.get("/review")
|
@router.get("/review")
|
||||||
@@ -62,22 +81,20 @@ async def review_queue(
|
|||||||
@router.post("/review/{extraction_id}")
|
@router.post("/review/{extraction_id}")
|
||||||
async def review(
|
async def review(
|
||||||
extraction_id: str,
|
extraction_id: str,
|
||||||
body: dict,
|
body: KnowledgeReviewRequest,
|
||||||
db: AsyncSession = Depends(get_db),
|
db: AsyncSession = Depends(get_db),
|
||||||
current_user: dict = Depends(require_permission("wiki:write")),
|
current_user: dict = Depends(require_permission("wiki:write")),
|
||||||
):
|
):
|
||||||
"""Approve or reject a knowledge extraction."""
|
"""Approve or reject a knowledge extraction."""
|
||||||
tenant_id = uuid.UUID(current_user["tenant_id"])
|
tenant_id = uuid.UUID(current_user["tenant_id"])
|
||||||
approved = body.get("approved", False)
|
|
||||||
notes = body.get("notes")
|
|
||||||
try:
|
try:
|
||||||
eid = uuid.UUID(extraction_id)
|
eid = uuid.UUID(extraction_id)
|
||||||
except ValueError:
|
except ValueError:
|
||||||
raise HTTPException(400, detail={"detail": "Invalid extraction_id", "code": "invalid_id"}) from None
|
raise HTTPException(400, detail={"detail": "Invalid extraction_id", "code": "invalid_id"}) from None
|
||||||
result = await review_extraction(
|
result = await review_extraction(
|
||||||
db=db, tenant_id=tenant_id, extraction_id=eid, approved=approved,
|
db=db, tenant_id=tenant_id, extraction_id=eid, approved=body.approved,
|
||||||
user_id=uuid.UUID(current_user["user_id"]) if current_user.get("user_id") else None,
|
user_id=uuid.UUID(current_user["user_id"]) if current_user.get("user_id") else None,
|
||||||
notes=notes,
|
notes=body.notes,
|
||||||
)
|
)
|
||||||
if "error" in result:
|
if "error" in result:
|
||||||
raise HTTPException(404, detail={"detail": result["error"], "code": "not_found"})
|
raise HTTPException(404, detail={"detail": result["error"], "code": "not_found"})
|
||||||
|
|||||||
@@ -1,10 +1,13 @@
|
|||||||
"""Knowledge extraction services — LLM-based entity/relationship extraction."""
|
"""Knowledge extraction services — LLM-based entity/relationship extraction."""
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import logging
|
import logging
|
||||||
import uuid
|
import uuid
|
||||||
from typing import Any
|
from typing import Any
|
||||||
from sqlalchemy import select, update
|
|
||||||
|
from sqlalchemy import select
|
||||||
from sqlalchemy.ext.asyncio import AsyncSession
|
from sqlalchemy.ext.asyncio import AsyncSession
|
||||||
|
|
||||||
from app.ai.llm_client import llm_complete
|
from app.ai.llm_client import llm_complete
|
||||||
from app.plugins.builtins.knowledge.models import KnowledgeExtraction
|
from app.plugins.builtins.knowledge.models import KnowledgeExtraction
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,7 @@
|
|||||||
|
-- Dual-path convergence (Gate B): add columns that Alembic migrations
|
||||||
|
-- 0120 (is_system) and 0140 (folder_id) add on the core path. Idempotent
|
||||||
|
-- so both install paths converge to the identical schema.
|
||||||
|
ALTER TABLE comm_conversations ADD COLUMN IF NOT EXISTS is_system BOOLEAN NOT NULL DEFAULT FALSE;
|
||||||
|
CREATE INDEX IF NOT EXISTS ix_comm_conversations_tenant_system ON comm_conversations(tenant_id, is_system);
|
||||||
|
ALTER TABLE comm_conversations ADD COLUMN IF NOT EXISTS folder_id UUID;
|
||||||
|
CREATE INDEX IF NOT EXISTS ix_comm_conversations_folder ON comm_conversations(folder_id);
|
||||||
@@ -39,7 +39,7 @@ class KommunikationPlugin(BasePlugin):
|
|||||||
"participant.left",
|
"participant.left",
|
||||||
"reaction.added",
|
"reaction.added",
|
||||||
],
|
],
|
||||||
migrations=["0001_initial.sql"],
|
migrations=["0001_initial.sql", "0002_system_channel_folders.sql"],
|
||||||
permissions=[
|
permissions=[
|
||||||
"comm:read",
|
"comm:read",
|
||||||
"comm:write",
|
"comm:write",
|
||||||
@@ -50,10 +50,10 @@ class KommunikationPlugin(BasePlugin):
|
|||||||
],
|
],
|
||||||
is_core=True,
|
is_core=True,
|
||||||
menu_items=[
|
menu_items=[
|
||||||
FrontendMenuItem(label_key='nav.communication', label='Kommunikation', path='/communication', icon='MessageSquare', order=80),
|
FrontendMenuItem(label_key='nav.communication', label='Kommunikation', path='/communication', icon='MessageSquare', order=80, permission='comm:read'),
|
||||||
],
|
],
|
||||||
page_routes=[
|
page_routes=[
|
||||||
FrontendPageRoute(path='/communication', component='@/pages/Communication', protected=True),
|
FrontendPageRoute(path='/communication', component='@/pages/Communication', protected=True, permission='comm:read'),
|
||||||
],
|
],
|
||||||
author="LeoCRM Team",
|
author="LeoCRM Team",
|
||||||
min_app_version="1.0.0",
|
min_app_version="1.0.0",
|
||||||
@@ -136,6 +136,13 @@ class KommunikationPlugin(BasePlugin):
|
|||||||
from app.plugins.builtins.contracts import get_contract_registry
|
from app.plugins.builtins.contracts import get_contract_registry
|
||||||
get_contract_registry().unregister(self.manifest.name)
|
get_contract_registry().unregister(self.manifest.name)
|
||||||
|
|
||||||
|
# Remove services registered in on_activate BEFORE super() so that
|
||||||
|
# event handlers being unsubscribed can no longer reach them (ARCH-033).
|
||||||
|
for service_name in ("comm_websocket", "comm_miniapps"):
|
||||||
|
if service_container.has(service_name):
|
||||||
|
service_container.remove(service_name)
|
||||||
|
logger.info("Removed '%s' from service container", service_name)
|
||||||
|
|
||||||
await super().on_deactivate(db, service_container, event_bus)
|
await super().on_deactivate(db, service_container, event_bus)
|
||||||
logger.info("Kommunikation plugin deactivated")
|
logger.info("Kommunikation plugin deactivated")
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,115 @@
|
|||||||
|
"""Mail Account Service for the Mail plugin.
|
||||||
|
|
||||||
|
Extracted from services.py as part of the God-object split (BUG-018 pilot).
|
||||||
|
Re-exported by ``app.plugins.builtins.mail.services``.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import logging
|
||||||
|
import uuid
|
||||||
|
|
||||||
|
from sqlalchemy.ext.asyncio import AsyncSession
|
||||||
|
|
||||||
|
from app.plugins.builtins.mail.crypto import encrypt_password, generate_salt
|
||||||
|
from app.plugins.builtins.mail.models import (
|
||||||
|
MailAccount,
|
||||||
|
MailFolder,
|
||||||
|
)
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
|
||||||
|
async def create_mail_account(
|
||||||
|
db: AsyncSession, *, tenant_id: uuid.UUID, user_id: uuid.UUID, data: dict
|
||||||
|
) -> MailAccount:
|
||||||
|
"""Create a new mail account with encrypted password."""
|
||||||
|
salt = generate_salt()
|
||||||
|
account = MailAccount(
|
||||||
|
tenant_id=tenant_id,
|
||||||
|
user_id=user_id,
|
||||||
|
owner_id=user_id,
|
||||||
|
email_address=data["email_address"],
|
||||||
|
display_name=data.get("display_name", ""),
|
||||||
|
imap_host=data["imap_host"],
|
||||||
|
imap_port=data.get("imap_port", 993),
|
||||||
|
imap_ssl=data.get("imap_ssl", True),
|
||||||
|
smtp_host=data["smtp_host"],
|
||||||
|
smtp_port=data.get("smtp_port", 587),
|
||||||
|
smtp_tls=data.get("smtp_tls", True),
|
||||||
|
username=data.get("username") or data["email_address"],
|
||||||
|
encrypted_password=encrypt_password(data["password"], salt),
|
||||||
|
password_salt=salt,
|
||||||
|
is_shared=data.get("is_shared", False),
|
||||||
|
is_active=True,
|
||||||
|
sent_folder_imap_name=data.get("sent_folder_imap_name"),
|
||||||
|
drafts_folder_imap_name=data.get("drafts_folder_imap_name"),
|
||||||
|
spam_folder_imap_name=data.get("spam_folder_imap_name"),
|
||||||
|
trash_folder_imap_name=data.get("trash_folder_imap_name"),
|
||||||
|
)
|
||||||
|
db.add(account)
|
||||||
|
await db.flush()
|
||||||
|
|
||||||
|
# Create INBOX first so subfolders can reference it as parent
|
||||||
|
inbox_folder = MailFolder(
|
||||||
|
tenant_id=tenant_id,
|
||||||
|
account_id=account.id,
|
||||||
|
name="Posteingang",
|
||||||
|
imap_name="INBOX",
|
||||||
|
is_standard=True,
|
||||||
|
)
|
||||||
|
db.add(inbox_folder)
|
||||||
|
await db.flush()
|
||||||
|
|
||||||
|
# Create standard subfolders under INBOX (IMAP server uses '.' delimiter)
|
||||||
|
for fname, imap_name in [
|
||||||
|
("Gesendet", "INBOX.Sent"),
|
||||||
|
("Entwürfe", "INBOX.Drafts"),
|
||||||
|
("Papierkorb", "INBOX.Trash"),
|
||||||
|
("Spam", "INBOX.spam"),
|
||||||
|
]:
|
||||||
|
folder = MailFolder(
|
||||||
|
tenant_id=tenant_id,
|
||||||
|
account_id=account.id,
|
||||||
|
name=fname,
|
||||||
|
imap_name=imap_name,
|
||||||
|
parent_id=inbox_folder.id,
|
||||||
|
is_standard=True,
|
||||||
|
)
|
||||||
|
db.add(folder)
|
||||||
|
await db.flush()
|
||||||
|
return account
|
||||||
|
|
||||||
|
|
||||||
|
async def update_mail_account(db: AsyncSession, account: MailAccount, data: dict) -> MailAccount:
|
||||||
|
"""Update a mail account, encrypting password if changed."""
|
||||||
|
field_map = {
|
||||||
|
"email": "email_address",
|
||||||
|
"email_address": "email_address",
|
||||||
|
"display_name": "display_name",
|
||||||
|
"imap_host": "imap_host",
|
||||||
|
"imap_port": "imap_port",
|
||||||
|
"imap_ssl": "imap_ssl",
|
||||||
|
"smtp_host": "smtp_host",
|
||||||
|
"smtp_port": "smtp_port",
|
||||||
|
"smtp_tls": "smtp_tls",
|
||||||
|
"username": "username",
|
||||||
|
"is_shared": "is_shared",
|
||||||
|
"is_active": "is_active",
|
||||||
|
"sent_folder_imap_name": "sent_folder_imap_name",
|
||||||
|
"drafts_folder_imap_name": "drafts_folder_imap_name",
|
||||||
|
"spam_folder_imap_name": "spam_folder_imap_name",
|
||||||
|
"trash_folder_imap_name": "trash_folder_imap_name",
|
||||||
|
}
|
||||||
|
for api_field, model_field in field_map.items():
|
||||||
|
if api_field in data and data[api_field] is not None:
|
||||||
|
setattr(account, model_field, data[api_field])
|
||||||
|
if "password" in data and data["password"] is not None:
|
||||||
|
new_salt = generate_salt()
|
||||||
|
account.password_salt = new_salt
|
||||||
|
account.encrypted_password = encrypt_password(data["password"], new_salt)
|
||||||
|
await db.flush()
|
||||||
|
await db.refresh(account)
|
||||||
|
return account
|
||||||
|
|
||||||
|
|
||||||
@@ -0,0 +1,117 @@
|
|||||||
|
"""Attachment storage helpers for the Mail plugin.
|
||||||
|
|
||||||
|
Extracted from services.py as part of the God-object split (BUG-018 pilot).
|
||||||
|
Re-exported by ``app.plugins.builtins.mail.services``.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import os
|
||||||
|
import re
|
||||||
|
import uuid
|
||||||
|
|
||||||
|
import aiofiles
|
||||||
|
|
||||||
|
from app.config import settings
|
||||||
|
from app.plugins.builtins.mail.models import MailAttachment
|
||||||
|
|
||||||
|
# ─── Attachment Storage Helpers ───
|
||||||
|
|
||||||
|
MAX_ATTACHMENT_SIZE = 25 * 1024 * 1024 # 25 MB
|
||||||
|
|
||||||
|
|
||||||
|
def _decode_mime_filename(filename: str) -> str:
|
||||||
|
"""Decode MIME-encoded filename, handling =?charset?Q?...?= and =?charset?B?...?= patterns."""
|
||||||
|
if not filename:
|
||||||
|
return "attachment"
|
||||||
|
# If no MIME encoding pattern, return as-is
|
||||||
|
if "=?" not in filename:
|
||||||
|
return filename
|
||||||
|
try:
|
||||||
|
from email.header import decode_header, make_header
|
||||||
|
|
||||||
|
return str(make_header(decode_header(filename)))
|
||||||
|
except Exception:
|
||||||
|
# Fallback: manually decode Q-encoding if decode_header fails
|
||||||
|
# This handles cases where the email parser partially processed the filename
|
||||||
|
try:
|
||||||
|
import base64
|
||||||
|
|
||||||
|
def decode_q(match):
|
||||||
|
charset, encoding, encoded = (
|
||||||
|
match.group(1),
|
||||||
|
match.group(2).upper(),
|
||||||
|
match.group(3),
|
||||||
|
)
|
||||||
|
if encoding == "B":
|
||||||
|
decoded = base64.b64decode(encoded).decode(
|
||||||
|
charset or "utf-8", errors="replace"
|
||||||
|
)
|
||||||
|
else: # Q encoding
|
||||||
|
decoded = encoded.replace("_", " ")
|
||||||
|
decoded = re.sub(
|
||||||
|
r"=([0-9A-Fa-f]{2})",
|
||||||
|
lambda m: chr(int(m.group(1), 16)),
|
||||||
|
decoded,
|
||||||
|
)
|
||||||
|
decoded = decoded.encode("latin-1").decode(
|
||||||
|
charset or "utf-8", errors="replace"
|
||||||
|
)
|
||||||
|
return decoded
|
||||||
|
|
||||||
|
return re.sub(r"=\?([^?]+)\?([BbQq])\?([^?]*)\?=", decode_q, filename)
|
||||||
|
except Exception:
|
||||||
|
return filename
|
||||||
|
|
||||||
|
|
||||||
|
def _sanitize_filename(filename: str) -> str:
|
||||||
|
"""Sanitize a filename to prevent path traversal attacks."""
|
||||||
|
# Remove any path components — keep only the basename
|
||||||
|
filename = os.path.basename(filename or "attachment")
|
||||||
|
# Replace potentially dangerous characters
|
||||||
|
filename = re.sub(r"[^a-zA-Z0-9._-]", "_", filename)
|
||||||
|
# Ensure non-empty
|
||||||
|
if not filename:
|
||||||
|
filename = "attachment"
|
||||||
|
# Limit length
|
||||||
|
if len(filename) > 200:
|
||||||
|
name, ext = os.path.splitext(filename)
|
||||||
|
filename = name[:200 - len(ext)] + ext
|
||||||
|
return filename
|
||||||
|
|
||||||
|
|
||||||
|
def _attachment_storage_path(mail_id: uuid.UUID, filename: str) -> str:
|
||||||
|
"""Build the on-disk storage path for a mail attachment."""
|
||||||
|
safe_name = _sanitize_filename(filename)
|
||||||
|
return os.path.join(
|
||||||
|
settings.storage_path,
|
||||||
|
"mail_attachments",
|
||||||
|
str(mail_id),
|
||||||
|
safe_name,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
async def _save_attachment_to_storage(
|
||||||
|
mail_id: uuid.UUID, filename: str, content: bytes
|
||||||
|
) -> str:
|
||||||
|
"""Save attachment content to disk and return the storage path."""
|
||||||
|
storage_path = _attachment_storage_path(mail_id, filename)
|
||||||
|
os.makedirs(os.path.dirname(storage_path), exist_ok=True)
|
||||||
|
async with aiofiles.open(storage_path, "wb") as f:
|
||||||
|
await f.write(content)
|
||||||
|
return storage_path
|
||||||
|
|
||||||
|
|
||||||
|
def attachment_to_response(att: MailAttachment) -> dict:
|
||||||
|
"""Convert a MailAttachment ORM object to a response dict."""
|
||||||
|
return {
|
||||||
|
"id": str(att.id),
|
||||||
|
"mail_id": str(att.mail_id),
|
||||||
|
"filename": att.filename,
|
||||||
|
"mime_type": att.mime_type,
|
||||||
|
"size_bytes": att.size_bytes,
|
||||||
|
"size": att.size_bytes, # alias for frontend compatibility
|
||||||
|
"content_id": att.content_id,
|
||||||
|
"is_inline": bool(att.content_id),
|
||||||
|
"dms_file_id": str(att.dms_file_id) if att.dms_file_id else None,
|
||||||
|
}
|
||||||
@@ -0,0 +1,74 @@
|
|||||||
|
"""AES-256 password encryption for Mail accounts (Fernet-based).
|
||||||
|
|
||||||
|
Extracted from services.py as part of the God-object split (BUG-018 pilot).
|
||||||
|
Re-exported by ``app.plugins.builtins.mail.services`` for backwards
|
||||||
|
compatibility.
|
||||||
|
|
||||||
|
⚠️ The legacy salt and MAIL_ENCRYPTION_KEY env guard are load-bearing:
|
||||||
|
changing them makes existing encrypted passwords unreadable.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import base64
|
||||||
|
import os
|
||||||
|
|
||||||
|
from cryptography.fernet import Fernet
|
||||||
|
from cryptography.hazmat.primitives import hashes
|
||||||
|
from cryptography.hazmat.primitives.kdf.pbkdf2 import PBKDF2HMAC
|
||||||
|
|
||||||
|
MAIL_ENCRYPTION_KEY = os.environ.get("MAIL_ENCRYPTION_KEY")
|
||||||
|
if not MAIL_ENCRYPTION_KEY:
|
||||||
|
raise RuntimeError(
|
||||||
|
"MAIL_ENCRYPTION_KEY environment variable is required. "
|
||||||
|
"Set it to a strong random value."
|
||||||
|
)
|
||||||
|
|
||||||
|
# Legacy salt for backward compatibility with existing encrypted passwords
|
||||||
|
_LEGACY_SALT = b"leocrm-mail-salt"
|
||||||
|
|
||||||
|
|
||||||
|
def _derive_key(password: str, salt: bytes) -> bytes:
|
||||||
|
"""Derive a 32-byte Fernet key from a password using PBKDF2 with the given salt."""
|
||||||
|
kdf = PBKDF2HMAC(
|
||||||
|
algorithm=hashes.SHA256(),
|
||||||
|
length=32,
|
||||||
|
salt=salt,
|
||||||
|
iterations=480000,
|
||||||
|
)
|
||||||
|
return base64.urlsafe_b64encode(kdf.derive(password.encode()))
|
||||||
|
|
||||||
|
|
||||||
|
def generate_salt() -> str:
|
||||||
|
"""Generate a random 32-byte salt and return as base64 string."""
|
||||||
|
salt = os.urandom(32)
|
||||||
|
return base64.urlsafe_b64encode(salt).decode()
|
||||||
|
|
||||||
|
|
||||||
|
def _get_fernet(salt_b64: str | None = None) -> Fernet:
|
||||||
|
"""Get a Fernet instance. If salt_b64 is provided, use it; otherwise use legacy salt."""
|
||||||
|
if salt_b64:
|
||||||
|
salt = base64.urlsafe_b64decode(salt_b64.encode())
|
||||||
|
else:
|
||||||
|
salt = _LEGACY_SALT
|
||||||
|
return Fernet(_derive_key(MAIL_ENCRYPTION_KEY, salt))
|
||||||
|
|
||||||
|
|
||||||
|
def encrypt_password(plaintext: str, salt_b64: str | None = None) -> str:
|
||||||
|
"""Encrypt a password using AES-256 (Fernet). Returns base64 ciphertext.
|
||||||
|
|
||||||
|
If salt_b64 is provided, uses that salt for key derivation.
|
||||||
|
If not, uses the legacy hardcoded salt (for backward compatibility).
|
||||||
|
"""
|
||||||
|
fernet = _get_fernet(salt_b64)
|
||||||
|
return fernet.encrypt(plaintext.encode()).decode()
|
||||||
|
|
||||||
|
|
||||||
|
def decrypt_password(ciphertext: str, salt_b64: str | None = None) -> str:
|
||||||
|
"""Decrypt a password encrypted with encrypt_password.
|
||||||
|
|
||||||
|
If salt_b64 is provided, uses that salt for key derivation.
|
||||||
|
If not, uses the legacy hardcoded salt (for backward compatibility).
|
||||||
|
"""
|
||||||
|
fernet = _get_fernet(salt_b64)
|
||||||
|
return fernet.decrypt(ciphertext.encode()).decode()
|
||||||
@@ -0,0 +1,546 @@
|
|||||||
|
"""Draft save/update and auto-sync for the Mail plugin.
|
||||||
|
|
||||||
|
Extracted from services.py as part of the God-object split (BUG-018 pilot).
|
||||||
|
Re-exported by ``app.plugins.builtins.mail.services``.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import logging
|
||||||
|
import uuid
|
||||||
|
from datetime import UTC, datetime
|
||||||
|
from email.message import EmailMessage
|
||||||
|
from email.utils import formataddr, formatdate, make_msgid
|
||||||
|
|
||||||
|
import aioimaplib
|
||||||
|
from sqlalchemy import and_, select
|
||||||
|
from sqlalchemy.ext.asyncio import AsyncSession
|
||||||
|
|
||||||
|
from app.core.notifications import create_notification
|
||||||
|
from app.plugins.builtins.mail.imap_ops import imap_delete_mail, imap_move_mail
|
||||||
|
from app.plugins.builtins.mail.imap_sync import (
|
||||||
|
get_account_password,
|
||||||
|
imap_sync_account,
|
||||||
|
)
|
||||||
|
from app.plugins.builtins.mail.models import (
|
||||||
|
Mail,
|
||||||
|
MailAccount,
|
||||||
|
MailFolder,
|
||||||
|
)
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
|
||||||
|
async def save_draft(
|
||||||
|
db: AsyncSession,
|
||||||
|
account_id: uuid.UUID,
|
||||||
|
tenant_id: uuid.UUID,
|
||||||
|
user_id: uuid.UUID,
|
||||||
|
data: dict,
|
||||||
|
) -> Mail:
|
||||||
|
"""Save a new draft mail to DB and IMAP Drafts folder."""
|
||||||
|
import logging
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
# 1. Find the Drafts folder (imap_name contains 'Drafts')
|
||||||
|
account = (
|
||||||
|
await db.execute(
|
||||||
|
select(MailAccount).where(
|
||||||
|
and_(MailAccount.id == account_id, MailAccount.tenant_id == tenant_id)
|
||||||
|
)
|
||||||
|
)
|
||||||
|
).scalar_one_or_none()
|
||||||
|
if not account:
|
||||||
|
raise ValueError("Account not found")
|
||||||
|
|
||||||
|
folders = (
|
||||||
|
await db.execute(
|
||||||
|
select(MailFolder).where(
|
||||||
|
and_(MailFolder.account_id == account_id, MailFolder.tenant_id == tenant_id)
|
||||||
|
)
|
||||||
|
)
|
||||||
|
).scalars().all()
|
||||||
|
|
||||||
|
drafts_folder = None
|
||||||
|
for f in folders:
|
||||||
|
if 'draft' in f.imap_name.lower():
|
||||||
|
drafts_folder = f
|
||||||
|
break
|
||||||
|
|
||||||
|
if not drafts_folder:
|
||||||
|
raise ValueError("No Drafts folder found for this account")
|
||||||
|
|
||||||
|
# 2. Create Mail record
|
||||||
|
to_str = ', '.join(data.get('to', []))
|
||||||
|
cc_str = ', '.join(data.get('cc', []))
|
||||||
|
bcc_str = ', '.join(data.get('bcc', []))
|
||||||
|
subject = data.get('subject', '')
|
||||||
|
body_text = data.get('body_text', '')
|
||||||
|
body_html = data.get('body_html', '')
|
||||||
|
|
||||||
|
msg_id = make_msgid()
|
||||||
|
now = datetime.now(UTC)
|
||||||
|
|
||||||
|
mail = Mail(
|
||||||
|
tenant_id=tenant_id,
|
||||||
|
account_id=account_id,
|
||||||
|
folder_id=drafts_folder.id,
|
||||||
|
message_id=msg_id,
|
||||||
|
thread_id=msg_id,
|
||||||
|
subject=subject,
|
||||||
|
from_address=account.email_address,
|
||||||
|
to_addresses=to_str,
|
||||||
|
cc_addresses=cc_str,
|
||||||
|
bcc_addresses=bcc_str,
|
||||||
|
body_text=body_text,
|
||||||
|
body_html=body_html,
|
||||||
|
body_html_sanitized=body_html,
|
||||||
|
is_seen=True,
|
||||||
|
is_flagged=False,
|
||||||
|
is_draft=True,
|
||||||
|
is_answered=False,
|
||||||
|
is_forwarded=False,
|
||||||
|
has_attachments=False,
|
||||||
|
size_bytes=len(body_text.encode('utf-8')),
|
||||||
|
received_at=now,
|
||||||
|
sent_at=None,
|
||||||
|
)
|
||||||
|
db.add(mail)
|
||||||
|
await db.flush()
|
||||||
|
|
||||||
|
# ── Notification: draft saved ──
|
||||||
|
try:
|
||||||
|
await create_notification(
|
||||||
|
db, tenant_id, user_id,
|
||||||
|
"mail_draft",
|
||||||
|
"Entwurf gespeichert",
|
||||||
|
subject or "Ohne Betreff",
|
||||||
|
)
|
||||||
|
await db.flush()
|
||||||
|
except Exception:
|
||||||
|
logger.debug("Ignored exception in mail service", exc_info=True)
|
||||||
|
|
||||||
|
# 3. Build RFC822 message and APPEND to IMAP Drafts folder
|
||||||
|
password = await get_account_password(account)
|
||||||
|
client = None
|
||||||
|
try:
|
||||||
|
client = aioimaplib.IMAP4_SSL(host=account.imap_host, port=account.imap_port)
|
||||||
|
await client.wait_hello_from_server()
|
||||||
|
await client.login(account.username, password)
|
||||||
|
|
||||||
|
# Build RFC822 message
|
||||||
|
email_msg = EmailMessage()
|
||||||
|
email_msg['From'] = formataddr((account.display_name, account.email_address))
|
||||||
|
if to_str:
|
||||||
|
email_msg['To'] = to_str
|
||||||
|
if cc_str:
|
||||||
|
email_msg['Cc'] = cc_str
|
||||||
|
email_msg['Subject'] = subject
|
||||||
|
email_msg['Date'] = formatdate(localtime=True)
|
||||||
|
email_msg['Message-ID'] = msg_id
|
||||||
|
email_msg.set_content(body_text if body_text else '')
|
||||||
|
if body_html:
|
||||||
|
email_msg.add_alternative(body_html, subtype='html')
|
||||||
|
|
||||||
|
rfc822_bytes = email_msg.as_bytes()
|
||||||
|
|
||||||
|
# APPEND to Drafts folder
|
||||||
|
append_resp = await client.append(
|
||||||
|
drafts_folder.imap_name,
|
||||||
|
r'(\\Draft)',
|
||||||
|
str(int(now.timestamp())),
|
||||||
|
rfc822_bytes,
|
||||||
|
)
|
||||||
|
if append_resp.result != 'OK':
|
||||||
|
logger.warning("save_draft: IMAP APPEND failed for drafts folder %s", drafts_folder.imap_name)
|
||||||
|
else:
|
||||||
|
logger.info("save_draft: appended draft %s to IMAP Drafts", mail.id)
|
||||||
|
|
||||||
|
except Exception as exc:
|
||||||
|
logger.warning("save_draft: IMAP append failed (non-critical): %s", exc)
|
||||||
|
finally:
|
||||||
|
if client is not None:
|
||||||
|
try:
|
||||||
|
await client.logout()
|
||||||
|
except Exception:
|
||||||
|
logger.debug("Ignored exception in mail service", exc_info=True)
|
||||||
|
|
||||||
|
return mail
|
||||||
|
|
||||||
|
|
||||||
|
async def update_draft(
|
||||||
|
db: AsyncSession,
|
||||||
|
mail_id: uuid.UUID,
|
||||||
|
tenant_id: uuid.UUID,
|
||||||
|
data: dict,
|
||||||
|
) -> Mail:
|
||||||
|
"""Update an existing draft."""
|
||||||
|
import logging
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
mail = (
|
||||||
|
await db.execute(
|
||||||
|
select(Mail).where(
|
||||||
|
and_(Mail.id == mail_id, Mail.tenant_id == tenant_id)
|
||||||
|
)
|
||||||
|
)
|
||||||
|
).scalar_one_or_none()
|
||||||
|
if not mail:
|
||||||
|
raise ValueError("Mail not found")
|
||||||
|
if not mail.is_draft:
|
||||||
|
raise ValueError("Mail is not a draft")
|
||||||
|
|
||||||
|
# 2. Update fields
|
||||||
|
to_str = ', '.join(data.get('to', []))
|
||||||
|
cc_str = ', '.join(data.get('cc', []))
|
||||||
|
bcc_str = ', '.join(data.get('bcc', []))
|
||||||
|
mail.to_addresses = to_str
|
||||||
|
mail.cc_addresses = cc_str
|
||||||
|
mail.bcc_addresses = bcc_str
|
||||||
|
mail.subject = data.get('subject', '')
|
||||||
|
mail.body_text = data.get('body_text', '')
|
||||||
|
mail.body_html = data.get('body_html', '')
|
||||||
|
mail.body_html_sanitized = data.get('body_html', '')
|
||||||
|
mail.size_bytes = len(mail.body_text.encode('utf-8'))
|
||||||
|
await db.flush()
|
||||||
|
|
||||||
|
# 3. Delete old IMAP copy and APPEND new one
|
||||||
|
folder = (
|
||||||
|
await db.execute(
|
||||||
|
select(MailFolder).where(
|
||||||
|
and_(MailFolder.id == mail.folder_id, MailFolder.tenant_id == tenant_id)
|
||||||
|
)
|
||||||
|
)
|
||||||
|
).scalar_one_or_none()
|
||||||
|
if not folder:
|
||||||
|
return mail
|
||||||
|
|
||||||
|
account = (
|
||||||
|
await db.execute(
|
||||||
|
select(MailAccount).where(
|
||||||
|
and_(MailAccount.id == folder.account_id, MailAccount.tenant_id == tenant_id)
|
||||||
|
)
|
||||||
|
)
|
||||||
|
).scalar_one_or_none()
|
||||||
|
if not account:
|
||||||
|
return mail
|
||||||
|
|
||||||
|
password = await get_account_password(account)
|
||||||
|
client = None
|
||||||
|
try:
|
||||||
|
client = aioimaplib.IMAP4_SSL(host=account.imap_host, port=account.imap_port)
|
||||||
|
await client.wait_hello_from_server()
|
||||||
|
await client.login(account.username, password)
|
||||||
|
|
||||||
|
# Delete old copy from IMAP
|
||||||
|
select_resp = await client.select(folder.imap_name)
|
||||||
|
if select_resp.result == 'OK' and mail.message_id:
|
||||||
|
search_resp = await client.uid_search(f'HEADER Message-ID "{mail.message_id}"')
|
||||||
|
uids_raw = search_resp[1][0] if search_resp[1] and search_resp[1][0] else b''
|
||||||
|
if isinstance(uids_raw, (bytes, bytearray)):
|
||||||
|
uids = uids_raw.split()
|
||||||
|
else:
|
||||||
|
uids = []
|
||||||
|
if uids:
|
||||||
|
uid_str = uids[0].decode() if isinstance(uids[0], bytes) else str(uids[0])
|
||||||
|
await client.uid('store', uid_str, r'+FLAGS (\\Deleted)')
|
||||||
|
await client.expunge()
|
||||||
|
|
||||||
|
# APPEND new copy
|
||||||
|
email_msg = EmailMessage()
|
||||||
|
email_msg['From'] = formataddr((account.display_name, account.email_address))
|
||||||
|
if to_str:
|
||||||
|
email_msg['To'] = to_str
|
||||||
|
if cc_str:
|
||||||
|
email_msg['Cc'] = cc_str
|
||||||
|
email_msg['Subject'] = mail.subject
|
||||||
|
email_msg['Date'] = formatdate(localtime=True)
|
||||||
|
email_msg['Message-ID'] = mail.message_id
|
||||||
|
email_msg.set_content(mail.body_text if mail.body_text else '')
|
||||||
|
if mail.body_html:
|
||||||
|
email_msg.add_alternative(mail.body_html, subtype='html')
|
||||||
|
|
||||||
|
rfc822_bytes = email_msg.as_bytes()
|
||||||
|
now = datetime.now(UTC)
|
||||||
|
append_resp = await client.append(
|
||||||
|
folder.imap_name,
|
||||||
|
r'(\\Draft)',
|
||||||
|
str(int(now.timestamp())),
|
||||||
|
rfc822_bytes,
|
||||||
|
)
|
||||||
|
if append_resp.result != 'OK':
|
||||||
|
logger.warning("update_draft: IMAP APPEND failed for drafts folder %s", folder.imap_name)
|
||||||
|
else:
|
||||||
|
logger.info("update_draft: appended updated draft %s to IMAP Drafts", mail.id)
|
||||||
|
|
||||||
|
except Exception as exc:
|
||||||
|
logger.warning("update_draft: IMAP sync failed (non-critical): %s", exc)
|
||||||
|
finally:
|
||||||
|
if client is not None:
|
||||||
|
try:
|
||||||
|
await client.logout()
|
||||||
|
except Exception:
|
||||||
|
logger.debug("Ignored exception in mail service", exc_info=True)
|
||||||
|
|
||||||
|
return mail
|
||||||
|
|
||||||
|
|
||||||
|
# ─── IMAP Folder Create / Delete ───
|
||||||
|
|
||||||
|
|
||||||
|
async def imap_create_folder(
|
||||||
|
db: AsyncSession, account_id: uuid.UUID, folder_name: str, tenant_id: uuid.UUID
|
||||||
|
) -> None:
|
||||||
|
"""Create folder on IMAP server.
|
||||||
|
|
||||||
|
Connects to IMAP, creates folder with CREATE command.
|
||||||
|
Non-critical: errors are logged, DB operation still succeeds.
|
||||||
|
"""
|
||||||
|
import logging
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
account = (
|
||||||
|
await db.execute(
|
||||||
|
select(MailAccount).where(
|
||||||
|
and_(MailAccount.id == account_id, MailAccount.tenant_id == tenant_id)
|
||||||
|
)
|
||||||
|
)
|
||||||
|
).scalar_one_or_none()
|
||||||
|
if not account:
|
||||||
|
logger.warning("imap_create_folder: account %s not found", account_id)
|
||||||
|
return
|
||||||
|
|
||||||
|
password = await get_account_password(account)
|
||||||
|
client = None
|
||||||
|
|
||||||
|
try:
|
||||||
|
client = aioimaplib.IMAP4_SSL(host=account.imap_host, port=account.imap_port)
|
||||||
|
await client.wait_hello_from_server()
|
||||||
|
await client.login(account.username, password)
|
||||||
|
|
||||||
|
resp = await client.create(folder_name)
|
||||||
|
if resp.result != "OK":
|
||||||
|
logger.warning(
|
||||||
|
"imap_create_folder: CREATE failed for %s: %s", folder_name, resp
|
||||||
|
)
|
||||||
|
else:
|
||||||
|
logger.info("imap_create_folder: created folder %s on IMAP", folder_name)
|
||||||
|
try:
|
||||||
|
await create_notification(
|
||||||
|
db, account.tenant_id, account.user_id,
|
||||||
|
"mail_folder",
|
||||||
|
"Ordner erstellt",
|
||||||
|
folder_name,
|
||||||
|
)
|
||||||
|
await db.flush()
|
||||||
|
except Exception:
|
||||||
|
logger.debug("Ignored exception in mail service", exc_info=True)
|
||||||
|
|
||||||
|
except Exception as exc:
|
||||||
|
logger.warning("imap_create_folder: failed (non-critical): %s", exc)
|
||||||
|
finally:
|
||||||
|
if client is not None:
|
||||||
|
try:
|
||||||
|
await client.logout()
|
||||||
|
except Exception:
|
||||||
|
logger.debug("Ignored exception in mail service", exc_info=True)
|
||||||
|
|
||||||
|
|
||||||
|
async def imap_delete_folder(
|
||||||
|
db: AsyncSession, folder_id: uuid.UUID, tenant_id: uuid.UUID
|
||||||
|
) -> None:
|
||||||
|
"""Delete folder from IMAP server.
|
||||||
|
|
||||||
|
Connects to IMAP, deletes folder with DELETE command.
|
||||||
|
Non-critical: errors are logged, DB operation still succeeds.
|
||||||
|
"""
|
||||||
|
import logging
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
folder = (
|
||||||
|
await db.execute(
|
||||||
|
select(MailFolder).where(
|
||||||
|
and_(MailFolder.id == folder_id, MailFolder.tenant_id == tenant_id)
|
||||||
|
)
|
||||||
|
)
|
||||||
|
).scalar_one_or_none()
|
||||||
|
if not folder:
|
||||||
|
logger.warning("imap_delete_folder: folder %s not found", folder_id)
|
||||||
|
return
|
||||||
|
|
||||||
|
account = (
|
||||||
|
await db.execute(
|
||||||
|
select(MailAccount).where(
|
||||||
|
and_(
|
||||||
|
MailAccount.id == folder.account_id,
|
||||||
|
MailAccount.tenant_id == tenant_id,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
)
|
||||||
|
).scalar_one_or_none()
|
||||||
|
if not account:
|
||||||
|
logger.warning("imap_delete_folder: account not found for folder %s", folder_id)
|
||||||
|
return
|
||||||
|
|
||||||
|
password = await get_account_password(account)
|
||||||
|
client = None
|
||||||
|
|
||||||
|
try:
|
||||||
|
client = aioimaplib.IMAP4_SSL(host=account.imap_host, port=account.imap_port)
|
||||||
|
await client.wait_hello_from_server()
|
||||||
|
await client.login(account.username, password)
|
||||||
|
|
||||||
|
resp = await client.delete(folder.imap_name)
|
||||||
|
if resp.result != "OK":
|
||||||
|
logger.warning(
|
||||||
|
"imap_delete_folder: DELETE failed for %s: %s", folder.imap_name, resp
|
||||||
|
)
|
||||||
|
else:
|
||||||
|
logger.info("imap_delete_folder: deleted folder %s on IMAP", folder.imap_name)
|
||||||
|
try:
|
||||||
|
await create_notification(
|
||||||
|
db, account.tenant_id, account.user_id,
|
||||||
|
"mail_folder",
|
||||||
|
"Ordner gelöscht",
|
||||||
|
folder.imap_name,
|
||||||
|
)
|
||||||
|
await db.flush()
|
||||||
|
except Exception:
|
||||||
|
logger.debug("Ignored exception in mail service", exc_info=True)
|
||||||
|
|
||||||
|
except Exception as exc:
|
||||||
|
logger.warning("imap_delete_folder: failed (non-critical): %s", exc)
|
||||||
|
finally:
|
||||||
|
if client is not None:
|
||||||
|
try:
|
||||||
|
await client.logout()
|
||||||
|
except Exception:
|
||||||
|
logger.debug("Ignored exception in mail service", exc_info=True)
|
||||||
|
|
||||||
|
|
||||||
|
# ─── Auto-Sync ───
|
||||||
|
|
||||||
|
|
||||||
|
async def auto_sync_all_accounts() -> None:
|
||||||
|
"""Auto-sync all active mail accounts.
|
||||||
|
|
||||||
|
Called periodically by the background scheduler.
|
||||||
|
Iterates all active mail accounts and syncs each one.
|
||||||
|
"""
|
||||||
|
import logging
|
||||||
|
|
||||||
|
from app.core.db import get_session_factory
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
factory = get_session_factory()
|
||||||
|
async with factory() as db:
|
||||||
|
accounts = (
|
||||||
|
await db.execute(
|
||||||
|
select(MailAccount).where(MailAccount.is_active.is_(True))
|
||||||
|
)
|
||||||
|
).scalars().all()
|
||||||
|
|
||||||
|
if not accounts:
|
||||||
|
return
|
||||||
|
|
||||||
|
logger.info("auto_sync_all_accounts: syncing %d active account(s)", len(accounts))
|
||||||
|
|
||||||
|
for account in accounts:
|
||||||
|
try:
|
||||||
|
result = await imap_sync_account(db, account.id, account.tenant_id)
|
||||||
|
logger.info(
|
||||||
|
"auto_sync_all_accounts: synced account %s (%s): %s",
|
||||||
|
account.id,
|
||||||
|
account.username,
|
||||||
|
result,
|
||||||
|
)
|
||||||
|
except Exception as exc:
|
||||||
|
logger.warning(
|
||||||
|
"auto_sync_all_accounts: failed for account %s: %s",
|
||||||
|
account.id,
|
||||||
|
exc,
|
||||||
|
)
|
||||||
|
try:
|
||||||
|
await create_notification(
|
||||||
|
db, account.tenant_id, account.user_id,
|
||||||
|
"mail_sync_error",
|
||||||
|
"Synchronisierung fehlgeschlagen",
|
||||||
|
f"Account {account.email_address}: {exc}",
|
||||||
|
)
|
||||||
|
except Exception:
|
||||||
|
logger.debug("Ignored exception in mail service", exc_info=True)
|
||||||
|
# commit per-account so partial progress is saved
|
||||||
|
try:
|
||||||
|
await db.commit()
|
||||||
|
except Exception:
|
||||||
|
await db.rollback()
|
||||||
|
|
||||||
|
|
||||||
|
async def process_sync_queue(db: AsyncSession) -> None:
|
||||||
|
"""Process pending IMAP operations from the sync queue.
|
||||||
|
|
||||||
|
Called at the start of each auto-sync loop iteration.
|
||||||
|
Retries failed delete/move operations.
|
||||||
|
"""
|
||||||
|
import logging
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
from app.plugins.builtins.mail.models import MailSyncQueue
|
||||||
|
|
||||||
|
pending = (
|
||||||
|
await db.execute(
|
||||||
|
select(MailSyncQueue).where(
|
||||||
|
and_(
|
||||||
|
MailSyncQueue.status == "pending",
|
||||||
|
MailSyncQueue.attempts < MailSyncQueue.max_attempts,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
)
|
||||||
|
).scalars().all()
|
||||||
|
|
||||||
|
if not pending:
|
||||||
|
return
|
||||||
|
|
||||||
|
logger.info("process_sync_queue: processing %d pending operation(s)", len(pending))
|
||||||
|
|
||||||
|
for entry in pending:
|
||||||
|
try:
|
||||||
|
if entry.operation == "delete":
|
||||||
|
is_permanent = entry.payload.get("permanent", False) if entry.payload else False
|
||||||
|
await imap_delete_mail(db, entry.mail_id, entry.tenant_id, permanent=is_permanent)
|
||||||
|
elif entry.operation == "move":
|
||||||
|
target_folder_id = uuid.UUID(entry.payload.get("target_folder_id", ""))
|
||||||
|
await imap_move_mail(db, entry.mail_id, target_folder_id, entry.tenant_id)
|
||||||
|
else:
|
||||||
|
logger.warning("process_sync_queue: unknown operation %s", entry.operation)
|
||||||
|
entry.status = "failed"
|
||||||
|
entry.last_error = f"Unknown operation: {entry.operation}"
|
||||||
|
continue
|
||||||
|
|
||||||
|
entry.status = "completed"
|
||||||
|
entry.updated_at = datetime.now(UTC)
|
||||||
|
logger.info("process_sync_queue: completed %s for mail %s", entry.operation, entry.mail_id)
|
||||||
|
|
||||||
|
except Exception as exc:
|
||||||
|
entry.attempts += 1
|
||||||
|
entry.last_error = str(exc)
|
||||||
|
entry.updated_at = datetime.now(UTC)
|
||||||
|
if entry.attempts >= entry.max_attempts:
|
||||||
|
entry.status = "failed"
|
||||||
|
logger.warning(
|
||||||
|
"process_sync_queue: giving up on %s for mail %s after %d attempts: %s",
|
||||||
|
entry.operation, entry.mail_id, entry.attempts, exc,
|
||||||
|
)
|
||||||
|
else:
|
||||||
|
logger.info(
|
||||||
|
"process_sync_queue: retry %d/%d for %s mail %s: %s",
|
||||||
|
entry.attempts, entry.max_attempts, entry.operation, entry.mail_id, exc,
|
||||||
|
)
|
||||||
|
|
||||||
|
await db.flush()
|
||||||
@@ -0,0 +1,452 @@
|
|||||||
|
"""IMAP operations (flags, delete, move, folders) for the Mail plugin.
|
||||||
|
|
||||||
|
Extracted from services.py as part of the God-object split (BUG-018 pilot).
|
||||||
|
Re-exported by ``app.plugins.builtins.mail.services``.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import logging
|
||||||
|
import uuid
|
||||||
|
|
||||||
|
import aioimaplib
|
||||||
|
from sqlalchemy import and_, select
|
||||||
|
from sqlalchemy.ext.asyncio import AsyncSession
|
||||||
|
|
||||||
|
from app.plugins.builtins.mail.imap_sync import (
|
||||||
|
_parse_imap_list_response,
|
||||||
|
get_account_password,
|
||||||
|
)
|
||||||
|
from app.plugins.builtins.mail.models import (
|
||||||
|
Mail,
|
||||||
|
MailAccount,
|
||||||
|
MailFolder,
|
||||||
|
)
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
|
||||||
|
async def imap_sync_mail_flags(
|
||||||
|
db: AsyncSession,
|
||||||
|
mail_id: uuid.UUID,
|
||||||
|
tenant_id: uuid.UUID,
|
||||||
|
) -> None:
|
||||||
|
"""Sync is_seen/is_flagged flags from DB to IMAP server.
|
||||||
|
|
||||||
|
Connects to the IMAP server, selects the mail's folder,
|
||||||
|
and uses UID STORE to set/remove \\Seen and \\Flagged flags.
|
||||||
|
Non-critical: logs warnings on failure but does not raise.
|
||||||
|
"""
|
||||||
|
import logging
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
# Load the mail with its folder and account
|
||||||
|
mail = (
|
||||||
|
await db.execute(
|
||||||
|
select(Mail).where(
|
||||||
|
and_(Mail.id == mail_id, Mail.tenant_id == tenant_id)
|
||||||
|
)
|
||||||
|
)
|
||||||
|
).scalar_one_or_none()
|
||||||
|
if not mail:
|
||||||
|
logger.warning("imap_sync_mail_flags: mail %s not found", mail_id)
|
||||||
|
return
|
||||||
|
|
||||||
|
folder = (
|
||||||
|
await db.execute(
|
||||||
|
select(MailFolder).where(
|
||||||
|
and_(MailFolder.id == mail.folder_id, MailFolder.tenant_id == tenant_id)
|
||||||
|
)
|
||||||
|
)
|
||||||
|
).scalar_one_or_none()
|
||||||
|
if not folder:
|
||||||
|
logger.warning("imap_sync_mail_flags: folder %s not found for mail %s", mail.folder_id, mail_id)
|
||||||
|
return
|
||||||
|
|
||||||
|
account = (
|
||||||
|
await db.execute(
|
||||||
|
select(MailAccount).where(
|
||||||
|
and_(MailAccount.id == folder.account_id, MailAccount.tenant_id == tenant_id)
|
||||||
|
)
|
||||||
|
)
|
||||||
|
).scalar_one_or_none()
|
||||||
|
if not account:
|
||||||
|
logger.warning("imap_sync_mail_flags: account not found for mail %s", mail_id)
|
||||||
|
return
|
||||||
|
|
||||||
|
if not mail.message_id:
|
||||||
|
logger.warning("imap_sync_mail_flags: mail %s has no message_id, cannot sync", mail_id)
|
||||||
|
return
|
||||||
|
|
||||||
|
password = await get_account_password(account)
|
||||||
|
client = None
|
||||||
|
|
||||||
|
try:
|
||||||
|
client = aioimaplib.IMAP4_SSL(host=account.imap_host, port=account.imap_port)
|
||||||
|
await client.wait_hello_from_server()
|
||||||
|
await client.login(account.username, password)
|
||||||
|
|
||||||
|
# Select the folder
|
||||||
|
select_resp = await client.select(folder.imap_name)
|
||||||
|
if select_resp.result != 'OK':
|
||||||
|
logger.warning("imap_sync_mail_flags: cannot select folder %s", folder.imap_name)
|
||||||
|
return
|
||||||
|
|
||||||
|
# Find the UID by searching for the Message-ID header
|
||||||
|
search_resp = await client.uid_search(f'HEADER Message-ID "{mail.message_id}"')
|
||||||
|
uids_raw = search_resp[1][0] if search_resp[1] and search_resp[1][0] else b''
|
||||||
|
if isinstance(uids_raw, (bytes, bytearray)):
|
||||||
|
uids = uids_raw.split()
|
||||||
|
else:
|
||||||
|
uids = []
|
||||||
|
|
||||||
|
if not uids:
|
||||||
|
logger.warning("imap_sync_mail_flags: no UID found for Message-ID %s", mail.message_id)
|
||||||
|
return
|
||||||
|
|
||||||
|
uid_str = uids[0].decode() if isinstance(uids[0], bytes) else str(uids[0])
|
||||||
|
|
||||||
|
# Sync \Seen flag
|
||||||
|
if mail.is_seen:
|
||||||
|
await client.uid('store', uid_str, '+FLAGS (\\Seen)')
|
||||||
|
else:
|
||||||
|
await client.uid('store', uid_str, '-FLAGS (\\Seen)')
|
||||||
|
|
||||||
|
# Sync \Flagged flag
|
||||||
|
if mail.is_flagged:
|
||||||
|
await client.uid('store', uid_str, '+FLAGS (\\Flagged)')
|
||||||
|
else:
|
||||||
|
await client.uid('store', uid_str, '-FLAGS (\\Flagged)')
|
||||||
|
|
||||||
|
logger.info("imap_sync_mail_flags: synced flags for mail %s (UID %s)", mail_id, uid_str)
|
||||||
|
|
||||||
|
except Exception as exc:
|
||||||
|
logger.warning("imap_sync_mail_flags: failed for mail %s: %s", mail_id, exc)
|
||||||
|
finally:
|
||||||
|
if client is not None:
|
||||||
|
try:
|
||||||
|
await client.logout()
|
||||||
|
except Exception:
|
||||||
|
logger.debug("Ignored exception in mail service", exc_info=True)
|
||||||
|
|
||||||
|
|
||||||
|
# ─── IMAP Delete ───
|
||||||
|
|
||||||
|
|
||||||
|
async def _find_trash_folder_name(
|
||||||
|
db: AsyncSession, account: MailAccount, tenant_id: uuid.UUID
|
||||||
|
) -> str | None:
|
||||||
|
"""Find the IMAP Trash folder name for an account.
|
||||||
|
|
||||||
|
Checks account.trash_folder_imap_name, then DB folders with 'trash' in imap_name,
|
||||||
|
then queries the IMAP server LIST for common Trash folder names.
|
||||||
|
"""
|
||||||
|
# 1) Explicit mapping on account
|
||||||
|
if account.trash_folder_imap_name:
|
||||||
|
return account.trash_folder_imap_name
|
||||||
|
|
||||||
|
# 2) DB folder with 'trash' in imap_name
|
||||||
|
db_folders = (
|
||||||
|
await db.execute(
|
||||||
|
select(MailFolder).where(
|
||||||
|
and_(MailFolder.account_id == account.id, MailFolder.tenant_id == tenant_id)
|
||||||
|
)
|
||||||
|
)
|
||||||
|
).scalars().all()
|
||||||
|
for f in db_folders:
|
||||||
|
if 'trash' in f.imap_name.lower():
|
||||||
|
return f.imap_name
|
||||||
|
|
||||||
|
# 3) Query IMAP server for common Trash folder names
|
||||||
|
password = await get_account_password(account)
|
||||||
|
client = None
|
||||||
|
try:
|
||||||
|
client = aioimaplib.IMAP4_SSL(host=account.imap_host, port=account.imap_port)
|
||||||
|
await client.wait_hello_from_server()
|
||||||
|
await client.login(account.username, password)
|
||||||
|
list_resp = await client.list('""', '"*"')
|
||||||
|
imap_folders, _ = _parse_imap_list_response(list_resp)
|
||||||
|
trash_candidates = ['Trash', 'INBOX.Trash', 'INBOX.Trash', 'Deleted', 'Deleted Items', 'Papierkorb']
|
||||||
|
for _, name in imap_folders:
|
||||||
|
if name in trash_candidates or 'trash' in name.lower():
|
||||||
|
return name
|
||||||
|
except Exception:
|
||||||
|
logger.debug("Ignored exception in mail service", exc_info=True)
|
||||||
|
finally:
|
||||||
|
if client:
|
||||||
|
try:
|
||||||
|
await client.logout()
|
||||||
|
except Exception:
|
||||||
|
logger.debug("Ignored exception in mail service", exc_info=True)
|
||||||
|
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
async def imap_delete_mail(
|
||||||
|
db: AsyncSession, mail_id: uuid.UUID, tenant_id: uuid.UUID, *, permanent: bool = False
|
||||||
|
) -> None:
|
||||||
|
"""Delete mail from IMAP server.
|
||||||
|
|
||||||
|
If permanent=False (default): MOVE mail to IMAP Trash folder (like real mail clients).
|
||||||
|
If permanent=True: STORE \\Deleted + EXPUNGE (permanent delete from server).
|
||||||
|
Uses UID MOVE if supported, otherwise COPY + STORE \\Deleted + EXPUNGE.
|
||||||
|
Raises exceptions on failure so caller can queue for retry.
|
||||||
|
"""
|
||||||
|
import logging
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
mail = (
|
||||||
|
await db.execute(
|
||||||
|
select(Mail).where(
|
||||||
|
and_(Mail.id == mail_id, Mail.tenant_id == tenant_id)
|
||||||
|
)
|
||||||
|
)
|
||||||
|
).scalar_one_or_none()
|
||||||
|
if not mail:
|
||||||
|
logger.warning("imap_delete_mail: mail %s not found", mail_id)
|
||||||
|
return
|
||||||
|
|
||||||
|
folder = (
|
||||||
|
await db.execute(
|
||||||
|
select(MailFolder).where(
|
||||||
|
and_(MailFolder.id == mail.folder_id, MailFolder.tenant_id == tenant_id)
|
||||||
|
)
|
||||||
|
)
|
||||||
|
).scalar_one_or_none()
|
||||||
|
if not folder:
|
||||||
|
logger.warning("imap_delete_mail: folder %s not found for mail %s", mail.folder_id, mail_id)
|
||||||
|
return
|
||||||
|
|
||||||
|
account = (
|
||||||
|
await db.execute(
|
||||||
|
select(MailAccount).where(
|
||||||
|
and_(MailAccount.id == folder.account_id, MailAccount.tenant_id == tenant_id)
|
||||||
|
)
|
||||||
|
)
|
||||||
|
).scalar_one_or_none()
|
||||||
|
if not account:
|
||||||
|
logger.warning("imap_delete_mail: account not found for mail %s", mail_id)
|
||||||
|
return
|
||||||
|
|
||||||
|
if not mail.imap_uid and not mail.message_id:
|
||||||
|
logger.warning("imap_delete_mail: mail %s has no imap_uid or message_id, cannot sync", mail_id)
|
||||||
|
return
|
||||||
|
|
||||||
|
# Lifecycle hook: mail.before_delete
|
||||||
|
from app.core.hooks import do_action
|
||||||
|
await do_action("mail.before_delete", mail_id=str(mail.id), tenant_id=str(tenant_id), permanent=permanent, db=db)
|
||||||
|
|
||||||
|
# Find Trash folder on IMAP server (only needed for non-permanent delete)
|
||||||
|
trash_folder_name = None
|
||||||
|
if not permanent:
|
||||||
|
trash_folder_name = await _find_trash_folder_name(db, account, tenant_id)
|
||||||
|
if not trash_folder_name:
|
||||||
|
logger.warning("imap_delete_mail: no Trash folder found for account %s, will permanent delete", account.id)
|
||||||
|
|
||||||
|
password = await get_account_password(account)
|
||||||
|
client = None
|
||||||
|
|
||||||
|
try:
|
||||||
|
client = aioimaplib.IMAP4_SSL(host=account.imap_host, port=account.imap_port)
|
||||||
|
await client.wait_hello_from_server()
|
||||||
|
await client.login(account.username, password)
|
||||||
|
|
||||||
|
select_resp = await client.select(folder.imap_name)
|
||||||
|
if select_resp.result != 'OK':
|
||||||
|
raise RuntimeError(f"cannot select folder {folder.imap_name}")
|
||||||
|
|
||||||
|
# Use stored imap_uid directly; fall back to Message-ID search
|
||||||
|
if mail.imap_uid:
|
||||||
|
uid_str = mail.imap_uid
|
||||||
|
else:
|
||||||
|
search_resp = await client.uid_search(f'HEADER Message-ID "{mail.message_id}"')
|
||||||
|
uids_raw = search_resp[1][0] if search_resp[1] and search_resp[1][0] else b''
|
||||||
|
if isinstance(uids_raw, (bytes, bytearray)):
|
||||||
|
uids = uids_raw.split()
|
||||||
|
else:
|
||||||
|
uids = []
|
||||||
|
if not uids:
|
||||||
|
raise RuntimeError(f"no UID found for Message-ID {mail.message_id}")
|
||||||
|
uid_str = uids[0].decode() if isinstance(uids[0], bytes) else str(uids[0])
|
||||||
|
|
||||||
|
if permanent or not trash_folder_name:
|
||||||
|
# Permanent delete: STORE \Deleted + EXPUNGE
|
||||||
|
await client.uid('store', uid_str, r'+FLAGS (\\Deleted)')
|
||||||
|
await client.expunge()
|
||||||
|
logger.info("imap_delete_mail: permanent deleted mail %s (UID %s)", mail_id, uid_str)
|
||||||
|
return
|
||||||
|
|
||||||
|
# Non-permanent: MOVE to Trash folder (like Thunderbird, Outlook, etc.)
|
||||||
|
# Don't move if already in Trash
|
||||||
|
if folder.imap_name.lower() == trash_folder_name.lower():
|
||||||
|
# Already in Trash — permanent delete
|
||||||
|
await client.uid('store', uid_str, r'+FLAGS (\\Deleted)')
|
||||||
|
await client.expunge()
|
||||||
|
logger.info("imap_delete_mail: permanent deleted mail %s (already in Trash, UID %s)", mail_id, uid_str)
|
||||||
|
return
|
||||||
|
|
||||||
|
# Try UID MOVE first
|
||||||
|
try:
|
||||||
|
move_resp = await client.uid('move', uid_str, trash_folder_name)
|
||||||
|
if move_resp.result == 'OK':
|
||||||
|
logger.info("imap_delete_mail: moved mail %s (UID %s) to Trash %s", mail_id, uid_str, trash_folder_name)
|
||||||
|
return
|
||||||
|
except Exception as move_exc:
|
||||||
|
logger.info("imap_delete_mail: UID MOVE not supported, falling back: %s", move_exc)
|
||||||
|
|
||||||
|
# Fallback: COPY to Trash + STORE \Deleted + EXPUNGE
|
||||||
|
copy_resp = await client.uid('copy', uid_str, trash_folder_name)
|
||||||
|
if copy_resp.result != 'OK':
|
||||||
|
raise RuntimeError(f"COPY to Trash failed for mail {mail_id}")
|
||||||
|
await client.uid('store', uid_str, r'+FLAGS (\\Deleted)')
|
||||||
|
await client.expunge()
|
||||||
|
logger.info("imap_delete_mail: moved mail %s (UID %s) to Trash via COPY+DELETE", mail_id, uid_str)
|
||||||
|
|
||||||
|
# Lifecycle hook: mail.after_delete
|
||||||
|
from app.core.hooks import do_action
|
||||||
|
await do_action("mail.after_delete", mail_id=str(mail.id), tenant_id=str(tenant_id), permanent=permanent, db=db)
|
||||||
|
|
||||||
|
except Exception as exc:
|
||||||
|
logger.warning("imap_delete_mail: failed for mail %s: %s", mail_id, exc)
|
||||||
|
raise
|
||||||
|
finally:
|
||||||
|
if client is not None:
|
||||||
|
try:
|
||||||
|
await client.logout()
|
||||||
|
except Exception:
|
||||||
|
logger.debug("Ignored exception in mail service", exc_info=True)
|
||||||
|
|
||||||
|
|
||||||
|
# ─── IMAP Move ───
|
||||||
|
|
||||||
|
|
||||||
|
async def imap_move_mail(
|
||||||
|
db: AsyncSession,
|
||||||
|
mail_id: uuid.UUID,
|
||||||
|
target_folder_id: uuid.UUID,
|
||||||
|
tenant_id: uuid.UUID,
|
||||||
|
) -> None:
|
||||||
|
"""Move mail to another folder on IMAP server.
|
||||||
|
|
||||||
|
Uses stored imap_uid directly; falls back to Message-ID search if missing.
|
||||||
|
Uses UID MOVE if supported, otherwise COPY + STORE \\Deleted + EXPUNGE.
|
||||||
|
Raises exceptions on failure so caller can queue for retry.
|
||||||
|
"""
|
||||||
|
import logging
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
mail = (
|
||||||
|
await db.execute(
|
||||||
|
select(Mail).where(
|
||||||
|
and_(Mail.id == mail_id, Mail.tenant_id == tenant_id)
|
||||||
|
)
|
||||||
|
)
|
||||||
|
).scalar_one_or_none()
|
||||||
|
if not mail:
|
||||||
|
logger.warning("imap_move_mail: mail %s not found", mail_id)
|
||||||
|
return
|
||||||
|
|
||||||
|
source_folder = (
|
||||||
|
await db.execute(
|
||||||
|
select(MailFolder).where(
|
||||||
|
and_(MailFolder.id == mail.folder_id, MailFolder.tenant_id == tenant_id)
|
||||||
|
)
|
||||||
|
)
|
||||||
|
).scalar_one_or_none()
|
||||||
|
if not source_folder:
|
||||||
|
logger.warning("imap_move_mail: source folder not found for mail %s", mail_id)
|
||||||
|
return
|
||||||
|
|
||||||
|
target_folder = (
|
||||||
|
await db.execute(
|
||||||
|
select(MailFolder).where(
|
||||||
|
and_(MailFolder.id == target_folder_id, MailFolder.tenant_id == tenant_id)
|
||||||
|
)
|
||||||
|
)
|
||||||
|
).scalar_one_or_none()
|
||||||
|
if not target_folder:
|
||||||
|
logger.warning("imap_move_mail: target folder %s not found", target_folder_id)
|
||||||
|
return
|
||||||
|
|
||||||
|
account = (
|
||||||
|
await db.execute(
|
||||||
|
select(MailAccount).where(
|
||||||
|
and_(MailAccount.id == source_folder.account_id, MailAccount.tenant_id == tenant_id)
|
||||||
|
)
|
||||||
|
)
|
||||||
|
).scalar_one_or_none()
|
||||||
|
if not account:
|
||||||
|
logger.warning("imap_move_mail: account not found for mail %s", mail_id)
|
||||||
|
return
|
||||||
|
|
||||||
|
if not mail.imap_uid and not mail.message_id:
|
||||||
|
logger.warning("imap_move_mail: mail %s has no imap_uid or message_id, cannot sync", mail_id)
|
||||||
|
return
|
||||||
|
|
||||||
|
# Lifecycle hook: mail.before_move
|
||||||
|
from app.core.hooks import do_action
|
||||||
|
await do_action("mail.before_move", mail_id=str(mail.id), tenant_id=str(tenant_id), source_folder_id=str(source_folder.id), target_folder_id=str(target_folder.id), db=db)
|
||||||
|
|
||||||
|
password = await get_account_password(account)
|
||||||
|
client = None
|
||||||
|
|
||||||
|
try:
|
||||||
|
client = aioimaplib.IMAP4_SSL(host=account.imap_host, port=account.imap_port)
|
||||||
|
await client.wait_hello_from_server()
|
||||||
|
await client.login(account.username, password)
|
||||||
|
|
||||||
|
select_resp = await client.select(source_folder.imap_name)
|
||||||
|
if select_resp.result != 'OK':
|
||||||
|
raise RuntimeError(f"cannot select folder {source_folder.imap_name}")
|
||||||
|
|
||||||
|
# Use stored imap_uid directly; fall back to Message-ID search
|
||||||
|
if mail.imap_uid:
|
||||||
|
uid_str = mail.imap_uid
|
||||||
|
else:
|
||||||
|
search_resp = await client.uid_search(f'HEADER Message-ID "{mail.message_id}"')
|
||||||
|
uids_raw = search_resp[1][0] if search_resp[1] and search_resp[1][0] else b''
|
||||||
|
if isinstance(uids_raw, (bytes, bytearray)):
|
||||||
|
uids = uids_raw.split()
|
||||||
|
else:
|
||||||
|
uids = []
|
||||||
|
if not uids:
|
||||||
|
raise RuntimeError(f"no UID found for Message-ID {mail.message_id}")
|
||||||
|
uid_str = uids[0].decode() if isinstance(uids[0], bytes) else str(uids[0])
|
||||||
|
|
||||||
|
# Try UID MOVE first; fall back to COPY + STORE \\Deleted + EXPUNGE
|
||||||
|
try:
|
||||||
|
move_resp = await client.uid('move', uid_str, target_folder.imap_name)
|
||||||
|
if move_resp.result == 'OK':
|
||||||
|
logger.info("imap_move_mail: moved mail %s (UID %s) via UID MOVE", mail_id, uid_str)
|
||||||
|
return
|
||||||
|
except Exception as move_exc:
|
||||||
|
logger.info("imap_move_mail: UID MOVE not supported, falling back: %s", move_exc)
|
||||||
|
|
||||||
|
# Fallback: COPY + STORE \\Deleted + EXPUNGE
|
||||||
|
copy_resp = await client.uid('copy', uid_str, target_folder.imap_name)
|
||||||
|
if copy_resp.result != 'OK':
|
||||||
|
raise RuntimeError(f"COPY failed for mail {mail_id}")
|
||||||
|
|
||||||
|
await client.uid('store', uid_str, r'+FLAGS (\\Deleted)')
|
||||||
|
await client.expunge()
|
||||||
|
|
||||||
|
logger.info("imap_move_mail: moved mail %s (UID %s) via COPY+DELETE", mail_id, uid_str)
|
||||||
|
|
||||||
|
# Lifecycle hook: mail.after_move
|
||||||
|
from app.core.hooks import do_action
|
||||||
|
await do_action("mail.after_move", mail_id=str(mail.id), tenant_id=str(tenant_id), source_folder_id=str(source_folder.id), target_folder_id=str(target_folder.id), db=db)
|
||||||
|
|
||||||
|
except Exception as exc:
|
||||||
|
logger.warning("imap_move_mail: failed for mail %s: %s", mail_id, exc)
|
||||||
|
raise
|
||||||
|
finally:
|
||||||
|
if client is not None:
|
||||||
|
try:
|
||||||
|
await client.logout()
|
||||||
|
except Exception:
|
||||||
|
logger.debug("Ignored exception in mail service", exc_info=True)
|
||||||
|
|
||||||
|
|
||||||
File diff suppressed because it is too large
Load Diff
@@ -4,7 +4,9 @@ from __future__ import annotations
|
|||||||
|
|
||||||
import uuid
|
import uuid
|
||||||
from datetime import UTC, datetime
|
from datetime import UTC, datetime
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
from pgvector.sqlalchemy import Vector
|
||||||
from sqlalchemy import (
|
from sqlalchemy import (
|
||||||
JSON,
|
JSON,
|
||||||
Boolean,
|
Boolean,
|
||||||
@@ -16,14 +18,12 @@ from sqlalchemy import (
|
|||||||
Text,
|
Text,
|
||||||
UniqueConstraint,
|
UniqueConstraint,
|
||||||
)
|
)
|
||||||
|
from sqlalchemy.dialects.postgresql import TSVECTOR
|
||||||
from sqlalchemy.dialects.postgresql import UUID as PGUUID
|
from sqlalchemy.dialects.postgresql import UUID as PGUUID
|
||||||
from sqlalchemy.orm import Mapped, mapped_column
|
from sqlalchemy.orm import Mapped, mapped_column
|
||||||
|
|
||||||
from app.core.db import Base, TenantMixin
|
from app.core.db import Base, TenantMixin
|
||||||
from app.models.owned_mixin import OwnedMixin
|
from app.models.owned_mixin import OwnedMixin
|
||||||
from sqlalchemy.dialects.postgresql import TSVECTOR
|
|
||||||
from pgvector.sqlalchemy import Vector
|
|
||||||
from typing import Any
|
|
||||||
|
|
||||||
# --- Mail Accounts (F-MAIL-14, F-MAIL-18) ---
|
# --- Mail Accounts (F-MAIL-14, F-MAIL-18) ---
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,48 @@
|
|||||||
|
"""PGP encryption/decryption for the Mail plugin using pgpy.
|
||||||
|
|
||||||
|
Extracted from services.py as part of the God-object split (BUG-018 pilot).
|
||||||
|
Re-exported by ``app.plugins.builtins.mail.services``.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import pgpy
|
||||||
|
|
||||||
|
|
||||||
|
def import_pgp_private_key(private_key_armored: str, passphrase: str = "") -> tuple[str, str]:
|
||||||
|
"""Import a PGP private key. Returns (key_id, public_key_armored)."""
|
||||||
|
key, _ = pgpy.PGPKey.from_blob(private_key_armored)
|
||||||
|
if key.is_protected:
|
||||||
|
with key.unlock(passphrase):
|
||||||
|
pub_key = key.pubkey
|
||||||
|
key_id = str(key.fingerprint).upper()[-16:]
|
||||||
|
return key_id, str(pub_key)
|
||||||
|
pub_key = key.pubkey
|
||||||
|
key_id = str(key.fingerprint).upper()[-16:]
|
||||||
|
return key_id, str(pub_key)
|
||||||
|
|
||||||
|
|
||||||
|
def import_pgp_public_key(public_key_armored: str) -> str:
|
||||||
|
"""Import a PGP public key. Returns key_id."""
|
||||||
|
key, _ = pgpy.PGPKey.from_blob(public_key_armored)
|
||||||
|
return str(key.fingerprint).upper()[-16:]
|
||||||
|
|
||||||
|
|
||||||
|
def pgp_encrypt_message(plaintext: str, recipient_public_key_armored: str) -> str:
|
||||||
|
"""Encrypt a message with recipient's public PGP key."""
|
||||||
|
pub_key, _ = pgpy.PGPKey.from_blob(recipient_public_key_armored)
|
||||||
|
msg = pgpy.PGPMessage.new(plaintext)
|
||||||
|
encrypted = pub_key.encrypt(msg)
|
||||||
|
return str(encrypted)
|
||||||
|
|
||||||
|
|
||||||
|
def pgp_decrypt_message(ciphertext: str, private_key_armored: str, passphrase: str = "") -> str:
|
||||||
|
"""Decrypt a PGP-encrypted message."""
|
||||||
|
key, _ = pgpy.PGPKey.from_blob(private_key_armored)
|
||||||
|
enc_msg = pgpy.PGPMessage.from_blob(ciphertext)
|
||||||
|
if key.is_protected:
|
||||||
|
with key.unlock(passphrase):
|
||||||
|
decrypted = key.decrypt(enc_msg)
|
||||||
|
return decrypted.message.decode("utf-8")
|
||||||
|
decrypted = key.decrypt(enc_msg)
|
||||||
|
return decrypted.message.decode("utf-8")
|
||||||
@@ -9,7 +9,6 @@ from typing import Any
|
|||||||
|
|
||||||
from app.plugins.base import BasePlugin
|
from app.plugins.base import BasePlugin
|
||||||
from app.plugins.manifest import (
|
from app.plugins.manifest import (
|
||||||
FrontendDetailTab,
|
|
||||||
FrontendMenuItem,
|
FrontendMenuItem,
|
||||||
FrontendPageRoute,
|
FrontendPageRoute,
|
||||||
FrontendSettingsPage,
|
FrontendSettingsPage,
|
||||||
@@ -115,7 +114,11 @@ async def _auto_sync_loop() -> None:
|
|||||||
class MailPlugin(BasePlugin):
|
class MailPlugin(BasePlugin):
|
||||||
"""Mail plugin for email management: IMAP sync, SMTP send, threading, rules, PGP."""
|
"""Mail plugin for email management: IMAP sync, SMTP send, threading, rules, PGP."""
|
||||||
|
|
||||||
_auto_sync_task: asyncio.Task | None = None
|
def __init__(self) -> None:
|
||||||
|
super().__init__()
|
||||||
|
# Instance attribute: multiple plugin instances must not share the
|
||||||
|
# background task state (ARCH-036).
|
||||||
|
self._auto_sync_task: asyncio.Task | None = None
|
||||||
|
|
||||||
manifest = PluginManifest(
|
manifest = PluginManifest(
|
||||||
name="mail",
|
name="mail",
|
||||||
@@ -137,24 +140,30 @@ class MailPlugin(BasePlugin):
|
|||||||
migrations=["0001_initial.sql", "0006_flag_type.sql", "0007_sync_queue.sql", "0008_sync_queue_deleted_at.sql", "0009_remove_mail_soft_delete.sql", "0010_add_deleted_at.sql"],
|
migrations=["0001_initial.sql", "0006_flag_type.sql", "0007_sync_queue.sql", "0008_sync_queue_deleted_at.sql", "0009_remove_mail_soft_delete.sql", "0010_add_deleted_at.sql"],
|
||||||
permissions=["mail:read", "mail:send", "mail:config", "mail:share", "mail:write", "mail:delete"],
|
permissions=["mail:read", "mail:send", "mail:config", "mail:share", "mail:write", "mail:delete"],
|
||||||
menu_items=[
|
menu_items=[
|
||||||
FrontendMenuItem(label_key='nav.mail', label='E-Mail', path='/mail', icon='Mail', order=30),
|
FrontendMenuItem(label_key='nav.mail', label='E-Mail', path='/mail', icon='Mail', order=30, permission='mail:read'),
|
||||||
],
|
],
|
||||||
page_routes=[
|
page_routes=[
|
||||||
FrontendPageRoute(path='/mail', component='@/pages/Mail', protected=True),
|
FrontendPageRoute(path='/mail', component='@/pages/Mail', protected=True, permission='mail:read'),
|
||||||
FrontendPageRoute(path='/mail/settings', component='@/pages/MailSettings', protected=True),
|
FrontendPageRoute(path='/mail/settings', component='@/pages/MailSettings', protected=True, permission='mail:config'),
|
||||||
],
|
],
|
||||||
settings_pages=[
|
settings_pages=[
|
||||||
FrontendSettingsPage(path='mail', label_key='settings.mail', label='Mail', component='@/pages/MailSettings', icon='Mail', order=50),
|
FrontendSettingsPage(path='mail', label_key='settings.mail', label='Mail', component='@/pages/MailSettings', icon='Mail', order=50),
|
||||||
],
|
],
|
||||||
detail_tabs=[
|
# BUG (ghost component): ContactMailTab does not exist in the
|
||||||
FrontendDetailTab(entity_type='contact', label_key='tabs.email', label='E-Mails', component='@/components/contact/ContactMailTab', icon='Mail', order=20, permission='mail:read'),
|
# frontend — tab removed until implemented (Block I-D).
|
||||||
],
|
detail_tabs=[],
|
||||||
author="LeoCRM Team",
|
author="LeoCRM Team",
|
||||||
min_app_version="1.0.0",
|
min_app_version="1.0.0",
|
||||||
hooks=["mail.before_send", "mail.after_send"],
|
hooks=["mail.before_send", "mail.after_send"],
|
||||||
contract_version="1.0.0",
|
contract_version="1.0.0",
|
||||||
)
|
)
|
||||||
|
|
||||||
|
def get_entity_models(self) -> dict[str, type]:
|
||||||
|
"""Entity types owned by Mail for the permission resolver."""
|
||||||
|
from app.plugins.builtins.mail.models import MailAccount
|
||||||
|
|
||||||
|
return {"mail_account": MailAccount}
|
||||||
|
|
||||||
async def on_activate(
|
async def on_activate(
|
||||||
self, db, service_container, event_bus
|
self, db, service_container, event_bus
|
||||||
) -> None:
|
) -> None:
|
||||||
|
|||||||
@@ -56,6 +56,7 @@ from app.plugins.builtins.mail.schemas import (
|
|||||||
MailRuleCreate,
|
MailRuleCreate,
|
||||||
MailSendRequest,
|
MailSendRequest,
|
||||||
MailSignatureCreate,
|
MailSignatureCreate,
|
||||||
|
MailSignatureUpdate,
|
||||||
MailTemplateCreate,
|
MailTemplateCreate,
|
||||||
PgpKeyImport,
|
PgpKeyImport,
|
||||||
SendPermissionCreate,
|
SendPermissionCreate,
|
||||||
@@ -892,7 +893,10 @@ async def list_threads(
|
|||||||
threads[tid]["mail_count"] += 1
|
threads[tid]["mail_count"] += 1
|
||||||
threads[tid]["mails"].append(mail_to_response(mail))
|
threads[tid]["mails"].append(mail_to_response(mail))
|
||||||
items = list(threads.values())
|
items = list(threads.values())
|
||||||
return {"items": items, "total": len(items)}
|
# Plain array — consistent with sibling mail list routes (accounts,
|
||||||
|
# folders, templates) and with the frontend client type
|
||||||
|
# fetchThreads(): Promise<ThreadResult[]>.
|
||||||
|
return items
|
||||||
|
|
||||||
|
|
||||||
# ─── Templates (F-MAIL-06) ───
|
# ─── Templates (F-MAIL-06) ───
|
||||||
@@ -992,6 +996,66 @@ async def list_signatures(
|
|||||||
return [signature_to_response(s) for s in sigs]
|
return [signature_to_response(s) for s in sigs]
|
||||||
|
|
||||||
|
|
||||||
|
@router.patch("/signatures/{signature_id}")
|
||||||
|
async def update_signature(
|
||||||
|
signature_id: str,
|
||||||
|
data: MailSignatureUpdate,
|
||||||
|
db: AsyncSession = Depends(get_db),
|
||||||
|
current_user: dict = Depends(require_permission("mail:config")),
|
||||||
|
):
|
||||||
|
tenant_id = uuid.UUID(current_user["tenant_id"])
|
||||||
|
user_id = uuid.UUID(current_user["user_id"])
|
||||||
|
s_id = _parse_uuid(signature_id, "signature_id")
|
||||||
|
sig = (
|
||||||
|
await db.execute(
|
||||||
|
select(MailSignature).where(
|
||||||
|
and_(MailSignature.id == s_id, MailSignature.tenant_id == tenant_id)
|
||||||
|
)
|
||||||
|
)
|
||||||
|
).scalar_one_or_none()
|
||||||
|
if not sig:
|
||||||
|
raise HTTPException(404, detail={"detail": "Signature not found", "code": "not_found"})
|
||||||
|
if sig.user_id != user_id:
|
||||||
|
raise HTTPException(403, detail={"detail": "Not your signature", "code": "forbidden"})
|
||||||
|
sig.name = data.name
|
||||||
|
sig.body_html = data.body_html
|
||||||
|
if data.is_default:
|
||||||
|
others = (
|
||||||
|
await db.execute(
|
||||||
|
select(MailSignature).where(MailSignature.user_id == user_id)
|
||||||
|
)
|
||||||
|
).scalars().all()
|
||||||
|
for o in others:
|
||||||
|
o.is_default = False
|
||||||
|
sig.is_default = data.is_default
|
||||||
|
await db.flush()
|
||||||
|
return signature_to_response(sig)
|
||||||
|
|
||||||
|
|
||||||
|
@router.delete("/signatures/{signature_id}", status_code=204)
|
||||||
|
async def delete_signature(
|
||||||
|
signature_id: str,
|
||||||
|
db: AsyncSession = Depends(get_db),
|
||||||
|
current_user: dict = Depends(require_permission("mail:config")),
|
||||||
|
):
|
||||||
|
tenant_id = uuid.UUID(current_user["tenant_id"])
|
||||||
|
user_id = uuid.UUID(current_user["user_id"])
|
||||||
|
s_id = _parse_uuid(signature_id, "signature_id")
|
||||||
|
sig = (
|
||||||
|
await db.execute(
|
||||||
|
select(MailSignature).where(
|
||||||
|
and_(MailSignature.id == s_id, MailSignature.tenant_id == tenant_id)
|
||||||
|
)
|
||||||
|
)
|
||||||
|
).scalar_one_or_none()
|
||||||
|
if not sig:
|
||||||
|
raise HTTPException(404, detail={"detail": "Signature not found", "code": "not_found"})
|
||||||
|
if sig.user_id != user_id:
|
||||||
|
raise HTTPException(403, detail={"detail": "Not your signature", "code": "forbidden"})
|
||||||
|
await db.delete(sig)
|
||||||
|
await db.flush()
|
||||||
|
|
||||||
|
|
||||||
# ─── Rules (F-MAIL-07) ───
|
# ─── Rules (F-MAIL-07) ───
|
||||||
|
|
||||||
|
|
||||||
@@ -1206,6 +1270,28 @@ async def list_labels(
|
|||||||
return [label_to_response(lbl) for lbl in labels]
|
return [label_to_response(lbl) for lbl in labels]
|
||||||
|
|
||||||
|
|
||||||
|
@router.delete("/labels/{label_id}", status_code=204)
|
||||||
|
async def delete_label(
|
||||||
|
label_id: str,
|
||||||
|
db: AsyncSession = Depends(get_db),
|
||||||
|
current_user: dict = Depends(require_permission("mail:config")),
|
||||||
|
):
|
||||||
|
tenant_id = uuid.UUID(current_user["tenant_id"])
|
||||||
|
user_id = uuid.UUID(current_user["user_id"])
|
||||||
|
l_id = _parse_uuid(label_id, "label_id")
|
||||||
|
label = (
|
||||||
|
await db.execute(
|
||||||
|
select(MailLabel).where(and_(MailLabel.id == l_id, MailLabel.tenant_id == tenant_id))
|
||||||
|
)
|
||||||
|
).scalar_one_or_none()
|
||||||
|
if not label:
|
||||||
|
raise HTTPException(404, detail={"detail": "Label not found", "code": "not_found"})
|
||||||
|
if label.user_id != user_id:
|
||||||
|
raise HTTPException(403, detail={"detail": "Not your label", "code": "forbidden"})
|
||||||
|
await db.delete(label)
|
||||||
|
await db.flush()
|
||||||
|
|
||||||
|
|
||||||
# ─── Contact PGP Keys (F-MAIL-12) ───
|
# ─── Contact PGP Keys (F-MAIL-12) ───
|
||||||
|
|
||||||
|
|
||||||
@@ -1494,8 +1580,8 @@ async def create_event_from_mail(
|
|||||||
try:
|
try:
|
||||||
from app.plugins.builtins.calendar.contracts import get_contract as get_calendar_contract
|
from app.plugins.builtins.calendar.contracts import get_contract as get_calendar_contract
|
||||||
_cal = get_calendar_contract()
|
_cal = get_calendar_contract()
|
||||||
calendar = _cal.calendar
|
calendar = _cal.Calendar
|
||||||
calendar_entry = _cal.calendar_entry
|
calendar_entry = _cal.CalendarEntry
|
||||||
except ImportError:
|
except ImportError:
|
||||||
return {"created": False, "error": "calendar plugin not available"}
|
return {"created": False, "error": "calendar plugin not available"}
|
||||||
cal_id = _parse_uuid(data.calendar_id, "calendar_id")
|
cal_id = _parse_uuid(data.calendar_id, "calendar_id")
|
||||||
@@ -1862,4 +1948,3 @@ async def get_mail(
|
|||||||
)
|
)
|
||||||
return mail_to_response(mail, attachments=list(attachments), labels=list(label_assignments))
|
return mail_to_response(mail, attachments=list(attachments), labels=list(label_assignments))
|
||||||
|
|
||||||
from app.core.audit import log_audit
|
|
||||||
@@ -0,0 +1,194 @@
|
|||||||
|
"""Rules, Templates & Vacation for the Mail plugin.
|
||||||
|
|
||||||
|
Extracted from services.py as part of the God-object split (BUG-018 pilot).
|
||||||
|
Re-exported by ``app.plugins.builtins.mail.services``.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import logging
|
||||||
|
import uuid
|
||||||
|
from datetime import UTC, datetime, timedelta
|
||||||
|
|
||||||
|
from sqlalchemy import and_, or_, select
|
||||||
|
from sqlalchemy.ext.asyncio import AsyncSession
|
||||||
|
|
||||||
|
from app.plugins.builtins.mail.models import (
|
||||||
|
Mail,
|
||||||
|
MailFolder,
|
||||||
|
MailLabelAssignment,
|
||||||
|
MailRule,
|
||||||
|
VacationSentLog,
|
||||||
|
)
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
|
||||||
|
def substitute_template_vars(template_body: str, variables: dict[str, str]) -> str:
|
||||||
|
"""Replace {{placeholder}} variables in template body."""
|
||||||
|
result = template_body
|
||||||
|
for key, value in variables.items():
|
||||||
|
result = result.replace(f"{{{{{key}}}}}", value)
|
||||||
|
result = result.replace(f"{{{{{key.lower()}}}}}", value)
|
||||||
|
result = result.replace(f"{{{{{key.upper()}}}}}", value)
|
||||||
|
return result
|
||||||
|
|
||||||
|
|
||||||
|
# ─── Mail Rule Engine (F-MAIL-07) ───
|
||||||
|
|
||||||
|
|
||||||
|
def matches_condition(mail: Mail, conditions: dict) -> bool:
|
||||||
|
"""Check if a mail matches all rule conditions."""
|
||||||
|
for field, expected in conditions.items():
|
||||||
|
if field == "from_contains":
|
||||||
|
if expected.lower() not in mail.from_address.lower():
|
||||||
|
return False
|
||||||
|
elif field == "subject_contains":
|
||||||
|
if expected.lower() not in mail.subject.lower():
|
||||||
|
return False
|
||||||
|
elif field == "to_contains":
|
||||||
|
if expected.lower() not in mail.to_addresses.lower():
|
||||||
|
return False
|
||||||
|
elif field == "body_contains":
|
||||||
|
body = (mail.body_text + mail.body_html).lower()
|
||||||
|
if expected.lower() not in body:
|
||||||
|
return False
|
||||||
|
elif field == "has_attachments":
|
||||||
|
if mail.has_attachments != bool(expected):
|
||||||
|
return False
|
||||||
|
elif field == "is_flagged":
|
||||||
|
if mail.is_flagged != bool(expected):
|
||||||
|
return False
|
||||||
|
return True
|
||||||
|
|
||||||
|
|
||||||
|
async def execute_rule_actions(
|
||||||
|
db: AsyncSession, mail: Mail, actions: dict, tenant_id: uuid.UUID
|
||||||
|
) -> dict:
|
||||||
|
"""Execute rule actions on a matching mail."""
|
||||||
|
results = {}
|
||||||
|
for action, value in actions.items():
|
||||||
|
if action == "move_to_folder":
|
||||||
|
folder_id = uuid.UUID(value) if isinstance(value, str) else value
|
||||||
|
folder = (
|
||||||
|
await db.execute(
|
||||||
|
select(MailFolder).where(
|
||||||
|
and_(MailFolder.id == folder_id, MailFolder.tenant_id == tenant_id)
|
||||||
|
)
|
||||||
|
)
|
||||||
|
).scalar_one_or_none()
|
||||||
|
if folder:
|
||||||
|
mail.folder_id = folder.id
|
||||||
|
results["moved"] = str(folder.id)
|
||||||
|
elif action == "label":
|
||||||
|
label_id = uuid.UUID(value) if isinstance(value, str) else value
|
||||||
|
existing = (
|
||||||
|
await db.execute(
|
||||||
|
select(MailLabelAssignment).where(
|
||||||
|
and_(
|
||||||
|
MailLabelAssignment.mail_id == mail.id,
|
||||||
|
MailLabelAssignment.label_id == label_id,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
)
|
||||||
|
).scalar_one_or_none()
|
||||||
|
if not existing:
|
||||||
|
assignment = MailLabelAssignment(
|
||||||
|
tenant_id=tenant_id,
|
||||||
|
mail_id=mail.id,
|
||||||
|
label_id=label_id,
|
||||||
|
)
|
||||||
|
db.add(assignment)
|
||||||
|
results["labeled"] = str(label_id)
|
||||||
|
elif action == "mark_seen":
|
||||||
|
mail.is_seen = bool(value)
|
||||||
|
results["seen"] = bool(value)
|
||||||
|
elif action == "mark_flagged":
|
||||||
|
mail.is_flagged = bool(value)
|
||||||
|
results["flagged"] = bool(value)
|
||||||
|
elif action == "forward_to":
|
||||||
|
results["forward_to"] = value
|
||||||
|
await db.flush()
|
||||||
|
return results
|
||||||
|
|
||||||
|
|
||||||
|
async def apply_rules_to_mail(db: AsyncSession, mail: Mail, tenant_id: uuid.UUID) -> list[dict]:
|
||||||
|
"""Find and apply all matching rules to a mail, sorted by priority."""
|
||||||
|
rules = (
|
||||||
|
(
|
||||||
|
await db.execute(
|
||||||
|
select(MailRule)
|
||||||
|
.where(
|
||||||
|
and_(
|
||||||
|
MailRule.tenant_id == tenant_id,
|
||||||
|
MailRule.is_active,
|
||||||
|
or_(
|
||||||
|
MailRule.account_id == mail.account_id,
|
||||||
|
MailRule.account_id.is_(None),
|
||||||
|
),
|
||||||
|
)
|
||||||
|
)
|
||||||
|
.order_by(MailRule.priority)
|
||||||
|
)
|
||||||
|
)
|
||||||
|
.scalars()
|
||||||
|
.all()
|
||||||
|
)
|
||||||
|
|
||||||
|
applied = []
|
||||||
|
for rule in rules:
|
||||||
|
conditions = json.loads(rule.conditions) if rule.conditions else {}
|
||||||
|
actions = json.loads(rule.actions) if rule.actions else {}
|
||||||
|
if matches_condition(mail, conditions):
|
||||||
|
result = await execute_rule_actions(db, mail, actions, tenant_id)
|
||||||
|
applied.append({"rule_id": str(rule.id), "rule_name": rule.name, "actions": result})
|
||||||
|
return applied
|
||||||
|
|
||||||
|
|
||||||
|
# ─── Vacation Auto-Reply (F-MAIL-08) ───
|
||||||
|
|
||||||
|
|
||||||
|
VACATION_DEDUP_HOURS = 24
|
||||||
|
|
||||||
|
|
||||||
|
async def should_send_vacation_reply(
|
||||||
|
db: AsyncSession,
|
||||||
|
account_id: uuid.UUID,
|
||||||
|
sender_address: str,
|
||||||
|
tenant_id: uuid.UUID,
|
||||||
|
) -> bool:
|
||||||
|
"""Check if vacation auto-reply should be sent (dedup within 24h)."""
|
||||||
|
cutoff = datetime.now(UTC) - timedelta(hours=VACATION_DEDUP_HOURS)
|
||||||
|
existing = (
|
||||||
|
await db.execute(
|
||||||
|
select(VacationSentLog).where(
|
||||||
|
and_(
|
||||||
|
VacationSentLog.account_id == account_id,
|
||||||
|
VacationSentLog.sender_address == sender_address,
|
||||||
|
VacationSentLog.sent_at >= cutoff,
|
||||||
|
VacationSentLog.tenant_id == tenant_id,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
)
|
||||||
|
).scalar_one_or_none()
|
||||||
|
return existing is None
|
||||||
|
|
||||||
|
|
||||||
|
async def log_vacation_sent(
|
||||||
|
db: AsyncSession,
|
||||||
|
account_id: uuid.UUID,
|
||||||
|
sender_address: str,
|
||||||
|
tenant_id: uuid.UUID,
|
||||||
|
) -> None:
|
||||||
|
"""Log that a vacation auto-reply was sent to a sender."""
|
||||||
|
log = VacationSentLog(
|
||||||
|
tenant_id=tenant_id,
|
||||||
|
account_id=account_id,
|
||||||
|
sender_address=sender_address,
|
||||||
|
sent_at=datetime.now(UTC),
|
||||||
|
)
|
||||||
|
db.add(log)
|
||||||
|
await db.flush()
|
||||||
|
|
||||||
|
|
||||||
@@ -0,0 +1,62 @@
|
|||||||
|
"""HTML sanitization for the Mail plugin using nh3.
|
||||||
|
|
||||||
|
Extracted from services.py as part of the God-object split (BUG-018 pilot).
|
||||||
|
Re-exported by ``app.plugins.builtins.mail.services``.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import nh3
|
||||||
|
|
||||||
|
|
||||||
|
def sanitize_html(raw_html: str) -> str:
|
||||||
|
"""Sanitize HTML using nh3 — removes script tags and dangerous attributes."""
|
||||||
|
if not raw_html:
|
||||||
|
return ""
|
||||||
|
return nh3.clean(
|
||||||
|
raw_html,
|
||||||
|
tags={
|
||||||
|
"a",
|
||||||
|
"b",
|
||||||
|
"br",
|
||||||
|
"div",
|
||||||
|
"em",
|
||||||
|
"h1",
|
||||||
|
"h2",
|
||||||
|
"h3",
|
||||||
|
"h4",
|
||||||
|
"h5",
|
||||||
|
"h6",
|
||||||
|
"hr",
|
||||||
|
"i",
|
||||||
|
"img",
|
||||||
|
"li",
|
||||||
|
"ol",
|
||||||
|
"p",
|
||||||
|
"span",
|
||||||
|
"strong",
|
||||||
|
"table",
|
||||||
|
"tbody",
|
||||||
|
"td",
|
||||||
|
"th",
|
||||||
|
"thead",
|
||||||
|
"tr",
|
||||||
|
"u",
|
||||||
|
"ul",
|
||||||
|
"blockquote",
|
||||||
|
"code",
|
||||||
|
"pre",
|
||||||
|
"font",
|
||||||
|
"center",
|
||||||
|
},
|
||||||
|
attributes={
|
||||||
|
"a": {"href", "title", "target"},
|
||||||
|
"img": {"src", "alt", "width", "height"},
|
||||||
|
"span": {"style"},
|
||||||
|
"div": {"style"},
|
||||||
|
"font": {"color", "size", "face"},
|
||||||
|
"p": {"style"},
|
||||||
|
"td": {"style"},
|
||||||
|
"th": {"style"},
|
||||||
|
},
|
||||||
|
)
|
||||||
@@ -259,6 +259,12 @@ class MailSignatureResponse(BaseModel):
|
|||||||
is_default: bool
|
is_default: bool
|
||||||
|
|
||||||
|
|
||||||
|
class MailSignatureUpdate(BaseModel):
|
||||||
|
name: str = Field(..., min_length=1, max_length=255)
|
||||||
|
body_html: str = ""
|
||||||
|
is_default: bool = False
|
||||||
|
|
||||||
|
|
||||||
# ─── Rules ───
|
# ─── Rules ───
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,169 @@
|
|||||||
|
"""Response serializers for the Mail plugin ORM models.
|
||||||
|
|
||||||
|
Extracted from services.py as part of the God-object split (BUG-018 pilot).
|
||||||
|
Re-exported by ``app.plugins.builtins.mail.services``.
|
||||||
|
|
||||||
|
account_to_response NEVER includes passwords — security-critical contract.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
|
||||||
|
from app.plugins.builtins.mail.models import (
|
||||||
|
Mail,
|
||||||
|
MailAccount,
|
||||||
|
MailAttachment,
|
||||||
|
MailFolder,
|
||||||
|
MailLabel,
|
||||||
|
MailRule,
|
||||||
|
MailSignature,
|
||||||
|
MailTemplate,
|
||||||
|
)
|
||||||
|
from app.plugins.builtins.mail.text_utils import (
|
||||||
|
extract_email_addresses, # noqa: F401 — re-exported for backwards compat
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def attachment_to_response(att: MailAttachment) -> dict:
|
||||||
|
"""Convert a MailAttachment ORM object to a response dict."""
|
||||||
|
return {
|
||||||
|
"id": str(att.id),
|
||||||
|
"mail_id": str(att.mail_id),
|
||||||
|
"filename": att.filename,
|
||||||
|
"mime_type": att.mime_type,
|
||||||
|
"size_bytes": att.size_bytes,
|
||||||
|
"size": att.size_bytes, # alias for frontend compatibility
|
||||||
|
"content_id": att.content_id,
|
||||||
|
"is_inline": bool(att.content_id),
|
||||||
|
"dms_file_id": str(att.dms_file_id) if att.dms_file_id else None,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def account_to_response(account: MailAccount) -> dict:
|
||||||
|
"""Convert MailAccount to response dict, NEVER including password."""
|
||||||
|
return {
|
||||||
|
"id": str(account.id),
|
||||||
|
"email": account.email_address,
|
||||||
|
"email_address": account.email_address,
|
||||||
|
"display_name": account.display_name,
|
||||||
|
"imap_host": account.imap_host,
|
||||||
|
"imap_port": account.imap_port,
|
||||||
|
"imap_ssl": account.imap_ssl,
|
||||||
|
"smtp_host": account.smtp_host,
|
||||||
|
"smtp_port": account.smtp_port,
|
||||||
|
"smtp_tls": account.smtp_tls,
|
||||||
|
"username": account.username,
|
||||||
|
"is_shared": account.is_shared,
|
||||||
|
"is_active": account.is_active,
|
||||||
|
"sent_folder_imap_name": account.sent_folder_imap_name,
|
||||||
|
"drafts_folder_imap_name": account.drafts_folder_imap_name,
|
||||||
|
"spam_folder_imap_name": account.spam_folder_imap_name,
|
||||||
|
"trash_folder_imap_name": account.trash_folder_imap_name,
|
||||||
|
"created_at": account.created_at,
|
||||||
|
"updated_at": account.updated_at,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def mail_to_response(
|
||||||
|
mail: Mail,
|
||||||
|
attachments: list[MailAttachment] | None = None,
|
||||||
|
labels: list[MailLabel] | None = None,
|
||||||
|
) -> dict:
|
||||||
|
"""Convert a Mail ORM object to a response dict."""
|
||||||
|
resp = {
|
||||||
|
"id": str(mail.id),
|
||||||
|
"account_id": str(mail.account_id),
|
||||||
|
"folder_id": str(mail.folder_id),
|
||||||
|
"message_id": mail.message_id,
|
||||||
|
"thread_id": mail.thread_id,
|
||||||
|
"in_reply_to": mail.in_reply_to,
|
||||||
|
"subject": mail.subject,
|
||||||
|
"from_address": mail.from_address,
|
||||||
|
"from_name": mail.from_address.split("<")[0].strip().strip('"') if "<" in mail.from_address else mail.from_address,
|
||||||
|
"to_addresses": extract_email_addresses(mail.to_addresses) if mail.to_addresses else [],
|
||||||
|
"cc_addresses": extract_email_addresses(mail.cc_addresses) if mail.cc_addresses else [],
|
||||||
|
"bcc_addresses": extract_email_addresses(mail.bcc_addresses) if mail.bcc_addresses else [],
|
||||||
|
"body_text": mail.body_text,
|
||||||
|
"body_html": mail.body_html if mail.body_html else None,
|
||||||
|
"body_html_sanitized": mail.body_html_sanitized,
|
||||||
|
"sanitized_html": mail.body_html_sanitized,
|
||||||
|
"date": mail.received_at.isoformat() if mail.received_at else (mail.sent_at.isoformat() if mail.sent_at else None),
|
||||||
|
"is_seen": mail.is_seen,
|
||||||
|
"is_flagged": mail.is_flagged,
|
||||||
|
"flag_type": getattr(mail, 'flag_type', None),
|
||||||
|
"is_draft": mail.is_draft,
|
||||||
|
"is_answered": mail.is_answered,
|
||||||
|
"is_forwarded": mail.is_forwarded,
|
||||||
|
"has_attachments": mail.has_attachments,
|
||||||
|
"size_bytes": mail.size_bytes,
|
||||||
|
"received_at": mail.received_at,
|
||||||
|
"sent_at": mail.sent_at,
|
||||||
|
"contact_id": str(mail.contact_id) if mail.contact_id else None,
|
||||||
|
"attachments": [],
|
||||||
|
"labels": [],
|
||||||
|
}
|
||||||
|
if attachments:
|
||||||
|
resp["attachments"] = [attachment_to_response(a) for a in attachments]
|
||||||
|
if labels:
|
||||||
|
resp["labels"] = [
|
||||||
|
{"id": str(lbl.id), "name": lbl.name, "color": lbl.color} for lbl in labels
|
||||||
|
]
|
||||||
|
return resp
|
||||||
|
|
||||||
|
|
||||||
|
def folder_to_response(folder: MailFolder) -> dict:
|
||||||
|
"""Convert MailFolder to response dict."""
|
||||||
|
return {
|
||||||
|
"id": str(folder.id),
|
||||||
|
"account_id": str(folder.account_id),
|
||||||
|
"name": folder.name,
|
||||||
|
"imap_name": folder.imap_name,
|
||||||
|
"parent_id": str(folder.parent_id) if folder.parent_id else None,
|
||||||
|
"is_standard": folder.is_standard,
|
||||||
|
"unread_count": folder.unread_count,
|
||||||
|
"total_count": folder.total_count,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def rule_to_response(rule: MailRule) -> dict:
|
||||||
|
"""Convert MailRule to response dict."""
|
||||||
|
return {
|
||||||
|
"id": str(rule.id),
|
||||||
|
"name": rule.name,
|
||||||
|
"account_id": str(rule.account_id) if rule.account_id else None,
|
||||||
|
"priority": rule.priority,
|
||||||
|
"is_active": rule.is_active,
|
||||||
|
"conditions": json.loads(rule.conditions) if rule.conditions else {},
|
||||||
|
"actions": json.loads(rule.actions) if rule.actions else {},
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def template_to_response(template: MailTemplate) -> dict:
|
||||||
|
"""Convert MailTemplate to response dict."""
|
||||||
|
return {
|
||||||
|
"id": str(template.id),
|
||||||
|
"name": template.name,
|
||||||
|
"subject": template.subject,
|
||||||
|
"body_html": template.body_html,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def signature_to_response(sig: MailSignature) -> dict:
|
||||||
|
"""Convert MailSignature to response dict."""
|
||||||
|
return {
|
||||||
|
"id": str(sig.id),
|
||||||
|
"name": sig.name,
|
||||||
|
"body_html": sig.body_html,
|
||||||
|
"account_id": str(sig.account_id) if sig.account_id else None,
|
||||||
|
"is_default": sig.is_default,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def label_to_response(label: MailLabel) -> dict:
|
||||||
|
"""Convert MailLabel to response dict."""
|
||||||
|
return {
|
||||||
|
"id": str(label.id),
|
||||||
|
"name": label.name,
|
||||||
|
"color": label.color,
|
||||||
|
}
|
||||||
+151
-3067
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,358 @@
|
|||||||
|
"""SMTP send service for the Mail plugin (F-MAIL-02).
|
||||||
|
|
||||||
|
Extracted from services.py as part of the God-object split (BUG-018 pilot).
|
||||||
|
Re-exported by ``app.plugins.builtins.mail.services``.
|
||||||
|
|
||||||
|
Contains send_mail_via_smtp, reply_to_mail and forward_mail.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import logging
|
||||||
|
import os
|
||||||
|
import uuid
|
||||||
|
from datetime import UTC, datetime
|
||||||
|
from email.message import EmailMessage
|
||||||
|
from email.utils import formataddr, formatdate, make_msgid
|
||||||
|
|
||||||
|
import aiofiles
|
||||||
|
import aioimaplib
|
||||||
|
import aiosmtplib
|
||||||
|
from sqlalchemy import and_, select
|
||||||
|
from sqlalchemy.ext.asyncio import AsyncSession
|
||||||
|
|
||||||
|
from app.core.notifications import create_notification
|
||||||
|
from app.plugins.builtins.mail.imap_sync import (
|
||||||
|
_compute_thread_id,
|
||||||
|
get_account_password,
|
||||||
|
)
|
||||||
|
from app.plugins.builtins.mail.models import (
|
||||||
|
Mail,
|
||||||
|
MailAccount,
|
||||||
|
MailFolder,
|
||||||
|
MailSignature,
|
||||||
|
)
|
||||||
|
from app.plugins.builtins.mail.sanitize import sanitize_html
|
||||||
|
from app.plugins.builtins.mail.text_utils import _strip_html
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
|
||||||
|
async def send_mail_via_smtp(
|
||||||
|
db: AsyncSession,
|
||||||
|
*,
|
||||||
|
tenant_id: uuid.UUID,
|
||||||
|
user_id: uuid.UUID,
|
||||||
|
account: MailAccount,
|
||||||
|
to_addrs: list[str],
|
||||||
|
cc_addrs: list[str] = None,
|
||||||
|
bcc_addrs: list[str] = None,
|
||||||
|
subject: str = "",
|
||||||
|
body_html: str = "",
|
||||||
|
body_text: str = "",
|
||||||
|
in_reply_to: str | None = None,
|
||||||
|
references_header: str | None = None,
|
||||||
|
signature: MailSignature | None = None,
|
||||||
|
attachment_paths: list[dict] | None = None,
|
||||||
|
) -> dict:
|
||||||
|
"""Send an email via SMTP using aiosmtplib.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
attachment_paths: list of dicts with keys 'path', 'filename', 'mime_type'
|
||||||
|
pointing to files on disk to attach.
|
||||||
|
"""
|
||||||
|
cc_addrs = cc_addrs or []
|
||||||
|
bcc_addrs = bcc_addrs or []
|
||||||
|
attachment_paths = attachment_paths or []
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
# Apply signature if provided
|
||||||
|
if signature and signature.body_html:
|
||||||
|
body_html = body_html + f"<br><br>{signature.body_html}"
|
||||||
|
if body_text:
|
||||||
|
body_text = body_text + "\n\n-- \n" + _strip_html(signature.body_html)
|
||||||
|
|
||||||
|
# Build email message
|
||||||
|
msg = EmailMessage()
|
||||||
|
msg["From"] = formataddr((account.display_name or "", account.email_address))
|
||||||
|
msg["To"] = ", ".join(to_addrs)
|
||||||
|
if cc_addrs:
|
||||||
|
msg["Cc"] = ", ".join(cc_addrs)
|
||||||
|
msg["Subject"] = subject
|
||||||
|
msg["Date"] = formatdate(localtime=True)
|
||||||
|
msg_id = make_msgid(
|
||||||
|
domain=account.email_address.split("@")[-1] if "@" in account.email_address else "localhost"
|
||||||
|
)
|
||||||
|
msg["Message-ID"] = msg_id
|
||||||
|
if in_reply_to:
|
||||||
|
msg["In-Reply-To"] = in_reply_to
|
||||||
|
if references_header:
|
||||||
|
msg["References"] = references_header
|
||||||
|
|
||||||
|
if body_html:
|
||||||
|
msg.set_content(body_text or _strip_html(body_html), subtype="plain")
|
||||||
|
msg.add_alternative(body_html, subtype="html")
|
||||||
|
else:
|
||||||
|
msg.set_content(body_text, subtype="plain")
|
||||||
|
|
||||||
|
# Add attachments to the message
|
||||||
|
for att_info in attachment_paths:
|
||||||
|
file_path = att_info.get("path", "")
|
||||||
|
filename = att_info.get("filename", os.path.basename(file_path))
|
||||||
|
mime_type = att_info.get("mime_type", "application/octet-stream")
|
||||||
|
if not file_path or not os.path.exists(file_path): # noqa: ASYNC240
|
||||||
|
continue
|
||||||
|
async with aiofiles.open(file_path, "rb") as f:
|
||||||
|
content = await f.read()
|
||||||
|
# Determine maintype/subtype from mime_type
|
||||||
|
if "/" in mime_type:
|
||||||
|
maintype, subtype = mime_type.split("/", 1)
|
||||||
|
else:
|
||||||
|
maintype, subtype = "application", "octet-stream"
|
||||||
|
msg.add_attachment(
|
||||||
|
content,
|
||||||
|
maintype=maintype,
|
||||||
|
subtype=subtype,
|
||||||
|
filename=filename,
|
||||||
|
)
|
||||||
|
|
||||||
|
# ── Hook: mail.before_send (Filter) ──
|
||||||
|
from app.core.hooks import apply_filters
|
||||||
|
mail_data = {
|
||||||
|
"subject": subject,
|
||||||
|
"body_html": body_html,
|
||||||
|
"body_text": body_text,
|
||||||
|
"to_addrs": to_addrs,
|
||||||
|
"cc_addrs": cc_addrs,
|
||||||
|
"bcc_addrs": bcc_addrs,
|
||||||
|
"attachment_paths": attachment_paths,
|
||||||
|
}
|
||||||
|
mail_data = await apply_filters("mail.before_send", mail_data)
|
||||||
|
|
||||||
|
# Send via SMTP
|
||||||
|
password = await get_account_password(account)
|
||||||
|
try:
|
||||||
|
smtp = aiosmtplib.SMTP(
|
||||||
|
hostname=account.smtp_host,
|
||||||
|
port=account.smtp_port,
|
||||||
|
use_tls=account.smtp_tls,
|
||||||
|
)
|
||||||
|
await smtp.connect()
|
||||||
|
await smtp.login(account.username, password)
|
||||||
|
recipients = to_addrs + cc_addrs + bcc_addrs
|
||||||
|
await smtp.send_message(msg, recipients=recipients)
|
||||||
|
await smtp.quit()
|
||||||
|
|
||||||
|
# ── Hook: mail.after_send (Action) ──
|
||||||
|
from app.core.hooks import do_action
|
||||||
|
await do_action("mail.after_send", mail_data, db=db, account=account, msg_id=msg_id)
|
||||||
|
await do_action("mail.after_update", mail_data, db=db, tenant_id=str(mail_data.get('tenant_id', '')) if isinstance(mail_data, dict) else None)
|
||||||
|
|
||||||
|
# Store sent mail in Sent folder — use configured mapping if set,
|
||||||
|
# otherwise flexible lookup to handle different IMAP naming conventions
|
||||||
|
if account.sent_folder_imap_name:
|
||||||
|
sent_folder = (
|
||||||
|
await db.execute(
|
||||||
|
select(MailFolder).where(
|
||||||
|
and_(
|
||||||
|
MailFolder.account_id == account.id,
|
||||||
|
MailFolder.imap_name == account.sent_folder_imap_name,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
)
|
||||||
|
).scalar_one_or_none()
|
||||||
|
else:
|
||||||
|
sent_folder = (
|
||||||
|
await db.execute(
|
||||||
|
select(MailFolder).where(
|
||||||
|
and_(
|
||||||
|
MailFolder.account_id == account.id,
|
||||||
|
MailFolder.imap_name.in_(
|
||||||
|
["Sent", "INBOX.Sent", "Sent Items", "Sent Mail"]
|
||||||
|
),
|
||||||
|
)
|
||||||
|
)
|
||||||
|
)
|
||||||
|
).scalar_one_or_none()
|
||||||
|
|
||||||
|
if not sent_folder:
|
||||||
|
sent_folder = (
|
||||||
|
await db.execute(
|
||||||
|
select(MailFolder).where(
|
||||||
|
and_(
|
||||||
|
MailFolder.account_id == account.id,
|
||||||
|
MailFolder.is_standard.is_(True),
|
||||||
|
MailFolder.imap_name.ilike("%sent%"),
|
||||||
|
)
|
||||||
|
)
|
||||||
|
)
|
||||||
|
).scalar_one_or_none()
|
||||||
|
|
||||||
|
if sent_folder:
|
||||||
|
thread_id = _compute_thread_id(msg_id, references_header or "", in_reply_to)
|
||||||
|
sent_mail = Mail(
|
||||||
|
tenant_id=tenant_id,
|
||||||
|
account_id=account.id,
|
||||||
|
folder_id=sent_folder.id,
|
||||||
|
message_id=msg_id,
|
||||||
|
thread_id=thread_id,
|
||||||
|
in_reply_to=in_reply_to,
|
||||||
|
references_header=references_header,
|
||||||
|
subject=subject,
|
||||||
|
from_address=account.email_address,
|
||||||
|
to_addresses=", ".join(to_addrs),
|
||||||
|
cc_addresses=", ".join(cc_addrs),
|
||||||
|
bcc_addresses=", ".join(bcc_addrs),
|
||||||
|
body_text=body_text or _strip_html(body_html),
|
||||||
|
body_html=body_html,
|
||||||
|
body_html_sanitized=sanitize_html(body_html),
|
||||||
|
is_seen=True,
|
||||||
|
is_answered=bool(in_reply_to),
|
||||||
|
sent_at=datetime.now(UTC),
|
||||||
|
)
|
||||||
|
db.add(sent_mail)
|
||||||
|
await db.flush()
|
||||||
|
|
||||||
|
# Upload sent mail to IMAP Sent folder via APPEND
|
||||||
|
try:
|
||||||
|
imap_client = aioimaplib.IMAP4_SSL(host=account.imap_host, port=account.imap_port)
|
||||||
|
await imap_client.wait_hello_from_server()
|
||||||
|
await imap_client.login(account.username, password)
|
||||||
|
# Build raw email bytes for APPEND
|
||||||
|
raw_email_bytes = msg.as_bytes()
|
||||||
|
append_resp = await imap_client.append(
|
||||||
|
sent_folder.imap_name,
|
||||||
|
r'(\Seen)',
|
||||||
|
None,
|
||||||
|
raw_email_bytes,
|
||||||
|
)
|
||||||
|
if append_resp.result == 'OK':
|
||||||
|
logger.info("send_mail: uploaded sent mail to IMAP folder %s", sent_folder.imap_name)
|
||||||
|
# Retrieve the new UID for the appended mail by searching for its Message-ID
|
||||||
|
try:
|
||||||
|
await imap_client.select(sent_folder.imap_name)
|
||||||
|
search_resp = await imap_client.uid_search(f'HEADER Message-ID "{msg_id}"')
|
||||||
|
uid_raw = search_resp[1][0] if search_resp[1] and search_resp[1][0] else b''
|
||||||
|
if isinstance(uid_raw, (bytes, bytearray)) and uid_raw:
|
||||||
|
new_uids = uid_raw.decode().split()
|
||||||
|
if new_uids:
|
||||||
|
sent_mail.imap_uid = new_uids[0]
|
||||||
|
logger.info("send_mail: set imap_uid=%s for sent mail %s", new_uids[0], sent_mail.id)
|
||||||
|
except Exception as uid_exc:
|
||||||
|
logger.warning("send_mail: could not retrieve UID after APPEND: %s", uid_exc)
|
||||||
|
else:
|
||||||
|
logger.warning("send_mail: IMAP APPEND failed for sent folder %s: %s", sent_folder.imap_name, append_resp)
|
||||||
|
await imap_client.logout()
|
||||||
|
except Exception as imap_exc:
|
||||||
|
logger.warning("send_mail: IMAP APPEND failed (non-critical): %s", imap_exc)
|
||||||
|
|
||||||
|
# ── Notification: mail sent ──
|
||||||
|
try:
|
||||||
|
await create_notification(
|
||||||
|
db, tenant_id, user_id,
|
||||||
|
"mail_sent",
|
||||||
|
"E-Mail gesendet",
|
||||||
|
subject,
|
||||||
|
)
|
||||||
|
await db.flush()
|
||||||
|
except Exception:
|
||||||
|
logger.debug("Ignored exception in mail service", exc_info=True)
|
||||||
|
|
||||||
|
return {"status": "sent", "message_id": msg_id}
|
||||||
|
except Exception as e:
|
||||||
|
# ── Notification: send error ──
|
||||||
|
try:
|
||||||
|
await create_notification(
|
||||||
|
db, tenant_id, user_id,
|
||||||
|
"mail_send_error",
|
||||||
|
"E-Mail konnte nicht gesendet werden",
|
||||||
|
str(e),
|
||||||
|
)
|
||||||
|
await db.flush()
|
||||||
|
except Exception:
|
||||||
|
logger.debug("Ignored exception in mail service", exc_info=True)
|
||||||
|
return {"status": "error", "error": str(e)}
|
||||||
|
|
||||||
|
|
||||||
|
async def reply_to_mail(
|
||||||
|
db: AsyncSession,
|
||||||
|
*,
|
||||||
|
tenant_id: uuid.UUID,
|
||||||
|
user_id: uuid.UUID,
|
||||||
|
original_mail: Mail,
|
||||||
|
account: MailAccount,
|
||||||
|
body_html: str,
|
||||||
|
body_text: str = "",
|
||||||
|
reply_to_all: bool = False,
|
||||||
|
signature: MailSignature | None = None,
|
||||||
|
) -> dict:
|
||||||
|
"""Reply to a mail, setting In-Reply-To and References headers (F-MAIL-02)."""
|
||||||
|
to_addrs = [original_mail.from_address]
|
||||||
|
if reply_to_all and original_mail.cc_addresses:
|
||||||
|
to_addrs.extend([a.strip() for a in original_mail.cc_addresses.split(",") if a.strip()])
|
||||||
|
|
||||||
|
refs = original_mail.references_header or ""
|
||||||
|
new_refs = f"{refs} {original_mail.message_id}".strip()
|
||||||
|
|
||||||
|
result = await send_mail_via_smtp(
|
||||||
|
db,
|
||||||
|
tenant_id=tenant_id,
|
||||||
|
user_id=user_id,
|
||||||
|
account=account,
|
||||||
|
to_addrs=to_addrs,
|
||||||
|
subject=f"Re: {original_mail.subject}".replace("Re: Re: ", "Re: "),
|
||||||
|
body_html=body_html,
|
||||||
|
body_text=body_text,
|
||||||
|
in_reply_to=original_mail.message_id,
|
||||||
|
references_header=new_refs,
|
||||||
|
signature=signature,
|
||||||
|
)
|
||||||
|
|
||||||
|
# Mark original as answered
|
||||||
|
original_mail.is_answered = True
|
||||||
|
await db.flush()
|
||||||
|
return result
|
||||||
|
|
||||||
|
|
||||||
|
async def forward_mail(
|
||||||
|
db: AsyncSession,
|
||||||
|
*,
|
||||||
|
tenant_id: uuid.UUID,
|
||||||
|
user_id: uuid.UUID,
|
||||||
|
original_mail: Mail,
|
||||||
|
account: MailAccount,
|
||||||
|
to_addrs: list[str],
|
||||||
|
cc_addrs: list[str] = None,
|
||||||
|
body_html: str = "",
|
||||||
|
body_text: str = "",
|
||||||
|
signature: MailSignature | None = None,
|
||||||
|
) -> dict:
|
||||||
|
"""Forward a mail with original as forwarded content (F-MAIL-02)."""
|
||||||
|
fwd_subject = f"Fwd: {original_mail.subject}".replace("Fwd: Fwd: ", "Fwd: ")
|
||||||
|
fwd_body = (
|
||||||
|
f"<br><br>----- Original Message -----<br>"
|
||||||
|
f"From: {original_mail.from_address}<br>"
|
||||||
|
f"Subject: {original_mail.subject}<br><br>"
|
||||||
|
f"{original_mail.body_html or original_mail.body_text}"
|
||||||
|
)
|
||||||
|
full_html = body_html + fwd_body
|
||||||
|
full_text = (body_text or _strip_html(body_html)) + "\n\n----- Original Message -----\n"
|
||||||
|
|
||||||
|
result = await send_mail_via_smtp(
|
||||||
|
db,
|
||||||
|
tenant_id=tenant_id,
|
||||||
|
user_id=user_id,
|
||||||
|
account=account,
|
||||||
|
to_addrs=to_addrs,
|
||||||
|
cc_addrs=cc_addrs or [],
|
||||||
|
subject=fwd_subject,
|
||||||
|
body_html=full_html,
|
||||||
|
body_text=full_text,
|
||||||
|
signature=signature,
|
||||||
|
)
|
||||||
|
|
||||||
|
original_mail.is_forwarded = True
|
||||||
|
await db.flush()
|
||||||
|
return result
|
||||||
|
|
||||||
|
|
||||||
@@ -0,0 +1,35 @@
|
|||||||
|
"""Text utilities for the Mail plugin.
|
||||||
|
|
||||||
|
Extracted from services.py as part of the God-object split (BUG-018 pilot).
|
||||||
|
Re-exported by ``app.plugins.builtins.mail.services``.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import re
|
||||||
|
|
||||||
|
|
||||||
|
def extract_email_addresses(text: str) -> list[str]:
|
||||||
|
"""Extract email addresses from a text string."""
|
||||||
|
if not text:
|
||||||
|
return []
|
||||||
|
return re.findall(r"[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}", text)
|
||||||
|
|
||||||
|
|
||||||
|
def _strip_html(html: str) -> str:
|
||||||
|
"""Simple HTML to text conversion for plain text fallback."""
|
||||||
|
if not html:
|
||||||
|
return ""
|
||||||
|
# Remove tags
|
||||||
|
text = re.sub(r"<[^>]+>", "", html)
|
||||||
|
# Replace HTML entities
|
||||||
|
text = (
|
||||||
|
text.replace(" ", " ")
|
||||||
|
.replace("&", "&")
|
||||||
|
.replace("<", "<")
|
||||||
|
.replace(">", ">")
|
||||||
|
.replace(""", '"')
|
||||||
|
.replace("'", "'")
|
||||||
|
)
|
||||||
|
# Collapse whitespace
|
||||||
|
return re.sub(r"\s+", " ", text).strip()
|
||||||
@@ -4,7 +4,7 @@ from __future__ import annotations
|
|||||||
|
|
||||||
import logging
|
import logging
|
||||||
import uuid
|
import uuid
|
||||||
from datetime import datetime
|
from datetime import UTC, datetime
|
||||||
from typing import Any
|
from typing import Any
|
||||||
|
|
||||||
from fastapi import APIRouter, Depends, HTTPException
|
from fastapi import APIRouter, Depends, HTTPException
|
||||||
@@ -170,7 +170,7 @@ async def list_server_tools(
|
|||||||
tools_resp = await client.list_tools()
|
tools_resp = await client.list_tools()
|
||||||
# Update last_connected_at
|
# Update last_connected_at
|
||||||
await db.execute(
|
await db.execute(
|
||||||
update(McpServerConfigModel).where(McpServerConfigModel.id == sid).values(last_connected_at=datetime.utcnow())
|
update(McpServerConfigModel).where(McpServerConfigModel.id == sid).values(last_connected_at=datetime.now(UTC))
|
||||||
)
|
)
|
||||||
await db.commit()
|
await db.commit()
|
||||||
return tools_resp
|
return tools_resp
|
||||||
|
|||||||
@@ -14,7 +14,7 @@ class McpServerPlugin(BasePlugin):
|
|||||||
version="1.0.0",
|
version="1.0.0",
|
||||||
display_name="MCP Server",
|
display_name="MCP Server",
|
||||||
description="Exposes LeoCRM data (contacts, calendar, mail, DMS) to external MCP clients via API-token auth.",
|
description="Exposes LeoCRM data (contacts, calendar, mail, DMS) to external MCP clients via API-token auth.",
|
||||||
dependencies=["permissions"],
|
dependencies=["permissions", "unified_search"],
|
||||||
routes=[
|
routes=[
|
||||||
PluginRouteDef(
|
PluginRouteDef(
|
||||||
path="/api/v1/mcp",
|
path="/api/v1/mcp",
|
||||||
|
|||||||
@@ -0,0 +1,9 @@
|
|||||||
|
"""Minimal Example plugin package.
|
||||||
|
|
||||||
|
Re-export the plugin class so ``discover_builtins`` finds it when scanning
|
||||||
|
the builtins package namespace (see guide chapter 29.1).
|
||||||
|
"""
|
||||||
|
|
||||||
|
from app.plugins.builtins.minimal_example.plugin import MinimalExamplePlugin
|
||||||
|
|
||||||
|
__all__ = ["MinimalExamplePlugin"]
|
||||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user