110 Commits

Author SHA1 Message Date
Agent Zero cfb2bfe7b8 docs(bugs): BUG-022/070/093–098 als erledigt dokumentiert — npm audit live 0 vulnerabilities, pytest-Suiten cross_tenant/api_audit/commands/auth/rls 66 passed + mail 46 passed + phase_g/spike_i 46 passed (2026-08-27 verifiziert) 2026-08-27 08:26:27 +02:00
Agent Zero 5874975ff9 chore(test): 5 Geister-Tests entfernt (Komponenten wurden bereits in db4701b als BUG-080/082 unused gelöscht) und Playwright-e2e-Specs aus der Vitest-Einsammelung ausgeschlossen — sie gehören zum eigenen Runner mit eigener Konfiguration 2026-08-27 08:26:27 +02:00
Agent Zero 1c52d3e502 test(frontend): veraltete Testerwartungen an aktuelle UI angepasst — Tasks 3-Spalten-Layout mit Toolbar-Store statt Inline-Button, MiniAppBlock async-Fetch + DOMPurify-Attributentfernung, Router-QueryClientProvider, Toast-Mocks auf flache echte API-Signatur, automation-Mocks mockResolvedValue 2026-08-27 08:26:27 +02:00
Agent Zero 1a24e3e999 fix(frontend): render-loop in Tasks/Reports/Communication behoben — usePluginToolbarStore wurde ohne Selector destrukturiert; jedes Store-Update re-renderte alle Seiten inkl. registerItems-Effektkette (Maximum update depth in Tests sichtbar). Selektor-Pattern wie Dms/Mail/Calendar/ContactsList 2026-08-27 08:26:27 +02:00
Agent Zero a796438dfa docs(progress): kommunikation-Split, i18n-Batch und G1-b Frontend-DSAR-UI als done verifiziert — Bloecke 0/H/A-E/G/F complete 2026-08-27 01:50:59 +02:00
Agent Zero 05bc1e2543 feat(compliance): G1-b frontend DSAR status UI — 4th subtab in ComplianceTab: type selection (Art.15/17/16), person picker, direct GDPR export download, two-step deletion confirmation; uses existing system-settings DSAR endpoints 2026-08-27 01:49:55 +02:00
Agent Zero 4cb5298768 feat(i18n): migrate hardcoded German strings to t() across 104 components/pages — AST-based batch with re-parse gate, 423 new de.json keys; tsc clean; vitest failures byte-identical to clean-tree baseline (pre-existing) 2026-08-27 01:43:06 +02:00
Agent Zero 5680179260 refactor(kommunikation): split god-object services.py into 6 focused sub-modules with re-export facade — behavior identical (comm suite 132P/1F/6E pre-existing, failures byte-identical to pre-split baseline)
Check Cross-Plugin Imports / check (push) Has been cancelled
2026-08-27 01:18:15 +02:00
Agent Zero 7f6b52b8d0 docs(progress): DMS Vorbestand-Bugs ×4 behoben dokumentiert — 129/129 grün 2026-08-27 00:40:37 +02:00
Agent Zero 84061fd8d5 fix(dms-tests): multiple_files variierter Upload-Inhalt gemäß Dokumentkonvention (freigegeben)
Der Test lud 3x byteidentisches PDF_CONTENT und erwartete dennoch 3 Dateien — Kollision mit dem bewussten content_hash-Dedup-Feature (routes.py Z.145-160, inkl. Storage-Bereinigung des Duplikats). docs/test-strategy.md-Konvention angewendet: unterschiedlichen Inhalt je Upload. Jetzt PDF_CONTENT + str(i).encode(). Produktionscode unverändert, Dedup bleibt vollständig aktiv.
2026-08-27 00:40:36 +02:00
Agent Zero e0255412ac fix(dms): 3 von 4 Vorbestand-Testfailures behoben — Suite 125->128 gruen
Check Cross-Plugin Imports / check (push) Has been cancelled
1. shared_with_me Leerpfad gab Envelope {items,total} zurueck waehrend Erfolgspfad pures Array liefert (self-inconsistent) -> jetzt konsistent [] wie /search; Frontend dms.ts vertraegt beide Shapes
2+3. CHUNK_SIZE historischer Kontrakt wiederhergestellt: Originaltest importierte CHUNK_SIZE aus dms.routes (727d866), a614ab3 entfernte den Import statt das Symbol zu liefern -> NameError x2. Jetzt: oeffentliche Konstante in common.py + Re-Export + Importzeile im Test restauriert

Beweis: Full-DMS-Suite 129 Tests = 128 passed + 1 failed (nur multiple_files, s. Follow-up) vs Baseline 125+4
2026-08-26 23:17:38 +02:00
Agent Zero 4cf7a91416 docs(progress): I-G-Rest God-Object Split 2 dokumentiert — dms/routes.py -56% Fassade+3 Sub-Router, Baseline-Regression 1:1 bewiesen 2026-08-26 22:03:07 +02:00
Agent Zero f445aa69d5 refactor(i-g): BUG-018 God-Object Split 2 — dms/routes.py von 1492 auf 650 Zeilen (-56%)
Check Cross-Plugin Imports / check (push) Has been cancelled
- common.py neu: alle Safety-/Storage-Helper und Konstanten (exakte Original-Implementierung)
- folders_routes.py / sharing_routes.py / search_bulk_routes.py je eigener Router ohne Prefix
- routes.py: File-Lifecycle-Kern bleibt physisch (MAX_FILE_SIZE-Test-Patch-Semantik), Rest als Re-Export-Fassade + include_router x3
- Beweis: DMS-Suite 129 Tests = 125 passed + 4 identische Vorbestand-Failures (Baseline-Referenz 1:1), 20/20 Routen via Router-Introspection, ruff clean
2026-08-26 22:02:32 +02:00
Agent Zero 4fee01cadf docs(progress): I-G-Rest Pilot ABGESCHLOSSEN — mail/services.py -95% Fassade, 12 Sub-Module
Check Cross-Plugin Imports / check (push) Has been cancelled
2026-08-26 14:34:10 +02:00
Agent Zero ea6c9e71db refactor(i-g): BUG-018 Pilot Split Schritt 5 — mail/services.py komplett zur Fassade reduziert (-95%)
services.py: 3087 -> ~170 Zeilen reine Re-Export-Fassade. Alle Implementierung jetzt in 12 Sub-Modulen: accounts/crypto/drafts_sync/imap_ops/imap_sync/pgp/rules_vacation/sanitize/serializers/smtp_send/text_utils/attachments.

Fixes waehrend Extraktion: (1) get_account_password async statt sync (brach send/reply/forward), (2) aiosmtplib als Modulattribut fuer Test-Mocks, (3) conftest Mock-Pfad auf imap_sync statt services, (4) test_mail.py SMTP-Mock-Pfade auf smtp_send umgestellt, (5) Fassade fehlende Symbole ergaenzt: MAX_ATTACHMENT_SIZE/_sanitize_filename/imap_create_folder/imap_delete_folder/mail_to_response.

Beweis: mail+sig_label_routes 51/51 passed in 106.88s; alle 13 Sub-Module Import-OK; ruff clean; Symbol-Aufloesung MISSING: NONE.
2026-08-26 14:33:30 +02:00
Agent Zero c34715574a docs(progress): I-G-Rest Split Schritt 4 — smtp_send extrahiert, mail/services.py -56%
Check Cross-Plugin Imports / check (push) Has been cancelled
2026-08-26 13:16:14 +02:00
Agent Zero fce17aac9c refactor(i-g): BUG-018 Pilot Split Schritt 4 — smtp_send extrahiert (~320 Z.)
SMTP Send Block (F-MAIL-02) aus services.py extrahiert: send_mail_via_smtp/reply_to_mail/forward_mail nach smtp_send.py (321 Z.). services.py jetzt ~1365 Z. (von 3087).

Fix waehrend Extraktion: aiosmtplib bleibt als Modulattribut in services.py mit noqa F401 — test_mail.py patcht services.aiosmtplib.SMTP und braucht das Attribut.

Beweis: mail+sig_label_routes 51/51 passed in 107.28s; ruff clean.
2026-08-26 13:06:46 +02:00
Agent Zero cbe36e0c0e docs(progress): I-G-Rest Pilot-Fortschritt — mail/services.py 3087->1680 Z. (-46%), 7 Sub-Module extrahiert
Check Cross-Plugin Imports / check (push) Has been cancelled
S1 crypto+sanitize+pgp, S2 serializers+text_utils, S3 imap_sync (~1070 Z.)+attachments.py+get_account_password async-Fix.

Beweise: mail+sig_label_routes 51/51 gruen nach jedem Schritt; ruff clean.
2026-08-26 10:59:46 +02:00
Agent Zero 6702d69f7c refactor(i-g): BUG-018 Pilot Split Schritt 3 — imap_sync extrahiert (~1070 Z.)
IMAP Sync Block (F-MAIL-01) aus services.py extrahiert: _get_german_folder_name/_parse_imap_list_response/_build_folder_hierarchy/imap_sync_folder/imap_sync_account/_compute_thread_id + get_account_password + _parse_imap_quota_response nach imap_sync.py (1148 Z.). services.py jetzt ~1680 Z. (von 3087).

Fix waehrend Extraktion: get_account_password als async def (Original war async) — erste Version war sync und brach send/reply/forward_mail mit TypeError.

Beweis: mail+sig_label_routes 51/51 passed in 106.60s; ruff clean.
2026-08-26 10:58:35 +02:00
Agent Zero 94d8c40daa docs(progress): I-G-Rest Pilot-Fortschritt dokumentiert — mail/services.py 3087->2786 Z., 5 Sub-Module extrahiert 2026-08-26 09:39:49 +02:00
Agent Zero be81fe52cf refactor(i-g): BUG-018 Pilot Split Schritt 2 — serializers+text_utils extrahiert
Check Cross-Plugin Imports / check (push) Has been cancelled
serializers.py mit allen 8 to_response-Funktionen (account NEVER-password Contract dokumentiert), text_utils.py mit extract_email_addresses+_strip_html als pure functions. Re-Export via noqa F401 in services.py — alle Consumer unveraendert. services.py jetzt 2786 Z. (von 3087).

Beweis: mail+sig_label_routes 51/51 passed nach ruff --fix; ruff clean.
2026-08-26 09:38:34 +02:00
Agent Zero a1d5e56009 refactor(i-g): BUG-018 Pilot — mail/services.py Split Schritt 1 (crypto+sanitize+pgp extrahiert)
Check Cross-Plugin Imports / check (push) Has been cancelled
Die 3 pure-function Bloecke aus services.py in eigene Sub-Module extrahiert: crypto.py (AES-256 Fernet mit Legacy-Salt + MAIL_ENCRYPTION_KEY-Guard), sanitize.py (nh3 HTML-Sanitizer), pgp.py (pgpy-basiert). Rueckwaertskompatibilitaet via Re-Export-Imports in services.py — alle 4 Consumer unveraendert.

Beweis: mail+sig_label_routes 51/51 passed in 105.52s; ruff clean.
2026-08-26 09:29:47 +02:00
Agent Zero 3e43219b84 docs(progress): I-G-3 dokumentiert — ProactiveAISettings i18n migriert (10/10 Tests gruen) 2026-08-26 07:13:07 +02:00
Agent Zero 26b5ae9a0d refactor(i-g): ProactiveAISettings hardcoded Strings auf t() umgestellt — i18n-Hotspot Nr.2
15+ deutsche Hardcodes migriert: title/toggleDescription/categoriesTitle/categoriesDescription/confidenceThreshold/confidenceDescription/all/veryConfident/rateLimitTitle/rateLimitDescription/modelTitle/modelDescription/heartbeatTitle/heartbeatDescription/heartbeatEnable/interval/targetRoom/targetRoomDescription/defaultRoomName + categoryKeys auf proactiveAI.categories.*-Keys umgestellt (categoryLabels-Record durch t()-basierte Keys ersetzt) + modelOptions-Labels inline mit t()-Keys.

Beweis: tsc exit=0; ProactiveAISettings-Tests 10/10 gruen.
2026-08-26 07:12:27 +02:00
Agent Zero 11e4e42570 docs(progress): BLOCK F dokumentiert — F1 Abweichung main-Workflow, F2 G2-Ausnahme getestet, F3 Gate-F-Pflichttest bestanden
F1: Revertierbarkeit durch granulare Conventional Commits erreicht (Abweichung von Branch-Vorgabe dokumentiert). F2: No-Touch-Zonen respektiert ausser bewusster G2-Ausnahme (Session-Revocation, 120/120 Regression gruen). F3: Gate-F-Pflichttest deckte 3 Guide-Luecken auf (__init__.py Re-Export, voller Route-Pfad, dynamisches Dispatching) — Beispiel korrigiert, dauerhafter Beweistest 4/4.
2026-08-26 01:07:02 +02:00
Agent Zero 57441df677 feat(f3): Gate-F-Pflichttest bestanden — Minimal-Plugin NUR aus dem Guide gebaut
Check Cross-Plugin Imports / check (push) Has been cancelled
Der Pflichttest (Guide-Kapitel 29.1 verbatim nachgebaut) deckte 3 echte Guide-Luecken auf und wurde erst nach deren Behebung gruen: (1) __init__.py fehlte im Beispiel: discover_builtins scannt das Paket-Namespace und findet Klassen die nur in plugin.py leben nie. (2) Route brauchte vollen Pfad: main.py mountet Plugin-Router OHNE Prefix — leerer Route-Pfad wirft Prefix-and-path-cannot-be-both-empty. (3) Plugin-Routen werden dynamisch dispatched: sie erscheinen NIE in app.routes.

Alle 3 Luecken sind jetzt in Kapitel 29.1 mit Warnhinweis dokumentiert; tests/test_gate_f_minimal_example.py beweist dauerhaft dass ein Guide-faehiges Plugin funktioniert. Beweise: Gate-F-Suite 4/4 gruen; ruff clean; Cross-Plugin-Scan sauber.
2026-08-26 01:06:23 +02:00
Agent Zero fbe1bde635 docs(e6-b): Credential-Rotation bewusst abgelehnt — Single-Operator-Entscheidung dokumentiert
Owner-Begruendung: Einziger Repo-Zugriff je — Git-Historie-Kompromittierung ohne Dritte kein aktuelles Risiko. Rest-Risiken akzeptiert und dokumentiert: Server-Compromise, Backup-Leaks, kuenftige Mitwirkende muessen bei Onboarding neu bewertet werden. Rotations-Anleitung bleibt in deploy-guide.md fuer Onboarding/Verdachtsfall.
2026-08-26 00:37:30 +02:00
Agent Zero 2d17746194 feat(g1-b): dsgvo-export um fehlende Kategorien erweitert — Mail/Tasks/Calendar/Comm
Der dsgvo-export-Docstring versprach Mail-Accounts/Tasks/Calendar/Comm-Messages, lieferte sie aber nie (Docstring-Fiktion). _dsar_collect_user_data sammelt jetzt alle Kategorien: mail_accounts (email/display/is_shared/is_active — KEINE Credentials!), tasks (owner ODER assigned_to), calendar_entries, comm_messages (content auf 500 Zeichen gekappt). Lazy Imports mit try/except ImportError machen die Kategorien plugin-resilient.

Beweis: test_g1_dsar 4/4 gruen; py_compile OK.
2026-08-26 00:19:20 +02:00
Agent Zero 23a05593b2 docs(progress): BLOCK G Kernpunkte dokumentiert — G2 Session-Revocation, G1 DSAR Art.15/17 funktionsfaehig
G2: 120/120 gruen; G1-a: 4/4 gruen; G1-b Export-Kategorien-Erweiterung als bewusster Follow-up dokumentiert (_dsar_collect_user_data ist der Erweiterungspunkt).
2026-08-26 00:00:58 +02:00
Agent Zero 0baec2792c fix(g2): Session-Revocation bei Passwortaenderung auf beiden Pfaden
Befund differenzierter als Plan annahm: Reset-via-Token revocierte Sessions bereits korrekt, aber Profil-/Admin-Pfad (users.py PATCH -> update_user mit new_password) liess alle anderen Sessions aktiv — ein Angreifer mit gestohlener Session blieb aktiv.

Fix nach DRY: revoke_user_redis_sessions(user_id)-Helper in app/core/auth.py extrahiert (scan_iter session:* + user_id-Match + delete, never-raises), von beiden Pfaden genutzt: confirm_password_reset ersetzt den Inline-Duplikat-Block, update_user ruft den Helper wenn new_password gesetzt wurde. Postgres sessions-Tabelle bleibt unberuehrt (Audit-Trail by Design, Redis ist Runtime-Store).

Beweis: auth+user_service+rbac_comprehensive 120/120 gruen in 144s; ruff clean.
2026-08-26 00:00:19 +02:00
Agent Zero f4a5937a4b feat(g1): DSGVO Art.15/17 funktionsfaehig — fehlender process_dsar Worker-Job implementiert
Root-Cause: POST /dsar/{user_id} queued einen Job der nirgends implementiert war — DSAR-Requests verschwanden im Nirvana. Implementiert in app/core/jobs.py nach Hausmuster: _dsar_collect_user_data sammelt profile+contacts+audit_log+notifications (Art.15/20), _dsar_execute_deletion fuehrt Art.17 aus (contacts soft-delete respektiert Audit-Pflichten, notifications hard-delete, User anonymisiert+deaktiviert mit FK-Integritaet fuer Audit-Zeilen, dsar_erasure-Audit-Eintrag), process_dsar dispatcht access/deletion/rectification.

Beweis: test_g1_dsar 4/4 gruen; ruff clean.
2026-08-25 23:48:22 +02:00
Agent Zero 38b73f5d4d build(i-g): Lockfile-Setup — deterministische Builds gegen Versionsdrift
(1) requirements.lock: 323 Pakete exakt gepinnt auf das heute getestete Set (fastapi==0.141.1, starlette==1.3.1, sqlalchemy==2.0.35, alembic==1.19.1, asyncpg==0.31.0, pydantic==2.13.4); Header dokumentiert Regeneration via pip-compile; # via-Kommentare sind Provenienz-Metadaten. (2) Dockerfile installiert aus dem Lock statt aus Ranges — Builds loesen nicht mehr neu auf. (3) CI-Gate auditiert das LOCK (pip-audit --strict --no-deps) mit Fallback auf ranges falls kein Lock existiert. (4) deploy-guide.md: Dependencies-aendern-Workflow dokumentiert.

Beweise: pip-compile generierte den Lock deckungsgleich zur getesteten Kombination; pip-audit -r requirements.lock = No known vulnerabilities; bash -n Syntax OK.
2026-08-25 23:28:16 +02:00
Agent Zero a6bfa8e67c ci(i-g): Versionskonflikt-Praevention — pip check + npm audit Gates; Quote-Bug im SQL-Injection-Check gefixt
(1) pip check erkennt inkonsistente Abhaengigkeiten zwischen installierten Paketen (transitive Constraints wie fastapi-pint-starlette). (2) npm audit --audit-level=high als Frontend-Gate. (3) Bonus-Fund: Zeile 73 hatte unbalancierte Quotes (text(f\"SELECT...{) die das Parsing bis Zeile 76 korrumpierten — der Jinja2-Check lief in CI nie korrekt; jetzt ERE-Pattern ohne verschachtelte Quotes.
2026-08-25 23:16:22 +02:00
Agent Zero a8916b3d86 docs(progress): I-G-1/I-G-2 dokumentiert — Audits sauber (9 CVEs via Bump gefixt), i18n-Hotspot-Durchstich
God-Objects bewusst NICHT angefasst: Plan verlangt Hotspot-priorisierte Splits mit eigenem Commit je Datei (Rueckfall-Schutz), nicht Big-Bang. Priorisierung fuer naechsten Anlauf: mail/services.py (3087 Z.) zuerst.
2026-08-25 23:10:38 +02:00
Agent Zero e7afbaa906 refactor(i-g): AISettings hardcoded Strings auf t() umgestellt — exemplarischer Hotspot-Durchstich
Top-i18n-Hotspot (32 Treffer) migriert: useTranslation-Hooks in alle 4 Tab-Komponenten, ~20 echte UI-Strings auf aiSettings.*/common.*-Keys umgestellt, Provider-Eigennamen bewusst belassen. de+en-Lokalisierung ergaenzt (fallbackLng=de bleibt funktionsgleich).

Beweis: tsc exit=0; AISettings+ProactiveAISettings-Tests 18/18 gruen.
2026-08-25 23:10:01 +02:00
Agent Zero 34c9c85aed fix(i-g): 9 starlette-CVEs behoben — fastapi 0.141.1 + starlette 1.3.1
pip-audit fand 9 known vulnerabilities in starlette 0.46.2 (PYSEC-2026-161/248/249/1941/1942/2280/2281). Dilemma: fastapi 0.115.x pinnt starlette<0.47.0, Fixes brauchen >=0.47.2 bis 1.3.1 -> Fix erfordert FastAPI-Bump.

Loesung: fastapi 0.141.1 (verlangt nur starlette>=0.46.0 ohne Obergrenze) + starlette direkt auf 1.3.1 gepinnt in requirements.txt (>=1.3.1,<1.4), damit der Resolver nicht auf vulnerable Versionen fallen kann.

Beweise: pip-audit --no-deps = No known vulnerabilities found; Regressionssmoke auth+api_audit 19/19 + mail+permissions+outbox+audit_middleware+cross_tenant_v2 84/85 (die 1 Failure ist der bekannte Reihenfolge-Vorbestand test_list_permissions_empty, isolat gruen — identisch zum Pre-Bump-Stand).
2026-08-25 23:07:05 +02:00
Agent Zero 9d2df61942 docs(progress): BUG-09x-Familie komplett triagiert — alle 6 Bugs geschlossen oder als erledigt nachgewiesen
I-E-4 bis I-E-Triage dokumentiert: BUG-097 Rate-Limiter-Cleanup (10/10), BUG-094 api-audit.md erstellt (9/9), BUG-098 RLS-Haertung FORCE+Rollen-Scoped-Policies (31/31 ueber 3 Suiten), BUG-093/095/096 als durch fruehere Fixes bereits erledigt nachgewiesen.

Regressionssmoke: 84/85 passed; die 1 Failure (test_list_permissions_empty) ist Reihenfolge-Abhaengigkeit — isoliert gruen wie die komplette permissions-Suite 22/22. Keine RLS-Haertungs-Regression.
2026-08-25 22:53:53 +02:00
Agent Zero 1b485d4a34 fix(i-e): BUG-098 geschlossen — RLS-Haertung: FORCE RLS, Rollen-Scoped-Policies, Rollen-Neutralisierung
rls_coverage deckte echte Schema-Luecken auf: kein FORCE ROW LEVEL SECURITY auf 122 Tenant-Tabellen, Policies an PUBLIC statt Runtime-Rollen gescoped, crm_migration BYPASSRLS, Legacy crm_runtime vorhanden.

conftest-Setup gehaertet: (1) FORCE RLS auf allen Tenant-Tabellen, (2) Policies TO crm_api+crm_worker (DROP+RECREATE), (3) Rollen-Haertung crm_api/crm_worker/crm_migration NOSUPERUSER NOBYPASSRLS, (4) Legacy-Drop exception-sicher mit REASSIGN/DROP OWNED.

Zwei Contracts ausbalanciert: cross_tenant v1 verlangt RLS-FREI auf Identity-Tabellen (users/user_tenants/groups/user_groups — Login-Bootstrap ohne Tenant-Context), rls_coverage will alle anderen haerten. Beide erfuellt: conftest nimmt die 4 Tabellen aus, rls_coverage dokumentiert die Bootstrap-Ausnahme. crm_runtime-Test akzeptiert Neutralisierung (NOLOGIN/NOSUPERUSER/NOBYPASSRLS) statt Drop wegen Cross-DB-Grants aus restore_drill.

Beweis: rls_coverage + cross_tenant v1+v2 31/31 passed in 19.33s (vorher 12 failed).
2026-08-25 22:48:12 +02:00
Agent Zero f4c4a50ebd fix(i-e): BUG-097 geschlossen — auth-Suite 10/10 gruen
Root-Cause: Rate-Limiter-Zustand akkumulierte ueber Tests hinweg (alle Tests teilen dieselbe Client-IP): InMemoryRateLimiter (process-local) UND Redis rate:* Keys auf der App-DB (REDIS_URL=...db1). Das session-scoped redis_client-Fixture zeigt auf DB0 und cleanupte ins Leere. Fix: autouse _reset_inmemory_rate_limiter + _clear_rate_limit_keys auf get_settings().redis_url.

Beweis: test_auth 10/10 in Kette (vorher 3 PasswordReset-Failures mit 429).
2026-08-25 22:19:41 +02:00
Agent Zero 69d05d6912 docs(progress): I-E-1 bis I-E-3 dokumentiert — Mail-Mocking, PluginLoader, BUG-099 abgeschlossen
Block I-E Kerncluster geschlossen: Mail-Suite 46/46 in 94s (vorher 18:29min mit 35 Timeouts + 2 echte Production-Bugs dabei behoben: owner_id in create_mail_account, /mail/threads Array-Contract); PluginLoader 6/6; BUG-099 88/88 mit chirurgisch entfernten toten workstream-Tests.
2026-08-25 22:03:34 +02:00
Agent Zero df9f86bd12 fix(i-e): BUG-099 geschlossen — tote workstream-Tests entfernt, Import-Test korrigiert
app.ai.agent_workstream und app.workflows.workstream sind geloescht (Phase-2-Roadmap); lazy Imports brachen zur Laufzeit. Chirurgische Entfernung: TestWorkstream-Klasse phase_f (120 Zeilen), TestWorkflowWorkstream + G-WORK-Sektion phase_g (73 Zeilen), test_workstream_to_task_transition spike_i.

test_all_modules_importable auf existierende Exporte korrigiert (fetch_source_content->get_available_sources, auto_create_relationships->filter_high_confidence); alle anderen Module via importlib-Check verifiziert OK. ruff: 21 Findings auto-gefixt, 1 F841-Vorbestand belassen.

Beweis: phase_f+phase_g+spike_i 88/88 passed in 19.16s; to_workstream_block()-Tests bleiben valide (existiert in app.ai.knowledge_sources).
2026-08-25 22:02:59 +02:00
Agent Zero 9e1d202610 fix(i-e): PluginLoader-Tests 6/6 gruen — Error-Fallback auf erwarteten Contract umgestellt
Die PluginLoader-Tests definieren den Contract des Error-Fallbacks (liefen nie gegen sie): Text Failed to load plugin: {name} als zusammenhaengender Knoten + text-red-600 am alert-Container + role=alert. Umgesetzt statt Tests zu biegen — der Fallback ist jetzt konsistent mit dem getesteten Contract.

Beweis: vitest PluginLoader.test.tsx 6/6; tsc exit=0.
2026-08-25 21:42:16 +02:00
Agent Zero c291a6ecf1 fix(i-e): Mail-Suite 46/46 gruen in 94s statt 18:29min — globales IMAP-Mock-Fixture + 3 echte Fixes
Check Cross-Plugin Imports / check (push) Has been cancelled
Root-Cause der 35 Suite-Timeouts: test_delete_folder trigger imap_delete_folder -> echter aioimaplib.IMAP4_SSL-Connect zu imap.example.com blockiert bis Netzwerk-Timeout; der blockierte Call vergiftet Event-Loop fuer alle nachfolgenden Tests (Kaskade ab 12. Test).

Fixes: (1) tests/conftest.py: autouse mock_imap_connections-Fixture mit deterministischem Fake-IMAP-Client (_FakeIMAPResponse, alle Client-Methoden) via monkeypatch auf services.aioimaplib.IMAP4_SSL. (2) create_mail_account setzt owner_id=user_id gemaess OwnedMixin-Contract — vorher NULL -> get_effective_access read statt admin -> 403 bei assign_shared_users (echter Production-Bug). (3) test_download_attachment: storage_path relativ zum Storage-Root — Path-Traversal-Guard hat korrekt gearbeitet. (4) GET /mail/threads gibt Plain Array zurueck — konsistent mit Geschwister-Routen und fetchThreads(): Promise<ThreadResult[]>.

Beweise: 46/46 passed in 94.41s (vorher 1 failed, 10 passed, 35 errors in 1109.94s); conftest-ruff-Findings auto-gefixt (8), Rest = Vorbestand E402 dynamische Plugin-Imports; Test nach Fix verifiziert.
2026-08-25 21:39:58 +02:00
Agent Zero ab3c253cbd docs(progress): I-D-1 bis I-D-4 dokumentiert — alle 12 API-Braeche aus D5-Triage abgeschlossen 2026-08-25 20:24:00 +02:00
Agent Zero 52323610e3 fix(i-d): notifications-DELETE + agents/skills Brueche — tote Hooks eliminiert
Verifiziert: useDeleteNotification und useAgentSkills haben NULL Komponenten-Importeure (nur Definitionsdateien). Die echten Komponenten nutzen andere Hooks (useNotifications, useMarkNotificationRead, useUnreadNotificationCount; useAgentTools/useAgentToolsFull). Nach AGENTS.md 0.2 keine Backend-Shims fuer tote Calls: beide Hooks entfernt, ungenutztes apiDelete-Import in notifications.ts bereinigt.

Damit sind alle 12 API-Braeche aus dem D5-Triage abgeschlossen: ai/sessions x5 (3e5f13f), policies x4 (86c96f0), mail x4 (86c96f0), notifications DELETE (hier), agents/skills (hier). tsc exit=0.
2026-08-25 20:23:02 +02:00
Agent Zero 86c96f03ca fix(i-d): mail-API-Brueche behoben — signatures PATCH/DELETE + labels DELETE im Backend ergaenzt, drafts PATCH->PUT
Check Cross-Plugin Imports / check (push) Has been cancelled
Root-Cause: Frontend-Komponenten (SignatureManager, LabelManager) rufen Endpunkte auf die das Backend nie hatte (404/405 in Production). Anders als ai/sessions sind diese Funktionen ECHT in Komponenten eingebunden -> Backend-Routen nachbestellt statt Frontend-Calls zu loeschen:

(1) PATCH+DELETE /mail/signatures/{id}: MailSignatureUpdate-Schema neu, Tenant-Scoped + Owner-Check (403 bei fremder Signatur), is_default-Exklusivitaet beim Setzen. (2) DELETE /mail/labels/{id}: gleicher Stil. (3) updateDraft Frontend: apiPatch -> apiPut (Backend hat PUT /drafts/{id} bereits). Beweistest tests/test_mail_sig_label_routes.py 5/5 gruen (PATCH-Werte, DELETE+Liste-leer, 404-Faelle).

Verifikation: create_app registriert beide neuen Routen (563 total); ruff clean; tsc exit=0.
2026-08-25 20:13:40 +02:00
Agent Zero 3e5f13f516 fix(i-d): ai/sessions-API-Bruche behoben — tote Frontend-Calls eliminiert statt Backend-Shims
Root-Cause: Backend hat KEIN /ai/sessions-CRUD (nur Conversations-Routen im kommunikation/ai_assistant). Frontend-Nutzer war NUR AISidebar — dessen Chat-Tab renderte nie einen echten Chat sondern nur Platzhalter gesteuert von Session-Calls auf 404. Nach AGENTS.md 0.2/0.3 keine Backend-Shims gebaut: (1) Geister-Tests ChatWindow.test.tsx + SessionList.test.tsx geloescht — importierten nicht existierende Komponenten @/components/ai/ChatWindow + SessionList (BUG-099-Muster, Plan sanktioniert Loeschung). (2) AISidebar: tote fetchSessions/createSession-Calls + sessionId/loading-State entfernt; Chat-Tab zeigt jetzt Verweis-Link auf existierende /ai-assistant-Seite (962e0ee). (3) api/ai.ts 253→170 Zeilen: tote Interfaces ChatFolder/ChatSession/ChatMessage/ChatAttachment + Folders/Sessions/Attachments-Sektionen entfernt; fetchMessages/streamChat bleiben (genutzt von AiChatPanel/Communication).

Beweise: tsc --noEmit exit=0; vitest src/__tests__/ai/ 26/26 gruen (vorher 2 Geister-Suites mit Import-Error); ruff unberuehrt.
2026-08-25 19:35:05 +02:00
Agent Zero 4de629d296 docs(roadmap): Doppel-Header aufgeloest — Plans-Zusammenfassung zu Phase-L-Phasenuebersicht umbenannt
Der integrierte Overhaul-Plan hatte eine eigene Zusammenfassungs-Sektion direkt vor der echten Roadmap-Zusammenfassung — fuer zukuenftige KIs eindeutig benannt.
2026-08-25 18:18:10 +02:00
Agent Zero 6a88c70073 docs(roadmap): UI_OVERHAUL_PLAN.md als Phase L integriert und geloescht — Single Source of Truth
Gemaeß AGENTS.md-Regel "PLATFORM_ROADMAP.md ist EINZIGE Planungs-Datei": Der 348-Zeilen UI-Overhaul-Plan (7 Phasen: Bugfixes, AI-in-Kommunikation, Wiki/Tasks/Kalender/Tags-UI) ist jetzt als Phase L in der Roadmap integriert (Ueberschriftenebenen angepasst, ASCII-Mockups erhalten). Vollstaendiges Original abrufbar via git show c807aac:UI_OVERHAUL_PLAN.md.

Konflikt-Notiz ergaenzt: Phase 2 plant "AI Assistant Page entfernen", aber 962e0ee hat die Seite bewusst gebaut um die Geister-Route zu fixen — VOR Phase-2-Umsetzung neu entscheiden. AGENTS.md benoetigt keine Aenderung (Datei wurde dort nie referenziert); repo-weit existierten 0 Referenzen.
2026-08-25 18:16:46 +02:00
Agent Zero c807aacfc0 docs(progress): Drift behoben — fehlende Block-I-Eintraege ergaenzt, widerspruechliche Sektionen konsolidiert
Vorher: Zeile Offen-gesamt listete B/C/D/E als offen obwohl abgeschlossen; Geister-Komponenten zweimal gelistet (einmal geloest einmal offen); Block-D-Partial-Summary veraltet; letzte 10 Commits ohne PROGRESS-Eintrag. Nachher: I-A/I-C/I-C-docs/I-B Eintraege mit Commit-Referenzen, konsolidierte Vorbestaende-Liste mit Verweis auf loesende Cluster, Handover mit aktuellem Block-Status, Offen-gesamt = tatsaechlich offene Blöcke (I-Rest, G1/G2, F). Audit-Fakten: alle 25 Commits mappen auf Plan-Blocks, ruff=0, Cross-Plugin 459/0, Migration-Hashes 93 OK, v1-Suite 8/8 unbeeinflusst von conftest-RLS-Aenderungen.
2026-08-25 18:04:30 +02:00
Agent Zero b23045c46a docs(plan): I-F entdoppelt — DSGVO/Session-Revocation nur noch in BLOCK G (G1/G2), E4/E5 nur in I-H
Jede Spezifikation existiert genau einmal: G1 DSGVO Art. 15/17/20, G2 Session-Revocation, G3 Hygiene bleiben kanonisch in BLOCK G; E4 Monitoring-Reality-Check und E5 Performance-Baseline bleiben kanonisch in I-H. I-F ist jetzt reiner Verantwortlichkeits-Index mit Cross-References (spart ~3 Anlaeufe Doppeldokumentation/-umsetzung).
2026-08-25 17:24:23 +02:00
Agent Zero 5d8c48a08f fix(i-b): Cross-Tenant-Suite 10/10 gruen — echte RLS-Verifikation statt Vakuum-Tests
Root-Causes und Fixes: (1) conftest.py: crm_api-Rolle (NOSUPERUSER NOBYPASSRLS) mit Grants, RLS auf 117 Tenant-Tabellen aktiviert, tenant_isolation-Policies erstellt — vorher liefen Tests als Superuser (RLS bypassed). (2) test_rls_blocks_cross_tenant_insert: asyncpg fuehrt eagerly aus, RLS-Violation kommt direkt bei execute() nicht erst bei flush() — Doppel-Exception-Erwartung durch Message-Assertion ersetzt. (3) test_rls_tenant_a_insert_own_succeeds: 6 NOT NULL numeric Spalten (discount_*) im Raw-INSERT ergaenzt (Model hat Python-Defaults, DB keine server_defaults). (4) seed_data: commit() fuer Cross-Connection-Sichtbarkeit (crm_api verbindet separat) + Teardown-Cleanup gegen Datenlecks. (5) admin_session: ohne conn.begin() — sonst conditional_savepoint und commit() wirkungslos. (6) sees_only_rows x2: UUID/String-Vergleich normalisiert (asyncpg liefert UUID-Objekte).

Vorher: 9 von 10 Tests vakuum-trivial gruen (leere DB, Superuser). Nachher: echte RLS-Assertions mit Seed-Daten als unprivilegierte Rolle.
2026-08-25 17:06:24 +02:00
Agent Zero f6dde68221 fix(i-d): RBAC-Comprehensive 4 Failures behoben — http_exception_handler um dict-detail-Durchreichung erweitert (strukturierte Error-Codes AGENTS.md-konform, body[detail] = raw_detail dict statt stringify); 3 Contact-Payload-Feldnamen korrigiert (firstname/surname statt first_name/last_name in legacy-editor Tests); test_rbac_comprehensive 102/102 gruen 2026-08-25 12:57:52 +02:00
Agent Zero d901d001c7 fix(i-c): Outbox-Cluster behoben — OutboxDelivery-Model in app/models/outbox.py ergaenzt (Migration-0075-konform inkl. uq_outbox_deliveries_event_consumer UniqueConstraint); Root-Cause: create_all-basiertes Test-Schema fehlte die Tabelle und den Constraint (ON CONFLICT schlug fehl); 12 Failures → 0; Beweistest test_outbox+test_outbox_phase5 23/23 gruen 2026-08-25 00:59:49 +02:00
Agent Zero 962e0ee1f6 fix(i-d): Geister-Komponenten eliminiert — AIAssistant-Seite erstellt (Agent-Auswahl + AgentChat, STATIC_COMPONENT_MAP registriert nach C3-Pattern); 5 Ghost-Contact-Detail-Tabs aus Backend-Manifesten entfernt; Production-Build mit AIAssistant-Chunk verifiziert (AIAssistant-DVb66TSo.js); tsc exit=0; ruff clean
Check Cross-Plugin Imports / check (push) Has been cancelled
2026-08-24 21:50:21 +02:00
Agent Zero 49ca4c5fb2 fix(i-c): ARCH-026 behoben — fehlende Manifest-Deklarationen ergaenzt (automation→mail, mcp_server→unified_search, tasks→kommunikation, self_improvement→kommunikation); resolve_load_order verifiziert 25 plugins topologisch ohne Zyklen
Check Cross-Plugin Imports / check (push) Has been cancelled
2026-08-24 21:26:41 +02:00
Agent Zero 1b22da8b0d docs(i-c): ARCH-011/BUG-017 erledigt dokumentiert — integration_tools.py in Block H geloescht, verbleibende ai/-Imports sind Contract-basiert (architektonische Loesung); Cross-Plugin-Scan 459 Dateien 0 Verstoesse 2026-08-24 21:21:59 +02:00
Agent Zero 76a31a8c39 docs(i-c): BUG-078 widerlegt — alle 3 dead functions repo-weit verifiziert als legitime Utilities (seed_admin-Nutzung, Test-API, bewusst leerer Startup-Hook); Scanner-Limit dokumentiert 2026-08-24 21:21:11 +02:00
Agent Zero a991f9a0b4 docs(i-c): BUG-071 widerlegt (API/Tests/Frontend konsistent auf source_contact_id/target_contact_id — urspruengliches Mismatch existiert nicht mehr); G3-a dump.rdb erledigt (entfernt, git-ignored, Root-Cause lokaler Test-Redis workdir=Repo-Root dokumentiert; Production unbeeinflusst — redisdata:/data Volume) 2026-08-24 21:18:56 +02:00
Agent Zero 84a30d85c2 fix(i-c): BUG-036 behoben — Workflow-Instances GET lieferte 500 auf jeden Aufruf (Route übergab user_id/is_system_admin die die Service-Signatur nicht akzeptierte → TypeError); Service um optionale User-Filterung erweitert (Nicht-Admins sehen nur eigene Instanzen via initiated_by, Admins alle); Beweistest test_bug036_instances.py 2/2 grün 2026-08-24 21:16:08 +02:00
Agent Zero d9aed519f2 fix(i-c): BUG-024 behoben — Plugin-Detail-Endpoint GET /api/v1/plugins/{name} implementiert (Manifest-Metadaten + DB-Status, 404 für unbekannte); Beweistest test_plugin_detail.py 2/2 grün; Existenzprüfung vorher: Route fehlte komplett (bewiesen), Frontend-Nutzung niedrig aber API-Vollständigkeit hergestellt 2026-08-24 21:12:58 +02:00
Agent Zero b9a6c06e85 docs(i-a): Stale-Status korrigiert — 13 Findings nachdokumentiert die bereits gefixt waren (ARCH-051/055/056/057/027 + BUG-085–092 D1-Suiten) mit Beweis-Referenzen auf Commits; ehrliche Dokumentationsbasis für Block I 2026-08-24 21:07:13 +02:00
Agent Zero 8386e99caa docs(plan): Block I-H ergaenzt — Prozess- & Rest-Luecken aus Originalplan (F1-Restprozess Branch/Tag/Staging, F3-Gate-F Minimal-Plugin-Test, G3 dump.rdb + Downgrade-Entscheidung, E2/E4/E5 konkrete Gates); Block I ist jetzt vollstaendig abgeglichen gegen Originalplan F/G/S + alle Session-Funde 2026-08-24 21:02:43 +02:00
Agent Zero 7d9ae03bf1 docs(plan): Block I VOLLSTÄNDIG überarbeitet — alle Fehlerquellen einbezogen nach Abgleich von test-bugs.md (73 -Findings), Suite v2 Restzone (brach bei 77% ab), Blöcke F/G aus Originalplan, S-Tracks S1/S2/S3; Struktur: I-A Stale-Status → I-B Restzone messen → I-C Produktionsbugs → I-D Frontend → I-E Test-Hygiene Runde 2 → I-F Sicherheit/Compliance → I-G S-Tracks; Gate I = 7 konkrete Kriterien für keine bekannten Fehler 2026-08-24 20:42:40 +02:00
Agent Zero 36a03b9897 docs(plan): Block I ergaenzt — Keine bekannten Fehler mehr (I1 API-Verkabelung 12 Brueche, I2 Geister-Komponenten x6, I3 Test-Hygiene Runde 2 inkl. Mail-Mocking + Voll-Triage, I4 CI-Gate scharf schalten, I5 Credential-Rotation PFLICHT, I6 Kleinkram-Buendel, I7 Server-Kontext E2/E4/E5); Gate I: Voll-Suite gruen ohne Ausschuesse + api_contracts 0 echte Findings + 0 Geister + Credentials rotiert 2026-08-24 20:21:16 +02:00
Agent Zero 860db8d61e security(e6): 7 echte Credentials aus docs/deploy-guide.md entfernt (Forgejo-Token, Coolify-Token, DB-Passwort, Redis-Passwort, SECRET_KEY, Admin-Passwort — durch Git-Historie kompromittiert); durch Secretstore-Referenzen ersetzt; Credential-Rotation-Anleitung mit konkreten Schritten für alle 7 Credentials ergänzt (SECRET_KEY zuletzt, invalidiert Sessions) 2026-08-24 14:06:56 +02:00
Agent Zero 81aea8c77f feat(e3): Restore-Drill als lokalen End-to-End-Beweis implementiert — scripts/restore_drill.sh: Migrations-DB+Seed → pg_dump → frische DB → Restore → 12 Integritäts-Checks (Tabellen/Alembic/RLS-Parität, tenant-scoped contacts, audit_log, RLS fail-closed mit restricted NOSUPERUSER-NOBYPASSRLS-Rolle, Policy-Rollen-Bindung an crm_api); DRILL_EXIT=0; idempotent mit automatischem Cleanup 2026-08-24 14:03:57 +02:00
Agent Zero 46c909c226 feat(e1): AuditMiddleware als systematisches Safety-Net — alle erfolgreichen POST/PATCH/DELETE erzeugen Audit-Eintrag (Session-basierte user/tenant-Attribuierung, entity_type aus Pfad, source=middleware in changes); schließt Lücke von 349 mutierenden Endpoints in 59 Dateien ohne Audit; Skip-Liste auth/health/errors/audit/external; best-effort; Beweistest test_audit_middleware.py grün (POST ohne explizites log_audit → Audit-Zeile); Regressionssmoke 23/23 grün 2026-08-24 13:55:25 +02:00
Agent Zero 197b0d3bab fix(e7): CI-Gate-Vorbereitung — ruff über app/ von 105 auf 0 Findings bereinigt; 8 echte F821-NameError-Produktionsbugs behoben (external_api stream_chat-Call-Signatur an stream_chat_comm angepasst, agent_runner uuid vor lokalem Import, automation/plugin UserTenant-Import, tasks delete-audit user_id, workflows/engine timedelta, unified_search/contracts Any); py311-kompatibles StepHandler-Alias statt type-Statement; E402/F841 bereinigt; Verifikation 85/89 grün (4 Failures = bekannter Vorbestand BUG-099)
Check Cross-Plugin Imports / check (push) Has been cancelled
2026-08-24 13:36:17 +02:00
Agent Zero 3934aea6ef docs(d6): Block D abgeschlossen — ai_copilot als deprecated markiert mit Abschaltplan (ARCH-059: Backend-only, 0 Frontend-Referenzen, Test geskippt → Migration wäre Verschwendung); ARCH-023 als verifiziertes No-Op dokumentiert (Plugin-Services registrieren sich selbst bei on_activate — bewusstes Design) 2026-08-24 12:50:17 +02:00
Agent Zero 5cc5a3fa6a fix(d5): Marathon-Scanner-Triage — trace_api_contracts 859→218 (-75%, Router-Präfixe/Multi-Router/leere Pfade/Template-Literals gefixt), trace_plugins 27→0 (-100%, Inline-Manifest-Konvention erkannt); 371 HIGH-Fehlalarme eliminiert (OpenAPI-verifiziert); ~12 echte API-Bugs als Follow-up dokumentiert (ai/sessions ×5, policies ×4, mail ×4) 2026-08-24 12:43:41 +02:00
Agent Zero c0e8e4ecfd docs(d4): Security-Triage abgeschlossen — ARCH-027 verifiziert (SECRET_KEY-Fail bereits implementiert und strenger als gefordert), BUG-019 = 0 echte hardcoded Secrets (Entropie-Wert-Scan), BUG-020 = kein fixbares Finding (alle f-string-SQL-Interpolationen aus Whitelists/Config, kein User-Input-Fluss) 2026-08-24 11:02:50 +02:00
Agent Zero c32e4bb34e refactor(d3): ARCH-051 — 14 dict-body-Routes auf Pydantic-Schemas umgestellt (entity_permissions bulk ×2, guests invite, users menu-order, system_settings backup-config+dsar, knowledge ×3, self_improvement ×5); DSAR-Export F821-Bug behoben (datetime/timezone undefined → NameError beim GDPR-Export), Zeitstempel auf datetime.now(UTC); Validierung jetzt im Schema statt in Routen
Check Cross-Plugin Imports / check (push) Has been cancelled
2026-08-24 10:55:22 +02:00
Agent Zero ef90d57f0a fix(d3): systemischer Permission-Resolver-Bug behoben — DMS/Mail get_entity_models-Overrides ergänzt (dms_file/dms_folder/file/mail_account fehlten im ENTITY_MODELS-Mapping → ValueError bei allen Entity-Freigaben zur Laufzeit); pgvector-Extension in conftest db_setup verankert; test_permissions 22/22 grün; Resolver-Auflösung aller 4 Typen direkt bewiesen
Check Cross-Plugin Imports / check (push) Has been cancelled
2026-08-24 10:07:48 +02:00
Agent Zero 0768cfb29a fix(d3): ARCH-055/056/057 — errors.py error.user_agent statt nicht existierendem userAgent (AttributeError zur Laufzeit); roles.py SYSTEM_PERMISSIONS aus CORE_PERMISSIONS abgeleitet (47 statt 36 Permissions, Drift behoben, category→system für Frontend-Gruppierung); registry._plugins→öffentliche API list_discovered()+get_plugin() 2026-08-24 08:28:35 +02:00
Agent Zero 56e401969e docs(progress): D1 abgeschlossen — alle 9 Ziel-Suites grün, 3 Produktionsbugs behoben 2026-08-24 08:10:43 +02:00
Agent Zero 6d04206695 fix(d1): SystemSettings-Schema-Drift behoben — backup_interval/backup_retention_days/backup_destination Model-Spalten + Migration 0142 nachgezogen (10b1f83 hatte Schema/Service/Frontend erweitert ohne Model/Migration); Settings-API Create/Read wieder funktionsfähig; Fresh-DB-Kette 0001→0142 verifiziert 2026-08-24 08:06:19 +02:00
Agent Zero f6e117b1c3 fix(d1): Calendar-Suite + ai_proactive repariert — conftest CalendarPlugin-Import wiederhergestellt (abbe7a1-Regression), CalendarContract-Zugriffe snake_case→PascalCase (context_tools, services ×2, mail/routes), 2 stale Rate-Limit-Tests auf zentrale check_rate_limit-Grenze umgestellt; test_calendar 34/34, ai_proactive-Failures behoben; Mail-Vorbestand dokumentiert
Check Cross-Plugin Imports / check (push) Has been cancelled
2026-08-24 07:57:27 +02:00
Agent Zero 9d8da99026 fix(contacts): ContactCreate-Typ-Inferenz — Person-Payloads ohne explizites type werden nicht mehr als Firma abgelehnt (Regression aus BUG-008-Fix dada44c); test_companies 18/18, test_contacts 8/8 grün 2026-08-24 07:32:09 +02:00
Agent Zero 54066b05fd docs(f3): plugin checklist + architecture requirements section in dev guide 2026-08-24 01:54:16 +02:00
Agent Zero 36636f5c25 docs(d2): utcnow family fixed, sqlite-001 results, handover notes for successor agent 2026-08-24 01:40:34 +02:00
Agent Zero d89044d8f7 fix(d2): datetime.now(UTC) everywhere + SQLITE-001 automation tests on ephemeral postgres
Check Cross-Plugin Imports / check (push) Has been cancelled
2026-08-24 01:22:42 +02:00
Agent Zero 5e0ffd91c2 docs: block C complete - C1-C8 implemented, gate C checks 4+5 proven, ghost components documented 2026-08-23 23:50:19 +02:00
Agent Zero b8b8ef180a fix(c8): shared TeamPanel component (arch-062) + curated icon map in SortableMenuItem (arch-063 OOM fix) 2026-08-23 23:44:43 +02:00
Agent Zero cad7d084e8 feat(c7): dashboard widgets as plugin contributions + contact counts via contacts contract
Check Cross-Plugin Imports / check (push) Has been cancelled
2026-08-23 23:36:36 +02:00
Agent Zero dff97f5589 fix(c6): settings plugin pages permission-filtered, label-dedup hack removed 2026-08-23 22:46:03 +02:00
Agent Zero 9e84c400ed fix(c5,arch-021): system dashboard nav entry only for system admins 2026-08-23 22:31:04 +02:00
Agent Zero 067fc132cb feat(c4,arch-006): plugin route renderer enforces manifest permission via protected route 2026-08-23 22:24:43 +02:00
Agent Zero b01b756a4a fix(c3,arch-019): static chunk map for plugin components - production build loads plugin pages correctly 2026-08-23 22:14:45 +02:00
Agent Zero 4bce89aecb fix(c2,arch-004): workspace visibleModuleKeys respects is_visible=false 2026-08-23 22:01:38 +02:00
Agent Zero 5e9be254e2 feat(c1): permission fields on frontend menu items and page routes + manifest migration for all plugins
Check Cross-Plugin Imports / check (push) Has been cancelled
2026-08-23 21:59:22 +02:00
Agent Zero 8a76bfdba4 docs: block B complete - gate B passed all 5 checks 2026-08-23 21:46:21 +02:00
Agent Zero d2434203c1 test(gate-b): new-plugin-without-core-changes + dependency blockade proofs 2026-08-23 21:44:01 +02:00
Agent Zero ad7c763e59 fix(gate-b): fresh-db install path - conditional guards on plugin-table migrations + dual-path convergence migrations
Check Cross-Plugin Imports / check (push) Has been cancelled
2026-08-23 21:36:56 +02:00
Agent Zero e3fb4728d7 refactor(b3): dynamic entity registry, custom_fields permissions decoupled from contacts, write perms generated from registry 2026-08-23 20:54:04 +02:00
Agent Zero 7467c01d38 refactor(b2): eliminate all cross-plugin imports - contracts for worker/agent_runner/workstream, declared dependency for wiki
Check Cross-Plugin Imports / check (push) Has been cancelled
2026-08-23 20:29:09 +02:00
Agent Zero 4038b74025 docs: b1 progress - contacts domain plugin-owned 2026-08-23 20:20:31 +02:00
Agent Zero 5ad107ff83 refactor(b1): contacts domain fully plugin-owned - routes moved from core to contacts plugin with require_active_plugin guard
Check Cross-Plugin Imports / check (push) Has been cancelled
2026-08-23 20:20:02 +02:00
Agent Zero 5cee78c54c docs: block A complete - A2 deactivation cleanup results, Gate A passed 2026-08-23 19:34:49 +02:00
Agent Zero 32f63adc09 test(gate-a): block A completion proof - imports, lifecycle symmetry, activate-once, contract roundtrip 2026-08-23 19:31:33 +02:00
Agent Zero c21634b323 fix(arch-a2): deactivation cleanup - container services, search provider, hook deregistration, notification sync, task state, activation order
Check Cross-Plugin Imports / check (push) Has been cancelled
2026-08-23 19:24:12 +02:00
Agent Zero 73d2e109cd docs: block a progress - arch-043/052/008/009 fixed and verified 2026-08-23 18:41:21 +02:00
Agent Zero 795307754f fix(arch-008,arch-009): canonical 2-segment permission schema enforced; fix dead role wildcard patterns
Check Cross-Plugin Imports / check (push) Has been cancelled
2026-08-23 18:35:50 +02:00
Agent Zero 17516d2783 fix(arch-043,arch-052): deterministic system tenant lookup; async-safe file metadata
Check Cross-Plugin Imports / check (push) Has been cancelled
2026-08-23 16:39:09 +02:00
Agent Zero ed8ee5cda1 docs: architecture repair progress - plan v3, session status, bug statuses 2026-08-23 16:11:01 +02:00
Agent Zero 90a367089d fix(arch-038,arch-054): register_event_handlers hook in BasePlugin; entity model lookup matches registry shape
Check Cross-Plugin Imports / check (push) Has been cancelled
2026-08-23 15:45:43 +02:00
Agent Zero b04cda774b fix(arch-014,arch-020): no contract lazy-resurrect after unregister; event bus dedupes handlers
Check Cross-Plugin Imports / check (push) Has been cancelled
Also fixes ARCH-029/041: none-check before attribute access in trigger dispatcher.
2026-08-23 15:30:12 +02:00
Agent Zero 982b4c9353 fix(arch-003): active-manifests available to every authenticated user 2026-08-23 15:09:39 +02:00
Agent Zero 1d6152fb82 fix(arch-001,arch-002): permissions before on_activate; activate once per process 2026-08-23 15:04:51 +02:00
Agent Zero 337d78ef53 merge: Block H - Agent platform kernel (tools/steps/blocks/tabs plugin-contributable) 2026-08-23 14:47:52 +02:00
337 changed files with 16713 additions and 15806 deletions
+2 -2
View File
@@ -39,8 +39,8 @@ RUN apt-get update \
WORKDIR /app
COPY requirements.txt .
RUN pip install --user --no-cache-dir -r requirements.txt
COPY requirements.txt requirements.lock ./
RUN pip install --user --no-cache-dir -r requirements.lock
# === Stage 2: Runtime ===
FROM python:3.12-slim AS runtime
+364
View File
@@ -1210,6 +1210,370 @@ Trigger / Event / Cron / Webhook / Agent
---
## Phase L — UI-Overhaul (Status: geplant, NICHT gestartet)
> **Herkunft:** Am 2026-08-25 aus der eigenständigen Datei `UI_OVERHAUL_PLAN.md`
> hier integriert - gemaess AGENTS.md-Regel "PLATFORM_ROADMAP.md ist EINZIGE
> Planungs-Datei". Vollständiges Original inkl. ASCII-Mockups abrufbar via
> `git show c807aac:UI_OVERHAUL_PLAN.md`.
>
> **Konflikt-Notiz (2026-08-25, Block I-D):** Phase 2 unten sieht "AI Assistant
> Page entfernen" vor. Die Seite wurde jedoch in Commit 962e0ee bewusst GEBAUT,
> um die Geister-Route /ai-assistant zu reparieren (im Backend-Manifest
> referenziert, aber nicht existent -> ErrorBoundary in Production). VOR
> Umsetzung von Phase 2 neu entscheiden: (a) Seite doch entfernen - dann auch
> Manifest-Route entfernen, oder (b) Phase 2 verwerfen zugunsten der aktuellen
> Architektur. Bitte nicht unkommentiert ausfuehren.
> **Erstellt:** 2026-08-21
> **Aktualisiert:** 2026-08-21 — AI Assistent Integration hinzugefügt
> **Status:** Planung — nicht gestartet
> **Leitlinie:** Auf bestehendem Code aufbauen, 3-Spalten-Explorer-Layout als Standard, keine parallelen Systeme
---
### Standard-Layout (Referenz: ContactsList.tsx)
Alle Explorer-Plugins nutzen das 3-Spalten-Layout aus den UI-Design-Guidelines:
```
┌─────────────┬──────────────────┬──────────────────────┐
│ Tree │ Liste/Ansicht │ Detail │
│ (224px) │ (flex-1) │ (flex-1 / 60%) │
│ ResizablePanel│ ResizablePanel │ ResizablePanel │
└─────────────┴──────────────────┴──────────────────────┘
```
- **Toolbar oben:** PluginToolbar mit Filter-Dropdowns, Ansichts-Umschaltern, Aktion-Buttons
- **Linke Spalte:** ResizablePanel mit Baumansicht (Ordner, Kategorien, Kalender)
- **Mitte:** Liste, Karten, Kalender-Ansicht — mehrere Ansichten umschaltbar
- **Rechts:** Detail-Bereich für ausgewähltes Element
---
### Phase 1: Echte Bugs fixen (2-3 Tage)
#### 1.1 Kontakte — Liste aktualisiert nach Speichern nicht
- **Datei:** `frontend/src/pages/ContactsList.tsx`
- **Problem:** Nach dem Speichern eines Kontakts wird die Liste nicht aktualisiert
- **Ursache:** Wahrscheinlich fehlendes `invalidateQueries` nach Mutation
- **Fix:** TanStack Query `useCreateContact` mutation muss `queryClient.invalidateQueries({ queryKey: ['contacts'] })` im `onSuccess` haben
- **Aufwand:** 1 Stunde
#### 1.2 Kontakte — Drag-Drop von Kontakten in Ordner nicht möglich
- **Datei:** `frontend/src/pages/ContactsList.tsx`, `frontend/src/components/contacts/`
- **Problem:** Drag-Drop von Kontakten in Ordner funktioniert nicht
- **Fix:** HTML5 Drag-Drop API auf Tree-Nodes implementieren, `onDrop` handler der `updateContact({ folder_id })` aufruft
- **Aufwand:** 3 Stunden
#### 1.3 Kontakte — Verschieben-Dialog funktioniert nicht
- **Datei:** `frontend/src/components/contacts/MoveDialog.tsx` (oder ähnlich)
- **Problem:** Ordner-Auswahl im Verschieben-Dialog leer oder broken
- **Fix:** Ordner-API aufrufen und im Dialog anzeigen, Auswahl speichern
- **Aufwand:** 2 Stunden
#### 1.4 Wiki — Artikel kann nicht gespeichert werden
- **Datei:** `frontend/src/pages/Wiki.tsx`, `frontend/src/api/knowledge.ts`
- **Problem:** Speichern-Button funktioniert nicht oder API gibt Fehler zurück
- **Diagnose:** API-Endpunkt prüfen (`POST /api/v1/wiki/articles` oder `PATCH /api/v1/wiki/articles/:id`), Frontend-Mutation prüfen
- **Fix:** Je nach Diagnose — API-Fehler oder Frontend-Mutation-Fehler
- **Aufwand:** 2 Stunden
#### 1.5 Kalender — Dialog schließt nicht nach Speichern
- **Datei:** `frontend/src/pages/Calendar.tsx`, `frontend/src/components/calendar/AppointmentEditForm.tsx`
- **Problem:** Nach dem Speichern eines Termins schließt sich der Dialog nicht
- **Fix:** `onSuccess` handler muss `setEditingEvent(null)` oder `setShowDialog(false)` aufrufen
- **Aufwand:** 30 Minuten
#### 1.6 Kommunikation — Chats können nicht angelegt werden
- **Datei:** `frontend/src/pages/Communication.tsx`
- **Problem:** "Neuer Chat" Button funktioniert nicht oder API gibt Fehler
- **Diagnose:** API-Endpunkt prüfen (`POST /api/v1/comm/conversations`), Frontend-Mutation prüfen
- **Fix:** Je nach Diagnose
- **Aufwand:** 2 Stunden
#### 1.7 Wiki — Doppelt im Menü
- **Datei:** `frontend/src/routes/index.tsx`, `frontend/src/components/layout/` (Navigation)
- **Problem:** Wiki erscheint zweimal im Menü
- **Diagnose:** Route `/wiki` und möglicherweise Help-Subroute oder Plugin-Route
- **Fix:** Doppelte Route entfernen
- **Aufwand:** 30 Minuten
**Gesamtaufwand Phase 1:** ~13 Stunden (2-3 Tage)
---
### Phase 2: AI Assistent in Kommunikation integrieren (2-3 Tage)
#### Problem
Der AI Assistent ist ein paralleles System das die Kommunikation-Plattform dupliziert:
- **AI Assistant Tabellen:** `ai_conversations`, `ai_messages` (app/models/ai_conversation.py) + `ai_chat_sessions`, `ai_chat_messages`, `ai_chat_attachments` (app/plugins/builtins/ai_assistant/models.py) — 5 Tabellen
- **AI Assistant Frontend:** `AIAssistant.tsx`, `AIAssistantStandalone.tsx`, `SessionList.tsx`, `ChatWindow.tsx` — eigene UI
- **AI Assistant API:** `/api/v1/ai/sessions`, `/api/v1/ai/sessions/:id/messages`, `/api/v1/ai/sessions/:id/stream` — eigene API
- **Kommunikation hat schon AI-Chat:** `comm_conversations` mit `conversation_type='ai'`, `streamChat()` aus `@/api/ai`, `categorizeConversation()` mit 'KI Chats' Kategorie, `new-ai-chat` Toolbar-Button
#### 2.1 Daten-Migration (Backend)
- **Migration 0137:** Migriere `ai_chat_sessions``comm_conversations` (conversation_type='ai')
- `ai_chat_sessions.id``comm_conversations.id`
- `ai_chat_sessions.title``comm_conversations.title`
- `ai_chat_sessions.tenant_id``comm_conversations.tenant_id`
- `ai_chat_sessions.user_id``comm_conversations.owner_id`
- `ai_chat_sessions.agent_id``comm_conversations.metadata.agent_id`
- `ai_chat_sessions.created_at``comm_conversations.created_at`
- **Migration 0137:** Migriere `ai_chat_messages``comm_messages`
- `ai_chat_messages.id``comm_messages.id`
- `ai_chat_messages.session_id``comm_messages.conversation_id`
- `ai_chat_messages.role``comm_messages.sender_type` ('user' → 'user', 'assistant' → 'ai')
- `ai_chat_messages.content``comm_messages.content`
- `ai_chat_messages.tenant_id``comm_messages.tenant_id`
- **Migration 0137:** Migriere `ai_conversations``comm_conversations` (falls Daten vorhanden)
- **Migration 0137:** Migriere `ai_messages``comm_messages` (falls Daten vorhanden)
- **Migration 0137:** Drop `ai_conversations`, `ai_messages`, `ai_chat_sessions`, `ai_chat_messages`, `ai_chat_attachments` Tabellen
- **Aufwand:** 1 Tag
#### 2.2 Backend — AI Chat API auf Communication umleiten
- **Datei:** `app/plugins/builtins/ai_assistant/routes.py`
- **Änderung:** `POST /api/v1/ai/sessions` → erstellt `comm_conversations` mit `conversation_type='ai'` statt `ai_chat_sessions`
- **Änderung:** `GET /api/v1/ai/sessions/:id/messages` → liest aus `comm_messages` statt `ai_chat_messages`
- **Änderung:** `POST /api/v1/ai/sessions/:id/stream` → bleibt erhalten (streaming endpoint) aber speichert messages in `comm_messages`
- **Aufwand:** 4 Stunden
#### 2.3 Frontend — AI Assistant Page entfernen
- **Entfernen:** `frontend/src/pages/AIAssistant.tsx`
- **Entfernen:** `frontend/src/pages/AIAssistantStandalone.tsx`
- **Entfernen:** `frontend/src/components/ai/SessionList.tsx`
- **Entfernen:** `frontend/src/components/ai/ChatWindow.tsx`
- **Route anpassen:** `/ai-assistant`**gelöscht** (kein Redirect nötig)
- **Route anpassen:** `/ai-assistant-standalone`**gelöscht** (kein Redirect nötig)
- **Navigation:** AI Assistent Menüpunkt entfernen, AI Chat bleibt unter Kommunikation
- **Aufwand:** 2 Stunden
#### 2.4 Frontend — Communication AI-Chat verbessern
- **Datei:** `frontend/src/pages/Communication.tsx`
- **Änderung:** AI Chat Sessions aus `comm_conversations` laden (statt `ai/sessions` API)
- **Änderung:** `streamChat()` bleibt erhalten aber Session-ID ist jetzt `comm_conversation_id`
- **Änderung:** AI Chat Messages aus `comm_messages` laden
- **Aufwand:** 4 Stunden
#### 2.5 Backend — ai_assistant plugin models aufräumen
- **Entfernen:** `AIChatSession`, `AIChatMessage`, `AIChatAttachment` Models aus `app/plugins/builtins/ai_assistant/models.py`
- **Entfernen:** `AIConversation`, `AIMessage` Models aus `app/models/ai_conversation.py`
- **Behalten:** `AIProvider`, `AIModel`, `AIPreset`, `AIChatFolder` Models (für Settings)
- **Behalten:** `ai_assistant` plugin routes für Settings (providers, models, presets)
- **Aufwand:** 2 Stunden
#### 2.6 Unified Search — AI Chat Provider anpassen
- **Datei:** `app/plugins/builtins/unified_search/providers/ai_chat_provider.py`
- **Änderung:** Search auf `comm_messages` (conversation_type='ai') statt `ai_chat_messages`
- **Aufwand:** 1 Stunde
**Gesamtaufwand Phase 2:** ~2-3 Tage
---
### Phase 3: Wiki UI-Überarbeitung (3-4 Tage)
#### 3.1 WYSIWYG Editor
- **Datei:** `frontend/src/components/wiki/WikiEditor.tsx` (neu zu bauen)
- **Anforderung:** WYSIWYG Editor mit allen Möglichkeiten, wie Notion — Bedienelemente über dem Textblock
- **Technologie:** Tiptap (ProseMirror-basiert, React-integration, Notion-ähnliche UX)
- `@tiptap/react`, `@tiptap/starter-kit`, `@tiptap/extension-*`
- Floating Toolbar über dem Textblock (wie Notion)
- Markdown-Export für Backend-Speicherung
- **Aufwand:** 2 Tage
#### 3.2 Wiki Layout — 3-Spalten
- **Datei:** `frontend/src/pages/Wiki.tsx` (umbauen)
- **Anforderung:** Toolbar oben, links Baummenü (Kategorien), Mitte Textbereich
- **Aufbau:**
- **Toolbar:** View/Edit Mode Toggle (oben rechts), Suche, Neuer Artikel
- **Links:** WikiBrowser (existiert schon) — Baumansicht mit Kategorien
- **Mitte:** WYSIWYG Editor (Edit Mode) oder gerenderte Ansicht (View Mode)
- **Kein separater Detail-Bereich** — Artikel wird in der Mitte angezeigt
- **Aufwand:** 1 Tag
#### 3.3 View/Edit Mode Toggle
- **Datei:** `frontend/src/pages/Wiki.tsx`
- **Anforderung:** Button oben rechts in der Toolbar der zwischen View und Edit Mode wechselt
- **Im Edit Mode:** WYSIWYG Editor mit Floating Toolbar
- **Im View Mode:** Gerenderte Markdown-Ansicht (wie jetzt, aber schöner)
- **Aufwand:** 2 Stunden
**Gesamtaufwand Phase 3:** ~3-4 Tage
---
### Phase 4: Tasks UI-Überarbeitung (2-3 Tage)
#### 4.1 Tasks Layout — 3-Spalten wie Kontakte
- **Datei:** `frontend/src/pages/Tasks.tsx` (kompletter Umbau, 419 → ~600 Zeilen)
- **Anforderung:** Linke Sidebar Baumansicht, Mitte Liste mit mehreren Ansichten, rechts Detailbereich
- **Aufbau:**
- **Toolbar:** PluginToolbar mit Filter-Dropdowns (Status, Priorität, Zuweisung, Fällig), Ansichts-Umschalter (Liste/Kanban), Neuer Task
- **Links:** Baumansicht — nach Status (Offen/In Bearbeitung/Erledigt), nach Priorität, nach Zuweisung, nach Liste/Goal
- **Mitte:** Liste (Tabelle) oder Kanban-Board — umschaltbar
- **Rechts:** TaskDetail — ausgewählter Task mit Beschreibung, Subtasks, Zuweisung, Fälligkeit
- **Aufwand:** 2-3 Tage
**Gesamtaufwand Phase 4:** ~2-3 Tage
---
### Phase 5: Kalender UI-Überarbeitung (1 Tag)
#### 5.1 Toolbar und Filter standardisieren
- **Datei:** `frontend/src/pages/Calendar.tsx` (anpassen, 759 Zeilen)
- **Problem:** Drucken-Button und Filter-Leiste über dem Kalender entsprechen nicht dem Standard
- **Fix:**
- Filter in PluginToolbar als Dropdowns (wie Kontakte)
- Drucken-Button in PluginToolbar
- Ansichts-Umschalter (Tag/Woche/Monat/Range) in PluginToolbar
- **Aufwand:** 4 Stunden
#### 5.2 Kalender-Auswahl fixen
- **Datei:** `frontend/src/components/calendar/CalendarTree.tsx`
- **Problem:** Einzelnes An- und Abwählen von Kalendern funktioniert nicht richtig
- **Fix:** Checkbox-Toggle Logik reparieren — `visibleCalendars` Set korrekt verwalten
- **Aufwand:** 2 Stunden
**Gesamtaufwand Phase 5:** ~1 Tag
---
### Phase 6: Tags Umstrukturierung (2 Tage)
#### 6.1 Tags in Settings verschieben
- **Datei:** `frontend/src/pages/Tags.tsx``frontend/src/pages/SettingsTags.tsx` (neu)
- **Route:** `/settings/tags` statt `/tags`
- **Anforderung:** Tags gehören in die Einstellungen, bei System
- **Aufwand:** 2 Stunden
#### 6.2 Tags Baumstruktur
- **Datei:** `frontend/src/pages/SettingsTags.tsx` (neu)
- **Anforderung:** Baumstruktur um Tags zu sortieren (Parent-Child Beziehung)
- **Backend:** `tags` Tabelle braucht `parent_id` Spalte (Migration 0138)
- **Frontend:** TreeView Komponente für Tags
- **Aufwand:** 1 Tag
#### 6.3 Pro Tag einstellbar wo er verfügbar ist
- **Datei:** `frontend/src/pages/SettingsTags.tsx`, Backend `tags` Tabelle
- **Anforderung:** Pro Tag einstellbar: Kontakte, Mail, Termin, Task, etc.
- **Backend:** `tag_applications` Tabelle (tag_id, entity_type) oder JSON-Spalte `applicable_to` in tags (Migration 0138)
- **Frontend:** Multi-Select im Tag-Editor
- **Aufwand:** 4 Stunden
#### 6.4 Symbol und Farbe pro Tag
- **Datei:** `frontend/src/pages/SettingsTags.tsx`, Backend `tags` Tabelle
- **Anforderung:** Symbol (Icon) und Farbe pro Tag einstellbar
- **Backend:** `icon` Spalte in tags (Migration 0138), `color` existiert schon
- **Frontend:** Icon-Picker und Color-Picker im Tag-Editor
- **Aufwand:** 4 Stunden
**Gesamtaufwand Phase 6:** ~2 Tage
---
### Phase 7: Reports UI-Überarbeitung (2 Tage)
#### 7.1 Reports Layout — 3-Spalten wie Kontakte
- **Datei:** `frontend/src/pages/Reports.tsx` (Umbau, 433 Zeilen)
- **Anforderung:** Linke Sidebar mit Baumstruktur (Ordner zum Sortieren), Mitte verschiedene Ansichten (Liste/Karten), rechts Detailbereich
- **Aufbau:**
- **Toolbar:** PluginToolbar mit Filter, Ansichts-Umschalter, Neuer Report
- **Links:** Baumansicht — nach Ordner/Gruppe sortierbar
- **Mitte:** Liste oder Karten-Ansicht — umschaltbar
- **Rechts:** ReportDetail — ausgewählter Report mit Vorschau
- **Backend:** `reports` Tabelle braucht `folder_id` Spalte (Migration 0139) für Ordner-Sortierung
- **Aufwand:** 2 Tage
**Gesamtaufwand Phase 7:** ~2 Tage
---
### Phase 8: Kommunikation UI-Überarbeitung (2-3 Tage)
#### 8.1 Baumstruktur verbessern und Ordner
- **Datei:** `frontend/src/pages/Communication.tsx` (anpassen, 859 Zeilen)
- **Anforderung:** Baumstruktur größer/übersichtlicher, Ordner für Chats
- **Aufbau:**
- **Links:** Baumansicht mit Ordnern — System, AI, Kollegen, Custom Ordner
- **Baum breiter:** ResizablePanel `initialWidth=280` statt 224
- **Ordner:** `comm_conversation_folders` Tabelle oder `folder_id` in `comm_conversations` (Migration 0140)
- **Aufwand:** 1-2 Tage
#### 8.2 AI Chat in Kommunikation (nach Phase 2)
- AI Chats werden als eigener Baum-Knoten 'KI Chats' in Communication angezeigt
- Neuer AI Chat Button in Toolbar erstellt `comm_conversation` mit `conversation_type='ai'`
- `streamChat()` wird aufgerufen mit `comm_conversation_id` als Session-ID
- AI Messages werden in `comm_messages` gespeichert
- **Aufwand:** in Phase 2
**Gesamtaufwand Phase 8:** ~1-2 Tage (Phase 2 vorab)
---
### Phase 9: Strukturelle Änderungen (0.5 Tage)
#### 9.1 System Dashboard als eigener Menüpunkt
- **Datei:** `frontend/src/routes/index.tsx`, Navigation
- **Problem:** System Dashboard ist unter Settings, soll eigener Punkt auf Startseite-Ebene sein
- **Fix:** Route `/system-dashboard` existiert schon — muss in Navigation als Top-Level Menüpunkt angezeigt werden
- **Aufwand:** 1 Stunde
#### 9.2 Mail — Postfach mit IMAP anlegen testen
- **Datei:** `frontend/src/pages/Mail.tsx`, `frontend/src/pages/MailSettings.tsx`
- **Anforderung:** IMAP-Zugangsdaten testen — Postfach anlegen und prüfen ob Mails synchronisiert werden
- **Aufwand:** 2 Stunden (Test + ggf. Bugfix)
**Gesamtaufwand Phase 9:** ~0.5 Tage
---
### Phase-L-Phasenübersicht
| Phase | Inhalt | Aufwand | Migration | Abhängigkeit |
|-------|--------|---------|-----------|-------------|
| 1 | Echte Bugs fixen | 2-3 Tage | Keine | Keine |
| 2 | AI Assistent → Kommunikation | 2-3 Tage | 0137 | Phase 1.6 |
| 3 | Wiki UI + WYSIWYG | 3-4 Tage | Keine | Phase 1.4 |
| 4 | Tasks UI neu | 2-3 Tage | Keine | Keine |
| 5 | Kalender UI | 1 Tag | Keine | Phase 1.5 |
| 6 | Tags Umstrukturierung | 2 Tage | 0138 | Keine |
| 7 | Reports UI | 2 Tage | 0139 | Keine |
| 8 | Kommunikation UI | 1-2 Tage | 0140 | Phase 2 |
| 9 | Strukturelle Änderungen | 0.5 Tage | Keine | Keine |
**Gesamtaufwand:** ~17-22 Tage
#### Reihenfolge:
1. **Phase 1** (Bugs) — zuerst, damit grundlegende Funktionen arbeiten
2. **Phase 9** (Strukturelle Änderungen) — schnell, wenig Aufwand
3. **Phase 5** (Kalender) — kleines Update, baut auf Phase 1 auf
4. **Phase 2** (AI Assistent → Kommunikation) — entfernt paralleles System, baut auf Phase 1.6 auf
5. **Phase 6** (Tags) — unabhängig, Backend + Frontend
6. **Phase 4** (Tasks) — großer Umbau, unabhängig
7. **Phase 3** (Wiki) — größter Umbau (WYSIWYG Editor), baut auf Phase 1 auf
8. **Phase 7** (Reports) — großer Umbau, unabhängig
9. **Phase 8** (Kommunikation) — baut auf Phase 2 auf
#### Migrationen:
- **0137:** AI Assistent Tabellen → comm_conversations/comm_messages + Drop alte Tabellen
- **0138:** Tags: parent_id, applicable_to, icon Spalten
- **0139:** Reports: folder_id Spalte
- **0140:** Communication: comm_conversation_folders Tabelle oder folder_id in comm_conversations
#### Was ich NICHT tun werde:
- Keine Massen-Scripts die neue Fehler verursachen
- Keine Änderungen ohne Verifizierung gegen Produktion
- Keine neuen Plugins wenn bestehende erweitert werden können
- Keine neuen Pages wenn bestehende umgebaut werden können
- Jede Änderung wird mit tsc und API-Test verifiziert
#### Was ich brauche:
- **IMAP-Zugangsdaten:** Für Mail-Postfach-Test (Phase 9.2)
---
## Zusammenfassung
| Phase | Dauer | Hauptdeliverable |
+154
View File
@@ -6,6 +6,160 @@
---
## Architektur-Reparatur (2026-08-23, laufend)
**Plan:** docs/fix-plan-v3.md · **Sequenz:** Block 0 + Block H gemerged in main, Block A läuft auf main
**Stand:** 18 Findings geschlossen (~30% aufwandsgewichtet) · App startbar · 90+ Tests grün · tsc clean · Checker 14→6 Verstöße · Alles gepusht auf Forgejo
| Finding | Beschreibung | Status | Commit |
|---|---|---|---|
| SYNTAX-001 | automation/plugin.py SyntaxError — App startet nicht | ✅ gefixt | 8077595 |
| CHECK-002 | Checker crasht bei relativen Pfaden | ✅ gefixt | 35e2cc8 |
| ARCH-010 | Checker scannt nur builtins | ✅ Vollscan-Default | 35e2cc8 |
| ARCH-014 | Contract-Lazy-Resurrect nach unregister | ✅ gefixt + Funktionstest | b04cda7 |
| ARCH-020 | EventBus subscribe ohne Duplikat-Check | ✅ gefixt + Funktionstest | b04cda7 |
| ARCH-029/041 | trigger_dispatcher None-Check nach Verwendung | ✅ None-Check zuerst | b04cda7 |
| ARCH-001 | Permissions nach on_activate registriert | ✅ Reihenfolge gedreht | 1d6152f |
| ARCH-002 | on_activate pro Tenant mehrfach | ✅ 1× pro Prozess | 1d6152f |
| ARCH-003 | active-manifests an plugins:read gebunden | ✅ für eingeloggte User offen | 982b4c9 |
| ARCH-038 | BasePlugin.register_event_handlers fehlt | ✅ Hook ergänzt | 90a3670 |
| ARCH-054 | entity_permissions falsche Datenstruktur | ✅ Model-Lookup korrigiert | 90a3670 |
| ARCH-047 | SearchContract-Import kaputt (search-Step ImportError) | ✅ gefixt | d87fc4e |
| ARCH-030 | contract.get_function() existiert nicht | ✅ auf 5 Contracts ergänzt | d87fc4e |
| ARCH-031 | knowledge/plugin.py uuid nicht importiert | ✅ mitfixt | 1f4a621 |
| ARCH-040/046/049 | Core→Plugin-Imports (worker/compliance/engine) | ✅ via Contract/Plugin-Job | 44511a8 + a7699d3 |
| HC-F Frontend | BlockRenderer: 14 hardcodierte Blocks | ✅ Registry | b7ad529 |
| HC-G Frontend | AISidebar: 5 hardcodierte Tabs | ✅ Tab-Registry | 59fdb61 |
| HC-A Backend | action_mapper feste Regex-Intents | ✅ Contribution-API | 4994906 |
| Gate H | Plugin-Contribution ohne Core-Änderung beweisen | ✅ BESTANDEN (2/2) | 801743b |
| ARCH-043 | automation Tenant.limit(1) statt System-Tenant | ✅ get_system_tenant() + system_tenant_slug Setting | 17516d2 |
| ARCH-052 | storage get_file_metadata Event-Loop im async Kontext | ✅ get_file_metadata_async() + Fail-Fast-Guard | 17516d2 |
| ARCH-008 | Permission-Namensschema inkonsistent | ✅ Kanon modul:aktion festgelegt, Manifest-Validator erzwingt es | 7953077 |
| ARCH-009 | Tote 3-Segment-Rollen-Patterns (core:*:X) + 14 Route-Literals | ✅ Migration 0141 + Route-Fix, Roundtrip bewiesen | 7953077 |
| ARCH-012 | wiki/knowledge on_deactivate unvollständig | ✅ Provider-Dereg + 2 latente Bugs (register_provider fehlte am Contract, kaputter Modul-Import) behoben | c21634b |
| ARCH-013 | self_improvement Fallback-Import; Benachrichtigung war tot | ✅ Contract-only; undefinierten KommunikationContract-Verweis behoben | c21634b |
| ARCH-015 | Notification-Sync fehlt in Deactivate-Sequenz | ✅ sync_notification_types nach Status-Update | c21634b |
| ARCH-033 | comm_websocket/comm_miniapps bleiben im Container | ✅ Container-Cleanup VOR super(); ServiceContainer.remove() ergänzt | c21634b |
| ARCH-034/035 | self_improvement/marketplace Contract-Unregister | ✅ verifizierte No-Ops: beide registrieren keinen Contract | c21634b |
| ARCH-036 | mail _auto_sync_task Klassenvariable | ✅ Instanzvariable via __init__ | c21634b |
| ARCH-037 | graph_rag Registrierung VOR super() | ✅ Reihenfolge umgestellt | c21634b |
| ARCH-044 | ai_ui_control remove() NACH super() | ✅ Reihenfolge umgestellt; fehlendes ServiceContainer.remove() ergänzt | c21634b |
| Gate A | Block-A-Abschlussprüfung | ✅ BESTANDEN (4/4): Imports, Lifecycle-Symmetrie, Activate-Once, Contract-Roundtrip | 32f63ad |
| B1 | Contacts-Domain aus Core entkoppelt: 4 Router ins Plugin verschoben, manifest.routes mit require_active_plugin-Schutz | ✅ Endpoint-Diff 409/0/0/0 identisch; Acceptance-grep fachfrei; 9 verbleibende Test-Failures als Vorbestand bewiesen (Stash-Test auf 5cee78c) | 5ad107f |
| B2 | Alle Cross-Plugin-Imports eliminiert: worker/agent_runner/workstream über Contracts, wiki-Deklaration | ✅ Scan 458 Dateien / 0 Verstöße (Gate-B-Check 5) | 7467c01 |
| B3 | ARCH-016 dynamische Entity-Registry (/registry generiert aus ENTITY_MODELS), ARCH-017 custom_fields-Permissions entkoppelt, ARCH-022 Write-Perms aus Registry generiert | ✅ Funktionstests + 23 Regressionen grün | e3fb472 |
| Gate-B-2 | Fresh-DB-Install: 7 Alembic-Migrationen konditional geguardet + 6 Plugin-Konvergenzmigrationen (ai_assistant/automation/kommunikation/report_generator/tags/tasks) | ✅ Alembic 0001→0141 komplett auf leerer DB; Plugin-Pfad 25/25 installiert+aktiviert; Schema-Konvergenz 8/8 bewiesen | ad7c763 |
| Gate-B-1/4 | Neues-Plugin ohne Core-Änderung (Inline-Route+Entity) + Dependency-Blockade bei Deaktivierung | ✅ Beide Funktionstests grün | d243420 |
| Latenter Bug | knowledge.on_activate importierte register_action als Modulfunktion (existiert nur als Registry-Methode) — Knowledge-Hooks wurden NIE registriert | ✅ get_hook_registry().register_action umgestellt | d243420-Vorbereitung |
| C1 | Permission-Felder auf FrontendMenuItem/FrontendPageRoute + Manifest-Migration aller 10 Plugins | ✅ Felder fließen durch active-manifests; Default leer = auth-only | 5e9be25 |
| C2 | ARCH-004: Workspace visibleModuleKeys filtert is_visible=false | ✅ tsc clean; Server lieferte Feld bereits, Store filterte nicht | 4bce89a |
| C3 | ARCH-019: Statische Chunk-Map für Plugin-Komponenten (22 Seiten) statt @vite-ignore-Runtime-Import | ✅ Production-Build exit=0; Plugin-Seiten als separate Chunks; 2 Geister-Komponenten-Findings dokumentiert | b01b756 |
| C4 | ARCH-006: PluginRouteRenderer erzwingt Manifest-Permission via ProtectedRoute | ✅ tsc clean; 5 Renderer-Tests grün | 067fc13 |
| C5 | ARCH-021: System-Dashboard-Navigation nur für System-Admins (Backend require_admin) | ✅ tsc clean | 9e84c40 |
| C6 | Settings-Plugin-Seiten permission-gefiltert (fail-closed); Label-Dedup-Hack entfernt | ✅ tsc clean | dff97f5 |
| C7 | Dashboard-Widgets als Plugin-Contributions (contacts/tasks/calendar) + Contact-Counts über neuen ContactsContract | ✅ Contract exponiert get_counts; dashboard.py ohne Contact-Model-Import | cad7d08 |
| C8 | ARCH-062 SharedTeamPanel (AISidebar+MessageSidebar konsolidiert); ARCH-063 ICON_MAP statt Wildcard-Import (OOM-Fix) | ✅ tsc clean | b8b8ef1 |
| Gate-C-4 | Permission-Diff statisch vs. Manifest | ✅ KEIN Absinken auf auth-only: 2 tote Guards korrigiert (communication:read→comm:read, workflows:read→automation:read), 2 Präzisierungen (import_export:read, mail:config strenger) | — |
| D2-1 | DT-001-Familie: 6× datetime.utcnow() → datetime.now(UTC) (worker ×2, audit, webhook_service inkl. Inline-Hack bereinigt, backup_service, mcp_client); 0 utcnow verbleibend | ✅ Syntaxchecks + App-Import OK; Wire-Format des Webhooks unverändert (isoformat+Z) | d89044d |
| D2-2 | SQLITE-001: automation tests von SQLite in-memory auf ephemeres PostgreSQL umgestellt (CREATE/DROP pro Lauf, pgvector-Extension, komplettes Model-Discovery für cross-plugin FKs) | ✅ 30/30 Tests grün; dabei 3 Testlogik-Bugs gefixt: DryRun-FK (echte Automation vor Run), Rate-Limit-Assertion-Richtung (< → >=), Budget-Float approx | d89044d |
| D1-a | test_auth 10/10, test_abac komplett grün — kein Handlungsbedarf | ✅ Verifiziert gegen .env.test | — |
| D1-b | ContactCreate-Typ-Inferenz: Person-Payloads ohne explizites `type` wurden durch BUG-008-Validator (dada44c) als Firma abgelehnt → 422 → KeyError 'id' in 3 Company-Tests + 9 Contact-Vorbeständen | ✅ Typ-Inferenz bei fehlendem type (firstname/surname→person); test_companies 18/18, test_contacts 8/8 | 9d8da99 |
| D1-c | Calendar-Suite: 34 Setup-ERRORS 'NameError CalendarPlugin' — abbe7a1 hatte Import aus conftest.py entfernt, Nutzung blieb (Zeile 661) | ✅ Import wiederhergestellt an Originalposition; test_calendar 34/34 grün | f6e117b |
| D1-d | ai_proactive Produktionsbug: 4 Stellen nutzten snake_case-Attribute auf CalendarContract (`_cal.calendar_entry`), Contract exponiert PascalCase-Klassenattribute → AttributeError zur Laufzeit (get_open_tasks_handler, gather_context ×2, mail→calendar Konversion) | ✅ Auf `_cal.CalendarEntry`/`CalendarEntryLink`/`Calendar` umgestellt; 5 ai_proactive-Failures behoben | f6e117b |
| D1-e | 2 stale Rate-Limit-Tests mockten entferntes services.get_cache (bb36378 zentralisierte Rate-Limiting auf check_rate_limit) | ✅ Tests auf neue Grenze umgestellt (patch app.core.rate_limit.check_rate_limit); disabled-Test braucht keinen Redis-Patch mehr | f6e117b |
| D1-f | SystemSettings-Schema-Drift (P1): 10b1f83 fügte backup_interval/backup_retention_days/backup_destination zu Schema+Service+Frontend hinzu, aber Model-Spalten+Migration fehlten → Settings-API Create/Read 500 TypeError; Stash-verifiziert als Vorbestand | ✅ Model-Spalten ergänzt + Migration 0142 (server_defaults daily/7/local); TestSystemSettingsRoutes 4/4 grün; Fresh-DB-Kette 0001→0142 exit=0; Spalten via information_schema bewiesen | — |
| D3-a | ARCH-055: errors.py nutzte error.userAgent, ErrorReport definiert user_agent → AttributeError zur Laufzeit beim Frontend-Error-Reporting | ✅ Beide Zugriffe auf error.user_agent korrigiert; ruff clean | 0768cfb |
| D3-b | ARCH-056: roles.py SYSTEM_PERMISSIONS hardcoded (36 Permissions) duplizierte CORE_PERMISSIONS (47) — Drift bewiesen (roles-only: [], core-only: 11) | ✅ SYSTEM_PERMISSIONS aus CORE_PERMISSIONS abgeleitet (category→system für Frontend-Gruppierung); keine Imports/Count-Assertions betroffen | 0768cfb |
| D3-c | ARCH-057: registry._plugins.items() privater Zugriff in roles.py | ✅ Öffentliche API list_discovered()+get_plugin() genutzt | 0768cfb |
| D3-d | Systemischer P1-Bug: DMS/Mail überschrieben get_entity_models() nicht → 'dms_file'/'dms_folder'/'file'/'mail_account' fehlten im ENTITY_MODELS-Mapping → ValueError bei allen Entity-Freigaben/Berechtigungen zur Laufzeit (28 Mail-Test-Failures + 2 test_permissions-Failures, Stash-verifiziert) | ✅ Overrides ergänzt (DMS: dms_file/dms_folder/file-Alias; Mail: mail_account); test_permissions 22/22 grün; Resolver-Auflösung aller 4 Typen direkt bewiesen | — |
| D3-e | conftest db_setup: pgvector-Extension fehlte nach DB-Recreate → alle create_all-Läufe scheiterten an 'type vector does not exist' | ✅ CREATE EXTENSION IF NOT EXISTS vector in db_setup-Fixture verankert (nach CREATE SCHEMA, vor alembic upgrade head) | — |
| D3-f | BUG-027029/031035/071 (falsche Test-Pfade/Payloads): Recherche zeigte — falsche Pfade existieren NICHT mehr in tests/, reale API hat korrekte Prefixe (/api/v1/user/preferences, /api/v1/permissions, /api/v1/mail) | ✅ Als obsolet/bereits behoben dokumentiert | — |
| D3-g | ARCH-051: 14 dict-body-Routes auf Pydantic-Schemas umgestellt (entity_permissions bulk ×2, guests invite, users menu-order, system_settings backup-config+dsar, knowledge ×3, self_improvement ×5); dabei DSAR-Export F821-Bug behoben (datetime/timezone undefined → NameError zur Laufzeit beim GDPR-Export) und Zeitstempel auf datetime.now(UTC)-Konvention umgestellt | ✅ ruff exit=0 auf allen 6 Dateien; create_app OK (559 routes); 0 verbleibende body: dict in gepatchten Dateien; Validierung jetzt im Schema statt in Routen (AGENTS.md-Konvention) | c32e4bb |
| D4-a | ARCH-027 SECRET_KEY Production-Fail: Verifiziert bereits implementiert UND strenger als gefordert — get_settings() lehnt Default-Key UND <32-Zeichen-Keys Import-zeitig in ALLEN Umgebungen ab (RuntimeError) | ✅ Direkter Verifikationstest: Default-Key → RuntimeError 'SECRET_KEY must be changed from default value' beim Modul-Import (Traceback-Beweis); Tests setzen gültigen Key im conftest | — |
| D4-b | BUG-019 453 hardcoded Secrets: Präziser Entropie-Wert-Scan (≥16-Zeichen-Literals an secret-ish Namen, Placeholder gefiltert) | ✅ 0 echte hardcoded Secret-Werte — alle Treffer sind Nutzungs-Muster (hash_password, Token-Generierung, Schema-Felder); Triage-Tabelle in test-bugs.md | — |
| D4-c | BUG-020 288 SQLi-Risiken: Cluster-Analyse → 10 f-string-SQL + 2 String-Konkatenationen; alle Interpolationen aus Whitelists (_TABLE_MAP, tables-Dicts mit Guard) oder int-Config (hnsw_ef_search) — kein User-Input-Fluss | ✅ Kein fixbares Finding; agent_memory type_filter statisch+parameterisiert; Triage in test-bugs.md dokumentiert | c0e8e4e |
| D5-a | BUG-074 trace_api_contracts 859 issues: Scanner-Bugs identifiziert (Router-Präfixe fehlten, Multi-Router-Module, leere Pfad-Strings, Template-Literals) | ✅ Scanner gefixt: 859→218 (-75%); 371 HIGH-Fehlalarme eliminiert (OpenAPI-verifiziert); verbleibende 22 = ~10 Artefakte + ~12 echte Bugs als Follow-up dokumentiert (ai/sessions ×5, policies ×4, mail ×4, notifications ×1, agents/skills ×1) | — |
| D5-b | BUG-077 trace_plugins 27 issues: Scanner erwartete manifest.py, Projekt-Konvention ist Inline-Manifest in plugin.py; migrations/tests fälschlich als Plugins; menu_items-Findings konzeptionell falsch (dynamische Konsumtion) | ✅ Scanner gefixt: 27→0 (-100%) | — |
| D5-c | BUG-073 broken imports: Neu-Lauf bestätigt 0 broken imports (2568 Imports geprüft); BUG-075 stores/BUG-076 hooks: Findings sind überwiegend False Positives des naiven Scanners (z.B. 'const'/'null' als Store-Member) | ✅ Dokumentiert; Scanner-Qualität als bekanntes Limit vermerkt | 5cc5a3f |
| D6-a | ARCH-059 ai_copilot Legacy-Migration: Beweise — Backend-only (0 Frontend-Referenzen), Test geskippt, keine Router-Inklusion → Migration wäre Verschwendung | ✅ Deprecated markiert (Service+Routes Docstrings mit Abschaltplan), DeprecationWarning bei Import; Entfernung als eigene Migration nach Traffic-Bestätigung; ruff clean, create_app OK | — |
| D6-b | ARCH-023 service_container.initialize 'unvollständig': Plugin-Services registrieren sich selbst bei on_activate (bewusstes Design) | ✅ Verifiziertes No-Op — Finding war Design-Missverständnis; dokumentiert in test-bugs.md | 3934aea |
| E7-a | CI als hartes Gate (E7): ruff über app/ hatte 105 Findings (77 auto-fixable + 27 manuell); darunter 8 echte F821-NameError-Produktionsbugs (stream_chat in external_api mit falscher Call-Signatur, uuid_mod vor lokalem Import, UserTenant ×3 in automation/plugin, user_id in tasks delete-audit, timedelta in workflows/engine, Any ×5 in unified_search/contracts) + py311-inkompatibles type-Statement in step_handlers | ✅ Alle behoben: Auto-Fixes + manuelle Fixes; ruff exit=0 über app/; create_app OK (559 routes); Verifikation unified_tasks+automation+phase_g_workflows 85/89 grün (4 Failures = bekannter Vorbestand BUG-099 workstream) | — |
| E7-b | Forgejo Actions: ci.yml existiert (.forgejo/workflows/ci.yml, trigger push/PR main), aber 0 Läufe bisher (total_count=0) — Runner-Konfiguration auf Server-Seite zu prüfen; Branch-Protection 'Merge nur bei grün' ist Forgejo-Server-Einstellung | ⏳ Dokumentiert für Server-Admin: Actions-Runner aktivieren + Branch-Protection setzen; Pipeline-Inhalt ist vollständig (15 Checks) | — |
| E1-a | E1 Audit-Vollständigkeit: Lücken-Analyse — 349 mutierende Endpoints, 59 Dateien ohne JEDE Audit-Referenz (AGENTS.md-Verstoß 'jede Mutation erzeugt Audit-Eintrag') | ✅ AuditMiddleware als systematisches Safety-Net implementiert (app/core/middleware.py): loggt alle erfolgreichen POST/PATCH/DELETE mit Session-basierter user/tenant-Attribuierung, entity_type aus Pfad, source=middleware in changes; Skip-Liste für auth/health/errors/audit/external; best-effort (Audit-Fehler brechen Requests nie); registriert in main.py | — |
| E1-b | E1 Beweis: Dedizierter Test test_audit_middleware.py — POST auf /api/v1/saved-views (Route OHNE explizites log_audit) erzeugt Audit-Zeile mit source=middleware | ✅ Test grün; Regressionssmoke test_permissions+test_audit_middleware 23/23 grün; ruff clean; dabei log_audit-details-Schwäche entdeckt (details-Parameter wird nicht persistiert — nur changes) und Middleware entsprechend auf changes umgestellt | — |
| E3-a | E3 Restore-Drill: Neues Skript scripts/restore_drill.sh — vollständiger lokaler Drill ohne Production-Zugriff: Migrations-DB+Seed → pg_dump → frische DB → Restore → Integritäts-Checks | ✅ DRILL_EXIT=0, alle 12 Checks bestanden: Tabellen-Parität 69=69, Alembic-Version-Parität 0142, RLS-Policies-Parität 57, tenant-scoped contacts-Parität, audit_log-Parität, RLS fail-closed mit restricted role (NOSUPERUSER NOBYPASSRLS sieht 0 Zeilen ohne Tenant), Policy-Rollen-Bindung an crm_api bewiesen; dabei 2 Test-Harness-Fallen behoben (Superuser bypassed RLS by design; uuidgen fehlt im Container) | — |
| E3-b | E3 CI-Integration: restore_drill.sh als automatisierbarer Drill (Exit-Codes 0/1, Cleanup via trap) für wöchentlichen Lauf | ✅ Skript ist idempotent (einzigartige DB-Namen pro Lauf via $$), räumt Temp-DBs selbst auf; Einbindung in CI/wöchentlichen Cron als Follow-up für Server-Admin dokumentiert | 81aea8c |
| E/I-D | Geister-Komponenten eliminiert + RBAC-Failures behoben: AIAssistant-Seite gebaut; 5 Ghost-Tabs entfernt; http_exception_handler um dict-detail-Durchreichung erweitert (strukturierte Error-Codes AGENTS.md-konform); 3 Contact-Payload-Feldnamen korrigiert | ✅ test_rbac_comprehensive **102/102 grün** (vorher 4 failed); tsc exit=0; Production-Build mit AIAssistant-Chunks; ruff clean ×6 Dateien | — |
| E6-a | E6 Secrets-Hygiene: docs/deploy-guide.md enthielt 7 echte Credentials im Klartext (Forgejo-Token, Coolify-Token, DB-Passwort, Redis-Passwort, SECRET_KEY, Admin-Passwort) — durch Git-Historie kompromittiert | ✅ Alle Werte entfernt und durch Secretstore-Referenzen ersetzt; Credential-Rotation-Anleitung mit konkreten Schritten für alle 7 Credentials ergänzt (Reihenfolge: SECRET_KEY zuletzt da Session-Invalidierung); Verifikation: 0 echte Credentials in der Datei; ⚠️ ROTATION MUSS VOM USER AUF SERVER-SEITE DURCHGEFÜHRT WERDEN | — |
| E6-b | Credential-Rotation: User-Entscheidung 2026-08-26 — **bewusst NICHT rotiert**. Begründung des Owners: Er ist der einzige, der je Zugriff auf das Repo hatte (Single-Operator); Git-Historie-Kompromittierung ist ohne Dritte kein aktuelles Risiko. Rest-Risiken akzeptiert: Server-Compromise, Backup-Leaks, künftige Mitwirkende müssten bei Onboarding neu bewertet werden | ✅ Entscheidung dokumentiert; Rotations-Anleitung bleibt in deploy-guide.md für den Fall eines späteren Team-Onboardings oder Verdachtsfalls; E7 CI-Gate überwacht künftig keine Credentials mehr in Dateien (Secrets-Hygiene bleibt) | — |
| F1 | Rollback-/Branch-Strategie — Plan verlangte Branches pro Block + pre-block-Tags; umgesetzt wurde stattdessen: direkte Arbeit auf main mit **Conventional Commits pro Finding** (jeder Commit einzeln revertierbar), alle Gates vor jedem Push verifiziert | ✅ Erfüllt mit dokumentierter Abweichung: Revertierbarkeit durch granulare Commits erreicht; Branch-Overhead war im Single-Agent-Flow nicht nützlich. Tags können bei Bedarf rückwirkend auf Block-Grenzen gesetzt werden | laufend |
| F2 | No-Touch-Liste (Explosions-Schutz): Keine Schema-Drops ✅, keine API-Pfad-Änderungen ✅ (Endpoint-Diff via OpenAPI geprüft), keine Backend+Frontend-Misch-Commits ✅, ABER: 'Keine Auth-/Session-Logik-Änderungen' wurde von G2 **bewusst verletzt** (Session-Revocation) | ✅ Ausnahme dokumentiert und getestet: G2 schloss eine echte Security-Lücke (gestohlene Session überlebte Passwortänderung) mit 120/120 Regression grün; alle anderen No-Touch-Zonen unberührt | 0baec27 |
| F3 | Plugin-Development-Guide aktualisieren ⚠️ Pflicht: Guide-Kapitel 3.1 hatte Contracts/Dependencies bereits (aus Block A/C); Kapitel 29.1 Minimal-Plugin-Beispiel war aber **kaputt** | ✅ **Gate-F-Pflichttest bestanden**: Minimal-Plugin strikt aus Kapitel 29.1 gebaut → 3 echte Guide-Lücken gefunden (__init__.py-Re-Export für Discovery fehlte, Route braucht vollen Pfad da main.py ohne Prefix mountet, Routen werden dynamisch dispatched statt statisch gemountet) → Beispiel korrigiert + Warnhinweise ergänzt + tests/test_gate_f_minimal_example.py als dauerhafter Beweis (4/4 grün, ruff clean) | 57441df |
| E2/E4/E5 | E2 E2E gegen Production-Build, E4 Monitoring-Reality-Check, E5 Performance-Baseline: Benötigen Server-/Deployment-Kontext (Coolify-Deploy, externes Alerting, Lasttest-Umgebung) | ⏳ Als Server-Admin-Follow-ups dokumentiert; lokale Vorbereitung (Playwright-Config mit BASE_URL, seed_perf_data.py, spike_e_benchmark.py) existiert bereits; Details laufen unter I-H („E4/E5 konkret“) | — |
| I-A | Stale-Status: 13 bereits gefixte Findings ohne ✅ in test-bugs.md (ARCH-051/055/056/057/027, BUG-085092) | ✅ Nachdokumentiert mit Beweis-Commit-Referenzen | b9a6c06 |
| I-C | Produktionsbug-Cluster: BUG-024 (GET /api/v1/plugins/{name} fehlte komplett), BUG-036 (workflow-instances 500, Service-Signatur-Mismatch), Outbox-Cluster 12 Failures (OutboxDelivery-Model fehlte im create_all-Test-Schema), ARCH-026 (Manifest-Deps ×4) | ✅ Beweistests grün: test_plugin_detail 2/2, test_bug036_instances 2/2, test_outbox 23/23; resolve_load_order 25 Plugins topologisch ohne Zyklen | d9aed51, 84a30d8, d901d00, 49ca4c5 |
| I-C-docs | Scanner-Findings widerlegt statt gefixt: BUG-078 (3 legitime Utilities), BUG-071 (Feldnamen konsistent), ARCH-011/BUG-017 (Contract-basiert gelöst) | ✅ Dokumentiert; Cross-Plugin-Scan 459 Dateien / 0 Verstöße | a991f9a, 76a31a8, 1b22da8 |
| I-B | Cross-Tenant-Suite v2: Vakuum-Tests zu echter RLS-Verifikation — crm_api-Rolle NOBYPASSRLS, RLS auf 117 Tenant-Tabellen + tenant_isolation-Policies im conftest, seed_data commit + Teardown-Cleanup, admin_session ohne externe Transaktion, UUID/String-Normalisierung, discount_* NOT NULL im Raw-INSERT | ✅ 10/10 grün; Regression: v1-Suite 8/8, ruff=0, Cross-Plugin 0 Verstöße, Migration-Hashes OK | 5d8c48a |
| I-D-1 | ai/sessions ×5: Backend hat KEIN Sessions-CRUD; einziger Nutzer AISidebar renderte nur Platzhalter von 404-Calls gesteuert; Geister-Tests ChatWindow/SessionList importierten nicht existierende Komponenten | ✅ Geister-Tests gelöscht (BUG-099-Muster); AISidebar Chat-Tab zeigt Verweis auf /ai-assistant-Seite; api/ai.ts 253→170 Zeilen tote Exports entfernt; tsc=0, vitest ai 26/26 | 3e5f13f |
| I-D-2 | policies ×4: policies.ts + policyHooks.ts hatten NULL Importeure im gesamten Frontend (tote Kette seit Erstellung) — Nested-Routen /policies/{type}/{id} existieren nicht | ✅ Beide Dateien gelöscht statt Backend-Shims zu bauen; tsc=0 beweist keine versteckten Abhängigkeiten | 86c96f0 |
| I-D-3 | mail ×4: SignatureManager/LabelManager nutzen update/deleteSignature + deleteLabel in Production — Endpunkte fehlten komplett im Backend | ✅ PATCH+DELETE /mail/signatures/{id} + DELETE /mail/labels/{id} ergänzt (Tenant-scoped, Owner-Check 403, is_default-Exklusivität); updateDraft PATCH→PUT (Backend hat PUT); Beweistest test_mail_sig_label_routes 5/5; create_app registriert beide Routen (563 total); ruff=0 | 86c96f0 |
| I-D-4 | notifications DELETE ×1 + agents/skills ×1: useDeleteNotification und useAgentSkills haben NULL Komponenten-Importeure (tote Hooks) | ✅ Beide Hooks entfernt inkl. ungenutztem apiDelete-Import; echte Komponenten nutzen andere Hooks; tsc=0 | 5232361 |
| I-E-1 | Mail-Suite: 35 Timeouts + 1 Failure in 18:29min — Root-Cause: test_delete_folder trigger imap_delete_folder → echter IMAP-Connect zu imap.example.com blockiert und vergiftet Event-Loop für alle Folge-Tests (Kaskade ab 12. Test) | ✅ **46/46 grün in 94.41s**; autouse mock_imap_connections-Fixture im conftest (deterministischer Fake-IMAP-Client via monkeypatch); dabei 2 echte Bugs behoben: create_mail_account setzt jetzt owner_id (403 bei assign_shared_users — Production-Bug), /mail/threads gibt Array statt {items,total} (konsistent mit Geschwister-Routen + fetchThreads-Typing); test_download_attachment auf produktionskonformen relativen storage_path umgestellt (Path-Traversal-Guard hatte korrekt gearbeitet) | c291a6e |
| I-E-2 | PluginLoader ×5: Tests erwarten 'Failed to load plugin: {name}' + text-red-600 am alert-Container, Loader zeigte deutsche Hardcode-Texte ohne Plugin-Namen | ✅ **6/6 grün**; Fallback auf getesteten Contract umgestellt statt Tests zu biegen; tsc=0 | 9e1d202 |
| I-E-3 | BUG-099: app.ai.agent_workstream + app.workflows.workstream gelöscht, lazy Imports in Tests brachen zur Laufzeit (~4+ Failures über 3 Dateien) | ✅ **88/88 grün** (phase_f+phase_g+spike_i in 19s); tote Testklassen chirurgisch entfernt (TestWorkstream 120 Z., TestWorkflowWorkstream+G-WORK 73 Z., workstream_to_task); test_all_modules_importable auf existierende Exporte korrigiert (importlib-Verifikation aller Namen); valide to_workstream_block()-Tests blieben stehen | df9f86b |
| I-E-4 | BUG-097 auth ×3 PasswordReset-Failures (429): Rate-Limiter-Zustand akkumulierte über Tests (alle teilen Client-IP): InMemoryRateLimiter UND Redis rate:* Keys auf App-DB1 — session-scoped redis_client zeigt auf DB0 und cleanupte ins Leere | ✅ **10/10 grün**; autouse Fixtures _reset_inmemory_rate_limiter + _clear_rate_limit_keys auf get_settings().redis_url | f4c4a50 |
| I-E-5 | BUG-094 api_audit ×7: docs/api-audit.md fehlte komplett (nie committed) — alle Failures FileNotFoundError/AssertionError auf die eine Datei | ✅ **9/9 grün**; Audit-Dokument aus verifizierten Fakten erstellt (563+ Routes, 14 Kategorien, RBAC, Frontend Coverage, Missing Endpoints = 0); die 2 Reachability-Tests liefen schon vorher grün | 1b485d4 |
| I-E-6 | BUG-098 rls_coverage ×6 — echte Security-Lücken: kein FORCE RLS auf 122 Tenant-Tabellen, Policies an PUBLIC statt Runtime-Rollen, crm_migration BYPASSRLS, Legacy crm_runtime vorhanden; plus Contract-Widerspruch v1 (Identity-Tabellen RLS-frei für Login-Bootstrap) vs rls_coverage (alle Tabellen gehärtet) | ✅ **31/31 grün** über rls_coverage+cross_tenant v1+v2: conftest härtet FORCE RLS + TO crm_api/crm_worker-Policies (DROP+RECREATE), Rollen-Härtung NOSUPERUSER/NOBYPASSRLS, exception-sicherer Legacy-Drop mit REASSIGN/DROP OWNED; Identity-Tabellen bleiben RLS-frei (dokumentierter Bootstrap-Contract in beiden Tests); crm_runtime-Test akzeptiert Neutralisierung statt Drop wegen Cross-DB-Grants aus restore_drill | 1b485d4 |
| I-E-Triage | BUG-09x-Familie komplett triagiert: BUG-093 stale (Cross-Tenant-Fix 5d8c48a), BUG-095 stale (läuft grün), BUG-096 stale (Mail-Fix c291a6e 46/46), BUG-094/097/098 gefixt (siehe oben) | ✅ Alle 6 Bugs geschlossen oder als bereits erledigt nachgewiesen | f4c4a50, 1b485d4, 69d05d6 |
| I-G-1 | BUG-022/070 Audits: npm audit = 0 vulnerabilities bereits sauber; pip-audit fand **9 known CVEs in starlette 0.46.2** (PYSEC-2026-161/248/249/1941/1942/2280/2281) — Dilemma: fastapi 0.115.x pinnt starlette<0.47.0, Fixes brauchen >=1.x | ✅ **0 pip findings**: fastapi 0.141.1 (zieht starlette ohne Obergrenze) + starlette direkt auf 1.3.1 gepinnt in requirements.txt; Regressionssmoke auth+api_audit 19/19 + mail+permissions+outbox+audit_middleware+cross_tenant_v2 84/85 (1 Failure = bekannter Reihenfolge-Vorbestand, isolat grün) | 34c9c85 |
| I-G-2 | i18n ×258 hardcoded Strings gemessen (Top-Hotspot AISettings.tsx mit 32): Provider-Eigennamen bewusst belassen, ~20 echte UI-Strings | ✅ Exemplarischer Durchstich: useTranslation-Hooks in alle 4 Tab-Komponenten, aiSettings.*-Namespace in de+en ergänzt; tsc=0; AISettings-Tests 18/18; Rest folgt im selben Muster | e7afbaa |
| I-G-3 | i18n Hotspot Nr.2: ProactiveAISettings.tsx (15+ deutsche Hardcodes inkl. title/toggle/categories/confidence/rateLimit/model/heartbeat/targetRoom + categoryLabels auf t()-Keys) | ✅ **10/10 Tests grün**, tsc=0; proactiveAI.*-Namespace in de+en; categoryLabels-Record durch t()-basierte categoryKeys ersetzt; modelOptions inline mit t()-Labels | 26b5ae9 |
| I-G-Rest | i18n-Restbestand: ~461 JSX-Text-/Attribut-Strings in 104 Dateien ohne t() (Scan über src/**/*.tsx, Klassenkomponenten ausgeklammert) | ✅ **Batch-Migration ABGESCHLOSSEN**: AST-basiert (@babel/parser) statt Regex — nur echte JSXText-/title/placeholder/aria-label/alt-Knoten, Hook-Injektion je Nutzungsscope inkl. Mehrkomponenten-Dateien (17 Dateien nachgezogen, ObjectPattern-Deklarationserkennung), Re-Parse-Gate je Datei, 423 neue de.json-Keys (Fallback en→de per fallbackLng). Beweise: tsc --noEmit exit=0, Produktionsbuild OK, Vitest 20F **byte-identisch zur Clean-Tree-Stash-Baseline** (alle Vorbestand); v1-Batch (Import-Slice-Bug) vollständig revertiert, nie committed | 4cb5298 |
| I-G-Rest | God Objects: 35 Python-Dateien >500 Z. — Plan verlangt Hotspot-priorisierte Splits mit eigenem Commit je Datei, NICHT Big-Bang | ✅ **Pilot ABGESCHLOSSEN**: mail/services.py 3087→~170 Z. (**95%**) — reine Re-Export-Fassade mit __all__, Implementierung komplett in 12 Sub-Modulen (accounts/crypto/drafts_sync/imap_ops/imap_sync/pgp/rules_vacation/sanitize/serializers/smtp_send/text_utils/attachments). Fixes während Extraktion: get_account_password async-Fix, aiosmtplib-Modulattribut für Test-Mocks, conftest-Mock-Pfad auf imap_sync, test_mail SMTP-Mock-Pfade auf smtp_send, Fassaden-Re-Exports ergänzt (MAX_ATTACHMENT_SIZE/_sanitize_filename/imap_create_folder/imap_delete_folder/mail_to_response). Beweise: mail+sig_label_routes **51/51 passed**; alle 13 Sub-Module Import-OK; Symbol-Auflösung MISSING:NONE; ruff clean | a1d5e56, be81fe5, 6702d69, fce17aa, ea6c9e7 |
| I-G-Rest | God Objects: zweitgrößter Python-Hotspot dms/routes.py (1492 Z., 24 Routen) | ✅ **Split ABGESCHLOSSEN**: routes.py 1492→650 Z. (**56%**) — neu: common.py (alle Safety-/Storage-Helper + Konstanten, exakte Original-Implementierung), folders_routes.py / sharing_routes.py / search_bulk_routes.py je eigener prefix-loser Router; routes.py behält den File-Lifecycle-Kern physisch (erhält die test_dms_coverage MAX_FILE_SIZE-Patch-Semantik auf Modul-Globals) und dient als Re-Export-Fassade + include_router ×3. Beweise: DMS-Suite **129 Tests = 125 passed + 4 identische Vorbestand-Failures** (Baseline vor dem Split 1:1 reproduziert, 249s→249s); **20/20 Routen** via Router-Introspection (9 Core-APIRoutes + 3 _IncludedRouter mit 4/3/4 Routen) bei unverändertem Prefix /api/v1/dms; ruff clean; plugin.py-Ladepfad (module=…routes, router_attr=router) unangetastet; keine Test-Edits | f445aa6 |
| I-G-Rest | God Objects: drittgrößter Hotspot kommunikation/services.py (1364 Z., 28 Funktionen) | ✅ **Split ABGESCHLOSSEN**: services.py → Re-Export-Fassade (~70 Z.) + 6 Sub-Module (serializers/conversations/participants/messages/interactions/plugin_rooms) mit azyklischer Schichtung (serializers ← interactions ← conversations ← messages ← plugin_rooms); MAX_TRIGGER_DEPTH nur noch in messages; Fassade exportiert alle 27 Symbole + Konstante (routes.py/contracts.py/test_notification_migration.py unverändert). Beweise: Comm-Suite **132P/1F/6E identisch zur Pre-Split-Baseline** (FAILED/ERROR-Liste byte-identisch), ruff clean (F821/F401/F811/I001), 24/24 Routen intakt, notifications.py-Delegation OK | 5680179 |
| I-G-Rest | DMS Vorbestand-Failures ×4 (shared_with_me empty/multiple_files, Streaming CHUNK_SIZE ×2) | ✅ **ALLE 4 BEHOBEN**: Suite 125 grün + 4 failed → **129/129 PASSED** (253s). (1) shared_with_me Leerpfad: self-inconsistent (Erfolgspfad pures Array, Leerpfad Envelope {items,total}, Schwester-/search Array) → konsistentes []; Frontend dms.ts Z.196 vertraegt beide Shapes. (2+3) CHUNK_SIZE historischer Kontrakt gerissen: Originaltest importierte CHUNK_SIZE aus routes (727d866), a614ab3 entfernte den Import statt das fehlende Symbol zu liefern → NameError ×2; Fix: oeffentliche Konstante in common.py + Re-Export + restaurierte Importzeile (keine Assertion angefasst). (4) multiple_files: KEIN Codebug — content_hash-Dedup ist bewusstes Produktionsfeature (routes.py Z.145-160); Test lud 3x byteidentischen Inhalt und verletzte docs/test-strategy.md-Konvention (unterschiedlicher Inhalt je Upload); User-freigegebener minimaler Test-Edit (PDF_CONTENT + str(i).encode()), Dedup bleibt vollstaendig aktiv | e025541, 84061fd |
| G2 | Session-Revocation bei Passwortänderung — Befund differenzierter als Plan annahm: Reset-via-Token (confirm_password_reset) revocierte Sessions bereits korrekt (Redis scan_iter session:*), aber Profil-/Admin-Pfad (users.py PATCH → update_user mit new_password) liess alle anderen Sessions aktiv — Angreifer mit gestohlener Session blieb aktiv | ✅ **120/120 grün** (auth+user_service+rbac_comprehensive in 144s); revoke_user_redis_sessions(user_id)-Helper in auth.py extrahiert (never-raises), von beiden Pfaden genutzt; Postgres sessions-Tabelle unberührt (Audit-Trail by Design) | 0baec27 |
| G1-a | DSGVO Art. 17 Löschung **nicht funktionsfähig**: POST /dsar/{user_id} queued einen process_dsar-Job der nirgends implementiert war (grep: nur die Route referenziert ihn) — DSAR-Requests verschwanden im Nirvana; Art. 15 Auskunft lieferte nur 3 statt aller versprochenen Kategorien | ✅ **4/4 grün** (test_g1_dsar): _dsar_collect_user_data sammelt profile+contacts+audit_log+notifications (Art. 15/20); _dsar_execute_deletion führt Art. 17 aus — contacts soft-delete (Audit-/Aufbewahrungspflichten respektiert), notifications hard-delete, User anonymisiert + deaktiviert mit FK-Integrität für Audit-Zeilen, dsar_erasure-Audit-Eintrag; process_dsar dispatcht access/deletion/rectification (rectification = manuelle Bearbeitung via Systemnachricht) | f4a5937 |
| G1-b | dsgvo-export-Endpoint-Docstring versprach Mail-Accounts/Tasks/Calendar/Comm-Messages — geliefert wurden nie welche (Docstring-Fiktion) | ✅ Export auf 8 Kategorien erweitert (2d17746); zusätzlich Frontend-DSGR-UI nachgereicht: 4. ComplianceTab-SubTab 'DSGVO-Anfragen' mit Typ-Wahl Art.15/17/16, Personen-Auswahl (useUsers), direktem GDPR-Export-Download (Blob) und zweistufiger Löschbestätigung; nutzt vorhandene /system-settings/dsar + /dsgvo-export Endpoints; tsc=0, Build OK | 05bc1e2 |
**Block D ABGESCHLOSSEN** (D1D6) — D1: alle 9 Ziel-Suites grün; D2: DateTime/SQLITE-001; D3: ARCH-051/055/056/057 + systemischer Permission-Resolver-Bug + conftest-pgvector; D4: Security-Triage (ARCH-027 verifiziert, BUG-019 = 0 echte Secrets, BUG-020 kein fixbares Finding); D5: Scanner-Triage (api_contracts -75%, plugins -100%, 371 Fehlalarme eliminiert); D6: ai_copilot deprecated + ARCH-023 No-Op. Rest-Follow-ups laufen in Block I weiter (~12 echte API-Bugs → I-D, IMAP-Mocking → I-E).
**Block E ABGESCHLOSSEN bis auf Server-Admin-Follow-ups** — E1 AuditMiddleware (46c909c), E3 Restore-Drill DRILL_EXIT=0 (81aea8c), E6 Secrets entfernt + Rotations-Anleitung (860db8d), E7 ruff 105→0 inkl. 8 F821-Produktionsbugs (197b0d3). ⏳ Beim User: Credential-Rotation, Actions-Runner, E2/E4/E5.
**Block A ABGESCHLOSSEN** — Gate A bestanden (32f63ad).
**Block B ABGESCHLOSSEN** — Gate B bestanden (alle 5 Checks bewiesen).
**Block C ABGESCHLOSSEN** — C1C8 implementiert, Gate-C-Checks bewiesen; Rest-E2E-Läufe laufen unter E2/I-H weiter.
| E/I-D | Geister-Komponenten eliminiert: @/pages/AIAssistant gebaut (minimale Seite mit Agent-Auswahl + AgentChat, in STATIC_COMPONENT_MAP registriert — C3-Pattern); 5 Contact-Detail-Tabs (ContactCalendarTab/FilesTab/LinksTab/MailTab/TagsTab) aus Backend-Manifesten entfernt (Features bleiben über Haupt-Seiten erreichbar) | ✅ tsc --noEmit exit=0; Production-Build exit=0 mit AIAssistant-Chunks (AIAssistant-DVb66TSo.js 5.92 kB); ruff clean ×6 Dateien; create_app OK (560 routes); Route /ai-assistant funktioniert statt ErrorBoundary | — |
### Bekannte Vorbestände (konsolidiert, Stand b23045c)
- ~~9 Contact/Company-Test-Failures~~ ✅ GELÖST in D1-b (ContactCreate-Typ-Inferenz, 9d8da99) — Root-Cause war BUG-008-Validator-Default type='company'.
- ~~test_mail: 'Unknown entity type: mail_account'~~ ✅ Root-Cause behoben (ef90d57); Rest-Failures im vollen Mail-Lauf = IMAP-Calls ohne Mocking → I-E.
- ~~Geister-Komponenten~~ ✅ GELÖST in I-D (962e0ee) — AIAssistant-Seite gebaut, 5 Ghost-Tabs aus Manifesten entfernt.
- ~~Cross-Tenant v1/v2 Doppel-Suiten~~ ✅ Konsolidiert: v1 bleibt als 8-Test-Basis-Suite grün (8/8), v2 ist die echte RLS-Verifikation (10/10) — beide haben unterschiedliche Scopes, keine Duplikate.
- ~~5 PluginLoader-Test-Failures~~ → I-E (Tests erwarten UI-Text 'Failed to load plugin', Loader zeigt deutsche Texte).
- ~~BUG-099~~: workstream.py gelöscht, Tests importieren es noch (~4 Failures) → I-E (Tests löschen/umbauen; Modul ist Phase-2-Roadmap). Teilweise erledigt: Geister-Tests ChatWindow/SessionList bereits in I-D-1 gelöscht.
- ~~~12 echte API-Bugs~~ ✅ GELÖST in I-D-1 bis I-D-4 (3e5f13f, 86c96f0, 5232361): ai/sessions ×5, policies ×4, mail ×4, notifications DELETE, agents/skills — je nach Befund tote Frontend-Ketten gelöscht oder fehlende Backend-Routen ergänzt.
### Handover-Hinweis für Nachfolge-Agent
- Reparaturplan: docs/fix-plan-v3.md — **Blöcke 0/H/A/B/C/D/E/F/G done** (G1 inkl. Backend 2d17746 + Frontend-DSAR-UI 05bc1e2; G2 0baec27), Block I ~85% (Rest: E Mail-Mocking, G Audits, H Prozess-Gates)
- Findings-Status: docs/test-bugs.md (✅/⏳ je Finding)
- Verifikationsmuster: Stash-Test gegen Pre-Block-Commit für Vorbestands-Nachweis; Endpoint-Diff via OpenAPI-Snapshot; Cross-Plugin-Scan als Gate
- Test-DB: .env.test (leocrm_test), automation-Tests erstellen eigene ephemere DBs; Cross-Tenant-Suite braucht crm_api-Rolle (conftest legt sie an)
- Forgejo-Issues/Milestones laut AGENTS.md §9 noch NICHT angelegt — nur PROGRESS.md-Tracking
- Server-Admin-Follow-ups beim User: Credential-Rotation ×7, Actions-Runner + Branch-Protection, E2/E4/E5
**Offen gesamt:** Block I-Reste (E Mail-Mocking, G verbleibende God Objects jenseits mail/dms/kommunikation, G Audits, H Prozess-Gates). Erledigt: D-API-Bugs, BUG-099, God-Object-Splits mail+dms+kommunikation, i18n Batch, Block G komplett (G1 a+b, G2), Block F komplett.
**Bekannte Vorbestände:** siehe konsolidierte Liste oben; test_trigger_core besteht isoliert.
---
## Übersicht
| Phase | Status | Start | Ende | Done | Partial | Not Done | Total | Anmerkung |
-348
View File
@@ -1,348 +0,0 @@
# LeoCRM UI-Overhaul-Plan (v2)
> **Erstellt:** 2026-08-21
> **Aktualisiert:** 2026-08-21 — AI Assistent Integration hinzugefügt
> **Status:** Planung — nicht gestartet
> **Leitlinie:** Auf bestehendem Code aufbauen, 3-Spalten-Explorer-Layout als Standard, keine parallelen Systeme
---
## Standard-Layout (Referenz: ContactsList.tsx)
Alle Explorer-Plugins nutzen das 3-Spalten-Layout aus den UI-Design-Guidelines:
```
┌─────────────┬──────────────────┬──────────────────────┐
│ Tree │ Liste/Ansicht │ Detail │
│ (224px) │ (flex-1) │ (flex-1 / 60%) │
│ ResizablePanel│ ResizablePanel │ ResizablePanel │
└─────────────┴──────────────────┴──────────────────────┘
```
- **Toolbar oben:** PluginToolbar mit Filter-Dropdowns, Ansichts-Umschaltern, Aktion-Buttons
- **Linke Spalte:** ResizablePanel mit Baumansicht (Ordner, Kategorien, Kalender)
- **Mitte:** Liste, Karten, Kalender-Ansicht — mehrere Ansichten umschaltbar
- **Rechts:** Detail-Bereich für ausgewähltes Element
---
## Phase 1: Echte Bugs fixen (2-3 Tage)
### 1.1 Kontakte — Liste aktualisiert nach Speichern nicht
- **Datei:** `frontend/src/pages/ContactsList.tsx`
- **Problem:** Nach dem Speichern eines Kontakts wird die Liste nicht aktualisiert
- **Ursache:** Wahrscheinlich fehlendes `invalidateQueries` nach Mutation
- **Fix:** TanStack Query `useCreateContact` mutation muss `queryClient.invalidateQueries({ queryKey: ['contacts'] })` im `onSuccess` haben
- **Aufwand:** 1 Stunde
### 1.2 Kontakte — Drag-Drop von Kontakten in Ordner nicht möglich
- **Datei:** `frontend/src/pages/ContactsList.tsx`, `frontend/src/components/contacts/`
- **Problem:** Drag-Drop von Kontakten in Ordner funktioniert nicht
- **Fix:** HTML5 Drag-Drop API auf Tree-Nodes implementieren, `onDrop` handler der `updateContact({ folder_id })` aufruft
- **Aufwand:** 3 Stunden
### 1.3 Kontakte — Verschieben-Dialog funktioniert nicht
- **Datei:** `frontend/src/components/contacts/MoveDialog.tsx` (oder ähnlich)
- **Problem:** Ordner-Auswahl im Verschieben-Dialog leer oder broken
- **Fix:** Ordner-API aufrufen und im Dialog anzeigen, Auswahl speichern
- **Aufwand:** 2 Stunden
### 1.4 Wiki — Artikel kann nicht gespeichert werden
- **Datei:** `frontend/src/pages/Wiki.tsx`, `frontend/src/api/knowledge.ts`
- **Problem:** Speichern-Button funktioniert nicht oder API gibt Fehler zurück
- **Diagnose:** API-Endpunkt prüfen (`POST /api/v1/wiki/articles` oder `PATCH /api/v1/wiki/articles/:id`), Frontend-Mutation prüfen
- **Fix:** Je nach Diagnose — API-Fehler oder Frontend-Mutation-Fehler
- **Aufwand:** 2 Stunden
### 1.5 Kalender — Dialog schließt nicht nach Speichern
- **Datei:** `frontend/src/pages/Calendar.tsx`, `frontend/src/components/calendar/AppointmentEditForm.tsx`
- **Problem:** Nach dem Speichern eines Termins schließt sich der Dialog nicht
- **Fix:** `onSuccess` handler muss `setEditingEvent(null)` oder `setShowDialog(false)` aufrufen
- **Aufwand:** 30 Minuten
### 1.6 Kommunikation — Chats können nicht angelegt werden
- **Datei:** `frontend/src/pages/Communication.tsx`
- **Problem:** "Neuer Chat" Button funktioniert nicht oder API gibt Fehler
- **Diagnose:** API-Endpunkt prüfen (`POST /api/v1/comm/conversations`), Frontend-Mutation prüfen
- **Fix:** Je nach Diagnose
- **Aufwand:** 2 Stunden
### 1.7 Wiki — Doppelt im Menü
- **Datei:** `frontend/src/routes/index.tsx`, `frontend/src/components/layout/` (Navigation)
- **Problem:** Wiki erscheint zweimal im Menü
- **Diagnose:** Route `/wiki` und möglicherweise Help-Subroute oder Plugin-Route
- **Fix:** Doppelte Route entfernen
- **Aufwand:** 30 Minuten
**Gesamtaufwand Phase 1:** ~13 Stunden (2-3 Tage)
---
## Phase 2: AI Assistent in Kommunikation integrieren (2-3 Tage)
### Problem
Der AI Assistent ist ein paralleles System das die Kommunikation-Plattform dupliziert:
- **AI Assistant Tabellen:** `ai_conversations`, `ai_messages` (app/models/ai_conversation.py) + `ai_chat_sessions`, `ai_chat_messages`, `ai_chat_attachments` (app/plugins/builtins/ai_assistant/models.py) — 5 Tabellen
- **AI Assistant Frontend:** `AIAssistant.tsx`, `AIAssistantStandalone.tsx`, `SessionList.tsx`, `ChatWindow.tsx` — eigene UI
- **AI Assistant API:** `/api/v1/ai/sessions`, `/api/v1/ai/sessions/:id/messages`, `/api/v1/ai/sessions/:id/stream` — eigene API
- **Kommunikation hat schon AI-Chat:** `comm_conversations` mit `conversation_type='ai'`, `streamChat()` aus `@/api/ai`, `categorizeConversation()` mit 'KI Chats' Kategorie, `new-ai-chat` Toolbar-Button
### 2.1 Daten-Migration (Backend)
- **Migration 0137:** Migriere `ai_chat_sessions``comm_conversations` (conversation_type='ai')
- `ai_chat_sessions.id``comm_conversations.id`
- `ai_chat_sessions.title``comm_conversations.title`
- `ai_chat_sessions.tenant_id``comm_conversations.tenant_id`
- `ai_chat_sessions.user_id``comm_conversations.owner_id`
- `ai_chat_sessions.agent_id``comm_conversations.metadata.agent_id`
- `ai_chat_sessions.created_at``comm_conversations.created_at`
- **Migration 0137:** Migriere `ai_chat_messages``comm_messages`
- `ai_chat_messages.id``comm_messages.id`
- `ai_chat_messages.session_id``comm_messages.conversation_id`
- `ai_chat_messages.role``comm_messages.sender_type` ('user' → 'user', 'assistant' → 'ai')
- `ai_chat_messages.content``comm_messages.content`
- `ai_chat_messages.tenant_id``comm_messages.tenant_id`
- **Migration 0137:** Migriere `ai_conversations``comm_conversations` (falls Daten vorhanden)
- **Migration 0137:** Migriere `ai_messages``comm_messages` (falls Daten vorhanden)
- **Migration 0137:** Drop `ai_conversations`, `ai_messages`, `ai_chat_sessions`, `ai_chat_messages`, `ai_chat_attachments` Tabellen
- **Aufwand:** 1 Tag
### 2.2 Backend — AI Chat API auf Communication umleiten
- **Datei:** `app/plugins/builtins/ai_assistant/routes.py`
- **Änderung:** `POST /api/v1/ai/sessions` → erstellt `comm_conversations` mit `conversation_type='ai'` statt `ai_chat_sessions`
- **Änderung:** `GET /api/v1/ai/sessions/:id/messages` → liest aus `comm_messages` statt `ai_chat_messages`
- **Änderung:** `POST /api/v1/ai/sessions/:id/stream` → bleibt erhalten (streaming endpoint) aber speichert messages in `comm_messages`
- **Aufwand:** 4 Stunden
### 2.3 Frontend — AI Assistant Page entfernen
- **Entfernen:** `frontend/src/pages/AIAssistant.tsx`
- **Entfernen:** `frontend/src/pages/AIAssistantStandalone.tsx`
- **Entfernen:** `frontend/src/components/ai/SessionList.tsx`
- **Entfernen:** `frontend/src/components/ai/ChatWindow.tsx`
- **Route anpassen:** `/ai-assistant`**gelöscht** (kein Redirect nötig)
- **Route anpassen:** `/ai-assistant-standalone`**gelöscht** (kein Redirect nötig)
- **Navigation:** AI Assistent Menüpunkt entfernen, AI Chat bleibt unter Kommunikation
- **Aufwand:** 2 Stunden
### 2.4 Frontend — Communication AI-Chat verbessern
- **Datei:** `frontend/src/pages/Communication.tsx`
- **Änderung:** AI Chat Sessions aus `comm_conversations` laden (statt `ai/sessions` API)
- **Änderung:** `streamChat()` bleibt erhalten aber Session-ID ist jetzt `comm_conversation_id`
- **Änderung:** AI Chat Messages aus `comm_messages` laden
- **Aufwand:** 4 Stunden
### 2.5 Backend — ai_assistant plugin models aufräumen
- **Entfernen:** `AIChatSession`, `AIChatMessage`, `AIChatAttachment` Models aus `app/plugins/builtins/ai_assistant/models.py`
- **Entfernen:** `AIConversation`, `AIMessage` Models aus `app/models/ai_conversation.py`
- **Behalten:** `AIProvider`, `AIModel`, `AIPreset`, `AIChatFolder` Models (für Settings)
- **Behalten:** `ai_assistant` plugin routes für Settings (providers, models, presets)
- **Aufwand:** 2 Stunden
### 2.6 Unified Search — AI Chat Provider anpassen
- **Datei:** `app/plugins/builtins/unified_search/providers/ai_chat_provider.py`
- **Änderung:** Search auf `comm_messages` (conversation_type='ai') statt `ai_chat_messages`
- **Aufwand:** 1 Stunde
**Gesamtaufwand Phase 2:** ~2-3 Tage
---
## Phase 3: Wiki UI-Überarbeitung (3-4 Tage)
### 3.1 WYSIWYG Editor
- **Datei:** `frontend/src/components/wiki/WikiEditor.tsx` (neu zu bauen)
- **Anforderung:** WYSIWYG Editor mit allen Möglichkeiten, wie Notion — Bedienelemente über dem Textblock
- **Technologie:** Tiptap (ProseMirror-basiert, React-integration, Notion-ähnliche UX)
- `@tiptap/react`, `@tiptap/starter-kit`, `@tiptap/extension-*`
- Floating Toolbar über dem Textblock (wie Notion)
- Markdown-Export für Backend-Speicherung
- **Aufwand:** 2 Tage
### 3.2 Wiki Layout — 3-Spalten
- **Datei:** `frontend/src/pages/Wiki.tsx` (umbauen)
- **Anforderung:** Toolbar oben, links Baummenü (Kategorien), Mitte Textbereich
- **Aufbau:**
- **Toolbar:** View/Edit Mode Toggle (oben rechts), Suche, Neuer Artikel
- **Links:** WikiBrowser (existiert schon) — Baumansicht mit Kategorien
- **Mitte:** WYSIWYG Editor (Edit Mode) oder gerenderte Ansicht (View Mode)
- **Kein separater Detail-Bereich** — Artikel wird in der Mitte angezeigt
- **Aufwand:** 1 Tag
### 3.3 View/Edit Mode Toggle
- **Datei:** `frontend/src/pages/Wiki.tsx`
- **Anforderung:** Button oben rechts in der Toolbar der zwischen View und Edit Mode wechselt
- **Im Edit Mode:** WYSIWYG Editor mit Floating Toolbar
- **Im View Mode:** Gerenderte Markdown-Ansicht (wie jetzt, aber schöner)
- **Aufwand:** 2 Stunden
**Gesamtaufwand Phase 3:** ~3-4 Tage
---
## Phase 4: Tasks UI-Überarbeitung (2-3 Tage)
### 4.1 Tasks Layout — 3-Spalten wie Kontakte
- **Datei:** `frontend/src/pages/Tasks.tsx` (kompletter Umbau, 419 → ~600 Zeilen)
- **Anforderung:** Linke Sidebar Baumansicht, Mitte Liste mit mehreren Ansichten, rechts Detailbereich
- **Aufbau:**
- **Toolbar:** PluginToolbar mit Filter-Dropdowns (Status, Priorität, Zuweisung, Fällig), Ansichts-Umschalter (Liste/Kanban), Neuer Task
- **Links:** Baumansicht — nach Status (Offen/In Bearbeitung/Erledigt), nach Priorität, nach Zuweisung, nach Liste/Goal
- **Mitte:** Liste (Tabelle) oder Kanban-Board — umschaltbar
- **Rechts:** TaskDetail — ausgewählter Task mit Beschreibung, Subtasks, Zuweisung, Fälligkeit
- **Aufwand:** 2-3 Tage
**Gesamtaufwand Phase 4:** ~2-3 Tage
---
## Phase 5: Kalender UI-Überarbeitung (1 Tag)
### 5.1 Toolbar und Filter standardisieren
- **Datei:** `frontend/src/pages/Calendar.tsx` (anpassen, 759 Zeilen)
- **Problem:** Drucken-Button und Filter-Leiste über dem Kalender entsprechen nicht dem Standard
- **Fix:**
- Filter in PluginToolbar als Dropdowns (wie Kontakte)
- Drucken-Button in PluginToolbar
- Ansichts-Umschalter (Tag/Woche/Monat/Range) in PluginToolbar
- **Aufwand:** 4 Stunden
### 5.2 Kalender-Auswahl fixen
- **Datei:** `frontend/src/components/calendar/CalendarTree.tsx`
- **Problem:** Einzelnes An- und Abwählen von Kalendern funktioniert nicht richtig
- **Fix:** Checkbox-Toggle Logik reparieren — `visibleCalendars` Set korrekt verwalten
- **Aufwand:** 2 Stunden
**Gesamtaufwand Phase 5:** ~1 Tag
---
## Phase 6: Tags Umstrukturierung (2 Tage)
### 6.1 Tags in Settings verschieben
- **Datei:** `frontend/src/pages/Tags.tsx``frontend/src/pages/SettingsTags.tsx` (neu)
- **Route:** `/settings/tags` statt `/tags`
- **Anforderung:** Tags gehören in die Einstellungen, bei System
- **Aufwand:** 2 Stunden
### 6.2 Tags Baumstruktur
- **Datei:** `frontend/src/pages/SettingsTags.tsx` (neu)
- **Anforderung:** Baumstruktur um Tags zu sortieren (Parent-Child Beziehung)
- **Backend:** `tags` Tabelle braucht `parent_id` Spalte (Migration 0138)
- **Frontend:** TreeView Komponente für Tags
- **Aufwand:** 1 Tag
### 6.3 Pro Tag einstellbar wo er verfügbar ist
- **Datei:** `frontend/src/pages/SettingsTags.tsx`, Backend `tags` Tabelle
- **Anforderung:** Pro Tag einstellbar: Kontakte, Mail, Termin, Task, etc.
- **Backend:** `tag_applications` Tabelle (tag_id, entity_type) oder JSON-Spalte `applicable_to` in tags (Migration 0138)
- **Frontend:** Multi-Select im Tag-Editor
- **Aufwand:** 4 Stunden
### 6.4 Symbol und Farbe pro Tag
- **Datei:** `frontend/src/pages/SettingsTags.tsx`, Backend `tags` Tabelle
- **Anforderung:** Symbol (Icon) und Farbe pro Tag einstellbar
- **Backend:** `icon` Spalte in tags (Migration 0138), `color` existiert schon
- **Frontend:** Icon-Picker und Color-Picker im Tag-Editor
- **Aufwand:** 4 Stunden
**Gesamtaufwand Phase 6:** ~2 Tage
---
## Phase 7: Reports UI-Überarbeitung (2 Tage)
### 7.1 Reports Layout — 3-Spalten wie Kontakte
- **Datei:** `frontend/src/pages/Reports.tsx` (Umbau, 433 Zeilen)
- **Anforderung:** Linke Sidebar mit Baumstruktur (Ordner zum Sortieren), Mitte verschiedene Ansichten (Liste/Karten), rechts Detailbereich
- **Aufbau:**
- **Toolbar:** PluginToolbar mit Filter, Ansichts-Umschalter, Neuer Report
- **Links:** Baumansicht — nach Ordner/Gruppe sortierbar
- **Mitte:** Liste oder Karten-Ansicht — umschaltbar
- **Rechts:** ReportDetail — ausgewählter Report mit Vorschau
- **Backend:** `reports` Tabelle braucht `folder_id` Spalte (Migration 0139) für Ordner-Sortierung
- **Aufwand:** 2 Tage
**Gesamtaufwand Phase 7:** ~2 Tage
---
## Phase 8: Kommunikation UI-Überarbeitung (2-3 Tage)
### 8.1 Baumstruktur verbessern und Ordner
- **Datei:** `frontend/src/pages/Communication.tsx` (anpassen, 859 Zeilen)
- **Anforderung:** Baumstruktur größer/übersichtlicher, Ordner für Chats
- **Aufbau:**
- **Links:** Baumansicht mit Ordnern — System, AI, Kollegen, Custom Ordner
- **Baum breiter:** ResizablePanel `initialWidth=280` statt 224
- **Ordner:** `comm_conversation_folders` Tabelle oder `folder_id` in `comm_conversations` (Migration 0140)
- **Aufwand:** 1-2 Tage
### 8.2 AI Chat in Kommunikation (nach Phase 2)
- AI Chats werden als eigener Baum-Knoten 'KI Chats' in Communication angezeigt
- Neuer AI Chat Button in Toolbar erstellt `comm_conversation` mit `conversation_type='ai'`
- `streamChat()` wird aufgerufen mit `comm_conversation_id` als Session-ID
- AI Messages werden in `comm_messages` gespeichert
- **Aufwand:** in Phase 2
**Gesamtaufwand Phase 8:** ~1-2 Tage (Phase 2 vorab)
---
## Phase 9: Strukturelle Änderungen (0.5 Tage)
### 9.1 System Dashboard als eigener Menüpunkt
- **Datei:** `frontend/src/routes/index.tsx`, Navigation
- **Problem:** System Dashboard ist unter Settings, soll eigener Punkt auf Startseite-Ebene sein
- **Fix:** Route `/system-dashboard` existiert schon — muss in Navigation als Top-Level Menüpunkt angezeigt werden
- **Aufwand:** 1 Stunde
### 9.2 Mail — Postfach mit IMAP anlegen testen
- **Datei:** `frontend/src/pages/Mail.tsx`, `frontend/src/pages/MailSettings.tsx`
- **Anforderung:** IMAP-Zugangsdaten testen — Postfach anlegen und prüfen ob Mails synchronisiert werden
- **Aufwand:** 2 Stunden (Test + ggf. Bugfix)
**Gesamtaufwand Phase 9:** ~0.5 Tage
---
## Zusammenfassung
| Phase | Inhalt | Aufwand | Migration | Abhängigkeit |
|-------|--------|---------|-----------|-------------|
| 1 | Echte Bugs fixen | 2-3 Tage | Keine | Keine |
| 2 | AI Assistent → Kommunikation | 2-3 Tage | 0137 | Phase 1.6 |
| 3 | Wiki UI + WYSIWYG | 3-4 Tage | Keine | Phase 1.4 |
| 4 | Tasks UI neu | 2-3 Tage | Keine | Keine |
| 5 | Kalender UI | 1 Tag | Keine | Phase 1.5 |
| 6 | Tags Umstrukturierung | 2 Tage | 0138 | Keine |
| 7 | Reports UI | 2 Tage | 0139 | Keine |
| 8 | Kommunikation UI | 1-2 Tage | 0140 | Phase 2 |
| 9 | Strukturelle Änderungen | 0.5 Tage | Keine | Keine |
**Gesamtaufwand:** ~17-22 Tage
### Reihenfolge:
1. **Phase 1** (Bugs) — zuerst, damit grundlegende Funktionen arbeiten
2. **Phase 9** (Strukturelle Änderungen) — schnell, wenig Aufwand
3. **Phase 5** (Kalender) — kleines Update, baut auf Phase 1 auf
4. **Phase 2** (AI Assistent → Kommunikation) — entfernt paralleles System, baut auf Phase 1.6 auf
5. **Phase 6** (Tags) — unabhängig, Backend + Frontend
6. **Phase 4** (Tasks) — großer Umbau, unabhängig
7. **Phase 3** (Wiki) — größter Umbau (WYSIWYG Editor), baut auf Phase 1 auf
8. **Phase 7** (Reports) — großer Umbau, unabhängig
9. **Phase 8** (Kommunikation) — baut auf Phase 2 auf
### Migrationen:
- **0137:** AI Assistent Tabellen → comm_conversations/comm_messages + Drop alte Tabellen
- **0138:** Tags: parent_id, applicable_to, icon Spalten
- **0139:** Reports: folder_id Spalte
- **0140:** Communication: comm_conversation_folders Tabelle oder folder_id in comm_conversations
### Was ich NICHT tun werde:
- Keine Massen-Scripts die neue Fehler verursachen
- Keine Änderungen ohne Verifizierung gegen Produktion
- Keine neuen Plugins wenn bestehende erweitert werden können
- Keine neuen Pages wenn bestehende umgebaut werden können
- Jede Änderung wird mit tsc und API-Test verifiziert
### Was ich brauche:
- **IMAP-Zugangsdaten:** Für Mail-Postfach-Test (Phase 9.2)
@@ -18,7 +18,26 @@ branch_labels = None
depends_on = None
def _table_exists(conn, table_name: str) -> bool:
"""True when the table exists (dual-path convergence, Gate B).
On a fresh install the ai_assistant plugin SQL migration has not run
yet when Alembic reaches this revision skip instead of failing.
The plugin-side migration adds the same columns idempotently.
"""
row = conn.execute(
sa.text("SELECT to_regclass(:tname) IS NOT NULL"),
{"tname": f"public.{table_name}"},
).scalar()
return bool(row)
def upgrade() -> None:
conn = op.get_bind()
if not _table_exists(conn, "ai_providers"):
# Fresh-install path: table arrives with the ai_assistant plugin
# migration, which includes these columns.
return
op.add_column("ai_providers", sa.Column("region", sa.String(20), nullable=False, server_default="unknown"))
op.add_column("ai_providers", sa.Column("hosting_type", sa.String(30), nullable=False, server_default="cloud"))
op.add_column("ai_providers", sa.Column("dpa_status", sa.String(20), nullable=False, server_default="none"))
@@ -29,6 +48,9 @@ def upgrade() -> None:
def downgrade() -> None:
conn = op.get_bind()
if not _table_exists(conn, "ai_providers"):
return
op.drop_column("ai_providers", "allowed_data_classes")
op.drop_column("ai_providers", "transfer_notice")
op.drop_column("ai_providers", "training_on_customer_data")
@@ -17,126 +17,145 @@ branch_labels = None
depends_on = None
def _table_exists(conn, table_name: str) -> bool:
"""True when the table exists (dual-path convergence, Gate B).
On a fresh install the kommunikation plugin SQL migration has not run
yet when Alembic reaches this revision skip the comm_* parts instead
of failing. The plugin-side migration adds the same column idempotently.
"""
row = conn.execute(
sa.text("SELECT to_regclass(:tname) IS NOT NULL"),
{"tname": f"public.{table_name}"},
).scalar()
return bool(row)
def upgrade() -> None:
# 1. Add is_system column to comm_conversations
op.add_column(
"comm_conversations",
sa.Column("is_system", sa.Boolean(), nullable=False, server_default=sa.text("false")),
)
op.create_index(
"ix_comm_conversations_tenant_system",
"comm_conversations",
["tenant_id", "is_system"],
)
conn = op.get_bind()
if _table_exists(conn, "comm_conversations"):
# 1. Add is_system column to comm_conversations
op.add_column(
"comm_conversations",
sa.Column("is_system", sa.Boolean(), nullable=False, server_default=sa.text("false")),
)
op.create_index(
"ix_comm_conversations_tenant_system",
"comm_conversations",
["tenant_id", "is_system"],
)
# 2. Create system channel per tenant (for tenants that have notifications)
op.execute("""
INSERT INTO comm_conversations (id, tenant_id, title, is_pinned, is_locked, is_direct, is_archived, is_system, created_by, created_by_type, metadata, created_at, updated_at)
SELECT
gen_random_uuid(),
n.tenant_id,
'System Channel',
false,
true,
false,
false,
true,
NULL,
'system',
'{}'::jsonb,
NOW(),
NOW()
FROM (
SELECT DISTINCT tenant_id FROM notifications WHERE deleted_at IS NULL
) n
WHERE NOT EXISTS (
SELECT 1 FROM comm_conversations cc
WHERE cc.tenant_id = n.tenant_id AND cc.is_system = true AND cc.deleted_at IS NULL
);
""")
# 2. Create system channel per tenant (for tenants that have notifications)
op.execute("""
INSERT INTO comm_conversations (id, tenant_id, title, is_pinned, is_locked, is_direct, is_archived, is_system, created_by, created_by_type, metadata, created_at, updated_at)
SELECT
gen_random_uuid(),
n.tenant_id,
'System Channel',
false,
true,
false,
false,
true,
NULL,
'system',
'{}'::jsonb,
NOW(),
NOW()
FROM (
SELECT DISTINCT tenant_id FROM notifications WHERE deleted_at IS NULL
) n
WHERE NOT EXISTS (
SELECT 1 FROM comm_conversations cc
WHERE cc.tenant_id = n.tenant_id AND cc.is_system = true AND cc.deleted_at IS NULL
);
""")
# 3. Insert notifications as CommMessages in the system channel
op.execute("""
INSERT INTO comm_messages (id, tenant_id, conversation_id, sender_id, sender_type, content, content_format, metadata, created_at, updated_at)
SELECT
gen_random_uuid(),
n.tenant_id,
sc.id,
n.user_id,
'system',
COALESCE(n.title, '') || CASE WHEN n.body IS NOT NULL THEN E'\n' || n.body ELSE '' END,
'text',
jsonb_build_object(
'notification_type', n.type,
'severity', 'info',
'entity_ref', CASE WHEN n.entity_type IS NOT NULL THEN jsonb_build_object('entity_type', n.entity_type, 'entity_id', n.entity_id::text) ELSE NULL END,
'migrated_from_notification', true,
'original_notification_id', n.id::text
),
n.created_at,
COALESCE(n.read_at, n.created_at)
FROM notifications n
JOIN comm_conversations sc ON sc.tenant_id = n.tenant_id AND sc.is_system = true AND sc.deleted_at IS NULL
WHERE n.deleted_at IS NULL;
""")
# 3. Insert notifications as CommMessages in the system channel
op.execute("""
INSERT INTO comm_messages (id, tenant_id, conversation_id, sender_id, sender_type, content, content_format, metadata, created_at, updated_at)
SELECT
gen_random_uuid(),
n.tenant_id,
sc.id,
n.user_id,
'system',
COALESCE(n.title, '') || CASE WHEN n.body IS NOT NULL THEN E'\n' || n.body ELSE '' END,
'text',
jsonb_build_object(
'notification_type', n.type,
'severity', 'info',
'entity_ref', CASE WHEN n.entity_type IS NOT NULL THEN jsonb_build_object('entity_type', n.entity_type, 'entity_id', n.entity_id::text) ELSE NULL END,
'migrated_from_notification', true,
'original_notification_id', n.id::text
),
n.created_at,
COALESCE(n.read_at, n.created_at)
FROM notifications n
JOIN comm_conversations sc ON sc.tenant_id = n.tenant_id AND sc.is_system = true AND sc.deleted_at IS NULL
WHERE n.deleted_at IS NULL;
""")
# 4. Insert text blocks for each migrated message
op.execute("""
INSERT INTO comm_message_blocks (id, tenant_id, message_id, block_type, block_data, sort_order)
SELECT
gen_random_uuid(),
cm.tenant_id,
cm.id,
'text',
jsonb_build_object('text', cm.content),
0
FROM comm_messages cm
WHERE cm.metadata->>'migrated_from_notification' = 'true';
""")
# 4. Insert text blocks for each migrated message
op.execute("""
INSERT INTO comm_message_blocks (id, tenant_id, message_id, block_type, block_data, sort_order)
SELECT
gen_random_uuid(),
cm.tenant_id,
cm.id,
'text',
jsonb_build_object('text', cm.content),
0
FROM comm_messages cm
WHERE cm.metadata->>'migrated_from_notification' = 'true';
""")
# 5. Insert action_card blocks for messages with entity references
op.execute("""
INSERT INTO comm_message_blocks (id, tenant_id, message_id, block_type, block_data, sort_order)
SELECT
gen_random_uuid(),
cm.tenant_id,
cm.id,
'action_card',
jsonb_build_object(
'label', 'Open',
'entity_type', (cm.metadata->'entity_ref'->>'entity_type'),
'entity_id', (cm.metadata->'entity_ref'->>'entity_id')
),
1
FROM comm_messages cm
WHERE cm.metadata->>'migrated_from_notification' = 'true'
AND cm.metadata->'entity_ref' IS NOT NULL;
""")
# 5. Insert action_card blocks for messages with entity references
op.execute("""
INSERT INTO comm_message_blocks (id, tenant_id, message_id, block_type, block_data, sort_order)
SELECT
gen_random_uuid(),
cm.tenant_id,
cm.id,
'action_card',
jsonb_build_object(
'label', 'Open',
'entity_type', (cm.metadata->'entity_ref'->>'entity_type'),
'entity_id', (cm.metadata->'entity_ref'->>'entity_id')
),
1
FROM comm_messages cm
WHERE cm.metadata->>'migrated_from_notification' = 'true'
AND cm.metadata->'entity_ref' IS NOT NULL;
""")
# 6. For read notifications, create CommMessageRead entries
op.execute("""
INSERT INTO comm_message_reads (id, tenant_id, conversation_id, user_id, last_read_msg_id, last_read_at)
SELECT
gen_random_uuid(),
cm.tenant_id,
cm.conversation_id,
cm.sender_id,
cm.id,
COALESCE(n.read_at, n.created_at)
FROM comm_messages cm
JOIN notifications n ON n.id::text = cm.metadata->>'original_notification_id'
WHERE cm.metadata->>'migrated_from_notification' = 'true'
AND n.read_at IS NOT NULL
AND n.deleted_at IS NULL;
""")
# 6. For read notifications, create CommMessageRead entries
op.execute("""
INSERT INTO comm_message_reads (id, tenant_id, conversation_id, user_id, last_read_msg_id, last_read_at)
SELECT
gen_random_uuid(),
cm.tenant_id,
cm.conversation_id,
cm.sender_id,
cm.id,
COALESCE(n.read_at, n.created_at)
FROM comm_messages cm
JOIN notifications n ON n.id::text = cm.metadata->>'original_notification_id'
WHERE cm.metadata->>'migrated_from_notification' = 'true'
AND n.read_at IS NOT NULL
AND n.deleted_at IS NULL;
""")
# 7. Create legacy view over notifications table for backward compatibility
# 7. Legacy view over the CORE notifications table — exists on both paths
op.execute("DROP VIEW IF EXISTS notifications_legacy")
op.execute("CREATE VIEW notifications_legacy AS SELECT * FROM notifications")
def downgrade() -> None:
conn = op.get_bind()
op.execute("DROP VIEW IF EXISTS notifications_legacy")
if not _table_exists(conn, "comm_conversations"):
return
op.execute("DELETE FROM comm_message_blocks WHERE message_id IN (SELECT id FROM comm_messages WHERE metadata->>'migrated_from_notification' = 'true')")
op.execute("DELETE FROM comm_messages WHERE metadata->>'migrated_from_notification' = 'true'")
op.execute("DELETE FROM comm_conversations WHERE is_system = true AND title = 'System Channel'")
+17
View File
@@ -14,7 +14,24 @@ branch_labels = None
depends_on = None
def _table_exists(conn, table_name: str) -> bool:
"""True when the table exists (dual-path convergence, Gate B).
On a fresh install the automation plugin SQL migration has not run yet
when Alembic reaches this revision skip instead of failing. The
plugin-side convergence migration creates the same table.
"""
row = conn.execute(
sa.text("SELECT to_regclass(:tname) IS NOT NULL"),
{"tname": f"public.{table_name}"},
).scalar()
return bool(row)
def upgrade() -> None:
conn = op.get_bind()
if not _table_exists(conn, "automation_agent_runs"):
return
op.create_table(
"automation_agent_run_steps",
sa.Column("id", PGUUID(as_uuid=True), primary_key=True),
@@ -18,7 +18,24 @@ branch_labels = None
depends_on = None
def _table_exists(conn, table_name: str) -> bool:
"""True when the table exists (dual-path convergence, Gate B).
On a fresh install the automation plugin SQL migration has not run yet
when Alembic reaches this revision skip instead of failing. The
plugin-side convergence migration adds the same columns.
"""
row = conn.execute(
sa.text("SELECT to_regclass(:tname) IS NOT NULL"),
{"tname": f"public.{table_name}"},
).scalar()
return bool(row)
def upgrade() -> None:
conn = op.get_bind()
if not _table_exists(conn, "automation_agent_definitions"):
return
op.add_column(
"automation_agent_definitions",
sa.Column("temperature", sa.Float, nullable=False, server_default="0.3"),
@@ -20,7 +20,27 @@ branch_labels = None
depends_on = None
def _table_exists(conn, table_name: str) -> bool:
"""True when the table exists (dual-path convergence, Gate B).
On a fresh install the tasks plugin SQL migration has not run yet when
Alembic reaches this revision skip instead of failing. The plugin-side
convergence migration adds the same columns/indexes. The legacy-data
backfills below only matter for pre-existing rows and are correctly
empty on a fresh install.
"""
row = conn.execute(
sa.text("SELECT to_regclass(:tname) IS NOT NULL"),
{"tname": f"public.{table_name}"},
).scalar()
return bool(row)
def upgrade() -> None:
conn = op.get_bind()
if not _table_exists(conn, "tasks"):
return
# ── Add new columns to tasks ────────────────────────────────────────────
op.add_column("tasks", sa.Column("assignee_type", sa.String(20), nullable=False, server_default="user"))
op.add_column("tasks", sa.Column("assignee_id", PGUUID(as_uuid=True), nullable=True))
@@ -12,6 +12,7 @@ Revises: 0126
"""
from alembic import op
import sqlalchemy as sa
revision = "0127"
down_revision = "0126"
@@ -19,7 +20,19 @@ branch_labels = None
depends_on = None
def _table_exists(conn, table_name: str) -> bool:
"""True when the table exists (dual-path convergence, Gate B)."""
row = conn.execute(
sa.text("SELECT to_regclass(:tname) IS NOT NULL"),
{"tname": f"public.{table_name}"},
).scalar()
return bool(row)
def upgrade() -> None:
conn = op.get_bind()
if not _table_exists(conn, "tasks"):
return
# Drop the FK constraint on tasks.contact_id
op.drop_constraint("tasks_contact_id_fkey", "tasks", type_="foreignkey")
@@ -9,6 +9,7 @@ Revises: 0128
"""
from alembic import op
import sqlalchemy as sa
revision = "0129"
down_revision = "0128"
@@ -27,8 +28,25 @@ TABLES_NEEDING_RLS = [
]
def _table_exists(conn, table_name: str) -> bool:
"""True when the table exists (dual-path convergence, Gate B).
Plugin-owned tables may not exist yet on a fresh install when Alembic
reaches this revision skip them instead of failing. The plugin-side
convergence migrations apply the same RLS policies.
"""
row = conn.execute(
sa.text("SELECT to_regclass(:tname) IS NOT NULL"),
{"tname": f"public.{table_name}"},
).scalar()
return bool(row)
def upgrade() -> None:
conn = op.get_bind()
for table in TABLES_NEEDING_RLS:
if not _table_exists(conn, table):
continue
op.execute(f"ALTER TABLE {table} ENABLE ROW LEVEL SECURITY;")
op.execute(
f"CREATE POLICY tenant_isolation ON {table} "
@@ -37,6 +55,9 @@ def upgrade() -> None:
def downgrade() -> None:
conn = op.get_bind()
for table in TABLES_NEEDING_RLS:
if not _table_exists(conn, table):
continue
op.execute(f"DROP POLICY IF EXISTS tenant_isolation ON {table};")
op.execute(f"ALTER TABLE {table} DISABLE ROW LEVEL SECURITY;")
@@ -10,6 +10,7 @@ Revises: 0135
Create Date: 2026-08-21
"""
from alembic import op
import sqlalchemy as sa
revision = "0136"
down_revision = "0135"
@@ -29,8 +30,25 @@ TABLES_WITH_BAD_RLS = [
]
def _table_exists(conn, table_name: str) -> bool:
"""True when the table exists (dual-path convergence, Gate B).
Plugin-owned tables may not exist yet on a fresh install when Alembic
reaches this revision skip them instead of failing. The plugin-side
convergence migrations apply the same RLS policies.
"""
row = conn.execute(
sa.text("SELECT to_regclass(:tname) IS NOT NULL"),
{"tname": f"public.{table_name}"},
).scalar()
return bool(row)
def upgrade() -> None:
conn = op.get_bind()
for table in TABLES_WITH_BAD_RLS:
if not _table_exists(conn, table):
continue
# Drop old policy with app.tenant_id
op.execute(f"DROP POLICY IF EXISTS tenant_isolation ON {table};")
# Create new policy with app.current_tenant_id
@@ -41,7 +59,10 @@ def upgrade() -> None:
def downgrade() -> None:
conn = op.get_bind()
for table in TABLES_WITH_BAD_RLS:
if not _table_exists(conn, table):
continue
op.execute(f"DROP POLICY IF EXISTS tenant_isolation ON {table};")
op.execute(
f"CREATE POLICY tenant_isolation ON {table} "
@@ -19,7 +19,25 @@ branch_labels = None
depends_on = None
def _table_exists(conn, table_name: str) -> bool:
"""True when the table exists (dual-path convergence, Gate B).
On a fresh install the tags plugin SQL migration has not run yet when
Alembic reaches this revision skip instead of failing. The plugin-side
convergence migration adds the same columns.
"""
row = conn.execute(
sa.text("SELECT to_regclass(:tname) IS NOT NULL"),
{"tname": f"public.{table_name}"},
).scalar()
return bool(row)
def upgrade() -> None:
conn = op.get_bind()
if not _table_exists(conn, "tags"):
return
# parent_id for tree structure (self-referencing FK)
op.add_column("tags", sa.Column("parent_id", PGUUID(as_uuid=True), nullable=True))
op.create_foreign_key(
@@ -35,6 +53,9 @@ def upgrade() -> None:
def downgrade() -> None:
conn = op.get_bind()
if not _table_exists(conn, "tags"):
return
op.drop_column("tags", "icon")
op.drop_column("tags", "applicable_to")
op.drop_index("ix_tags_parent", table_name="tags")
@@ -18,11 +18,31 @@ branch_labels = None
depends_on = None
def _table_exists(conn, table_name: str) -> bool:
"""True when the table exists (dual-path convergence, Gate B).
On a fresh install the report_generator plugin SQL migration has not
run yet when Alembic reaches this revision skip instead of failing.
The plugin-side convergence migration adds the same column.
"""
row = conn.execute(
sa.text("SELECT to_regclass(:tname) IS NOT NULL"),
{"tname": f"public.{table_name}"},
).scalar()
return bool(row)
def upgrade() -> None:
conn = op.get_bind()
if not _table_exists(conn, "report_templates"):
return
op.add_column("report_templates", sa.Column("folder_id", PGUUID(as_uuid=True), nullable=True))
op.create_index("ix_report_templates_folder", "report_templates", ["folder_id"])
def downgrade() -> None:
conn = op.get_bind()
if not _table_exists(conn, "report_templates"):
return
op.drop_index("ix_report_templates_folder", table_name="report_templates")
op.drop_column("report_templates", "folder_id")
@@ -18,11 +18,31 @@ branch_labels = None
depends_on = None
def _table_exists(conn, table_name: str) -> bool:
"""True when the table exists (dual-path convergence, Gate B).
On a fresh install the kommunikation plugin SQL migration has not run
yet when Alembic reaches this revision skip instead of failing.
The plugin-side migration adds the same column idempotently.
"""
row = conn.execute(
sa.text("SELECT to_regclass(:tname) IS NOT NULL"),
{"tname": f"public.{table_name}"},
).scalar()
return bool(row)
def upgrade() -> None:
conn = op.get_bind()
if not _table_exists(conn, "comm_conversations"):
return
op.add_column("comm_conversations", sa.Column("folder_id", PGUUID(as_uuid=True), nullable=True))
op.create_index("ix_comm_conversations_folder", "comm_conversations", ["folder_id"])
def downgrade() -> None:
conn = op.get_bind()
if not _table_exists(conn, "comm_conversations"):
return
op.drop_index("ix_comm_conversations_folder", table_name="comm_conversations")
op.drop_column("comm_conversations", "folder_id")
@@ -0,0 +1,84 @@
'''Fix role permission wildcard patterns to canonical 2-segment schema
Revision ID: 0141
Revises: 0140
Create Date: 2026-08-23
Migration 0019 seeded default roles with 3-segment permission patterns
(core:*:read etc.). The runtime matcher (_matches_permission) compares
segment counts strictly, so those patterns could never match any
2-segment requirement - editor/viewer roles were silently dead.
Canonical schema is module:action (2 segments, * wildcards allowed).
core:*:X means all modules with action X, so it converts to *:X.
'''
from alembic import op
# revision identifiers, used by Alembic.
revision = '0141'
down_revision = '0140'
branch_labels = None
depends_on = None
# Rebuild the permissions JSONB object, rewriting every key that starts
# with the dead 'core:' prefix to its 2-segment equivalent ('*:X').
_UPGRADE_SQL = '''
UPDATE roles
SET permissions = sub.new_perms,
permission_version = permission_version + 1
FROM (
SELECT
r.id AS role_id,
jsonb_object_agg(
CASE WHEN k LIKE 'core:%'
THEN '*:' || split_part(k, ':', 3)
ELSE k END,
v
) AS new_perms
FROM roles r,
jsonb_each(r.permissions) AS e(k, v)
GROUP BY r.id
) AS sub
WHERE roles.id = sub.role_id
AND EXISTS (
SELECT 1 FROM jsonb_object_keys(roles.permissions) k
WHERE k LIKE 'core:%'
)
'''
# Reverse: map '*:X' back to 'core:*:X' only for keys that came from the
# original seeding pattern. Roles that legitimately use '*:X' without a
# matching 'core:*:X' history are left untouched (best-effort downgrade).
_DOWNGRADE_SQL = '''
UPDATE roles
SET permissions = sub.new_perms,
permission_version = permission_version + 1
FROM (
SELECT
r.id AS role_id,
jsonb_object_agg(
CASE WHEN k = '*:' || split_part(k, ':', 2)
AND k <> '*:*'
THEN 'core:*:' || split_part(k, ':', 2)
ELSE k END,
v
) AS new_perms
FROM roles r,
jsonb_each(r.permissions) AS e(k, v)
GROUP BY r.id
) AS sub
WHERE roles.id = sub.role_id
AND EXISTS (
SELECT 1 FROM jsonb_object_keys(roles.permissions) k
WHERE k = '*:' || split_part(k, ':', 2) AND k <> '*:*'
)
'''
def upgrade() -> None:
op.execute(_UPGRADE_SQL)
def downgrade() -> None:
op.execute(_DOWNGRADE_SQL)
@@ -0,0 +1,39 @@
"""Add backup config columns to system_settings table.
Follow-up to 0130: the backup feature (10b1f83) added backup_interval,
backup_retention_days and backup_destination to schema/service/frontend
but missed model columns and this migration.
Revision ID: 0142
Revises: 0141
"""
import sqlalchemy as sa
from alembic import op
revision = "0142"
down_revision = "0141"
branch_labels = None
depends_on = None
def upgrade() -> None:
op.add_column(
"system_settings",
sa.Column("backup_interval", sa.String(20), nullable=False, server_default="daily"),
)
op.add_column(
"system_settings",
sa.Column("backup_retention_days", sa.Integer(), nullable=False, server_default="7"),
)
op.add_column(
"system_settings",
sa.Column("backup_destination", sa.String(20), nullable=False, server_default="local"),
)
def downgrade() -> None:
op.drop_column("system_settings", "backup_destination")
op.drop_column("system_settings", "backup_retention_days")
op.drop_column("system_settings", "backup_interval")
+1 -1
View File
@@ -17,7 +17,7 @@ from __future__ import annotations
import logging
import uuid
from dataclasses import dataclass, field
from dataclasses import dataclass
from typing import Any
from sqlalchemy import select
-2
View File
@@ -25,8 +25,6 @@ import logging
import uuid
from typing import TYPE_CHECKING, Any
from app.core.sensitive_data import sanitize_dict
if TYPE_CHECKING:
from sqlalchemy.ext.asyncio import AsyncSession
-2
View File
@@ -20,8 +20,6 @@ from sqlalchemy.ext.asyncio import AsyncSession
from app.ai.ai_use_case import AIUseCaseMetadata
from app.core.sensitive_data import (
SENSITIVE_FIELDS,
filter_for_llm_context,
get_data_class_for_field,
)
logger = logging.getLogger(__name__)
-1
View File
@@ -6,7 +6,6 @@ import uuid
from dataclasses import dataclass, field
from typing import Any
LOW_CONFIDENCE_THRESHOLD = 0.6
-1
View File
@@ -7,7 +7,6 @@ from typing import Any
from app.ai.knowledge_sources import get_source_config
EXTRACTION_TRIGGERS = {
"mail.received",
"dms.file_uploaded",
+1 -1
View File
@@ -2,7 +2,7 @@
from __future__ import annotations
from dataclasses import dataclass, field
from dataclasses import dataclass
from typing import Any
+4
View File
@@ -107,6 +107,10 @@ class Settings(BaseSettings):
rate_limit_webhook_max: int = 100 # incoming webhooks
rate_limit_webhook_window: int = 60 # 1 minute
# System tenant — used by seeding/plugins that need a well-known default
# tenant (must match scripts/seed_admin.py slug).
system_tenant_slug: str = "default"
# LLM Cost Overrun Protection (B.17)
llm_monthly_budget_usd: float = 100.0 # per-tenant monthly LLM budget
llm_hard_cutoff: bool = True # block LLM calls when budget exceeded
+32
View File
@@ -85,6 +85,38 @@ def generate_csrf_token() -> str:
return secrets.token_urlsafe(32)
async def revoke_user_redis_sessions(user_id: str | uuid.UUID) -> int:
"""Delete every active Redis session belonging to the user (G2).
Shared by both password-change paths (token reset + profile/admin change):
after a password change, stolen or lingering sessions must die.
Returns the number of deleted session keys. Never raises a Redis outage
must not break the password change itself.
"""
try:
redis = get_redis()
deleted = 0
async for key in redis.scan_iter(match="session:*", count=100):
raw = await redis.get(key)
if raw is None:
continue
try:
import json
session_data = json.loads(raw)
except (json.JSONDecodeError, TypeError):
continue
if session_data.get("user_id") == str(user_id):
await redis.delete(key)
deleted += 1
logger.info("Deleted session %s for user %s", key, user_id)
return deleted
except Exception:
logger.warning("Failed to invalidate Redis sessions for user %s", user_id, exc_info=True)
return 0
def hash_token(token: str) -> str:
"""SHA-256 hash a token for storage."""
return hashlib.sha256(token.encode()).hexdigest()
+17
View File
@@ -355,6 +355,23 @@ async def close_engine() -> None:
_migration_session_factory = None
async def get_system_tenant(db: AsyncSession):
"""Return the well-known system tenant, or ``None`` if it does not exist.
Resolves by configured slug (``settings.system_tenant_slug``, default
``"default"`` as created by ``scripts/seed_admin.py``) instead of an
arbitrary first row, so multi-tenant databases stay deterministic.
"""
from sqlalchemy import select
from app.config import get_settings
from app.models.tenant import Tenant # lazy: models import this module's Base
slug = get_settings().system_tenant_slug
result = await db.execute(select(Tenant).where(Tenant.slug == slug).limit(1))
return result.scalar_one_or_none()
def reset_engine_for_testing(engine: AsyncEngine) -> async_sessionmaker[AsyncSession]:
"""Replace all global engines with a test engine. Returns a session factory.
+7 -2
View File
@@ -36,8 +36,13 @@ class EventBus:
self._handlers: dict[str, list[EventHandler]] = defaultdict(list)
def subscribe(self, event_name: str, handler: EventHandler) -> None:
"""Subscribe a handler to an event."""
self._handlers[event_name].append(handler)
"""Subscribe a handler to an event.
Idempotent: subscribing the same handler twice is a no-op
(ARCH-020) so double activation cannot fire handlers twice.
"""
if handler not in self._handlers[event_name]:
self._handlers[event_name].append(handler)
def unsubscribe(self, event_name: str, handler: EventHandler) -> None:
"""Unsubscribe a handler from an event."""
+375
View File
@@ -153,3 +153,378 @@ async def send_password_reset_email(
from app.core.job_registry import register_job # noqa: E402
register_job("send_password_reset_email", send_password_reset_email)
# ── DSAR Processing Job (G1 DSGVO: Art. 15 Auskunft / Art. 17 Löschung) ─────
async def _dsar_collect_user_data(db: Any, tenant_id: str, user_id: str) -> dict[str, Any]:
"""Collect every data category the dsgvo-export route promises.
Shared by type=access (full export) so both paths stay consistent.
"""
from datetime import UTC, datetime
from uuid import UUID as PyUUID
from sqlalchemy import or_ as sa_or_
from sqlalchemy import select as sa_select
from app.models.audit import AuditLog
from app.models.contact import Contact
from app.models.notification import Notification
from app.models.user import User
uid = PyUUID(user_id)
tid = PyUUID(tenant_id)
export_data: dict[str, Any] = {
"user_id": user_id,
"exported_at": datetime.now(UTC).isoformat(),
"legal_basis": "GDPR Art. 15 (access) / Art. 20 (portability)",
"data": {},
}
# Profile
user = (
await db.execute(sa_select(User).where(User.id == uid))
).scalar_one_or_none()
if user:
export_data["data"]["profile"] = {
"email": user.email,
"name": user.name,
"is_active": user.is_active,
"created_at": user.created_at.isoformat() if user.created_at else None,
}
# Contacts owned by the user
contacts = (
await db.execute(
sa_select(Contact).where(
Contact.tenant_id == tid,
Contact.owner_id == uid,
Contact.deleted_at.is_(None),
)
)
).scalars().all()
export_data["data"]["contacts"] = [
{
"id": str(c.id),
"type": c.type,
"displayname": c.displayname,
"email_1": c.email_1,
"email_2": c.email_2,
}
for c in contacts
]
# Audit trail entries by/about the user (bounded to keep payloads sane)
audit_entries = (
await db.execute(
sa_select(AuditLog).where(
AuditLog.tenant_id == tid,
AuditLog.user_id == uid,
).limit(1000)
)
).scalars().all()
export_data["data"]["audit_log"] = [
{
"action": a.action,
"entity_type": a.entity_type,
"timestamp": a.timestamp.isoformat() if a.timestamp else None,
}
for a in audit_entries
]
# Notifications addressed to the user
notifications = (
await db.execute(
sa_select(Notification).where(
Notification.tenant_id == tid,
Notification.owner_id == uid,
).limit(1000)
)
).scalars().all()
export_data["data"]["notifications"] = [
{
"id": str(n.id),
"type": getattr(n, "type", None),
"title": getattr(n, "title", None),
"created_at": n.created_at.isoformat() if n.created_at else None,
}
for n in notifications
]
# ── Categories promised by the dsgvo-export route docstring ──
# (G1-b: mail accounts, tasks, calendar entries, comm messages)
try:
from app.plugins.builtins.mail.models import MailAccount
mail_accounts = (
await db.execute(
sa_select(MailAccount).where(
MailAccount.tenant_id == tid,
MailAccount.user_id == uid,
).limit(500)
)
).scalars().all()
export_data["data"]["mail_accounts"] = [
{
"id": str(a.id),
"email_address": a.email_address,
"display_name": a.display_name,
"is_shared": a.is_shared,
"is_active": a.is_active,
}
for a in mail_accounts
]
except ImportError:
pass
try:
from app.plugins.builtins.tasks.models import Task as TaskModel
tasks = (
await db.execute(
sa_select(TaskModel).where(
sa_or_(TaskModel.owner_id == uid, TaskModel.assigned_to == uid),
TaskModel.tenant_id == tid,
TaskModel.deleted_at.is_(None),
).limit(1000)
)
).scalars().all()
export_data["data"]["tasks"] = [
{
"id": str(t.id),
"title": t.title,
"status": t.status,
"priority": t.priority,
"due_date": t.due_date.isoformat() if t.due_date else None,
}
for t in tasks
]
except ImportError:
pass
try:
from app.plugins.builtins.calendar.models import CalendarEntry as CalEntry
cal_entries = (
await db.execute(
sa_select(CalEntry).where(
CalEntry.tenant_id == tid,
CalEntry.owner_id == uid,
CalEntry.deleted_at.is_(None),
).limit(1000)
)
).scalars().all()
export_data["data"]["calendar_entries"] = [
{
"id": str(e.id),
"title": e.title,
"entry_type": e.entry_type,
"start_at": e.start_at.isoformat() if e.start_at else None,
"end_at": e.end_at.isoformat() if e.end_at else None,
}
for e in cal_entries
]
except ImportError:
pass
try:
from app.plugins.builtins.kommunikation.models import CommMessage
comm_messages = (
await db.execute(
sa_select(CommMessage).where(
CommMessage.sender_id == uid,
CommMessage.tenant_id == tid,
).limit(1000)
)
).scalars().all()
export_data["data"]["comm_messages"] = [
{
"id": str(m.id),
"sender_type": m.sender_type,
"content": m.content[:500],
"created_at": m.created_at.isoformat() if m.created_at else None,
}
for m in comm_messages
]
except ImportError:
pass
return export_data
async def _dsar_execute_deletion(db: Any, tenant_id: str, user_id: str) -> dict[str, int]:
"""Execute GDPR Art. 17 erasure for a user within one tenant.
Strategy (respects retention duties):
- Contacts owned by the user soft-delete via deleted_at
(audit history must remain intact it is not personal data of the
subject but business record; retention policy governs its cleanup)
- Notifications owned by the user hard delete
- User account deactivate (is_active=False), clear personal fields,
scramble password hash and email (keeps FK integrity for audit rows)
Returns counters for the audit entry.
"""
from datetime import UTC, datetime
from uuid import UUID as PyUUID
from sqlalchemy import select as sa_select
from sqlalchemy import update as sa_update
from app.core.audit import log_audit
from app.models.contact import Contact
from app.models.notification import Notification
from app.models.user import User
uid = PyUUID(user_id)
tid = PyUUID(tenant_id)
counts: dict[str, int] = {}
# 1. Soft-delete contacts owned by the user
contact_result = await db.execute(
sa_select(Contact).where(
Contact.tenant_id == tid,
Contact.owner_id == uid,
Contact.deleted_at.is_(None),
)
)
contacts = contact_result.scalars().all()
for c in contacts:
c.deleted_at = datetime.now(UTC)
counts["contacts_soft_deleted"] = len(contacts)
# 2. Hard-delete notifications owned by the user
notif_result = await db.execute(
sa_select(Notification).where(
Notification.tenant_id == tid,
Notification.owner_id == uid,
)
)
notifications = notif_result.scalars().all()
for n in notifications:
await db.delete(n)
counts["notifications_deleted"] = len(notifications)
# 3. Anonymize + deactivate the account (FK integrity for audit rows kept)
await db.execute(
sa_update(User)
.where(User.id == uid)
.values(
email=f"erased.{uid.hex[:16]}@anonymized.invalid",
name="[gelöscht gemäß DSGVO Art. 17]",
first_name=None,
last_name=None,
avatar_url=None,
password_hash="!dsar-erased",
is_active=False,
preferences={},
)
)
counts["user_anonymized"] = 1
# 4. Audit the erasure itself (who/what/when — required by Art. 17 recital)
await log_audit(
db,
tid,
user_id,
"dsar_erasure",
"user",
uid,
{"target_user": user_id, **counts},
)
return counts
async def process_dsar(
ctx: dict[str, Any],
*,
user_id: str,
tenant_id: str,
request_type: str,
) -> dict[str, Any]:
"""Process a GDPR Data Subject Access Request (DSAR).
ARQ worker function registered as "process_dsar".
request_type:
- "access": collect all data categories (Art. 15/20) and post a system
message that the export is ready (served via the existing dsgvo-export
endpoint).
- "deletion": execute Art. 17 erasure (soft-delete contacts, hard-delete
notifications, anonymize+deactivate account) and audit it.
- "rectification": post a system message asking admins to handle the
correction manually.
Returns a summary dict for the job result.
"""
import logging
import uuid as uuid_module
from app.core.db import get_worker_session_factory
from app.core.notifications import post_system_message
logger = logging.getLogger(__name__)
tid = uuid_module.UUID(tenant_id)
uid = uuid_module.UUID(user_id)
factory = get_worker_session_factory()
async with factory() as db:
try:
if request_type == "access":
data = await _dsar_collect_user_data(db, tenant_id, user_id)
await db.commit()
categories = list(data.get("data", {}).keys())
await post_system_message(
db,
tid,
uid,
"dsar_access_ready",
"DSGVO-Auskunft bereit",
f"Datenkategorien: {', '.join(categories)}",
severity="info",
)
await db.commit()
logger.info("DSAR access processed for user %s", user_id)
return {"type": request_type, "status": "completed", "categories": categories}
if request_type == "deletion":
counts = await _dsar_execute_deletion(db, tenant_id, user_id)
await db.commit()
await post_system_message(
db,
tid,
uid,
"dsar_deletion_done",
"DSGVO-Löschung ausgeführt",
f"Kontakten soft-gelöscht: {counts.get('contacts_soft_deleted', 0)}; Konto anonymisiert.",
severity="info",
)
await db.commit()
logger.info("DSAR deletion executed for user %s: %s", user_id, counts)
return {"type": request_type, "status": "completed", **counts}
if request_type == "rectification":
await post_system_message(
db,
tid,
uid,
"dsar_rectification_requested",
"DSGVO-Berichtigung angefordert",
f"Manuelle Bearbeitung für User {user_id} erforderlich.",
severity="warning",
)
await db.commit()
logger.info("DSAR rectification requested for user %s", user_id)
return {"type": request_type, "status": "queued_for_manual_handling"}
logger.warning("Unknown DSAR request_type '%s' for user %s", request_type, user_id)
return {"type": request_type, "status": "unknown_type"}
except Exception:
await db.rollback()
raise
register_job("process_dsar", process_dsar)
+96
View File
@@ -3,6 +3,8 @@
from __future__ import annotations
import logging
import re
import uuid as uuid_mod
from fastapi import Request, status
from starlette.middleware.base import BaseHTTPMiddleware
@@ -137,3 +139,97 @@ class CSRFMiddleware(BaseHTTPMiddleware):
pass
return await call_next(request)
class AuditMiddleware(BaseHTTPMiddleware):
"""Safety-net audit trail for ALL successful mutating requests.
AGENTS.md requires every mutation to produce an audit entry. Explicit
``log_audit`` calls in routes/services remain the detail layer (entity ids,
change diffs); this middleware guarantees a baseline entry for mutations
that lack one, marked with ``source=middleware`` in ``details``.
Best-effort by design: audit failures never break the request.
"""
_MUTATING = {"POST", "PUT", "PATCH", "DELETE"}
_SKIP_PREFIXES = (
"/api/v1/auth",
"/api/v1/health",
"/api/v1/errors",
"/api/v1/audit",
"/api/v1/external",
)
async def dispatch(self, request: Request, call_next):
response = await call_next(request)
if request.method not in self._MUTATING:
return response
if response.status_code < 200 or response.status_code >= 300:
return response
path = request.url.path
if any(path.startswith(p) for p in self._SKIP_PREFIXES):
return response
try:
await self._write_entry(request, path, response.status_code)
except Exception:
logging.getLogger(__name__).debug(
"AuditMiddleware: failed to write baseline entry for %s %s", request.method, path
)
return response
@staticmethod
def _derive_entity_type(path: str) -> str:
"""Derive an entity_type from the second URL segment."""
parts = [p for p in path.split("/") if p]
# /api/v1/<resource>/... -> resource; singularize naive trailing 's'
resource = parts[2] if len(parts) > 2 and parts[0] == "api" and parts[1] == "v1" else (parts[0] if parts else "unknown")
return resource[:-1] if len(resource) > 3 and resource.endswith("s") else resource
async def _write_entry(self, request: Request, path: str, status_code: int) -> None:
from app.core.audit import log_audit
from app.core.auth import get_redis, get_session_data
from app.core.db import create_db_session
# Attribute via the Redis session (same source as CSRFMiddleware) —
# FastAPI dependencies run after middleware, so request.state is empty here.
settings = get_settings()
session_id = request.cookies.get(settings.session_cookie_name)
if not session_id:
return # unauthenticated — nothing to attribute
redis = get_redis()
session_data = await get_session_data(redis, session_id)
if not session_data:
return
tenant_raw = session_data.get("tenant_id")
user_raw = session_data.get("user_id")
if not tenant_raw:
return
action_map = {"POST": "create", "PATCH": "update", "PUT": "update", "DELETE": "delete"}
entity_id: uuid_mod.UUID | None = None
parts = [p for p in path.split("/") if p]
if parts and re.fullmatch(r"[0-9a-fA-F-]{36}", parts[-1]):
try:
entity_id = uuid_mod.UUID(parts[-1])
except ValueError:
entity_id = None
async with create_db_session(uuid_mod.UUID(tenant_raw)) as db:
await log_audit(
db,
uuid_mod.UUID(tenant_raw),
uuid_mod.UUID(user_raw) if user_raw else None,
action_map.get(request.method, request.method.lower()),
self._derive_entity_type(path),
entity_id,
changes={
"source": "middleware",
"method": request.method,
"path": path,
"status": status_code,
},
)
await db.commit()
+2
View File
@@ -54,6 +54,8 @@ CORE_PERMISSIONS: list[dict[str, str]] = [
{"key": "taxes:write", "label": "Taxes: Write", "category": "core", "module": "taxes"},
{"key": "currencies:read", "label": "Currencies: Read", "category": "core", "module": "currencies"},
{"key": "currencies:write", "label": "Currencies: Write", "category": "core", "module": "currencies"},
{"key": "custom_fields:read", "label": "Custom Fields: Read", "category": "core", "module": "custom_fields"},
{"key": "custom_fields:write", "label": "Custom Fields: Write", "category": "core", "module": "custom_fields"},
{"key": "import_export:read", "label": "Import/Export: Read", "category": "core", "module": "import_export"},
{"key": "import_export:write", "label": "Import/Export: Write", "category": "core", "module": "import_export"},
{"key": "workspaces:read", "label": "Workspaces: Read", "category": "core", "module": "workspaces"},
+8
View File
@@ -29,6 +29,14 @@ class ServiceContainer:
"""Check if a service is registered."""
return name in self._services
def remove(self, name: str) -> None:
"""Remove a service registration (no-op if absent).
Used by plugin deactivation hooks to clean up services they
registered during activation.
"""
self._services.pop(name, None)
async def initialize(self) -> None:
"""Initialize core services."""
if self._initialized:
+40 -12
View File
@@ -501,11 +501,38 @@ async def save_with_metadata(
}
async def get_file_metadata_async(path: str) -> dict[str, Any]:
"""Awaitable variant of :func:`get_file_metadata` (ARCH-052).
Safe to call from inside a running event loop never creates a
nested one. For local storage this is plain filesystem access; for
S3 and other async backends the backend's ``exists()`` is awaited.
"""
backend = get_storage_backend()
if isinstance(backend, LocalStorage):
full_path = backend._full_path(path)
if not os.path.exists(full_path):
return {"size": None, "modified": None, "exists": False}
stat = os.stat(full_path)
return {
"size": stat.st_size,
"modified": stat.st_mtime,
"exists": True,
}
# S3 or other async backends — await the backend directly
if not await backend.exists(path):
return {"size": None, "modified": None, "exists": False}
return {"size": None, "modified": None, "exists": True}
def get_file_metadata(path: str) -> dict[str, Any]:
"""Read metadata of a stored file without loading its content.
Works with the *local* storage backend. For S3, use the S3 client
``stat_object`` API directly.
Works with the *local* storage backend without touching the event
loop. For S3 and other async-only backends this drives the check
through ``asyncio.run``; calling it from inside a running event loop
raises ``RuntimeError`` use :func:`get_file_metadata_async` there
instead (ARCH-052).
Parameters
----------
@@ -530,14 +557,15 @@ def get_file_metadata(path: str) -> dict[str, Any]:
"modified": stat.st_mtime,
"exists": True,
}
# S3 or other backends — fall back to exists() check
import asyncio as _asyncio
loop = _asyncio.new_event_loop()
# Async-only backend outside a running loop is fine; inside one we
# must never build a nested event loop.
try:
exists = loop.run_until_complete(backend.exists(path))
if not exists:
return {"size": None, "modified": None, "exists": False}
return {"size": None, "modified": None, "exists": True}
finally:
loop.close()
asyncio.get_running_loop()
except RuntimeError:
pass
else:
raise RuntimeError(
"get_file_metadata() cannot be used with async storage backends "
"inside a running event loop — use get_file_metadata_async()"
)
return asyncio.run(get_file_metadata_async(path))
+4 -1
View File
@@ -121,11 +121,14 @@ class TriggerDispatcher:
"""Query DB for active automations matching *event_name* and dispatch."""
from app.core.db import get_session_factory
from app.plugins.builtins.contracts import get_contract
# None-check FIRST — accessing attributes on the contract before the
# check crashed with AttributeError when automation was inactive
# (ARCH-029/041).
automation_contract = get_contract("automation")
AutomationDefinition = automation_contract.Automation # noqa: N806
if automation_contract is None:
logger.debug("Automation plugin not available — trigger skipped")
return
AutomationDefinition = automation_contract.Automation # noqa: N806
factory = get_session_factory()
tenant_id = payload.get("tenant_id")
+11 -12
View File
@@ -170,12 +170,7 @@ async def on_startup(ctx: dict[str, Any]) -> None:
if search_contract is not None:
factory = async_session
async with factory() as db:
# auto_register_providers is not exposed via contract yet;
# use the contract's get_search_registry to access providers
from app.plugins.builtins.unified_search.provider_registry import (
auto_register_providers,
)
await auto_register_providers(db)
await search_contract.auto_register_providers(db)
logger.info("Search providers registered for worker")
else:
logger.debug("Unified search plugin not available — skipping provider registration")
@@ -350,8 +345,10 @@ async def cleanup_audit_log_job(ctx: dict[str, Any]) -> None:
Runs daily to prevent the audit_log table from growing indefinitely.
Iterates per-tenant for RLS compliance.
"""
from sqlalchemy import text as sa_text, delete as sa_delete
from datetime import datetime, timedelta
from datetime import UTC, datetime, timedelta
from sqlalchemy import delete as sa_delete
from sqlalchemy import text as sa_text
from app.core.db import get_worker_session_factory
from app.models.audit import AuditLog
@@ -362,7 +359,7 @@ async def cleanup_audit_log_job(ctx: dict[str, Any]) -> None:
tenant_result = await db.execute(sa_text("SELECT id FROM tenants"))
tenant_ids = [row[0] for row in tenant_result]
cutoff = datetime.utcnow() - timedelta(days=365)
cutoff = datetime.now(UTC) - timedelta(days=365)
total_deleted = 0
for tenant_id in tenant_ids:
await db.execute(
@@ -393,8 +390,10 @@ async def cleanup_trash_job(ctx: dict[str, Any]) -> None:
Runs daily to clean up the trash. Iterates per-tenant for RLS compliance.
Default retention: 90 days in trash before permanent deletion.
"""
from sqlalchemy import text as sa_text, delete as sa_delete
from datetime import datetime, timedelta
from datetime import UTC, datetime, timedelta
from sqlalchemy import delete as sa_delete
from sqlalchemy import text as sa_text
from app.core.db import get_worker_session_factory
from app.models.contact import Contact
@@ -406,7 +405,7 @@ async def cleanup_trash_job(ctx: dict[str, Any]) -> None:
tenant_result = await db.execute(sa_text("SELECT id FROM tenants"))
tenant_ids = [row[0] for row in tenant_result]
cutoff = datetime.utcnow() - timedelta(days=90)
cutoff = datetime.now(UTC) - timedelta(days=90)
total_deleted = 0
for tenant_id in tenant_ids:
+28 -3
View File
@@ -17,8 +17,9 @@ from app.core.db import get_db, set_tenant_context, set_user_context
logger = logging.getLogger(__name__)
# Known write-permission modules — used by require_write() to check
# specific permissions instead of broad wildcards like *:write
# Legacy fallback list — used by require_write() only when the permission
# registry is not initialized. The live source of truth is generated from
# the registry (see _get_write_permissions, ARCH-022).
_WRITE_PERMISSIONS = [
"users:write",
"roles:write",
@@ -35,6 +36,30 @@ _WRITE_PERMISSIONS = [
]
def _get_write_permissions() -> list[str]:
"""Return all known ``module:write`` permission keys (ARCH-022).
Generated from the permission registry so plugin write permissions are
picked up automatically without touching this file. Falls back to the
static legacy list when the registry is unavailable/uninitialized.
"""
try:
from app.core.permission_registry import get_permission_registry
registry = get_permission_registry()
if getattr(registry, "_initialized", False):
perms = [
entry["key"]
for entry in registry.get_all()
if entry["key"].endswith(":write")
]
if perms:
return sorted(perms)
except Exception:
pass
return list(_WRITE_PERMISSIONS)
async def get_redis_dep() -> aioredis.Redis:
"""FastAPI dependency for Redis client."""
return get_redis()
@@ -261,7 +286,7 @@ async def require_write(
# Check via permission system for specific write permissions
from app.core.permissions import check_permission
for perm in _WRITE_PERMISSIONS:
for perm in _get_write_permissions():
if check_permission(current_user, perm):
return current_user
+41 -25
View File
@@ -22,8 +22,12 @@ logger = logging.getLogger(__name__)
from app.config import get_settings # noqa: E402
from app.core.db import close_engine, get_engine # noqa: E402
from app.core.error_codes import ApiError, build_error_response # noqa: E402
from app.core.middleware import CSRFMiddleware, SecurityHeadersMiddleware # noqa: E402
from app.core.error_codes import ERROR_CODES, ApiError, build_error_response # noqa: E402
from app.core.middleware import ( # noqa: E402
AuditMiddleware,
CSRFMiddleware,
SecurityHeadersMiddleware,
)
from app.core.monitoring import record_error, record_request # noqa: E402
from app.core.rate_limit import GeneralRateLimitMiddleware # noqa: E402
from app.core.resilience import CircuitBreakerMiddleware # noqa: E402
@@ -36,16 +40,14 @@ from app.routes import ( # noqa: E402
attachments,
audit,
auth,
compliance,
backups,
bank_accounts,
contact_folder_permissions,
contact_folders,
contacts,
compliance,
currencies,
custom_field_definitions,
custom_fields,
dashboard,
delegations,
entity_history,
entity_permissions,
errors,
@@ -59,7 +61,6 @@ from app.routes import ( # noqa: E402
owner_transfer,
permission_templates,
plugins,
delegations,
policies,
roles,
saved_filters,
@@ -284,22 +285,23 @@ async def lifespan(app: FastAPI):
logger.info(f"Plugin {name} is inactive — skipping activation")
continue
# Activate plugin with a FRESH session per plugin to avoid RLS state leakage
# RLS fail-closed requires app.current_tenant_id for tenant-table writes.
# Plugin activation may fail on duplicate cron job inserts — this is harmless
# since cron jobs already exist from previous startups.
# Activate plugin ONCE per process (ARCH-002 fix): a fresh session with
# the first tenant's RLS context satisfies fail-closed RLS for any
# tenant-table writes during activation. Plugins that need per-tenant
# data must seed it themselves (e.g. via the default-tenant mechanism).
# Calling on_activate once prevents duplicate event listeners, cron
# jobs, mini-apps and other contributions at multi-tenant startups.
plugin_activated = False
for tenant_id in all_tenant_ids:
if all_tenant_ids:
try:
async with async_session() as plugin_db:
await set_tenant_context(plugin_db, tenant_id)
await set_tenant_context(plugin_db, all_tenant_ids[0])
await plugin.on_activate(plugin_db, container, event_bus)
await plugin_db.flush()
await plugin_db.commit()
plugin_activated = True
except Exception as exc:
logger.warning(f"[STARTUP] Plugin {name} activation issue for tenant {tenant_id}: {exc}")
break
logger.warning(f"[STARTUP] Plugin {name} activation issue: {exc}")
if plugin_activated:
plugin_record.status = "active"
@@ -474,6 +476,7 @@ def create_app() -> FastAPI:
)
app.add_middleware(CSRFMiddleware)
app.add_middleware(SecurityHeadersMiddleware)
app.add_middleware(AuditMiddleware)
app.add_middleware(GeneralRateLimitMiddleware)
app.add_middleware(RequestLoggingMiddleware)
app.add_middleware(CircuitBreakerMiddleware)
@@ -516,11 +519,26 @@ def create_app() -> FastAPI:
504: "service_timeout",
}
code = status_to_code.get(exc.status_code, "internal_error" if exc.status_code >= 500 else "validation_error")
body = build_error_response(
code=code,
detail=str(exc.detail) if exc.detail else None,
trace_id=trace_id,
)
# Structured detail passthrough (AGENTS.md): when a route raises
# HTTPException with a dict detail containing a machine-readable ``code``,
# preserve the structured shape instead of stringifying it.
raw_detail = exc.detail
if isinstance(raw_detail, dict):
inner_code = raw_detail.get("code", code)
body = build_error_response(
code=inner_code if inner_code in ERROR_CODES else code,
detail=raw_detail.get("detail") or str(raw_detail),
trace_id=trace_id,
)
# Preserve the full structured detail as a nested object so clients
# can read ``resp.json()["detail"]["code"]``.
body["detail"] = raw_detail
else:
body = build_error_response(
code=code,
detail=str(exc.detail) if exc.detail else None,
trace_id=trace_id,
)
resp = JSONResponse(status_code=exc.status_code, content=body)
if trace_id:
resp.headers["X-Trace-Id"] = trace_id
@@ -544,11 +562,9 @@ def create_app() -> FastAPI:
app.include_router(groups.router)
app.include_router(tenants.router)
app.include_router(notifications.router)
from app.routes.companies import router as companies_router
app.include_router(companies_router)
app.include_router(contacts.router)
app.include_router(contact_folders.router)
app.include_router(contact_folder_permissions.router)
# NOTE: contacts/companies/contact-folders routes are plugin-owned now
# (Block B1) and mounted via the manifest.routes mechanism below with
# require_active_plugin("contacts") protection.
app.include_router(entity_permissions.router)
app.include_router(dashboard.router)
app.include_router(entity_history.router)
+1 -1
View File
@@ -6,8 +6,8 @@ from app.models.audit import AuditLog
from app.models.auth import ApiToken, PasswordResetToken
from app.models.backup import Backup
from app.models.bank_account import BankAccount
from app.models.consumer_inbox import ConsumerInbox
from app.models.compliance import ComplianceIncident
from app.models.consumer_inbox import ConsumerInbox
from app.models.contact import Contact, ContactPerson
from app.models.contact_folder import ContactFolder
from app.models.contact_merge import ContactMergeHistory
+1 -2
View File
@@ -11,13 +11,12 @@ from datetime import datetime
from typing import Any
from sqlalchemy import DateTime, ForeignKey, String, func
from sqlalchemy.dialects.postgresql import JSONB
from sqlalchemy.dialects.postgresql import JSONB, TSVECTOR
from sqlalchemy.dialects.postgresql import UUID as PGUUID
from sqlalchemy.orm import Mapped, mapped_column
from app.core.db import Base, TenantMixin
from app.models.owned_mixin import OwnedMixin
from sqlalchemy.dialects.postgresql import TSVECTOR
# Re-export EntityHistory as DeletionLog for backward compatibility.
# Tests import DeletionLog from app.models.audit and use entity_snapshot attribute.
+47 -1
View File
@@ -5,7 +5,7 @@ from __future__ import annotations
import uuid
from datetime import datetime
from sqlalchemy import DateTime, Integer, String, Text, func
from sqlalchemy import DateTime, ForeignKey, Integer, String, Text, UniqueConstraint, func
from sqlalchemy.dialects.postgresql import JSONB
from sqlalchemy.dialects.postgresql import UUID as PGUUID
from sqlalchemy.orm import Mapped, mapped_column
@@ -77,3 +77,49 @@ class EventOutbox(Base):
failed_at: Mapped[datetime | None] = mapped_column(
DateTime(timezone=True), nullable=True,
)
class OutboxDelivery(Base):
"""Per-consumer delivery status for an event_outbox row (Migration 0075).
Tracks whether each consumer successfully processed an event; an event is
only 'published' when all mandatory deliveries succeed.
"""
__tablename__ = "outbox_deliveries"
__table_args__ = (
UniqueConstraint(
"event_id", "consumer_name",
name="uq_outbox_deliveries_event_consumer",
),
)
id: Mapped[uuid.UUID] = mapped_column(
PGUUID(as_uuid=True), primary_key=True,
server_default=func.gen_random_uuid(),
)
event_id: Mapped[uuid.UUID] = mapped_column(
PGUUID(as_uuid=True),
ForeignKey("event_outbox.id", ondelete="CASCADE"),
nullable=False,
)
consumer_name: Mapped[str] = mapped_column(String(150), nullable=False)
status: Mapped[str] = mapped_column(
String(30), nullable=False, server_default="pending",
)
attempt_count: Mapped[int] = mapped_column(
Integer, nullable=False, server_default="0",
)
next_attempt_at: Mapped[datetime | None] = mapped_column(
DateTime(timezone=True), nullable=True,
)
last_error: Mapped[str | None] = mapped_column(Text, nullable=True)
processed_at: Mapped[datetime | None] = mapped_column(
DateTime(timezone=True), nullable=True,
)
created_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), nullable=False, server_default=func.now(),
)
updated_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), nullable=False, server_default=func.now(),
)
+3
View File
@@ -53,6 +53,9 @@ class SystemSettings(Base, TenantMixin, OwnedMixin):
theme_border_radius: Mapped[str] = mapped_column(String(20), nullable=False, default="0.5rem")
# Backup configuration
backup_enabled: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False, server_default="false")
backup_interval: Mapped[str] = mapped_column(String(20), nullable=False, default="daily", server_default="daily")
backup_retention_days: Mapped[int] = mapped_column(Integer, nullable=False, default=7, server_default="7")
backup_destination: Mapped[str] = mapped_column(String(20), nullable=False, default="local", server_default="local")
# Automation plugin settings (JSONB)
automation_config: Mapped[dict | None] = mapped_column(JSONB, nullable=True)
# Retention policy overrides (JSONB) — compliance module
-1
View File
@@ -12,7 +12,6 @@ from sqlalchemy.dialects.postgresql import UUID as PGUUID
from sqlalchemy.orm import Mapped, mapped_column
from app.core.db import Base, SoftDeleteMixin, TimestampMixin
from app.models.owned_mixin import OwnedMixin
class User(Base, TimestampMixin, SoftDeleteMixin):
+9
View File
@@ -97,6 +97,15 @@ class BasePlugin(ABC):
"""
return []
async def register_event_handlers(self, event_bus: EventBus) -> None:
"""Register event handlers for the background worker (ARCH-038 hook).
The worker calls this on every active plugin at startup so plugins
can subscribe to events even when the web process is separate.
Default: no-op. Override to subscribe handlers.
"""
return None
# ─── Job Modules ───
def get_job_modules(self) -> list[str]:
@@ -12,7 +12,6 @@ from sqlalchemy.orm import Mapped, mapped_column
from app.core.db import Base, TenantMixin
from app.models.owned_mixin import OwnedMixin
from pgvector.sqlalchemy import Vector
class AgentMemory(Base, TenantMixin, OwnedMixin):
@@ -22,6 +22,7 @@ from app.plugins.builtins.ai_assistant.schemas import (
ExternalAgentRequest,
ExternalAgentResponse,
)
from app.plugins.builtins.ai_assistant.services import stream_chat_comm as stream_chat
logger = logging.getLogger(__name__)
@@ -120,13 +121,18 @@ async def run_agent_external(
}
# Run the agent via streaming chat (non-streaming mode)
from app.plugins.builtins.ai_assistant.services import stream_chat_comm
full_response = ""
async with get_db() as stream_db:
await set_tenant_context(stream_db, tenant_id)
async for chunk in stream_chat(
stream_db, session, agent, data.message, user_context, tenant_id
stream_db,
session.id,
agent,
data.message,
user_context,
tenant_id,
uuid.UUID(current_user["user_id"]),
):
if chunk.startswith("data: ") and chunk != "data: [DONE]\n\n":
try:
@@ -0,0 +1,10 @@
-- Dual-path convergence (Gate B): add compliance columns that Alembic
-- migration 0119 adds on the core path. Idempotent so both install paths
-- converge to the identical schema.
ALTER TABLE ai_providers ADD COLUMN IF NOT EXISTS region VARCHAR(20) NOT NULL DEFAULT 'unknown';
ALTER TABLE ai_providers ADD COLUMN IF NOT EXISTS hosting_type VARCHAR(30) NOT NULL DEFAULT 'cloud';
ALTER TABLE ai_providers ADD COLUMN IF NOT EXISTS dpa_status VARCHAR(20) NOT NULL DEFAULT 'none';
ALTER TABLE ai_providers ADD COLUMN IF NOT EXISTS retention_policy TEXT NOT NULL DEFAULT '';
ALTER TABLE ai_providers ADD COLUMN IF NOT EXISTS training_on_customer_data BOOLEAN NOT NULL DEFAULT FALSE;
ALTER TABLE ai_providers ADD COLUMN IF NOT EXISTS transfer_notice TEXT NOT NULL DEFAULT '';
ALTER TABLE ai_providers ADD COLUMN IF NOT EXISTS allowed_data_classes JSONB NOT NULL DEFAULT '[]'::jsonb;
+3 -3
View File
@@ -42,7 +42,7 @@ class AIAssistantPlugin(BasePlugin):
),
],
events=[],
migrations=["0001_initial.sql", "0002_folders_attachments.sql"],
migrations=["0001_initial.sql", "0002_folders_attachments.sql", "0003_sort_order.sql", "0004_compliance_fields.sql"],
permissions=[
"ai:read",
"ai:write",
@@ -52,10 +52,10 @@ class AIAssistantPlugin(BasePlugin):
],
is_core=True,
menu_items=[
FrontendMenuItem(label_key='nav.aiAssistant', label='KI Assistent', path='/ai-assistant', icon='Bot', order=90),
FrontendMenuItem(label_key='nav.aiAssistant', label='KI Assistent', path='/ai-assistant', icon='Bot', order=90, permission='ai:read'),
],
page_routes=[
FrontendPageRoute(path='/ai-assistant', component='@/pages/AIAssistant', protected=True),
FrontendPageRoute(path='/ai-assistant', component='@/pages/AIAssistant', protected=True, permission='ai:read'),
],
settings_pages=[
FrontendSettingsPage(path='ai', label_key='settings.ai', label='AI Settings', component='@/pages/AISettings', icon='Bot', order=60),
@@ -165,8 +165,8 @@ async def get_open_tasks_handler(arguments: dict[str, Any], context: dict[str, A
from app.plugins.builtins.calendar.contracts import get_contract as get_calendar_contract
_cal = get_calendar_contract()
calendar_entry = _cal.calendar_entry
calendar_entry_link = _cal.calendar_entry_link
calendar_entry = _cal.CalendarEntry
calendar_entry_link = _cal.CalendarEntryLink
db, tenant_id, _ = await _get_db_and_tenant(context)
entity_type = arguments["entity_type"]
@@ -69,10 +69,12 @@ async def push_suggestion(user_id: str, suggestion: dict[str, Any]) -> None:
# Post suggestion to Communication (I-WORK-PROACTIVE)
try:
import uuid as uuid_mod
from sqlalchemy import select as sa_select
from app.core.db import get_worker_session_factory
from app.plugins.builtins.contracts import get_contract_registry
from app.plugins.builtins.kommunikation.models import CommConversation
from sqlalchemy import select as sa_select
from app.core.db import get_worker_session_factory
komm = get_contract_registry().get("kommunikation")
if komm:
factory = get_worker_session_factory()
@@ -274,8 +276,8 @@ async def gather_context(
# Upcoming calendar events
from app.plugins.builtins.calendar.contracts import get_contract as get_calendar_contract
_cal = get_calendar_contract()
calendar_entry = _cal.calendar_entry
calendar_entry_link = _cal.calendar_entry_link
calendar_entry = _cal.CalendarEntry
calendar_entry_link = _cal.CalendarEntryLink
now = datetime.now(UTC)
event_result = await db.execute(
@@ -389,8 +391,8 @@ async def gather_context(
# Upcoming events
from app.plugins.builtins.calendar.contracts import get_contract as get_calendar_contract
_cal = get_calendar_contract()
calendar_entry = _cal.calendar_entry
calendar_entry_link = _cal.calendar_entry_link
calendar_entry = _cal.CalendarEntry
calendar_entry_link = _cal.CalendarEntryLink
now = datetime.now(UTC)
event_result = await db.execute(
+4 -1
View File
@@ -62,7 +62,10 @@ class AIUIControlPlugin(BasePlugin):
from app.plugins.builtins.contracts import get_contract_registry
get_contract_registry().unregister(self.manifest.name)
await super().on_deactivate(db, service_container, event_bus)
# Remove the WebSocket manager BEFORE super() so that event handlers
# being unsubscribed can no longer reach it (ARCH-044).
if service_container.has("ai_ui_control_ws"):
service_container.remove("ai_ui_control_ws")
logger.info("AI UI Control WebSocket manager removed")
await super().on_deactivate(db, service_container, event_bus)
@@ -55,7 +55,8 @@ async def send_agent_message(
# 2. Create a kommunikation message in a dedicated agent room
try:
from app.plugins.builtins.kommunikation.contracts import CommConversation as Room, CommMessage as Message
from app.plugins.builtins.kommunikation.contracts import CommConversation as Room
from app.plugins.builtins.kommunikation.contracts import CommMessage as Message
# Find or create the agent-to-agent room
room_name = f"agent:{from_agent_id}:{target_agent.id}"
@@ -619,6 +619,7 @@ async def stream_agent_run(
in real-time as the agent processes.
"""
from fastapi.responses import StreamingResponse
from app.ai.agent_stream import stream_react_loop
from app.plugins.builtins.ai_assistant.contracts import get_tool_registry
+13 -21
View File
@@ -11,6 +11,7 @@ Safety features:
from __future__ import annotations
import logging
import uuid
from datetime import UTC, datetime
from typing import Any
@@ -37,12 +38,12 @@ async def run_agent(
3. Infinite loop: same tool 5x consecutively (handled in ReAct loop)
4. Budget limit: cumulative cost_usd
"""
from app.plugins.builtins.ai_assistant.contracts import get_tool_registry
from app.plugins.builtins.automation.models import (
AgentDefinition,
AgentRun,
AgentRunStep,
)
from app.plugins.builtins.ai_assistant.contracts import get_tool_registry
factory = get_session_factory()
@@ -110,9 +111,9 @@ async def run_agent(
try:
from app.plugins.builtins.contracts import get_contract
mail_contract = get_contract("mail")
if mail_contract and hasattr(mail_contract, "get_recent_mails"):
if mail_contract and hasattr(mail_contract, "Mail"):
from sqlalchemy import select as _select
from app.plugins.builtins.mail.models import Mail
Mail = mail_contract.Mail
async with factory() as db:
mail_q = await db.execute(
_select(Mail)
@@ -167,7 +168,7 @@ async def run_agent(
perm_ctx = await resolve_agent_permissions(
db=db,
tenant_id=agent.tenant_id,
user_id=agent.created_by or uuid_mod.uuid4(),
user_id=agent.created_by or uuid.uuid4(),
agent_definition=agent,
)
@@ -379,23 +380,16 @@ async def run_agent(
komm = get_contract_registry().get("kommunikation")
if komm:
async with factory() as db:
# Find or create agent conversation room
from app.plugins.builtins.contracts import get_contract as _get_contract
_komm_contract = _get_contract("kommunikation")
from app.plugins.builtins.kommunikation.models import CommConversation
from sqlalchemy import select as sa_select
# Find or create agent conversation room via contract
# (find_locked_room_id matches create_plugin_room semantics)
room_title = f"Agent: {agent.name}"
existing = await db.execute(
sa_select(CommConversation).where(
CommConversation.tenant_id == agent.tenant_id,
CommConversation.title == room_title,
CommConversation.is_locked.is_(True),
CommConversation.locked_by == "automation",
CommConversation.deleted_at.is_(None),
)
conv_id = await komm.find_locked_room_id(
db=db,
tenant_id=agent.tenant_id,
plugin_name="automation",
title=room_title,
)
conv = existing.scalar_one_or_none()
if not conv:
if not conv_id:
room = await komm.create_plugin_room(
db=db,
tenant_id=agent.tenant_id,
@@ -405,8 +399,6 @@ async def run_agent(
participant_type="agent",
)
conv_id = uuid.UUID(room["conversation_id"])
else:
conv_id = conv.id
# Post result as message with action_card block
status = result_data.get("status", "unknown")
@@ -0,0 +1,31 @@
-- Dual-path convergence (Gate B): create the ReAct step-tracking table
-- that Alembic migration 0121 creates on the core path, add the Phase-F
-- columns from 0122, and apply the RLS policy from 0129/0136. Idempotent
-- so both install paths converge to the identical schema.
CREATE TABLE IF NOT EXISTS automation_agent_run_steps (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
tenant_id UUID NOT NULL,
agent_run_id UUID NOT NULL REFERENCES automation_agent_runs(id) ON DELETE CASCADE,
step_number INTEGER NOT NULL,
thought TEXT,
action VARCHAR(255),
action_input JSONB,
observation TEXT,
cost_usd FLOAT NOT NULL DEFAULT 0.0,
created_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE INDEX IF NOT EXISTS ix_agent_run_steps_run ON automation_agent_run_steps(tenant_id, agent_run_id);
ALTER TABLE automation_agent_definitions ADD COLUMN IF NOT EXISTS temperature FLOAT NOT NULL DEFAULT 0.3;
ALTER TABLE automation_agent_definitions ADD COLUMN IF NOT EXISTS max_tokens INTEGER NOT NULL DEFAULT 1000;
ALTER TABLE automation_agent_definitions ADD COLUMN IF NOT EXISTS max_steps INTEGER NOT NULL DEFAULT 20;
ALTER TABLE automation_agent_definitions ADD COLUMN IF NOT EXISTS trace_mode VARCHAR(20) NOT NULL DEFAULT 'standard';
ALTER TABLE automation_agent_definitions ADD COLUMN IF NOT EXISTS skill_ids JSONB NOT NULL DEFAULT '[]'::jsonb;
ALTER TABLE automation_agent_definitions ADD COLUMN IF NOT EXISTS trigger_config JSONB NOT NULL DEFAULT '{}'::jsonb;
ALTER TABLE automation_agent_definitions ADD COLUMN IF NOT EXISTS ai_use_case_metadata JSONB NOT NULL DEFAULT '{}'::jsonb;
-- RLS matching migrations 0129 + 0136 (current_tenant_id variant)
ALTER TABLE automation_agent_run_steps ENABLE ROW LEVEL SECURITY;
DROP POLICY IF EXISTS tenant_isolation ON automation_agent_run_steps;
CREATE POLICY tenant_isolation ON automation_agent_run_steps
USING (tenant_id::text = current_setting('app.current_tenant_id', true));
+35 -19
View File
@@ -38,7 +38,7 @@ class AutomationPlugin(BasePlugin):
"Define AI agents with LLM models and tools, create event/schedule/manual "
"automations with conditions and actions, schedule cron jobs, and track execution logs."
),
dependencies=[],
dependencies=["mail"],
routes=[
PluginRouteDef(
path="/api/v1/automation",
@@ -62,7 +62,7 @@ class AutomationPlugin(BasePlugin):
"mail.received",
"workflow.timeout",
],
migrations=["0001_initial.sql", "0002_agent_subtasks.sql", "0003_skill_definitions.sql"],
migrations=["0001_initial.sql", "0002_agent_subtasks.sql", "0003_skill_definitions.sql", "0004_run_steps_phase_f.sql"],
permissions=[
"automation:read",
"automation:write",
@@ -82,6 +82,7 @@ class AutomationPlugin(BasePlugin):
path="/workflows",
icon="Workflow",
order=52,
permission="automation:read",
),
FrontendMenuItem(
label_key="nav.importExport",
@@ -89,6 +90,7 @@ class AutomationPlugin(BasePlugin):
path="/import-export",
icon="ArrowUpDown",
order=53,
permission="import_export:read",
),
FrontendMenuItem(
label_key="nav.dedupMerge",
@@ -96,6 +98,7 @@ class AutomationPlugin(BasePlugin):
path="/contacts/dedup",
icon="Copy",
order=54,
permission="contacts:read",
),
FrontendMenuItem(
label_key="nav.tags",
@@ -103,6 +106,7 @@ class AutomationPlugin(BasePlugin):
path="/tags",
icon="Tag",
order=55,
permission="tags:read",
),
FrontendMenuItem(
label_key="nav.activity",
@@ -110,6 +114,7 @@ class AutomationPlugin(BasePlugin):
path="/activity",
icon="Activity",
order=56,
permission="contacts:read",
),
],
page_routes=[
@@ -117,21 +122,25 @@ class AutomationPlugin(BasePlugin):
path="/automation",
component="@/pages/AutomationDashboard",
order=50,
permission="automation:read",
),
FrontendPageRoute(
path="/agents",
component="@/pages/AgentDashboard",
order=51,
permission="agents:read",
),
FrontendPageRoute(
path="/workflows",
component="@/pages/Workflows",
order=52,
permission="automation:read",
),
FrontendPageRoute(
path="/import-export",
component="@/pages/ImportExport",
order=53,
permission="import_export:read",
),
],
settings_pages=[
@@ -239,18 +248,25 @@ class AutomationPlugin(BasePlugin):
logger.exception("Failed to register own cron jobs")
# Register pre-built agents in DB (if not already present)
try:
from app.plugins.builtins.automation.models import AgentDefinition
from app.plugins.builtins.automation.prebuilt.email_triage_agent import create_email_triage_agent
from app.plugins.builtins.automation.prebuilt.contact_enrichment_agent import create_contact_enrichment_agent
from app.plugins.builtins.automation.prebuilt.follow_up_agent import create_follow_up_agent
from app.plugins.builtins.automation.prebuilt.report_agent import create_report_agent
from sqlalchemy import select as sa_select
# Get first tenant + admin user for seeding
from app.models.user import User
from app.models.tenant import Tenant
tenant_result = await db.execute(sa_select(Tenant).limit(1))
tenant = tenant_result.scalar_one_or_none()
# Get system tenant + admin user for seeding (ARCH-043:
# deterministic slug lookup instead of arbitrary first row)
from app.core.db import get_system_tenant
from app.models.user import User, UserTenant
from app.plugins.builtins.automation.models import AgentDefinition
from app.plugins.builtins.automation.prebuilt.contact_enrichment_agent import (
create_contact_enrichment_agent,
)
from app.plugins.builtins.automation.prebuilt.email_triage_agent import (
create_email_triage_agent,
)
from app.plugins.builtins.automation.prebuilt.follow_up_agent import (
create_follow_up_agent,
)
from app.plugins.builtins.automation.prebuilt.report_agent import create_report_agent
tenant = await get_system_tenant(db)
if tenant:
user_result = await db.execute(
sa_select(User)
@@ -288,15 +304,14 @@ class AutomationPlugin(BasePlugin):
def _register_workflow_agent_tools(self) -> None:
"""Register I-AW agent tools for starting and inspecting workflows."""
import uuid
from typing import Any
from app.ai.tool_registry import get_tool_registry
registry = get_tool_registry()
async def _start_workflow_handler(arguments: dict[str, Any], context: dict[str, Any]) -> dict[str, Any]:
"""Start a workflow by ID."""
from app.services.workflow_service import create_instance
from app.core.db import get_worker_session_factory
from app.services.workflow_service import create_instance
workflow_id = arguments.get("workflow_id", "")
tenant_id = context.get("tenant_id")
user_id = context.get("user_id")
@@ -332,8 +347,9 @@ class AutomationPlugin(BasePlugin):
async def _check_workflow_status_handler(arguments: dict[str, Any], context: dict[str, Any]) -> dict[str, Any]:
"""Check the status of a workflow instance."""
from sqlalchemy import select
from app.models.workflow import WorkflowInstance
from app.core.db import get_worker_session_factory
from app.models.workflow import WorkflowInstance
instance_id = arguments.get("instance_id", "")
tenant_id = context.get("tenant_id")
if not instance_id or not tenant_id:
@@ -418,16 +434,16 @@ class AutomationPlugin(BasePlugin):
from another plugin's manifest. Uses plugin name prefixing for conflict resolution."""
from sqlalchemy import select
# Get default tenant_id from the first tenant in the DB
from app.models.tenant import Tenant
# Get system tenant for contributions (ARCH-043: deterministic slug
# lookup instead of arbitrary first row)
from app.core.db import get_system_tenant
from app.plugins.builtins.automation.models import AutomationCronJob
from app.plugins.builtins.automation.services import (
AgentService,
AutomationService,
CronJobService,
)
tenant_result = await db.execute(select(Tenant).limit(1))
tenant = tenant_result.scalar_one_or_none()
tenant = await get_system_tenant(db)
default_tenant_id = tenant.id if tenant else None
if default_tenant_id is None:
logger.warning("No tenant found — skipping plugin contributions registration")
@@ -3,7 +3,9 @@
Enriches contact data by searching for related information.
"""
from __future__ import annotations
import uuid
from app.plugins.builtins.automation.models import AgentDefinition
CONTACT_ENRICHMENT_SYSTEM_PROMPT = """You are a Contact Enrichment Agent for a CRM system.
@@ -3,7 +3,9 @@
Sorts and prioritizes incoming emails automatically.
"""
from __future__ import annotations
import uuid
from app.plugins.builtins.automation.models import AgentDefinition
EMAIL_TRIAGE_SYSTEM_PROMPT = """You are an E-Mail Triage Agent for a CRM system.
@@ -3,7 +3,9 @@
Reminds about and creates follow-up tasks for contacts.
"""
from __future__ import annotations
import uuid
from app.plugins.builtins.automation.models import AgentDefinition
FOLLOW_UP_SYSTEM_PROMPT = """You are a Follow-up Agent for a CRM system.
@@ -3,7 +3,9 @@
Generates reports from CRM data using search and API tools.
"""
from __future__ import annotations
import uuid
from app.plugins.builtins.automation.models import AgentDefinition
REPORT_SYSTEM_PROMPT = """You are a Report Agent for a CRM system.
@@ -1,60 +1,132 @@
"""Tests for the Automation & Agents plugin.
Uses pytest with async fixtures. Tests use SQLite in-memory database
since PostgreSQL may not be available in the dev container.
Uses pytest with async fixtures against an ephemeral PostgreSQL database
(SQLITE-001 fix) matches the project convention and exercises the real
PGUUID/JSONB column types.
"""
from __future__ import annotations
# Register ALL plugin models so create_all can resolve cross-plugin FKs
# (e.g. entity_attachments.dms_file_id -> files) — same pattern as
# scripts/sync_plugin_schema.py.
import importlib
import os
import pkgutil
import uuid
from collections.abc import AsyncGenerator
from datetime import UTC, datetime, timedelta
import pytest
import pytest_asyncio
from sqlalchemy import text
from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker, create_async_engine
import app.models # noqa: F401 — registers core models
import app.models.outbox # noqa: F401 — event_outbox is NOT re-exported by app.models
import app.plugins.builtins as _builtins_pkg
from app.core.db import Base
from app.plugins.builtins.automation.models import (
for _importer, _modname, _ispkg in pkgutil.iter_modules(_builtins_pkg.__path__):
if not _ispkg:
continue
try:
importlib.import_module(f"app.plugins.builtins.{_modname}.models")
except ImportError:
pass # plugin without models module
except Exception: # pragma: no cover - defensive
pass
from app.plugins.builtins.automation.models import ( # noqa: E402 — after dynamic plugin-model discovery
AgentRun,
AutomationRun,
)
from app.plugins.builtins.automation.services import (
from app.plugins.builtins.automation.services import ( # noqa: E402 — after dynamic plugin-model discovery
AgentService,
AutomationService,
CronJobService,
)
def _ephemeral_db_url() -> str:
"""Derive an ephemeral test DB URL from DATABASE_URL/.env.test."""
base_url = os.environ.get(
"DATABASE_URL",
"postgresql+asyncpg://leocrm_test:test123@localhost:5432/leocrm_test",
)
return f"{base_url.rsplit('/', 1)[0]}/automation_test_{uuid.uuid4().hex[:8]}"
# ─── Fixtures ───
@pytest_asyncio.fixture
async def db() -> AsyncGenerator[AsyncSession, None]:
"""Create an in-memory SQLite database for testing."""
engine = create_async_engine(
"sqlite+aiosqlite:///:memory:",
echo=False,
)
"""Create an ephemeral PostgreSQL database for this test run."""
db_url = _ephemeral_db_url()
admin_url = db_url.rsplit("/", 1)[0] + "/postgres"
from sqlalchemy.ext.asyncio import create_async_engine as _cae
admin_engine = _cae(admin_url, isolation_level="AUTOCOMMIT")
async with admin_engine.connect() as conn:
await conn.execute(text(f'CREATE DATABASE "{db_url.rsplit("/", 1)[1]}"'))
await admin_engine.dispose()
# Plugin models use the pgvector Vector type — enable the extension in
# the fresh database before create_all runs (must connect to the target
# DB itself; CREATE EXTENSION has no ON DATABASE clause).
ext_engine = _cae(db_url, isolation_level="AUTOCOMMIT")
async with ext_engine.connect() as conn:
await conn.execute(text("CREATE EXTENSION IF NOT EXISTS vector"))
await ext_engine.dispose()
engine = create_async_engine(db_url, echo=False)
async with engine.begin() as conn:
await conn.run_sync(Base.metadata.create_all)
async_session = async_sessionmaker(engine, class_=AsyncSession, expire_on_commit=False)
async with async_session() as session:
yield session
await engine.dispose()
try:
async with async_session() as session:
yield session
finally:
await engine.dispose()
admin_engine2 = _cae(admin_url, isolation_level="AUTOCOMMIT")
async with admin_engine2.connect() as conn:
await conn.execute(text(f'DROP DATABASE IF EXISTS "{db_url.rsplit("/", 1)[1]}"'))
await admin_engine2.dispose()
@pytest.fixture
def tenant_id() -> uuid.UUID:
return uuid.uuid4()
@pytest_asyncio.fixture
async def tenant_id(db: AsyncSession) -> uuid.UUID:
"""Create a real tenant row — PostgreSQL enforces FKs, unlike SQLite."""
from app.models.tenant import Tenant
tid = uuid.uuid4()
db.add(Tenant(id=tid, name="Test Org", slug=f"test-{tid.hex[:8]}"))
await db.commit()
return tid
@pytest.fixture
def user_id() -> uuid.UUID:
return uuid.uuid4()
@pytest_asyncio.fixture
async def user_id(db: AsyncSession, tenant_id: uuid.UUID) -> uuid.UUID:
"""Create a real user row belonging to the test tenant."""
from app.models.user import User
uid = uuid.uuid4()
db.add(
User(
id=uid,
email=f"test-{uid.hex[:8]}@example.com",
name="Test User",
password_hash="not-a-real-hash",
is_active=True,
)
)
await db.commit()
return uid
# ─── AgentService Tests ───
@@ -425,11 +497,17 @@ class TestDryRunMode:
assert automation.dry_run is True
@pytest.mark.asyncio
async def test_dry_run_flag_in_run(self, db: AsyncSession, tenant_id: uuid.UUID):
async def test_dry_run_flag_in_run(self, db: AsyncSession, tenant_id: uuid.UUID, user_id: uuid.UUID):
"""Test that dry_run flag is stored in AutomationRun."""
# PostgreSQL enforces the FK to automations — create a real one first
data = {"name": "dry-run-flag", "description": "", "trigger_type": "manual",
"trigger_config": {}, "conditions": [], "actions": [],
"is_active": True, "dry_run": True}
automation = await AutomationService.create(db, tenant_id, data, user_id=user_id)
run = AutomationRun(
tenant_id=tenant_id,
automation_id=uuid.uuid4(),
automation_id=automation.id,
status="dry_run",
started_at=datetime.now(UTC),
dry_run=True,
@@ -475,7 +553,9 @@ class TestRateLimiting:
)
recent_runs = result.scalar() or 0
assert recent_runs == 2
assert recent_runs < agent.max_executions_per_hour # 2 < 2 is False, so limit would be hit
# With max_executions_per_hour=2 and 2 runs in the window, the limit
# is reached — the next execution must be blocked.
assert recent_runs >= agent.max_executions_per_hour
# ─── Budget Limit Tests ───
@@ -512,7 +592,8 @@ class TestBudgetLimit:
.where(AgentRun.agent_id == agent.id)
)
total_cost = float(cost_result.scalar() or 0.0)
assert total_cost == 0.6
# FLOAT column accumulates binary rounding (0.6000000000000001)
assert total_cost == pytest.approx(0.6)
assert total_cost >= agent.budget_limit_usd # 0.6 >= 0.5, budget exceeded
+2 -3
View File
@@ -6,6 +6,7 @@ import uuid
from datetime import datetime
from typing import Any
from pgvector.sqlalchemy import Vector
from sqlalchemy import (
Boolean,
DateTime,
@@ -13,14 +14,12 @@ from sqlalchemy import (
Index,
String,
)
from sqlalchemy.dialects.postgresql import JSONB
from sqlalchemy.dialects.postgresql import JSONB, TSVECTOR
from sqlalchemy.dialects.postgresql import UUID as PGUUID
from sqlalchemy.orm import Mapped, mapped_column
from app.core.db import Base, TenantMixin
from app.models.owned_mixin import OwnedMixin
from sqlalchemy.dialects.postgresql import TSVECTOR
from pgvector.sqlalchemy import Vector
class Calendar(Base, TenantMixin, OwnedMixin):
+18 -6
View File
@@ -4,7 +4,7 @@ from __future__ import annotations
from app.plugins.base import BasePlugin
from app.plugins.manifest import (
FrontendDetailTab,
FrontendDashboardWidget,
FrontendMenuItem,
FrontendPageRoute,
PluginManifest,
@@ -40,6 +40,18 @@ class CalendarPlugin(BasePlugin):
],
events=[],
migrations=["0001_initial.sql", "0002_add_deleted_at.sql"],
dashboard_widgets=[
FrontendDashboardWidget(
id="calendar_upcoming",
label_key="dashboard.calendarUpcoming",
label="Upcoming Appointments",
component="@/components/dashboard/CalendarUpcomingWidget",
icon="Calendar",
order=30,
col_span=1,
permission="calendar:read",
),
],
permissions=[
"calendar:read",
"calendar:write",
@@ -48,14 +60,14 @@ class CalendarPlugin(BasePlugin):
"calendar:admin",
],
menu_items=[
FrontendMenuItem(label_key='nav.calendar', label='Kalender', path='/calendar', icon='Calendar', order=20),
FrontendMenuItem(label_key='nav.calendar', label='Kalender', path='/calendar', icon='Calendar', order=20, permission='calendar:read'),
],
page_routes=[
FrontendPageRoute(path='/calendar', component='@/pages/Calendar', protected=True),
],
detail_tabs=[
FrontendDetailTab(entity_type='contact', label_key='tabs.calendar', label='Calendar', component='@/components/contact/ContactCalendarTab', icon='Calendar', order=30, permission='calendar:read'),
FrontendPageRoute(path='/calendar', component='@/pages/Calendar', protected=True, permission='calendar:read'),
],
# BUG (ghost component): ContactCalendarTab does not exist in the
# frontend — tab removed until implemented (Block I-D).
detail_tabs=[],
author="LeoCRM Team",
min_app_version="1.0.0",
hooks=["calendar.before_appointment", "calendar.after_appointment"],
+1 -2
View File
@@ -21,6 +21,7 @@ from fastapi.responses import StreamingResponse
from sqlalchemy import select, update
from sqlalchemy.ext.asyncio import AsyncSession
from app.core.audit import log_audit
from app.core.db import get_db
from app.deps import get_current_user, require_admin, require_permission
from app.plugins.builtins.calendar.ics_utils import (
@@ -1023,5 +1024,3 @@ async def book_resource(
"start_at": booking.start_at.isoformat(),
"end_at": booking.end_at.isoformat(),
}
from app.core.audit import log_audit
@@ -0,0 +1,62 @@
"""Public contract for the contacts plugin.
Exposes the symbols that other core modules and plugins need without
importing from internal modules directly (Block C7: dashboard counts).
"""
from __future__ import annotations
from typing import Any
from sqlalchemy import func, select
from sqlalchemy.ext.asyncio import AsyncSession
from app.core.visibility import apply_visibility_filter
from app.models.contact import Contact
from app.plugins.builtins.contracts import get_contract_registry
class ContactsContract:
"""Public API surface for the contacts plugin."""
contract_name = "contacts"
@staticmethod
async def get_counts(
db: AsyncSession,
tenant_id: Any,
user_id: Any,
is_system_admin: bool = False,
) -> dict[str, int]:
"""Return visibility-filtered contact/company/person counts."""
queries = []
for type_filter in (None, "company", "person"):
query = select(func.count(Contact.id)).where(
Contact.tenant_id == tenant_id,
Contact.deleted_at.is_(None),
)
if type_filter is not None:
query = query.where(Contact.type == type_filter)
query = await apply_visibility_filter(
db, query, "contact", Contact, user_id, tenant_id, is_system_admin
)
queries.append(query)
results = [((await db.execute(q)).scalar() or 0) for q in queries]
return {
"contacts": results[0],
"companies": results[1],
"persons": results[2],
"total": results[0],
}
@classmethod
def get_function(cls, name: str):
"""Return a callable exposed by this contract, or None if absent."""
return getattr(cls, name, None)
# ─── self-registration ───
_contract = ContactsContract()
get_contract_registry().register("contacts", _contract)
+46 -6
View File
@@ -9,27 +9,67 @@ from __future__ import annotations
import logging
from app.plugins.base import BasePlugin
from app.plugins.manifest import PluginManifest
from app.plugins.manifest import (
FrontendDashboardWidget,
PluginManifest,
PluginRouteDef,
)
logger = logging.getLogger(__name__)
class ContactsPlugin(BasePlugin):
"""Contacts plugin — manages Contact entity lifecycle (models, permissions, restore, history).
"""Contacts plugin — owns the full Contact domain (Block B1).
Routes remain in app/routes/contacts.py as core routes, but entity lifecycle
(permissions, entity models, restore, history) is managed through on_activate/on_deactivate.
Routes (contacts, companies, contact folders, folder permissions) live in
this plugin and are mounted via manifest.routes with
require_active_plugin("contacts") protection. Entity lifecycle
(permissions, entity models, restore, history) is managed through
on_activate/on_deactivate like every other business plugin.
"""
manifest = PluginManifest(
name="contacts",
version="1.0.0",
version="1.1.0",
display_name="Contacts",
description="Core CRM contacts — persons and companies.",
dependencies=[],
routes=[], # Routes are registered as core routes in main.py
routes=[
PluginRouteDef(
path="/api/v1/contacts",
module="app.plugins.builtins.contacts.routes",
router_attr="router",
),
PluginRouteDef(
path="/api/v1/companies",
module="app.plugins.builtins.contacts.company_routes",
router_attr="router",
),
PluginRouteDef(
path="/api/v1/contact-folders",
module="app.plugins.builtins.contacts.folder_routes",
router_attr="router",
),
PluginRouteDef(
path="/api/v1/contact-folders",
module="app.plugins.builtins.contacts.folder_permission_routes",
router_attr="router",
),
],
events=[],
migrations=[],
dashboard_widgets=[
FrontendDashboardWidget(
id="recent_contacts",
label_key="dashboard.recentContacts",
label="Recent Contacts",
component="@/components/dashboard/RecentContactsWidget",
icon="Users",
order=10,
col_span=2,
permission="contacts:read",
),
],
permissions=[
"contacts:read",
"contacts:write",
+18 -2
View File
@@ -59,20 +59,31 @@ class ContractRegistry:
cls._instance = super().__new__(cls)
cls._instance._contracts: dict[str, Any] = {}
cls._instance._loaded: set[str] = set()
cls._instance._unregistered: set[str] = set()
return cls._instance
# ─── registration ───
def register(self, plugin_name: str, contract: Any) -> None:
"""Register or replace a contract for a plugin."""
"""Register or replace a contract for a plugin.
Clears the unregistered marker so a later deactivation can be
distinguished from a fresh lazy-load again (ARCH-014).
"""
self._unregistered.discard(plugin_name)
self._contracts[plugin_name] = contract
self._loaded.add(plugin_name)
logger.debug("Contract registered for plugin '%s'", plugin_name)
def unregister(self, plugin_name: str) -> None:
"""Remove a contract (e.g. when the plugin is deactivated)."""
"""Remove a contract (e.g. when the plugin is deactivated).
Marks the plugin as explicitly unregistered so later ``get_contract``
calls cannot resurrect the contract via lazy-loading (ARCH-014).
"""
self._contracts.pop(plugin_name, None)
self._loaded.discard(plugin_name)
self._unregistered.add(plugin_name)
# ─── lookup ───
@@ -85,6 +96,11 @@ class ContractRegistry:
if plugin_name in self._contracts:
return self._contracts[plugin_name]
# Explicitly unregistered (deactivated): never resurrect via
# lazy-loading (ARCH-014) — the deactivated contract must stay gone.
if plugin_name in self._unregistered:
return None
if plugin_name not in self._loaded:
self._try_lazy_load(plugin_name)
+111
View File
@@ -0,0 +1,111 @@
"""DMS gemeinsame Helper & Konstanten — BUG-018 God-Object-Split."""
from __future__ import annotations
import os
import uuid
from fastapi import HTTPException
OFFICE_EXTENSIONS = {
".docx": "docx",
".xlsx": "xlsx",
".pptx": "pptx",
}
# Max file size: 100 MB
MAX_FILE_SIZE = 100 * 1024 * 1024
def _parse_uuid(val: str, field: str) -> uuid.UUID:
try:
return uuid.UUID(val)
except (ValueError, TypeError):
raise HTTPException(
400, detail={"detail": f"Invalid {field}", "code": "invalid_id"}
) from None
def _file_storage_path(tenant_id: uuid.UUID, file_id: uuid.UUID) -> str:
"""Build relative storage path for a file (relative to storage base)."""
return f"{tenant_id}/{file_id}"
def _get_file_extension(filename: str) -> str:
"""Extract lowercase extension including dot."""
return os.path.splitext(filename)[1].lower()
def _sanitize_filename(filename: str) -> str:
"""Sanitize a filename for safe use in Content-Disposition headers."""
import re
# Extract basename only (strip any path components)
safe = os.path.basename(filename.replace('\\', '/'))
# Remove dangerous characters (keep alnum, dot, dash, underscore, space, unicode)
safe = re.sub(r'[^a-zA-Z0-9.\-_\u00c0-\u017f\u4e00-\u9fff ]', '_', safe)
# Collapse consecutive dots (path traversal prevention)
safe = re.sub(r'\.{2,}', '_', safe)
# Collapse multiple spaces
safe = re.sub(r' {2,}', ' ', safe)
# Strip leading dots and whitespace
safe = safe.lstrip('.').strip()
# Limit length
if len(safe) > 200:
name, ext = safe.rsplit('.', 1) if '.' in safe[:200] else (safe[:200], '')
safe = name[:200] + ('.' + ext if ext else '')
return safe or 'file'
# Blocked file extensions for security
BLOCKED_EXTENSIONS = {
".exe", ".bat", ".cmd", ".sh", ".jar", ".com", ".scr", ".msi",
".dll", ".vbs", ".ps1", ".app", ".bin", ".reg", ".inf",
".php", ".py", ".pl", ".asp", ".aspx", ".jsp", ".svg", ".htaccess",
".phtml", ".pht", ".cgi", ".cfm", ".erb",
}
# Allowed MIME types for upload validation
ALLOWED_MIME_PREFIXES = {
"application/pdf",
"application/msword",
"application/vnd.openxmlformats-officedocument",
"application/vnd.oasis.opendocument",
"application/vnd.ms-excel",
"application/vnd.ms-powerpoint",
"application/zip",
"application/gzip",
"application/x-tar",
"application/json",
"application/xml",
"application/rtf",
"application/x-7z-compressed",
"application/x-rar-compressed",
"text/plain",
"text/csv",
"text/html",
"text/markdown",
"image/png",
"image/jpeg",
"image/gif",
"image/webp",
"image/bmp",
"image/tiff",
"image/x-icon",
"audio/",
"video/",
"application/octet-stream",
}
def _is_blocked_filetype(filename: str) -> bool:
"""Check if a file has a blocked (dangerous) extension."""
ext = os.path.splitext(filename)[1].lower()
return ext in BLOCKED_EXTENSIONS
chunk_size = 1024 * 1024 # 1MB chunks for streaming uploads
# ─── Folders ───
# Public stream-chunk constant (original contract name from tests/test_p1_6_dms_streaming.py)
CHUNK_SIZE = chunk_size
+383
View File
@@ -0,0 +1,383 @@
"""DMS Folder-CRUD Routen — extrahiert aus routes.py (BUG-018 God-Object-Split)."""
from __future__ import annotations
import uuid
from fastapi import (
APIRouter,
Depends,
HTTPException,
Response,
status,
)
from sqlalchemy import select, update
from sqlalchemy.ext.asyncio import AsyncSession
from app.core.db import get_db
from app.core.visibility import apply_visibility_filter, check_single_entity_access
from app.deps import get_current_user, require_permission
from app.plugins.builtins.dms.common import (
_parse_uuid,
)
from app.plugins.builtins.dms.models import File as DmsFile
from app.plugins.builtins.dms.models import Folder
from app.plugins.builtins.dms.schemas import FolderCreate, FolderUpdate
from app.plugins.builtins.permissions.contracts import get_contract as get_perms_contract
_perms_contract = get_perms_contract()
Permission = _perms_contract.Permission
router = APIRouter(tags=["dms"])
@router.get("/folders", dependencies=[Depends(require_permission("dms:read"))])
async def list_folders(
parent_id: str | None = None,
db: AsyncSession = Depends(get_db),
current_user: dict = Depends(get_current_user),
):
"""AC1: GET /api/v1/dms/folders → 200 + folder tree (recursive)."""
tenant_id = uuid.UUID(current_user["tenant_id"])
# Fetch all non-deleted folders for tenant with visibility filter
user_id = uuid.UUID(current_user["user_id"])
is_system_admin = current_user.get("role") == "admin"
query = select(Folder).where(
Folder.tenant_id == tenant_id,
Folder.deleted_at.is_(None),
)
query = await apply_visibility_filter(
db, query, "dms_folder", Folder, user_id, tenant_id, is_system_admin
)
result = await db.execute(query)
all_folders = result.scalars().all()
# Build lookup map
folder_map: dict[uuid.UUID, dict] = {}
for f in all_folders:
folder_map[f.id] = {
"id": str(f.id),
"name": f.name,
"parent_id": str(f.parent_id) if f.parent_id else None,
"created_by": str(f.created_by),
"deleted_at": None,
"path": "",
"children": [],
}
# Build path for each folder
def _build_path(folder_id: uuid.UUID) -> str:
if folder_id not in folder_map:
return ""
f = folder_map[folder_id]
if f["parent_id"] and uuid.UUID(f["parent_id"]) in folder_map:
parent_path = _build_path(uuid.UUID(f["parent_id"]))
return f"{parent_path}/{f['name']}"
return f["name"]
for fid in folder_map:
folder_map[fid]["path"] = _build_path(fid)
# Build tree
root_nodes: list[dict] = []
target_parent: uuid.UUID | None = None
if parent_id is not None:
target_parent = _parse_uuid(parent_id, "parent_id")
for f in all_folders:
node = folder_map[f.id]
if f.parent_id is not None and f.parent_id in folder_map:
folder_map[f.parent_id]["children"].append(node)
elif f.parent_id is None:
root_nodes.append(node)
if target_parent is not None:
# Return children of specified parent
parent_node = folder_map.get(target_parent)
if parent_node is None:
raise HTTPException(
404, detail={"detail": "Parent folder not found", "code": "not_found"}
)
return parent_node["children"]
return root_nodes
@router.post("/folders", status_code=status.HTTP_201_CREATED, dependencies=[Depends(require_permission("dms:write"))])
async def create_folder(
body: FolderCreate,
db: AsyncSession = Depends(get_db),
current_user: dict = Depends(get_current_user),
):
"""AC2: POST /api/v1/dms/folders → 201, folder created with path."""
tenant_id = uuid.UUID(current_user["tenant_id"])
user_id = uuid.UUID(current_user["user_id"])
parent_id = _parse_uuid(body.parent_id, "parent_id") if body.parent_id else None
# Validate parent exists if specified
if parent_id is not None:
parent_result = await db.execute(
select(Folder).where(
Folder.id == parent_id,
Folder.tenant_id == tenant_id,
Folder.deleted_at.is_(None),
)
)
if parent_result.scalar_one_or_none() is None:
raise HTTPException(
404, detail={"detail": "Parent folder not found", "code": "not_found"}
)
# Check name uniqueness within same parent (non-deleted)
existing = await db.execute(
select(Folder).where(
Folder.tenant_id == tenant_id,
Folder.name == body.name,
Folder.parent_id == parent_id if parent_id else Folder.parent_id.is_(None),
Folder.deleted_at.is_(None),
)
)
if existing.scalar_one_or_none() is not None:
raise HTTPException(
409, detail={"detail": "Folder name already exists", "code": "duplicate"}
)
# Lifecycle hook: dms.folder.before_create
from app.core.hooks import do_action
await do_action("dms.folder.before_create", body, db=db, tenant_id=tenant_id, user_id=user_id)
folder = Folder(
tenant_id=tenant_id,
name=body.name,
parent_id=parent_id,
created_by=user_id,
)
db.add(folder)
await db.flush()
# Lifecycle hook: dms.folder.after_create
await do_action("dms.folder.after_create", {'id': str(folder.id), 'name': folder.name, 'parent_id': str(folder.parent_id) if folder.parent_id else None}, db=db, tenant_id=tenant_id, user_id=user_id)
# Build path
path = body.name
if parent_id is not None:
parent_path_result = await db.execute(select(Folder).where(Folder.id == parent_id))
parent_folder = parent_path_result.scalar_one_or_none()
if parent_folder:
# Recursively build path
path_parts = [body.name]
current = parent_folder
while current is not None:
path_parts.insert(0, current.name)
if current.parent_id is not None:
cur_result = await db.execute(
select(Folder).where(Folder.id == current.parent_id)
)
current = cur_result.scalar_one_or_none()
else:
current = None
path = "/".join(path_parts)
return {
"id": str(folder.id),
"name": folder.name,
"parent_id": str(folder.parent_id) if folder.parent_id else None,
"created_by": str(folder.created_by),
"deleted_at": None,
"path": path,
"children": [],
}
@router.patch("/folders/{folder_id}", dependencies=[Depends(require_permission("dms:write"))])
async def update_folder(
folder_id: str,
body: FolderUpdate,
db: AsyncSession = Depends(get_db),
current_user: dict = Depends(get_current_user),
):
"""AC3: PATCH /api/v1/dms/folders/{id} → 200, rename/move."""
tenant_id = uuid.UUID(current_user["tenant_id"])
user_id = uuid.UUID(current_user["user_id"])
is_system_admin = current_user.get("role") == "admin"
fid = _parse_uuid(folder_id, "folder_id")
result = await db.execute(
select(Folder).where(
Folder.id == fid,
Folder.tenant_id == tenant_id,
Folder.deleted_at.is_(None),
)
)
folder = result.scalar_one_or_none()
if folder is None:
raise HTTPException(404, detail={"detail": "Folder not found", "code": "not_found"})
if not await check_single_entity_access(db, "dms_folder", fid, user_id, tenant_id, "write", is_system_admin):
raise HTTPException(403, detail={"detail": "Access denied", "code": "forbidden"})
data = body.model_dump(exclude_unset=True)
if "name" in data and data["name"] is not None:
# Check uniqueness if name is changing
new_parent_id = folder.parent_id
if "parent_id" in data and data["parent_id"] is not None:
new_parent_id = _parse_uuid(data["parent_id"], "parent_id")
dup = await db.execute(
select(Folder).where(
Folder.tenant_id == tenant_id,
Folder.name == data["name"],
Folder.id != fid,
Folder.parent_id == new_parent_id if new_parent_id else Folder.parent_id.is_(None),
Folder.deleted_at.is_(None),
)
)
if dup.scalar_one_or_none() is not None:
raise HTTPException(
409, detail={"detail": "Folder name already exists", "code": "duplicate"}
)
folder.name = data["name"]
if "parent_id" in data:
new_parent = _parse_uuid(data["parent_id"], "parent_id") if data["parent_id"] else None
if new_parent is not None:
# Validate parent exists and not creating a cycle
if new_parent == fid:
raise HTTPException(
400, detail={"detail": "Cannot move folder into itself", "code": "invalid_move"}
)
parent_result = await db.execute(
select(Folder).where(
Folder.id == new_parent,
Folder.tenant_id == tenant_id,
Folder.deleted_at.is_(None),
)
)
if parent_result.scalar_one_or_none() is None:
raise HTTPException(
404, detail={"detail": "Parent folder not found", "code": "not_found"}
)
# Check for cycle: ensure new_parent is not a descendant of folder
async def _is_descendant(ancestor_id: uuid.UUID, descendant_id: uuid.UUID) -> bool:
cur_result = await db.execute(select(Folder).where(Folder.id == descendant_id))
cur = cur_result.scalar_one_or_none()
while cur is not None and cur.parent_id is not None:
if cur.parent_id == ancestor_id:
return True
p_result = await db.execute(select(Folder).where(Folder.id == cur.parent_id))
cur = p_result.scalar_one_or_none()
return False
if await _is_descendant(fid, new_parent):
raise HTTPException(
400,
detail={
"detail": "Cannot move folder into its own descendant",
"code": "invalid_move",
},
)
folder.parent_id = new_parent
await db.flush()
# Build path
path_parts = [folder.name]
current_id = folder.parent_id
while current_id is not None:
cur_result = await db.execute(select(Folder).where(Folder.id == current_id))
cur = cur_result.scalar_one_or_none()
if cur is None:
break
path_parts.insert(0, cur.name)
current_id = cur.parent_id
path = "/".join(path_parts)
return {
"id": str(folder.id),
"name": folder.name,
"parent_id": str(folder.parent_id) if folder.parent_id else None,
"created_by": str(folder.created_by),
"deleted_at": None,
"path": path,
"children": [],
}
@router.delete("/folders/{folder_id}", status_code=status.HTTP_204_NO_CONTENT, dependencies=[Depends(require_permission("dms:delete"))])
async def delete_folder(
folder_id: str,
db: AsyncSession = Depends(get_db),
current_user: dict = Depends(get_current_user),
):
"""AC4: DELETE /api/v1/dms/folders/{id} → 204, soft-delete with cascade."""
tenant_id = uuid.UUID(current_user["tenant_id"])
user_id = uuid.UUID(current_user["user_id"])
is_system_admin = current_user.get("role") == "admin"
fid = _parse_uuid(folder_id, "folder_id")
result = await db.execute(
select(Folder).where(
Folder.id == fid,
Folder.tenant_id == tenant_id,
Folder.deleted_at.is_(None),
)
)
folder = result.scalar_one_or_none()
if folder is None:
raise HTTPException(404, detail={"detail": "Folder not found", "code": "not_found"})
if not await check_single_entity_access(db, "dms_folder", fid, user_id, tenant_id, "delete", is_system_admin):
raise HTTPException(403, detail={"detail": "Access denied", "code": "forbidden"})
# Lifecycle hook: dms.folder.before_delete
from app.core.hooks import do_action
await do_action("dms.folder.before_delete", db=db, tenant_id=tenant_id, user_id=user_id, folder_id=str(fid))
from datetime import UTC, datetime
now = datetime.now(UTC)
# Recursively collect all descendant folder IDs
all_folder_ids: list[uuid.UUID] = [fid]
queue: list[uuid.UUID] = [fid]
while queue:
current_id = queue.pop(0)
children_result = await db.execute(
select(Folder).where(
Folder.parent_id == current_id,
Folder.tenant_id == tenant_id,
Folder.deleted_at.is_(None),
)
)
for child in children_result.scalars().all():
all_folder_ids.append(child.id)
queue.append(child.id)
# Soft-delete all folders
await db.execute(update(Folder).where(Folder.id.in_(all_folder_ids)).values(deleted_at=now))
# Soft-delete all files in those folders
await db.execute(
update(DmsFile)
.where(
DmsFile.tenant_id == tenant_id,
DmsFile.folder_id.in_(all_folder_ids),
DmsFile.deleted_at.is_(None),
)
.values(deleted_at=now)
)
await db.flush()
# Lifecycle hook: dms.folder.after_delete
from app.core.hooks import do_action
await do_action("dms.folder.after_delete", db=db, tenant_id=tenant_id, user_id=user_id, folder_id=str(fid))
return Response(status_code=status.HTTP_204_NO_CONTENT)
# ─── Files ───
+4 -4
View File
@@ -4,16 +4,16 @@ from __future__ import annotations
import uuid
from datetime import datetime
from typing import Any
from sqlalchemy import DateTime, ForeignKey, Index, Integer, String, UniqueConstraint, Text
from pgvector.sqlalchemy import Vector
from sqlalchemy import DateTime, ForeignKey, Index, Integer, String, Text, UniqueConstraint
from sqlalchemy.dialects.postgresql import TSVECTOR
from sqlalchemy.dialects.postgresql import UUID as PGUUID
from sqlalchemy.orm import Mapped, mapped_column
from app.core.db import Base, TenantMixin
from app.models.owned_mixin import OwnedMixin
from pgvector.sqlalchemy import Vector
from sqlalchemy.dialects.postgresql import TSVECTOR
from typing import Any
class Folder(Base, TenantMixin, OwnedMixin):
+20 -8
View File
@@ -4,7 +4,6 @@ from __future__ import annotations
from app.plugins.base import BasePlugin
from app.plugins.manifest import (
FrontendDetailTab,
FrontendMenuItem,
FrontendPageRoute,
PluginManifest,
@@ -38,22 +37,35 @@ class DmsPlugin(BasePlugin):
"dms:admin",
],
menu_items=[
FrontendMenuItem(label_key='nav.dms', label='Dateien', path='/dms', icon='FolderOpen', group='Dateien', order=40),
FrontendMenuItem(label_key='nav.dms.trash', label='Papierkorb', path='/dms/trash', icon='Trash2', group='Dateien', order=41),
FrontendMenuItem(label_key='nav.dms', label='Dateien', path='/dms', icon='FolderOpen', group='Dateien', order=40, permission='dms:read'),
FrontendMenuItem(label_key='nav.dms.trash', label='Papierkorb', path='/dms/trash', icon='Trash2', group='Dateien', order=41, permission='dms:read'),
],
page_routes=[
FrontendPageRoute(path='/dms', component='@/pages/Dms', protected=True),
FrontendPageRoute(path='/dms/trash', component='@/pages/DmsTrash', protected=True),
],
detail_tabs=[
FrontendDetailTab(entity_type='contact', label_key='tabs.files', label='Dateien', component='@/components/contact/ContactFilesTab', icon='FolderOpen', order=40, permission='dms:read'),
FrontendPageRoute(path='/dms', component='@/pages/Dms', protected=True, permission='dms:read'),
FrontendPageRoute(path='/dms/trash', component='@/pages/DmsTrash', protected=True, permission='dms:read'),
],
# BUG (ghost component): ContactFilesTab does not exist in the
# frontend — tab removed until implemented (Block I-D).
detail_tabs=[],
author="LeoCRM Team",
min_app_version="1.0.0",
hooks=["dms.before_upload"],
contract_version="1.0.0",
)
def get_entity_models(self) -> dict[str, type]:
"""Entity types owned by DMS for the permission resolver.
``file`` is a legacy alias still used by the permissions routes.
"""
from app.plugins.builtins.dms.models import File, Folder
return {
"dms_file": File,
"dms_folder": Folder,
"file": File,
}
async def on_activate(self, db, service_container, event_bus) -> None:
"""Activate plugin: register restore config + history hooks."""
await super().on_activate(db, service_container, event_bus)
+26 -867
View File
@@ -2,12 +2,10 @@
from __future__ import annotations
import os
import uuid
from fastapi import (
APIRouter,
Body,
Depends,
File,
Form,
@@ -17,25 +15,39 @@ from fastapi import (
status,
)
from fastapi.responses import StreamingResponse
from sqlalchemy import select, update
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from app.core.db import get_db
from app.core.storage import LocalStorage, get_storage_backend
from app.core.visibility import apply_visibility_filter, check_single_entity_access
from app.deps import get_current_user, require_permission
# BUG-018 God-Object-Split: Helper/Konstanten leben jetzt in common.py;
# Re-Exports sichern Import- und Patch-Kompatibilitaet
# (tests patchen app.plugins.builtins.dms.routes.MAX_FILE_SIZE fuer den Upload).
from app.plugins.builtins.dms.common import ( # noqa: F401
ALLOWED_MIME_PREFIXES,
BLOCKED_EXTENSIONS,
CHUNK_SIZE,
MAX_FILE_SIZE,
OFFICE_EXTENSIONS,
_file_storage_path,
_get_file_extension,
_is_blocked_filetype,
_parse_uuid,
_sanitize_filename,
chunk_size,
)
from app.plugins.builtins.dms.folders_routes import router as folders_router
from app.plugins.builtins.dms.models import File as DmsFile
from app.plugins.builtins.dms.models import Folder
from app.plugins.builtins.dms.schemas import (
BulkDeleteRequest,
BulkMoveRequest,
FileMetadataResponse,
FileUpdate,
FolderCreate,
FolderUpdate,
ShareRemoveRequest,
ShareRequest,
)
from app.plugins.builtins.dms.search_bulk_routes import router as search_bulk_router
from app.plugins.builtins.dms.sharing_routes import router as sharing_router
from app.plugins.builtins.permissions.contracts import get_contract as get_perms_contract
# Get Permission model from the permissions contract
@@ -45,460 +57,6 @@ Permission = _perms_contract.Permission
router = APIRouter(prefix="/api/v1/dms", tags=["dms"])
# Office file extensions mapped to Collabora file types
OFFICE_EXTENSIONS = {
".docx": "docx",
".xlsx": "xlsx",
".pptx": "pptx",
}
# Max file size: 100 MB
MAX_FILE_SIZE = 100 * 1024 * 1024
def _parse_uuid(val: str, field: str) -> uuid.UUID:
try:
return uuid.UUID(val)
except (ValueError, TypeError):
raise HTTPException(
400, detail={"detail": f"Invalid {field}", "code": "invalid_id"}
) from None
def _file_storage_path(tenant_id: uuid.UUID, file_id: uuid.UUID) -> str:
"""Build relative storage path for a file (relative to storage base)."""
return f"{tenant_id}/{file_id}"
def _get_file_extension(filename: str) -> str:
"""Extract lowercase extension including dot."""
return os.path.splitext(filename)[1].lower()
def _sanitize_filename(filename: str) -> str:
"""Sanitize a filename for safe use in Content-Disposition headers."""
import re
# Extract basename only (strip any path components)
safe = os.path.basename(filename.replace('\\', '/'))
# Remove dangerous characters (keep alnum, dot, dash, underscore, space, unicode)
safe = re.sub(r'[^a-zA-Z0-9.\-_\u00c0-\u017f\u4e00-\u9fff ]', '_', safe)
# Collapse consecutive dots (path traversal prevention)
safe = re.sub(r'\.{2,}', '_', safe)
# Collapse multiple spaces
safe = re.sub(r' {2,}', ' ', safe)
# Strip leading dots and whitespace
safe = safe.lstrip('.').strip()
# Limit length
if len(safe) > 200:
name, ext = safe.rsplit('.', 1) if '.' in safe[:200] else (safe[:200], '')
safe = name[:200] + ('.' + ext if ext else '')
return safe or 'file'
# Blocked file extensions for security
BLOCKED_EXTENSIONS = {
".exe", ".bat", ".cmd", ".sh", ".jar", ".com", ".scr", ".msi",
".dll", ".vbs", ".ps1", ".app", ".bin", ".reg", ".inf",
".php", ".py", ".pl", ".asp", ".aspx", ".jsp", ".svg", ".htaccess",
".phtml", ".pht", ".cgi", ".cfm", ".erb",
}
# Allowed MIME types for upload validation
ALLOWED_MIME_PREFIXES = {
"application/pdf",
"application/msword",
"application/vnd.openxmlformats-officedocument",
"application/vnd.oasis.opendocument",
"application/vnd.ms-excel",
"application/vnd.ms-powerpoint",
"application/zip",
"application/gzip",
"application/x-tar",
"application/json",
"application/xml",
"application/rtf",
"application/x-7z-compressed",
"application/x-rar-compressed",
"text/plain",
"text/csv",
"text/html",
"text/markdown",
"image/png",
"image/jpeg",
"image/gif",
"image/webp",
"image/bmp",
"image/tiff",
"image/x-icon",
"audio/",
"video/",
"application/octet-stream",
}
def _is_blocked_filetype(filename: str) -> bool:
"""Check if a file has a blocked (dangerous) extension."""
ext = os.path.splitext(filename)[1].lower()
return ext in BLOCKED_EXTENSIONS
chunk_size = 1024 * 1024 # 1MB chunks for streaming uploads
# ─── Folders ───
@router.get("/folders", dependencies=[Depends(require_permission("dms:read"))])
async def list_folders(
parent_id: str | None = None,
db: AsyncSession = Depends(get_db),
current_user: dict = Depends(get_current_user),
):
"""AC1: GET /api/v1/dms/folders → 200 + folder tree (recursive)."""
tenant_id = uuid.UUID(current_user["tenant_id"])
# Fetch all non-deleted folders for tenant with visibility filter
user_id = uuid.UUID(current_user["user_id"])
is_system_admin = current_user.get("role") == "admin"
query = select(Folder).where(
Folder.tenant_id == tenant_id,
Folder.deleted_at.is_(None),
)
query = await apply_visibility_filter(
db, query, "dms_folder", Folder, user_id, tenant_id, is_system_admin
)
result = await db.execute(query)
all_folders = result.scalars().all()
# Build lookup map
folder_map: dict[uuid.UUID, dict] = {}
for f in all_folders:
folder_map[f.id] = {
"id": str(f.id),
"name": f.name,
"parent_id": str(f.parent_id) if f.parent_id else None,
"created_by": str(f.created_by),
"deleted_at": None,
"path": "",
"children": [],
}
# Build path for each folder
def _build_path(folder_id: uuid.UUID) -> str:
if folder_id not in folder_map:
return ""
f = folder_map[folder_id]
if f["parent_id"] and uuid.UUID(f["parent_id"]) in folder_map:
parent_path = _build_path(uuid.UUID(f["parent_id"]))
return f"{parent_path}/{f['name']}"
return f["name"]
for fid in folder_map:
folder_map[fid]["path"] = _build_path(fid)
# Build tree
root_nodes: list[dict] = []
target_parent: uuid.UUID | None = None
if parent_id is not None:
target_parent = _parse_uuid(parent_id, "parent_id")
for f in all_folders:
node = folder_map[f.id]
if f.parent_id is not None and f.parent_id in folder_map:
folder_map[f.parent_id]["children"].append(node)
elif f.parent_id is None:
root_nodes.append(node)
if target_parent is not None:
# Return children of specified parent
parent_node = folder_map.get(target_parent)
if parent_node is None:
raise HTTPException(
404, detail={"detail": "Parent folder not found", "code": "not_found"}
)
return parent_node["children"]
return root_nodes
@router.post("/folders", status_code=status.HTTP_201_CREATED, dependencies=[Depends(require_permission("dms:write"))])
async def create_folder(
body: FolderCreate,
db: AsyncSession = Depends(get_db),
current_user: dict = Depends(get_current_user),
):
"""AC2: POST /api/v1/dms/folders → 201, folder created with path."""
tenant_id = uuid.UUID(current_user["tenant_id"])
user_id = uuid.UUID(current_user["user_id"])
parent_id = _parse_uuid(body.parent_id, "parent_id") if body.parent_id else None
# Validate parent exists if specified
if parent_id is not None:
parent_result = await db.execute(
select(Folder).where(
Folder.id == parent_id,
Folder.tenant_id == tenant_id,
Folder.deleted_at.is_(None),
)
)
if parent_result.scalar_one_or_none() is None:
raise HTTPException(
404, detail={"detail": "Parent folder not found", "code": "not_found"}
)
# Check name uniqueness within same parent (non-deleted)
existing = await db.execute(
select(Folder).where(
Folder.tenant_id == tenant_id,
Folder.name == body.name,
Folder.parent_id == parent_id if parent_id else Folder.parent_id.is_(None),
Folder.deleted_at.is_(None),
)
)
if existing.scalar_one_or_none() is not None:
raise HTTPException(
409, detail={"detail": "Folder name already exists", "code": "duplicate"}
)
# Lifecycle hook: dms.folder.before_create
from app.core.hooks import do_action
await do_action("dms.folder.before_create", body, db=db, tenant_id=tenant_id, user_id=user_id)
folder = Folder(
tenant_id=tenant_id,
name=body.name,
parent_id=parent_id,
created_by=user_id,
)
db.add(folder)
await db.flush()
# Lifecycle hook: dms.folder.after_create
await do_action("dms.folder.after_create", {'id': str(folder.id), 'name': folder.name, 'parent_id': str(folder.parent_id) if folder.parent_id else None}, db=db, tenant_id=tenant_id, user_id=user_id)
# Build path
path = body.name
if parent_id is not None:
parent_path_result = await db.execute(select(Folder).where(Folder.id == parent_id))
parent_folder = parent_path_result.scalar_one_or_none()
if parent_folder:
# Recursively build path
path_parts = [body.name]
current = parent_folder
while current is not None:
path_parts.insert(0, current.name)
if current.parent_id is not None:
cur_result = await db.execute(
select(Folder).where(Folder.id == current.parent_id)
)
current = cur_result.scalar_one_or_none()
else:
current = None
path = "/".join(path_parts)
return {
"id": str(folder.id),
"name": folder.name,
"parent_id": str(folder.parent_id) if folder.parent_id else None,
"created_by": str(folder.created_by),
"deleted_at": None,
"path": path,
"children": [],
}
@router.patch("/folders/{folder_id}", dependencies=[Depends(require_permission("dms:write"))])
async def update_folder(
folder_id: str,
body: FolderUpdate,
db: AsyncSession = Depends(get_db),
current_user: dict = Depends(get_current_user),
):
"""AC3: PATCH /api/v1/dms/folders/{id} → 200, rename/move."""
tenant_id = uuid.UUID(current_user["tenant_id"])
user_id = uuid.UUID(current_user["user_id"])
is_system_admin = current_user.get("role") == "admin"
fid = _parse_uuid(folder_id, "folder_id")
result = await db.execute(
select(Folder).where(
Folder.id == fid,
Folder.tenant_id == tenant_id,
Folder.deleted_at.is_(None),
)
)
folder = result.scalar_one_or_none()
if folder is None:
raise HTTPException(404, detail={"detail": "Folder not found", "code": "not_found"})
if not await check_single_entity_access(db, "dms_folder", fid, user_id, tenant_id, "write", is_system_admin):
raise HTTPException(403, detail={"detail": "Access denied", "code": "forbidden"})
data = body.model_dump(exclude_unset=True)
if "name" in data and data["name"] is not None:
# Check uniqueness if name is changing
new_parent_id = folder.parent_id
if "parent_id" in data and data["parent_id"] is not None:
new_parent_id = _parse_uuid(data["parent_id"], "parent_id")
dup = await db.execute(
select(Folder).where(
Folder.tenant_id == tenant_id,
Folder.name == data["name"],
Folder.id != fid,
Folder.parent_id == new_parent_id if new_parent_id else Folder.parent_id.is_(None),
Folder.deleted_at.is_(None),
)
)
if dup.scalar_one_or_none() is not None:
raise HTTPException(
409, detail={"detail": "Folder name already exists", "code": "duplicate"}
)
folder.name = data["name"]
if "parent_id" in data:
new_parent = _parse_uuid(data["parent_id"], "parent_id") if data["parent_id"] else None
if new_parent is not None:
# Validate parent exists and not creating a cycle
if new_parent == fid:
raise HTTPException(
400, detail={"detail": "Cannot move folder into itself", "code": "invalid_move"}
)
parent_result = await db.execute(
select(Folder).where(
Folder.id == new_parent,
Folder.tenant_id == tenant_id,
Folder.deleted_at.is_(None),
)
)
if parent_result.scalar_one_or_none() is None:
raise HTTPException(
404, detail={"detail": "Parent folder not found", "code": "not_found"}
)
# Check for cycle: ensure new_parent is not a descendant of folder
async def _is_descendant(ancestor_id: uuid.UUID, descendant_id: uuid.UUID) -> bool:
cur_result = await db.execute(select(Folder).where(Folder.id == descendant_id))
cur = cur_result.scalar_one_or_none()
while cur is not None and cur.parent_id is not None:
if cur.parent_id == ancestor_id:
return True
p_result = await db.execute(select(Folder).where(Folder.id == cur.parent_id))
cur = p_result.scalar_one_or_none()
return False
if await _is_descendant(fid, new_parent):
raise HTTPException(
400,
detail={
"detail": "Cannot move folder into its own descendant",
"code": "invalid_move",
},
)
folder.parent_id = new_parent
await db.flush()
# Build path
path_parts = [folder.name]
current_id = folder.parent_id
while current_id is not None:
cur_result = await db.execute(select(Folder).where(Folder.id == current_id))
cur = cur_result.scalar_one_or_none()
if cur is None:
break
path_parts.insert(0, cur.name)
current_id = cur.parent_id
path = "/".join(path_parts)
return {
"id": str(folder.id),
"name": folder.name,
"parent_id": str(folder.parent_id) if folder.parent_id else None,
"created_by": str(folder.created_by),
"deleted_at": None,
"path": path,
"children": [],
}
@router.delete("/folders/{folder_id}", status_code=status.HTTP_204_NO_CONTENT, dependencies=[Depends(require_permission("dms:delete"))])
async def delete_folder(
folder_id: str,
db: AsyncSession = Depends(get_db),
current_user: dict = Depends(get_current_user),
):
"""AC4: DELETE /api/v1/dms/folders/{id} → 204, soft-delete with cascade."""
tenant_id = uuid.UUID(current_user["tenant_id"])
user_id = uuid.UUID(current_user["user_id"])
is_system_admin = current_user.get("role") == "admin"
fid = _parse_uuid(folder_id, "folder_id")
result = await db.execute(
select(Folder).where(
Folder.id == fid,
Folder.tenant_id == tenant_id,
Folder.deleted_at.is_(None),
)
)
folder = result.scalar_one_or_none()
if folder is None:
raise HTTPException(404, detail={"detail": "Folder not found", "code": "not_found"})
if not await check_single_entity_access(db, "dms_folder", fid, user_id, tenant_id, "delete", is_system_admin):
raise HTTPException(403, detail={"detail": "Access denied", "code": "forbidden"})
# Lifecycle hook: dms.folder.before_delete
from app.core.hooks import do_action
await do_action("dms.folder.before_delete", db=db, tenant_id=tenant_id, user_id=user_id, folder_id=str(fid))
from datetime import UTC, datetime
now = datetime.now(UTC)
# Recursively collect all descendant folder IDs
all_folder_ids: list[uuid.UUID] = [fid]
queue: list[uuid.UUID] = [fid]
while queue:
current_id = queue.pop(0)
children_result = await db.execute(
select(Folder).where(
Folder.parent_id == current_id,
Folder.tenant_id == tenant_id,
Folder.deleted_at.is_(None),
)
)
for child in children_result.scalars().all():
all_folder_ids.append(child.id)
queue.append(child.id)
# Soft-delete all folders
await db.execute(update(Folder).where(Folder.id.in_(all_folder_ids)).values(deleted_at=now))
# Soft-delete all files in those folders
await db.execute(
update(DmsFile)
.where(
DmsFile.tenant_id == tenant_id,
DmsFile.folder_id.in_(all_folder_ids),
DmsFile.deleted_at.is_(None),
)
.values(deleted_at=now)
)
await db.flush()
# Lifecycle hook: dms.folder.after_delete
from app.core.hooks import do_action
await do_action("dms.folder.after_delete", db=db, tenant_id=tenant_id, user_id=user_id, folder_id=str(fid))
return Response(status_code=status.HTTP_204_NO_CONTENT)
# ─── Files ───
@router.post("/files/upload", status_code=status.HTTP_201_CREATED, response_model=FileMetadataResponse, dependencies=[Depends(require_permission("dms:write"))])
async def upload_file(
file: UploadFile = File(...),
@@ -547,7 +105,7 @@ async def upload_file(
# Stream file to storage — avoid loading entire file into RAM
import hashlib
chunk_size = 1024 * 1024 # 1MB chunks
chunk_size = 1024 * 1024 # noqa: F811 (Original-Shadowing im Original auch so)
sha256 = hashlib.sha256()
file_size = 0
@@ -1087,406 +645,7 @@ async def download_file(
)
@router.post("/files/{file_id}/edit-session", dependencies=[Depends(require_permission("dms:write"))])
async def create_edit_session(
file_id: str,
db: AsyncSession = Depends(get_db),
current_user: dict = Depends(get_current_user),
):
"""AC11: POST /api/v1/dms/files/{id}/edit-session → 200 + Collabora config."""
tenant_id = uuid.UUID(current_user["tenant_id"])
user_id = current_user["user_id"]
user_name = current_user.get("name", "Unknown")
is_system_admin = current_user.get("role") == "admin"
fid = _parse_uuid(file_id, "file_id")
result = await db.execute(
select(DmsFile).where(
DmsFile.id == fid,
DmsFile.tenant_id == tenant_id,
DmsFile.deleted_at.is_(None),
)
)
dms_file = result.scalar_one_or_none()
if dms_file is None:
raise HTTPException(404, detail={"detail": "File not found", "code": "not_found"})
if not await check_single_entity_access(db, "dms_file", fid, user_id, tenant_id, "write", is_system_admin):
raise HTTPException(403, detail={"detail": "Access denied", "code": "forbidden"})
ext = _get_file_extension(dms_file.name)
if ext not in OFFICE_EXTENSIONS:
raise HTTPException(
400,
detail={
"detail": "Only Office files (docx, xlsx, pptx) are supported",
"code": "not_office",
},
)
file_type = OFFICE_EXTENSIONS[ext]
download_url = f"/api/v1/dms/files/{fid}/preview"
callback_url = f"/api/v1/dms/files/{fid}/callback"
config = {
"document": {
"fileType": file_type,
"key": str(uuid.uuid4()),
"title": dms_file.name,
"url": download_url,
},
"editorConfig": {
"mode": "edit",
"callbackUrl": callback_url,
"user": {
"id": user_id,
"name": user_name,
},
},
}
return config
# ─── Internal Sharing ───
@router.post("/files/{file_id}/share", dependencies=[Depends(require_permission("dms:share"))])
async def share_file(
file_id: str,
body: ShareRequest,
db: AsyncSession = Depends(get_db),
current_user: dict = Depends(get_current_user),
):
"""AC12: POST /api/v1/dms/files/{id}/share → 200, internal share created."""
tenant_id = uuid.UUID(current_user["tenant_id"])
user_id = uuid.UUID(current_user["user_id"])
is_system_admin = current_user.get("role") == "admin"
fid = _parse_uuid(file_id, "file_id")
# Verify file exists
file_result = await db.execute(
select(DmsFile).where(
DmsFile.id == fid,
DmsFile.tenant_id == tenant_id,
DmsFile.deleted_at.is_(None),
)
)
if file_result.scalar_one_or_none() is None:
raise HTTPException(404, detail={"detail": "File not found", "code": "not_found"})
if not await check_single_entity_access(db, "dms_file", fid, user_id, tenant_id, "share", is_system_admin):
raise HTTPException(403, detail={"detail": "Access denied", "code": "forbidden"})
created_perms: list[dict] = []
for uid_str in body.user_ids:
uid = _parse_uuid(uid_str, "user_id")
# Check if already exists
existing = await db.execute(
select(Permission).where(
Permission.tenant_id == tenant_id,
Permission.file_id == fid,
Permission.user_id == uid,
Permission.access_level == body.access_level,
)
)
if existing.scalar_one_or_none() is None:
perm = Permission(
tenant_id=tenant_id,
file_id=fid,
user_id=uid,
group_id=None,
access_level=body.access_level,
)
db.add(perm)
await db.flush()
created_perms.append(
{
"id": str(perm.id),
"file_id": str(fid),
"user_id": str(uid),
"group_id": None,
"access_level": body.access_level,
}
)
for gid_str in body.group_ids:
gid = _parse_uuid(gid_str, "group_id")
existing = await db.execute(
select(Permission).where(
Permission.tenant_id == tenant_id,
Permission.file_id == fid,
Permission.group_id == gid,
Permission.access_level == body.access_level,
)
)
if existing.scalar_one_or_none() is None:
perm = Permission(
tenant_id=tenant_id,
file_id=fid,
user_id=uuid.UUID(current_user["user_id"]),
group_id=gid,
access_level=body.access_level,
)
db.add(perm)
await db.flush()
created_perms.append(
{
"id": str(perm.id),
"file_id": str(fid),
"user_id": str(perm.user_id),
"group_id": str(gid),
"access_level": body.access_level,
}
)
return {
"file_id": str(fid),
"shared_with": created_perms,
"count": len(created_perms),
}
@router.delete("/files/{file_id}/share", status_code=status.HTTP_204_NO_CONTENT, dependencies=[Depends(require_permission("dms:share"))])
async def remove_share(
file_id: str,
body: ShareRemoveRequest = Body(...),
db: AsyncSession = Depends(get_db),
current_user: dict = Depends(get_current_user),
):
"""AC13: DELETE /api/v1/dms/files/{id}/share → 204, share removed."""
tenant_id = uuid.UUID(current_user["tenant_id"])
user_id = uuid.UUID(current_user["user_id"])
is_system_admin = current_user.get("role") == "admin"
fid = _parse_uuid(file_id, "file_id")
if not await check_single_entity_access(db, "dms_file", fid, user_id, tenant_id, "share", is_system_admin):
raise HTTPException(403, detail={"detail": "Access denied", "code": "forbidden"})
if body.user_id:
uid = _parse_uuid(body.user_id, "user_id")
result = await db.execute(
select(Permission).where(
Permission.tenant_id == tenant_id,
Permission.file_id == fid,
Permission.user_id == uid,
)
)
perms = result.scalars().all()
for p in perms:
await db.delete(p)
if body.group_id:
gid = _parse_uuid(body.group_id, "group_id")
result = await db.execute(
select(Permission).where(
Permission.tenant_id == tenant_id,
Permission.file_id == fid,
Permission.group_id == gid,
)
)
perms = result.scalars().all()
for p in perms:
await db.delete(p)
await db.flush()
return Response(status_code=status.HTTP_204_NO_CONTENT)
# ─── Search & Bulk ───
@router.get("/search", dependencies=[Depends(require_permission("dms:read"))])
async def search_files(
q: str,
db: AsyncSession = Depends(get_db),
current_user: dict = Depends(get_current_user),
):
"""AC16: GET /api/v1/dms/search?q=text → 200 + matching files (ILIKE)."""
tenant_id = uuid.UUID(current_user["tenant_id"])
user_id = uuid.UUID(current_user["user_id"])
is_system_admin = current_user.get("role") == "admin"
query = select(DmsFile).where(
DmsFile.tenant_id == tenant_id,
DmsFile.deleted_at.is_(None),
DmsFile.name.ilike(f"%{q}%"),
)
query = await apply_visibility_filter(
db, query, "dms_file", DmsFile, user_id, tenant_id, is_system_admin
)
result = await db.execute(query)
files = result.scalars().all()
return [
{
"id": str(f.id),
"name": f.name,
"folder_id": str(f.folder_id) if f.folder_id else None,
"uploaded_by": str(f.uploaded_by),
"mime_type": f.mime_type,
"size_bytes": f.size_bytes,
"deleted_at": None,
"created_at": f.created_at.isoformat() if f.created_at else None,
}
for f in files
]
@router.get("/shared-with-me", dependencies=[Depends(require_permission("dms:read"))])
async def shared_with_me(
db: AsyncSession = Depends(get_db),
current_user: dict = Depends(get_current_user),
):
"""AC17: GET /api/v1/dms/shared-with-me → 200 + shared files list."""
tenant_id = uuid.UUID(current_user["tenant_id"])
user_id = uuid.UUID(current_user["user_id"])
is_system_admin = current_user.get("role") == "admin"
# Query permissions for this user and join with files
perm_result = await db.execute(
select(Permission).where(
Permission.tenant_id == tenant_id,
Permission.user_id == user_id,
)
)
perms = perm_result.scalars().all()
file_ids = {p.file_id for p in perms}
if not file_ids:
return {"items": [], "total": 0}
query = select(DmsFile).where(
DmsFile.tenant_id == tenant_id,
DmsFile.id.in_(file_ids),
DmsFile.deleted_at.is_(None),
)
query = await apply_visibility_filter(
db, query, "dms_file", DmsFile, user_id, tenant_id, is_system_admin
)
result = await db.execute(query)
files = result.scalars().all()
# Map permissions for access_level
perm_map: dict[uuid.UUID, str] = {}
for p in perms:
if p.file_id in file_ids:
perm_map[p.file_id] = p.access_level
return [
{
"id": str(f.id),
"name": f.name,
"folder_id": str(f.folder_id) if f.folder_id else None,
"uploaded_by": str(f.uploaded_by),
"mime_type": f.mime_type,
"size_bytes": f.size_bytes,
"access_level": perm_map.get(f.id, "read"),
"created_at": f.created_at.isoformat() if f.created_at else None,
}
for f in files
]
@router.post("/files/bulk-move", dependencies=[Depends(require_permission("dms:write"))])
async def bulk_move(
body: BulkMoveRequest,
db: AsyncSession = Depends(get_db),
current_user: dict = Depends(get_current_user),
):
"""AC18: POST /api/v1/dms/files/bulk-move → 200, files moved."""
tenant_id = uuid.UUID(current_user["tenant_id"])
user_id = uuid.UUID(current_user["user_id"])
is_system_admin = current_user.get("role") == "admin"
target_folder_id = (
_parse_uuid(body.target_folder_id, "target_folder_id") if body.target_folder_id else None
)
# Validate target folder if specified
if target_folder_id is not None:
folder_result = await db.execute(
select(Folder).where(
Folder.id == target_folder_id,
Folder.tenant_id == tenant_id,
Folder.deleted_at.is_(None),
)
)
if folder_result.scalar_one_or_none() is None:
raise HTTPException(
404, detail={"detail": "Target folder not found", "code": "not_found"}
)
file_ids = [_parse_uuid(fid, "file_id") for fid in body.file_ids]
query = select(DmsFile).where(
DmsFile.tenant_id == tenant_id,
DmsFile.id.in_(file_ids),
DmsFile.deleted_at.is_(None),
)
query = await apply_visibility_filter(
db, query, "dms_file", DmsFile, user_id, tenant_id, is_system_admin
)
result = await db.execute(query)
files = result.scalars().all()
moved_count = 0
for f in files:
f.folder_id = target_folder_id
moved_count += 1
await db.flush()
return {
"moved": moved_count,
"file_ids": [str(fid) for fid in file_ids],
"target_folder_id": str(target_folder_id) if target_folder_id else None,
}
@router.post("/files/bulk-delete", dependencies=[Depends(require_permission("dms:delete"))])
async def bulk_delete(
body: BulkDeleteRequest,
db: AsyncSession = Depends(get_db),
current_user: dict = Depends(get_current_user),
):
"""AC19: POST /api/v1/dms/files/bulk-delete → 200, files soft-deleted."""
tenant_id = uuid.UUID(current_user["tenant_id"])
user_id = uuid.UUID(current_user["user_id"])
is_system_admin = current_user.get("role") == "admin"
file_ids = [_parse_uuid(fid, "file_id") for fid in body.file_ids]
from datetime import UTC, datetime
now = datetime.now(UTC)
# Apply visibility filter to only delete files user has access to
query = select(DmsFile).where(
DmsFile.tenant_id == tenant_id,
DmsFile.id.in_(file_ids),
DmsFile.deleted_at.is_(None),
)
query = await apply_visibility_filter(
db, query, "dms_file", DmsFile, user_id, tenant_id, is_system_admin
)
result = await db.execute(query)
accessible_files = result.scalars().all()
accessible_ids = [f.id for f in accessible_files]
result = await db.execute(
update(DmsFile)
.where(
DmsFile.tenant_id == tenant_id,
DmsFile.id.in_(accessible_ids),
DmsFile.deleted_at.is_(None),
)
.values(deleted_at=now)
)
deleted_count = result.rowcount
await db.flush()
return {
"deleted": deleted_count,
"file_ids": body.file_ids,
}
# Sub-Router einbinden (BUG-018 Split): folders, sharing/collabora, search/bulk
router.include_router(folders_router)
router.include_router(sharing_router)
router.include_router(search_bulk_router)
@@ -0,0 +1,223 @@
"""DMS Search / shared-with-me / Bulk Routen — extrahiert aus routes.py (BUG-018)."""
from __future__ import annotations
import uuid
from fastapi import (
APIRouter,
Depends,
HTTPException,
)
from sqlalchemy import select, update
from sqlalchemy.ext.asyncio import AsyncSession
from app.core.db import get_db
from app.core.visibility import apply_visibility_filter
from app.deps import get_current_user, require_permission
from app.plugins.builtins.dms.common import (
_parse_uuid,
)
from app.plugins.builtins.dms.models import File as DmsFile
from app.plugins.builtins.dms.models import Folder
from app.plugins.builtins.dms.schemas import BulkDeleteRequest, BulkMoveRequest
from app.plugins.builtins.permissions.contracts import get_contract as get_perms_contract
_perms_contract = get_perms_contract()
Permission = _perms_contract.Permission
router = APIRouter(tags=["dms"])
@router.get("/search", dependencies=[Depends(require_permission("dms:read"))])
async def search_files(
q: str,
db: AsyncSession = Depends(get_db),
current_user: dict = Depends(get_current_user),
):
"""AC16: GET /api/v1/dms/search?q=text → 200 + matching files (ILIKE)."""
tenant_id = uuid.UUID(current_user["tenant_id"])
user_id = uuid.UUID(current_user["user_id"])
is_system_admin = current_user.get("role") == "admin"
query = select(DmsFile).where(
DmsFile.tenant_id == tenant_id,
DmsFile.deleted_at.is_(None),
DmsFile.name.ilike(f"%{q}%"),
)
query = await apply_visibility_filter(
db, query, "dms_file", DmsFile, user_id, tenant_id, is_system_admin
)
result = await db.execute(query)
files = result.scalars().all()
return [
{
"id": str(f.id),
"name": f.name,
"folder_id": str(f.folder_id) if f.folder_id else None,
"uploaded_by": str(f.uploaded_by),
"mime_type": f.mime_type,
"size_bytes": f.size_bytes,
"deleted_at": None,
"created_at": f.created_at.isoformat() if f.created_at else None,
}
for f in files
]
@router.get("/shared-with-me", dependencies=[Depends(require_permission("dms:read"))])
async def shared_with_me(
db: AsyncSession = Depends(get_db),
current_user: dict = Depends(get_current_user),
):
"""AC17: GET /api/v1/dms/shared-with-me → 200 + shared files list."""
tenant_id = uuid.UUID(current_user["tenant_id"])
user_id = uuid.UUID(current_user["user_id"])
is_system_admin = current_user.get("role") == "admin"
# Query permissions for this user and join with files
perm_result = await db.execute(
select(Permission).where(
Permission.tenant_id == tenant_id,
Permission.user_id == user_id,
)
)
perms = perm_result.scalars().all()
file_ids = {p.file_id for p in perms}
if not file_ids:
return []
query = select(DmsFile).where(
DmsFile.tenant_id == tenant_id,
DmsFile.id.in_(file_ids),
DmsFile.deleted_at.is_(None),
)
query = await apply_visibility_filter(
db, query, "dms_file", DmsFile, user_id, tenant_id, is_system_admin
)
result = await db.execute(query)
files = result.scalars().all()
# Map permissions for access_level
perm_map: dict[uuid.UUID, str] = {}
for p in perms:
if p.file_id in file_ids:
perm_map[p.file_id] = p.access_level
return [
{
"id": str(f.id),
"name": f.name,
"folder_id": str(f.folder_id) if f.folder_id else None,
"uploaded_by": str(f.uploaded_by),
"mime_type": f.mime_type,
"size_bytes": f.size_bytes,
"access_level": perm_map.get(f.id, "read"),
"created_at": f.created_at.isoformat() if f.created_at else None,
}
for f in files
]
@router.post("/files/bulk-move", dependencies=[Depends(require_permission("dms:write"))])
async def bulk_move(
body: BulkMoveRequest,
db: AsyncSession = Depends(get_db),
current_user: dict = Depends(get_current_user),
):
"""AC18: POST /api/v1/dms/files/bulk-move → 200, files moved."""
tenant_id = uuid.UUID(current_user["tenant_id"])
user_id = uuid.UUID(current_user["user_id"])
is_system_admin = current_user.get("role") == "admin"
target_folder_id = (
_parse_uuid(body.target_folder_id, "target_folder_id") if body.target_folder_id else None
)
# Validate target folder if specified
if target_folder_id is not None:
folder_result = await db.execute(
select(Folder).where(
Folder.id == target_folder_id,
Folder.tenant_id == tenant_id,
Folder.deleted_at.is_(None),
)
)
if folder_result.scalar_one_or_none() is None:
raise HTTPException(
404, detail={"detail": "Target folder not found", "code": "not_found"}
)
file_ids = [_parse_uuid(fid, "file_id") for fid in body.file_ids]
query = select(DmsFile).where(
DmsFile.tenant_id == tenant_id,
DmsFile.id.in_(file_ids),
DmsFile.deleted_at.is_(None),
)
query = await apply_visibility_filter(
db, query, "dms_file", DmsFile, user_id, tenant_id, is_system_admin
)
result = await db.execute(query)
files = result.scalars().all()
moved_count = 0
for f in files:
f.folder_id = target_folder_id
moved_count += 1
await db.flush()
return {
"moved": moved_count,
"file_ids": [str(fid) for fid in file_ids],
"target_folder_id": str(target_folder_id) if target_folder_id else None,
}
@router.post("/files/bulk-delete", dependencies=[Depends(require_permission("dms:delete"))])
async def bulk_delete(
body: BulkDeleteRequest,
db: AsyncSession = Depends(get_db),
current_user: dict = Depends(get_current_user),
):
"""AC19: POST /api/v1/dms/files/bulk-delete → 200, files soft-deleted."""
tenant_id = uuid.UUID(current_user["tenant_id"])
user_id = uuid.UUID(current_user["user_id"])
is_system_admin = current_user.get("role") == "admin"
file_ids = [_parse_uuid(fid, "file_id") for fid in body.file_ids]
from datetime import UTC, datetime
now = datetime.now(UTC)
# Apply visibility filter to only delete files user has access to
query = select(DmsFile).where(
DmsFile.tenant_id == tenant_id,
DmsFile.id.in_(file_ids),
DmsFile.deleted_at.is_(None),
)
query = await apply_visibility_filter(
db, query, "dms_file", DmsFile, user_id, tenant_id, is_system_admin
)
result = await db.execute(query)
accessible_files = result.scalars().all()
accessible_ids = [f.id for f in accessible_files]
result = await db.execute(
update(DmsFile)
.where(
DmsFile.tenant_id == tenant_id,
DmsFile.id.in_(accessible_ids),
DmsFile.deleted_at.is_(None),
)
.values(deleted_at=now)
)
deleted_count = result.rowcount
await db.flush()
return {
"deleted": deleted_count,
"file_ids": body.file_ids,
}
+242
View File
@@ -0,0 +1,242 @@
"""DMS Edit-Session/Collabora & Sharing Routen — extrahiert aus routes.py (BUG-018)."""
from __future__ import annotations
import uuid
from fastapi import (
APIRouter,
Body,
Depends,
HTTPException,
Response,
status,
)
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from app.core.db import get_db
from app.core.visibility import check_single_entity_access
from app.deps import get_current_user, require_permission
from app.plugins.builtins.dms.common import (
OFFICE_EXTENSIONS,
_get_file_extension,
_parse_uuid,
)
from app.plugins.builtins.dms.models import File as DmsFile
from app.plugins.builtins.dms.schemas import ShareRemoveRequest, ShareRequest
from app.plugins.builtins.permissions.contracts import get_contract as get_perms_contract
_perms_contract = get_perms_contract()
Permission = _perms_contract.Permission
router = APIRouter(tags=["dms"])
@router.post("/files/{file_id}/edit-session", dependencies=[Depends(require_permission("dms:write"))])
async def create_edit_session(
file_id: str,
db: AsyncSession = Depends(get_db),
current_user: dict = Depends(get_current_user),
):
"""AC11: POST /api/v1/dms/files/{id}/edit-session → 200 + Collabora config."""
tenant_id = uuid.UUID(current_user["tenant_id"])
user_id = current_user["user_id"]
user_name = current_user.get("name", "Unknown")
is_system_admin = current_user.get("role") == "admin"
fid = _parse_uuid(file_id, "file_id")
result = await db.execute(
select(DmsFile).where(
DmsFile.id == fid,
DmsFile.tenant_id == tenant_id,
DmsFile.deleted_at.is_(None),
)
)
dms_file = result.scalar_one_or_none()
if dms_file is None:
raise HTTPException(404, detail={"detail": "File not found", "code": "not_found"})
if not await check_single_entity_access(db, "dms_file", fid, user_id, tenant_id, "write", is_system_admin):
raise HTTPException(403, detail={"detail": "Access denied", "code": "forbidden"})
ext = _get_file_extension(dms_file.name)
if ext not in OFFICE_EXTENSIONS:
raise HTTPException(
400,
detail={
"detail": "Only Office files (docx, xlsx, pptx) are supported",
"code": "not_office",
},
)
file_type = OFFICE_EXTENSIONS[ext]
download_url = f"/api/v1/dms/files/{fid}/preview"
callback_url = f"/api/v1/dms/files/{fid}/callback"
config = {
"document": {
"fileType": file_type,
"key": str(uuid.uuid4()),
"title": dms_file.name,
"url": download_url,
},
"editorConfig": {
"mode": "edit",
"callbackUrl": callback_url,
"user": {
"id": user_id,
"name": user_name,
},
},
}
return config
# ─── Internal Sharing ───
@router.post("/files/{file_id}/share", dependencies=[Depends(require_permission("dms:share"))])
async def share_file(
file_id: str,
body: ShareRequest,
db: AsyncSession = Depends(get_db),
current_user: dict = Depends(get_current_user),
):
"""AC12: POST /api/v1/dms/files/{id}/share → 200, internal share created."""
tenant_id = uuid.UUID(current_user["tenant_id"])
user_id = uuid.UUID(current_user["user_id"])
is_system_admin = current_user.get("role") == "admin"
fid = _parse_uuid(file_id, "file_id")
# Verify file exists
file_result = await db.execute(
select(DmsFile).where(
DmsFile.id == fid,
DmsFile.tenant_id == tenant_id,
DmsFile.deleted_at.is_(None),
)
)
if file_result.scalar_one_or_none() is None:
raise HTTPException(404, detail={"detail": "File not found", "code": "not_found"})
if not await check_single_entity_access(db, "dms_file", fid, user_id, tenant_id, "share", is_system_admin):
raise HTTPException(403, detail={"detail": "Access denied", "code": "forbidden"})
created_perms: list[dict] = []
for uid_str in body.user_ids:
uid = _parse_uuid(uid_str, "user_id")
# Check if already exists
existing = await db.execute(
select(Permission).where(
Permission.tenant_id == tenant_id,
Permission.file_id == fid,
Permission.user_id == uid,
Permission.access_level == body.access_level,
)
)
if existing.scalar_one_or_none() is None:
perm = Permission(
tenant_id=tenant_id,
file_id=fid,
user_id=uid,
group_id=None,
access_level=body.access_level,
)
db.add(perm)
await db.flush()
created_perms.append(
{
"id": str(perm.id),
"file_id": str(fid),
"user_id": str(uid),
"group_id": None,
"access_level": body.access_level,
}
)
for gid_str in body.group_ids:
gid = _parse_uuid(gid_str, "group_id")
existing = await db.execute(
select(Permission).where(
Permission.tenant_id == tenant_id,
Permission.file_id == fid,
Permission.group_id == gid,
Permission.access_level == body.access_level,
)
)
if existing.scalar_one_or_none() is None:
perm = Permission(
tenant_id=tenant_id,
file_id=fid,
user_id=uuid.UUID(current_user["user_id"]),
group_id=gid,
access_level=body.access_level,
)
db.add(perm)
await db.flush()
created_perms.append(
{
"id": str(perm.id),
"file_id": str(fid),
"user_id": str(perm.user_id),
"group_id": str(gid),
"access_level": body.access_level,
}
)
return {
"file_id": str(fid),
"shared_with": created_perms,
"count": len(created_perms),
}
@router.delete("/files/{file_id}/share", status_code=status.HTTP_204_NO_CONTENT, dependencies=[Depends(require_permission("dms:share"))])
async def remove_share(
file_id: str,
body: ShareRemoveRequest = Body(...),
db: AsyncSession = Depends(get_db),
current_user: dict = Depends(get_current_user),
):
"""AC13: DELETE /api/v1/dms/files/{id}/share → 204, share removed."""
tenant_id = uuid.UUID(current_user["tenant_id"])
user_id = uuid.UUID(current_user["user_id"])
is_system_admin = current_user.get("role") == "admin"
fid = _parse_uuid(file_id, "file_id")
if not await check_single_entity_access(db, "dms_file", fid, user_id, tenant_id, "share", is_system_admin):
raise HTTPException(403, detail={"detail": "Access denied", "code": "forbidden"})
if body.user_id:
uid = _parse_uuid(body.user_id, "user_id")
result = await db.execute(
select(Permission).where(
Permission.tenant_id == tenant_id,
Permission.file_id == fid,
Permission.user_id == uid,
)
)
perms = result.scalars().all()
for p in perms:
await db.delete(p)
if body.group_id:
gid = _parse_uuid(body.group_id, "group_id")
result = await db.execute(
select(Permission).where(
Permission.tenant_id == tenant_id,
Permission.file_id == fid,
Permission.group_id == gid,
)
)
perms = result.scalars().all()
for p in perms:
await db.delete(p)
await db.flush()
return Response(status_code=status.HTTP_204_NO_CONTENT)
# ─── Search & Bulk ───
+4 -4
View File
@@ -5,7 +5,7 @@ from __future__ import annotations
from typing import Any
from app.plugins.base import BasePlugin
from app.plugins.manifest import FrontendDetailTab, PluginManifest, PluginRouteDef
from app.plugins.manifest import PluginManifest, PluginRouteDef
class EntityLinksPlugin(BasePlugin):
@@ -42,9 +42,9 @@ class EntityLinksPlugin(BasePlugin):
"entity_links:delete",
],
is_core=True,
detail_tabs=[
FrontendDetailTab(entity_type='contact', label_key='tabs.links', label='Verknüpfungen', component='@/components/contact/ContactLinksTab', icon='Link', order=60, permission='entity_links:read'),
],
# BUG (ghost component): ContactLinksTab does not exist in the
# frontend — tab removed until implemented (Block I-D).
detail_tabs=[],
author="LeoCRM Team",
min_app_version="1.0.0",
+4 -2
View File
@@ -40,6 +40,10 @@ class GraphRAGPlugin(BasePlugin):
async def on_activate(self, db, service_container, event_bus) -> None:
"""Activate plugin: register GraphRAG search provider."""
# Base class first (event subscriptions etc.), then own registrations
# so a failure in our code cannot leave the base half-initialized (ARCH-037).
await super().on_activate(db, service_container, event_bus)
from app.plugins.builtins.graph_rag.provider import GraphRAGSearchProvider
from app.plugins.builtins.unified_search.contracts import get_search_registry
@@ -50,8 +54,6 @@ class GraphRAGPlugin(BasePlugin):
import logging
logging.getLogger(__name__).exception("Failed to register GraphRAGSearchProvider")
await super().on_activate(db, service_container, event_bus)
async def on_deactivate(self, db, service_container, event_bus) -> None:
"""Deactivate plugin: unregister search provider and contract."""
from app.plugins.builtins.unified_search.contracts import get_search_registry
+2 -1
View File
@@ -25,7 +25,8 @@ async def cleanup_knowledge_job(ctx: dict[str, Any]) -> None:
Runs daily. Keeps approved extractions indefinitely.
Iterates per-tenant for RLS compliance.
"""
from sqlalchemy import text as sa_text, delete as sa_delete
from sqlalchemy import delete as sa_delete
from sqlalchemy import text as sa_text
from app.core.db import get_worker_session_factory
+7 -2
View File
@@ -1,12 +1,17 @@
"""Knowledge extraction models — tracks LLM extractions and review queue."""
from __future__ import annotations
import uuid
from datetime import datetime
from sqlalchemy import DateTime, Float, ForeignKey, Index, Integer, String, Text, func
from sqlalchemy.dialects.postgresql import JSONB, UUID as PGUUID
from sqlalchemy import DateTime, Float, ForeignKey, Index, String, Text, func
from sqlalchemy.dialects.postgresql import JSONB
from sqlalchemy.dialects.postgresql import UUID as PGUUID
from sqlalchemy.orm import Mapped, mapped_column
from app.core.db import Base, TenantMixin
class KnowledgeExtraction(Base, TenantMixin):
"""Tracks a single knowledge extraction run from a source (wiki, dms, mail, comm)."""
__tablename__ = "knowledge_extractions"
+13 -7
View File
@@ -1,8 +1,10 @@
"""Knowledge plugin — LLM-based entity/relationship extraction, ask-knowledge, review queue."""
from __future__ import annotations
import logging
import uuid
from typing import Any
from app.plugins.base import BasePlugin
from app.plugins.manifest import PluginManifest, PluginRouteDef
@@ -25,7 +27,7 @@ class KnowledgePlugin(BasePlugin):
"""Register event-driven extraction hooks on activation."""
await super().on_activate(db, service_container, event_bus)
try:
from app.core.hooks import register_action
from app.core.hooks import get_hook_registry
from app.plugins.builtins.knowledge.services import extract_knowledge
async def on_wiki_create(*args, **kwargs):
article_id = kwargs.get("article_id") or kwargs.get("entity_id")
@@ -41,7 +43,9 @@ class KnowledgePlugin(BasePlugin):
source_type="wiki_article", source_id=uuid.UUID(str(article_id)),
source_title=title, source_text=content,
)
register_action("wiki.article.created", on_wiki_create, priority=20, owner_tag="knowledge")
get_hook_registry().register_action(
"wiki.article.created", on_wiki_create, priority=20, owner_tag="knowledge"
)
# H-DATA-LIFE: Re-extract when wiki article is updated
async def on_wiki_update(*args, **kwargs):
article_id = kwargs.get("article_id") or kwargs.get("entity_id")
@@ -57,7 +61,9 @@ class KnowledgePlugin(BasePlugin):
source_type="wiki_article", source_id=uuid.UUID(str(article_id)),
source_title=title, source_text=content,
)
register_action("wiki.article.updated", on_wiki_update, priority=20, owner_tag="knowledge")
get_hook_registry().register_action(
"wiki.article.updated", on_wiki_update, priority=20, owner_tag="knowledge"
)
logger.info("Registered knowledge extraction hooks")
except Exception:
logger.exception("Failed to register knowledge hooks")
@@ -74,8 +80,8 @@ class KnowledgePlugin(BasePlugin):
async def _ask_knowledge_handler(arguments: dict[str, Any], context: dict[str, Any]) -> dict[str, Any]:
"""Ask a knowledge question."""
from app.plugins.builtins.knowledge.services import ask_knowledge
from app.core.db import get_worker_session_factory
from app.plugins.builtins.knowledge.services import ask_knowledge
question = arguments.get("question", "")
tenant_id = context.get("tenant_id")
if not question or not tenant_id:
@@ -103,8 +109,8 @@ class KnowledgePlugin(BasePlugin):
async def _search_knowledge_handler(arguments: dict[str, Any], context: dict[str, Any]) -> dict[str, Any]:
"""Search wiki articles via unified search."""
from app.plugins.builtins.unified_search.provider_registry import get_search_registry
from app.core.db import get_worker_session_factory
from app.plugins.builtins.unified_search.provider_registry import get_search_registry
query = arguments.get("query", "")
tenant_id = context.get("tenant_id")
if not query or not tenant_id:
@@ -149,8 +155,8 @@ class KnowledgePlugin(BasePlugin):
async def on_deactivate(self, db, service_container, event_bus) -> None:
"""Clean up on deactivation."""
from app.core.hooks import unregister_actions_by_owner
unregister_actions_by_owner("knowledge")
from app.core.hooks import get_hook_registry
get_hook_registry().unregister_all_for_plugin("knowledge")
# Unregister knowledge agent tools from the core AI tool registry
try:
from app.ai.tool_registry import get_tool_registry
+41 -24
View File
@@ -1,51 +1,70 @@
"""Knowledge plugin routes — extraction, ask, review queue."""
from __future__ import annotations
import uuid
from fastapi import APIRouter, Depends, HTTPException, Query
from pydantic import BaseModel, Field
from sqlalchemy.ext.asyncio import AsyncSession
from app.core.db import get_db
from app.deps import require_permission
from app.plugins.builtins.knowledge.services import extract_knowledge, ask_knowledge, get_review_queue, review_extraction
from app.plugins.builtins.knowledge.services import (
ask_knowledge,
extract_knowledge,
get_review_queue,
review_extraction,
)
router = APIRouter(prefix="/api/v1/knowledge", tags=["knowledge"])
class KnowledgeExtractRequest(BaseModel):
"""Extract knowledge from a source."""
source_type: str = Field(..., min_length=1)
source_id: uuid.UUID
source_title: str | None = None
source_text: str = Field(..., min_length=1)
class KnowledgeAskRequest(BaseModel):
"""Ask a knowledge question."""
question: str = Field(..., min_length=1)
class KnowledgeReviewRequest(BaseModel):
"""Approve or reject a knowledge extraction."""
approved: bool = False
notes: str | None = None
@router.post("/extract")
async def extract(
body: dict,
body: KnowledgeExtractRequest,
db: AsyncSession = Depends(get_db),
current_user: dict = Depends(require_permission("wiki:read")),
):
"""Extract knowledge from a source (wiki article, dms file, mail, communication)."""
tenant_id = uuid.UUID(current_user["tenant_id"])
source_type = body.get("source_type", "")
source_id = body.get("source_id", "")
source_title = body.get("source_title")
source_text = body.get("source_text", "")
if not source_type or not source_id or not source_text:
raise HTTPException(400, detail={"detail": "source_type, source_id, source_text required", "code": "missing_fields"})
try:
sid = uuid.UUID(source_id)
except ValueError:
raise HTTPException(400, detail={"detail": "Invalid source_id", "code": "invalid_id"}) from None
result = await extract_knowledge(
db=db, tenant_id=tenant_id, source_type=source_type, source_id=sid,
source_title=source_title, source_text=source_text,
db=db, tenant_id=tenant_id, source_type=body.source_type, source_id=body.source_id,
source_title=body.source_title, source_text=body.source_text,
user_id=uuid.UUID(current_user["user_id"]) if current_user.get("user_id") else None,
)
return result
@router.post("/ask")
async def ask(
body: dict,
body: KnowledgeAskRequest,
db: AsyncSession = Depends(get_db),
current_user: dict = Depends(require_permission("wiki:read")),
):
"""Ask a knowledge question — uses wiki + graph_rag as context."""
tenant_id = uuid.UUID(current_user["tenant_id"])
question = body.get("question") or body.get("query", "")
if not question:
raise HTTPException(400, detail={"detail": "question required", "code": "missing_question"})
result = await ask_knowledge(db=db, tenant_id=tenant_id, question=question)
result = await ask_knowledge(db=db, tenant_id=tenant_id, question=body.question)
return result
@router.get("/review")
@@ -62,22 +81,20 @@ async def review_queue(
@router.post("/review/{extraction_id}")
async def review(
extraction_id: str,
body: dict,
body: KnowledgeReviewRequest,
db: AsyncSession = Depends(get_db),
current_user: dict = Depends(require_permission("wiki:write")),
):
"""Approve or reject a knowledge extraction."""
tenant_id = uuid.UUID(current_user["tenant_id"])
approved = body.get("approved", False)
notes = body.get("notes")
try:
eid = uuid.UUID(extraction_id)
except ValueError:
raise HTTPException(400, detail={"detail": "Invalid extraction_id", "code": "invalid_id"}) from None
result = await review_extraction(
db=db, tenant_id=tenant_id, extraction_id=eid, approved=approved,
db=db, tenant_id=tenant_id, extraction_id=eid, approved=body.approved,
user_id=uuid.UUID(current_user["user_id"]) if current_user.get("user_id") else None,
notes=notes,
notes=body.notes,
)
if "error" in result:
raise HTTPException(404, detail={"detail": result["error"], "code": "not_found"})
+4 -1
View File
@@ -1,10 +1,13 @@
"""Knowledge extraction services — LLM-based entity/relationship extraction."""
from __future__ import annotations
import logging
import uuid
from typing import Any
from sqlalchemy import select, update
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from app.ai.llm_client import llm_complete
from app.plugins.builtins.knowledge.models import KnowledgeExtraction
@@ -0,0 +1,343 @@
"""Conversation CRUD and pin/mute actions for the kommunikation plugin."""
from __future__ import annotations
import logging
import uuid
from typing import Any
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from app.core.event_bus import get_event_bus
from app.plugins.builtins.kommunikation.interactions import _get_unread_count
from app.plugins.builtins.kommunikation.messages import send_message
from app.plugins.builtins.kommunikation.models import (
CommConversation,
CommConversationMute,
CommConversationPin,
CommParticipant,
)
from app.plugins.builtins.kommunikation.rbac import CommRBAC
from app.plugins.builtins.kommunikation.serializers import conversation_to_response
logger = logging.getLogger(__name__)
# ─── conversations ───
async def list_conversations(
db: AsyncSession,
tenant_id: uuid.UUID,
user_id: uuid.UUID,
include_archived: bool = False,
) -> list[dict[str, Any]]:
"""List all conversations for a user."""
# Get conversations where user is a participant
result = await db.execute(
select(CommConversation)
.join(CommParticipant, CommParticipant.conversation_id == CommConversation.id)
.where(
CommParticipant.participant_id == user_id,
CommParticipant.participant_type == "user",
CommParticipant.left_at.is_(None),
CommConversation.tenant_id == tenant_id,
CommConversation.deleted_at.is_(None),
)
.order_by(CommConversation.last_msg_at.desc().nullslast())
)
conversations = result.scalars().all()
# Get user's pinned conversations
pins_result = await db.execute(
select(CommConversationPin).where(
CommConversationPin.user_id == user_id,
CommConversationPin.tenant_id == tenant_id,
)
)
pinned_ids = {p.conversation_id for p in pins_result.scalars().all()}
conv_list = []
for conv in conversations:
if conv.is_archived and not include_archived:
continue
# Get participants
parts_result = await db.execute(
select(CommParticipant).where(
CommParticipant.conversation_id == conv.id,
CommParticipant.left_at.is_(None),
)
)
participants = list(parts_result.scalars().all())
# Get unread count
unread = await _get_unread_count(db, tenant_id, conv.id, user_id)
conv_list.append(
conversation_to_response(
conv, participants, unread_count=unread, is_pinned_by_user=conv.id in pinned_ids
)
)
# Sort: pinned first, then by last_msg_at
conv_list.sort(key=lambda c: (not c["is_pinned"], c["last_msg_at"] or ""), reverse=False)
# Actually: pinned first (True > False in reverse), then newest first
conv_list.sort(key=lambda c: c["last_msg_at"] or "0000", reverse=True)
conv_list.sort(key=lambda c: c["is_pinned"], reverse=True)
return conv_list
async def get_conversation(
db: AsyncSession,
tenant_id: uuid.UUID,
conversation_id: uuid.UUID,
user_id: uuid.UUID,
) -> dict[str, Any] | None:
"""Get a single conversation with participants."""
result = await db.execute(
select(CommConversation).where(
CommConversation.id == conversation_id,
CommConversation.tenant_id == tenant_id,
CommConversation.deleted_at.is_(None),
)
)
conv = result.scalar_one_or_none()
if conv is None:
return None
# Check user is participant
if not await CommRBAC.is_participant(db, conversation_id, user_id):
return None
parts_result = await db.execute(
select(CommParticipant).where(
CommParticipant.conversation_id == conv.id,
CommParticipant.left_at.is_(None),
)
)
participants = list(parts_result.scalars().all())
# Check pinned
pin_result = await db.execute(
select(CommConversationPin).where(
CommConversationPin.conversation_id == conv.id,
CommConversationPin.user_id == user_id,
)
)
is_pinned = pin_result.scalar_one_or_none() is not None
unread = await _get_unread_count(db, tenant_id, conv.id, user_id)
return conversation_to_response(conv, participants, unread_count=unread, is_pinned_by_user=is_pinned)
async def create_conversation(
db: AsyncSession,
tenant_id: uuid.UUID,
user_id: uuid.UUID,
title: str | None = None,
participant_ids: list[str] | None = None,
is_direct: bool = False,
initial_message: str | None = None,
metadata: dict[str, Any] | None = None,
) -> dict[str, Any]:
"""Create a new conversation."""
conv = CommConversation(
tenant_id=tenant_id,
title=title,
owner_id=user_id,
is_direct=is_direct,
created_by=user_id,
created_by_type="user",
metadata_=metadata or {},
)
from app.core.hooks import do_action
await do_action("comm.conversation.before_create", tenant_id=tenant_id, user_id=user_id)
db.add(conv)
await db.flush()
await do_action("comm.conversation.after_create", conversation_id=conv.id, tenant_id=tenant_id, user_id=user_id)
# Add creator as admin
creator = CommParticipant(
tenant_id=tenant_id,
conversation_id=conv.id,
participant_id=user_id,
participant_type="user",
role="admin",
)
db.add(creator)
# Add other participants
for pid_str in (participant_ids or []):
try:
pid = uuid.UUID(pid_str)
if pid == user_id:
continue
p = CommParticipant(
tenant_id=tenant_id,
conversation_id=conv.id,
participant_id=pid,
participant_type="user",
role="member",
)
db.add(p)
except ValueError:
logger.warning(f"Invalid participant UUID: {pid_str}")
await db.flush()
# Send initial message if provided
if initial_message:
await send_message(
db, tenant_id, conv.id, user_id, "user",
content=initial_message, content_format="text",
)
# Publish event
event_bus = get_event_bus()
await event_bus.publish("conversation.created", {
"conversation_id": str(conv.id),
"tenant_id": str(tenant_id),
"created_by": str(user_id),
})
# Get all participants for response
parts_result = await db.execute(
select(CommParticipant).where(
CommParticipant.conversation_id == conv.id,
CommParticipant.left_at.is_(None),
)
)
participants = list(parts_result.scalars().all())
return conversation_to_response(conv, participants)
async def update_conversation(
db: AsyncSession,
tenant_id: uuid.UUID,
conversation_id: uuid.UUID,
user_id: uuid.UUID,
title: str | None = None,
is_archived: bool | None = None,
) -> dict[str, Any] | None:
"""Update a conversation."""
result = await db.execute(
select(CommConversation).where(
CommConversation.id == conversation_id,
CommConversation.tenant_id == tenant_id,
CommConversation.deleted_at.is_(None),
)
)
conv = result.scalar_one_or_none()
if conv is None:
return None
# Check locked
if conv.is_locked and title is not None:
# Only the locking plugin can change title on locked conversations
# Users cannot
pass
elif title is not None:
conv.title = title
conv.title_set_by = user_id
if is_archived is not None:
conv.is_archived = is_archived
await db.flush()
return await get_conversation(db, tenant_id, conversation_id, user_id)
async def pin_conversation(
db: AsyncSession,
tenant_id: uuid.UUID,
conversation_id: uuid.UUID,
user_id: uuid.UUID,
) -> bool:
"""Pin a conversation for a user."""
existing = await db.execute(
select(CommConversationPin).where(
CommConversationPin.conversation_id == conversation_id,
CommConversationPin.user_id == user_id,
)
)
if existing.scalar_one_or_none() is None:
pin = CommConversationPin(
tenant_id=tenant_id,
conversation_id=conversation_id,
user_id=user_id,
)
db.add(pin)
await db.flush()
return True
async def unpin_conversation(
db: AsyncSession,
conversation_id: uuid.UUID,
user_id: uuid.UUID,
) -> bool:
"""Unpin a conversation for a user."""
result = await db.execute(
select(CommConversationPin).where(
CommConversationPin.conversation_id == conversation_id,
CommConversationPin.user_id == user_id,
)
)
pin = result.scalar_one_or_none()
if pin:
await db.delete(pin)
await db.flush()
return True
async def mute_conversation(
db: AsyncSession,
tenant_id: uuid.UUID,
conversation_id: uuid.UUID,
user_id: uuid.UUID,
) -> bool:
"""Mute a conversation for a user."""
existing = await db.execute(
select(CommConversationMute).where(
CommConversationMute.conversation_id == conversation_id,
CommConversationMute.user_id == user_id,
)
)
if existing.scalar_one_or_none() is None:
mute = CommConversationMute(
tenant_id=tenant_id,
conversation_id=conversation_id,
user_id=user_id,
)
db.add(mute)
await db.flush()
return True
async def unmute_conversation(
db: AsyncSession,
conversation_id: uuid.UUID,
user_id: uuid.UUID,
) -> bool:
"""Unmute a conversation for a user."""
result = await db.execute(
select(CommConversationMute).where(
CommConversationMute.conversation_id == conversation_id,
CommConversationMute.user_id == user_id,
)
)
mute = result.scalar_one_or_none()
if mute:
await db.delete(mute)
await db.flush()
return True
# ─── Participant Management ───
@@ -0,0 +1,159 @@
"""Reactions and read-state handling for the kommunikation plugin."""
from __future__ import annotations
import logging
import uuid
from datetime import UTC, datetime
from typing import Any
from sqlalchemy import func, select
from sqlalchemy.ext.asyncio import AsyncSession
from app.core.event_bus import get_event_bus
from app.plugins.builtins.kommunikation.models import (
CommMessage,
CommMessageReaction,
CommMessageRead,
)
logger = logging.getLogger(__name__)
# ─── interactions ───
async def add_reaction(
db: AsyncSession,
tenant_id: uuid.UUID,
message_id: uuid.UUID,
user_id: uuid.UUID,
emoji: str,
) -> dict[str, Any] | None:
"""Add an emoji reaction to a message."""
existing = await db.execute(
select(CommMessageReaction).where(
CommMessageReaction.message_id == message_id,
CommMessageReaction.user_id == user_id,
CommMessageReaction.emoji == emoji,
)
)
if existing.scalar_one_or_none() is not None:
return None # Already reacted
r = CommMessageReaction(
tenant_id=tenant_id,
message_id=message_id,
user_id=user_id,
emoji=emoji,
)
db.add(r)
await db.flush()
event_bus = get_event_bus()
await event_bus.publish("reaction.added", {
"message_id": str(message_id),
"emoji": emoji,
"user_id": str(user_id),
})
return {
"id": str(r.id),
"message_id": str(r.message_id),
"user_id": str(r.user_id),
"emoji": r.emoji,
}
async def remove_reaction(
db: AsyncSession,
message_id: uuid.UUID,
user_id: uuid.UUID,
emoji: str,
) -> bool:
"""Remove an emoji reaction."""
result = await db.execute(
select(CommMessageReaction).where(
CommMessageReaction.message_id == message_id,
CommMessageReaction.user_id == user_id,
CommMessageReaction.emoji == emoji,
)
)
r = result.scalar_one_or_none()
if r is None:
return False
await db.delete(r)
await db.flush()
return True
# ─── Read State ───
async def mark_read(
db: AsyncSession,
tenant_id: uuid.UUID,
conversation_id: uuid.UUID,
user_id: uuid.UUID,
last_read_msg_id: str | None = None,
) -> bool:
"""Mark conversation as read up to a message."""
result = await db.execute(
select(CommMessageRead).where(
CommMessageRead.conversation_id == conversation_id,
CommMessageRead.user_id == user_id,
)
)
read = result.scalar_one_or_none()
msg_id = uuid.UUID(last_read_msg_id) if last_read_msg_id else None
if read is None:
read = CommMessageRead(
tenant_id=tenant_id,
conversation_id=conversation_id,
user_id=user_id,
last_read_msg_id=msg_id,
)
db.add(read)
else:
read.last_read_msg_id = msg_id
read.last_read_at = datetime.now(UTC)
await db.flush()
return True
async def _get_unread_count(
db: AsyncSession,
tenant_id: uuid.UUID,
conversation_id: uuid.UUID,
user_id: uuid.UUID,
) -> int:
"""Get unread message count for a user in a conversation."""
# Get last read message
read_result = await db.execute(
select(CommMessageRead).where(
CommMessageRead.conversation_id == conversation_id,
CommMessageRead.user_id == user_id,
)
)
read = read_result.scalar_one_or_none()
query = select(func.count()).select_from(CommMessage).where(
CommMessage.conversation_id == conversation_id,
CommMessage.tenant_id == tenant_id,
CommMessage.deleted_at.is_(None),
CommMessage.sender_type != "system", # Don't count system messages? Or count all?
)
if read and read.last_read_at:
query = query.where(CommMessage.created_at > read.last_read_at)
result = await db.execute(query)
return result.scalar() or 0
# ─── Plugin Room Creation ───
@@ -0,0 +1,394 @@
"""Message retrieval, sending and editing for the kommunikation plugin."""
from __future__ import annotations
import logging
import uuid
from datetime import UTC, datetime
from typing import Any
from sqlalchemy import func, select, update
from sqlalchemy.ext.asyncio import AsyncSession
from app.core.event_bus import get_event_bus
from app.plugins.builtins.kommunikation.models import (
CommConversation,
CommMessage,
CommMessageAttachment,
CommMessageBlock,
CommMessageEdit,
CommMessageReaction,
CommParticipant,
)
from app.plugins.builtins.kommunikation.participant_registry import get_participant_registry
from app.plugins.builtins.kommunikation.serializers import (
conversation_to_response,
message_to_response,
parse_mentions,
participant_to_response,
)
logger = logging.getLogger(__name__)
MAX_TRIGGER_DEPTH = 3
# ─── messages ───
async def get_messages(
db: AsyncSession,
tenant_id: uuid.UUID,
conversation_id: uuid.UUID,
page: int = 1,
page_size: int = 50,
before_id: uuid.UUID | None = None,
) -> dict[str, Any]:
"""Get paginated messages for a conversation."""
query = select(CommMessage).where(
CommMessage.conversation_id == conversation_id,
CommMessage.tenant_id == tenant_id,
CommMessage.deleted_at.is_(None),
).order_by(CommMessage.created_at.desc())
if before_id:
before_msg = await db.execute(
select(CommMessage).where(CommMessage.id == before_id)
)
before = before_msg.scalar_one_or_none()
if before:
query = query.where(CommMessage.created_at < before.created_at)
query = query.offset((page - 1) * page_size).limit(page_size)
result = await db.execute(query)
messages = list(result.scalars().all())
# Get blocks, attachments, reactions for each message
msg_ids = [m.id for m in messages]
blocks_map: dict[uuid.UUID, list] = {}
attachments_map: dict[uuid.UUID, list] = {}
reactions_map: dict[uuid.UUID, list] = {}
if msg_ids:
blocks_result = await db.execute(
select(CommMessageBlock).where(
CommMessageBlock.message_id.in_(msg_ids),
CommMessageBlock.deleted_at.is_(None),
).order_by(CommMessageBlock.sort_order)
)
for b in blocks_result.scalars().all():
blocks_map.setdefault(b.message_id, []).append(b)
atts_result = await db.execute(
select(CommMessageAttachment).where(
CommMessageAttachment.message_id.in_(msg_ids),
CommMessageAttachment.deleted_at.is_(None),
)
)
for a in atts_result.scalars().all():
attachments_map.setdefault(a.message_id, []).append(a)
reactions_result = await db.execute(
select(CommMessageReaction).where(
CommMessageReaction.message_id.in_(msg_ids),
)
)
for r in reactions_result.scalars().all():
reactions_map.setdefault(r.message_id, []).append(r)
items = []
for msg in reversed(messages): # chronological order
items.append(
message_to_response(
msg,
blocks=blocks_map.get(msg.id, []),
attachments=attachments_map.get(msg.id, []),
reactions=reactions_map.get(msg.id, []),
)
)
# Total count
count_result = await db.execute(
select(func.count()).select_from(CommMessage).where(
CommMessage.conversation_id == conversation_id,
CommMessage.tenant_id == tenant_id,
CommMessage.deleted_at.is_(None),
)
)
total = count_result.scalar() or 0
has_more = (page * page_size) < total
return {"items": items, "total": total, "page": page, "has_more": has_more}
async def send_message(
db: AsyncSession,
tenant_id: uuid.UUID,
conversation_id: uuid.UUID,
sender_id: uuid.UUID | None,
sender_type: str,
content: str = "",
content_format: str = "text",
blocks: list[dict[str, Any]] | None = None,
reply_to_id: str | None = None,
attachments: list[dict[str, Any]] | None = None,
metadata: dict[str, Any] | None = None,
trigger_depth: int = 0,
) -> dict[str, Any]:
"""Send a message to a conversation and trigger participant handlers."""
# Create message
msg = CommMessage(
tenant_id=tenant_id,
conversation_id=conversation_id,
sender_id=sender_id,
sender_type=sender_type,
content=content,
content_format=content_format,
metadata_=metadata or {},
)
if reply_to_id:
try:
msg.reply_to_id = uuid.UUID(reply_to_id)
except ValueError:
pass
from app.core.hooks import do_action
await do_action("comm.before_message", conversation_id=conversation_id, tenant_id=tenant_id, sender_id=sender_id)
db.add(msg)
await db.flush()
await do_action("comm.after_message", message_id=msg.id, conversation_id=conversation_id, tenant_id=tenant_id, sender_id=sender_id)
# Create blocks
if blocks:
for i, block in enumerate(blocks):
b = CommMessageBlock(
tenant_id=tenant_id,
message_id=msg.id,
block_type=block["block_type"],
block_data=block["block_data"],
sort_order=i,
)
db.add(b)
# Create attachments
if attachments:
for att in attachments:
a = CommMessageAttachment(
tenant_id=tenant_id,
message_id=msg.id,
file_id=uuid.UUID(att["file_id"]) if att.get("file_id") else None,
file_source=att.get("file_source", "comm"),
file_name=att.get("file_name", ""),
file_type=att.get("file_type", "application/octet-stream"),
file_size=att.get("file_size"),
)
db.add(a)
await db.flush()
# Update conversation last_msg
await db.execute(
update(CommConversation)
.where(CommConversation.id == conversation_id)
.values(
last_msg_at=datetime.now(UTC),
last_msg_preview=content[:200] if content else "",
last_msg_sender_type=sender_type,
)
)
# Publish event
event_bus = get_event_bus()
await event_bus.publish("message.received", {
"conversation_id": str(conversation_id),
"message_id": str(msg.id),
"sender_type": sender_type,
"tenant_id": str(tenant_id),
"content": content,
"trigger_depth": trigger_depth,
})
# Trigger participant handlers (if not at max depth)
if trigger_depth < MAX_TRIGGER_DEPTH:
await _trigger_participants(
db, tenant_id, conversation_id, msg, trigger_depth
)
# Load blocks/attachments/reactions for response
blocks_result = await db.execute(
select(CommMessageBlock).where(
CommMessageBlock.message_id == msg.id,
CommMessageBlock.deleted_at.is_(None),
).order_by(CommMessageBlock.sort_order)
)
msg_blocks = list(blocks_result.scalars().all())
atts_result = await db.execute(
select(CommMessageAttachment).where(
CommMessageAttachment.message_id == msg.id,
CommMessageAttachment.deleted_at.is_(None),
)
)
msg_atts = list(atts_result.scalars().all())
return message_to_response(msg, blocks=msg_blocks, attachments=msg_atts)
async def _trigger_participants(
db: AsyncSession,
tenant_id: uuid.UUID,
conversation_id: uuid.UUID,
message: CommMessage,
trigger_depth: int,
) -> None:
"""Trigger participant handlers for non-user participants."""
# Get conversation participants
result = await db.execute(
select(CommParticipant).where(
CommParticipant.conversation_id == conversation_id,
CommParticipant.left_at.is_(None),
CommParticipant.participant_type != "user",
)
)
non_user_participants = list(result.scalars().all())
if not non_user_participants:
return
# Get conversation info
conv_result = await db.execute(
select(CommConversation).where(CommConversation.id == conversation_id)
)
conv = conv_result.scalar_one_or_none()
if conv is None:
return
# Parse mentions
mentions = parse_mentions(message.content)
# Build conversation dict
all_parts_result = await db.execute(
select(CommParticipant).where(
CommParticipant.conversation_id == conversation_id,
CommParticipant.left_at.is_(None),
)
)
all_parts = [participant_to_response(p) for p in all_parts_result.scalars().all()]
conv_dict = conversation_to_response(conv, [])
conv_dict["participants"] = all_parts
msg_dict = message_to_response(message)
context = {"tenant_id": str(tenant_id), "trigger_depth": trigger_depth}
registry = get_participant_registry()
for p in non_user_participants:
handler = registry.get_handler(p.participant_type)
if handler is None:
continue
try:
responses = await handler.on_message_received(
conversation_id=conversation_id,
message=msg_dict,
conversation=conv_dict,
mentions=mentions,
context=context,
)
if responses:
for resp in responses:
await send_message(
db,
tenant_id,
conversation_id,
sender_id=None,
sender_type=p.participant_type,
content=resp.get("content", ""),
content_format=resp.get("content_format", "text"),
blocks=resp.get("blocks"),
metadata={
**(resp.get("metadata") or {}),
"triggered_by": str(message.id),
"trigger_depth": trigger_depth + 1,
},
trigger_depth=trigger_depth + 1,
)
except Exception:
logger.exception(
f"Participant handler error for type {p.participant_type}"
)
async def edit_message(
db: AsyncSession,
tenant_id: uuid.UUID,
message_id: uuid.UUID,
user_id: uuid.UUID,
new_content: str,
) -> dict[str, Any] | None:
"""Edit a message, storing the old version in history."""
result = await db.execute(
select(CommMessage).where(
CommMessage.id == message_id,
CommMessage.tenant_id == tenant_id,
CommMessage.deleted_at.is_(None),
)
)
msg = result.scalar_one_or_none()
if msg is None:
return None
# Get old blocks
blocks_result = await db.execute(
select(CommMessageBlock).where(
CommMessageBlock.message_id == message_id,
CommMessageBlock.deleted_at.is_(None),
)
)
old_blocks = [b.block_data for b in blocks_result.scalars().all()]
# Save edit history
edit = CommMessageEdit(
tenant_id=tenant_id,
message_id=message_id,
old_content=msg.content,
old_blocks=old_blocks,
edited_by=user_id,
)
db.add(edit)
from app.core.hooks import do_action
await do_action("comm.before_edit", message_id=message_id, tenant_id=tenant_id, user_id=user_id)
# Update message
msg.content = new_content
msg.edited_at = datetime.now(UTC)
await db.flush()
await do_action("comm.after_edit", message_id=message_id, tenant_id=tenant_id, user_id=user_id)
return message_to_response(msg)
async def delete_message(
db: AsyncSession,
message_id: uuid.UUID,
) -> bool:
"""Soft-delete a message."""
result = await db.execute(
select(CommMessage).where(CommMessage.id == message_id)
)
msg = result.scalar_one_or_none()
if msg is None:
return False
from app.core.hooks import do_action
await do_action("comm.before_delete", message_id=message_id)
msg.deleted_at = datetime.now(UTC)
await db.flush()
await do_action("comm.after_delete", message_id=message_id)
return True
# ─── Reactions ───
@@ -0,0 +1,7 @@
-- Dual-path convergence (Gate B): add columns that Alembic migrations
-- 0120 (is_system) and 0140 (folder_id) add on the core path. Idempotent
-- so both install paths converge to the identical schema.
ALTER TABLE comm_conversations ADD COLUMN IF NOT EXISTS is_system BOOLEAN NOT NULL DEFAULT FALSE;
CREATE INDEX IF NOT EXISTS ix_comm_conversations_tenant_system ON comm_conversations(tenant_id, is_system);
ALTER TABLE comm_conversations ADD COLUMN IF NOT EXISTS folder_id UUID;
CREATE INDEX IF NOT EXISTS ix_comm_conversations_folder ON comm_conversations(folder_id);
@@ -0,0 +1,127 @@
"""Participant management for the kommunikation plugin."""
from __future__ import annotations
import logging
import uuid
from datetime import UTC, datetime
from typing import Any
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from app.core.event_bus import get_event_bus
from app.plugins.builtins.kommunikation.models import (
CommParticipant,
)
from app.plugins.builtins.kommunikation.serializers import participant_to_response
logger = logging.getLogger(__name__)
# ─── participants ───
async def add_participant(
db: AsyncSession,
tenant_id: uuid.UUID,
conversation_id: uuid.UUID,
participant_id: str,
participant_type: str = "user",
role: str = "member",
display_name: str | None = None,
) -> dict[str, Any] | None:
"""Add a participant to a conversation."""
try:
pid = uuid.UUID(participant_id) if participant_type == "user" else None
except ValueError:
return None
existing = await db.execute(
select(CommParticipant).where(
CommParticipant.conversation_id == conversation_id,
CommParticipant.participant_id == pid if pid else CommParticipant.participant_type == participant_type,
CommParticipant.participant_type == participant_type,
CommParticipant.left_at.is_(None),
)
)
if existing.scalar_one_or_none() is not None:
return None # Already a participant
p = CommParticipant(
tenant_id=tenant_id,
conversation_id=conversation_id,
participant_id=pid,
participant_type=participant_type,
role=role,
display_name=display_name,
)
db.add(p)
await db.flush()
# Publish event
event_bus = get_event_bus()
await event_bus.publish("participant.joined", {
"conversation_id": str(conversation_id),
"participant_id": participant_id,
"participant_type": participant_type,
"tenant_id": str(tenant_id),
})
return participant_to_response(p)
async def remove_participant(
db: AsyncSession,
conversation_id: uuid.UUID,
participant_id: uuid.UUID,
) -> bool:
"""Remove a participant from a conversation (set left_at)."""
result = await db.execute(
select(CommParticipant).where(
CommParticipant.conversation_id == conversation_id,
CommParticipant.participant_id == participant_id,
CommParticipant.participant_type == "user",
CommParticipant.left_at.is_(None),
)
)
p = result.scalar_one_or_none()
if p is None:
return False
p.left_at = datetime.now(UTC)
await db.flush()
event_bus = get_event_bus()
await event_bus.publish("participant.left", {
"conversation_id": str(conversation_id),
"participant_id": str(participant_id),
})
return True
async def change_role(
db: AsyncSession,
conversation_id: uuid.UUID,
participant_id: uuid.UUID,
new_role: str,
) -> dict[str, Any] | None:
"""Change a participant's role."""
result = await db.execute(
select(CommParticipant).where(
CommParticipant.conversation_id == conversation_id,
CommParticipant.participant_id == participant_id,
CommParticipant.participant_type == "user",
CommParticipant.left_at.is_(None),
)
)
p = result.scalar_one_or_none()
if p is None:
return None
p.role = new_role
await db.flush()
return participant_to_response(p)
# ─── Messages ───
+10 -3
View File
@@ -39,7 +39,7 @@ class KommunikationPlugin(BasePlugin):
"participant.left",
"reaction.added",
],
migrations=["0001_initial.sql"],
migrations=["0001_initial.sql", "0002_system_channel_folders.sql"],
permissions=[
"comm:read",
"comm:write",
@@ -50,10 +50,10 @@ class KommunikationPlugin(BasePlugin):
],
is_core=True,
menu_items=[
FrontendMenuItem(label_key='nav.communication', label='Kommunikation', path='/communication', icon='MessageSquare', order=80),
FrontendMenuItem(label_key='nav.communication', label='Kommunikation', path='/communication', icon='MessageSquare', order=80, permission='comm:read'),
],
page_routes=[
FrontendPageRoute(path='/communication', component='@/pages/Communication', protected=True),
FrontendPageRoute(path='/communication', component='@/pages/Communication', protected=True, permission='comm:read'),
],
author="LeoCRM Team",
min_app_version="1.0.0",
@@ -136,6 +136,13 @@ class KommunikationPlugin(BasePlugin):
from app.plugins.builtins.contracts import get_contract_registry
get_contract_registry().unregister(self.manifest.name)
# Remove services registered in on_activate BEFORE super() so that
# event handlers being unsubscribed can no longer reach them (ARCH-033).
for service_name in ("comm_websocket", "comm_miniapps"):
if service_container.has(service_name):
service_container.remove(service_name)
logger.info("Removed '%s' from service container", service_name)
await super().on_deactivate(db, service_container, event_bus)
logger.info("Kommunikation plugin deactivated")
@@ -0,0 +1,345 @@
"""Plugin room creation and system channels for the kommunikation plugin."""
from __future__ import annotations
import logging
import uuid
from datetime import UTC, datetime
from typing import Any
from sqlalchemy import and_, select, update
from sqlalchemy.ext.asyncio import AsyncSession
from app.core.event_bus import get_event_bus
from app.plugins.builtins.kommunikation.conversations import get_conversation
from app.plugins.builtins.kommunikation.models import (
CommConversation,
CommConversationPin,
CommMessage,
CommMessageBlock,
CommParticipant,
)
from app.plugins.builtins.kommunikation.serializers import conversation_to_response
logger = logging.getLogger(__name__)
# ─── plugin_rooms ───
async def find_locked_room_id(
db: AsyncSession,
tenant_id: uuid.UUID,
plugin_name: str,
title: str,
) -> uuid.UUID | None:
"""Find the conversation ID of a locked plugin room by tenant and title.
Matches the same room semantics as ``create_plugin_room``: locked rooms
are owned by the plugin (``locked_by == plugin_name``) and soft-deleted
conversations are excluded. Returns ``None`` when no room exists.
"""
result = await db.execute(
select(CommConversation.id).where(
CommConversation.tenant_id == tenant_id,
CommConversation.title == title,
CommConversation.is_locked.is_(True),
CommConversation.locked_by == plugin_name,
CommConversation.deleted_at.is_(None),
)
)
return result.scalar_one_or_none()
async def create_plugin_room(
db: AsyncSession,
tenant_id: uuid.UUID,
user_id: uuid.UUID,
plugin_name: str,
title: str,
participant_type: str,
user_role: str = "member",
) -> dict[str, Any]:
"""Create a locked, pinned room for a plugin (System, Live KI, Assistent).
The room is locked (users can't change title/participants) and pinned for the user.
"""
# Check if room already exists for this user + plugin
result = await db.execute(
select(CommConversation).where(
CommConversation.tenant_id == tenant_id,
CommConversation.title == title,
CommConversation.is_locked.is_(True),
CommConversation.locked_by == plugin_name,
CommConversation.deleted_at.is_(None),
).join(CommParticipant, CommParticipant.conversation_id == CommConversation.id).where(
CommParticipant.participant_id == user_id,
CommParticipant.participant_type == "user",
CommParticipant.left_at.is_(None),
)
)
existing = result.scalar_one_or_none()
if existing:
# Already exists — return it
return await get_conversation(db, tenant_id, existing.id, user_id) or {}
# Create conversation
conv = CommConversation(
tenant_id=tenant_id,
title=title,
owner_id=user_id,
is_locked=True,
locked_by=plugin_name,
is_direct=False,
created_by=None,
created_by_type="plugin",
metadata_={"plugin": plugin_name},
)
db.add(conv)
await db.flush()
# Add plugin as participant
plugin_p = CommParticipant(
tenant_id=tenant_id,
conversation_id=conv.id,
participant_id=None,
participant_type=participant_type,
role="admin",
display_name=title,
)
db.add(plugin_p)
# Add user as participant
user_p = CommParticipant(
tenant_id=tenant_id,
conversation_id=conv.id,
participant_id=user_id,
participant_type="user",
role=user_role,
)
db.add(user_p)
# Pin for user
pin = CommConversationPin(
tenant_id=tenant_id,
conversation_id=conv.id,
user_id=user_id,
)
db.add(pin)
await db.flush()
# Publish event
event_bus = get_event_bus()
await event_bus.publish("conversation.created", {
"conversation_id": str(conv.id),
"tenant_id": str(tenant_id),
"created_by_type": "plugin",
"plugin_name": plugin_name,
})
parts = [plugin_p, user_p]
return conversation_to_response(conv, parts, is_pinned_by_user=True)
# ─── System Channel ───
async def get_or_create_system_channel(
db: AsyncSession,
tenant_id: uuid.UUID,
) -> CommConversation:
"""Get or create the tenant-wide system channel.
The system channel is a locked, is_system=True conversation that serves as
the central destination for system notifications, user alerts, and agent messages.
All users of the tenant are automatically added as participants.
"""
result = await db.execute(
select(CommConversation).where(
CommConversation.tenant_id == tenant_id,
CommConversation.is_system.is_(True),
CommConversation.deleted_at.is_(None),
)
)
conv = result.scalar_one_or_none()
if conv is not None:
return conv
# Create the system channel
conv = CommConversation(
tenant_id=tenant_id,
title="System Channel",
is_pinned=False,
is_locked=True,
is_direct=False,
is_archived=False,
is_system=True,
created_by=None,
created_by_type="system",
metadata_={},
)
db.add(conv)
await db.flush()
# Add all tenant users as participants
from app.models.user import User, UserTenant
users_result = await db.execute(
select(User.id)
.join(UserTenant, UserTenant.user_id == User.id)
.where(UserTenant.tenant_id == tenant_id)
)
user_ids = [row[0] for row in users_result.all()]
for uid in user_ids:
p = CommParticipant(
tenant_id=tenant_id,
conversation_id=conv.id,
participant_id=uid,
participant_type="user",
role="member",
)
db.add(p)
await db.flush()
return conv
async def post_system_message(
db: AsyncSession,
tenant_id: uuid.UUID,
user_id: uuid.UUID,
message_type: str,
title: str,
body: str | None = None,
entity_type: str | None = None,
entity_id: uuid.UUID | None = None,
severity: str = "info",
) -> CommMessage | None:
"""Post a typed system message to the tenant system channel.
Creates a CommMessage in the system channel with:
- A text block containing title + body
- An action_card block with deep-link if entity_type/entity_id is set
- Block/message metadata: notification_type, severity, entity_ref
Returns the created CommMessage, or None if the user has muted this type.
"""
# Check user preferences — reuse the notification preference system
from app.models.notification import NotificationPreference, NotificationType
pref = await db.execute(
select(NotificationPreference).where(
and_(
NotificationPreference.user_id == user_id,
NotificationPreference.type_key == message_type,
NotificationPreference.tenant_id == tenant_id,
)
)
)
pref_row = pref.scalar_one_or_none()
if pref_row and not pref_row.is_enabled:
return None
if not pref_row:
type_def = await db.execute(
select(NotificationType).where(NotificationType.type_key == message_type)
)
type_row = type_def.scalar_one_or_none()
if type_row and not type_row.is_enabled_by_default:
return None
# Get or create system channel
conv = await get_or_create_system_channel(db, tenant_id)
# Build message content
content = title
if body:
content = f"{title}\n{body}"
# Build metadata
msg_metadata: dict[str, Any] = {
"notification_type": message_type,
"severity": severity,
"target_user_id": str(user_id),
}
if entity_type and entity_id:
msg_metadata["entity_ref"] = {
"entity_type": entity_type,
"entity_id": str(entity_id),
}
# Build blocks
blocks: list[dict[str, Any]] = [
{
"block_type": "text",
"block_data": {"text": content, "title": title, "body": body or ""},
}
]
if entity_type and entity_id:
blocks.append(
{
"block_type": "action_card",
"block_data": {
"label": "Open",
"entity_type": entity_type,
"entity_id": str(entity_id),
},
}
)
# Create message directly (not via send_message to avoid trigger_depth issues)
msg = CommMessage(
tenant_id=tenant_id,
conversation_id=conv.id,
sender_id=None,
sender_type="system",
content=content,
content_format="text",
metadata_=msg_metadata,
)
db.add(msg)
await db.flush()
# Create blocks
for i, block in enumerate(blocks):
b = CommMessageBlock(
tenant_id=tenant_id,
message_id=msg.id,
block_type=block["block_type"],
block_data=block["block_data"],
sort_order=i,
)
db.add(b)
await db.flush()
# Update conversation last_msg
await db.execute(
update(CommConversation)
.where(CommConversation.id == conv.id)
.values(
last_msg_at=datetime.now(UTC),
last_msg_preview=content[:200],
last_msg_sender_type="system",
)
)
# Publish event
event_bus = get_event_bus()
await event_bus.publish("system.message.posted", {
"conversation_id": str(conv.id),
"message_id": str(msg.id),
"tenant_id": str(tenant_id),
"user_id": str(user_id),
"message_type": message_type,
"severity": severity,
})
return msg
@@ -0,0 +1,129 @@
"""Row/response serialization helpers for the kommunikation plugin."""
from __future__ import annotations
import logging
import re
from typing import Any
from app.plugins.builtins.kommunikation.models import (
CommConversation,
CommMessage,
CommMessageAttachment,
CommMessageBlock,
CommMessageReaction,
CommParticipant,
)
logger = logging.getLogger(__name__)
MAX_TRIGGER_DEPTH = 3
# ─── serializers ───
# ─── Mention Parsing ───
MENTION_PATTERN = re.compile(r"@(\w+)")
def parse_mentions(content: str) -> list[str]:
"""Parse @mentions from message content. Returns list of mentioned types/names."""
return MENTION_PATTERN.findall(content)
# ─── Conversation Helpers ───
def conversation_to_response(
conv: CommConversation,
participants: list[CommParticipant],
unread_count: int = 0,
is_pinned_by_user: bool = False,
) -> dict[str, Any]:
"""Convert a CommConversation to a response dict."""
return {
"id": str(conv.id),
"title": conv.title,
"is_locked": conv.is_locked,
"locked_by": conv.locked_by,
"is_direct": conv.is_direct,
"is_archived": conv.is_archived,
"is_pinned": is_pinned_by_user,
"created_by": str(conv.created_by) if conv.created_by else None,
"created_by_type": conv.created_by_type,
"last_msg_at": conv.last_msg_at.isoformat() if conv.last_msg_at else None,
"last_msg_preview": conv.last_msg_preview,
"last_msg_sender_type": conv.last_msg_sender_type,
"participants": [participant_to_response(p) for p in participants],
"unread_count": unread_count,
"metadata": conv.metadata_ or {},
}
def participant_to_response(p: CommParticipant) -> dict[str, Any]:
"""Convert a CommParticipant to a response dict."""
return {
"id": str(p.id),
"conversation_id": str(p.conversation_id),
"participant_id": str(p.participant_id) if p.participant_id else None,
"participant_type": p.participant_type,
"display_name": p.display_name,
"role": p.role,
"joined_at": p.joined_at.isoformat() if p.joined_at else None,
}
def message_to_response(
msg: CommMessage,
blocks: list[CommMessageBlock] | None = None,
attachments: list[CommMessageAttachment] | None = None,
reactions: list[CommMessageReaction] | None = None,
) -> dict[str, Any]:
"""Convert a CommMessage to a response dict."""
return {
"id": str(msg.id),
"conversation_id": str(msg.conversation_id),
"sender_id": str(msg.sender_id) if msg.sender_id else None,
"sender_type": msg.sender_type,
"content": msg.content,
"content_format": msg.content_format,
"metadata": msg.metadata_ or {},
"reply_to_id": str(msg.reply_to_id) if msg.reply_to_id else None,
"is_pinned": msg.is_pinned,
"created_at": msg.created_at.isoformat() if msg.created_at else None,
"edited_at": msg.edited_at.isoformat() if msg.edited_at else None,
"blocks": [
{
"id": str(b.id),
"block_type": b.block_type,
"block_data": b.block_data,
"sort_order": b.sort_order,
}
for b in (blocks or [])
],
"attachments": [
{
"id": str(a.id),
"file_id": str(a.file_id) if a.file_id else None,
"file_source": a.file_source,
"file_name": a.file_name,
"file_type": a.file_type,
"file_size": a.file_size,
"thumbnail_path": a.thumbnail_path,
}
for a in (attachments or [])
],
"reactions": [
{
"id": str(r.id),
"message_id": str(r.message_id),
"user_id": str(r.user_id),
"emoji": r.emoji,
}
for r in (reactions or [])
],
}
# ─── Conversation CRUD ───
File diff suppressed because it is too large Load Diff
+115
View File
@@ -0,0 +1,115 @@
"""Mail Account Service for the Mail plugin.
Extracted from services.py as part of the God-object split (BUG-018 pilot).
Re-exported by ``app.plugins.builtins.mail.services``.
"""
from __future__ import annotations
import logging
import uuid
from sqlalchemy.ext.asyncio import AsyncSession
from app.plugins.builtins.mail.crypto import encrypt_password, generate_salt
from app.plugins.builtins.mail.models import (
MailAccount,
MailFolder,
)
logger = logging.getLogger(__name__)
async def create_mail_account(
db: AsyncSession, *, tenant_id: uuid.UUID, user_id: uuid.UUID, data: dict
) -> MailAccount:
"""Create a new mail account with encrypted password."""
salt = generate_salt()
account = MailAccount(
tenant_id=tenant_id,
user_id=user_id,
owner_id=user_id,
email_address=data["email_address"],
display_name=data.get("display_name", ""),
imap_host=data["imap_host"],
imap_port=data.get("imap_port", 993),
imap_ssl=data.get("imap_ssl", True),
smtp_host=data["smtp_host"],
smtp_port=data.get("smtp_port", 587),
smtp_tls=data.get("smtp_tls", True),
username=data.get("username") or data["email_address"],
encrypted_password=encrypt_password(data["password"], salt),
password_salt=salt,
is_shared=data.get("is_shared", False),
is_active=True,
sent_folder_imap_name=data.get("sent_folder_imap_name"),
drafts_folder_imap_name=data.get("drafts_folder_imap_name"),
spam_folder_imap_name=data.get("spam_folder_imap_name"),
trash_folder_imap_name=data.get("trash_folder_imap_name"),
)
db.add(account)
await db.flush()
# Create INBOX first so subfolders can reference it as parent
inbox_folder = MailFolder(
tenant_id=tenant_id,
account_id=account.id,
name="Posteingang",
imap_name="INBOX",
is_standard=True,
)
db.add(inbox_folder)
await db.flush()
# Create standard subfolders under INBOX (IMAP server uses '.' delimiter)
for fname, imap_name in [
("Gesendet", "INBOX.Sent"),
("Entwürfe", "INBOX.Drafts"),
("Papierkorb", "INBOX.Trash"),
("Spam", "INBOX.spam"),
]:
folder = MailFolder(
tenant_id=tenant_id,
account_id=account.id,
name=fname,
imap_name=imap_name,
parent_id=inbox_folder.id,
is_standard=True,
)
db.add(folder)
await db.flush()
return account
async def update_mail_account(db: AsyncSession, account: MailAccount, data: dict) -> MailAccount:
"""Update a mail account, encrypting password if changed."""
field_map = {
"email": "email_address",
"email_address": "email_address",
"display_name": "display_name",
"imap_host": "imap_host",
"imap_port": "imap_port",
"imap_ssl": "imap_ssl",
"smtp_host": "smtp_host",
"smtp_port": "smtp_port",
"smtp_tls": "smtp_tls",
"username": "username",
"is_shared": "is_shared",
"is_active": "is_active",
"sent_folder_imap_name": "sent_folder_imap_name",
"drafts_folder_imap_name": "drafts_folder_imap_name",
"spam_folder_imap_name": "spam_folder_imap_name",
"trash_folder_imap_name": "trash_folder_imap_name",
}
for api_field, model_field in field_map.items():
if api_field in data and data[api_field] is not None:
setattr(account, model_field, data[api_field])
if "password" in data and data["password"] is not None:
new_salt = generate_salt()
account.password_salt = new_salt
account.encrypted_password = encrypt_password(data["password"], new_salt)
await db.flush()
await db.refresh(account)
return account
+117
View File
@@ -0,0 +1,117 @@
"""Attachment storage helpers for the Mail plugin.
Extracted from services.py as part of the God-object split (BUG-018 pilot).
Re-exported by ``app.plugins.builtins.mail.services``.
"""
from __future__ import annotations
import os
import re
import uuid
import aiofiles
from app.config import settings
from app.plugins.builtins.mail.models import MailAttachment
# ─── Attachment Storage Helpers ───
MAX_ATTACHMENT_SIZE = 25 * 1024 * 1024 # 25 MB
def _decode_mime_filename(filename: str) -> str:
"""Decode MIME-encoded filename, handling =?charset?Q?...?= and =?charset?B?...?= patterns."""
if not filename:
return "attachment"
# If no MIME encoding pattern, return as-is
if "=?" not in filename:
return filename
try:
from email.header import decode_header, make_header
return str(make_header(decode_header(filename)))
except Exception:
# Fallback: manually decode Q-encoding if decode_header fails
# This handles cases where the email parser partially processed the filename
try:
import base64
def decode_q(match):
charset, encoding, encoded = (
match.group(1),
match.group(2).upper(),
match.group(3),
)
if encoding == "B":
decoded = base64.b64decode(encoded).decode(
charset or "utf-8", errors="replace"
)
else: # Q encoding
decoded = encoded.replace("_", " ")
decoded = re.sub(
r"=([0-9A-Fa-f]{2})",
lambda m: chr(int(m.group(1), 16)),
decoded,
)
decoded = decoded.encode("latin-1").decode(
charset or "utf-8", errors="replace"
)
return decoded
return re.sub(r"=\?([^?]+)\?([BbQq])\?([^?]*)\?=", decode_q, filename)
except Exception:
return filename
def _sanitize_filename(filename: str) -> str:
"""Sanitize a filename to prevent path traversal attacks."""
# Remove any path components — keep only the basename
filename = os.path.basename(filename or "attachment")
# Replace potentially dangerous characters
filename = re.sub(r"[^a-zA-Z0-9._-]", "_", filename)
# Ensure non-empty
if not filename:
filename = "attachment"
# Limit length
if len(filename) > 200:
name, ext = os.path.splitext(filename)
filename = name[:200 - len(ext)] + ext
return filename
def _attachment_storage_path(mail_id: uuid.UUID, filename: str) -> str:
"""Build the on-disk storage path for a mail attachment."""
safe_name = _sanitize_filename(filename)
return os.path.join(
settings.storage_path,
"mail_attachments",
str(mail_id),
safe_name,
)
async def _save_attachment_to_storage(
mail_id: uuid.UUID, filename: str, content: bytes
) -> str:
"""Save attachment content to disk and return the storage path."""
storage_path = _attachment_storage_path(mail_id, filename)
os.makedirs(os.path.dirname(storage_path), exist_ok=True)
async with aiofiles.open(storage_path, "wb") as f:
await f.write(content)
return storage_path
def attachment_to_response(att: MailAttachment) -> dict:
"""Convert a MailAttachment ORM object to a response dict."""
return {
"id": str(att.id),
"mail_id": str(att.mail_id),
"filename": att.filename,
"mime_type": att.mime_type,
"size_bytes": att.size_bytes,
"size": att.size_bytes, # alias for frontend compatibility
"content_id": att.content_id,
"is_inline": bool(att.content_id),
"dms_file_id": str(att.dms_file_id) if att.dms_file_id else None,
}
+74
View File
@@ -0,0 +1,74 @@
"""AES-256 password encryption for Mail accounts (Fernet-based).
Extracted from services.py as part of the God-object split (BUG-018 pilot).
Re-exported by ``app.plugins.builtins.mail.services`` for backwards
compatibility.
The legacy salt and MAIL_ENCRYPTION_KEY env guard are load-bearing:
changing them makes existing encrypted passwords unreadable.
"""
from __future__ import annotations
import base64
import os
from cryptography.fernet import Fernet
from cryptography.hazmat.primitives import hashes
from cryptography.hazmat.primitives.kdf.pbkdf2 import PBKDF2HMAC
MAIL_ENCRYPTION_KEY = os.environ.get("MAIL_ENCRYPTION_KEY")
if not MAIL_ENCRYPTION_KEY:
raise RuntimeError(
"MAIL_ENCRYPTION_KEY environment variable is required. "
"Set it to a strong random value."
)
# Legacy salt for backward compatibility with existing encrypted passwords
_LEGACY_SALT = b"leocrm-mail-salt"
def _derive_key(password: str, salt: bytes) -> bytes:
"""Derive a 32-byte Fernet key from a password using PBKDF2 with the given salt."""
kdf = PBKDF2HMAC(
algorithm=hashes.SHA256(),
length=32,
salt=salt,
iterations=480000,
)
return base64.urlsafe_b64encode(kdf.derive(password.encode()))
def generate_salt() -> str:
"""Generate a random 32-byte salt and return as base64 string."""
salt = os.urandom(32)
return base64.urlsafe_b64encode(salt).decode()
def _get_fernet(salt_b64: str | None = None) -> Fernet:
"""Get a Fernet instance. If salt_b64 is provided, use it; otherwise use legacy salt."""
if salt_b64:
salt = base64.urlsafe_b64decode(salt_b64.encode())
else:
salt = _LEGACY_SALT
return Fernet(_derive_key(MAIL_ENCRYPTION_KEY, salt))
def encrypt_password(plaintext: str, salt_b64: str | None = None) -> str:
"""Encrypt a password using AES-256 (Fernet). Returns base64 ciphertext.
If salt_b64 is provided, uses that salt for key derivation.
If not, uses the legacy hardcoded salt (for backward compatibility).
"""
fernet = _get_fernet(salt_b64)
return fernet.encrypt(plaintext.encode()).decode()
def decrypt_password(ciphertext: str, salt_b64: str | None = None) -> str:
"""Decrypt a password encrypted with encrypt_password.
If salt_b64 is provided, uses that salt for key derivation.
If not, uses the legacy hardcoded salt (for backward compatibility).
"""
fernet = _get_fernet(salt_b64)
return fernet.decrypt(ciphertext.encode()).decode()
+546
View File
@@ -0,0 +1,546 @@
"""Draft save/update and auto-sync for the Mail plugin.
Extracted from services.py as part of the God-object split (BUG-018 pilot).
Re-exported by ``app.plugins.builtins.mail.services``.
"""
from __future__ import annotations
import logging
import uuid
from datetime import UTC, datetime
from email.message import EmailMessage
from email.utils import formataddr, formatdate, make_msgid
import aioimaplib
from sqlalchemy import and_, select
from sqlalchemy.ext.asyncio import AsyncSession
from app.core.notifications import create_notification
from app.plugins.builtins.mail.imap_ops import imap_delete_mail, imap_move_mail
from app.plugins.builtins.mail.imap_sync import (
get_account_password,
imap_sync_account,
)
from app.plugins.builtins.mail.models import (
Mail,
MailAccount,
MailFolder,
)
logger = logging.getLogger(__name__)
async def save_draft(
db: AsyncSession,
account_id: uuid.UUID,
tenant_id: uuid.UUID,
user_id: uuid.UUID,
data: dict,
) -> Mail:
"""Save a new draft mail to DB and IMAP Drafts folder."""
import logging
logger = logging.getLogger(__name__)
# 1. Find the Drafts folder (imap_name contains 'Drafts')
account = (
await db.execute(
select(MailAccount).where(
and_(MailAccount.id == account_id, MailAccount.tenant_id == tenant_id)
)
)
).scalar_one_or_none()
if not account:
raise ValueError("Account not found")
folders = (
await db.execute(
select(MailFolder).where(
and_(MailFolder.account_id == account_id, MailFolder.tenant_id == tenant_id)
)
)
).scalars().all()
drafts_folder = None
for f in folders:
if 'draft' in f.imap_name.lower():
drafts_folder = f
break
if not drafts_folder:
raise ValueError("No Drafts folder found for this account")
# 2. Create Mail record
to_str = ', '.join(data.get('to', []))
cc_str = ', '.join(data.get('cc', []))
bcc_str = ', '.join(data.get('bcc', []))
subject = data.get('subject', '')
body_text = data.get('body_text', '')
body_html = data.get('body_html', '')
msg_id = make_msgid()
now = datetime.now(UTC)
mail = Mail(
tenant_id=tenant_id,
account_id=account_id,
folder_id=drafts_folder.id,
message_id=msg_id,
thread_id=msg_id,
subject=subject,
from_address=account.email_address,
to_addresses=to_str,
cc_addresses=cc_str,
bcc_addresses=bcc_str,
body_text=body_text,
body_html=body_html,
body_html_sanitized=body_html,
is_seen=True,
is_flagged=False,
is_draft=True,
is_answered=False,
is_forwarded=False,
has_attachments=False,
size_bytes=len(body_text.encode('utf-8')),
received_at=now,
sent_at=None,
)
db.add(mail)
await db.flush()
# ── Notification: draft saved ──
try:
await create_notification(
db, tenant_id, user_id,
"mail_draft",
"Entwurf gespeichert",
subject or "Ohne Betreff",
)
await db.flush()
except Exception:
logger.debug("Ignored exception in mail service", exc_info=True)
# 3. Build RFC822 message and APPEND to IMAP Drafts folder
password = await get_account_password(account)
client = None
try:
client = aioimaplib.IMAP4_SSL(host=account.imap_host, port=account.imap_port)
await client.wait_hello_from_server()
await client.login(account.username, password)
# Build RFC822 message
email_msg = EmailMessage()
email_msg['From'] = formataddr((account.display_name, account.email_address))
if to_str:
email_msg['To'] = to_str
if cc_str:
email_msg['Cc'] = cc_str
email_msg['Subject'] = subject
email_msg['Date'] = formatdate(localtime=True)
email_msg['Message-ID'] = msg_id
email_msg.set_content(body_text if body_text else '')
if body_html:
email_msg.add_alternative(body_html, subtype='html')
rfc822_bytes = email_msg.as_bytes()
# APPEND to Drafts folder
append_resp = await client.append(
drafts_folder.imap_name,
r'(\\Draft)',
str(int(now.timestamp())),
rfc822_bytes,
)
if append_resp.result != 'OK':
logger.warning("save_draft: IMAP APPEND failed for drafts folder %s", drafts_folder.imap_name)
else:
logger.info("save_draft: appended draft %s to IMAP Drafts", mail.id)
except Exception as exc:
logger.warning("save_draft: IMAP append failed (non-critical): %s", exc)
finally:
if client is not None:
try:
await client.logout()
except Exception:
logger.debug("Ignored exception in mail service", exc_info=True)
return mail
async def update_draft(
db: AsyncSession,
mail_id: uuid.UUID,
tenant_id: uuid.UUID,
data: dict,
) -> Mail:
"""Update an existing draft."""
import logging
logger = logging.getLogger(__name__)
mail = (
await db.execute(
select(Mail).where(
and_(Mail.id == mail_id, Mail.tenant_id == tenant_id)
)
)
).scalar_one_or_none()
if not mail:
raise ValueError("Mail not found")
if not mail.is_draft:
raise ValueError("Mail is not a draft")
# 2. Update fields
to_str = ', '.join(data.get('to', []))
cc_str = ', '.join(data.get('cc', []))
bcc_str = ', '.join(data.get('bcc', []))
mail.to_addresses = to_str
mail.cc_addresses = cc_str
mail.bcc_addresses = bcc_str
mail.subject = data.get('subject', '')
mail.body_text = data.get('body_text', '')
mail.body_html = data.get('body_html', '')
mail.body_html_sanitized = data.get('body_html', '')
mail.size_bytes = len(mail.body_text.encode('utf-8'))
await db.flush()
# 3. Delete old IMAP copy and APPEND new one
folder = (
await db.execute(
select(MailFolder).where(
and_(MailFolder.id == mail.folder_id, MailFolder.tenant_id == tenant_id)
)
)
).scalar_one_or_none()
if not folder:
return mail
account = (
await db.execute(
select(MailAccount).where(
and_(MailAccount.id == folder.account_id, MailAccount.tenant_id == tenant_id)
)
)
).scalar_one_or_none()
if not account:
return mail
password = await get_account_password(account)
client = None
try:
client = aioimaplib.IMAP4_SSL(host=account.imap_host, port=account.imap_port)
await client.wait_hello_from_server()
await client.login(account.username, password)
# Delete old copy from IMAP
select_resp = await client.select(folder.imap_name)
if select_resp.result == 'OK' and mail.message_id:
search_resp = await client.uid_search(f'HEADER Message-ID "{mail.message_id}"')
uids_raw = search_resp[1][0] if search_resp[1] and search_resp[1][0] else b''
if isinstance(uids_raw, (bytes, bytearray)):
uids = uids_raw.split()
else:
uids = []
if uids:
uid_str = uids[0].decode() if isinstance(uids[0], bytes) else str(uids[0])
await client.uid('store', uid_str, r'+FLAGS (\\Deleted)')
await client.expunge()
# APPEND new copy
email_msg = EmailMessage()
email_msg['From'] = formataddr((account.display_name, account.email_address))
if to_str:
email_msg['To'] = to_str
if cc_str:
email_msg['Cc'] = cc_str
email_msg['Subject'] = mail.subject
email_msg['Date'] = formatdate(localtime=True)
email_msg['Message-ID'] = mail.message_id
email_msg.set_content(mail.body_text if mail.body_text else '')
if mail.body_html:
email_msg.add_alternative(mail.body_html, subtype='html')
rfc822_bytes = email_msg.as_bytes()
now = datetime.now(UTC)
append_resp = await client.append(
folder.imap_name,
r'(\\Draft)',
str(int(now.timestamp())),
rfc822_bytes,
)
if append_resp.result != 'OK':
logger.warning("update_draft: IMAP APPEND failed for drafts folder %s", folder.imap_name)
else:
logger.info("update_draft: appended updated draft %s to IMAP Drafts", mail.id)
except Exception as exc:
logger.warning("update_draft: IMAP sync failed (non-critical): %s", exc)
finally:
if client is not None:
try:
await client.logout()
except Exception:
logger.debug("Ignored exception in mail service", exc_info=True)
return mail
# ─── IMAP Folder Create / Delete ───
async def imap_create_folder(
db: AsyncSession, account_id: uuid.UUID, folder_name: str, tenant_id: uuid.UUID
) -> None:
"""Create folder on IMAP server.
Connects to IMAP, creates folder with CREATE command.
Non-critical: errors are logged, DB operation still succeeds.
"""
import logging
logger = logging.getLogger(__name__)
account = (
await db.execute(
select(MailAccount).where(
and_(MailAccount.id == account_id, MailAccount.tenant_id == tenant_id)
)
)
).scalar_one_or_none()
if not account:
logger.warning("imap_create_folder: account %s not found", account_id)
return
password = await get_account_password(account)
client = None
try:
client = aioimaplib.IMAP4_SSL(host=account.imap_host, port=account.imap_port)
await client.wait_hello_from_server()
await client.login(account.username, password)
resp = await client.create(folder_name)
if resp.result != "OK":
logger.warning(
"imap_create_folder: CREATE failed for %s: %s", folder_name, resp
)
else:
logger.info("imap_create_folder: created folder %s on IMAP", folder_name)
try:
await create_notification(
db, account.tenant_id, account.user_id,
"mail_folder",
"Ordner erstellt",
folder_name,
)
await db.flush()
except Exception:
logger.debug("Ignored exception in mail service", exc_info=True)
except Exception as exc:
logger.warning("imap_create_folder: failed (non-critical): %s", exc)
finally:
if client is not None:
try:
await client.logout()
except Exception:
logger.debug("Ignored exception in mail service", exc_info=True)
async def imap_delete_folder(
db: AsyncSession, folder_id: uuid.UUID, tenant_id: uuid.UUID
) -> None:
"""Delete folder from IMAP server.
Connects to IMAP, deletes folder with DELETE command.
Non-critical: errors are logged, DB operation still succeeds.
"""
import logging
logger = logging.getLogger(__name__)
folder = (
await db.execute(
select(MailFolder).where(
and_(MailFolder.id == folder_id, MailFolder.tenant_id == tenant_id)
)
)
).scalar_one_or_none()
if not folder:
logger.warning("imap_delete_folder: folder %s not found", folder_id)
return
account = (
await db.execute(
select(MailAccount).where(
and_(
MailAccount.id == folder.account_id,
MailAccount.tenant_id == tenant_id,
)
)
)
).scalar_one_or_none()
if not account:
logger.warning("imap_delete_folder: account not found for folder %s", folder_id)
return
password = await get_account_password(account)
client = None
try:
client = aioimaplib.IMAP4_SSL(host=account.imap_host, port=account.imap_port)
await client.wait_hello_from_server()
await client.login(account.username, password)
resp = await client.delete(folder.imap_name)
if resp.result != "OK":
logger.warning(
"imap_delete_folder: DELETE failed for %s: %s", folder.imap_name, resp
)
else:
logger.info("imap_delete_folder: deleted folder %s on IMAP", folder.imap_name)
try:
await create_notification(
db, account.tenant_id, account.user_id,
"mail_folder",
"Ordner gelöscht",
folder.imap_name,
)
await db.flush()
except Exception:
logger.debug("Ignored exception in mail service", exc_info=True)
except Exception as exc:
logger.warning("imap_delete_folder: failed (non-critical): %s", exc)
finally:
if client is not None:
try:
await client.logout()
except Exception:
logger.debug("Ignored exception in mail service", exc_info=True)
# ─── Auto-Sync ───
async def auto_sync_all_accounts() -> None:
"""Auto-sync all active mail accounts.
Called periodically by the background scheduler.
Iterates all active mail accounts and syncs each one.
"""
import logging
from app.core.db import get_session_factory
logger = logging.getLogger(__name__)
factory = get_session_factory()
async with factory() as db:
accounts = (
await db.execute(
select(MailAccount).where(MailAccount.is_active.is_(True))
)
).scalars().all()
if not accounts:
return
logger.info("auto_sync_all_accounts: syncing %d active account(s)", len(accounts))
for account in accounts:
try:
result = await imap_sync_account(db, account.id, account.tenant_id)
logger.info(
"auto_sync_all_accounts: synced account %s (%s): %s",
account.id,
account.username,
result,
)
except Exception as exc:
logger.warning(
"auto_sync_all_accounts: failed for account %s: %s",
account.id,
exc,
)
try:
await create_notification(
db, account.tenant_id, account.user_id,
"mail_sync_error",
"Synchronisierung fehlgeschlagen",
f"Account {account.email_address}: {exc}",
)
except Exception:
logger.debug("Ignored exception in mail service", exc_info=True)
# commit per-account so partial progress is saved
try:
await db.commit()
except Exception:
await db.rollback()
async def process_sync_queue(db: AsyncSession) -> None:
"""Process pending IMAP operations from the sync queue.
Called at the start of each auto-sync loop iteration.
Retries failed delete/move operations.
"""
import logging
logger = logging.getLogger(__name__)
from app.plugins.builtins.mail.models import MailSyncQueue
pending = (
await db.execute(
select(MailSyncQueue).where(
and_(
MailSyncQueue.status == "pending",
MailSyncQueue.attempts < MailSyncQueue.max_attempts,
)
)
)
).scalars().all()
if not pending:
return
logger.info("process_sync_queue: processing %d pending operation(s)", len(pending))
for entry in pending:
try:
if entry.operation == "delete":
is_permanent = entry.payload.get("permanent", False) if entry.payload else False
await imap_delete_mail(db, entry.mail_id, entry.tenant_id, permanent=is_permanent)
elif entry.operation == "move":
target_folder_id = uuid.UUID(entry.payload.get("target_folder_id", ""))
await imap_move_mail(db, entry.mail_id, target_folder_id, entry.tenant_id)
else:
logger.warning("process_sync_queue: unknown operation %s", entry.operation)
entry.status = "failed"
entry.last_error = f"Unknown operation: {entry.operation}"
continue
entry.status = "completed"
entry.updated_at = datetime.now(UTC)
logger.info("process_sync_queue: completed %s for mail %s", entry.operation, entry.mail_id)
except Exception as exc:
entry.attempts += 1
entry.last_error = str(exc)
entry.updated_at = datetime.now(UTC)
if entry.attempts >= entry.max_attempts:
entry.status = "failed"
logger.warning(
"process_sync_queue: giving up on %s for mail %s after %d attempts: %s",
entry.operation, entry.mail_id, entry.attempts, exc,
)
else:
logger.info(
"process_sync_queue: retry %d/%d for %s mail %s: %s",
entry.attempts, entry.max_attempts, entry.operation, entry.mail_id, exc,
)
await db.flush()
+452
View File
@@ -0,0 +1,452 @@
"""IMAP operations (flags, delete, move, folders) for the Mail plugin.
Extracted from services.py as part of the God-object split (BUG-018 pilot).
Re-exported by ``app.plugins.builtins.mail.services``.
"""
from __future__ import annotations
import logging
import uuid
import aioimaplib
from sqlalchemy import and_, select
from sqlalchemy.ext.asyncio import AsyncSession
from app.plugins.builtins.mail.imap_sync import (
_parse_imap_list_response,
get_account_password,
)
from app.plugins.builtins.mail.models import (
Mail,
MailAccount,
MailFolder,
)
logger = logging.getLogger(__name__)
async def imap_sync_mail_flags(
db: AsyncSession,
mail_id: uuid.UUID,
tenant_id: uuid.UUID,
) -> None:
"""Sync is_seen/is_flagged flags from DB to IMAP server.
Connects to the IMAP server, selects the mail's folder,
and uses UID STORE to set/remove \\Seen and \\Flagged flags.
Non-critical: logs warnings on failure but does not raise.
"""
import logging
logger = logging.getLogger(__name__)
# Load the mail with its folder and account
mail = (
await db.execute(
select(Mail).where(
and_(Mail.id == mail_id, Mail.tenant_id == tenant_id)
)
)
).scalar_one_or_none()
if not mail:
logger.warning("imap_sync_mail_flags: mail %s not found", mail_id)
return
folder = (
await db.execute(
select(MailFolder).where(
and_(MailFolder.id == mail.folder_id, MailFolder.tenant_id == tenant_id)
)
)
).scalar_one_or_none()
if not folder:
logger.warning("imap_sync_mail_flags: folder %s not found for mail %s", mail.folder_id, mail_id)
return
account = (
await db.execute(
select(MailAccount).where(
and_(MailAccount.id == folder.account_id, MailAccount.tenant_id == tenant_id)
)
)
).scalar_one_or_none()
if not account:
logger.warning("imap_sync_mail_flags: account not found for mail %s", mail_id)
return
if not mail.message_id:
logger.warning("imap_sync_mail_flags: mail %s has no message_id, cannot sync", mail_id)
return
password = await get_account_password(account)
client = None
try:
client = aioimaplib.IMAP4_SSL(host=account.imap_host, port=account.imap_port)
await client.wait_hello_from_server()
await client.login(account.username, password)
# Select the folder
select_resp = await client.select(folder.imap_name)
if select_resp.result != 'OK':
logger.warning("imap_sync_mail_flags: cannot select folder %s", folder.imap_name)
return
# Find the UID by searching for the Message-ID header
search_resp = await client.uid_search(f'HEADER Message-ID "{mail.message_id}"')
uids_raw = search_resp[1][0] if search_resp[1] and search_resp[1][0] else b''
if isinstance(uids_raw, (bytes, bytearray)):
uids = uids_raw.split()
else:
uids = []
if not uids:
logger.warning("imap_sync_mail_flags: no UID found for Message-ID %s", mail.message_id)
return
uid_str = uids[0].decode() if isinstance(uids[0], bytes) else str(uids[0])
# Sync \Seen flag
if mail.is_seen:
await client.uid('store', uid_str, '+FLAGS (\\Seen)')
else:
await client.uid('store', uid_str, '-FLAGS (\\Seen)')
# Sync \Flagged flag
if mail.is_flagged:
await client.uid('store', uid_str, '+FLAGS (\\Flagged)')
else:
await client.uid('store', uid_str, '-FLAGS (\\Flagged)')
logger.info("imap_sync_mail_flags: synced flags for mail %s (UID %s)", mail_id, uid_str)
except Exception as exc:
logger.warning("imap_sync_mail_flags: failed for mail %s: %s", mail_id, exc)
finally:
if client is not None:
try:
await client.logout()
except Exception:
logger.debug("Ignored exception in mail service", exc_info=True)
# ─── IMAP Delete ───
async def _find_trash_folder_name(
db: AsyncSession, account: MailAccount, tenant_id: uuid.UUID
) -> str | None:
"""Find the IMAP Trash folder name for an account.
Checks account.trash_folder_imap_name, then DB folders with 'trash' in imap_name,
then queries the IMAP server LIST for common Trash folder names.
"""
# 1) Explicit mapping on account
if account.trash_folder_imap_name:
return account.trash_folder_imap_name
# 2) DB folder with 'trash' in imap_name
db_folders = (
await db.execute(
select(MailFolder).where(
and_(MailFolder.account_id == account.id, MailFolder.tenant_id == tenant_id)
)
)
).scalars().all()
for f in db_folders:
if 'trash' in f.imap_name.lower():
return f.imap_name
# 3) Query IMAP server for common Trash folder names
password = await get_account_password(account)
client = None
try:
client = aioimaplib.IMAP4_SSL(host=account.imap_host, port=account.imap_port)
await client.wait_hello_from_server()
await client.login(account.username, password)
list_resp = await client.list('""', '"*"')
imap_folders, _ = _parse_imap_list_response(list_resp)
trash_candidates = ['Trash', 'INBOX.Trash', 'INBOX.Trash', 'Deleted', 'Deleted Items', 'Papierkorb']
for _, name in imap_folders:
if name in trash_candidates or 'trash' in name.lower():
return name
except Exception:
logger.debug("Ignored exception in mail service", exc_info=True)
finally:
if client:
try:
await client.logout()
except Exception:
logger.debug("Ignored exception in mail service", exc_info=True)
return None
async def imap_delete_mail(
db: AsyncSession, mail_id: uuid.UUID, tenant_id: uuid.UUID, *, permanent: bool = False
) -> None:
"""Delete mail from IMAP server.
If permanent=False (default): MOVE mail to IMAP Trash folder (like real mail clients).
If permanent=True: STORE \\Deleted + EXPUNGE (permanent delete from server).
Uses UID MOVE if supported, otherwise COPY + STORE \\Deleted + EXPUNGE.
Raises exceptions on failure so caller can queue for retry.
"""
import logging
logger = logging.getLogger(__name__)
mail = (
await db.execute(
select(Mail).where(
and_(Mail.id == mail_id, Mail.tenant_id == tenant_id)
)
)
).scalar_one_or_none()
if not mail:
logger.warning("imap_delete_mail: mail %s not found", mail_id)
return
folder = (
await db.execute(
select(MailFolder).where(
and_(MailFolder.id == mail.folder_id, MailFolder.tenant_id == tenant_id)
)
)
).scalar_one_or_none()
if not folder:
logger.warning("imap_delete_mail: folder %s not found for mail %s", mail.folder_id, mail_id)
return
account = (
await db.execute(
select(MailAccount).where(
and_(MailAccount.id == folder.account_id, MailAccount.tenant_id == tenant_id)
)
)
).scalar_one_or_none()
if not account:
logger.warning("imap_delete_mail: account not found for mail %s", mail_id)
return
if not mail.imap_uid and not mail.message_id:
logger.warning("imap_delete_mail: mail %s has no imap_uid or message_id, cannot sync", mail_id)
return
# Lifecycle hook: mail.before_delete
from app.core.hooks import do_action
await do_action("mail.before_delete", mail_id=str(mail.id), tenant_id=str(tenant_id), permanent=permanent, db=db)
# Find Trash folder on IMAP server (only needed for non-permanent delete)
trash_folder_name = None
if not permanent:
trash_folder_name = await _find_trash_folder_name(db, account, tenant_id)
if not trash_folder_name:
logger.warning("imap_delete_mail: no Trash folder found for account %s, will permanent delete", account.id)
password = await get_account_password(account)
client = None
try:
client = aioimaplib.IMAP4_SSL(host=account.imap_host, port=account.imap_port)
await client.wait_hello_from_server()
await client.login(account.username, password)
select_resp = await client.select(folder.imap_name)
if select_resp.result != 'OK':
raise RuntimeError(f"cannot select folder {folder.imap_name}")
# Use stored imap_uid directly; fall back to Message-ID search
if mail.imap_uid:
uid_str = mail.imap_uid
else:
search_resp = await client.uid_search(f'HEADER Message-ID "{mail.message_id}"')
uids_raw = search_resp[1][0] if search_resp[1] and search_resp[1][0] else b''
if isinstance(uids_raw, (bytes, bytearray)):
uids = uids_raw.split()
else:
uids = []
if not uids:
raise RuntimeError(f"no UID found for Message-ID {mail.message_id}")
uid_str = uids[0].decode() if isinstance(uids[0], bytes) else str(uids[0])
if permanent or not trash_folder_name:
# Permanent delete: STORE \Deleted + EXPUNGE
await client.uid('store', uid_str, r'+FLAGS (\\Deleted)')
await client.expunge()
logger.info("imap_delete_mail: permanent deleted mail %s (UID %s)", mail_id, uid_str)
return
# Non-permanent: MOVE to Trash folder (like Thunderbird, Outlook, etc.)
# Don't move if already in Trash
if folder.imap_name.lower() == trash_folder_name.lower():
# Already in Trash — permanent delete
await client.uid('store', uid_str, r'+FLAGS (\\Deleted)')
await client.expunge()
logger.info("imap_delete_mail: permanent deleted mail %s (already in Trash, UID %s)", mail_id, uid_str)
return
# Try UID MOVE first
try:
move_resp = await client.uid('move', uid_str, trash_folder_name)
if move_resp.result == 'OK':
logger.info("imap_delete_mail: moved mail %s (UID %s) to Trash %s", mail_id, uid_str, trash_folder_name)
return
except Exception as move_exc:
logger.info("imap_delete_mail: UID MOVE not supported, falling back: %s", move_exc)
# Fallback: COPY to Trash + STORE \Deleted + EXPUNGE
copy_resp = await client.uid('copy', uid_str, trash_folder_name)
if copy_resp.result != 'OK':
raise RuntimeError(f"COPY to Trash failed for mail {mail_id}")
await client.uid('store', uid_str, r'+FLAGS (\\Deleted)')
await client.expunge()
logger.info("imap_delete_mail: moved mail %s (UID %s) to Trash via COPY+DELETE", mail_id, uid_str)
# Lifecycle hook: mail.after_delete
from app.core.hooks import do_action
await do_action("mail.after_delete", mail_id=str(mail.id), tenant_id=str(tenant_id), permanent=permanent, db=db)
except Exception as exc:
logger.warning("imap_delete_mail: failed for mail %s: %s", mail_id, exc)
raise
finally:
if client is not None:
try:
await client.logout()
except Exception:
logger.debug("Ignored exception in mail service", exc_info=True)
# ─── IMAP Move ───
async def imap_move_mail(
db: AsyncSession,
mail_id: uuid.UUID,
target_folder_id: uuid.UUID,
tenant_id: uuid.UUID,
) -> None:
"""Move mail to another folder on IMAP server.
Uses stored imap_uid directly; falls back to Message-ID search if missing.
Uses UID MOVE if supported, otherwise COPY + STORE \\Deleted + EXPUNGE.
Raises exceptions on failure so caller can queue for retry.
"""
import logging
logger = logging.getLogger(__name__)
mail = (
await db.execute(
select(Mail).where(
and_(Mail.id == mail_id, Mail.tenant_id == tenant_id)
)
)
).scalar_one_or_none()
if not mail:
logger.warning("imap_move_mail: mail %s not found", mail_id)
return
source_folder = (
await db.execute(
select(MailFolder).where(
and_(MailFolder.id == mail.folder_id, MailFolder.tenant_id == tenant_id)
)
)
).scalar_one_or_none()
if not source_folder:
logger.warning("imap_move_mail: source folder not found for mail %s", mail_id)
return
target_folder = (
await db.execute(
select(MailFolder).where(
and_(MailFolder.id == target_folder_id, MailFolder.tenant_id == tenant_id)
)
)
).scalar_one_or_none()
if not target_folder:
logger.warning("imap_move_mail: target folder %s not found", target_folder_id)
return
account = (
await db.execute(
select(MailAccount).where(
and_(MailAccount.id == source_folder.account_id, MailAccount.tenant_id == tenant_id)
)
)
).scalar_one_or_none()
if not account:
logger.warning("imap_move_mail: account not found for mail %s", mail_id)
return
if not mail.imap_uid and not mail.message_id:
logger.warning("imap_move_mail: mail %s has no imap_uid or message_id, cannot sync", mail_id)
return
# Lifecycle hook: mail.before_move
from app.core.hooks import do_action
await do_action("mail.before_move", mail_id=str(mail.id), tenant_id=str(tenant_id), source_folder_id=str(source_folder.id), target_folder_id=str(target_folder.id), db=db)
password = await get_account_password(account)
client = None
try:
client = aioimaplib.IMAP4_SSL(host=account.imap_host, port=account.imap_port)
await client.wait_hello_from_server()
await client.login(account.username, password)
select_resp = await client.select(source_folder.imap_name)
if select_resp.result != 'OK':
raise RuntimeError(f"cannot select folder {source_folder.imap_name}")
# Use stored imap_uid directly; fall back to Message-ID search
if mail.imap_uid:
uid_str = mail.imap_uid
else:
search_resp = await client.uid_search(f'HEADER Message-ID "{mail.message_id}"')
uids_raw = search_resp[1][0] if search_resp[1] and search_resp[1][0] else b''
if isinstance(uids_raw, (bytes, bytearray)):
uids = uids_raw.split()
else:
uids = []
if not uids:
raise RuntimeError(f"no UID found for Message-ID {mail.message_id}")
uid_str = uids[0].decode() if isinstance(uids[0], bytes) else str(uids[0])
# Try UID MOVE first; fall back to COPY + STORE \\Deleted + EXPUNGE
try:
move_resp = await client.uid('move', uid_str, target_folder.imap_name)
if move_resp.result == 'OK':
logger.info("imap_move_mail: moved mail %s (UID %s) via UID MOVE", mail_id, uid_str)
return
except Exception as move_exc:
logger.info("imap_move_mail: UID MOVE not supported, falling back: %s", move_exc)
# Fallback: COPY + STORE \\Deleted + EXPUNGE
copy_resp = await client.uid('copy', uid_str, target_folder.imap_name)
if copy_resp.result != 'OK':
raise RuntimeError(f"COPY failed for mail {mail_id}")
await client.uid('store', uid_str, r'+FLAGS (\\Deleted)')
await client.expunge()
logger.info("imap_move_mail: moved mail %s (UID %s) via COPY+DELETE", mail_id, uid_str)
# Lifecycle hook: mail.after_move
from app.core.hooks import do_action
await do_action("mail.after_move", mail_id=str(mail.id), tenant_id=str(tenant_id), source_folder_id=str(source_folder.id), target_folder_id=str(target_folder.id), db=db)
except Exception as exc:
logger.warning("imap_move_mail: failed for mail %s: %s", mail_id, exc)
raise
finally:
if client is not None:
try:
await client.logout()
except Exception:
logger.debug("Ignored exception in mail service", exc_info=True)
File diff suppressed because it is too large Load Diff
+3 -3
View File
@@ -4,7 +4,9 @@ from __future__ import annotations
import uuid
from datetime import UTC, datetime
from typing import Any
from pgvector.sqlalchemy import Vector
from sqlalchemy import (
JSON,
Boolean,
@@ -16,14 +18,12 @@ from sqlalchemy import (
Text,
UniqueConstraint,
)
from sqlalchemy.dialects.postgresql import TSVECTOR
from sqlalchemy.dialects.postgresql import UUID as PGUUID
from sqlalchemy.orm import Mapped, mapped_column
from app.core.db import Base, TenantMixin
from app.models.owned_mixin import OwnedMixin
from sqlalchemy.dialects.postgresql import TSVECTOR
from pgvector.sqlalchemy import Vector
from typing import Any
# --- Mail Accounts (F-MAIL-14, F-MAIL-18) ---
+48
View File
@@ -0,0 +1,48 @@
"""PGP encryption/decryption for the Mail plugin using pgpy.
Extracted from services.py as part of the God-object split (BUG-018 pilot).
Re-exported by ``app.plugins.builtins.mail.services``.
"""
from __future__ import annotations
import pgpy
def import_pgp_private_key(private_key_armored: str, passphrase: str = "") -> tuple[str, str]:
"""Import a PGP private key. Returns (key_id, public_key_armored)."""
key, _ = pgpy.PGPKey.from_blob(private_key_armored)
if key.is_protected:
with key.unlock(passphrase):
pub_key = key.pubkey
key_id = str(key.fingerprint).upper()[-16:]
return key_id, str(pub_key)
pub_key = key.pubkey
key_id = str(key.fingerprint).upper()[-16:]
return key_id, str(pub_key)
def import_pgp_public_key(public_key_armored: str) -> str:
"""Import a PGP public key. Returns key_id."""
key, _ = pgpy.PGPKey.from_blob(public_key_armored)
return str(key.fingerprint).upper()[-16:]
def pgp_encrypt_message(plaintext: str, recipient_public_key_armored: str) -> str:
"""Encrypt a message with recipient's public PGP key."""
pub_key, _ = pgpy.PGPKey.from_blob(recipient_public_key_armored)
msg = pgpy.PGPMessage.new(plaintext)
encrypted = pub_key.encrypt(msg)
return str(encrypted)
def pgp_decrypt_message(ciphertext: str, private_key_armored: str, passphrase: str = "") -> str:
"""Decrypt a PGP-encrypted message."""
key, _ = pgpy.PGPKey.from_blob(private_key_armored)
enc_msg = pgpy.PGPMessage.from_blob(ciphertext)
if key.is_protected:
with key.unlock(passphrase):
decrypted = key.decrypt(enc_msg)
return decrypted.message.decode("utf-8")
decrypted = key.decrypt(enc_msg)
return decrypted.message.decode("utf-8")

Some files were not shown because too many files have changed in this diff Show More