f15c3bec46
fix: 4 API bugs found by integration tests
...
Check Cross-Plugin Imports / check (push) Has been cancelled
1. tags.owner_id column missing — Migration 0105 adds owner_id to tags table
2. contacts trigger first_name vs firstname — Migration 0105 recreates
unified_search TSV trigger with correct column names (firstname, surname, etc.)
3. create_webhook() missing is_system_admin param — Add to webhook_service.py
4. Missing GET /api/v1/search endpoint — Add to unified_search/routes.py
with shared _do_search() helper for GET+POST
Alembic head: 0104 → 0105
2026-08-04 22:57:37 +02:00
fcc1c92b33
fix: Migration 0104 — check table existence before ALTER, remove companies table
2026-08-04 22:42:25 +02:00
6881e8abde
fix: Add CREATE EXTENSION vector to migration 0104
2026-08-04 22:40:38 +02:00
5d408934ba
fix: Add Alembic migration 0104 for missing embedding + audit_log columns
...
Two critical bugs found by API integration tests:
1. contacts.embedding (vector(768)) — ORM model updated in Phase 5.3 but
plugin migration 0002_embeddings.sql was never run as Alembic migration.
Also adds embedding columns to mails, companies, files, calendar_entries, tags.
2. audit_log.created_at, updated_at, deleted_at — AuditLog inherits TenantMixin
which expects these columns, but they were never added to the DB table.
Also adds to deletion_log.
Migration uses IF NOT EXISTS checks for all columns/indexes.
Alembic head: 0103 → 0104
2026-08-04 22:39:07 +02:00
157e454fcc
chore: Delete all outdated plan files (Sanierungsplan, FIX-PLAN, UMBAU_PLAN, etc.)
...
Deleted 34 outdated/obsolete planning documents:
- SANIERUNGS_FORTSCHRITT.md, UMBAU_PLAN.md, FIX-PLAN.md, FIX-PLAN-V2.md
- MASTER-PLAN.md, PLUGIN-SYSTEM-UMBAUPLAN.md, PROGRESS.md
- ENTERPRISE_RBAC_PLAN.md, RBAC_PROGRESS.md
- docs/ABSCHLUSSBERICHT_PHASE0_PHASE1.md, docs/RECOVERY_SCOPE.md
- docs/phase0_phase1_acceptance_report.md, docs/phase0_error_list.md
- quality-gate-phase1/2/2-r2/2-r3.md, security-review-phase2.md
- requirements.md, requirements-review.md, test_report.md
- frontend-gap-analysis.md, codebase-vs-requirements.md
- architecture-feasibility-review.md, extracted-architecture-details.md
- docs/migration_history_audit.md, docs/infrastructure_audit_report.md
- docs/RECOVERY_ACCEPTANCE_REPORT.md, AGENTS.md.bak
Also: Removed Sanierungsplan reference from alembic migration comment
2026-08-04 15:11:28 +02:00
e7edc46286
Phase 3: WebSocket CSRF, SameSite=Lax, Tenant FK CASCADE
2026-08-04 09:27:10 +02:00
51a2c44238
Fix: Migration 0102 checks table existence before adding owner_id
2026-08-04 00:18:06 +02:00
e17b9c9e56
Phase 2: Visibility Filter & Owner ID
...
Check Cross-Plugin Imports / check (push) Has been cancelled
- Add OwnedMixin to 15 models (contact_folder, user_preference, workspace,
mcp_server_config, agent_definition, automation_definition, report_template,
report_instance, entity_link, comm_conversation, proactive_suggestion,
ai_agent, ai_chat_session, tag, share_link)
- Migration 0102: Add owner_id column to 15 tables with backfill from user_id
- Fix EntityPermission Registry: remove notification, add entity_attachment,
entity_history, subtask, calendar, folder; fix wrong class names
(DmsFile→File, CalendarEvent→CalendarEntry, Mailbox→MailAccount)
- Add apply_visibility_filter to list endpoints in tags, tasks, mcp_client,
automation, report_generator, ai_assistant routes
- Add owner_id to create handlers for all new OwnedMixin models
- Patch tasks/services.py and automation/services.py list methods with
user_id and is_system_admin parameters
2026-08-04 00:03:29 +02:00
93a330ae40
Fix: Simplify migration 0101 — only disable RLS on auth tables
2026-08-03 23:14:26 +02:00
d43407ca77
Fix: Disable RLS on auth tables, correct policy syntax
2026-08-03 23:06:01 +02:00
4f970a11eb
Phase 1: Critical security fixes - 59 permissions, grants, RLS, mass-assignment, ownership, leaks, MIME
Check Cross-Plugin Imports / check (push) Has been cancelled
2026-08-03 22:32:03 +02:00
5d35f0064e
Performance Optimierungen: Rate Limit, created_at Index, Keyset-Pagination
...
1. Rate Limit erhoeht: 60 -> 300 Requests/Minute (fuer 100+ User)
2. Migration 0099: created_at DESC Index auf allen Tabellen (Order by Performance)
3. Keyset-Pagination: optionaler cursor Parameter fuer contacts API
- cursor=UUID nutzt WHERE id > cursor statt OFFSET
- Backward compatible: ohne cursor wird page/page_size genutzt
- next_cursor in Response fuer naechste Seite
Tests: 43/43 bestanden
2026-08-03 19:18:09 +02:00
fe6a4fdd54
Fix: Migration 0094 prueft Spalten-Existenz vor GIN-Index-Erstellung
2026-08-03 16:14:43 +02:00
f4364f30e0
Phase 8.1+8.2: CI Pipeline und Migrations-Release-Gate
...
8.1 Merge-CI:
- Backend Tests und Frontend Tests zu ci_pipeline.sh hinzugefuegt
- Migration Hash Check (<=0092) mit check_migration_hashes.py
- npm ci --legacy-peer-deps in Forgejo Workflow und ci_pipeline.sh
- 93 Migration-Hashes generiert und verifiziert
8.2 Migrations-Release-Gate:
- migration_release_gate.sh: Fresh Install, Schema Snapshot, RLS/Grants Check, Cross-Tenant Test, Data Integrity
- Prueft leere DB Installation mit Alembic Head + Plugin-Migrationen
- Verifiziert RLS >= 100 Tabellen, 4 DB-Rollen, kein BYPASSRLS auf crm_api
- Cross-Tenant: 0 rows ohne/fake tenant context
2026-08-03 15:49:03 +02:00
29d55cb187
Phase 6: DMS & Attachments — Streaming, Deduplikation, API-Bereinigung
...
Check Cross-Plugin Imports / check (push) Has been cancelled
6.4 Upload streamen:
- attachment_service.save_attachment: Streamt in 1MB Chunks statt await file.read()
- routes/attachments.py: Uebergibt UploadFile direkt statt bytes
6.5 Download streamen:
- DMS preview_file: FileResponse fuer LocalStorage (automatisches Streaming)
- Kein storage.read() mehr fuer LocalStorage
6.6 Tenantlokale Deduplikation:
- DMS Upload: Prueft content_hash vor Erstellung, wiederverwendet existierendes File
- attachment_service: Dedup bereits vorhanden, jetzt mit Streaming kompatibel
- Migration 0098: Partial Unique Index (tenant_id, content_hash) WHERE content_hash IS NOT NULL AND deleted_at IS NULL
6.7 API-Ausgabe bereinigt:
- attachment_service: storage_path und content_hash aus API-Ausgaben entfernt
- DMS routes: content_hash aus 4 API-Endpunkten entfernt
Tests: 54/54 bestanden (17 Workspace + 13 API Token + 24 Command)
2026-08-03 14:21:43 +02:00
4efdc8e036
Fix: Migration 0097 — api_tokens.updated_at Spalte hinzufuegen
...
ApiToken Modell erbt von TenantMixin (TimestampMixin) das updated_at erwartet.
Migration 0001 hat api_tokens ohne updated_at erstellt.
Migration 0083 hat deleted_at hinzugefuegt aber updated_at verpasst.
2026-08-03 14:10:04 +02:00
3eb11b1745
Phase 1: Migrationsaudit + Forward-Migrationen 0093-0096
...
Audit (docs/migration_history_audit.md):
- files.size_bytes: INTEGER (Alembic) vs BIGINT (Produktion/Plugin)
- GIN-Indizes: Fehlendes USING GIN in Alembic 0002
- guest_users: ix_guest_users_email_tenant fehlt UNIQUE in Alembic 0059
- plugins.name: Doppelter Unique-Index in Produktion
Forward-Migrationen:
- 0093: files.size_bytes INTEGER → BIGINT
- 0094: GIN-Indizes reparieren + plugins.name doppelten Index entfernen
- 0095: guest_users email+tenant_id UNIQUE INDEX (mit Dubletten-Check)
- 0096: Workspace tenant_integrity (tenant-bound FKs)
Tests: 41/41 bestanden (17 Workspace + 24 Command)
Alembic Head: 0096
2026-08-03 13:29:16 +02:00
07a99975ec
Phase 5: Outbox DLQ, Monitoring, Consumer-Registry
...
- Migration 0092: DLQ columns (error_message, failed_at) + consumer_inbox RLS fix
- outbox.py: DLQ logic, replay functions, stats, consumer registry
- app/routes/outbox.py: 5 API endpoints (stats, failed, replay, replay-all, consumer-registry)
- outbox_deliveries tracking per consumer handler
- 18/18 tests passing
2026-08-02 23:25:54 +02:00
745bc4f2d8
feat: Phase 2 — Migration 0091: FK-Constraints für 74 Tenant-Tabellen
...
- 74 Tabellen erhalten FOREIGN KEY (tenant_id) REFERENCES tenants(id) ON DELETE CASCADE
- 10 globale Tabellen ausgeschlossen (sequences, system_settings, currencies, etc.)
- Orphan-Cleanup: SET tenant_id = NULL für verwaiste Einträge
- Idempotent: IF NOT EXISTS für alle Constraints
- Downgrade: Drop aller FK-Constraints
2026-08-01 23:04:15 +02:00
89b775b9ef
fix: legacy app.tenant_id policies on _old tables + seed_admin.py rewrite
...
- Migration 0090: Drop legacy tenant_isolation policies on companies_old,
company_contacts_old, contacts_old that used app.tenant_id variable.
Create new policies using app.current_tenant_id for crm_api/crm_worker.
- seed_admin.py: Rewrite to use migration engine (crm_migration) for
bootstrap, set tenant context, create Tenant + Role + User + UserTenant.
No longer passes tenant_id as User parameter.
Fixes: 3 legacy app.tenant_id policies found in Gate 2 verification.
Fixes: seed_admin.py incompatible with current User model.
2026-07-31 22:23:38 +02:00
b5191f0d11
gate2: migration 0089 — add updated_at to sessions table (model uses TimestampMixin but table was missing column)
2026-07-31 22:15:29 +02:00
10296137e9
gate2: fix migration 0085 — revoke default privileges before dropping crm_runtime, handle dependent_objects_still_exist
2026-07-31 21:20:59 +02:00
010ef448e7
gate2: fix all migrations for fresh DB installation
2026-07-31 19:16:11 +02:00
d37388423d
gate2: fix ix_contacts_tenant_id conflict — drop old index before recreate in 0021
2026-07-31 18:51:15 +02:00
e43a906cde
gate2: fix ix_contacts_tenant_id duplicate (index=True in 0021 vs create_index in 0002)
2026-07-31 18:34:35 +02:00
dd7ad461d8
gate2: fix duplicate column/index in migrations for fresh DB installation
2026-07-31 18:20:09 +02:00
224a5ea9af
gate2: fix migration 0019 duplicate deleted_at on roles (IF NOT EXISTS)
2026-07-31 17:50:07 +02:00
3032ad2cbf
gate4: migration 0088 — auth RLS policies for password_reset_tokens and audit_log
2026-07-31 12:04:27 +02:00
437c107ee8
gate: migration 0087 add timestamps to password_reset_tokens, backup uploaded to Forgejo
2026-07-31 09:23:34 +02:00
1a980ba9d8
gate: migration 0086, crm_migration BYPASSRLS, audit_log fix, CI test for app.tenant_id
...
- Migration 0086: Remove FORCE RLS from 5 global tables
- Migration 0085: crm_migration keeps BYPASSRLS for data migrations
- Migration 0085: Remove audit_log from crm_auth grants
- auth_service.py: Audit log via separate API session (crm_api with tenant context)
- tests/test_no_legacy_tenant_var.py: CI test for app.tenant_id in policies
2026-07-31 09:02:40 +02:00
15f0a07d4e
phase1: fix auth_service tenant context for audit_log, add sessions+audit_log to crm_auth grants
...
- auth_service.py: set tenant context before audit log write in login
- migration 0085: add sessions and audit_log to AUTH_TABLES for crm_auth
- Login now works on production with RLS enabled
2026-07-31 02:28:29 +02:00
100b9f705c
phase1: separate DB roles, RLS restoration, login on crm_auth
...
Check Cross-Plugin Imports / check (push) Has been cancelled
- config.py: add auth_database_url, worker_database_url, migration_database_url
- db/__init__.py: separate engines for auth/worker/migration + get_auth_db/get_worker_db
- auth.py: all auth endpoints use get_auth_db (crm_auth role)
- auth_service.py: remove login fallback, require active membership, check status
- auth_service.py: switch_tenant checks active membership status
- alembic/env.py: use migration_database_url for Alembic
- docker-compose.yml: add AUTH_DATABASE_URL, WORKER_DATABASE_URL
- .env.example: add all 4 DB URLs with separate roles
- migration 0085: transfer ownership to crm_migration, fix BYPASSRLS,
enable RLS+FORCE on all tenant tables, drop old policies, create new
fail-closed policies scoped to crm_api+crm_worker, revoke excessive grants,
grant minimal crm_auth access, drop crm_runtime, set default privileges
- tests/test_rls_coverage.py: automated RLS coverage check (13 tests)
- tests/test_cross_tenant_security_v2.py: RLS tests with unprivileged role
2026-07-31 02:05:16 +02:00
0692fce2e4
fix: RLS fail-closed migration + per-tenant startup code
2026-07-31 01:31:41 +02:00
7fbbe420bd
fix: comprehensive system audit fixes (55+ issues)
...
Check Cross-Plugin Imports / check (push) Has been cancelled
CRITICAL:
- Fix SQL injection in prestart.sh (parameterized query)
- Fix secret key validation (always validate, not just production)
- Fix workspace model partial index bug (func.text -> text)
- Fix HealthResponse schema (add checks field)
- Fix Tenant import in permissions.py (NameError on every auth request)
- Fix README tech stack (React instead of Alpine.js)
- Delete broken test_cross_tenant_security_v2.py
- Add fail-closed RLS migration 0084 (48 tenant tables)
HIGH:
- Add GeneralRateLimitMiddleware for all API routes
- Add file type blocklist for DMS and attachment uploads
- Fix guest auth: Pydantic schema, tenant_slug required, CSRF bypass
- Fix CSRF bypass path matching (in -> endswith)
- Add worker healthcheck in docker-compose.yml
- Add ARQ max_tries=3 for job retries
- Fix 28 bare pass in mail services (-> logger.debug)
- Fix print() -> logger in main.py and ai_assistant
- Fix duplicate email handling (catch IntegrityError -> 409)
- Add session revocation (invalidate_all_user_sessions)
- Add resource limits to all containers
- Fix CORS default (localhost -> production domain)
- Fix SameSite=Lax -> Strict
- Fix Redis password visibility in healthcheck
- Fix npm vulnerabilities (19 -> 9)
- Fix Sidebar OOM (wildcard lucide import -> curated ICON_MAP)
MEDIUM:
- Localize ErrorBoundary to German
- Wire Mail.tsx save/delete filter to API
- Document system_notif plugin (no routes needed)
- Fix datetime.utcnow() -> datetime.now(UTC)
- Pin litellm version (>=1.0,<2.0)
- Move CSRF token from sessionStorage to in-memory
- Fix restore_backup error handling and transaction
- Fix Dms.tsx useEffect cleanup
- Add skip-to-content link for accessibility
- Add selectinload imports to 3 services
- Add .env.example missing variables
- Fix AppShell/TopBar/Sidebar test mocks
NEW TESTS:
- test_guest_auth.py (6 tests)
- test_user_service.py (8 tests)
- test_backup_service.py (5 tests)
NEW SCHEMAS:
- saved_filter, saved_view, user_preference, workspace, entity_policy
Tests: 22/22 PASSED
2026-07-31 00:58:05 +02:00
076134b445
migration: add missing deleted_at columns to 9 TenantMixin tables (0083)
2026-07-30 10:16:26 +02:00
8acc00c559
migration: add sensitivity column to custom_field_definitions (0082)
2026-07-30 09:27:04 +02:00
d4ffbeca50
phase12: disable RLS on all system/auth/config/plugin tables (final migration)
2026-07-30 00:47:43 +02:00
8833444dcb
phase12: disable RLS on audit_log and sessions (written during login)
2026-07-30 00:20:20 +02:00
02af9ebaa2
phase12: disable RLS on all system/auth/config tables for crm_api startup
2026-07-30 00:15:58 +02:00
42d004c2c9
phase12: disable RLS on automation tables (written at startup)
2026-07-30 00:06:43 +02:00
0d7602db3a
phase12: disable RLS on tax_rates (startup table)
2026-07-29 23:42:25 +02:00
1611b2450e
phase12: disable RLS on startup tables (system_settings, currencies, taxes, sequences, saved_filters, saved_views, webhooks)
2026-07-29 23:33:56 +02:00
54c275580f
phase6: standardized event envelope (aggregate_type, aggregate_id, occurred_at, correlation_id, schema_version) + outbox_deliveries table
2026-07-29 22:50:27 +02:00
bd50a85483
fix: add created_at/updated_at to workspace_users (TenantMixin inherits TimestampMixin)
2026-07-29 18:40:09 +02:00
8094b6d13f
fix: add deleted_at to workspace tables (TenantMixin includes SoftDeleteMixin)
2026-07-29 18:38:06 +02:00
2423053477
phase5: workspace backend — models, service, routes, migration 0072
2026-07-29 18:32:35 +02:00
8322adb73f
phase4: entity_attachments table + DMS unified storage + attachment service rewritten + download via DMS
2026-07-29 17:52:55 +02:00
840795b5b9
phase2: 4 DB roles (crm_migration/api/worker/auth) + docker-compose updated + GRANT USAGE + RLS verified with unprivileged role
2026-07-29 16:49:09 +02:00
8da803156e
phase1: RLS simplified to tenant isolation only + canAccess fallback removed + useUserPermissions hook + security kernel docs
2026-07-29 16:36:51 +02:00
0448962d08
fix: visibility.py Defense-in-Depth tenant_id filter + entity_permissions deleted_at migration + cross-tenant tests
2026-07-29 16:12:04 +02:00
Agent Zero