Commit Graph

930 Commits

Author SHA1 Message Date
Agent Zero 5b7d93cd0e refactor(deploy): remove old multi-resource code, document single docker-compose workflow
- Remove POSTGRES_COMPOSE, REDIS_COMPOSE templates (unused)
- Remove create_service(), create_api_application(), generate_worker_compose()
- Remove deploy_worker(), deploy_worker_only(), verify_worker_service()
- Remove resolve_worker_uuid() and all worker_uuid references
- Remove get_worker_envs(), get_api_envs(), get_postgres_envs(), get_redis_envs()
- Remove set_service_envs(), set_application_envs() (dead code)
- Remove _extract_deploy_uuid(), _wait_service_healthy() (only used by deploy_worker)
- Remove seed_admin_user() (only used by old deploy_full)
- Remove DB_HOST, REDIS_HOST, WORKER_UUID, WORKER_NAME config vars
- Remove --worker-only CLI arg
- Replace old deploy_full() with simple redeploy via /api/v1/deploy
- Update run_verification() to remove worker_uuid param
- Add KI workflow comment at top of deploy.py
- Update DEPLOY.md: single docker-compose stack workflow
- Update COOLIFY_SETUP.md: single docker-compose stack, remove 3-resource setup
- Update docs/INSTALL.md: automated --initial workflow

deploy.py: 1370 → 893 lines (-477 lines, -35%)
2026-08-06 01:22:09 +02:00
Agent Zero 85fcb90b32 fix(security): disable RLS on roles/permissions — blocks login (0109) 2026-08-06 01:05:33 +02:00
Agent Zero 6631615bef fix(migration): exclude login tables from RLS in 0108 — RLS blocks crm_auth login
LOGIN_TABLES (users, user_tenants, tenants, sessions, password_reset_tokens)
added to skip list. RLS on these tables blocked crm_auth from reading users
during login → 401 Invalid email or password.

crm_auth grants applied directly (no RLS) matching 0085 AUTH_TABLES.

⚠️ LOGIN-TABELLEN DÜRFEN KEIN RLS BEKOMMEN — RLS blockiert crm_auth beim Login.
Siehe 0085 AUTH_TABLES für die korrekten Grants.
2026-08-06 00:56:19 +02:00
Agent Zero 0ae8db4932 fix(security): enable RLS for all tenant tables missing RLS (0108) 2026-08-06 00:51:28 +02:00
Agent Zero 407c373173 fix(security): enable RLS automatically for plugin-created tenant tables
Check Cross-Plugin Imports / check (push) Has been cancelled
Plugin migrations run after core migration 0085 which sets up RLS for
all known core tables. Plugin-created tables were left without RLS,
creating a critical multi-tenant isolation gap (84 tables affected).

The migration runner now automatically enables RLS on all newly created
tenant tables after validation:
- ENABLE + FORCE ROW LEVEL SECURITY
- Idempotent DROP IF EXISTS + CREATE fail-closed tenant isolation policy
- GRANT CRUD to crm_api and crm_worker
- ALTER TABLE OWNER TO crm_migration

Global tables (-- GLOBAL TABLE comment) are skipped.
2026-08-06 00:38:41 +02:00
Agent Zero acbf144329 fix: dynamic container discovery for verification + verify=False for SSL 2026-08-06 00:12:34 +02:00
Agent Zero 4b41b4f7af docs: KI-Kommentare für deploy.py und docker-compose.yaml — 100% zukunftssicher 2026-08-05 23:12:23 +02:00
Agent Zero eb074bfb4d fix: 2-phase deploy (first deploy, then set domain without :443 + redeploy) 2026-08-05 22:57:25 +02:00
Agent Zero 48e6b15bb2 fix: service names with underscores (crm_app, crm_worker) + docker_compose_domains with crm_app 2026-08-05 22:46:33 +02:00
Agent Zero b3133abbc1 fix: Magic ENV SERVICE_FQDN_CRM_APP_8000 for auto domain + SSL 2026-08-05 22:29:41 +02:00
Agent Zero 549c11018c fix: domain with :443 port for SSL certificate 2026-08-05 22:24:57 +02:00
Agent Zero 2d25dc35e0 fix: docker_compose_domains + worker depends_on crm-app healthy 2026-08-05 22:18:12 +02:00
Agent Zero c278597757 fix: deploy.py --initial als einzelner docker-compose Stack + docker-compose.yaml rename 2026-08-05 22:11:29 +02:00
Agent Zero 4b72530566 fix: Widen notification_types.type_key from VARCHAR(20) to VARCHAR(100) (migration 0107)
Plugin activation was broken for ALL inactive plugins because
sync_notification_types() tried to INSERT search_reindex_complete (22 chars)
into type_key VARCHAR(20), causing StringDataRightTruncationError.

Alembic head: 0106 → 0107
2026-08-04 23:24:39 +02:00
Agent Zero 92d60badd3 fix: Grant DELETE on notification_types to app DB roles (migration 0106)
unified_search plugin activation calls sync_notification_types() which
DELETEs stale rows from notification_types. App DB user (crm_api) lacked
DELETE permission, causing plugin activation to fail with
InsufficientPrivilegeError.

Alembic head: 0105 → 0106
2026-08-04 23:02:28 +02:00
Agent Zero f15c3bec46 fix: 4 API bugs found by integration tests
Check Cross-Plugin Imports / check (push) Has been cancelled
1. tags.owner_id column missing — Migration 0105 adds owner_id to tags table
2. contacts trigger first_name vs firstname — Migration 0105 recreates
   unified_search TSV trigger with correct column names (firstname, surname, etc.)
3. create_webhook() missing is_system_admin param — Add to webhook_service.py
4. Missing GET /api/v1/search endpoint — Add to unified_search/routes.py
   with shared _do_search() helper for GET+POST

Alembic head: 0104 → 0105
2026-08-04 22:57:37 +02:00
Agent Zero b115d8211e fix: _extract_global_table_names needs self parameter
Check Cross-Plugin Imports / check (push) Has been cancelled
2026-08-04 22:48:37 +02:00
Agent Zero 16648f543a fix: Plugin migration runner + unified_search + marketplace migrations
Check Cross-Plugin Imports / check (push) Has been cancelled
Fixes 3 issues found by API integration tests:
1. migration_runner.py: Add GLOBAL TABLE exemption for tables without tenant_id
   - New _extract_global_table_names() method parses -- GLOBAL TABLE: comments
   - marketplace_listings is intentionally global (no tenant_id)
2. unified_search/migrations/0002_embeddings.sql: Remove companies table (does not exist),
   add DO $$ BEGIN END $$ blocks to check table existence before ALTER
3. marketplace/migrations/0001_initial.sql: Add -- GLOBAL TABLE: marketplace_listings comment
2026-08-04 22:47:26 +02:00
Agent Zero fcc1c92b33 fix: Migration 0104 — check table existence before ALTER, remove companies table 2026-08-04 22:42:25 +02:00
Agent Zero 6881e8abde fix: Add CREATE EXTENSION vector to migration 0104 2026-08-04 22:40:38 +02:00
Agent Zero 5d408934ba fix: Add Alembic migration 0104 for missing embedding + audit_log columns
Two critical bugs found by API integration tests:
1. contacts.embedding (vector(768)) — ORM model updated in Phase 5.3 but
   plugin migration 0002_embeddings.sql was never run as Alembic migration.
   Also adds embedding columns to mails, companies, files, calendar_entries, tags.
2. audit_log.created_at, updated_at, deleted_at — AuditLog inherits TenantMixin
   which expects these columns, but they were never added to the DB table.
   Also adds to deletion_log.

Migration uses IF NOT EXISTS checks for all columns/indexes.
Alembic head: 0103 → 0104
2026-08-04 22:39:07 +02:00
Agent Zero b60500d455 test: Fix E2E Playwright tests (25/34 pass) + cleanup old briefings
E2E Test Fixes (12 tests fixed, 25/34 now pass):
- helpers.ts: Fix API mock routes, response shapes, welcome dialog dismissal
- auth.spec.ts: Fix logout selector (duplicate button match)
- calendar.spec.ts: Fix strict mode violations, modal close assertions
- dms.spec.ts: Fix strict mode violations, modal close assertions
- contact-crud.spec.ts: Fix modal close assertion
- mail.spec.ts: Fix modal close assertion
- search.spec.ts: Fix search result expectations, empty query test

9 remaining failures: Playwright route interception with glob patterns
does not match when Vite dev proxy is configured (calendar/mail/plugins).

Cleanup:
- Delete 13 old .a0/briefings/ files
- Delete test-results/, docs/test_raw_output.md, e2e_test_report.md, test_report.md
- Delete .a0/known_errors.md (circuit breaker bug is fixed)
2026-08-04 20:53:51 +02:00
Agent Zero efba5ceb9c fix: Circuit Breaker only triggers on transient DB errors, not HTTP exceptions
The CircuitBreakerMiddleware was blocking all requests (503 circuit_open) because
every exception in get_db() — including 401 Unauthorized, 403 Forbidden, 404 Not Found —
was calling record_failure() on the DB circuit breaker. This caused the circuit to
trip after 5 non-DB errors (e.g. failed login attempts during security testing).

Fix: Only call record_failure() when _is_transient_db_error(exc) returns True,
filtering out HTTP exceptions that are not DB-related.
2026-08-04 19:43:47 +02:00
Agent Zero 17765e47b4 fix: Fix all 68 frontend test failures
P1 Code Bugs:
- SettingsPlugins.tsx: Array.isArray guard for plugins.map (9 tests)
- HtmlBlock.tsx: javascript: URL sanitization in href attributes (1 test, security fix)

P2 Test-Setup (QueryClientProvider):
- Dashboard.test.tsx: Add QueryClientProvider + dashboard mock (11 tests)
- CalendarPage.test.tsx: Add QueryClientProvider + savedFilters mock (8 tests)
- SessionList.test.tsx: Add QueryClientProvider (6 tests)
- MailPage.test.tsx: Add QueryClientProvider + savedFilters mock (8 tests fixed)
- DmsPage.test.tsx: Add QueryClientProvider + DMS API mocks (2 tests fixed)
- SettingsSystem.test.tsx: Add QueryClientProvider + sub-page mocks (1 test fixed)

P2 Test-Setup (ChevronDown Mock):
- Reports.test.tsx: Add ChevronDown to lucide-react mock (5 tests)

P3 Text Fix:
- UploadDropzone.test.tsx: Fix umlaut Auswaehlen -> Auswahlen (1 test)

Pre-existing Test Fixes (18 tests):
- MailPage.test.tsx: Remove 6 obsolete tests (compose-btn, shared-mailbox-selector, etc. — now plugin toolbar actions)
- DmsPage.test.tsx: Adapt 3 tests to new testids, remove 3 obsolete tests (upload/folder/search now plugin toolbar actions)
- SettingsSystem.test.tsx: Remove 4 obsolete tests (form fields moved to sub-pages)
- PluginRouteRenderer.test.tsx: Adapt test to loading spinner behavior
- ShareDialog.test.tsx: Fix button text i18n mismatch
2026-08-04 19:25:44 +02:00
Agent Zero 2bacadabc2 fix: Add missing SubtaskListResponse import in automation routes
Check Cross-Plugin Imports / check (push) Has been cancelled
2026-08-04 17:05:02 +02:00
Agent Zero 9fd17e7a00 chore: Delete outdated IMPLEMENTATION_PLAN.md — all 14 features already implemented 2026-08-04 17:00:44 +02:00
Agent Zero 7d976276ae test: Add 126 tests for Phase 5 plugins + fix 2 source bugs
Check Cross-Plugin Imports / check (push) Has been cancelled
Tests (5 files, 126 tests, all passing):
- test_agent_memory.py: 22 tests (store, retrieve, delete, routes, tenant isolation)
- test_graph_rag.py: 22 tests (create, traverse BFS, bidirectional, max_hops, cycles, routes)
- test_marketplace.py: 26 tests (fetch, download, verify, install, categories, routes)
- test_agent_subtasks.py: 25 tests (create, wait, cancel, aggregate, list, model)
- test_external_agent_api.py: 31 tests (run, status, stream, auth, rate limit)

Bugfixes:
- graph_rag/models.py: metadata -> meta (SQLAlchemy reserved attribute)
- marketplace/routes.py: fix default parameter validation
2026-08-04 16:02:36 +02:00
Agent Zero 25a97356d8 docs: Update .a0/worklog.md with Phase 4-5 and cleanup entries 2026-08-04 15:34:17 +02:00
Agent Zero aaf2784a9a chore: Clean up dump.rdb, templates/, test_report.md; update .a0 status files 2026-08-04 15:12:36 +02:00
Agent Zero 157e454fcc chore: Delete all outdated plan files (Sanierungsplan, FIX-PLAN, UMBAU_PLAN, etc.)
Deleted 34 outdated/obsolete planning documents:
- SANIERUNGS_FORTSCHRITT.md, UMBAU_PLAN.md, FIX-PLAN.md, FIX-PLAN-V2.md
- MASTER-PLAN.md, PLUGIN-SYSTEM-UMBAUPLAN.md, PROGRESS.md
- ENTERPRISE_RBAC_PLAN.md, RBAC_PROGRESS.md
- docs/ABSCHLUSSBERICHT_PHASE0_PHASE1.md, docs/RECOVERY_SCOPE.md
- docs/phase0_phase1_acceptance_report.md, docs/phase0_error_list.md
- quality-gate-phase1/2/2-r2/2-r3.md, security-review-phase2.md
- requirements.md, requirements-review.md, test_report.md
- frontend-gap-analysis.md, codebase-vs-requirements.md
- architecture-feasibility-review.md, extracted-architecture-details.md
- docs/migration_history_audit.md, docs/infrastructure_audit_report.md
- docs/RECOVERY_ACCEPTANCE_REPORT.md, AGENTS.md.bak

Also: Removed Sanierungsplan reference from alembic migration comment
2026-08-04 15:11:28 +02:00
Agent Zero b77b40c34f docs: Mark Security Fix Plan Phase 5.5-5.9 as complete 2026-08-04 15:07:50 +02:00
Agent Zero 000c969b13 Phase 5.5-5.9: Plugin-Marketplace, Agent Memory, GraphRAG, Subagents, External Agent API
Check Cross-Plugin Imports / check (push) Has been cancelled
5.5 Plugin-Marketplace:
- New plugin: marketplace/ (models, routes, services, schemas, config)
- MarketplaceListing model (global, no tenant_id)
- Ed25519 signature verification via PluginSignature
- Endpoints: list, detail, install, verify, categories
- Config: MARKETPLACE_SERVER_URL setting

5.6 Agent Memory (persistent):
- New plugin: agent_memory/ (models, routes, services, schemas)
- AgentMemory model with embedding vector(768) + HNSW index
- store_memory() with auto-embedding
- retrieve_relevant_memories() with pgvector cosine similarity
- Semantic search endpoint

5.7 GraphRAG:
- New plugin: graph_rag/ (models, routes, services, provider, schemas)
- EntityRelationship model (source/target type+id, relationship_type, metadata)
- BFS graph traversal (bidirectional, configurable depth)
- GraphRAGSearchProvider registered in unified_search

5.8 Subagents / Multi-Agent:
- AgentCoordinator class (create_subtask, wait_for_subtask, aggregate, cancel)
- AgentSubtask model + migration 0002_agent_subtasks.sql
- 6 new API endpoints for subtask management
- Tools registered in AI tool registry

5.9 External Agent API:
- external_api.py: POST /run, GET /status, POST /stream (SSE)
- Bearer API token authentication
- Rate limiting: 10 req/min per token
- ExternalAgentRequest/Response schemas

3 new plugins registered in main.py and __init__.py
All files py_compile clean
2026-08-04 15:06:23 +02:00
Agent Zero 597aea1c23 docs: Mark Security Fix Plan Phase 5.1-5.4 as complete 2026-08-04 14:51:14 +02:00
Agent Zero cfb4c5ae8b Phase 5.1-5.4: PWA, Public Plugin Endpoints, Contacts Embedding, Search Coverage
Check Cross-Plugin Imports / check (push) Has been cancelled
5.1 Public Plugin Endpoints:
- PluginRouteDef.is_public field in manifest.py
- main.py: public routes mounted without auth dependency
- permissions/public_routes.py: token-based share link access (info, verify, download)
- permissions/plugin.py: public share route registered with is_public=True

5.2 PWA:
- vite.config.ts: VitePWA plugin configured (autoUpdate, workbox, runtime caching)
- frontend/public/manifest.json: PWA manifest with icons
- index.html: theme-color, manifest link, apple-touch-icon, apple-mobile-web-app meta
- Build generates sw.js + workbox (90 precache entries)

5.3 Contacts Embedding:
- contact.py: embedding column (Vector(768)) added to Contact model
- Migration 0002_embeddings.sql already exists (adds embedding + HNSW index)
- ContactSearchProvider already queries embedding column

5.4 Search Coverage:
- 5 new search providers: task, contactperson, tag, conversation, user
- All providers implement FTS search with tenant_id + deleted_at filters
- TagSearchProvider also supports vector search (384-dim embedding)
- provider_registry.py: all 5 new providers auto-registered
- Total: 10 search providers (was 5)
2026-08-04 14:49:35 +02:00
Agent Zero f704f7b032 docs: Mark Security Fix Plan Phase 4 as complete 2026-08-04 14:37:54 +02:00
Agent Zero a26405f15e Phase 4: Circuit Breaker, DB Retry, Redis Graceful Degradation
- app/core/resilience.py: CircuitBreaker (CLOSED/OPEN/HALF_OPEN), retry_db,
  redis_call_with_fallback, InMemoryRateLimiter, CircuitBreakerMiddleware
- app/core/auth.py: get_session_data now falls back to PostgreSQL sessions
  table when Redis is unavailable
- app/core/permissions.py: get_cached_permissions falls back to direct DB
  resolution when Redis circuit is open
- app/core/rate_limit.py: check_rate_limit falls back to in-memory limiter
  when Redis is down; reset_rate_limit clears both Redis and in-memory
- app/core/middleware.py: CSRF validation uses get_session_data (Redis+DB
  fallback); sliding session TTL is best-effort during outage
- app/core/db/__init__.py: get_db() wraps session creation with retry_db
  for transient connection errors; records circuit breaker success/failure
- app/deps.py: refresh_session_ttl wrapped in try/except for Redis outage
- app/main.py: CircuitBreakerMiddleware registered (returns 503 when DB
  circuit is OPEN, skips health/metrics endpoints)
- app/config.py: Added resilience settings (thresholds, cooldown, retries)
- tests/test_resilience.py: 30 tests covering all patterns

30/30 resilience tests pass. No regressions in plugin lifecycle tests.
2026-08-04 14:34:06 +02:00
Agent Zero 247d4165ea docs: Final cleanup — remove all old UUIDs from all files
- .env.example: Add ADMIN_EMAIL/ADMIN_PASSWORD
- SANIERUNGS_FORTSCHRITT.md: Update UUIDs
- IMPLEMENTATION_PLAN.md: Update UUID
- promptinclude: Worker/DB/Redis are now part of Docker-Compose-App
2026-08-04 13:43:54 +02:00
Agent Zero 0ce3d43ae2 docs: Update UUIDs and installation docs
- Replace old UUID stvabl4vaqru7jclx4ittzr3 with dx4pqdziu4uj6x9fxs1u5z0x
- Remove old worker UUID asxqaq3566to108xordck0ff
- Update INSTALL.md: Stand 2026-08-04, Commit 0ebc411, Alembic-Head 0103
- Update DEPLOY.md: New UUID and auto-resolve via APP_DOMAIN
2026-08-04 13:16:28 +02:00
Agent Zero 0ebc411fd8 feat: Auto-seed admin user on container start
- prestart.sh: runs seed_admin.py after migrations
- seed_admin.py: reads ADMIN_EMAIL and ADMIN_PASSWORD from env vars
- Creates default tenant + admin role + admin user if not exists
2026-08-04 12:21:38 +02:00
Agent Zero 4b00204b63 feat: Domain-based app names + admin user seeding
- deploy.py: APP_NAME derived from APP_DOMAIN (e.g. crm.media-on.de → crm)
- deploy.py: seed_admin_user() runs seed_admin.py after deploy
- fast-deploy.sh: Same domain-based name derivation
- No hardcoded leocrm-api/leocrm-worker defaults
2026-08-04 12:17:01 +02:00
Agent Zero 0d06e73fe5 Fix: Simplify docker-compose.yml — remove custom networks and labels
- No custom crm-net network — Coolify manages networking
- No custom Traefik labels — Coolify generates them
- No hardcoded domains — all from environment variables
- Simplified volumes — no custom names
2026-08-04 11:55:04 +02:00
Agent Zero 51c9b467b2 Fix: verify_ws_origin async + await callers
Check Cross-Plugin Imports / check (push) Has been cancelled
- auth.py: verify_ws_origin is now async def
- kommunikation/routes.py: await verify_ws_origin
- ai_ui_control/routes.py: await verify_ws_origin
2026-08-04 11:41:18 +02:00
Agent Zero 7d3007b6c4 Fix: Remove hardcoded UUIDs and secrets from deploy scripts
- deploy.py: UUIDs from env vars or Coolify API lookup by name
- fast-deploy.sh: No hardcoded UUIDs, APP_DOMAIN from env
- docker-compose.yml: All secrets from env vars, no hardcoded values
- .env.example: All required vars documented
- Deleted obsolete fast-frontend-deploy.sh with hardcoded container name
2026-08-04 11:13:44 +02:00
Agent Zero e7edc46286 Phase 3: WebSocket CSRF, SameSite=Lax, Tenant FK CASCADE 2026-08-04 09:27:10 +02:00
Agent Zero 4a104af615 Fix duplicate networks key in crm-app service 2026-08-04 00:37:58 +02:00
Agent Zero 6481996334 Add Traefik labels and coolify network for crm.media-on.de routing 2026-08-04 00:31:56 +02:00
Agent Zero 51a2c44238 Fix: Migration 0102 checks table existence before adding owner_id 2026-08-04 00:18:06 +02:00
Agent Zero 40e9943e69 Fix: Remove external port binding from crm-app (conflicts with Coolify on port 8000) 2026-08-04 00:11:30 +02:00
Agent Zero e17b9c9e56 Phase 2: Visibility Filter & Owner ID
Check Cross-Plugin Imports / check (push) Has been cancelled
- Add OwnedMixin to 15 models (contact_folder, user_preference, workspace,
  mcp_server_config, agent_definition, automation_definition, report_template,
  report_instance, entity_link, comm_conversation, proactive_suggestion,
  ai_agent, ai_chat_session, tag, share_link)
- Migration 0102: Add owner_id column to 15 tables with backfill from user_id
- Fix EntityPermission Registry: remove notification, add entity_attachment,
  entity_history, subtask, calendar, folder; fix wrong class names
  (DmsFile→File, CalendarEvent→CalendarEntry, Mailbox→MailAccount)
- Add apply_visibility_filter to list endpoints in tags, tasks, mcp_client,
  automation, report_generator, ai_assistant routes
- Add owner_id to create handlers for all new OwnedMixin models
- Patch tasks/services.py and automation/services.py list methods with
  user_id and is_system_admin parameters
2026-08-04 00:03:29 +02:00
Agent Zero 93a330ae40 Fix: Simplify migration 0101 — only disable RLS on auth tables 2026-08-03 23:14:26 +02:00