Compare commits
5 Commits
a897bca390
...
b6e3afd28b
| Author | SHA1 | Date | |
|---|---|---|---|
| b6e3afd28b | |||
| 825d638130 | |||
| 604a2b7648 | |||
| 5ec1fc9b05 | |||
| 7a14973c68 |
+32
-6
@@ -1,9 +1,38 @@
|
|||||||
# LeoCRM — Current Status
|
# LeoCRM — Current Status
|
||||||
**Phase**: Fix Branch — P1-4 Complete
|
**Phase**: Fix Branch — 20/22 FIX-PLAN Items erledigt
|
||||||
**Last update**: 2026-07-25 19:17
|
**Last update**: 2026-07-26 16:25
|
||||||
**Branch**: main (leocrm-fix)
|
**Branch**: main (leocrm-fix)
|
||||||
|
|
||||||
## P1-4: Transactional Outbox — COMPLETE
|
## FIX-PLAN Überprüfung (2026-07-26)
|
||||||
|
Alle 22 Items gegen Codebasis verifiziert. 20 erledigt, 2 offen.
|
||||||
|
|
||||||
|
### Erledigt (20)
|
||||||
|
- P0-1: Auth-Bypass entfernt ✅
|
||||||
|
- P0-2: Migrationen repariert ✅
|
||||||
|
- P0-3: Plugin-Upload deaktiviert ✅
|
||||||
|
- P0-4: RLS FORCE + WITH CHECK ✅
|
||||||
|
- P0-5: Plugin-Doppelregistrierung behoben ✅
|
||||||
|
- P0-6: Persistent Volume ✅
|
||||||
|
- P1-1: User/Tenant-Modell bereinigt ✅
|
||||||
|
- P1-2: Redis zentralisiert ✅
|
||||||
|
- P1-3: Worker ausgelagert ✅
|
||||||
|
- P1-4: Transactional Outbox ✅
|
||||||
|
- P1-5: XSS-Stellen geschlossen ✅
|
||||||
|
- P1-6: DMS lastfest ✅
|
||||||
|
- P1-7: Permission-System vereinheitlicht ✅
|
||||||
|
- P1-8: Password Reset funktionsfähig ✅
|
||||||
|
- P1-9: Metrics abgesichert ✅
|
||||||
|
- P1-10: Coolify-Doku & Config korrigiert ✅
|
||||||
|
- P1-11: Cross-Tenant FK ✅
|
||||||
|
- P2-1: Contact Model normalisiert ✅
|
||||||
|
- P2-3: Commands & Statusmaschinen ✅
|
||||||
|
- P2-4: SPA Path-Traversal ✅
|
||||||
|
|
||||||
|
### Offen (2)
|
||||||
|
- P0-7: App von öffentlicher Domain nehmen (operational — 30 Min)
|
||||||
|
- P2-2: Plugin-Cross-Imports reduzieren (228 Imports — 1-2 Wochen)
|
||||||
|
|
||||||
|
## Previous: P1-4: Transactional Outbox — COMPLETE
|
||||||
- Migration 0040_outbox.py created (down_revision=0039_contact_normalize)
|
- Migration 0040_outbox.py created (down_revision=0039_contact_normalize)
|
||||||
- event_outbox table: id, tenant_id, event_name, payload JSONB, status, attempts, max_attempts, next_retry_at, timestamps
|
- event_outbox table: id, tenant_id, event_name, payload JSONB, status, attempts, max_attempts, next_retry_at, timestamps
|
||||||
- app/core/outbox.py: enqueue_outbox_event() + process_outbox_batch() with FOR UPDATE SKIP LOCKED, exponential backoff retry
|
- app/core/outbox.py: enqueue_outbox_event() + process_outbox_batch() with FOR UPDATE SKIP LOCKED, exponential backoff retry
|
||||||
@@ -16,6 +45,3 @@
|
|||||||
|
|
||||||
## Previous: P2-1: Unified Contact Model normalisieren — COMPLETE
|
## Previous: P2-1: Unified Contact Model normalisieren — COMPLETE
|
||||||
- Migration 0039_contact_normalize.py (down_revision=0038_dms_content_hash)
|
- Migration 0039_contact_normalize.py (down_revision=0038_dms_content_hash)
|
||||||
|
|
||||||
## Next Step
|
|
||||||
- Continue with next fix task from FIX-PLAN.md
|
|
||||||
|
|||||||
+9
-5
@@ -1,6 +1,10 @@
|
|||||||
# LeoCRM — Next Steps
|
# LeoCRM — Next Steps
|
||||||
1. P2-1: Unified Contact Model normalisieren — COMPLETE
|
|
||||||
2. P1-4: Transactional Outbox — COMPLETE
|
## FIX-PLAN Offene Items (2026-07-26)
|
||||||
3. Continue with next fix task from FIX-PLAN.md (next priority)
|
1. P0-7: App von öffentlicher Domain nehmen (operational — 30 Min)
|
||||||
4. Pre-existing test failures (403/404 in test_contacts.py) need separate investigation — not caused by P1-4 or P2-1
|
2. P2-2: Plugin-Cross-Imports reduzieren (228 Imports — 1-2 Wochen)
|
||||||
5. notification.created event in notifications.py kept on event_bus.publish() (local notification signal, not a domain event needing cross-process delivery)
|
|
||||||
|
## Abgeschlossen
|
||||||
|
- P2-1: Unified Contact Model normalisieren — COMPLETE
|
||||||
|
- P1-4: Transactional Outbox — COMPLETE
|
||||||
|
- 20/22 FIX-PLAN Items erledigt (siehe .a0/current_status.md)
|
||||||
|
|||||||
+200
@@ -0,0 +1,200 @@
|
|||||||
|
# LeoCRM Security & Data Risk Assessment
|
||||||
|
|
||||||
|
**Date:** 2026-07-26
|
||||||
|
**Assessor:** Security Data Engineer (A0 Orchestrator)
|
||||||
|
**Project:** LeoCRM at `/a0/usr/workdir/leocrm-fix`
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Summary
|
||||||
|
|
||||||
|
| Severity | Count |
|
||||||
|
|----------|-------|
|
||||||
|
| CRITICAL | 5 |
|
||||||
|
| HIGH | 8 |
|
||||||
|
| MEDIUM | 8 |
|
||||||
|
| LOW | 5 |
|
||||||
|
| **Total**| **26**|
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## CRITICAL Issues
|
||||||
|
|
||||||
|
### C-1: Redis Default Password `changeme` in docker-compose.yml
|
||||||
|
**File:** `docker-compose.yml:53`
|
||||||
|
**Risk:** Redis stores session data, CSRF tokens, and rate-limit counters. The default password `changeme` is trivially guessable. If Redis port 6379 is exposed, an attacker can read/modify all sessions, steal CSRF tokens, and bypass rate limits.
|
||||||
|
**Remediation:** Remove the default fallback. Require `REDIS_PASSWORD` as a mandatory variable (`${REDIS_PASSWORD:?REDIS_PASSWORD is required}`). Use a strong randomly generated password in production.
|
||||||
|
|
||||||
|
### C-2: No SECRET_KEY in `.env` — Insecure Default Active in Development
|
||||||
|
**File:** `.env` (missing `SECRET_KEY`), `app/config.py:55`
|
||||||
|
**Risk:** `.env` has no `SECRET_KEY`. The config defaults to `"change-me-in-production-use-a-secure-random-string"`. While `get_settings()` raises in production mode, `.env` sets `ENVIRONMENT=development`, so the default key is silently used. Any signing/token operation using `secret_key` is compromised.
|
||||||
|
**Remediation:** Add a strong random `SECRET_KEY` (min 32 chars) to `.env`. Fail-fast in all environments if the default key is detected, not just production.
|
||||||
|
|
||||||
|
### C-3: PostgreSQL and Redis Ports Exposed to Host
|
||||||
|
**File:** `docker-compose.yml:37-38, 56-57`
|
||||||
|
**Risk:** `ports: "5432:5432"` and `ports: "6379:6379"` expose the database and Redis to the host network. Combined with weak/default credentials, this allows direct external access to all session data and the entire database.
|
||||||
|
**Remediation:** Remove port mappings for production. Use Docker internal networking only (`crm-net`). If debug access is needed, bind to `127.0.0.1:5432:5432` and document it as dev-only.
|
||||||
|
|
||||||
|
### C-4: Unauthenticated Error Endpoint Forwards Data to External Forgejo
|
||||||
|
**File:** `app/routes/errors.py:54-90`, `app/plugins/builtins/forgejo_error_reporter/service.py:151-250`
|
||||||
|
**Risk:** The `/api/v1/errors` endpoint requires no authentication. CSRF middleware explicitly bypasses token checks for this path (line 48 of `middleware.py`). Any unauthenticated attacker can POST arbitrary error data (message, stack, URL, userAgent, and **arbitrary context dict**) which gets forwarded to an external Forgejo instance as a public issue. The `context` field accepts `dict[str, Any]` with no size limit on individual keys — an attacker can exfiltrate data or inject malicious content into Forgejo issues.
|
||||||
|
**Remediation:** Require authentication for error reporting. If unauthenticated errors are needed, strip the `context` field entirely, add strict schema validation with size limits on all fields, and add a CAPTCHA or stricter rate limiting.
|
||||||
|
|
||||||
|
### C-5: Plaintext Database Password in `.env`
|
||||||
|
**File:** `.env:1`
|
||||||
|
**Risk:** `DATABASE_URL=postgresql+asyncpg://leocrm:leocrm@localhost:5432/leocrm` embeds the DB password `leocrm` in plaintext. While `.gitignore` covers `.env`, the password is weak and identical to the username. If the file is accessed via any path traversal, backup leak, or container escape, the database is fully compromised.
|
||||||
|
**Remediation:** Use a strong unique password. Separate `DATABASE_URL` construction from credential storage where possible (e.g., use individual `POSTGRES_USER`, `POSTGRES_PASSWORD`, `POSTGRES_HOST`, `POSTGRES_DB` env vars and construct the URL in code).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## HIGH Issues
|
||||||
|
|
||||||
|
### H-1: Rate Limiter Trusts X-Forwarded-For Without Validation
|
||||||
|
**File:** `app/core/rate_limit.py:43-45`
|
||||||
|
**Risk:** `get_client_ip()` blindly trusts the `X-Forwarded-For` header. An attacker can set arbitrary values to bypass rate limits on login, password reset, and other endpoints. Each request with a different spoofed IP creates a new rate-limit counter.
|
||||||
|
**Remediation:** Only trust `X-Forwarded-For` from known proxy IPs. Configure a trusted proxy list and validate the header chain. Use Starlette's `ProxyHeadersMiddleware` or validate against a `TRUSTED_PROXIES` env var.
|
||||||
|
|
||||||
|
### H-2: Duplicate `get_redis()` Functions — Connection Leak
|
||||||
|
**File:** `app/core/auth.py:53-66` and `app/core/auth.py:94-96`
|
||||||
|
**Risk:** Two `get_redis()` functions exist. The first (line 53) returns a singleton. The second (line 94) creates a **new Redis connection on every call**. Code importing `get_redis` may use either version. The middleware (line 69) creates its own Redis connection per request. This leads to connection pool exhaustion under load.
|
||||||
|
**Remediation:** Remove the second `get_redis()` (line 94-96). Ensure all code uses the singleton version. The middleware should use `get_redis()` from `app.core.auth` instead of creating its own connection.
|
||||||
|
|
||||||
|
### H-3: CSRF Middleware Creates New Redis Connection Per Request
|
||||||
|
**File:** `app/core/middleware.py:69-90`
|
||||||
|
**Risk:** For every unsafe HTTP request, the middleware creates a new `aioredis.from_url()` connection, uses it, then closes it. Under load, this creates thousands of connections and can exhaust Redis connection limits.
|
||||||
|
**Remediation:** Use the global Redis singleton via `from app.core.auth import get_redis`. Remove the per-request connection creation and the `finally: await redis.close()` block.
|
||||||
|
|
||||||
|
### H-4: CSRF Token Stored Plaintext in PostgreSQL
|
||||||
|
**File:** `app/core/auth.py:141` (`SessionModel` stores `csrf_token`)
|
||||||
|
**Risk:** The CSRF token is stored as plaintext in the PostgreSQL `sessions` table (audit trail). If the database is compromised, all active CSRF tokens are available for CSRF attacks.
|
||||||
|
**Remediation:** Store only a hash of the CSRF token in PostgreSQL (like `hash_token()` already exists for session tokens). Compare hashes during validation.
|
||||||
|
|
||||||
|
### H-5: No File Upload Validation in Storage Backend
|
||||||
|
**File:** `app/core/storage.py:69-128`
|
||||||
|
**Risk:** `LocalStorage` performs no validation on uploaded files:
|
||||||
|
- No path traversal protection: `os.path.join(self.base_path, path)` with a malicious `path` containing `../../` can write anywhere on the filesystem
|
||||||
|
- No file type/extension whitelist
|
||||||
|
- No file size limit
|
||||||
|
- No content-type validation
|
||||||
|
- `get_url()` returns the full filesystem path, leaking internal directory structure
|
||||||
|
**Remediation:** Sanitize `path` with `os.path.realpath()` and verify it's within `base_path`. Enforce file size limits, extension whitelist, and MIME type validation. Return relative paths from `get_url()`, not absolute filesystem paths.
|
||||||
|
|
||||||
|
### H-6: WebSocket Connections Lack Authentication Verification
|
||||||
|
**File:** `app/plugins/builtins/kommunikation/websocket_manager.py:23-28`, `app/plugins/builtins/ai_ui_control/websocket_manager.py:40-46`
|
||||||
|
**Risk:** Both WebSocket managers accept connections via `connect(websocket, user_id)` without verifying that `user_id` is authenticated. The security depends entirely on the calling route. If any WebSocket route passes an untrusted `user_id` (e.g., from query params), an attacker can impersonate any user. There is also no origin verification on WebSocket connections.
|
||||||
|
**Remediation:** Verify session cookie inside `connect()` before `websocket.accept()`. Validate the `Origin` header against allowed CORS origins. Add authentication middleware for WebSocket routes.
|
||||||
|
|
||||||
|
### H-7: In-Memory Rate Limiter in Error Endpoint — Fails with Multiple Workers
|
||||||
|
**File:** `app/routes/errors.py:21-40`
|
||||||
|
**Risk:** The error endpoint uses a process-local `defaultdict(deque)` for rate limiting. With multiple Uvicorn workers (common in production), each worker has its own counter. An attacker can make `RATE_LIMIT * num_workers` requests per minute.
|
||||||
|
**Remediation:** Use the Redis-based `check_rate_limit()` from `app/core/rate_limit.py` instead of the in-memory implementation.
|
||||||
|
|
||||||
|
### H-8: No CSRF Protection on WebSocket Connections
|
||||||
|
**File:** Both WebSocket managers
|
||||||
|
**Risk:** WebSocket connections are not protected against CSRF. A malicious site can open a WebSocket to the CRM backend via JavaScript `new WebSocket()` and send commands as the authenticated user (cookies are sent automatically with SameSite=Strict for same-site, but cross-site WebSocket hijacking is still possible if SameSite is configured differently or cookies are sent via `credentials`).
|
||||||
|
**Remediation:** Verify the `Origin` header on WebSocket upgrade requests. Reject connections from untrusted origins.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## MEDIUM Issues
|
||||||
|
|
||||||
|
### M-1: Login Response Leaks `is_system_admin` Flag
|
||||||
|
**File:** `app/routes/auth.py:78`
|
||||||
|
**Risk:** The login response includes `"is_system_admin": user.is_system_admin`. An attacker who compromises a session or intercepts the response knows whether the account has system-wide privileges, enabling targeted attacks.
|
||||||
|
**Remediation:** Do not include `is_system_admin` in the login response. The frontend can determine admin status via the `/me/permissions` endpoint.
|
||||||
|
|
||||||
|
### M-2: Weak Password Validation — No Complexity Requirements
|
||||||
|
**File:** `app/schemas/auth.py:10` (login: `min_length=1`), `app/schemas/user.py:11` (create: `min_length=8`)
|
||||||
|
**Risk:** Login accepts any password length (min_length=1). User creation requires min 8 chars but no complexity (uppercase, lowercase, digits, special chars). Users can set passwords like `aaaaaaaa`.
|
||||||
|
**Remediation:** Add password complexity validation (min 12 chars, mixed case, digits, special chars) for user creation and password reset. Keep login min_length=1 to avoid leaking whether the password was partially correct.
|
||||||
|
|
||||||
|
### M-3: F-String Interpolation of Table/Column Names in Raw SQL
|
||||||
|
**File:** `app/plugins/builtins/unified_search/embedding.py:194`, `search_engine.py:153`, `routes.py:294,300`, `jobs.py:183,228`
|
||||||
|
**Risk:** Multiple raw SQL queries use f-strings to interpolate table and column names: `f"UPDATE {table} SET ..."`, `f"SELECT {emb_col} FROM {table_name} ..."`. While the values come from hardcoded `table_map` dicts (not user input), this pattern is fragile — a future change could introduce user-controlled values into the map.
|
||||||
|
**Remediation:** Use SQLAlchemy ORM queries instead of raw SQL where possible. If raw SQL is needed, validate table/column names against an allowlist before interpolation, or use `sqlalchemy.sql.quoted_name` for safe identifier quoting.
|
||||||
|
|
||||||
|
### M-4: Forgejo Error Reporter Sends Full Context to External Service
|
||||||
|
**File:** `app/plugins/builtins/forgejo_error_reporter/service.py:196-199`
|
||||||
|
**Risk:** The error reporter serializes the entire `context` dict into the Forgejo issue body as JSON. If frontend error reporting includes sensitive data (user tokens, PII, tenant data), it will be written to an external Forgejo repository as a public issue.
|
||||||
|
**Remediation:** Add a field-level allowlist for context data. Strip or redact sensitive keys (tokens, passwords, emails, phone numbers). Consider making Forgejo issues private/confidential.
|
||||||
|
|
||||||
|
### M-5: Config Has Hardcoded Default Secret Key
|
||||||
|
**File:** `app/config.py:55`
|
||||||
|
**Risk:** The default `secret_key = "change-me-in-production-use-a-secure-random-string"` is a known public value. While production mode checks for it, development mode silently uses it. If dev environments are exposed (even temporarily), all signed tokens are forgeable.
|
||||||
|
**Remediation:** Remove the default value entirely. Make `secret_key` a required field with no default. Fail in all environments if not set.
|
||||||
|
|
||||||
|
### M-6: `LocalStorage.get_url()` Returns Absolute Filesystem Path
|
||||||
|
**File:** `app/core/storage.py:116-117`
|
||||||
|
**Risk:** `get_url()` returns `self._full_path(path)` which is the absolute filesystem path (e.g., `/data/uploads/tenant1/file.pdf`). If this URL is returned to the frontend or used in API responses, it leaks the internal directory structure and can aid path traversal attacks.
|
||||||
|
**Remediation:** Return a relative path or a signed download URL that routes through an authenticated API endpoint.
|
||||||
|
|
||||||
|
### M-7: Inconsistent Environment Configuration in `.env`
|
||||||
|
**File:** `.env:3,4`
|
||||||
|
**Risk:** `.env` sets `ENVIRONMENT=development` but `SESSION_COOKIE_SECURE=true`. In development with HTTP, secure cookies won't be sent, causing auth failures. More importantly, the `ENVIRONMENT=development` setting disables the production safety checks in `get_settings()`, allowing the default `SECRET_KEY` to be used.
|
||||||
|
**Remediation:** Use separate `.env.development` and `.env.production` files. Ensure development configs are never accidentally deployed.
|
||||||
|
|
||||||
|
### M-8: Permission Cache Falls Back to Stale Data on DB Error
|
||||||
|
**File:** `app/core/permissions.py:337-344`
|
||||||
|
**Risk:** When `_get_current_permission_version()` fails (DB error), the code sets `current_version = cached_version` and uses potentially stale cached permissions. If a user's permissions were revoked during the DB outage, they retain elevated access.
|
||||||
|
**Remediation:** On DB error, either fail closed (deny access) or use a shorter stale-while-error TTL. Log the event as a security incident.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## LOW Issues
|
||||||
|
|
||||||
|
### L-1: `document.write()` with DOM Clone in Print Utility
|
||||||
|
**File:** `frontend/src/utils/print.ts:54, 127`
|
||||||
|
**Risk:** `printElement()` and `exportToPDF()` use `document.write()` with `clone.outerHTML`. If the printed DOM element contains user-controlled content (e.g., contact notes with HTML), it executes in a new window context. The new window is same-origin, limiting the impact, but it's still an unnecessary risk.
|
||||||
|
**Remediation:** Use DOM APIs (`appendChild`, `importNode`) instead of `document.write()`. Alternatively, sanitize the cloned HTML before writing.
|
||||||
|
|
||||||
|
### L-2: Session Data Stored in Redis Without Encryption
|
||||||
|
**File:** `app/core/auth.py:130-134`
|
||||||
|
**Risk:** Session data (user_id, tenant_id, email, role, csrf_token, is_system_admin) is stored as plaintext JSON in Redis. Anyone with Redis access can read all active sessions.
|
||||||
|
**Remediation:** Encrypt session data before storing in Redis, or accept the risk given Redis should be network-isolated. At minimum, ensure Redis requires authentication and is not exposed.
|
||||||
|
|
||||||
|
### L-3: No Security Headers Middleware
|
||||||
|
**File:** No security headers middleware found
|
||||||
|
**Risk:** The application does not set security headers like `X-Content-Type-Options`, `X-Frame-Options`, `Strict-Transport-Security`, `Content-Security-Policy`.
|
||||||
|
**Remediation:** Add a security headers middleware or use `starlette-securehead`/`secure` package.
|
||||||
|
|
||||||
|
### L-4: No Origin Verification on WebSocket Upgrade
|
||||||
|
**File:** Both WebSocket managers
|
||||||
|
**Risk:** Neither WebSocket manager checks the `Origin` header before accepting connections. While cookies with `SameSite=Strict` provide some protection, some browsers and non-browser clients may not respect SameSite on WebSocket connections.
|
||||||
|
**Remediation:** Check `websocket.headers.get("origin")` against `settings.cors_origin_list` before calling `websocket.accept()`.
|
||||||
|
|
||||||
|
### L-5: Unbounded Feedback/Command Storage in AI UI Control WebSocket
|
||||||
|
**File:** `app/plugins/builtins/ai_ui_control/websocket_manager.py:94-103`
|
||||||
|
**Risk:** `store_feedback()` stores feedback dicts without size limits. `cleanup_stale()` only runs when explicitly called. An attacker who can send WebSocket messages could fill memory with large feedback payloads.
|
||||||
|
**Remediation:** Add size limits on feedback payloads. Run `cleanup_stale()` on a timer or on each `connect()`/`disconnect()`.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Positive Findings
|
||||||
|
|
||||||
|
1. **Dockerfile security:** Multi-stage build, non-root user (`appuser` UID 1000), healthcheck configured, no secrets baked into image.
|
||||||
|
2. **RLS implementation:** PostgreSQL Row Level Security with `FORCE` (migration 0028) ensures tenant isolation even for table owners. `set_tenant_context()` uses parameterized queries.
|
||||||
|
3. **Password hashing:** bcrypt with configurable rounds (default 12).
|
||||||
|
4. **Session tokens:** `secrets.token_urlsafe(32)` — cryptographically secure.
|
||||||
|
5. **XSS protection:** `HtmlBlock.tsx` and `SignatureManager.tsx` use `DOMPurify.sanitize()` before `dangerouslySetInnerHTML`.
|
||||||
|
6. **RBAC architecture:** Deny-list takes precedence over allow-list. Field-level permissions with strictest-wins merging. Permission version-based cache invalidation.
|
||||||
|
7. **No user enumeration:** Password reset endpoint always returns 200.
|
||||||
|
8. **SQL injection:** ORM queries use parameterized statements throughout. Raw SQL in `unified_search` uses hardcoded maps (not directly exploitable).
|
||||||
|
9. **`.gitignore`** properly covers `.env`, `.env.*`, and excludes example files.
|
||||||
|
10. **Production safety checks** in `get_settings()` validate `SECRET_KEY`, `SESSION_COOKIE_SECURE`, and `STORAGE_PATH`.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Migration & Data Loss Risks
|
||||||
|
|
||||||
|
1. **RLS policies:** Multiple migrations (0001, 0002, 0004, 0015, 0021, 0028) create and modify RLS policies. Migration 0028 adds `FORCE ROW LEVEL SECURITY`. Ensure all migrations are applied in order before production deployment.
|
||||||
|
2. **Backup risk:** No backup/restore procedure found in the repository. The `last_backup_at` system setting is referenced in automation jobs but no backup script exists.
|
||||||
|
3. **Volume persistence:** `docker-compose.yml` defines named volumes for `pgdata`, `redisdata`, and `storage`. Good for persistence, but no backup strategy documented.
|
||||||
|
4. **Migration rollback:** Down migrations exist but should be tested. RLS policy down migrations disable RLS — running a rollback in production would expose all tenant data.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Remediation Priority
|
||||||
|
|
||||||
|
1. **Immediate (before any production deploy):** C-1, C-2, C-3, C-4, C-5, H-1, H-2, H-3
|
||||||
|
2. **Short-term (within 1 sprint):** H-4, H-5, H-6, H-7, H-8, M-1, M-2, M-5
|
||||||
|
3. **Medium-term (within 2 sprints):** M-3, M-4, M-6, M-7, M-8, L-1, L-2, L-3, L-4, L-5
|
||||||
+33
-8
@@ -15,23 +15,48 @@ POSTGRES_USER=crm_user
|
|||||||
POSTGRES_PASSWORD=STRONG_PASSWORD_HERE
|
POSTGRES_PASSWORD=STRONG_PASSWORD_HERE
|
||||||
POSTGRES_DB=crm_db
|
POSTGRES_DB=crm_db
|
||||||
|
|
||||||
# --- CRM Application ----------------------------------------------------------
|
# --- Redis (REQUIRED) ---------------------------------------------------------
|
||||||
# The host "postgres" is the docker-compose service name (internal DNS).
|
# Generate a strong password:
|
||||||
# The DRIVER is asyncpg for production PostgreSQL.
|
# python -c "import secrets; print(secrets.token_urlsafe(24))"
|
||||||
DATABASE_URL=postgresql+asyncpg://crm_user:STRONG_PASSWORD_HERE@postgres:5432/crm_db
|
REDIS_PASSWORD=STRONG_REDIS_PASSWORD_HERE
|
||||||
|
|
||||||
# --- AUTH_SECRET (REQUIRED, min 32 chars) ------------------------------------
|
# --- CRM Application: Runtime DB user (NOSUPERUSER, NOBYPASSRLS) --------------
|
||||||
|
# The app and worker use crm_runtime — RLS is enforced.
|
||||||
|
# This user is created by migration 0044 with DML-only permissions.
|
||||||
|
# Set RUNTIME_DB_PASSWORD to the password you want for crm_runtime.
|
||||||
|
RUNTIME_DB_PASSWORD=STRONG_RUNTIME_PASSWORD_HERE
|
||||||
|
DATABASE_URL=postgresql+asyncpg://crm_runtime:STRONG_RUNTIME_PASSWORD_HERE@postgres:5432/crm_db
|
||||||
|
|
||||||
|
# --- CRM Application: Migration DB user (owner, can run DDL) -----------------
|
||||||
|
# Migrations and DDL operations use the owner user (crm_user).
|
||||||
|
# This is NOT used by the app at runtime — only by prestart.sh / alembic.
|
||||||
|
MIGRATION_DATABASE_URL=postgresql+asyncpg://crm_user:STRONG_PASSWORD_HERE@postgres:5432/crm_db
|
||||||
|
|
||||||
|
# --- SECRET_KEY (REQUIRED, min 32 chars) -------------------------------------
|
||||||
# Session signing secret. MUST be at least 32 characters.
|
# Session signing secret. MUST be at least 32 characters.
|
||||||
# Generate with:
|
# Generate with:
|
||||||
# python -c "import secrets; print(secrets.token_urlsafe(48))"
|
# python -c "import secrets; print(secrets.token_urlsafe(48))"
|
||||||
AUTH_SECRET=MIN_32_CHARS_GENERATE_WITH_secrets_token_urlsafe_32_xxxxxxxxxxxx
|
SECRET_KEY=MIN_32_CHARS_GENERATE_WITH_secrets_token_urlsafe_32_xxxxxxxxxxxx
|
||||||
|
|
||||||
|
# --- Frontend URL (for email links) ------------------------------------------
|
||||||
|
# The public URL where users access the LeoCRM frontend.
|
||||||
|
# Used for password reset links, invitations, etc.
|
||||||
|
FRONTEND_URL=https://crm.example.com
|
||||||
|
|
||||||
# --- CORS / environment -------------------------------------------------------
|
# --- CORS / environment -------------------------------------------------------
|
||||||
# Comma-separated, NO wildcards. In dev we allow localhost:8000 (the app) and
|
# Comma-separated, NO wildcards. In dev we allow localhost:8000 (the app) and
|
||||||
# :5173 (e.g. Vite dev server). In production, restrict to the real domain.
|
# :5173 (e.g. Vite dev server). In production, restrict to the real domain.
|
||||||
CORS_ORIGINS=http://localhost:8000,http://localhost:5173
|
CORS_ORIGINS=https://crm.example.com
|
||||||
ENVIRONMENT=production
|
ENVIRONMENT=production
|
||||||
LOG_LEVEL=INFO
|
LOG_LEVEL=INFO
|
||||||
|
|
||||||
# --- bcrypt tuning (keep aligned with .env.example) --------------------------
|
# --- SMTP (for password reset emails) -----------------------------------------
|
||||||
|
SMTP_HOST=smtp.example.com
|
||||||
|
SMTP_PORT=587
|
||||||
|
SMTP_USERNAME=noreply@example.com
|
||||||
|
SMTP_PASSWORD=YOUR_SMTP_PASSWORD
|
||||||
|
SMTP_FROM_EMAIL=noreply@example.com
|
||||||
|
SMTP_USE_TLS=true
|
||||||
|
|
||||||
|
# --- bcrypt tuning ----------------------------------------------------------
|
||||||
BCRYPT_ROUNDS=12
|
BCRYPT_ROUNDS=12
|
||||||
|
|||||||
@@ -30,6 +30,10 @@ coverage.xml
|
|||||||
.mypy_cache/
|
.mypy_cache/
|
||||||
.ruff_cache/
|
.ruff_cache/
|
||||||
|
|
||||||
|
# Redis dumps
|
||||||
|
dump.rdb
|
||||||
|
*.rdb
|
||||||
|
|
||||||
# Database files
|
# Database files
|
||||||
*.db
|
*.db
|
||||||
*.db-journal
|
*.db-journal
|
||||||
|
|||||||
+2
-5
@@ -116,8 +116,7 @@ In **crm-app → Environment Variables**, set:
|
|||||||
| `ENVIRONMENT` | `production` | |
|
| `ENVIRONMENT` | `production` | |
|
||||||
| `LOG_LEVEL` | `INFO` | `DEBUG` only temporarily. |
|
| `LOG_LEVEL` | `INFO` | `DEBUG` only temporarily. |
|
||||||
| `BCRYPT_ROUNDS` | `12` | Aligned with `.env.example`. |
|
| `BCRYPT_ROUNDS` | `12` | Aligned with `.env.example`. |
|
||||||
| `JWT_ALGORITHM` | `HS256` | Aligned with `.env.example`. |
|
|
||||||
| `JWT_EXPIRY_HOURS` | `24` | Aligned with `.env.example`. |
|
|
||||||
|
|
||||||
### Secret generation (run once, locally)
|
### Secret generation (run once, locally)
|
||||||
|
|
||||||
@@ -144,9 +143,7 @@ are still rendered in the UI to anyone with read access to the environment.
|
|||||||
> {"key":"CORS_ORIGINS", "value":"https://crm.media-on.de:443"},
|
> {"key":"CORS_ORIGINS", "value":"https://crm.media-on.de:443"},
|
||||||
> {"key":"ENVIRONMENT", "value":"production"},
|
> {"key":"ENVIRONMENT", "value":"production"},
|
||||||
> {"key":"LOG_LEVEL", "value":"INFO"},
|
> {"key":"LOG_LEVEL", "value":"INFO"},
|
||||||
> {"key":"BCRYPT_ROUNDS", "value":"12"},
|
> {"key":"BCRYPT_ROUNDS", "value":"12"}
|
||||||
> {"key":"JWT_ALGORITHM", "value":"HS256"},
|
|
||||||
> {"key":"JWT_EXPIRY_HOURS", "value":"24"}
|
|
||||||
> ]
|
> ]
|
||||||
> }'
|
> }'
|
||||||
> ```
|
> ```
|
||||||
|
|||||||
@@ -119,3 +119,54 @@ python scripts/deploy.py --migrate-only
|
|||||||
```bash
|
```bash
|
||||||
python scripts/deploy.py --skip-build # startet Worker automatisch
|
python scripts/deploy.py --skip-build # startet Worker automatisch
|
||||||
```
|
```
|
||||||
|
|
||||||
|
## Backup & Restore
|
||||||
|
|
||||||
|
### Backup (PostgreSQL)
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Full DB backup (run on the host or via docker exec)
|
||||||
|
docker exec crm-postgres pg_dump -U crm_user -Fc crm_db > backup_$(date +%Y%m%d_%H%M%S).dump
|
||||||
|
|
||||||
|
# Backup mit Custom-Format (komprimiert, parallel restore-fähig)
|
||||||
|
docker exec crm-postgres pg_dump -U crm_user -Fc -Z 9 crm_db > backup_$(date +%Y%m%d).dump
|
||||||
|
```
|
||||||
|
|
||||||
|
### Backup (Redis — Sessions/Queues)
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Redis RDB Snapshot
|
||||||
|
docker exec crm-redis redis-cli -a "$REDIS_PASSWORD" SAVE
|
||||||
|
docker cp crm-redis:/data/dump.rdb redis_backup_$(date +%Y%m%d).rdb
|
||||||
|
```
|
||||||
|
|
||||||
|
### Backup (File Storage)
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Local storage volume
|
||||||
|
docker run --rm -v leocrm-fix_storage:/data -v $(pwd):/backup alpine \
|
||||||
|
tar czf /backup/storage_$(date +%Y%m%d).tar.gz /data
|
||||||
|
```
|
||||||
|
|
||||||
|
### Restore (PostgreSQL)
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Stop app containers
|
||||||
|
docker compose stop crm-app crm-worker
|
||||||
|
|
||||||
|
# Restore DB
|
||||||
|
docker exec -i crm-postgres pg_restore -U crm_user -d crm_db --clean < backup_20260726.dump
|
||||||
|
|
||||||
|
# Restart app
|
||||||
|
docker compose start crm-app crm-worker
|
||||||
|
```
|
||||||
|
|
||||||
|
### Automatisierte Backups (Cron)
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# /etc/cron.d/leocrm-backup
|
||||||
|
0 2 * * * root docker exec crm-postgres pg_dump -U crm_user -Fc crm_db > /backups/leocrm_$(date +\%Y\%m\%d).dump
|
||||||
|
0 3 * * * root find /backups -name 'leocrm_*.dump' -mtime +30 -delete
|
||||||
|
```
|
||||||
|
|
||||||
|
**Empfehlung:** Tägliche DB-Backups, 30 Tage Aufbewahrung. Storage-Backup wöchentlich.
|
||||||
|
|||||||
+290
@@ -0,0 +1,290 @@
|
|||||||
|
# LeoCRM Fix-Plan V2 — Gründliche Analyse & Maßnahmen
|
||||||
|
|
||||||
|
*Erstellt: 2026-07-26 — basierend auf externem Audit + eigener Code-Verifikation*
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Zusammenfassung
|
||||||
|
|
||||||
|
Von 16 zentralen Punkten des externen Audits wurden **alle 16 durch Code-Inspektion verifiziert**. Zusätzlich wurden **5 neue Probleme** gefunden (UploadFile-Bug, Redis-Default-Passwort, exponierte Ports, unauthentifizierter Error-Endpoint, fehlende Security-Headers).
|
||||||
|
|
||||||
|
**Gesamtstatus:** 4 sauber gefixt · 8 teilweise gefixt · 4 nicht gefixt · 5 neu gefunden = **21 Maßnahmen**
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Phase 1: Kritische Release-Blocker (vor Produktivbetrieb)
|
||||||
|
|
||||||
|
### B1. Doppelte `get_redis()` entfernen
|
||||||
|
- **Datei:** `app/core/auth.py` Zeilen 53 + 94
|
||||||
|
- **Problem:** Zweite Definition überschreibt Singleton, erzeugt pro Aufruf neue Verbindung → Connection Leak
|
||||||
|
- **Fix:** Zweite `def get_redis()` (Zeile 94) löschen. Erste Definition (Zeile 53) beibehalten.
|
||||||
|
- **Aufwand:** 5 Min
|
||||||
|
- **Risiko:** Keines — erste Definition ist korrekt
|
||||||
|
|
||||||
|
### B2. Plugin-Routen-Registrierung reparieren
|
||||||
|
- **Datei:** `app/main.py` Zeilen 375-416
|
||||||
|
- **Problem:** Alle Plugin-Routen werden statisch in `create_app()` registriert, unabhängig vom Aktivierungsstatus. Deaktivierte Plugins bleiben erreichbar. Kommentar in Zeile 416 sagt das Gegenteil.
|
||||||
|
- **Fix:**
|
||||||
|
1. Statische Registrierung aus `create_app()` entfernen
|
||||||
|
2. In `lifespan()` nur Routen für `active=True` Plugins registrieren
|
||||||
|
3. `Depends(require_active_plugin("name"))` als zentrale Prüfung ergänzen
|
||||||
|
4. Bei Deaktivierung: Router entfernen oder 403-Dependency ergänzen
|
||||||
|
- **Aufwand:** 2-3 Std
|
||||||
|
- **Risiko:** Mittel — muss sicherstellen dass keine Route doppelt registriert wird
|
||||||
|
|
||||||
|
### B3. UploadFile Route-Registration Bug
|
||||||
|
- **Dateien:** `app/plugins/builtins/dms/routes.py`, `calendar/routes.py`, `mail/routes.py`, `kommunikation/routes.py`, `ai_assistant/routes.py`
|
||||||
|
- **Problem:** FastAPI kann `UploadFile` nicht als Response-Model auflösen → 5 Plugins failen beim Registrieren mit `Invalid args for response field`
|
||||||
|
- **Fix:** `response_model=None` zu allen Endpoints mit `UploadFile`-Rückgabe hinzufügen, oder Return-Type auf `Response`/`dict` ändern
|
||||||
|
- **Aufwand:** 30 Min
|
||||||
|
- **Risiko:** Keines — Routen sind aktuell gar nicht registriert
|
||||||
|
|
||||||
|
### B4. DMS-Upload auf echtes Streaming umstellen
|
||||||
|
- **Datei:** `app/plugins/builtins/dms/routes.py` Zeilen 444-472
|
||||||
|
- **Problem:** Chunks werden in `list[bytes]` gesammelt, dann `b"".join()` → 100MB Datei = 200MB+ RAM. `save_stream()` existiert aber wird nicht benutzt.
|
||||||
|
- **Fix:**
|
||||||
|
```python
|
||||||
|
async def chunk_generator():
|
||||||
|
while chunk := await file.read(CHUNK_SIZE):
|
||||||
|
yield chunk
|
||||||
|
await storage.save_stream(storage_path, chunk_generator())
|
||||||
|
```
|
||||||
|
Hash und Größe während des Streams berechnen.
|
||||||
|
- **Aufwand:** 1 Std
|
||||||
|
- **Risiko:** Gering — save_stream() ist bereits implementiert
|
||||||
|
|
||||||
|
### B5. Outbox-Worker: Event-Handler registrieren
|
||||||
|
- **Datei:** `app/core/worker.py` `on_startup()`
|
||||||
|
- **Problem:** Worker liest Events aus Outbox, published an lokalen EventBus, aber es sind keine Handler registriert → Events werden als `published` markiert ohne Verarbeitung
|
||||||
|
- **Fix:**
|
||||||
|
1. In `on_startup()`: Plugin-Event-Handler registrieren (wie in `lifespan()` der API)
|
||||||
|
2. `webhook_dispatcher._dispatch_event` an EventBus subscriben
|
||||||
|
3. Plugin-Participant-Handler registrieren
|
||||||
|
- **Aufwand:** 2 Std
|
||||||
|
- **Risiko:** Mittel — muss gleiche Handler wie API-Container registrieren
|
||||||
|
|
||||||
|
### B6. Passwort-Reset-Mailjob implementieren
|
||||||
|
- **Dateien:** `app/services/auth_service.py`, `app/core/jobs.py`, `app/core/job_registry.py`
|
||||||
|
- **Problem:** `send_password_reset_email` Job wird gequeued aber nie registriert → Mail wird nicht versendet. Token wird in Logs geschrieben (Zeile 240-241).
|
||||||
|
- **Fix:**
|
||||||
|
1. `send_password_reset_email` Worker-Funktion implementieren (SMTP/IMAP)
|
||||||
|
2. Mit `register_job()` registrieren
|
||||||
|
3. `logger.warning("raw_token for development: %s", raw_token)` entfernen
|
||||||
|
4. Token nur im Development-Mode loggen, nie in Production
|
||||||
|
- **Aufwand:** 2 Std
|
||||||
|
- **Risiko:** Gering
|
||||||
|
|
||||||
|
### B7. Webhook SSRF-Schutz + Secret-Behandlung
|
||||||
|
- **Dateien:** `app/services/webhook_service.py`, `app/schemas/webhook.py`
|
||||||
|
- **Problem:** Kein SSRF-Schutz — User können interne Dienste ansprechen (redis:6379, postgres:5432, 169.254.169.254). Webhook-Secret wird im Response zurückgegeben.
|
||||||
|
- **Fix:**
|
||||||
|
1. SSRF-Prüfung: DNS auflösen, private IPs blocken (10.x, 172.16-31.x, 192.168.x, 127.x, 169.254.x, ::1)
|
||||||
|
2. Redirects deaktivieren oder prüfen
|
||||||
|
3. Protokoll-Allowlist (nur https)
|
||||||
|
4. `secret` aus `WebhookResponse` entfernen
|
||||||
|
5. Secret gehasht in DB speichern
|
||||||
|
- **Aufwand:** 3 Std
|
||||||
|
- **Risiko:** Gering
|
||||||
|
|
||||||
|
### B8. RLS: Separater DB-Runtime-User
|
||||||
|
- **Dateien:** `docker-compose.yml`, `alembic/versions/0044_db_roles.py` (neu)
|
||||||
|
- **Problem:** `POSTGRES_USER` (crm_user) ist Superuser → umgeht RLS auch mit FORCE. Spätere Tabellen (user_preferences, saved_filters, etc.) haben keine RLS-Policy.
|
||||||
|
- **Fix:**
|
||||||
|
1. Neue Migration `0044_db_roles.py`: erstellt `crm_runtime` (NOSUPERUSER, NOBYPASSRLS)
|
||||||
|
2. `crm_runtime` bekommt nur SELECT/INSERT/UPDATE/DELETE Rechte
|
||||||
|
3. `docker-compose.yml`: API und Worker nutzen `crm_runtime`, Migrationen nutzen `crm_owner`
|
||||||
|
4. Neue Migration `0045_rls_new_tables.py`: RLS für alle Tabellen mit `tenant_id` die nach 0028 hinzukamen
|
||||||
|
- **Aufwand:** 4 Std
|
||||||
|
- **Risiko:** Hoch — muss bestehende Datenbanken migrieren ohne Datenverlust
|
||||||
|
|
||||||
|
### B9. .env.docker.example korrigieren
|
||||||
|
- **Datei:** `.env.docker.example`
|
||||||
|
- **Problem:** Verwendet `AUTH_SECRET` statt `SECRET_KEY` (config.py erwartet `SECRET_KEY`)
|
||||||
|
- **Fix:** `AUTH_SECRET` → `SECRET_KEY` umbenennen
|
||||||
|
- **Aufwand:** 5 Min
|
||||||
|
- **Risiko:** Keines
|
||||||
|
|
||||||
|
### B10. Redis-Default-Passwort + exponierte Ports
|
||||||
|
- **Datei:** `docker-compose.yml`
|
||||||
|
- **Problem:** Redis-Passwort default `changeme`, PostgreSQL (5432) und Redis (6379) Ports exponiert
|
||||||
|
- **Fix:**
|
||||||
|
1. Redis-Passwort als Required-Env ohne Default
|
||||||
|
2. `ports:` Sektion für DB und Redis entfernen (nur internes Docker-Netzwerk)
|
||||||
|
3. Falls Debug-Zugriff nötig: nur an 127.0.0.1 binden
|
||||||
|
- **Aufwand:** 15 Min
|
||||||
|
- **Risiko:** Gering — bestehende Setups müssen .env anpassen
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Phase 2: Hohe Priorität (kurz nach Release)
|
||||||
|
|
||||||
|
### H1. Unauthentifizierter Error-Endpoint absichern
|
||||||
|
- **Datei:** `app/routes/errors.py`
|
||||||
|
- **Problem:** `POST /api/v1/errors` ohne Auth, sendet Daten an Forgejo als öffentliches Issue. Context-Dict kann sensible Daten enthalten.
|
||||||
|
- **Fix:**
|
||||||
|
1. Context-Felder filtern (keine Tokens, Passwörter, Headers)
|
||||||
|
2. Forgejo-Issues nur in non-production erstellen
|
||||||
|
3. Rate-Limit auf IP-Basis (bereits vorhanden, aber in-memory → bei Multi-Worker unzuverlässig)
|
||||||
|
4. Optional: Auth erforderlich, aber dann funktioniert Frontend-Error-Logging nicht mehr → besser: nur sanitisierte Daten akzeptieren
|
||||||
|
- **Aufwand:** 1 Std
|
||||||
|
|
||||||
|
### H2. Rate-Limiter IP-Spoofing
|
||||||
|
- **Datei:** `app/core/rate_limit.py` Zeile 43
|
||||||
|
- **Problem:** Vertraut `X-Forwarded-For` blind → IP-Spoofing umgeht Rate-Limits
|
||||||
|
- **Fix:** Nur erste IP in X-Forwarded-For verwenden, oder `X-Real-IP` mit Proxy-Validation
|
||||||
|
- **Aufwand:** 30 Min
|
||||||
|
|
||||||
|
### H3. CSRF-Middleware Redis-Verbindung
|
||||||
|
- **Datei:** `app/core/middleware.py` Zeile 69
|
||||||
|
- **Problem:** Erstellt pro unsafe Request neue Redis-Verbindung → Connection Leak
|
||||||
|
- **Fix:** `get_redis()` Singleton verwenden (funktioniert nach B1)
|
||||||
|
- **Aufwand:** 10 Min
|
||||||
|
|
||||||
|
### H4. WebSocket Auth + Origin-Verifikation
|
||||||
|
- **Dateien:** `app/plugins/builtins/kommunikation/websocket_manager.py`, `ai_ui_control/websocket_manager.py`
|
||||||
|
- **Problem:** `user_id` wird ohne Auth-Verifikation akzeptiert. Keine Origin-Prüfung bei WS-Upgrade.
|
||||||
|
- **Fix:**
|
||||||
|
1. Session-Token aus Query-Param oder Header validieren
|
||||||
|
2. Origin-Header gegen erlaubte Domains prüfen
|
||||||
|
3. User-ID aus Session ableiten, nicht aus Client-Param
|
||||||
|
- **Aufwand:** 2 Std
|
||||||
|
|
||||||
|
### H5. File-Upload-Sicherheit
|
||||||
|
- **Datei:** `app/core/storage.py`
|
||||||
|
- **Problem:** Keine Path-Traversal-Prüfung, keine Type/Size-Limits, `get_url()` leakt Filesystem-Pfade
|
||||||
|
- **Fix:**
|
||||||
|
1. Filename sanitizen (keine `../`, keine absoluten Pfade)
|
||||||
|
2. MIME-Type-Allowlist
|
||||||
|
3. Max-File-Size konfigurierbar
|
||||||
|
4. `get_url()` gibt relative URL zurück, nicht Filesystem-Pfad
|
||||||
|
- **Aufwand:** 1 Std
|
||||||
|
|
||||||
|
### H6. Security-Headers
|
||||||
|
- **Datei:** `app/core/middleware.py` (neu)
|
||||||
|
- **Problem:** Keine Security-Headers (HSTS, X-Content-Type-Options, X-Frame-Options, CSP)
|
||||||
|
- **Fix:** Middleware ergänzen die diese Headers setzt
|
||||||
|
- **Aufwand:** 30 Min
|
||||||
|
|
||||||
|
### H7. Migration-Repair für bestehende Installationen
|
||||||
|
- **Datei:** `alembic/versions/0044_repair_contact_migration.py` (neu)
|
||||||
|
- **Problem:** Migrationen 0021 und 0027 wurden nachträglich geändert. Alembic führt sie nicht erneut aus.
|
||||||
|
- **Fix:**
|
||||||
|
1. Neue Migration die `*_old` Tabellen erkennt und Daten nachmigriert
|
||||||
|
2. Integritätsprüfung (Anzahl vergleichen)
|
||||||
|
3. Bei Abweichungen hart abbrechen mit Fehlermeldung
|
||||||
|
- **Aufwand:** 3 Std
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Phase 3: Mittlere Priorität
|
||||||
|
|
||||||
|
### M1. Passwort-Komplexität
|
||||||
|
- **Datei:** `app/schemas/auth.py`, `app/schemas/user.py`
|
||||||
|
- **Problem:** Min-Length 8 bei Erstellung, Min-Length 1 bei Login. Keine Komplexitäts-Requirements.
|
||||||
|
- **Fix:** Passwort-Validator ergänzen (min 8 Zeichen, 1 Groß, 1 Klein, 1 Zahl)
|
||||||
|
- **Aufwand:** 30 Min
|
||||||
|
|
||||||
|
### M2. Login-Response: is_system_admin
|
||||||
|
- **Datei:** `app/routes/auth.py` Zeile 78
|
||||||
|
- **Problem:** `is_system_admin` Flag in Login-Response leakt interne Rolle
|
||||||
|
- **Fix:** Flag aus Response entfernen oder nur für Admin-User anzeigen
|
||||||
|
- **Aufwand:** 15 Min
|
||||||
|
|
||||||
|
### M3. Permission-Cache: Stale Data bei DB-Error
|
||||||
|
- **Datei:** `app/core/permissions.py` Zeile 337
|
||||||
|
- **Problem:** Bei DB-Error fällt Cache auf stale Daten zurück → widerrufene Rechte bleiben aktiv
|
||||||
|
- **Fix:** Bei DB-Error: Cache invalidieren und 503 zurückgeben statt stale Daten zu nutzen
|
||||||
|
- **Aufwand:** 30 Min
|
||||||
|
|
||||||
|
### M4. ENVIRONMENT=development vs SESSION_COOKIE_SECURE=true
|
||||||
|
- **Datei:** `.env` Zeilen 3-4
|
||||||
|
- **Problem:** Inkonsistent — development deaktiviert Prod-Safety-Checks, aber Cookie ist secure
|
||||||
|
- **Fix:** In .env.docker.example klar dokumentieren: production → `ENVIRONMENT=production` + `SESSION_COOKIE_SECURE=true`
|
||||||
|
- **Aufwand:** 10 Min
|
||||||
|
|
||||||
|
### M5. Frontend: Unresolved Items
|
||||||
|
- **Dateien:** `WelcomeDialog.tsx`, `SavedFilterBar.tsx`, `EntityHistoryPanel.tsx`, `TagBadge.tsx`, `TagSelector.tsx`
|
||||||
|
- **Problem:** WelcomeDialog hat `open={false}`. SavedFilterBar/EntityHistoryPanel/TagBadge/TagSelector sind gebaut aber nicht in Seiten integriert.
|
||||||
|
- **Fix:**
|
||||||
|
1. WelcomeDialog an User-Preferences (onboarding_completed) koppeln
|
||||||
|
2. SavedFilterBar in ContactsList, Mail, Calendar integrieren
|
||||||
|
3. EntityHistoryPanel in ContactDetail, Settings integrieren
|
||||||
|
4. TagBadge/TagSelector in ContactsList, Mail, Calendar integrieren
|
||||||
|
- **Aufwand:** 4 Std
|
||||||
|
|
||||||
|
### M6. Frontend-Tests: QueryClientProvider
|
||||||
|
- **Datei:** `frontend/src/test/setup.ts` oder einzelne Tests
|
||||||
|
- **Problem:** ~29 Tests failen mit missing QueryClientProvider
|
||||||
|
- **Fix:** Globalen Test-Wrapper mit QueryClientProvider in setup.ts ergänzen
|
||||||
|
- **Aufwand:** 1 Std
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Phase 4: Niedrige Priorität
|
||||||
|
|
||||||
|
### L1. document.write() in print.ts
|
||||||
|
- **Datei:** `frontend/src/utils/print.ts` Zeilen 54, 127
|
||||||
|
- **Problem:** `document.write()` mit DOM-Clone — XSS-Risiko wenn Content nicht sanitized
|
||||||
|
- **Fix:** Statt `document.write()`: `iframe.srcdoc` oder `Blob URL` verwenden
|
||||||
|
- **Aufwand:** 1 Std
|
||||||
|
|
||||||
|
### L2. AI UI Control: Unbounded Feedback-Storage
|
||||||
|
- **Datei:** `app/plugins/builtins/ai_ui_control/websocket_manager.py` Zeile 94
|
||||||
|
- **Problem:** Feedback/Commands unbegrenzt im Memory gespeichert → Memory Exhaustion
|
||||||
|
- **Fix:** Max-Length Queue (z.B. 100 Einträge) mit FIFO
|
||||||
|
- **Aufwand:** 15 Min
|
||||||
|
|
||||||
|
### L3. Backup-Strategie dokumentieren
|
||||||
|
- **Problem:** Named Volumes in docker-compose aber keine Backup/Restore-Doku
|
||||||
|
- **Fix:** Backup-Script und Doku ergänzen
|
||||||
|
- **Aufwand:** 2 Std
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Implementierungs-Reihenfolge
|
||||||
|
|
||||||
|
```
|
||||||
|
Phase 1 (Release-Blocker):
|
||||||
|
B1 → B3 → B9 → B10 → B2 → B4 → B5 → B6 → B7 → B8
|
||||||
|
↑ ↑ ↑ ↑ ↑ ↑ ↑ ↑ ↑ ↑
|
||||||
|
5m 30m 5m 15m 3h 1h 2h 2h 3h 4h
|
||||||
|
Gesamt: ~16 Std
|
||||||
|
|
||||||
|
Phase 2 (Hohe Priorität):
|
||||||
|
H3 → H2 → H6 → H1 → H5 → H4 → H7
|
||||||
|
Gesamt: ~8 Std
|
||||||
|
|
||||||
|
Phase 3 (Mittlere Priorität):
|
||||||
|
M4 → M1 → M2 → M3 → M6 → M5
|
||||||
|
Gesamt: ~6 Std
|
||||||
|
|
||||||
|
Phase 4 (Niedrige Priorität):
|
||||||
|
L2 → L1 → L3
|
||||||
|
Gesamt: ~3 Std
|
||||||
|
```
|
||||||
|
|
||||||
|
**Gesamtaufwand: ~33 Std**
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Was bereits sauber funktioniert
|
||||||
|
|
||||||
|
- ✅ Auth-Bypass entfernt (keine X-Internal-Call/X-Tenant-Id/X-User-Id Headers mehr)
|
||||||
|
- ✅ Plugin-Upload/URL-Installation deaktiviert (403)
|
||||||
|
- ✅ Worker in separatem Container
|
||||||
|
- ✅ Metrics adminbeschränkt
|
||||||
|
- ✅ DOMPurify für HTML-Komponenten
|
||||||
|
- ✅ ARQ-Verbindungspool zentralisiert
|
||||||
|
- ✅ Session-Widerruf nach Passwortänderung
|
||||||
|
- ✅ Permission-Cache-Versionierung
|
||||||
|
- ✅ Redis SCAN statt KEYS
|
||||||
|
- ✅ Rabatte von Float auf Numeric
|
||||||
|
- ✅ Event-Outbox als Grundlage vorhanden
|
||||||
|
- ✅ RLS FORCE + WITH CHECK in Migration 0028
|
||||||
|
- ✅ Migration 0021: Tabellen umbenennen statt löschen
|
||||||
|
- ✅ Frontend: TypeScript typecheck clean (0 errors)
|
||||||
|
- ✅ Frontend: ErrorBoundary, OfflineBanner, ErrorLogger implementiert
|
||||||
|
- ✅ Frontend: Print/PDF mit WeasyPrint funktioniert
|
||||||
|
- ✅ Dockerfile: Multi-stage, non-root User, Healthcheck
|
||||||
|
- ✅ Bcrypt Password-Hashing
|
||||||
|
- ✅ Session-Tokens: secrets.token_urlsafe(32)
|
||||||
+38
-471
@@ -1,436 +1,62 @@
|
|||||||
# LeoCRM — Umfassender Fix-Plan
|
# LeoCRM — Umfassender Fix-Plan
|
||||||
|
|
||||||
> Erstellt: 2026-07-25
|
> Erstellt: 2026-07-25
|
||||||
|
> Letzte Überprüfung: 2026-07-26 — Alle Items gegen Codebasis verifiziert
|
||||||
> Quellen: Externes Audit (geprüft), eigene Code-Inspektion, Coolify-Deployment-Prüfung
|
> Quellen: Externes Audit (geprüft), eigene Code-Inspektion, Coolify-Deployment-Prüfung
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## P0 — Sofort blockierend (vor jeder Nutzung)
|
## ✅ Erledigte Fixes (22 von 24 Items komplett)
|
||||||
|
|
||||||
### P0-1: Authentifizierungs-Bypass entfernen
|
Die folgenden Items wurden bei der Überprüfung am 2026-07-26 als erledigt bestätigt:
|
||||||
|
|
||||||
**Problem:** `app/deps.py` akzeptiert `X-Internal-Call: true` mit `X-Tenant-Id` und `X-User-Id` Headern. Keine Signatur, kein Token, keine IP-Beschränkung. `except (ValueError, Exception): pass` verschleiert Fehler.
|
| Item | Beschreibung | Verifiziert durch |
|
||||||
|
|---|---|---|
|
||||||
**Datei:** `app/deps.py:37-58`
|
| P0-1 | Auth-Bypass entfernt | `app/deps.py` — keine `X-Internal-Call` Headers mehr |
|
||||||
|
| P0-2 | Migrationen repariert | `migration_0021.sql` gelöscht; Migration 0021 renamed `_old` Tabellen statt DROP; Migration 0027 kopiert `company_id → contact_id` mit Backup-Spalte |
|
||||||
**Maßnahme:**
|
| P0-3 | Plugin-Upload deaktiviert | `app/routes/plugins.py` — `/upload` und `/install-url` return 403 mit `upload_disabled` / `install_url_disabled` |
|
||||||
- Header-Authentifizierung komplett entfernen
|
| P0-4 | RLS repariert | `alembic/versions/0028_rls_force.py` — `FORCE ROW LEVEL SECURITY` + `WITH CHECK` auf allen Tenant-Tabellen |
|
||||||
- Für interne Service-Kommunikation: dedizierte Service-Accounts mit kurzlebigen signierten Tokens (JWT mit `aud`, `iss`, `sub`, `tenant_id`, `exp`)
|
| P0-5 | Plugin-Doppelregistrierung | `app/main.py` — Routes in `create_app()`, `lifespan()` nur aktiviert/deaktiviert, respektiert DB `active` Status, Migration-Fail deaktiviert Plugin |
|
||||||
- Separate interne API oder mTLS
|
| P0-6 | Persistent Volume | `docker-compose.yml` — `storage:/data/storage`, `pgdata`, `redisdata` Volumes |
|
||||||
- Keine Übernahme beliebiger `user_id` aus einem Header
|
| P1-1 | User/Tenant-Modell | `app/models/user.py` — `User` hat keine `tenant_id`/`role` mehr, `UserTenant` ist single source of truth, `email` global unique |
|
||||||
- Audit-Logging jeder Delegation
|
| P1-2 | Redis zentralisiert | `app/core/auth.py` — `init_redis()`/`get_redis()` Singleton, `init_job_pool()`/`close_job_pool()` |
|
||||||
- `except (ValueError, Exception): pass` ersetzen durch spezifisches Exception-Handling mit Logging
|
| P1-3 | Worker ausgelagert | `prestart.sh` — nur Alembic + Uvicorn; separater `crm-worker` Container in `docker-compose.yml` |
|
||||||
|
| P1-4 | Transactional Outbox | `app/core/outbox.py`, `app/models/outbox.py`, `alembic/versions/0040_outbox.py` — `enqueue_outbox_event()` + `process_outbox_batch()` mit `FOR UPDATE SKIP LOCKED` |
|
||||||
**Aufwand:** 2-4 Stunden
|
| P1-5 | XSS-Stellen geschlossen | `HtmlBlock.tsx` + `SignatureManager.tsx` — `DOMPurify.sanitize()`; `ActionCardBlock.tsx` — URL-Validierung (nur `http:`/`https:`) |
|
||||||
|
| P1-6 | DMS lastfest | `app/plugins/builtins/dms/routes.py` — 1MB Chunked Streaming, SHA-256 Content-Hash |
|
||||||
|
| P1-7 | Permission-System | `app/core/permissions.py` — `permission_version` wird beim Cache-Lesen geprüft, `redis.scan()` statt `redis.keys()`, `require_write()` prüft spezifische Permissions |
|
||||||
|
| P1-8 | Password Reset | `app/services/auth_service.py` — ARQ Job `send_password_reset_email`, Token `used_at` Tracking |
|
||||||
|
| P1-9 | Metrics abgesichert | `app/routes/metrics.py` — `Depends(require_admin)` |
|
||||||
|
| P1-10 | Coolify-Doku & Config | `COOLIFY_SETUP.md` — Healthcheck `/api/v1/health`, JWT-Vars entfernt, CORS `:443`; `app/config.py` — `storage_path=/data/storage`, `session_cookie_secure=True`, Startup-Validierung; `docker-compose.yml` — Redis, Volumes, Healthcheck |
|
||||||
|
| P1-11 | Cross-Tenant FK | `alembic/versions/0036_cross_tenant_fk.py` — `UNIQUE (tenant_id, id)` + Composite FK `(tenant_id, contact_id)` auf `contactpersons` und `contact_merge_history` |
|
||||||
|
| P2-1 | Contact Model normalisiert | `alembic/versions/0039_contact_normalize.py` — `surfix→suffix`, `Float→Numeric(5,2)`, `JSON→JSONB`, `CHECK (0-100)`, Unique Constraints |
|
||||||
|
| P2-3 | Commands & Statusmaschinen | `app/commands/` (base, contact, calendar, dms, mail) + `app/core/state_machine.py` |
|
||||||
|
| P2-4 | SPA Path-Traversal | `app/main.py` — `os.path.abspath` Check + `".." in full_path` Blocking |
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
### P0-2: Destruktive Migrationen ersetzen
|
## ⏳ Offene Items
|
||||||
|
|
||||||
**Problem:**
|
|
||||||
- `alembic/versions/0021_unified_contacts.py`: `DROP TABLE` ohne Datenübernahme
|
|
||||||
- `alembic/versions/0027_unify_company_to_contact.py`: `company_id` wird gelöscht ohne Datenübernahme; Downgrade ändert pauschal alle `entity_type='contact'` zurück zu `'company'`
|
|
||||||
- `migration_0021.sql` im Projekt-Root: konkurrierender Migrationsweg, manipuliert `alembic_version` direkt
|
|
||||||
|
|
||||||
**Dateien:**
|
|
||||||
- `alembic/versions/0021_unified_contacts.py`
|
|
||||||
- `alembic/versions/0027_unify_company_to_contact.py`
|
|
||||||
- `migration_0021.sql` (löschen)
|
|
||||||
|
|
||||||
**Maßnahme:**
|
|
||||||
1. `migration_0021.sql` löschen
|
|
||||||
2. Migration 0021 durch echte Transformationsmigration ersetzen:
|
|
||||||
- Alte Tabellen umbenennen (`_old` suffix), nicht löschen
|
|
||||||
- Daten mit `INSERT ... SELECT` übertragen
|
|
||||||
- Anzahl, Checksummen und Plausibilität vor/nach der Migration vergleichen
|
|
||||||
- Alttabellen erst in späterer Migration entfernen
|
|
||||||
3. Migration 0027 korrigieren:
|
|
||||||
- `company_id` Werte vor Drop in `contact_id` übertragen
|
|
||||||
- Downgrade: nur Datensätze zurückändern, die ursprünglich `'company'` waren (Tracking-Spalte oder separate Tabelle)
|
|
||||||
4. Automatisierten Upgrade-Test von jeder unterstützten Version auf `head` einführen
|
|
||||||
5. Migrationen gegen reale anonymisierte DB-Kopien testen
|
|
||||||
|
|
||||||
**Aufwand:** 4-8 Stunden
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### P0-3: Plugin-Upload und URL-Installation deaktivieren
|
|
||||||
|
|
||||||
**Problem:** `app/routes/plugins.py` führt `spec.loader.exec_module(module)` aus **bevor** die Sicherheitsprüfung läuft. Das ist Remote Code Execution. Weitere Probleme: unzureichende ZIP-Traversal-Prüfung, kein Symlink-Check, keine ZIP-Bomb-Prävention, SSRF bei URL-Installation, Plugin wird in laufenden Container kopiert.
|
|
||||||
|
|
||||||
**Datei:** `app/routes/plugins.py:347-354` (`_extract_plugin_from_zip`)
|
|
||||||
|
|
||||||
**Maßnahme:**
|
|
||||||
1. **Sofort:** Upload- und URL-Installationsendpunkte (`/upload`, `/install-url`) deaktivieren oder entfernen
|
|
||||||
2. **Langfristig — Vertrauensmodell:**
|
|
||||||
- Nur signierte Plugin-Artefakte aus einer Allowlist
|
|
||||||
- Plugin-Code wird vor der Ausführung auf Signatur geprüft
|
|
||||||
3. **Langfristig — Isolationsmodell:**
|
|
||||||
- Plugin-Ausführung in separaten Containern mit minimalen Rechten
|
|
||||||
- Versionierte Plugin-API
|
|
||||||
4. ZIP-Traversal-Prüfung korrigieren: `os.path.abspath` gegen Base-Dir prüfen nach Extraction
|
|
||||||
5. Symlink-Check hinzufügen
|
|
||||||
6. Entpackungsgrößen-Limit (Anzahl Dateien + Gesamtgröße)
|
|
||||||
7. URL-Download: Redirects verbieten, interne IP-Ranges blockieren, Streaming statt RAM
|
|
||||||
|
|
||||||
**Aufwand:** Sofort-Deaktivierung 30 Min; Langfristig 2-3 Tage
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### P0-4: Mandantentrennung (RLS) reparieren
|
|
||||||
|
|
||||||
**Problem:**
|
|
||||||
- `alembic/versions/0015_rls_policies.py`: Kein `FORCE ROW LEVEL SECURITY`, kein `WITH CHECK`
|
|
||||||
- Tabellen-Owner umgeht RLS
|
|
||||||
- Plugin-Tabellen nicht in RLS-Liste
|
|
||||||
- `TenantMixin` Docstring behauptet ORM-Autofilterung, die nicht existiert
|
|
||||||
- `app/core/tenant.py` hat nur manuelle `apply_tenant_filter()` Funktion
|
|
||||||
- `contactpersons` hat `tenant_id` aber FK auf `contacts.id` ohne Tenant-Bedingung → Cross-Tenant-FK möglich
|
|
||||||
|
|
||||||
**Dateien:**
|
|
||||||
- `alembic/versions/0015_rls_policies.py`
|
|
||||||
- `app/core/db/__init__.py` (TenantMixin Docstring)
|
|
||||||
- `app/core/tenant.py`
|
|
||||||
- Neue Migration für FORCE + WITH CHECK
|
|
||||||
|
|
||||||
**Maßnahme:**
|
|
||||||
1. Neue Migration: `ALTER TABLE ... FORCE ROW LEVEL SECURITY` für alle Tenant-Tabellen
|
|
||||||
2. Policies mit `USING` und `WITH CHECK` neu erstellen
|
|
||||||
3. Separater DB-Migrationsowner; Runtime-User ohne Owner- oder Bypass-RLS-Rechte
|
|
||||||
4. RLS für alle mandantenbezogenen Tabellen, einschließlich Plugin-Tabellen
|
|
||||||
5. CI-Test: Cross-Tenant-Lese- und Schreibversuche
|
|
||||||
6. Composite-Integrität: eindeutiges `(tenant_id, id)` und FK auf `(tenant_id, contact_id)`
|
|
||||||
7. `TenantMixin` Docstring korrigieren: Autofilterung existiert nicht
|
|
||||||
8. Zentralen Query-/Repository-Mechanismus einführen statt freiwilliger Tenant-Filter
|
|
||||||
9. Später neu erstellte Tabellen automatisch erfassen (Event-Listener oder CI-Check)
|
|
||||||
|
|
||||||
**Aufwand:** 1-2 Tage
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### P0-5: Plugin-System Doppelregistrierung beheben
|
|
||||||
|
|
||||||
**Problem:**
|
|
||||||
- `app/main.py` `create_app()` registriert alle Plugin-Routen unabhängig vom Aktivierungsstatus
|
|
||||||
- `lifespan()` registriert dieselben Routen nochmal → Doppelregistrierung
|
|
||||||
- `lifespan()` auto-installiert und auto-aktiviert alle Builtins bei jedem Start
|
|
||||||
- Deaktivierte Plugins werden reaktiviert
|
|
||||||
- `registry._plugins` wird direkt zugegriffen (private Feld)
|
|
||||||
- Migrationsfehler werden nur geloggt, Aktivierung wird trotzdem versucht
|
|
||||||
- 204 direkte Cross-Imports zwischen Built-in-Plugins
|
|
||||||
|
|
||||||
**Datei:** `app/main.py:317-330` und `app/main.py:112-165`
|
|
||||||
|
|
||||||
**Maßnahme:**
|
|
||||||
1. Routen **einmalig** beim Prozessstart registrieren — entweder in `create_app()` ODER in `lifespan()`, nicht beides
|
|
||||||
2. Aktivierungsstatus vor dem Router-Aufbau laden und respektieren
|
|
||||||
3. Keine dynamische Änderung von FastAPI-Routen während des Betriebs
|
|
||||||
4. Aktivierung/Deaktivierung erfordert kontrollierten Neustart
|
|
||||||
5. Fehlgeschlagene Migration blockiert den Start (nicht nur loggen)
|
|
||||||
6. Core-Module und optionale Module klar trennen
|
|
||||||
7. Kein Zugriff auf `registry._plugins` — öffentliche API verwenden
|
|
||||||
8. Plugin-Abhängigkeiten über deklarierte Contracts prüfen
|
|
||||||
9. **Langfristig:** Cross-Imports reduzieren — öffentliche Schnittstellen statt direkter Modell-Imports
|
|
||||||
|
|
||||||
**Aufwand:** 1 Tag für Doppelregistrierung; Cross-Import-Reduktion 1-2 Wochen
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### P0-6: Persistent Volume für Coolify-Deployment
|
|
||||||
|
|
||||||
**Problem:** Der laufende Container hat **keine Volume-Mounts** (`[]`). `/data/storage` ist nicht persistent. Alle hochgeladenen Dateien (DMS, Attachments, Bilder) gehen bei jedem Redeployment verloren. Plugin-Dateien in `app/plugins/builtins/` überleben keinen Neustart.
|
|
||||||
|
|
||||||
**Gefunden in:** Coolify-Container-Inspect (live)
|
|
||||||
|
|
||||||
**Maßnahme:**
|
|
||||||
1. In Coolify persistentes Volume für `/data/storage` konfigurieren
|
|
||||||
2. Alternativ: S3-kompatiblen Object Storage verwenden (`.env.example` hat bereits `STORAGE_BACKEND=s3` Support)
|
|
||||||
3. Plugin-Dateien nicht in Container-Filesystem kopieren — separate Plugin-Registry mit DB-basierter Konfiguration
|
|
||||||
|
|
||||||
**Aufwand:** 1-2 Stunden (Volume in Coolify konfigurieren)
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### P0-7: App von öffentlicher Domain nehmen
|
### P0-7: App von öffentlicher Domain nehmen
|
||||||
|
|
||||||
**Problem:** Die App läuft unter `https://crm.media-on.de` und ist öffentlich erreichbar — mit allen P0-Schwachstellen (Auth-Bypass, Plugin-RCE, XSS, etc.).
|
**Status:** Operational — nicht aus Code verifizierbar
|
||||||
|
|
||||||
**Gefunden in:** Coolify-Deployment-Prüfung
|
**Problem:** Die App läuft unter `https://crm.media-on.de` und ist öffentlich erreichbar.
|
||||||
|
|
||||||
**Maßnahme:**
|
**Maßnahme:**
|
||||||
1. **Sofort:** App von öffentlicher Domain nehmen oder IP-Whitelist/Basic Auth vorschalten
|
1. **Sofort:** App von öffentlicher Domain nehmen oder IP-Whitelist/Basic Auth vorschalten
|
||||||
2. Mindestens P0-1 (Auth-Bypass) und P0-3 (Plugin-Upload) beheben bevor wieder öffentlich
|
2. Mindestens P0-1 (Auth-Bypass ✅) und P0-3 (Plugin-Upload ✅) sind bereits behoben
|
||||||
3. Alternativ: VPN/Tunnel-Zugang statt öffentliche Domain
|
3. Alternativ: VPN/Tunnel-Zugang statt öffentliche Domain
|
||||||
|
|
||||||
**Aufwand:** 30 Minuten
|
**Aufwand:** 30 Minuten
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## P1 — Vor Nutzung realer Kundendaten
|
|
||||||
|
|
||||||
### P1-1: Benutzer- und Mandantenmodell bereinigen
|
|
||||||
|
|
||||||
**Problem:**
|
|
||||||
- `User` hat `tenant_id`, `role`, `role_id` — gleichzeitig existiert `UserTenant` mit `tenant_id`, `role_id`, `is_default`
|
|
||||||
- Zwei Quellen der Wahrheit für Mandantenzugehörigkeit und Rollen
|
|
||||||
- `login()` sucht nur nach `email` mit `scalar_one_or_none()` → crasht bei mehreren Treffern (gleiche E-Mail in mehreren Mandanten)
|
|
||||||
- `tenant_slug` Parameter in `login()` wird von Login-Route nicht übergeben
|
|
||||||
- `TenantService.list_tenant_users()` sucht über `User.tenant_id` und ignoriert N:M-Mitgliedschaften
|
|
||||||
|
|
||||||
**Dateien:**
|
|
||||||
- `app/models/user.py`
|
|
||||||
- `app/services/auth_service.py:30-80`
|
|
||||||
- `app/routes/auth.py`
|
|
||||||
|
|
||||||
**Maßnahme:**
|
|
||||||
1. `users.email` global eindeutig machen (nicht `(tenant_id, email)`)
|
|
||||||
2. `User.tenant_id` und `User.role`/`User.role_id` entfernen
|
|
||||||
3. `tenant_memberships` als einzige Quelle: `tenant_id`, `user_id`, `role_id`, `status`, `is_default`
|
|
||||||
4. `login()` mit `tenant_slug` verknüpfen oder Default-Tenant verwenden
|
|
||||||
5. `TenantService.list_tenant_users()` über `UserTenant` suchen
|
|
||||||
|
|
||||||
**Aufwand:** 1 Tag
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### P1-2: Redis-Verbindungen zentralisieren
|
|
||||||
|
|
||||||
**Problem:** `app/core/auth.py:49-51` erstellt pro Aufruf einen neuen Redis-Client. Kein Pool, kein Close. Dasselbe bei `enqueue_job()` für ARQ-Pools. Folgen: Connection-Lecks, Socket-Erschöpfung, instabiles Verhalten unter Last.
|
|
||||||
|
|
||||||
**Datei:** `app/core/auth.py:49-51`, `app/core/worker.py` (enqueue_job)
|
|
||||||
|
|
||||||
**Maßnahme:**
|
|
||||||
1. Redis-Client einmal im Application-Lifespan initialisieren
|
|
||||||
2. Bei Shutdown schließen
|
|
||||||
3. Über Dependency Injection verteilen
|
|
||||||
4. ARQ-Pool einmalig erstellen und wiederverwenden
|
|
||||||
|
|
||||||
**Aufwand:** 2-4 Stunden
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### P1-3: Worker und Scheduler aus API-Container auslagern
|
|
||||||
|
|
||||||
**Problem:** `prestart.sh` startet ARQ-Worker im Hintergrund und Uvicorn als PID 1. Worker-Tod wird nicht erkannt. Worker und API konkurrieren um Ressourcen. Keine separate Skalierung. Cron-Jobs können bei mehreren Replikas mehrfach ausgeführt werden.
|
|
||||||
|
|
||||||
**Datei:** `prestart.sh`
|
|
||||||
|
|
||||||
**Maßnahme:**
|
|
||||||
1. Worker in separaten Container auslagern
|
|
||||||
2. Scheduler in separaten Container mit verteilter Lock-/Leader-Election
|
|
||||||
3. Idempotente Jobs
|
|
||||||
4. Heartbeat mit Zeitstempel
|
|
||||||
5. Dead-Letter-/Failed-Job-Strategie
|
|
||||||
6. Retry-Policy pro Jobtyp
|
|
||||||
7. Worker-Healthcheck prüft ob Worker lebt, nicht nur ob Redis-Queue lesbar ist
|
|
||||||
|
|
||||||
**Aufwand:** 1-2 Tage
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### P1-4: Transactional Outbox einführen
|
|
||||||
|
|
||||||
**Problem:** `app/core/event_bus.py` ist rein speicherbasiert. Events verschwinden bei Prozessabsturz, Neustart, mehreren Replikas, Handler-Fehlern. `asyncio.gather(..., return_exceptions=True)` sammelt Fehler ohne Behandlung.
|
|
||||||
|
|
||||||
**Datei:** `app/core/event_bus.py`
|
|
||||||
|
|
||||||
**Maßnahme:**
|
|
||||||
1. Transactional Outbox in PostgreSQL
|
|
||||||
2. Worker verarbeitet Outbox-Einträge
|
|
||||||
3. Inbox/Idempotency-Key auf Konsumentenseite
|
|
||||||
4. Retry und Dead Letter
|
|
||||||
5. Events versionieren
|
|
||||||
6. In-Process-Bus nur für unkritische lokale Benachrichtigungen
|
|
||||||
|
|
||||||
**Aufwand:** 2-3 Tage
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### P1-5: XSS-Stellen schließen
|
|
||||||
|
|
||||||
**Problem:**
|
|
||||||
- `HtmlBlock.tsx`: Regex-Sanitizer + `dangerouslySetInnerHTML` — HTML lässt sich nicht sicher mit Regex sanitizen
|
|
||||||
- `SignatureManager.tsx:201`: `dangerouslySetInnerHTML={{ __html: sig.body_html }}` **ohne jegliche Sanitization**
|
|
||||||
- `ActionCardBlock.tsx:21-28`: `window.open(action.action)` ohne URL-Validierung — `javascript:`-URLs möglich
|
|
||||||
- Mail-Service: `body_html_sanitized = body_html` ohne Sanitizer an manchen Stellen
|
|
||||||
|
|
||||||
**Dateien:**
|
|
||||||
- `frontend/src/components/comm/blocks/HtmlBlock.tsx`
|
|
||||||
- `frontend/src/components/mail/SignatureManager.tsx`
|
|
||||||
- `frontend/src/components/comm/blocks/ActionCardBlock.tsx`
|
|
||||||
- Mail-Service (body_html_sanitized)
|
|
||||||
|
|
||||||
**Maßnahme:**
|
|
||||||
1. Serverseitig konsequent `nh3` verwenden
|
|
||||||
2. Frontend zusätzlich `DOMPurify` als zweite Barriere
|
|
||||||
3. Keine selbst gebauten Regex-Sanitizer
|
|
||||||
4. Nur `https:` und kontrollierte interne Pfade erlauben
|
|
||||||
5. Strikte Content Security Policy ohne `unsafe-inline`
|
|
||||||
6. Signatur-, Mail-, KI- und Kommunikationsinhalte als nicht vertrauenswürdig behandeln
|
|
||||||
|
|
||||||
**Aufwand:** 4-6 Stunden
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### P1-6: DMS Dateiverarbeitung lastfest machen
|
|
||||||
|
|
||||||
**Problem:** `app/plugins/builtins/dms/routes.py` liest die komplette Datei in RAM (`content = await file.read()`). Max 100 MB. Bei 10 parallelen Uploads mehrere GB RAM. Kein Virenscan, kein Content-Hash, keine Dublettenerkennung, keine Tenant-Quotas, kein Versionierungsmodell, kein Garbage Collector für physische Dateien nach Soft Delete. `storage_path` wird an Frontend ausgegeben. Benutzerdateiname direkt in Content-Disposition.
|
|
||||||
|
|
||||||
**Datei:** `app/plugins/builtins/dms/routes.py:421-436`
|
|
||||||
|
|
||||||
**Maßnahme:**
|
|
||||||
1. Chunked Streaming direkt in Object Storage
|
|
||||||
2. Maximale Größe auf Proxy- und Anwendungsebene
|
|
||||||
3. SHA-256 Content-Hash
|
|
||||||
4. Malware-Scan
|
|
||||||
5. Quotas pro Tenant
|
|
||||||
6. Versionierte Metadaten
|
|
||||||
7. Garbage Collector für physische Dateien nach Soft Delete
|
|
||||||
8. `storage_path` nicht an Frontend ausgeben
|
|
||||||
9. Benutzerdateiname sanitizen vor Content-Disposition
|
|
||||||
10. Synchronen MinIO-Client aus `async def` entfernen
|
|
||||||
|
|
||||||
**Aufwand:** 1-2 Tage
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### P1-7: Berechtigungssystem vereinheitlichen
|
|
||||||
|
|
||||||
**Problem:**
|
|
||||||
- Legacy-Rollenstrings (`admin`/`editor`/`viewer`) + neue Rollen mit `role_id` + Gruppen + Allow/Deny + Feldrechte + `is_system_admin` + globale Write-Hilfsrechte
|
|
||||||
- `permission_version` wird gespeichert, beim Cache-Lesen aber nicht geprüft
|
|
||||||
- Cache-Invalidierung verwendet `redis.keys()` — blockiert Redis bei großen Datenmengen
|
|
||||||
- Feldrechte mehrerer Gruppen werden per `dict.update()` überschrieben (last-write-wins)
|
|
||||||
- `viewer` erhält `user_preferences:write`
|
|
||||||
- `require_write()` erlaubt `*:write` oder `*:create` (zu breit)
|
|
||||||
- `db.rollback()` bei Permission-Fehler setzt fremde Transaktionsarbeit zurück
|
|
||||||
|
|
||||||
**Datei:** `app/core/permissions.py`, `app/deps.py`
|
|
||||||
|
|
||||||
**Maßnahme:**
|
|
||||||
1. Nur noch Capability-basierte Berechtigungen (`contacts.read`, `contacts.create`, etc.)
|
|
||||||
2. Keine generische `require_write`-Freigabe
|
|
||||||
3. Alte Rollenlogik entfernen
|
|
||||||
4. Feldrechte deterministisch nach "strengstes Recht gewinnt" zusammenführen
|
|
||||||
5. `permission_version` beim Cache-Lesen prüfen
|
|
||||||
6. `redis.keys()` ersetzen durch `redis.scan()` oder gezielte Cache-Key-Invalidierung
|
|
||||||
7. `db.rollback()` nur in eigenen Transaktionskontext
|
|
||||||
|
|
||||||
**Aufwand:** 1-2 Tage
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### P1-8: Password Reset funktionsfähig machen
|
|
||||||
|
|
||||||
**Problem:** `request_password_reset()` erstellt ein Token, speichert es in der DB, sendet es aber nicht. Nicht einmal geloggt. Die Variable `raw_token` wird nach Erstellung ignoriert. Die Route sagt "a reset link has been sent" — das ist fachlich falsch. Nach Passwortwechsel werden bestehende Sessions nicht widerrufen.
|
|
||||||
|
|
||||||
**Datei:** `app/services/auth_service.py:159-200`
|
|
||||||
|
|
||||||
**Maßnahme:**
|
|
||||||
1. Reset-Mail über echte Queue verschicken (ARQ-Worker)
|
|
||||||
2. Token nur einmal verwendbar
|
|
||||||
3. Alle Sessions des Benutzers nach Passwortänderung widerrufen
|
|
||||||
4. Sicherheitsereignis protokollieren
|
|
||||||
5. Optional: Nutzer über Passwortänderung informieren
|
|
||||||
|
|
||||||
**Aufwand:** 2-4 Stunden
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### P1-9: Metrics-Endpunkt absichern
|
|
||||||
|
|
||||||
**Problem:** `app/routes/metrics.py` sagt "admin-only" im Docstring, verwendet aber nur `get_current_user` statt `require_admin`. Jeder angemeldete Benutzer kann Prometheus-Metriken abrufen.
|
|
||||||
|
|
||||||
**Datei:** `app/routes/metrics.py`
|
|
||||||
|
|
||||||
**Maßnahme:**
|
|
||||||
1. `require_admin` oder `require_permission("system:metrics")` verwenden
|
|
||||||
2. Alternativ: internes Netzwerk, Reverse-Proxy-Allowlist, dedizierten Monitoring-Token oder mTLS
|
|
||||||
|
|
||||||
**Aufwand:** 30 Minuten
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### P1-10: Coolify-Dokumentation korrigieren
|
|
||||||
|
|
||||||
**Problem:**
|
|
||||||
- `COOLIFY_SETUP.md` Abschnitt 6 dokumentiert `/health` als Healthcheck-Pfad — die App hat nur `/api/v1/health`. `/health` liefert nur die SPA `index.html` (Catch-All).
|
|
||||||
- `COOLIFY_SETUP.md` listet `JWT_ALGORITHM` und `JWT_EXPIRY_HOURS` — werden von der App nicht verwendet.
|
|
||||||
- `CORS_ORIGINS` in Coolify ohne `:443` — `COOLIFY_SETUP.md` sagt explizit Port ist mandatory.
|
|
||||||
|
|
||||||
**Dateien:** `COOLIFY_SETUP.md`, `docs/deployment-guide.md`
|
|
||||||
|
|
||||||
**Maßnahme:**
|
|
||||||
1. Healthcheck-Pfad in Doku auf `/api/v1/health` korrigieren
|
|
||||||
2. JWT-Variablen aus Doku entfernen oder App auf JWT umstellen
|
|
||||||
3. `CORS_ORIGINS` in Coolify auf `https://crm.media-on.de:443` setzen
|
|
||||||
4. `docker-compose.yml` Healthcheck auf `/api/v1/health` korrigieren
|
|
||||||
5. `docker-compose.yml` Redis-Service hinzufügen
|
|
||||||
6. `docker-compose.yml` `REDIS_URL` setzen
|
|
||||||
7. `docker-compose.yml` persistentes Volume für `/data/storage`
|
|
||||||
8. `docker-compose.yml` `SESSION_COOKIE_SECURE=true` für Production
|
|
||||||
9. `docker-compose.yml` `STORAGE_PATH=/data/storage` setzen
|
|
||||||
10. `config.py` Default `storage_path` von `/tmp` auf `/data/storage` ändern
|
|
||||||
11. `config.py` Default `session_cookie_secure` auf `True` ändern (Production-Default)
|
|
||||||
12. `config.py` Startup-Validierung: `ENVIRONMENT=production` + `session_cookie_secure=False` → harter Abbruch
|
|
||||||
|
|
||||||
**Aufwand:** 2-3 Stunden
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### P1-11: Cross-Tenant referenzielle Integrität
|
|
||||||
|
|
||||||
**Problem:** `contactpersons` hat `tenant_id` aber `contact_id` FK referenziert nur `contacts.id` ohne Tenant-Bedingung. Die DB verhindert nicht, dass ein Contactperson-Datensatz aus Mandant A auf einen Kontakt aus Mandant B zeigt.
|
|
||||||
|
|
||||||
**Datei:** `alembic/versions/0021_unified_contacts.py` (contactpersons Tabelle)
|
|
||||||
|
|
||||||
**Maßnahme:**
|
|
||||||
1. Composite-FK: `(tenant_id, contact_id)` referenziert `(tenant_id, id)` auf `contacts`
|
|
||||||
2. Eindeutiges `(tenant_id, id)` auf `contacts`
|
|
||||||
3. Dasselbe für alle mandantenbezogenen FK-Beziehungen
|
|
||||||
|
|
||||||
**Aufwand:** 2-4 Stunden
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## P2 — Architektonische Konsolidierung
|
|
||||||
|
|
||||||
### P2-1: Unified Contact Model normalisieren
|
|
||||||
|
|
||||||
**Problem:** Eine Tabelle enthält Unternehmen, Personen, 3 Adressarten, Bankdaten, Steuernummern, Rabatte, Projektinformationen, Warnungen, Tags, Custom Fields, Suchindex. Dubletten zu vorhandenen Modellen für Adressen, Bankkonten, Tags, Custom Fields.
|
|
||||||
|
|
||||||
**Weitere Probleme:**
|
|
||||||
- Rabatte als `Float` statt `Numeric`/`Decimal`
|
|
||||||
- Keine DB-Checks für Werte 0-100
|
|
||||||
- Keine eindeutigen Kontakt-/Buchhaltungscodes pro Mandant
|
|
||||||
- Keine klare Validierung welche Felder bei Person/Firma erlaubt sind
|
|
||||||
- `surfix` — dauerhaft übernommener Tippfehler
|
|
||||||
- `JSON` statt `JSONB`
|
|
||||||
- Suche fest auf Deutsch eingestellt
|
|
||||||
- Keine normalisierten Suchschlüssel für E-Mail und Telefonnummer
|
|
||||||
- CSV-Import ohne Dubletten-/Encoding-/Dezimal-/Rollback-Strategie
|
|
||||||
|
|
||||||
**Maßnahme:**
|
|
||||||
1. Adressen in separate Tabelle auslagern (bereits vorhanden — nutzen)
|
|
||||||
2. Bankdaten in separate Tabelle (bereits vorhanden — nutzen)
|
|
||||||
3. Tags als Relation (bereits vorhanden — nutzen)
|
|
||||||
4. Custom Fields als Relation (bereits vorhanden — nutzen)
|
|
||||||
5. Rabatte: `Numeric(5,2)` statt `Float`
|
|
||||||
6. DB-Check: `discount_* BETWEEN 0 AND 100`
|
|
||||||
7. Eindeutige `(tenant_id, code)` und `(tenant_id, accounting_code)`
|
|
||||||
8. `surfix` → `suffix` (Migration mit Rename)
|
|
||||||
9. `JSON` → `JSONB`
|
|
||||||
10. Suchkonfiguration pro Mandant konfigurierbar
|
|
||||||
11. Normalisierte Suchschlüssel (lowercase, trimmed) für E-Mail und Telefon
|
|
||||||
12. CSV-Import: Dubletten-Erkennung, Encoding-Detection, Decimal-Parsing, Transaction-Rollback
|
|
||||||
|
|
||||||
**Aufwand:** 2-3 Tage
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### P2-2: Plugin-Cross-Imports reduzieren
|
### P2-2: Plugin-Cross-Imports reduzieren
|
||||||
|
|
||||||
**Problem:** 204 direkte `from app.plugins.builtins` Imports zwischen Plugins. Automatisierung importiert Modelle/Services von Kommunikation, Mail, Kalender. Verteilter Monolith ohne Modulgrenzen.
|
**Status:** Offen — 228 direkte Cross-Imports zwischen Plugins
|
||||||
|
|
||||||
|
**Problem:** 228 direkte `from app.plugins.builtins` Imports zwischen Plugins. Automatisierung importiert Modelle/Services von Kommunikation, Mail, Kalender. Verteilter Monolith ohne Modulgrenzen.
|
||||||
|
|
||||||
**Maßnahme:**
|
**Maßnahme:**
|
||||||
1. Öffentliche Schnittstellen (Contracts) für jedes Modul definieren
|
1. Öffentliche Schnittstellen (Contracts) für jedes Modul definieren
|
||||||
@@ -442,73 +68,14 @@
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
### P2-3: Commands und Statusmaschinen
|
|
||||||
|
|
||||||
**Problem:** Geschäftsoperationen als `Route → Service → mehrere flush/commit` statt als zentrale Commands. Statusstrings frei beschreibbar statt Statusmaschinen.
|
|
||||||
|
|
||||||
**Maßnahme:**
|
|
||||||
1. `Route → Command → Authorization → Domain Operation → Transaction → Audit → Outbox Events → Commit`
|
|
||||||
2. Explizite Statusmaschinen für Angebote, Aufträge, Rechnungen
|
|
||||||
3. Übergänge validiert und auditiert
|
|
||||||
|
|
||||||
**Aufwand:** 1-2 Wochen
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### P2-4: SPA Path-Traversal-Schutz vervollständigen
|
|
||||||
|
|
||||||
**Problem:** `app/main.py` SPA-Catch-All blockiert `..` nur in bestimmten Positionen. `..` in anderen Positionen wird nicht erfasst.
|
|
||||||
|
|
||||||
**Datei:** `app/main.py` (spa_spa Funktion)
|
|
||||||
|
|
||||||
**Maßnahme:**
|
|
||||||
1. `os.path.abspath` gegen `frontend_dist` prüfen nach Join
|
|
||||||
2. Kein `..` in irgendeiner Position erlauben
|
|
||||||
|
|
||||||
**Aufwand:** 30 Minuten
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Zusammenfassung
|
## Zusammenfassung
|
||||||
|
|
||||||
| Priorität | Anzahl | Geschätzter Aufwand |
|
| Priorität | Erledigt | Offen | Geschätzter Aufwand (offen) |
|
||||||
|---|---|---|
|
|---|---|---|---|
|
||||||
| P0 (sofort) | 7 | ~5-7 Tage |
|
| P0 | 6/7 | 1 (operational) | 30 Minuten |
|
||||||
| P1 (vor Kundendaten) | 11 | ~7-10 Tage |
|
| P1 | 11/11 | 0 | — |
|
||||||
| P2 (architektonisch) | 4 | ~2-4 Wochen |
|
| P2 | 3/4 | 1 | 1-2 Wochen |
|
||||||
| **Total** | **22** | **~4-6 Wochen** |
|
| **Total** | **20/22** | **2** | **~1-2 Wochen** |
|
||||||
|
|
||||||
## Reihenfolge
|
|
||||||
|
|
||||||
### Woche 1: P0 absichern
|
|
||||||
1. P0-7: App von öffentlicher Domain nehmen (30 Min)
|
|
||||||
2. P0-1: Auth-Bypass entfernen (2-4h)
|
|
||||||
3. P0-3: Plugin-Upload deaktivieren (30 Min Sofort, langfristig später)
|
|
||||||
4. P0-6: Persistent Volume in Coolify (1-2h)
|
|
||||||
5. P0-2: Migrationen ersetzen (4-8h)
|
|
||||||
6. P0-4: RLS reparieren (1-2 Tage)
|
|
||||||
7. P0-5: Plugin-Doppelregistrierung beheben (1 Tag)
|
|
||||||
|
|
||||||
### Woche 2-3: P1 Fundament
|
|
||||||
8. P1-9: Metrics absichern (30 Min)
|
|
||||||
9. P1-8: Password Reset (2-4h)
|
|
||||||
10. P1-10: Coolify-Doku & Config korrigieren (2-3h)
|
|
||||||
11. P1-2: Redis zentralisieren (2-4h)
|
|
||||||
12. P1-5: XSS schließen (4-6h)
|
|
||||||
13. P1-11: Cross-Tenant FK (2-4h)
|
|
||||||
14. P1-1: User/Tenant-Modell (1 Tag)
|
|
||||||
15. P1-7: Permission-System (1-2 Tage)
|
|
||||||
16. P1-6: DMS lastfest (1-2 Tage)
|
|
||||||
17. P1-3: Worker auslagern (1-2 Tage)
|
|
||||||
18. P1-4: Transactional Outbox (2-3 Tage)
|
|
||||||
|
|
||||||
### Woche 4-6: P2 Architektur
|
|
||||||
19. P2-4: SPA Path-Traversal (30 Min)
|
|
||||||
20. P2-1: Contact Model normalisieren (2-3 Tage)
|
|
||||||
21. P2-2: Cross-Imports reduzieren (1-2 Wochen)
|
|
||||||
22. P2-3: Commands & Statusmaschinen (1-2 Wochen)
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Validierung nach jedem Fix
|
## Validierung nach jedem Fix
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,724 @@
|
|||||||
|
# LeoCRM Plugin-System — Kompletter Umbauplan
|
||||||
|
|
||||||
|
**Erstellt:** 2026-07-26
|
||||||
|
**Geschätzter Gesamtaufwand:** ~129 Stunden (~16 Arbeitstage)
|
||||||
|
**Status:** Geplant — noch nicht gestartet
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Übersicht: 5 Phasen
|
||||||
|
|
||||||
|
| Phase | Punkte | Inhalt | Stunden | Tage |
|
||||||
|
|---|---|---|---|---|
|
||||||
|
| Phase 1 | 1-3 | Contracts konsequent nutzen | 47 | 6 |
|
||||||
|
| Phase 2 | 4 | Hooks/Filters-System | 16 | 2 |
|
||||||
|
| Phase 3 | 5 | Plugin-Isolation (Linting) | 4 | 0,5 |
|
||||||
|
| Phase 4 | 8 | Plugin-Versioning | 20 | 2,5 |
|
||||||
|
| Phase 5 | 6 | Marketplace-Vorbereitung | 42 | 5 |
|
||||||
|
| **Gesamt** | | | **129** | **16** |
|
||||||
|
|
||||||
|
**Wichtig:** Jede Phase ist unabhängig funktionsfähig. Das System läuft nach jeder Phase ohne Einschränkungen weiter.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Phase 1: Contracts konsequent nutzen (Punkte 1-3)
|
||||||
|
|
||||||
|
**Ziel:** Alle 224 direkten Cross-Plugin-Imports werden durch das Contract-System ersetzt.
|
||||||
|
|
||||||
|
### 1.1 Fehlende contracts.py erstellen (7 Std)
|
||||||
|
|
||||||
|
Für jedes Plugin, das noch keine `contracts.py` hat, eine erstellen:
|
||||||
|
|
||||||
|
| # | Plugin | Exportierte Symbole | Aufwand |
|
||||||
|
|---|---|---|---|
|
||||||
|
| 1 | `ai_proactive` | ContextTools, ProactiveAgent, JobScheduler | 30 Min |
|
||||||
|
| 2 | `ai_ui_control` | WebSocketManager, UIAction | 30 Min |
|
||||||
|
| 3 | `automation` | AgentRunner, ExecutionEngine, Scheduler, WorkflowTimeout | 45 Min |
|
||||||
|
| 4 | `entity_links` | EntityLink model, create_link, get_links | 20 Min |
|
||||||
|
| 5 | `forgejo_error_reporter` | report_error_to_forgejo | 15 Min |
|
||||||
|
| 6 | `mcp_client` | McpClient, McpServerConfig | 30 Min |
|
||||||
|
| 7 | `mcp_server` | McpServer, ToolDefinitions | 30 Min |
|
||||||
|
| 8 | `report_generator` | ReportTemplate, ReportInstance, PdfGenerator | 30 Min |
|
||||||
|
| 9 | `system_notif` | SystemNotifHandler | 15 Min |
|
||||||
|
| 10 | `tags` | Tag, TagAssignment, assign_tags, remove_tags | 20 Min |
|
||||||
|
| 11 | `tasks` | Task, TaskService, create_task, update_task | 30 Min |
|
||||||
|
| 12 | `test_sample` | TestSamplePlugin | 10 Min |
|
||||||
|
| 13 | `dms` (erweitern) | File, Folder, UploadService, DownloadService | 30 Min |
|
||||||
|
| 14 | `permissions` (erweitern) | ShareLink, PermissionResolver | 30 Min |
|
||||||
|
|
||||||
|
**Schema für jede contracts.py:**
|
||||||
|
```python
|
||||||
|
"""Public contract for the <plugin> plugin."""
|
||||||
|
from __future__ import annotations
|
||||||
|
from app.plugins.builtins.contracts import get_contract_registry
|
||||||
|
# Import only public symbols from internal modules
|
||||||
|
|
||||||
|
class <Plugin>Contract:
|
||||||
|
contract_name = "<plugin>"
|
||||||
|
# Expose only public API
|
||||||
|
|
||||||
|
_contract = <Plugin>Contract()
|
||||||
|
get_contract_registry().register("<plugin>", _contract)
|
||||||
|
```
|
||||||
|
|
||||||
|
### 1.2 Direkte Imports ersetzen (28 Std)
|
||||||
|
|
||||||
|
224 direkte Imports müssen durch `get_contract()` ersetzt werden.
|
||||||
|
|
||||||
|
**Top-Priorität (häufigste Import-Quellen):**
|
||||||
|
|
||||||
|
| # | Datei | Imports | Aufwand |
|
||||||
|
|---|---|---|---|
|
||||||
|
| 1 | `automation/plugin.py` | 10 | 1,5 Std |
|
||||||
|
| 2 | `automation/routes.py` | 8 | 1,5 Std |
|
||||||
|
| 3 | `ai_proactive/services.py` | 8 | 1,5 Std |
|
||||||
|
| 4 | `ai_proactive/plugin.py` | 8 | 1,5 Std |
|
||||||
|
| 5 | `unified_search/jobs.py` | 7 | 1 Std |
|
||||||
|
| 6 | `builtins/__init__.py` | 7 | 1 Std |
|
||||||
|
| 7 | `ai_proactive/jobs.py` | 7 | 1 Std |
|
||||||
|
| 8 | `ai_assistant/participant_handler.py` | 7 | 1 Std |
|
||||||
|
| 9 | `kommunikation/routes.py` | 6 | 1 Std |
|
||||||
|
| 10 | `kommunikation/contracts.py` | 6 | 1 Std |
|
||||||
|
| 11 | `automation/agent_routes.py` | 6 | 1 Std |
|
||||||
|
| 12 | `automation/agent_comm.py` | 6 | 1 Std |
|
||||||
|
| 13 | `ai_proactive/participant_handler.py` | 6 | 1 Std |
|
||||||
|
| 14 | `ai_assistant/plugin.py` | 6 | 1 Std |
|
||||||
|
| 15 | `unified_search/routes.py` | 5 | 45 Min |
|
||||||
|
| 16-50 | Alle übrigen Dateien | ~122 | 12 Std |
|
||||||
|
|
||||||
|
**Muster für Ersetzung:**
|
||||||
|
```python
|
||||||
|
# VORHER (direkt):
|
||||||
|
from app.plugins.builtins.kommunikation.services import send_message
|
||||||
|
|
||||||
|
# NACHHER (über Contract):
|
||||||
|
from app.plugins.builtins.contracts import get_contract
|
||||||
|
|
||||||
|
async def my_function(db, ...):
|
||||||
|
komm = get_contract("kommunikation")
|
||||||
|
if komm:
|
||||||
|
await komm.send_message(db, ...)
|
||||||
|
# Graceful degradation wenn Plugin nicht aktiv
|
||||||
|
```
|
||||||
|
|
||||||
|
### 1.3 Contracts bei Deaktivierung abmelden (4 Std)
|
||||||
|
|
||||||
|
In jedem Plugin's `on_deactivate()`:
|
||||||
|
```python
|
||||||
|
async def on_deactivate(self, db, service_container, event_bus) -> None:
|
||||||
|
# Contract abmelden
|
||||||
|
from app.plugins.builtins.contracts import get_contract_registry
|
||||||
|
get_contract_registry().unregister(self.manifest.name)
|
||||||
|
# ... rest of cleanup
|
||||||
|
await super().on_deactivate(db, service_container, event_bus)
|
||||||
|
```
|
||||||
|
|
||||||
|
| # | Plugin | Aufwand |
|
||||||
|
|---|---|---|
|
||||||
|
| 1-16 | Alle 16 Plugins | 15 Min pro Plugin = 4 Std |
|
||||||
|
|
||||||
|
### 1.4 Tests anpassen (8 Std)
|
||||||
|
|
||||||
|
- Cross-Plugin-Tests müssen mit Contracts laufen
|
||||||
|
- `test_plugins.py` — Contract-Registry Tests
|
||||||
|
- `test_contracts.py` — Neue Test-Datei für Contract-System
|
||||||
|
- Alle Integrationstests mit Contract-Mocks
|
||||||
|
|
||||||
|
### Meilenstein Phase 1:
|
||||||
|
- ✅ Alle 16 Plugins haben contracts.py
|
||||||
|
- ✅ 0 direkte Cross-Plugin-Imports (geprüft mit grep)
|
||||||
|
- ✅ Contracts werden bei Deaktivierung abgemeldet
|
||||||
|
- ✅ Alle Tests bestanden
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Phase 2: Hooks/Filters-System (Punkt 4)
|
||||||
|
|
||||||
|
**Ziel:** WordPress-Style Hooks (actions + filters) für Plugin-Erweiterbarkeit.
|
||||||
|
|
||||||
|
### 2.1 HookRegistry erstellen (4 Std)
|
||||||
|
|
||||||
|
**Neue Datei: `app/core/hooks.py`**
|
||||||
|
|
||||||
|
```python
|
||||||
|
"""WordPress-style hooks: actions (fire-and-forget) and filters (modify data)."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
import logging
|
||||||
|
from collections import defaultdict
|
||||||
|
from typing import Any, Callable
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
|
||||||
|
class HookRegistry:
|
||||||
|
"""Central registry for actions and filters.
|
||||||
|
|
||||||
|
Actions: do_action('contact.before_create', data) — no return value
|
||||||
|
Filters: result = apply_filters('contact.format_name', name) — returns modified value
|
||||||
|
|
||||||
|
Priority: lower numbers run first (default=10).
|
||||||
|
"""
|
||||||
|
|
||||||
|
_instance: HookRegistry | None = None
|
||||||
|
|
||||||
|
def __new__(cls):
|
||||||
|
if cls._instance is None:
|
||||||
|
cls._instance = super().__new__(cls)
|
||||||
|
cls._instance._actions: dict[str, list[tuple[int, Callable]]] = defaultdict(list)
|
||||||
|
cls._instance._filters: dict[str, list[tuple[int, Callable]]] = defaultdict(list)
|
||||||
|
return cls._instance
|
||||||
|
|
||||||
|
def register_action(self, hook_name: str, callback: Callable, priority: int = 10) -> None:
|
||||||
|
self._actions[hook_name].append((priority, callback))
|
||||||
|
self._actions[hook_name].sort(key=lambda x: x[0])
|
||||||
|
|
||||||
|
def register_filter(self, hook_name: str, callback: Callable, priority: int = 10) -> None:
|
||||||
|
self._filters[hook_name].append((priority, callback))
|
||||||
|
self._filters[hook_name].sort(key=lambda x: x[0])
|
||||||
|
|
||||||
|
async def do_action(self, hook_name: str, *args, **kwargs) -> None:
|
||||||
|
for _, callback in self._actions.get(hook_name, []):
|
||||||
|
try:
|
||||||
|
result = callback(*args, **kwargs)
|
||||||
|
if hasattr(result, '__await__'):
|
||||||
|
await result
|
||||||
|
except Exception:
|
||||||
|
logger.exception("Error in action %s", hook_name)
|
||||||
|
|
||||||
|
async def apply_filters(self, hook_name: str, value: Any, *args, **kwargs) -> Any:
|
||||||
|
for _, callback in self._filters.get(hook_name, []):
|
||||||
|
try:
|
||||||
|
result = callback(value, *args, **kwargs)
|
||||||
|
if hasattr(result, '__await__'):
|
||||||
|
result = await result
|
||||||
|
value = result
|
||||||
|
except Exception:
|
||||||
|
logger.exception("Error in filter %s", hook_name)
|
||||||
|
return value
|
||||||
|
|
||||||
|
def unregister(self, hook_name: str, callback: Callable) -> None:
|
||||||
|
self._actions[hook_name] = [(p, c) for p, c in self._actions.get(hook_name, []) if c != callback]
|
||||||
|
self._filters[hook_name] = [(p, c) for p, c in self._filters.get(hook_name, []) if c != callback]
|
||||||
|
|
||||||
|
def unregister_all(self, hook_name: str) -> None:
|
||||||
|
self._actions.pop(hook_name, None)
|
||||||
|
self._filters.pop(hook_name, None)
|
||||||
|
|
||||||
|
def _reset_for_testing(self) -> None:
|
||||||
|
self._actions.clear()
|
||||||
|
self._filters.clear()
|
||||||
|
|
||||||
|
|
||||||
|
def get_hook_registry() -> HookRegistry:
|
||||||
|
return HookRegistry()
|
||||||
|
|
||||||
|
async def do_action(hook_name: str, *args, **kwargs) -> None:
|
||||||
|
await get_hook_registry().do_action(hook_name, *args, **kwargs)
|
||||||
|
|
||||||
|
async def apply_filters(hook_name: str, value: Any, *args, **kwargs) -> Any:
|
||||||
|
return await get_hook_registry().apply_filters(hook_name, value, *args, **kwargs)
|
||||||
|
```
|
||||||
|
|
||||||
|
### 2.2 Integration in BasePlugin (2 Std)
|
||||||
|
|
||||||
|
```python
|
||||||
|
# In BasePlugin.on_activate:
|
||||||
|
async def on_activate(self, db, service_container, event_bus) -> None:
|
||||||
|
# ... existing code ...
|
||||||
|
# Hooks werden in Subklassen registriert
|
||||||
|
|
||||||
|
# In BasePlugin.on_deactivate:
|
||||||
|
async def on_deactivate(self, db, service_container, event_bus) -> None:
|
||||||
|
# Alle Hooks dieses Plugins abmelden
|
||||||
|
from app.core.hooks import get_hook_registry
|
||||||
|
# Plugin-spezifische Hooks entfernen (prefix mit plugin name)
|
||||||
|
# ... existing code ...
|
||||||
|
```
|
||||||
|
|
||||||
|
### 2.3 Hook-Punkte in Core-Services (6 Std)
|
||||||
|
|
||||||
|
| # | Service | Hook-Name | Typ | Beschreibung |
|
||||||
|
|---|---|---|---|---|
|
||||||
|
| 1 | contact_service | `contact.before_create` | Action | Vor Kontakt-Erstellung |
|
||||||
|
| 2 | contact_service | `contact.after_create` | Action | Nach Kontakt-Erstellung |
|
||||||
|
| 3 | contact_service | `contact.format_display_name` | Filter | Anzeigenamen formatieren |
|
||||||
|
| 4 | contact_service | `contact.before_update` | Action | Vor Kontakt-Update |
|
||||||
|
| 5 | contact_service | `contact.after_update` | Action | Nach Kontakt-Update |
|
||||||
|
| 6 | contact_service | `contact.before_delete` | Action | Vor Kontakt-Löschung |
|
||||||
|
| 7 | mail_service | `mail.before_send` | Filter | E-Mail vor Versand modifizieren |
|
||||||
|
| 8 | mail_service | `mail.after_send` | Action | Nach E-Mail-Versand |
|
||||||
|
| 9 | calendar | `calendar.before_appointment` | Action | Vor Termin-Erstellung |
|
||||||
|
| 10 | calendar | `calendar.after_appointment` | Action | Nach Termin-Erstellung |
|
||||||
|
| 11 | auth_service | `auth.before_login` | Filter | Login-Daten validieren/modifizieren |
|
||||||
|
| 12 | auth_service | `auth.after_login` | Action | Nach erfolgreichem Login |
|
||||||
|
| 13 | user_service | `user.before_create` | Action | Vor User-Erstellung |
|
||||||
|
| 14 | user_service | `user.after_create` | Action | Nach User-Erstellung |
|
||||||
|
| 15 | dms | `dms.before_upload` | Filter | Datei-Upload validieren/modifizieren |
|
||||||
|
|
||||||
|
### 2.4 Tests für Hooks/Filters (4 Std)
|
||||||
|
|
||||||
|
- `test_hooks.py` — HookRegistry Tests
|
||||||
|
- Integrationstests: Plugin registriert Hook, Core-Service löst Hook aus
|
||||||
|
- Filter-Tests: Wert wird korrekt modifiziert
|
||||||
|
- Priority-Tests: Reihenfolge wird eingehalten
|
||||||
|
- Unregister-Tests: Hooks werden bei Deaktivierung entfernt
|
||||||
|
|
||||||
|
### Meilenstein Phase 2:
|
||||||
|
- ✅ `app/core/hooks.py` mit HookRegistry
|
||||||
|
- ✅ 15 Hook-Punkte in Core-Services
|
||||||
|
- ✅ BasePlugin registriert/unregistriert Hooks automatisch
|
||||||
|
- ✅ Tests bestanden
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Phase 3: Plugin-Isolation (Punkt 5)
|
||||||
|
|
||||||
|
**Ziel:** Direkte Cross-Plugin-Imports werden durch Linting verhindert.
|
||||||
|
|
||||||
|
### 3.1 Linting-Regel erstellen (2 Std)
|
||||||
|
|
||||||
|
**Neue Datei: `.ruff/rules/no_cross_plugin_imports.py`**
|
||||||
|
|
||||||
|
```python
|
||||||
|
"""Ruff rule: forbid direct imports from app.plugins.builtins.* (except contracts)."""
|
||||||
|
|
||||||
|
# Erlaubt:
|
||||||
|
# from app.plugins.builtins.contracts import get_contract
|
||||||
|
# from app.plugins.builtins.<name>.contracts import ...
|
||||||
|
#
|
||||||
|
# Verboten:
|
||||||
|
# from app.plugins.builtins.<name>.services import ...
|
||||||
|
# from app.plugins.builtins.<name>.models import ...
|
||||||
|
# from app.plugins.builtins.<name>.routes import ...
|
||||||
|
```
|
||||||
|
|
||||||
|
### 3.2 CI/CD Integration (1 Std)
|
||||||
|
|
||||||
|
- `ruff check` in GitHub Actions / Forgejo CI
|
||||||
|
- Pre-commit Hook für lokale Entwicklung
|
||||||
|
- Fehler bei direkten Cross-Plugin-Imports
|
||||||
|
|
||||||
|
### 3.3 Ausnahmen definieren (1 Std)
|
||||||
|
|
||||||
|
- `conftest.py` — Tests dürfen direkt importieren
|
||||||
|
- `app/plugins/builtins/__init__.py` — Plugin-Discovery
|
||||||
|
- `app/plugins/registry.py` — Registry darf importieren
|
||||||
|
|
||||||
|
### Meilenstein Phase 3:
|
||||||
|
- ✅ Linting-Regel aktiv
|
||||||
|
- ✅ CI/CD prüft bei jedem Commit
|
||||||
|
- ✅ 0 direkte Cross-Plugin-Imports (automatisch erzwungen)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Phase 4: Plugin-Versioning (Punkt 8)
|
||||||
|
|
||||||
|
**Ziel:** Vollständige Versionsverwaltung mit SemVer, Rollback und Kompatibilitäts-Check.
|
||||||
|
|
||||||
|
### 4.1 SemVer-Vergleich (3 Std)
|
||||||
|
|
||||||
|
**Neue Datei: `app/plugins/semver.py`**
|
||||||
|
|
||||||
|
```python
|
||||||
|
"""Semantic version comparison for plugin versions."""
|
||||||
|
|
||||||
|
from dataclasses import dataclass
|
||||||
|
import re
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class SemVer:
|
||||||
|
major: int
|
||||||
|
minor: int
|
||||||
|
patch: int
|
||||||
|
prerelease: str = ""
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def parse(cls, version: str) -> "SemVer":
|
||||||
|
match = re.match(r"(\d+)\.(\d+)\.(\d+)(?:-(.+))?", version)
|
||||||
|
if not match:
|
||||||
|
raise ValueError(f"Invalid semver: {version}")
|
||||||
|
return cls(int(match[1]), int(match[2]), int(match[3]), match[4] or "")
|
||||||
|
|
||||||
|
def __lt__(self, other): ...
|
||||||
|
def __eq__(self, other): ...
|
||||||
|
def __le__(self, other): ...
|
||||||
|
def __gt__(self, other): ...
|
||||||
|
|
||||||
|
def is_breaking_change(self, other: "SemVer") -> bool:
|
||||||
|
return self.major != other.major
|
||||||
|
|
||||||
|
def is_compatible_with(self, min_version: "SemVer") -> bool:
|
||||||
|
return self >= min_version
|
||||||
|
```
|
||||||
|
|
||||||
|
**Änderung in `registry.py`:**
|
||||||
|
```python
|
||||||
|
# VORHER: String-Vergleich
|
||||||
|
if record.version != plugin.manifest.version:
|
||||||
|
|
||||||
|
# NACHHER: SemVer-Vergleich
|
||||||
|
old_ver = SemVer.parse(record.version)
|
||||||
|
new_ver = SemVer.parse(plugin.manifest.version)
|
||||||
|
if old_ver != new_ver:
|
||||||
|
if new_ver < old_ver:
|
||||||
|
# Downgrade — nur mit Rollback-Migration
|
||||||
|
...
|
||||||
|
```
|
||||||
|
|
||||||
|
### 4.2 Rollback-Migrationen (6 Std)
|
||||||
|
|
||||||
|
**Erweiterung des Migration-Systems:**
|
||||||
|
|
||||||
|
```python
|
||||||
|
# MigrationRunner erweitern:
|
||||||
|
async def run_migration_down(self, db, plugin_name, migration_filename):
|
||||||
|
"""Run rollback (down) migration."""
|
||||||
|
# Suche <filename>_down.sql oder parse DOWNGRADE-Block
|
||||||
|
|
||||||
|
async def rollback_to_version(self, db, plugin_name, target_version: str):
|
||||||
|
"""Rollback plugin to a specific version."""
|
||||||
|
# 1. Finde alle Migrationen nach target_version
|
||||||
|
# 2. Führe sie in umgekehrter Reihenfolge aus
|
||||||
|
# 3. Aktualisiere DB-Version
|
||||||
|
```
|
||||||
|
|
||||||
|
**Migration-Datei-Format:**
|
||||||
|
```sql
|
||||||
|
-- 0001_initial.sql
|
||||||
|
-- UP:
|
||||||
|
CREATE TABLE ...;
|
||||||
|
-- DOWN:
|
||||||
|
DROP TABLE ... CASCADE;
|
||||||
|
```
|
||||||
|
|
||||||
|
Oder separate Dateien:
|
||||||
|
- `0001_initial_up.sql`
|
||||||
|
- `0001_initial_down.sql`
|
||||||
|
|
||||||
|
### 4.3 Version-Kompatibilitäts-Check (3 Std)
|
||||||
|
|
||||||
|
**Manifest-Erweiterung:**
|
||||||
|
```python
|
||||||
|
class PluginManifest(BaseModel):
|
||||||
|
# ... existing fields ...
|
||||||
|
min_app_version: str = Field(
|
||||||
|
default="0.0.0",
|
||||||
|
description="Minimum LeoCRM version required"
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
|
**Check bei Installation:**
|
||||||
|
```python
|
||||||
|
async def install(self, db, name):
|
||||||
|
plugin = self.get_plugin(name)
|
||||||
|
# Check app version compatibility
|
||||||
|
app_version = SemVer.parse(settings.app_version)
|
||||||
|
min_version = SemVer.parse(plugin.manifest.min_app_version)
|
||||||
|
if app_version < min_version:
|
||||||
|
raise ValueError(
|
||||||
|
f"Plugin '{name}' requires LeoCRM >= {plugin.manifest.min_app_version}, "
|
||||||
|
f"but current version is {settings.app_version}"
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
|
### 4.4 Update-Benachrichtigung im Frontend (4 Std)
|
||||||
|
|
||||||
|
**Backend:**
|
||||||
|
- `GET /api/v1/plugins/updates` — Liste Plugins mit verfügbarer neuer Version
|
||||||
|
- Vergleich mit Marketplace-Registry (wenn verfügbar) oder lokaler Version
|
||||||
|
|
||||||
|
**Frontend:**
|
||||||
|
- Badge im Plugin-Settings: "Update verfügbar (1.2.0 → 1.3.0)"
|
||||||
|
- Update-Button: Löst Update aus (führt neue Migrationen aus)
|
||||||
|
- Changelog-Anzeige (optional)
|
||||||
|
|
||||||
|
### 4.5 Tests (4 Std)
|
||||||
|
|
||||||
|
- `test_semver.py` — SemVer-Vergleich, Parse, Edge Cases
|
||||||
|
- `test_versioning.py` — Upgrade, Downgrade, Kompatibilitäts-Check
|
||||||
|
- `test_rollback.py` — Rollback-Migrationen
|
||||||
|
- Integrationstests: Version-Update löst Migrationen aus
|
||||||
|
|
||||||
|
### Meilenstein Phase 4:
|
||||||
|
- ✅ SemVer-Vergleich statt String-Vergleich
|
||||||
|
- ✅ Rollback-Migrationen funktionieren
|
||||||
|
- ✅ min_app_version wird geprüft
|
||||||
|
- ✅ Frontend zeigt Update-Benachrichtigungen
|
||||||
|
- ✅ Tests bestanden
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Phase 5: Marketplace-Vorbereitung (Punkt 6)
|
||||||
|
|
||||||
|
**Ziel:** Code so vorbereiten, dass ein Marketplace nur noch gebaut werden muss — ohne Systemänderungen.
|
||||||
|
|
||||||
|
**Wichtig:** Funktioniert auch OHNE Marketplace — Built-in Plugins laufen normal weiter.
|
||||||
|
|
||||||
|
### 5.1 Externe Plugin-Discovery (6 Std)
|
||||||
|
|
||||||
|
**Erweiterung `registry.py`:**
|
||||||
|
|
||||||
|
```python
|
||||||
|
class PluginRegistry:
|
||||||
|
|
||||||
|
def discover_all(self) -> list[str]:
|
||||||
|
"""Discover built-in AND external plugins."""
|
||||||
|
discovered = self.discover_builtins()
|
||||||
|
discovered.extend(self.discover_external())
|
||||||
|
return discovered
|
||||||
|
|
||||||
|
def discover_external(self) -> list[str]:
|
||||||
|
"""Discover plugins from external plugins/ directory."""
|
||||||
|
external_dir = Path(settings.external_plugins_path or "plugins")
|
||||||
|
if not external_dir.exists():
|
||||||
|
return []
|
||||||
|
|
||||||
|
discovered = []
|
||||||
|
for plugin_dir in external_dir.iterdir():
|
||||||
|
if not plugin_dir.is_dir() or plugin_dir.name.startswith("_"):
|
||||||
|
continue
|
||||||
|
# Look for plugin.py or __init__.py with BasePlugin subclass
|
||||||
|
plugin_file = plugin_dir / "plugin.py"
|
||||||
|
if not plugin_file.exists():
|
||||||
|
continue
|
||||||
|
# Import and register
|
||||||
|
import sys
|
||||||
|
sys.path.insert(0, str(external_dir))
|
||||||
|
try:
|
||||||
|
module = importlib.import_module(f"{plugin_dir.name}.plugin")
|
||||||
|
# ... find BasePlugin subclass ...
|
||||||
|
finally:
|
||||||
|
sys.path.remove(str(external_dir))
|
||||||
|
return discovered
|
||||||
|
```
|
||||||
|
|
||||||
|
### 5.2 Plugin-Signatur-Validierung (8 Std)
|
||||||
|
|
||||||
|
**Neue Datei: `app/plugins/signature.py`**
|
||||||
|
|
||||||
|
```python
|
||||||
|
"""Plugin signature verification for external plugins."""
|
||||||
|
|
||||||
|
from pathlib import Path
|
||||||
|
import hashlib
|
||||||
|
import hmac
|
||||||
|
|
||||||
|
# Ed25519 oder HMAC-SHA256 Signatur
|
||||||
|
|
||||||
|
class PluginSignature:
|
||||||
|
"""Verify plugin package signatures."""
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def verify_signature(zip_path: Path, signature: bytes, public_key: bytes) -> bool:
|
||||||
|
"""Verify Ed25519 signature of plugin ZIP."""
|
||||||
|
# 1. Read ZIP content
|
||||||
|
# 2. Compute hash
|
||||||
|
# 3. Verify signature with public key
|
||||||
|
pass
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def compute_hash(zip_path: Path) -> bytes:
|
||||||
|
"""Compute SHA-256 hash of plugin ZIP."""
|
||||||
|
pass
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def sign_plugin(zip_path: Path, private_key: bytes) -> bytes:
|
||||||
|
"""Sign a plugin ZIP (for plugin authors)."""
|
||||||
|
pass
|
||||||
|
```
|
||||||
|
|
||||||
|
### 5.3 Plugin-Allowlist (4 Std)
|
||||||
|
|
||||||
|
**Neue Alembic-Migration: `0044_plugin_allowlist.py`**
|
||||||
|
|
||||||
|
```python
|
||||||
|
# Tabelle: plugin_allowlist
|
||||||
|
# - id: UUID
|
||||||
|
# - plugin_name: VARCHAR(80)
|
||||||
|
# - allowed_hash: VARCHAR(64) # SHA-256
|
||||||
|
# - allowed_signature: TEXT # Ed25519 signature
|
||||||
|
# - added_by: UUID (user)
|
||||||
|
# - created_at: TIMESTAMPTZ
|
||||||
|
# - is_active: BOOLEAN
|
||||||
|
```
|
||||||
|
|
||||||
|
### 5.4 Plugin-Metadata-Erweiterung (4 Std)
|
||||||
|
|
||||||
|
**Manifest-Erweiterung:**
|
||||||
|
```python
|
||||||
|
class PluginManifest(BaseModel):
|
||||||
|
# ... existing fields ...
|
||||||
|
author: str = Field(default="", description="Plugin author")
|
||||||
|
author_email: str = Field(default="", description="Author contact")
|
||||||
|
homepage: str = Field(default="", description="Plugin homepage URL")
|
||||||
|
license: str = Field(default="MIT", description="License")
|
||||||
|
min_app_version: str = Field(default="0.0.0")
|
||||||
|
icon: str = Field(default="", description="Icon URL or emoji")
|
||||||
|
screenshots: list[str] = Field(default_factory=list)
|
||||||
|
changelog: str = Field(default="", description="Changelog URL or text")
|
||||||
|
tags: list[str] = Field(default_factory=list, description="Marketplace categories")
|
||||||
|
price: float = Field(default=0.0, description="Price (0 = free)")
|
||||||
|
```
|
||||||
|
|
||||||
|
### 5.5 Plugin-Download-Endpoint (4 Std)
|
||||||
|
|
||||||
|
**Neue Route: `POST /api/v1/plugins/install-marketplace`**
|
||||||
|
|
||||||
|
```python
|
||||||
|
@router.post("/install-marketplace")
|
||||||
|
async def install_from_marketplace(
|
||||||
|
body: MarketplaceInstall,
|
||||||
|
db: AsyncSession = Depends(get_db),
|
||||||
|
current_user: dict = Depends(require_permission("plugins:configure")),
|
||||||
|
):
|
||||||
|
"""Install a plugin from the marketplace.
|
||||||
|
|
||||||
|
1. Download ZIP from marketplace URL
|
||||||
|
2. Verify signature against allowlist
|
||||||
|
3. Validate manifest
|
||||||
|
4. Check dangerous imports
|
||||||
|
5. Validate migration SQL
|
||||||
|
6. Install (migrations + DB record)
|
||||||
|
7. Activate (optional)
|
||||||
|
"""
|
||||||
|
# 1. Download
|
||||||
|
async with httpx.AsyncClient() as client:
|
||||||
|
resp = await client.get(body.url)
|
||||||
|
zip_data = resp.content
|
||||||
|
|
||||||
|
# 2. Verify signature
|
||||||
|
if not PluginSignature.verify_signature(zip_data, body.signature, public_key):
|
||||||
|
raise HTTPException(403, "Invalid plugin signature")
|
||||||
|
|
||||||
|
# 3-6. Validate and install
|
||||||
|
# ... (reuse existing validation + install logic)
|
||||||
|
```
|
||||||
|
|
||||||
|
### 5.6 Plugin-Update-Check (4 Std)
|
||||||
|
|
||||||
|
```python
|
||||||
|
@router.get("/updates")
|
||||||
|
async def check_plugin_updates(
|
||||||
|
db: AsyncSession = Depends(get_db),
|
||||||
|
current_user: dict = Depends(require_permission("plugins:read")),
|
||||||
|
):
|
||||||
|
"""Check for available plugin updates from marketplace."""
|
||||||
|
# 1. Query marketplace registry (if configured)
|
||||||
|
# 2. Compare versions with installed plugins
|
||||||
|
# 3. Return list of available updates
|
||||||
|
```
|
||||||
|
|
||||||
|
### 5.7 Plugin-Quarantine (4 Std)
|
||||||
|
|
||||||
|
```python
|
||||||
|
async def _quarantine_plugin(zip_path: Path) -> Path:
|
||||||
|
"""Extract plugin to temp dir, validate, then move to plugins/ dir.
|
||||||
|
|
||||||
|
1. Extract to /tmp/plugin_upload_<uuid>/
|
||||||
|
2. Validate manifest exists
|
||||||
|
3. Check dangerous imports
|
||||||
|
4. Validate migration SQL
|
||||||
|
5. Check signature
|
||||||
|
6. If all OK: move to plugins/ dir
|
||||||
|
7. If any fail: delete temp dir, raise error
|
||||||
|
"""
|
||||||
|
```
|
||||||
|
|
||||||
|
### 5.8 Tests (8 Std)
|
||||||
|
|
||||||
|
- `test_marketplace.py` — Download, Verify, Install Flow
|
||||||
|
- `test_signature.py` — Signatur-Validierung
|
||||||
|
- `test_allowlist.py` — Allowlist-Management
|
||||||
|
- `test_quarantine.py` — Quarantine-Validierung
|
||||||
|
- `test_external_discovery.py` — Externe Plugin-Discovery
|
||||||
|
- Integrationstests: Vollständiger Marketplace-Flow
|
||||||
|
|
||||||
|
### Meilenstein Phase 5:
|
||||||
|
- ✅ Externe Plugins können entdeckt werden
|
||||||
|
- ✅ Signatur-Validierung funktioniert
|
||||||
|
- ✅ Allowlist schützt vor nicht autorisierten Plugins
|
||||||
|
- ✅ Marketplace-Endpoint ist vorbereitet (deaktiviert bis Marketplace live)
|
||||||
|
- ✅ Plugin-Upload bleibt deaktiviert
|
||||||
|
- ✅ Built-in Plugins laufen ohne Marketplace
|
||||||
|
- ✅ Tests bestanden
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Zeitplan
|
||||||
|
|
||||||
|
```
|
||||||
|
Woche 1 (Tag 1-5): Phase 1 — Contracts (Teil 1: contracts.py + Imports)
|
||||||
|
Woche 2 (Tag 6-8): Phase 1 — Contracts (Teil 2: Deaktivierung + Tests)
|
||||||
|
(Tag 9-10): Phase 2 — Hooks/Filters-System
|
||||||
|
Woche 3 (Tag 11): Phase 3 — Plugin-Isolation
|
||||||
|
(Tag 12-14): Phase 4 — Plugin-Versioning
|
||||||
|
Woche 4 (Tag 15-19): Phase 5 — Marketplace-Vorbereitung
|
||||||
|
(Tag 20): Puffer / Bugfixes / Doku
|
||||||
|
```
|
||||||
|
|
||||||
|
### Abhängigkeiten
|
||||||
|
```
|
||||||
|
Phase 1 (Contracts) ──→ Phase 3 (Isolation: Linting braucht Contracts als Ausnahme)
|
||||||
|
│
|
||||||
|
└──→ Phase 2 (Hooks: unabhängig, kann parallel)
|
||||||
|
│
|
||||||
|
└──→ Phase 4 (Versioning: braucht Contracts für min_app_version)
|
||||||
|
│
|
||||||
|
└──→ Phase 5 (Marketplace: braucht alles)
|
||||||
|
```
|
||||||
|
|
||||||
|
### Parallelisierungsmöglichkeiten
|
||||||
|
- Phase 1 und Phase 2 können **parallel** laufen (verschiedene Entwickler)
|
||||||
|
- Phase 3 kann erst nach Phase 1 starten
|
||||||
|
- Phase 4 kann nach Phase 1 starten
|
||||||
|
- Phase 5 kann erst nach Phase 1+4 starten
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Risiken
|
||||||
|
|
||||||
|
| Risiko | Wahrscheinlichkeit | Auswirkung | Mitigation |
|
||||||
|
|---|---|---|---|
|
||||||
|
| Contract-Refactoring bricht bestehende Funktionalität | Mittel | Hoch | Tests nach jedem Plugin, schrittweise Migration |
|
||||||
|
| Hooks/Filters verändern Core-Verhalten | Niedrig | Mittel | Tests für alle Hook-Punkte, Priority-System |
|
||||||
|
| Externe Plugin-Discovery hat Sicherheitslücken | Mittel | Hoch | Signatur-Validierung, Quarantine, Allowlist |
|
||||||
|
| SemVer-Parse-Fehler bei bestehenden Versionen | Niedrig | Niedrig | Fallback auf String-Vergleich |
|
||||||
|
| Rollback-Migrationen löschen Daten | Mittel | Hoch | Bestätigungs-Prompt, Backup vor Rollback |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Erfolgskriterien
|
||||||
|
|
||||||
|
Nach Abschluss aller 5 Phasen:
|
||||||
|
|
||||||
|
1. ✅ **0 direkte Cross-Plugin-Imports** (grep-verifiziert, linting-enforced)
|
||||||
|
2. ✅ **Alle 16 Plugins haben contracts.py** mit klarer öffentlicher API
|
||||||
|
3. ✅ **Contracts werden bei Deaktivierung abgemeldet**
|
||||||
|
4. ✅ **Hooks/Filters-System** mit 15+ Hook-Punkten in Core-Services
|
||||||
|
5. ✅ **Plugin-Isolation** durch Linting-Regeln erzwungen
|
||||||
|
6. ✅ **SemVer-Vergleich** statt String-Vergleich
|
||||||
|
7. ✅ **Rollback-Migrationen** für alle Plugins verfügbar
|
||||||
|
8. ✅ **min_app_version** wird bei Installation geprüft
|
||||||
|
9. ✅ **Update-Benachrichtigung** im Frontend
|
||||||
|
10. ✅ **Marketplace-Endpoint** vorbereitet (deaktiviert)
|
||||||
|
11. ✅ **Signatur-Validierung** für externe Plugins
|
||||||
|
12. ✅ **Allowlist** schützt vor nicht autorisierten Plugins
|
||||||
|
13. ✅ **Externe Plugin-Discovery** funktioniert
|
||||||
|
14. ✅ **Alle Tests bestanden**
|
||||||
|
15. ✅ **Built-in Plugins laufen ohne Marketplace**
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Dokumentation
|
||||||
|
|
||||||
|
Nach Abschluss jeder Phase:
|
||||||
|
- `docs/plugin-system/phase-N.md` — Was wurde gemacht, was geändert
|
||||||
|
- `docs/plugin-system/contracts-api.md` — Contract-API Referenz
|
||||||
|
- `docs/plugin-system/hooks-api.md` — Hooks/Filters Referenz
|
||||||
|
- `docs/plugin-system/marketplace-api.md` — Marketplace-API Referenz
|
||||||
|
- `docs/plugin-system/plugin-development-guide.md` — Wie man ein Plugin entwickelt
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
**Dieser Plan ist vollständig. Alle Aufgaben, Aufwände, Abhängigkeiten und Risiken sind erfasst.**
|
||||||
@@ -0,0 +1,125 @@
|
|||||||
|
"""RLS repair + separate DB runtime user.
|
||||||
|
|
||||||
|
Revision ID: 0044
|
||||||
|
Revises: 0043
|
||||||
|
Created: 2026-07-26
|
||||||
|
|
||||||
|
This migration:
|
||||||
|
1. Re-discovers ALL tenant-scoped tables and ensures RLS is enabled
|
||||||
|
with FORCE + WITH CHECK (covers tables added after migration 0028).
|
||||||
|
2. Creates a separate ``crm_runtime`` role with NOSUPERUSER and
|
||||||
|
NOBYPASSRLS so the application cannot bypass RLS.
|
||||||
|
3. Grants only DML permissions (SELECT/INSERT/UPDATE/DELETE) to
|
||||||
|
``crm_runtime`` on all tenant-scoped tables.
|
||||||
|
|
||||||
|
IMPORTANT: After this migration, the application's DATABASE_URL must
|
||||||
|
use ``crm_runtime`` (not the superuser) for API and worker containers.
|
||||||
|
Migration/DDL operations continue to use the owner user (crm_user).
|
||||||
|
"""
|
||||||
|
|
||||||
|
from alembic import op
|
||||||
|
import sqlalchemy as sa
|
||||||
|
import logging
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
revision = "0044"
|
||||||
|
down_revision = "0043"
|
||||||
|
branch_labels = None
|
||||||
|
depends_on = None
|
||||||
|
|
||||||
|
|
||||||
|
def _discover_tenant_tables(conn) -> list[str]:
|
||||||
|
"""Return all table names in the public schema that have a tenant_id column."""
|
||||||
|
result = conn.execute(
|
||||||
|
sa.text(
|
||||||
|
"SELECT table_name FROM information_schema.columns "
|
||||||
|
"WHERE table_schema = 'public' AND column_name = 'tenant_id' "
|
||||||
|
"ORDER BY table_name"
|
||||||
|
)
|
||||||
|
)
|
||||||
|
return [row[0] for row in result]
|
||||||
|
|
||||||
|
|
||||||
|
def _discover_existing_policies(conn, table_name: str) -> list[str]:
|
||||||
|
"""Return all policy names on *table_name* that contain 'tenant' or 'isolation'."""
|
||||||
|
result = conn.execute(
|
||||||
|
sa.text(
|
||||||
|
"SELECT policyname FROM pg_policies "
|
||||||
|
"WHERE schemaname = 'public' AND tablename = :t "
|
||||||
|
"AND (policyname LIKE '%tenant%' OR policyname LIKE '%isolation%')"
|
||||||
|
),
|
||||||
|
{"t": table_name},
|
||||||
|
)
|
||||||
|
return [row[0] for row in result]
|
||||||
|
|
||||||
|
|
||||||
|
def upgrade() -> None:
|
||||||
|
conn = op.get_bind()
|
||||||
|
|
||||||
|
# ── 1. RLS Repair: ensure all tenant tables have RLS + WITH CHECK ──
|
||||||
|
tenant_tables = _discover_tenant_tables(conn)
|
||||||
|
logger.info("RLS repair: discovered %d tenant-scoped tables: %s", len(tenant_tables), tenant_tables)
|
||||||
|
|
||||||
|
for table_name in tenant_tables:
|
||||||
|
# Enable RLS
|
||||||
|
op.execute(f'ALTER TABLE "{table_name}" ENABLE ROW LEVEL SECURITY')
|
||||||
|
# Force RLS (applies to table owner too)
|
||||||
|
op.execute(f'ALTER TABLE "{table_name}" FORCE ROW LEVEL SECURITY')
|
||||||
|
|
||||||
|
# Drop existing tenant policies
|
||||||
|
existing_policies = _discover_existing_policies(conn, table_name)
|
||||||
|
for policy_name in existing_policies:
|
||||||
|
op.execute(f'DROP POLICY IF EXISTS "{policy_name}" ON "{table_name}"')
|
||||||
|
logger.info("Dropped policy %s on %s", policy_name, table_name)
|
||||||
|
|
||||||
|
# Create unified tenant isolation policy with WITH CHECK
|
||||||
|
op.execute(
|
||||||
|
f'CREATE POLICY tenant_isolation ON "{table_name}" '
|
||||||
|
f"USING (tenant_id = current_setting('app.tenant_id', true)::uuid) "
|
||||||
|
f"WITH CHECK (tenant_id = current_setting('app.tenant_id', true)::uuid)"
|
||||||
|
)
|
||||||
|
logger.info("Created/updated tenant_isolation policy on %s (USING + WITH CHECK)", table_name)
|
||||||
|
|
||||||
|
# ── 2. Create crm_runtime role (NOSUPERUSER, NOBYPASSRLS) ──
|
||||||
|
# Use DO block for idempotent creation
|
||||||
|
op.execute(
|
||||||
|
sa.text(
|
||||||
|
"DO $$ "
|
||||||
|
"BEGIN "
|
||||||
|
" IF NOT EXISTS (SELECT 1 FROM pg_roles WHERE rolname = 'crm_runtime') THEN "
|
||||||
|
" CREATE ROLE crm_runtime LOGIN NOSUPERUSER NOCREATEDB NOCREATEROLE "
|
||||||
|
" NOREPLICATION NOBYPASSRLS; "
|
||||||
|
" END IF; "
|
||||||
|
"END $$;"
|
||||||
|
)
|
||||||
|
)
|
||||||
|
logger.info("Ensured crm_runtime role exists (NOSUPERUSER, NOBYPASSRLS)")
|
||||||
|
|
||||||
|
# ── 3. Grant DML permissions to crm_runtime on all tenant tables ──
|
||||||
|
for table_name in tenant_tables:
|
||||||
|
op.execute(
|
||||||
|
f'GRANT SELECT, INSERT, UPDATE, DELETE ON "{table_name}" TO crm_runtime'
|
||||||
|
)
|
||||||
|
|
||||||
|
# Grant usage on sequences (for SERIAL/IDENTITY columns)
|
||||||
|
op.execute("GRANT USAGE, SELECT ON ALL SEQUENCES IN SCHEMA public TO crm_runtime")
|
||||||
|
|
||||||
|
logger.info("Granted DML permissions to crm_runtime on %d tables", len(tenant_tables))
|
||||||
|
|
||||||
|
|
||||||
|
def downgrade() -> None:
|
||||||
|
conn = op.get_bind()
|
||||||
|
|
||||||
|
# Revoke permissions from crm_runtime
|
||||||
|
tenant_tables = _discover_tenant_tables(conn)
|
||||||
|
for table_name in tenant_tables:
|
||||||
|
op.execute(f'REVOKE SELECT, INSERT, UPDATE, DELETE ON "{table_name}" FROM crm_runtime')
|
||||||
|
op.execute("REVOKE USAGE, SELECT ON ALL SEQUENCES IN SCHEMA public FROM crm_runtime")
|
||||||
|
|
||||||
|
# Drop crm_runtime role
|
||||||
|
op.execute("DROP ROLE IF EXISTS crm_runtime")
|
||||||
|
logger.info("Dropped crm_runtime role")
|
||||||
|
|
||||||
|
# Note: RLS policies are NOT reverted here to avoid weakening security.
|
||||||
|
# Migration 0028's downgrade handles the original set of tables.
|
||||||
@@ -0,0 +1,184 @@
|
|||||||
|
"""Forward-repair migration for databases that ran the original 0021/0027.
|
||||||
|
|
||||||
|
Revision ID: 0045
|
||||||
|
Revises: 0044
|
||||||
|
Created: 2026-07-26
|
||||||
|
|
||||||
|
Problem:
|
||||||
|
Migrations 0021 and 0027 were retroactively rewritten to be safer
|
||||||
|
(rename old tables, INSERT ... SELECT, preserve *_old tables).
|
||||||
|
However, Alembic only tracks whether a revision was applied — it does
|
||||||
|
NOT re-run modified revisions. Databases that already had 0021/0027
|
||||||
|
marked as applied will NOT benefit from the safer versions.
|
||||||
|
|
||||||
|
This migration:
|
||||||
|
1. Detects *_old tables (left behind by the rewritten 0021).
|
||||||
|
2. Compares row counts between *_old and current tables.
|
||||||
|
3. Migrates any missing rows from *_old to the current tables.
|
||||||
|
4. Logs discrepancies and aborts on data integrity issues.
|
||||||
|
5. Also repairs entity_type='company' → 'contact' (from rewritten 0027).
|
||||||
|
|
||||||
|
Safe to run on fresh installations (no *_old tables → no-op).
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import logging
|
||||||
|
from typing import Sequence, Union
|
||||||
|
|
||||||
|
from alembic import op
|
||||||
|
import sqlalchemy as sa
|
||||||
|
|
||||||
|
logger = logging.getLogger("alembic.migration.0045")
|
||||||
|
|
||||||
|
revision = "0045"
|
||||||
|
down_revision = "0044"
|
||||||
|
branch_labels: Union[str, Sequence[str], None] = None
|
||||||
|
depends_on: Union[str, Sequence[str], None] = None
|
||||||
|
|
||||||
|
|
||||||
|
def _table_exists(conn, table_name: str) -> bool:
|
||||||
|
"""Check whether *table_name* exists in the public schema."""
|
||||||
|
result = conn.execute(
|
||||||
|
sa.text(
|
||||||
|
"SELECT EXISTS (SELECT 1 FROM information_schema.tables "
|
||||||
|
"WHERE table_schema = 'public' AND table_name = :t)"
|
||||||
|
),
|
||||||
|
{"t": table_name},
|
||||||
|
)
|
||||||
|
return result.scalar()
|
||||||
|
|
||||||
|
|
||||||
|
def _row_count(conn, table_name: str) -> int:
|
||||||
|
"""Return the number of rows in *table_name*, or 0 if it doesn't exist."""
|
||||||
|
if not _table_exists(conn, table_name):
|
||||||
|
return -1
|
||||||
|
result = conn.execute(sa.text(f'SELECT COUNT(*) FROM "{table_name}"'))
|
||||||
|
return result.scalar()
|
||||||
|
|
||||||
|
|
||||||
|
def upgrade() -> None:
|
||||||
|
conn = op.get_bind()
|
||||||
|
|
||||||
|
# ── 1. Check for *_old tables from rewritten migration 0021 ──
|
||||||
|
old_tables = ["contacts_old", "companies_old", "addresses_old"]
|
||||||
|
found_old = [t for t in old_tables if _table_exists(conn, t)]
|
||||||
|
|
||||||
|
if not found_old:
|
||||||
|
logger.info("0045: No *_old tables found — fresh install or already repaired. Skipping.")
|
||||||
|
else:
|
||||||
|
logger.info("0045: Found *_old tables: %s — checking data integrity...", found_old)
|
||||||
|
|
||||||
|
# Compare contacts_old → contacts
|
||||||
|
if _table_exists(conn, "contacts_old"):
|
||||||
|
old_count = _row_count(conn, "contacts_old")
|
||||||
|
new_count = _row_count(conn, "contacts")
|
||||||
|
logger.info("0045: contacts_old=%d rows, contacts=%d rows", old_count, new_count)
|
||||||
|
|
||||||
|
if old_count > new_count:
|
||||||
|
# Migrate missing rows from contacts_old to contacts
|
||||||
|
missing = old_count - new_count
|
||||||
|
logger.warning("0045: %d contacts missing from current table — migrating...", missing)
|
||||||
|
op.execute(
|
||||||
|
sa.text(
|
||||||
|
"INSERT INTO contacts (id, tenant_id, type, first_name, last_name, "
|
||||||
|
"email, phone, is_active, created_at, updated_at) "
|
||||||
|
"SELECT id, tenant_id, type, first_name, last_name, email, phone, "
|
||||||
|
"is_active, created_at, updated_at "
|
||||||
|
"FROM contacts_old "
|
||||||
|
"WHERE id NOT IN (SELECT id FROM contacts)"
|
||||||
|
)
|
||||||
|
)
|
||||||
|
logger.info("0045: Migrated %d missing contacts", missing)
|
||||||
|
|
||||||
|
# Compare companies_old → contacts (type='company')
|
||||||
|
if _table_exists(conn, "companies_old"):
|
||||||
|
old_count = _row_count(conn, "companies_old")
|
||||||
|
new_count = conn.execute(
|
||||||
|
sa.text("SELECT COUNT(*) FROM contacts WHERE type = 'company'")
|
||||||
|
).scalar()
|
||||||
|
logger.info("0045: companies_old=%d rows, contacts(type=company)=%d rows", old_count, new_count)
|
||||||
|
|
||||||
|
if old_count > new_count:
|
||||||
|
missing = old_count - new_count
|
||||||
|
logger.warning("0045: %d companies missing — migrating...", missing)
|
||||||
|
op.execute(
|
||||||
|
sa.text(
|
||||||
|
"INSERT INTO contacts (id, tenant_id, type, first_name, email, phone, "
|
||||||
|
"is_active, created_at, updated_at) "
|
||||||
|
"SELECT id, tenant_id, 'company' as type, name as first_name, email, phone, "
|
||||||
|
"is_active, created_at, updated_at "
|
||||||
|
"FROM companies_old "
|
||||||
|
"WHERE id NOT IN (SELECT id FROM contacts)"
|
||||||
|
)
|
||||||
|
)
|
||||||
|
logger.info("0045: Migrated %d missing companies", missing)
|
||||||
|
|
||||||
|
# ── 2. Repair entity_type='company' → 'contact' (from rewritten 0027) ──
|
||||||
|
# Check if any rows still have entity_type='company' in relevant tables
|
||||||
|
repair_tables = [
|
||||||
|
("entity_links", "entity_type"),
|
||||||
|
("tag_assignments", "entity_type"),
|
||||||
|
("calendar_entry_links", "entity_type"),
|
||||||
|
("addresses", "entity_type"),
|
||||||
|
]
|
||||||
|
|
||||||
|
for table, col in repair_tables:
|
||||||
|
if not _table_exists(conn, table):
|
||||||
|
continue
|
||||||
|
try:
|
||||||
|
result = conn.execute(
|
||||||
|
sa.text(f"SELECT COUNT(*) FROM \"{table}\" WHERE {col} = 'company'")
|
||||||
|
)
|
||||||
|
count = result.scalar()
|
||||||
|
if count > 0:
|
||||||
|
logger.warning("0045: Found %d rows with entity_type='company' in %s — repairing...", count, table)
|
||||||
|
op.execute(
|
||||||
|
sa.text(f"UPDATE \"{table}\" SET {col} = 'contact' WHERE {col} = 'company'")
|
||||||
|
)
|
||||||
|
logger.info("0045: Repaired %d rows in %s", count, table)
|
||||||
|
except Exception as exc:
|
||||||
|
logger.warning("0045: Could not check/repair %s: %s", table, exc)
|
||||||
|
|
||||||
|
# ── 3. Repair mails.company_id → contact_id (from rewritten 0027) ──
|
||||||
|
if _table_exists(conn, "mails"):
|
||||||
|
# Check if company_id column still exists
|
||||||
|
col_result = conn.execute(
|
||||||
|
sa.text(
|
||||||
|
"SELECT EXISTS (SELECT 1 FROM information_schema.columns "
|
||||||
|
"WHERE table_schema = 'public' AND table_name = 'mails' "
|
||||||
|
"AND column_name = 'company_id')"
|
||||||
|
)
|
||||||
|
)
|
||||||
|
has_company_id = col_result.scalar()
|
||||||
|
|
||||||
|
if has_company_id:
|
||||||
|
# Copy company_id → contact_id where contact_id is NULL
|
||||||
|
result = conn.execute(
|
||||||
|
sa.text(
|
||||||
|
"SELECT COUNT(*) FROM mails "
|
||||||
|
"WHERE company_id IS NOT NULL AND contact_id IS NULL"
|
||||||
|
)
|
||||||
|
)
|
||||||
|
count = result.scalar()
|
||||||
|
if count > 0:
|
||||||
|
logger.warning("0045: Found %d mails with company_id but no contact_id — repairing...", count)
|
||||||
|
op.execute(
|
||||||
|
sa.text(
|
||||||
|
"UPDATE mails SET contact_id = company_id "
|
||||||
|
"WHERE company_id IS NOT NULL AND contact_id IS NULL"
|
||||||
|
)
|
||||||
|
)
|
||||||
|
logger.info("0045: Repaired %d mail contact_id references", count)
|
||||||
|
|
||||||
|
# Drop company_id column (safe now that data is copied)
|
||||||
|
op.execute(sa.text("ALTER TABLE mails DROP COLUMN IF EXISTS company_id"))
|
||||||
|
logger.info("0045: Dropped mails.company_id column")
|
||||||
|
|
||||||
|
logger.info("0045: Forward-repair migration completed")
|
||||||
|
|
||||||
|
|
||||||
|
def downgrade() -> None:
|
||||||
|
# This migration is a repair — no meaningful downgrade.
|
||||||
|
# The *_old tables and original data are preserved by migration 0021.
|
||||||
|
logger.info("0045: Downgrade is a no-op (repair migration)")
|
||||||
@@ -57,6 +57,12 @@ class Settings(BaseSettings):
|
|||||||
# CORS
|
# CORS
|
||||||
cors_origins: str = "http://localhost:5173,http://localhost:3000"
|
cors_origins: str = "http://localhost:5173,http://localhost:3000"
|
||||||
|
|
||||||
|
# Frontend URL for email links (password reset, invitations, etc.)
|
||||||
|
frontend_url: str = "http://localhost:5173"
|
||||||
|
|
||||||
|
# Trusted proxy CIDRs (comma-separated) — only these proxies can set X-Forwarded-For
|
||||||
|
trusted_proxy_cidrs: str = ""
|
||||||
|
|
||||||
# Rate Limiting
|
# Rate Limiting
|
||||||
rate_limit_login_max: int = 5
|
rate_limit_login_max: int = 5
|
||||||
rate_limit_login_window: int = 900 # 15 min
|
rate_limit_login_window: int = 900 # 15 min
|
||||||
|
|||||||
+15
-3
@@ -91,9 +91,21 @@ def hash_token(token: str) -> str:
|
|||||||
return hashlib.sha256(token.encode()).hexdigest()
|
return hashlib.sha256(token.encode()).hexdigest()
|
||||||
|
|
||||||
|
|
||||||
def get_redis() -> aioredis.Redis:
|
def verify_ws_origin(websocket) -> bool:
|
||||||
"""Get a Redis client instance."""
|
"""Verify that the WebSocket upgrade request comes from an allowed origin.
|
||||||
return aioredis.from_url(get_settings().redis_url, decode_responses=True)
|
|
||||||
|
Checks the Origin header against the configured CORS origins.
|
||||||
|
Returns True if the origin is allowed or if no CORS restriction is configured.
|
||||||
|
"""
|
||||||
|
from app.config import get_settings
|
||||||
|
settings = get_settings()
|
||||||
|
allowed_origins = settings.cors_origin_list
|
||||||
|
if not allowed_origins:
|
||||||
|
return True
|
||||||
|
origin = websocket.headers.get("origin", "")
|
||||||
|
if not origin:
|
||||||
|
return True # Non-browser clients don't send Origin
|
||||||
|
return origin in allowed_origins
|
||||||
|
|
||||||
|
|
||||||
async def create_session(
|
async def create_session(
|
||||||
|
|||||||
@@ -80,3 +80,72 @@ async def get_job_status(job_id: str) -> dict[str, Any] | None:
|
|||||||
"start_time": job_info.start_time.isoformat() if job_info.start_time else None,
|
"start_time": job_info.start_time.isoformat() if job_info.start_time else None,
|
||||||
"finish_time": job_info.finish_time.isoformat() if job_info.finish_time else None,
|
"finish_time": job_info.finish_time.isoformat() if job_info.finish_time else None,
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
|
# ── Password Reset Email Job ─────────────────────────────────────────────────
|
||||||
|
|
||||||
|
async def send_password_reset_email(
|
||||||
|
ctx: dict[str, Any],
|
||||||
|
*,
|
||||||
|
user_id: str,
|
||||||
|
email: str,
|
||||||
|
raw_token: str,
|
||||||
|
expires_at: str,
|
||||||
|
) -> None:
|
||||||
|
"""Send a password reset email via SMTP.
|
||||||
|
|
||||||
|
This is an ARQ worker function. It is registered with the job registry
|
||||||
|
so the worker can execute it when the auth service enqueues it.
|
||||||
|
"""
|
||||||
|
import aiosmtplib
|
||||||
|
from email.mime.text import MIMEText
|
||||||
|
from email.mime.multipart import MIMEMultipart
|
||||||
|
|
||||||
|
settings = get_settings()
|
||||||
|
|
||||||
|
# Build the reset URL
|
||||||
|
reset_url = f"{settings.frontend_url.rstrip('/')}/reset-password?token={raw_token}"
|
||||||
|
|
||||||
|
# Build the email
|
||||||
|
msg = MIMEMultipart("alternative")
|
||||||
|
msg["From"] = settings.smtp_from_email
|
||||||
|
msg["To"] = email
|
||||||
|
msg["Subject"] = "LeoCRM — Passwort zurücksetzen"
|
||||||
|
|
||||||
|
text_body = (
|
||||||
|
f"Sie haben angefordert, Ihr Passwort zurückzusetzen.\n\n"
|
||||||
|
f"Klicken Sie auf den folgenden Link, um ein neues Passwort zu setzen:\n"
|
||||||
|
f"{reset_url}\n\n"
|
||||||
|
f"Dieser Link ist gültig bis {expires_at}.\n\n"
|
||||||
|
f"Falls Sie diese Anfrage nicht gestellt haben, können Sie diese\n"
|
||||||
|
f"E-Mail ignorieren. Ihr Passwort bleibt unverändert.\n"
|
||||||
|
)
|
||||||
|
html_body = (
|
||||||
|
f"<html><body>"
|
||||||
|
f"<h2>Passwort zurücksetzen</h2>"
|
||||||
|
f"<p>Sie haben angefordert, Ihr Passwort zurückzusetzen.</p>"
|
||||||
|
f"<p><a href=\"{reset_url}\">Passwort jetzt zurücksetzen</a></p>"
|
||||||
|
f"<p>Dieser Link ist gültig bis {expires_at}.</p>"
|
||||||
|
f"<p>Falls Sie diese Anfrage nicht gestellt haben, können Sie diese "
|
||||||
|
f"E-Mail ignorieren. Ihr Passwort bleibt unverändert.</p>"
|
||||||
|
f"</body></html>"
|
||||||
|
)
|
||||||
|
msg.attach(MIMEText(text_body, "plain", "utf-8"))
|
||||||
|
msg.attach(MIMEText(html_body, "html", "utf-8"))
|
||||||
|
|
||||||
|
# Send via SMTP
|
||||||
|
await aiosmtplib.send(
|
||||||
|
msg,
|
||||||
|
hostname=settings.smtp_host,
|
||||||
|
port=settings.smtp_port,
|
||||||
|
username=settings.smtp_username,
|
||||||
|
password=settings.smtp_password,
|
||||||
|
start_tls=settings.smtp_use_tls,
|
||||||
|
)
|
||||||
|
logger.info("Password reset email sent to %s for user %s", email, user_id)
|
||||||
|
|
||||||
|
|
||||||
|
# Register the job so the worker can find it
|
||||||
|
from app.core.job_registry import register_job # noqa: E402
|
||||||
|
|
||||||
|
register_job("send_password_reset_email", send_password_reset_email)
|
||||||
|
|||||||
+62
-21
@@ -14,6 +14,50 @@ from app.config import get_settings
|
|||||||
logger = logging.getLogger(__name__)
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
|
||||||
|
class SecurityHeadersMiddleware(BaseHTTPMiddleware):
|
||||||
|
"""Add security headers to all responses."""
|
||||||
|
|
||||||
|
async def dispatch(self, request: Request, call_next):
|
||||||
|
response = await call_next(request)
|
||||||
|
settings = get_settings()
|
||||||
|
is_production = settings.environment == "production"
|
||||||
|
|
||||||
|
# HSTS — only in production (HTTPS assumed behind proxy)
|
||||||
|
if is_production:
|
||||||
|
response.headers["Strict-Transport-Security"] = (
|
||||||
|
"max-age=63072000; includeSubDomains; preload"
|
||||||
|
)
|
||||||
|
|
||||||
|
# Prevent MIME type sniffing
|
||||||
|
response.headers["X-Content-Type-Options"] = "nosniff"
|
||||||
|
|
||||||
|
# Prevent clickjacking
|
||||||
|
response.headers["X-Frame-Options"] = "DENY"
|
||||||
|
|
||||||
|
# Content Security Policy — restrictive but allows inline styles for SPA
|
||||||
|
response.headers["Content-Security-Policy"] = (
|
||||||
|
"default-src 'self'; "
|
||||||
|
"script-src 'self'; "
|
||||||
|
"style-src 'self' 'unsafe-inline'; "
|
||||||
|
"img-src 'self' data: blob:; "
|
||||||
|
"font-src 'self'; "
|
||||||
|
"connect-src 'self' wss: ws:; "
|
||||||
|
"frame-ancestors 'none'; "
|
||||||
|
"base-uri 'self'; "
|
||||||
|
"form-action 'self'"
|
||||||
|
)
|
||||||
|
|
||||||
|
# Referrer policy
|
||||||
|
response.headers["Referrer-Policy"] = "strict-origin-when-cross-origin"
|
||||||
|
|
||||||
|
# Permissions policy
|
||||||
|
response.headers["Permissions-Policy"] = (
|
||||||
|
"geolocation=(), microphone=(), camera=()"
|
||||||
|
)
|
||||||
|
|
||||||
|
return response
|
||||||
|
|
||||||
|
|
||||||
class CSRFMiddleware(BaseHTTPMiddleware):
|
class CSRFMiddleware(BaseHTTPMiddleware):
|
||||||
"""Validate Origin header and CSRF token on all state-changing requests.
|
"""Validate Origin header and CSRF token on all state-changing requests.
|
||||||
|
|
||||||
@@ -63,30 +107,27 @@ class CSRFMiddleware(BaseHTTPMiddleware):
|
|||||||
content={"detail": "No session for CSRF validation", "code": "csrf_no_session"},
|
content={"detail": "No session for CSRF validation", "code": "csrf_no_session"},
|
||||||
)
|
)
|
||||||
|
|
||||||
# Look up CSRF token from Redis session
|
# Look up CSRF token from Redis session (use singleton)
|
||||||
import redis.asyncio as aioredis
|
from app.core.auth import get_redis
|
||||||
|
|
||||||
redis = aioredis.from_url(settings.redis_url, decode_responses=True)
|
redis = get_redis()
|
||||||
try:
|
raw = await redis.get(f"session:{session_id}")
|
||||||
raw = await redis.get(f"session:{session_id}")
|
if raw is None:
|
||||||
if raw is None:
|
return JSONResponse(
|
||||||
return JSONResponse(
|
status_code=status.HTTP_403_FORBIDDEN,
|
||||||
status_code=status.HTTP_403_FORBIDDEN,
|
content={"detail": "Session expired for CSRF validation", "code": "csrf_session_expired"},
|
||||||
content={"detail": "Session expired for CSRF validation", "code": "csrf_session_expired"},
|
)
|
||||||
)
|
|
||||||
|
|
||||||
session_data = json.loads(raw)
|
session_data = json.loads(raw)
|
||||||
stored_token = session_data.get("csrf_token")
|
stored_token = session_data.get("csrf_token")
|
||||||
|
|
||||||
if not stored_token or stored_token != csrf_header:
|
if not stored_token or stored_token != csrf_header:
|
||||||
return JSONResponse(
|
return JSONResponse(
|
||||||
status_code=status.HTTP_403_FORBIDDEN,
|
status_code=status.HTTP_403_FORBIDDEN,
|
||||||
content={"detail": "CSRF token mismatch", "code": "csrf_token_mismatch"},
|
content={"detail": "CSRF token mismatch", "code": "csrf_token_mismatch"},
|
||||||
)
|
)
|
||||||
|
|
||||||
# Sliding session: also extend TTL on CSRF-validated unsafe requests
|
# Sliding session: also extend TTL on CSRF-validated unsafe requests
|
||||||
await redis.expire(f"session:{session_id}", settings.session_ttl_seconds)
|
await redis.expire(f"session:{session_id}", settings.session_ttl_seconds)
|
||||||
finally:
|
|
||||||
await redis.close()
|
|
||||||
|
|
||||||
return await call_next(request)
|
return await call_next(request)
|
||||||
|
|||||||
@@ -337,11 +337,13 @@ async def get_cached_permissions(
|
|||||||
except Exception:
|
except Exception:
|
||||||
logger.warning(
|
logger.warning(
|
||||||
"Failed to query current permission_version for cache validation "
|
"Failed to query current permission_version for cache validation "
|
||||||
"(user=%s, tenant=%s) — using cached data",
|
"(user=%s, tenant=%s) — invalidating cache and re-resolving",
|
||||||
user_id, tenant_id,
|
user_id, tenant_id,
|
||||||
exc_info=True,
|
exc_info=True,
|
||||||
)
|
)
|
||||||
current_version = cached_version # assume cache is valid if we can't check
|
# Invalidate stale cache — do NOT trust cached permissions on DB error
|
||||||
|
await redis.delete(cache_key)
|
||||||
|
return None # Fall through to re-resolution from DB
|
||||||
|
|
||||||
if cached_version == current_version:
|
if cached_version == current_version:
|
||||||
return data
|
return data
|
||||||
|
|||||||
@@ -1,14 +1,19 @@
|
|||||||
"""Plugin error isolation wrapper."""
|
"""Plugin error isolation wrapper."""
|
||||||
import logging
|
import logging
|
||||||
import functools
|
import functools
|
||||||
|
from fastapi import UploadFile as _UploadFile # noqa: F401 — needed for ForwardRef resolution
|
||||||
from fastapi.responses import JSONResponse
|
from fastapi.responses import JSONResponse
|
||||||
|
|
||||||
logger = logging.getLogger(__name__)
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
|
||||||
def wrap_plugin_route(handler):
|
def wrap_plugin_route(handler):
|
||||||
"""Decorator that isolates plugin route errors and returns structured JSON."""
|
"""Decorator that isolates plugin route errors and returns structured JSON.
|
||||||
@functools.wraps(handler)
|
|
||||||
|
Does NOT use functools.wraps to avoid copying __annotations__ and
|
||||||
|
__wrapped__ — FastAPI would otherwise try to resolve
|
||||||
|
``ForwardRef('UploadFile')`` from the original handler's signature.
|
||||||
|
"""
|
||||||
async def wrapper(*args, **kwargs):
|
async def wrapper(*args, **kwargs):
|
||||||
try:
|
try:
|
||||||
return await handler(*args, **kwargs)
|
return await handler(*args, **kwargs)
|
||||||
@@ -18,4 +23,8 @@ def wrap_plugin_route(handler):
|
|||||||
status_code=500,
|
status_code=500,
|
||||||
content={'detail': f'Plugin error: {exc}', 'code': 'plugin_error'}
|
content={'detail': f'Plugin error: {exc}', 'code': 'plugin_error'}
|
||||||
)
|
)
|
||||||
|
# Preserve identity for debugging but NOT __wrapped__ or __annotations__
|
||||||
|
wrapper.__name__ = getattr(handler, '__name__', 'wrapper')
|
||||||
|
wrapper.__module__ = getattr(handler, '__module__', __name__)
|
||||||
|
wrapper.__qualname__ = getattr(handler, '__qualname__', 'wrapper')
|
||||||
return wrapper
|
return wrapper
|
||||||
|
|||||||
+35
-5
@@ -39,8 +39,38 @@ async def reset_rate_limit(redis_key: str) -> None:
|
|||||||
|
|
||||||
|
|
||||||
def get_client_ip(request: Request) -> str:
|
def get_client_ip(request: Request) -> str:
|
||||||
"""Extract client IP from request."""
|
"""Extract client IP from request.
|
||||||
forwarded = request.headers.get("x-forwarded-for")
|
|
||||||
if forwarded:
|
Only trusts X-Forwarded-For if the direct client is a trusted proxy
|
||||||
return forwarded.split(",")[0].strip()
|
(configured via TRUSTED_PROXY_CIDRS env var, comma-separated CIDRs).
|
||||||
return request.client.host if request.client else "unknown"
|
This prevents IP spoofing to bypass rate limits.
|
||||||
|
"""
|
||||||
|
direct_ip = request.client.host if request.client else "unknown"
|
||||||
|
|
||||||
|
# Check if the direct client is a trusted proxy
|
||||||
|
from app.config import get_settings
|
||||||
|
settings = get_settings()
|
||||||
|
trusted_proxies = getattr(settings, "trusted_proxy_cidrs", "")
|
||||||
|
|
||||||
|
if trusted_proxies:
|
||||||
|
import ipaddress
|
||||||
|
try:
|
||||||
|
client_ip = ipaddress.ip_address(direct_ip)
|
||||||
|
for cidr in trusted_proxies.split(","):
|
||||||
|
cidr = cidr.strip()
|
||||||
|
if cidr and client_ip in ipaddress.ip_network(cidr, strict=False):
|
||||||
|
# Trusted proxy — use X-Forwarded-For
|
||||||
|
forwarded = request.headers.get("x-forwarded-for")
|
||||||
|
if forwarded:
|
||||||
|
# Use the leftmost (original client) IP
|
||||||
|
return forwarded.split(",")[0].strip()
|
||||||
|
# Fallback to X-Real-IP
|
||||||
|
real_ip = request.headers.get("x-real-ip")
|
||||||
|
if real_ip:
|
||||||
|
return real_ip.strip()
|
||||||
|
break
|
||||||
|
except (ValueError, TypeError):
|
||||||
|
pass
|
||||||
|
|
||||||
|
# Not a trusted proxy or no trusted proxies configured — use direct IP
|
||||||
|
return direct_ip
|
||||||
|
|||||||
+9
-4
@@ -74,8 +74,12 @@ class LocalStorage(StorageBackend):
|
|||||||
os.makedirs(self.base_path, exist_ok=True)
|
os.makedirs(self.base_path, exist_ok=True)
|
||||||
|
|
||||||
def _full_path(self, path: str) -> str:
|
def _full_path(self, path: str) -> str:
|
||||||
"""Get the full filesystem path."""
|
"""Get the full filesystem path with path traversal protection."""
|
||||||
return os.path.join(self.base_path, path)
|
# Normalize and ensure the path stays within base_path
|
||||||
|
full = os.path.normpath(os.path.join(self.base_path, path))
|
||||||
|
if not full.startswith(os.path.normpath(self.base_path)):
|
||||||
|
raise ValueError(f"Path traversal detected: {path}")
|
||||||
|
return full
|
||||||
|
|
||||||
async def save(self, path: str, data: bytes) -> str:
|
async def save(self, path: str, data: bytes) -> str:
|
||||||
full_path = self._full_path(path)
|
full_path = self._full_path(path)
|
||||||
@@ -113,8 +117,9 @@ class LocalStorage(StorageBackend):
|
|||||||
return os.path.exists(self._full_path(path))
|
return os.path.exists(self._full_path(path))
|
||||||
|
|
||||||
async def get_url(self, path: str, expires: int = 3600) -> str:
|
async def get_url(self, path: str, expires: int = 3600) -> str:
|
||||||
# Local storage returns the file path for direct access
|
"""Return a relative URL path for the file (not the filesystem path)."""
|
||||||
return self._full_path(path)
|
# Return a relative path that can be served by the app
|
||||||
|
return f"/api/v1/dms/files/{path}"
|
||||||
|
|
||||||
async def list_files(self, prefix: str) -> list[str]:
|
async def list_files(self, prefix: str) -> list[str]:
|
||||||
full_prefix = self._full_path(prefix)
|
full_prefix = self._full_path(prefix)
|
||||||
|
|||||||
+52
-2
@@ -90,11 +90,59 @@ def _get_redis_settings() -> RedisSettings:
|
|||||||
async def on_startup(ctx: dict[str, Any]) -> None:
|
async def on_startup(ctx: dict[str, Any]) -> None:
|
||||||
"""Called when worker starts."""
|
"""Called when worker starts."""
|
||||||
logger.info("ARQ worker starting...")
|
logger.info("ARQ worker starting...")
|
||||||
|
|
||||||
|
# Initialize Redis singleton (same as API lifespan)
|
||||||
|
from app.core.auth import init_redis
|
||||||
|
await init_redis()
|
||||||
|
|
||||||
|
# Initialize service container
|
||||||
|
from app.core.service_container import get_container
|
||||||
|
container = get_container()
|
||||||
|
await container.initialize()
|
||||||
|
|
||||||
|
# Initialize plugin registry and discover built-in plugins
|
||||||
|
from app.plugins.registry import get_registry
|
||||||
|
from app.core.db import get_engine
|
||||||
|
from app.core.event_bus import get_event_bus
|
||||||
|
from app.core.webhook_dispatcher import register_webhook_event_handlers
|
||||||
|
from sqlalchemy import select as sa_select
|
||||||
|
from app.models.plugin import Plugin as PluginModel
|
||||||
|
from sqlalchemy.ext.asyncio import async_sessionmaker
|
||||||
|
|
||||||
|
registry = get_registry()
|
||||||
|
registry.initialize(get_engine(), app=None)
|
||||||
|
registry.discover_builtins()
|
||||||
|
|
||||||
|
event_bus = get_event_bus()
|
||||||
|
async_session = async_sessionmaker(get_engine(), expire_on_commit=False)
|
||||||
|
|
||||||
|
# Activate plugins that are marked active in DB (register event handlers)
|
||||||
|
async with async_session() as db:
|
||||||
|
for name in registry.resolve_load_order():
|
||||||
|
plugin = registry.get_plugin(name)
|
||||||
|
if plugin is None:
|
||||||
|
continue
|
||||||
|
result = await db.execute(
|
||||||
|
sa_select(PluginModel).where(PluginModel.name == name)
|
||||||
|
)
|
||||||
|
plugin_record = result.scalar_one_or_none()
|
||||||
|
if plugin_record is None or not plugin_record.active:
|
||||||
|
continue
|
||||||
|
try:
|
||||||
|
await plugin.on_activate(db, container, event_bus)
|
||||||
|
logger.info(f"Worker: activated plugin {name}")
|
||||||
|
except Exception as exc:
|
||||||
|
logger.error(f"Worker: failed to activate plugin {name}: {exc}")
|
||||||
|
await db.commit()
|
||||||
|
|
||||||
|
# Register webhook dispatcher on the event bus
|
||||||
|
register_webhook_event_handlers(event_bus)
|
||||||
|
logger.info("Worker: webhook event handlers registered")
|
||||||
|
|
||||||
# Register search providers (normally done by app startup)
|
# Register search providers (normally done by app startup)
|
||||||
try:
|
try:
|
||||||
from app.core.db import get_session_factory
|
|
||||||
from app.plugins.builtins.unified_search.provider_registry import auto_register_providers
|
from app.plugins.builtins.unified_search.provider_registry import auto_register_providers
|
||||||
factory = get_session_factory()
|
factory = async_session
|
||||||
async with factory() as db:
|
async with factory() as db:
|
||||||
await auto_register_providers(db)
|
await auto_register_providers(db)
|
||||||
logger.info("Search providers registered for worker")
|
logger.info("Search providers registered for worker")
|
||||||
@@ -105,6 +153,8 @@ async def on_startup(ctx: dict[str, Any]) -> None:
|
|||||||
async def on_shutdown(ctx: dict[str, Any]) -> None:
|
async def on_shutdown(ctx: dict[str, Any]) -> None:
|
||||||
"""Called when worker shuts down."""
|
"""Called when worker shuts down."""
|
||||||
logger.info("ARQ worker shutting down...")
|
logger.info("ARQ worker shutting down...")
|
||||||
|
from app.core.auth import close_redis
|
||||||
|
await close_redis()
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
|
|||||||
+31
@@ -224,3 +224,34 @@ async def get_current_user_id(
|
|||||||
) -> uuid.UUID:
|
) -> uuid.UUID:
|
||||||
"""Extract user_id from current user session."""
|
"""Extract user_id from current user session."""
|
||||||
return uuid.UUID(current_user["user_id"])
|
return uuid.UUID(current_user["user_id"])
|
||||||
|
|
||||||
|
|
||||||
|
def require_active_plugin(plugin_name: str):
|
||||||
|
"""FastAPI dependency factory: require that a plugin is active.
|
||||||
|
|
||||||
|
Returns 403 if the plugin is not active in the permission registry.
|
||||||
|
This allows routes to be registered at app creation time while
|
||||||
|
enforcing activation status at request time.
|
||||||
|
"""
|
||||||
|
async def _check(
|
||||||
|
current_user: dict[str, Any] = Depends(get_current_user),
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
from app.core.permission_registry import get_permission_registry
|
||||||
|
try:
|
||||||
|
registry = get_permission_registry()
|
||||||
|
if not registry.is_plugin_active(plugin_name):
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_403_FORBIDDEN,
|
||||||
|
detail={
|
||||||
|
"detail": f"Plugin '{plugin_name}' is not active",
|
||||||
|
"code": "plugin_inactive",
|
||||||
|
},
|
||||||
|
)
|
||||||
|
except HTTPException:
|
||||||
|
raise
|
||||||
|
except Exception:
|
||||||
|
# If registry not initialized yet, allow request (startup race)
|
||||||
|
pass
|
||||||
|
return current_user
|
||||||
|
|
||||||
|
return _check
|
||||||
|
|||||||
+17
-6
@@ -6,7 +6,7 @@ import time
|
|||||||
import traceback
|
import traceback
|
||||||
from contextlib import asynccontextmanager
|
from contextlib import asynccontextmanager
|
||||||
|
|
||||||
from fastapi import FastAPI, HTTPException, Request
|
from fastapi import FastAPI, HTTPException, Request, Depends
|
||||||
from fastapi.middleware.cors import CORSMiddleware
|
from fastapi.middleware.cors import CORSMiddleware
|
||||||
from fastapi.responses import FileResponse, JSONResponse
|
from fastapi.responses import FileResponse, JSONResponse
|
||||||
from fastapi.staticfiles import StaticFiles
|
from fastapi.staticfiles import StaticFiles
|
||||||
@@ -20,7 +20,7 @@ logger = logging.getLogger(__name__)
|
|||||||
from app.config import get_settings
|
from app.config import get_settings
|
||||||
from app.core.db import close_engine, get_engine
|
from app.core.db import close_engine, get_engine
|
||||||
from app.core.error_codes import ApiError
|
from app.core.error_codes import ApiError
|
||||||
from app.core.middleware import CSRFMiddleware
|
from app.core.middleware import CSRFMiddleware, SecurityHeadersMiddleware
|
||||||
from app.core.monitoring import record_error, record_request
|
from app.core.monitoring import record_error, record_request
|
||||||
from app.core.plugin_error_handler import wrap_plugin_route
|
from app.core.plugin_error_handler import wrap_plugin_route
|
||||||
from app.core.service_container import get_container
|
from app.core.service_container import get_container
|
||||||
@@ -208,6 +208,11 @@ async def lifespan(app: FastAPI):
|
|||||||
init_permission_registry(active_plugin_names)
|
init_permission_registry(active_plugin_names)
|
||||||
logger.info("Permission registry initialized with %d active plugins", len(active_plugin_names))
|
logger.info("Permission registry initialized with %d active plugins", len(active_plugin_names))
|
||||||
|
|
||||||
|
# Register webhook dispatcher on the event bus
|
||||||
|
from app.core.webhook_dispatcher import register_webhook_event_handlers
|
||||||
|
register_webhook_event_handlers(event_bus)
|
||||||
|
logger.info("Webhook event handlers registered")
|
||||||
|
|
||||||
# Register field definitions from active plugins only
|
# Register field definitions from active plugins only
|
||||||
from app.core.permission_registry import get_permission_registry
|
from app.core.permission_registry import get_permission_registry
|
||||||
for name in active_plugin_names:
|
for name in active_plugin_names:
|
||||||
@@ -309,6 +314,7 @@ def create_app() -> FastAPI:
|
|||||||
max_age=3600,
|
max_age=3600,
|
||||||
)
|
)
|
||||||
app.add_middleware(CSRFMiddleware)
|
app.add_middleware(CSRFMiddleware)
|
||||||
|
app.add_middleware(SecurityHeadersMiddleware)
|
||||||
app.add_middleware(RequestLoggingMiddleware)
|
app.add_middleware(RequestLoggingMiddleware)
|
||||||
|
|
||||||
# ── Global exception handler — catch ALL unhandled exceptions ──
|
# ── Global exception handler — catch ALL unhandled exceptions ──
|
||||||
@@ -368,9 +374,10 @@ def create_app() -> FastAPI:
|
|||||||
app.include_router(errors.router)
|
app.include_router(errors.router)
|
||||||
|
|
||||||
# ── Register plugin routes for all built-in plugins ──
|
# ── Register plugin routes for all built-in plugins ──
|
||||||
# Routes are registered here (before app start); activation status
|
# Routes are registered at app creation time so OpenAPI docs are complete.
|
||||||
# is enforced at runtime via require_permission and plugin checks.
|
# Activation status is enforced per-request via require_active_plugin().
|
||||||
import importlib
|
import importlib
|
||||||
|
from app.deps import require_active_plugin
|
||||||
# Discover all built-in plugin modules and register their routes
|
# Discover all built-in plugin modules and register their routes
|
||||||
plugin_modules = [
|
plugin_modules = [
|
||||||
"app.plugins.builtins.tags",
|
"app.plugins.builtins.tags",
|
||||||
@@ -399,6 +406,7 @@ def create_app() -> FastAPI:
|
|||||||
for attr_name in dir(mod):
|
for attr_name in dir(mod):
|
||||||
attr = getattr(mod, attr_name)
|
attr = getattr(mod, attr_name)
|
||||||
if isinstance(attr, type) and hasattr(attr, "manifest") and hasattr(attr.manifest, "routes"):
|
if isinstance(attr, type) and hasattr(attr, "manifest") and hasattr(attr.manifest, "routes"):
|
||||||
|
plugin_name = getattr(attr.manifest, "name", mod_name.split(".")[-1])
|
||||||
for route_def in attr.manifest.routes:
|
for route_def in attr.manifest.routes:
|
||||||
try:
|
try:
|
||||||
router_module = importlib.import_module(route_def.module)
|
router_module = importlib.import_module(route_def.module)
|
||||||
@@ -407,13 +415,16 @@ def create_app() -> FastAPI:
|
|||||||
for route in router.routes:
|
for route in router.routes:
|
||||||
if hasattr(route, 'endpoint'):
|
if hasattr(route, 'endpoint'):
|
||||||
route.endpoint = wrap_plugin_route(route.endpoint)
|
route.endpoint = wrap_plugin_route(route.endpoint)
|
||||||
app.include_router(router)
|
# Add active-plugin check as a router-level dependency
|
||||||
|
app.include_router(
|
||||||
|
router,
|
||||||
|
dependencies=[Depends(require_active_plugin(plugin_name))],
|
||||||
|
)
|
||||||
except Exception as exc:
|
except Exception as exc:
|
||||||
logger.error(f"Failed to register route {route_def.module}.{route_def.router_attr}: {exc}")
|
logger.error(f"Failed to register route {route_def.module}.{route_def.router_attr}: {exc}")
|
||||||
break
|
break
|
||||||
except Exception as exc:
|
except Exception as exc:
|
||||||
logger.error(f"Failed to register plugin routes for {mod_name}: {exc}")
|
logger.error(f"Failed to register plugin routes for {mod_name}: {exc}")
|
||||||
# Do NOT register plugin routes here — lifespan() handles it for active plugins only
|
|
||||||
|
|
||||||
# ── Serve frontend static files (SPA) ──────────────────────────────
|
# ── Serve frontend static files (SPA) ──────────────────────────────
|
||||||
# Mount built frontend assets (JS, CSS, images)
|
# Mount built frontend assets (JS, CSS, images)
|
||||||
|
|||||||
@@ -5,7 +5,7 @@ from __future__ import annotations
|
|||||||
import uuid
|
import uuid
|
||||||
from datetime import datetime
|
from datetime import datetime
|
||||||
|
|
||||||
from sqlalchemy import BigInteger, DateTime, String, Text
|
from sqlalchemy import BigInteger, DateTime, String, Text, func
|
||||||
from sqlalchemy.dialects.postgresql import UUID as PGUUID
|
from sqlalchemy.dialects.postgresql import UUID as PGUUID
|
||||||
from sqlalchemy.orm import Mapped, mapped_column
|
from sqlalchemy.orm import Mapped, mapped_column
|
||||||
|
|
||||||
@@ -39,7 +39,7 @@ class Backup(Base, TenantMixin):
|
|||||||
PGUUID(as_uuid=True), nullable=True
|
PGUUID(as_uuid=True), nullable=True
|
||||||
)
|
)
|
||||||
created_at: Mapped[datetime] = mapped_column(
|
created_at: Mapped[datetime] = mapped_column(
|
||||||
DateTime(timezone=True), nullable=False, default=datetime.utcnow
|
DateTime(timezone=True), nullable=False, server_default=func.now()
|
||||||
)
|
)
|
||||||
completed_at: Mapped[datetime | None] = mapped_column(
|
completed_at: Mapped[datetime | None] = mapped_column(
|
||||||
DateTime(timezone=True), nullable=True, default=None
|
DateTime(timezone=True), nullable=True, default=None
|
||||||
|
|||||||
@@ -35,7 +35,7 @@ class Notification(Base, TenantMixin):
|
|||||||
user_id: Mapped[uuid.UUID] = mapped_column(
|
user_id: Mapped[uuid.UUID] = mapped_column(
|
||||||
PGUUID(as_uuid=True), ForeignKey("users.id", ondelete="CASCADE"), nullable=False, index=True
|
PGUUID(as_uuid=True), ForeignKey("users.id", ondelete="CASCADE"), nullable=False, index=True
|
||||||
)
|
)
|
||||||
type: Mapped[str] = mapped_column(String(20), nullable=False)
|
type: Mapped[str] = mapped_column(String(100), nullable=False)
|
||||||
title: Mapped[str] = mapped_column(String(200), nullable=False)
|
title: Mapped[str] = mapped_column(String(200), nullable=False)
|
||||||
body: Mapped[str | None] = mapped_column(Text, nullable=True)
|
body: Mapped[str | None] = mapped_column(Text, nullable=True)
|
||||||
read_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
|
read_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
|
||||||
@@ -83,5 +83,5 @@ class NotificationPreference(Base, TenantMixin):
|
|||||||
ForeignKey("users.id", ondelete="CASCADE"),
|
ForeignKey("users.id", ondelete="CASCADE"),
|
||||||
nullable=False,
|
nullable=False,
|
||||||
)
|
)
|
||||||
type_key: Mapped[str] = mapped_column(String(20), nullable=False)
|
type_key: Mapped[str] = mapped_column(String(100), nullable=False)
|
||||||
is_enabled: Mapped[bool] = mapped_column(Boolean, nullable=False, default=True)
|
is_enabled: Mapped[bool] = mapped_column(Boolean, nullable=False, default=True)
|
||||||
|
|||||||
@@ -6,6 +6,7 @@ import uuid
|
|||||||
from datetime import datetime
|
from datetime import datetime
|
||||||
|
|
||||||
from sqlalchemy import DateTime, ForeignKey, Index, Integer, String
|
from sqlalchemy import DateTime, ForeignKey, Index, Integer, String
|
||||||
|
from sqlalchemy.dialects.postgresql import JSONB
|
||||||
from sqlalchemy.dialects.postgresql import UUID as PGUUID
|
from sqlalchemy.dialects.postgresql import UUID as PGUUID
|
||||||
from sqlalchemy.orm import Mapped, mapped_column
|
from sqlalchemy.orm import Mapped, mapped_column
|
||||||
|
|
||||||
@@ -51,5 +52,4 @@ class SystemSettings(Base, TenantMixin):
|
|||||||
theme_font_family: Mapped[str] = mapped_column(String(100), nullable=False, default="Inter")
|
theme_font_family: Mapped[str] = mapped_column(String(100), nullable=False, default="Inter")
|
||||||
theme_border_radius: Mapped[str] = mapped_column(String(20), nullable=False, default="0.5rem")
|
theme_border_radius: Mapped[str] = mapped_column(String(20), nullable=False, default="0.5rem")
|
||||||
# Automation plugin settings (JSONB)
|
# Automation plugin settings (JSONB)
|
||||||
from sqlalchemy.dialects.postgresql import JSONB
|
|
||||||
automation_config: Mapped[dict | None] = mapped_column(JSONB, nullable=True)
|
automation_config: Mapped[dict | None] = mapped_column(JSONB, nullable=True)
|
||||||
|
|||||||
+2
-1
@@ -4,6 +4,7 @@ from __future__ import annotations
|
|||||||
|
|
||||||
import uuid
|
import uuid
|
||||||
from datetime import datetime
|
from datetime import datetime
|
||||||
|
from decimal import Decimal
|
||||||
|
|
||||||
from sqlalchemy import Boolean, DateTime, Index, Numeric, String
|
from sqlalchemy import Boolean, DateTime, Index, Numeric, String
|
||||||
from sqlalchemy.dialects.postgresql import UUID as PGUUID
|
from sqlalchemy.dialects.postgresql import UUID as PGUUID
|
||||||
@@ -25,6 +26,6 @@ class TaxRate(Base, TenantMixin):
|
|||||||
PGUUID(as_uuid=True), primary_key=True, default=uuid.uuid4
|
PGUUID(as_uuid=True), primary_key=True, default=uuid.uuid4
|
||||||
)
|
)
|
||||||
name: Mapped[str] = mapped_column(String(100), nullable=False)
|
name: Mapped[str] = mapped_column(String(100), nullable=False)
|
||||||
rate: Mapped[float] = mapped_column(Numeric(5, 2), nullable=False)
|
rate: Mapped[Decimal] = mapped_column(Numeric(5, 2), nullable=False)
|
||||||
is_default: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False)
|
is_default: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False)
|
||||||
country: Mapped[str | None] = mapped_column(String(2), nullable=True)
|
country: Mapped[str | None] = mapped_column(String(2), nullable=True)
|
||||||
|
|||||||
@@ -41,3 +41,8 @@ class Webhook(Base, TenantMixin):
|
|||||||
updated_by: Mapped[uuid.UUID | None] = mapped_column(
|
updated_by: Mapped[uuid.UUID | None] = mapped_column(
|
||||||
PGUUID(as_uuid=True), nullable=True
|
PGUUID(as_uuid=True), nullable=True
|
||||||
)
|
)
|
||||||
|
|
||||||
|
@property
|
||||||
|
def has_secret(self) -> bool:
|
||||||
|
"""Return True if a webhook secret is set (never expose the secret itself)."""
|
||||||
|
return self.secret is not None and len(self.secret) > 0
|
||||||
|
|||||||
@@ -698,7 +698,7 @@ ATTACHMENT_DIR = Path(os.environ.get("STORAGE_PATH", "/data/storage")) / "ai_att
|
|||||||
MAX_ATTACHMENT_SIZE = 25 * 1024 * 1024 # 25MB
|
MAX_ATTACHMENT_SIZE = 25 * 1024 * 1024 # 25MB
|
||||||
|
|
||||||
|
|
||||||
@router.post("/sessions/{session_id}/attachments", dependencies=[Depends(require_permission("ai:write"))])
|
@router.post("/sessions/{session_id}/attachments", response_model=None, dependencies=[Depends(require_permission("ai:write"))])
|
||||||
async def upload_attachment(
|
async def upload_attachment(
|
||||||
session_id: str,
|
session_id: str,
|
||||||
file: UploadFile,
|
file: UploadFile,
|
||||||
|
|||||||
@@ -205,10 +205,14 @@ async def ai_ui_control_ws(websocket: WebSocket):
|
|||||||
Authentication: via session cookie (same pattern as kommunikation plugin).
|
Authentication: via session cookie (same pattern as kommunikation plugin).
|
||||||
"""
|
"""
|
||||||
from app.config import get_settings
|
from app.config import get_settings
|
||||||
from app.core.auth import get_session_data, get_redis
|
from app.core.auth import get_session_data, get_redis, verify_ws_origin
|
||||||
from app.core.service_container import get_container
|
from app.core.service_container import get_container
|
||||||
|
|
||||||
settings = get_settings()
|
settings = get_settings()
|
||||||
|
if not verify_ws_origin(websocket):
|
||||||
|
await websocket.close(code=4003, reason="Origin not allowed")
|
||||||
|
return
|
||||||
|
|
||||||
session_id = websocket.cookies.get(settings.session_cookie_name)
|
session_id = websocket.cookies.get(settings.session_cookie_name)
|
||||||
if not session_id:
|
if not session_id:
|
||||||
await websocket.close(code=4001, reason="Not authenticated")
|
await websocket.close(code=4001, reason="Not authenticated")
|
||||||
|
|||||||
@@ -92,10 +92,20 @@ class AIUIControlWSManager:
|
|||||||
return None
|
return None
|
||||||
|
|
||||||
def store_feedback(self, feedback: dict[str, Any]) -> None:
|
def store_feedback(self, feedback: dict[str, Any]) -> None:
|
||||||
"""Store feedback from frontend after command execution."""
|
"""Store feedback from frontend after command execution.
|
||||||
|
|
||||||
|
Maintains a maximum of 100 feedback entries to prevent memory exhaustion.
|
||||||
|
Oldest entries are removed when the limit is reached.
|
||||||
|
"""
|
||||||
command_id = feedback.get("command_id")
|
command_id = feedback.get("command_id")
|
||||||
if command_id:
|
if command_id:
|
||||||
self._feedback[command_id] = feedback
|
self._feedback[command_id] = feedback
|
||||||
|
# Enforce max feedback entries (FIFO eviction)
|
||||||
|
MAX_FEEDBACK_ENTRIES = 100
|
||||||
|
if len(self._feedback) > MAX_FEEDBACK_ENTRIES:
|
||||||
|
keys_to_remove = list(self._feedback.keys())[:-MAX_FEEDBACK_ENTRIES]
|
||||||
|
for key in keys_to_remove:
|
||||||
|
del self._feedback[key]
|
||||||
logger.debug(f"AI UI Control: feedback stored for command {command_id}: {feedback.get('status')}")
|
logger.debug(f"AI UI Control: feedback stored for command {command_id}: {feedback.get('status')}")
|
||||||
|
|
||||||
def get_feedback(self, command_id: str) -> dict[str, Any] | None:
|
def get_feedback(self, command_id: str) -> dict[str, Any] | None:
|
||||||
|
|||||||
@@ -820,7 +820,7 @@ async def ics_feed(
|
|||||||
return Response(content=ics_content, media_type="text/calendar")
|
return Response(content=ics_content, media_type="text/calendar")
|
||||||
|
|
||||||
|
|
||||||
@router.post("/calendar/import", dependencies=[Depends(require_permission("calendar:write"))])
|
@router.post("/calendar/import", response_model=None, dependencies=[Depends(require_permission("calendar:write"))])
|
||||||
async def import_ics(
|
async def import_ics(
|
||||||
file: UploadFile = File(...),
|
file: UploadFile = File(...),
|
||||||
calendar_id: str | None = None,
|
calendar_id: str | None = None,
|
||||||
|
|||||||
@@ -417,7 +417,7 @@ async def delete_folder(
|
|||||||
# ─── Files ───
|
# ─── Files ───
|
||||||
|
|
||||||
|
|
||||||
@router.post("/files/upload", status_code=status.HTTP_201_CREATED, dependencies=[Depends(require_permission("dms:write"))])
|
@router.post("/files/upload", status_code=status.HTTP_201_CREATED, response_model=None, dependencies=[Depends(require_permission("dms:write"))])
|
||||||
async def upload_file(
|
async def upload_file(
|
||||||
file: UploadFile = File(...),
|
file: UploadFile = File(...),
|
||||||
folder_id: str | None = Form(None),
|
folder_id: str | None = Form(None),
|
||||||
@@ -441,35 +441,35 @@ async def upload_file(
|
|||||||
if folder_result.scalar_one_or_none() is None:
|
if folder_result.scalar_one_or_none() is None:
|
||||||
raise HTTPException(404, detail={"detail": "Folder not found", "code": "not_found"})
|
raise HTTPException(404, detail={"detail": "Folder not found", "code": "not_found"})
|
||||||
|
|
||||||
# Stream file in chunks — avoid loading entire file into RAM
|
# Stream file to storage — avoid loading entire file into RAM
|
||||||
import hashlib
|
import hashlib
|
||||||
CHUNK_SIZE = 1024 * 1024 # 1MB chunks
|
CHUNK_SIZE = 1024 * 1024 # 1MB chunks
|
||||||
sha256 = hashlib.sha256()
|
sha256 = hashlib.sha256()
|
||||||
file_size = 0
|
file_size = 0
|
||||||
chunks: list[bytes] = []
|
|
||||||
|
|
||||||
while True:
|
async def chunk_stream():
|
||||||
chunk = await file.read(CHUNK_SIZE)
|
nonlocal file_size
|
||||||
if not chunk:
|
while True:
|
||||||
break
|
chunk = await file.read(CHUNK_SIZE)
|
||||||
file_size += len(chunk)
|
if not chunk:
|
||||||
if file_size > MAX_FILE_SIZE:
|
break
|
||||||
raise HTTPException(
|
file_size += len(chunk)
|
||||||
413, detail={"detail": "File too large (max 100MB)", "code": "file_too_large"}
|
if file_size > MAX_FILE_SIZE:
|
||||||
)
|
raise HTTPException(
|
||||||
sha256.update(chunk)
|
413, detail={"detail": "File too large (max 100MB)", "code": "file_too_large"}
|
||||||
chunks.append(chunk)
|
)
|
||||||
|
sha256.update(chunk)
|
||||||
content_hash = sha256.hexdigest()
|
yield chunk
|
||||||
|
|
||||||
# Create file record
|
# Create file record
|
||||||
file_id = uuid.uuid4()
|
file_id = uuid.uuid4()
|
||||||
storage_path = _file_storage_path(tenant_id, file_id)
|
storage_path = _file_storage_path(tenant_id, file_id)
|
||||||
|
|
||||||
# Save file via storage backend
|
# Save file via storage backend (true streaming, no RAM accumulation)
|
||||||
storage = get_storage_backend()
|
storage = get_storage_backend()
|
||||||
await storage.save(storage_path, b"".join(chunks))
|
await storage.save_stream(storage_path, chunk_stream())
|
||||||
del chunks # Free memory
|
|
||||||
|
content_hash = sha256.hexdigest()
|
||||||
|
|
||||||
mime_type = file.content_type or "application/octet-stream"
|
mime_type = file.content_type or "application/octet-stream"
|
||||||
|
|
||||||
|
|||||||
@@ -338,7 +338,7 @@ async def delete_msg(
|
|||||||
|
|
||||||
# ─── Attachments ───
|
# ─── Attachments ───
|
||||||
|
|
||||||
@router.post("/messages/{message_id}/attachments", dependencies=[Depends(require_permission("comm:write"))])
|
@router.post("/messages/{message_id}/attachments", response_model=None, dependencies=[Depends(require_permission("comm:write"))])
|
||||||
async def upload_attachment(
|
async def upload_attachment(
|
||||||
message_id: str,
|
message_id: str,
|
||||||
file: UploadFile = File(...),
|
file: UploadFile = File(...),
|
||||||
@@ -471,11 +471,15 @@ async def websocket_endpoint(
|
|||||||
|
|
||||||
Authenticates via session cookie. On connect, subscribes user to all their conversations.
|
Authenticates via session cookie. On connect, subscribes user to all their conversations.
|
||||||
"""
|
"""
|
||||||
# Authenticate via session cookie
|
# Verify Origin header against allowed CORS origins
|
||||||
from app.config import get_settings
|
from app.config import get_settings
|
||||||
from app.core.auth import get_session_data, get_redis
|
from app.core.auth import get_session_data, get_redis, verify_ws_origin
|
||||||
|
|
||||||
settings = get_settings()
|
settings = get_settings()
|
||||||
|
if not verify_ws_origin(websocket):
|
||||||
|
await websocket.close(code=4003, reason="Origin not allowed")
|
||||||
|
return
|
||||||
|
|
||||||
session_id = websocket.cookies.get(settings.session_cookie_name)
|
session_id = websocket.cookies.get(settings.session_cookie_name)
|
||||||
if not session_id:
|
if not session_id:
|
||||||
await websocket.close(code=4001, reason="Not authenticated")
|
await websocket.close(code=4001, reason="Not authenticated")
|
||||||
|
|||||||
@@ -722,7 +722,7 @@ async def sync_folder(
|
|||||||
# ─── Attachment Upload (F-MAIL-04) ───
|
# ─── Attachment Upload (F-MAIL-04) ───
|
||||||
|
|
||||||
|
|
||||||
@router.post("/upload-attachment")
|
@router.post("/upload-attachment", response_model=None)
|
||||||
async def upload_attachment(
|
async def upload_attachment(
|
||||||
file: UploadFile = File(...),
|
file: UploadFile = File(...),
|
||||||
db: AsyncSession = Depends(get_db),
|
db: AsyncSession = Depends(get_db),
|
||||||
|
|||||||
@@ -0,0 +1,49 @@
|
|||||||
|
"""Test sample plugin for LeoCRM plugin system testing."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
from app.plugins.base import BasePlugin
|
||||||
|
from app.plugins.manifest import PluginManifest
|
||||||
|
|
||||||
|
|
||||||
|
class TestSamplePlugin(BasePlugin):
|
||||||
|
"""A sample plugin for testing the plugin lifecycle."""
|
||||||
|
|
||||||
|
manifest = PluginManifest(
|
||||||
|
name="test_sample",
|
||||||
|
version="1.0.0",
|
||||||
|
display_name="Test Sample Plugin",
|
||||||
|
description="A sample plugin for testing install/activate/deactivate/uninstall lifecycle.",
|
||||||
|
dependencies=[],
|
||||||
|
routes=[],
|
||||||
|
events=["contact.created"],
|
||||||
|
migrations=["0001_test_plugin.sql"],
|
||||||
|
permissions=[],
|
||||||
|
)
|
||||||
|
|
||||||
|
def __init__(self) -> None:
|
||||||
|
super().__init__()
|
||||||
|
self.install_called = False
|
||||||
|
self.activate_called = False
|
||||||
|
self.deactivate_called = False
|
||||||
|
self.uninstall_called = False
|
||||||
|
self.event_log: list[dict[str, Any]] = []
|
||||||
|
|
||||||
|
async def on_install(self, db, service_container) -> None:
|
||||||
|
self.install_called = True
|
||||||
|
|
||||||
|
async def on_activate(self, db, service_container, event_bus) -> None:
|
||||||
|
self.activate_called = True
|
||||||
|
await super().on_activate(db, service_container, event_bus)
|
||||||
|
|
||||||
|
async def on_deactivate(self, db, service_container, event_bus) -> None:
|
||||||
|
self.deactivate_called = True
|
||||||
|
await super().on_deactivate(db, service_container, event_bus)
|
||||||
|
|
||||||
|
async def on_uninstall(self, db, service_container) -> None:
|
||||||
|
self.uninstall_called = True
|
||||||
|
|
||||||
|
async def on_contact_created(self, payload: dict[str, Any]) -> None:
|
||||||
|
self.event_log.append({"event": "contact.created", "payload": payload})
|
||||||
@@ -0,0 +1,9 @@
|
|||||||
|
-- Test sample plugin migration: creates a test table with tenant_id
|
||||||
|
CREATE TABLE IF NOT EXISTS test_sample_items (
|
||||||
|
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||||
|
tenant_id UUID NOT NULL REFERENCES tenants(id) ON DELETE CASCADE,
|
||||||
|
name VARCHAR(100) NOT NULL,
|
||||||
|
created_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||||
|
);
|
||||||
|
|
||||||
|
CREATE INDEX IF NOT EXISTS ix_test_sample_items_tenant ON test_sample_items(tenant_id);
|
||||||
@@ -75,7 +75,6 @@ async def login(
|
|||||||
"email": user.email,
|
"email": user.email,
|
||||||
"name": user.name,
|
"name": user.name,
|
||||||
"role": role,
|
"role": role,
|
||||||
"is_system_admin": user.is_system_admin,
|
|
||||||
"tenant_id": str(tenant.id),
|
"tenant_id": str(tenant.id),
|
||||||
"tenant_name": tenant.name,
|
"tenant_name": tenant.name,
|
||||||
"csrf_token": csrf_token,
|
"csrf_token": csrf_token,
|
||||||
|
|||||||
+37
-4
@@ -2,12 +2,14 @@
|
|||||||
|
|
||||||
No auth required so errors can be logged even during logout.
|
No auth required so errors can be logged even during logout.
|
||||||
Rate-limited to 10 requests per minute per IP (simple in-memory implementation).
|
Rate-limited to 10 requests per minute per IP (simple in-memory implementation).
|
||||||
|
Context data is sanitized to prevent leaking sensitive information.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import time
|
import time
|
||||||
import logging
|
import logging
|
||||||
|
import re
|
||||||
from collections import defaultdict, deque
|
from collections import defaultdict, deque
|
||||||
from typing import Any
|
from typing import Any
|
||||||
|
|
||||||
@@ -23,6 +25,13 @@ RATE_LIMIT = 10 # max requests
|
|||||||
RATE_WINDOW = 60 # seconds
|
RATE_WINDOW = 60 # seconds
|
||||||
_ip_requests: dict[str, deque[float]] = defaultdict(deque)
|
_ip_requests: dict[str, deque[float]] = defaultdict(deque)
|
||||||
|
|
||||||
|
# -- Sensitive key patterns to strip from context --
|
||||||
|
_SENSITIVE_PATTERNS = re.compile(
|
||||||
|
r"(?i)(token|password|secret|authorization|cookie|session|api[_-]?key|"
|
||||||
|
r"access[_-]?token|refresh[_-]?token|csrf|bearer|private[_-]?key|"
|
||||||
|
r"client[_-]?secret|x[_-]?auth|x[_-]?api[_-]?key)",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
def _is_rate_limited(client_ip: str) -> bool:
|
def _is_rate_limited(client_ip: str) -> bool:
|
||||||
"""Return True if the IP has exceeded the rate limit."""
|
"""Return True if the IP has exceeded the rate limit."""
|
||||||
@@ -40,6 +49,25 @@ def _is_rate_limited(client_ip: str) -> bool:
|
|||||||
return False
|
return False
|
||||||
|
|
||||||
|
|
||||||
|
def _sanitize_context(context: Any, max_depth: int = 3, _depth: int = 0) -> Any:
|
||||||
|
"""Recursively remove sensitive keys and limit depth/size of context data."""
|
||||||
|
if _depth > max_depth:
|
||||||
|
return "[truncated]"
|
||||||
|
if isinstance(context, dict):
|
||||||
|
sanitized = {}
|
||||||
|
for key, value in context.items():
|
||||||
|
if _SENSITIVE_PATTERNS.search(str(key)):
|
||||||
|
sanitized[key] = "[redacted]"
|
||||||
|
else:
|
||||||
|
sanitized[key] = _sanitize_context(value, max_depth, _depth + 1)
|
||||||
|
return sanitized
|
||||||
|
if isinstance(context, list):
|
||||||
|
return [_sanitize_context(item, max_depth, _depth + 1) for item in context[:20]]
|
||||||
|
if isinstance(context, str) and len(context) > 500:
|
||||||
|
return context[:500] + "[truncated]"
|
||||||
|
return context
|
||||||
|
|
||||||
|
|
||||||
# -- Request schema --
|
# -- Request schema --
|
||||||
|
|
||||||
class ErrorReport(BaseModel):
|
class ErrorReport(BaseModel):
|
||||||
@@ -54,11 +82,16 @@ class ErrorReport(BaseModel):
|
|||||||
@router.post("", status_code=status.HTTP_204_NO_CONTENT)
|
@router.post("", status_code=status.HTTP_204_NO_CONTENT)
|
||||||
async def report_error(error: ErrorReport, request: Request) -> Response:
|
async def report_error(error: ErrorReport, request: Request) -> Response:
|
||||||
"""Log a frontend error. No auth required. Rate-limited per IP."""
|
"""Log a frontend error. No auth required. Rate-limited per IP."""
|
||||||
client_ip = request.client.host if request.client else "unknown"
|
from app.core.rate_limit import get_client_ip
|
||||||
|
|
||||||
|
client_ip = get_client_ip(request)
|
||||||
|
|
||||||
if _is_rate_limited(client_ip):
|
if _is_rate_limited(client_ip):
|
||||||
return Response(status_code=status.HTTP_429_TOO_MANY_REQUESTS)
|
return Response(status_code=status.HTTP_429_TOO_MANY_REQUESTS)
|
||||||
|
|
||||||
|
# Sanitize context to prevent leaking sensitive data
|
||||||
|
sanitized_context = _sanitize_context(error.context) if error.context else None
|
||||||
|
|
||||||
# Log with structured info
|
# Log with structured info
|
||||||
logger.error(
|
logger.error(
|
||||||
"Frontend error reported: %s",
|
"Frontend error reported: %s",
|
||||||
@@ -67,14 +100,14 @@ async def report_error(error: ErrorReport, request: Request) -> Response:
|
|||||||
"error_timestamp": error.timestamp,
|
"error_timestamp": error.timestamp,
|
||||||
"error_message": error.message,
|
"error_message": error.message,
|
||||||
"error_stack": error.stack,
|
"error_stack": error.stack,
|
||||||
"error_context": error.context,
|
"error_context": sanitized_context,
|
||||||
"error_url": error.url,
|
"error_url": error.url,
|
||||||
"error_user_agent": error.userAgent,
|
"error_user_agent": error.userAgent,
|
||||||
"client_ip": client_ip,
|
"client_ip": client_ip,
|
||||||
},
|
},
|
||||||
)
|
)
|
||||||
|
|
||||||
# If forgejo_error_reporter plugin is active, forward error
|
# If forgejo_error_reporter plugin is active, forward sanitized error
|
||||||
try:
|
try:
|
||||||
from app.plugins.builtins.forgejo_error_reporter.service import report_error_to_forgejo
|
from app.plugins.builtins.forgejo_error_reporter.service import report_error_to_forgejo
|
||||||
entry = {
|
entry = {
|
||||||
@@ -83,7 +116,7 @@ async def report_error(error: ErrorReport, request: Request) -> Response:
|
|||||||
"url": error.url,
|
"url": error.url,
|
||||||
"userAgent": error.userAgent,
|
"userAgent": error.userAgent,
|
||||||
"timestamp": error.timestamp,
|
"timestamp": error.timestamp,
|
||||||
"context": error.context,
|
"context": sanitized_context,
|
||||||
}
|
}
|
||||||
await report_error_to_forgejo(entry)
|
await report_error_to_forgejo(entry)
|
||||||
except Exception:
|
except Exception:
|
||||||
|
|||||||
+7
-1
@@ -2,7 +2,8 @@
|
|||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
from pydantic import BaseModel, EmailStr, Field
|
from pydantic import BaseModel, EmailStr, Field, field_validator
|
||||||
|
from app.schemas.user import _validate_password_complexity
|
||||||
|
|
||||||
|
|
||||||
class LoginRequest(BaseModel):
|
class LoginRequest(BaseModel):
|
||||||
@@ -19,6 +20,11 @@ class PasswordResetConfirm(BaseModel):
|
|||||||
token: str = Field(..., min_length=1)
|
token: str = Field(..., min_length=1)
|
||||||
new_password: str = Field(..., min_length=8)
|
new_password: str = Field(..., min_length=8)
|
||||||
|
|
||||||
|
@field_validator("new_password")
|
||||||
|
@classmethod
|
||||||
|
def validate_password(cls, v: str) -> str:
|
||||||
|
return _validate_password_complexity(v)
|
||||||
|
|
||||||
|
|
||||||
class SwitchTenantRequest(BaseModel):
|
class SwitchTenantRequest(BaseModel):
|
||||||
tenant_id: str = Field(..., min_length=1)
|
tenant_id: str = Field(..., min_length=1)
|
||||||
|
|||||||
+23
-1
@@ -2,7 +2,24 @@
|
|||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
from pydantic import BaseModel, EmailStr, Field
|
import re
|
||||||
|
from pydantic import BaseModel, EmailStr, Field, field_validator
|
||||||
|
|
||||||
|
|
||||||
|
def _validate_password_complexity(password: str) -> str:
|
||||||
|
"""Validate password meets complexity requirements.
|
||||||
|
|
||||||
|
Requires: min 8 chars, at least 1 uppercase, 1 lowercase, 1 digit.
|
||||||
|
"""
|
||||||
|
if len(password) < 8:
|
||||||
|
raise ValueError("Password must be at least 8 characters")
|
||||||
|
if not re.search(r"[A-Z]", password):
|
||||||
|
raise ValueError("Password must contain at least one uppercase letter")
|
||||||
|
if not re.search(r"[a-z]", password):
|
||||||
|
raise ValueError("Password must contain at least one lowercase letter")
|
||||||
|
if not re.search(r"\d", password):
|
||||||
|
raise ValueError("Password must contain at least one digit")
|
||||||
|
return password
|
||||||
|
|
||||||
|
|
||||||
class UserCreate(BaseModel):
|
class UserCreate(BaseModel):
|
||||||
@@ -13,6 +30,11 @@ class UserCreate(BaseModel):
|
|||||||
role_id: str | None = Field(default=None, description="UUID of a custom Role", examples=["550e8400-e29b-41d4-a716-446655440000"])
|
role_id: str | None = Field(default=None, description="UUID of a custom Role", examples=["550e8400-e29b-41d4-a716-446655440000"])
|
||||||
is_active: bool = True
|
is_active: bool = True
|
||||||
|
|
||||||
|
@field_validator("password")
|
||||||
|
@classmethod
|
||||||
|
def validate_password(cls, v: str) -> str:
|
||||||
|
return _validate_password_complexity(v)
|
||||||
|
|
||||||
|
|
||||||
class UserUpdate(BaseModel):
|
class UserUpdate(BaseModel):
|
||||||
name: str | None = Field(None, min_length=1, max_length=200)
|
name: str | None = Field(None, min_length=1, max_length=200)
|
||||||
|
|||||||
@@ -39,7 +39,7 @@ class WebhookResponse(BaseModel):
|
|||||||
tenant_id: uuid.UUID
|
tenant_id: uuid.UUID
|
||||||
url: str
|
url: str
|
||||||
events: list[str]
|
events: list[str]
|
||||||
secret: str | None = None
|
has_secret: bool = False # Only indicate if a secret is set, never return it
|
||||||
is_active: bool = True
|
is_active: bool = True
|
||||||
retry_count: int = 3
|
retry_count: int = 3
|
||||||
timeout_seconds: int = 30
|
timeout_seconds: int = 30
|
||||||
|
|||||||
@@ -237,8 +237,7 @@ class AuthService:
|
|||||||
except Exception:
|
except Exception:
|
||||||
logger.warning(
|
logger.warning(
|
||||||
"ARQ enqueue failed for password reset email — "
|
"ARQ enqueue failed for password reset email — "
|
||||||
"raw_token for development: %s",
|
"email will not be sent. Check Redis/ARQ connectivity.",
|
||||||
raw_token,
|
|
||||||
exc_info=True,
|
exc_info=True,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|||||||
@@ -4,10 +4,13 @@ from __future__ import annotations
|
|||||||
|
|
||||||
import hashlib
|
import hashlib
|
||||||
import hmac
|
import hmac
|
||||||
|
import ipaddress
|
||||||
import json
|
import json
|
||||||
import logging
|
import logging
|
||||||
|
import socket
|
||||||
import uuid
|
import uuid
|
||||||
from typing import Any
|
from typing import Any
|
||||||
|
from urllib.parse import urlparse
|
||||||
|
|
||||||
import httpx
|
import httpx
|
||||||
from sqlalchemy import select, delete
|
from sqlalchemy import select, delete
|
||||||
@@ -18,6 +21,45 @@ from app.models.webhook import Webhook
|
|||||||
logger = logging.getLogger(__name__)
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
|
||||||
|
def _validate_webhook_url(url: str) -> None:
|
||||||
|
"""Validate a webhook URL to prevent SSRF attacks.
|
||||||
|
|
||||||
|
Blocks:
|
||||||
|
- Non-http(s) schemes
|
||||||
|
- Private/internal IP ranges (10.x, 172.16-31.x, 192.168.x, 127.x, 169.254.x, ::1)
|
||||||
|
- Hostnames that resolve to private IPs (DNS rebinding)
|
||||||
|
- Redirects (handled by httpx follow_redirects=False)
|
||||||
|
"""
|
||||||
|
parsed = urlparse(url)
|
||||||
|
|
||||||
|
# Protocol allowlist
|
||||||
|
if parsed.scheme not in ("http", "https"):
|
||||||
|
raise ValueError(f"Webhook URL must use http or https, got: {parsed.scheme}")
|
||||||
|
|
||||||
|
hostname = parsed.hostname
|
||||||
|
if not hostname:
|
||||||
|
raise ValueError("Webhook URL has no hostname")
|
||||||
|
|
||||||
|
# Check if hostname is an IP address
|
||||||
|
try:
|
||||||
|
ip = ipaddress.ip_address(hostname)
|
||||||
|
if ip.is_private or ip.is_loopback or ip.is_link_local or ip.is_reserved:
|
||||||
|
raise ValueError(f"Webhook URL points to private/reserved IP: {ip}")
|
||||||
|
except ValueError as exc:
|
||||||
|
if "points to private" in str(exc) or "must use" in str(exc):
|
||||||
|
raise
|
||||||
|
# Not an IP — resolve hostname and check
|
||||||
|
try:
|
||||||
|
resolved = socket.getaddrinfo(hostname, None)
|
||||||
|
for family, _, _, _, sockaddr in resolved:
|
||||||
|
addr = sockaddr[0]
|
||||||
|
ip_obj = ipaddress.ip_address(addr)
|
||||||
|
if ip_obj.is_private or ip_obj.is_loopback or ip_obj.is_link_local or ip_obj.is_reserved:
|
||||||
|
raise ValueError(f"Webhook hostname '{hostname}' resolves to private IP: {addr}")
|
||||||
|
except socket.gaierror:
|
||||||
|
raise ValueError(f"Cannot resolve webhook hostname: {hostname}")
|
||||||
|
|
||||||
|
|
||||||
async def list_webhooks(
|
async def list_webhooks(
|
||||||
db: AsyncSession,
|
db: AsyncSession,
|
||||||
tenant_id: uuid.UUID,
|
tenant_id: uuid.UUID,
|
||||||
@@ -151,8 +193,18 @@ async def send_webhook(
|
|||||||
signature = _sign_payload(body_bytes, webhook.secret)
|
signature = _sign_payload(body_bytes, webhook.secret)
|
||||||
headers["X-Webhook-Signature"] = f"sha256={signature}"
|
headers["X-Webhook-Signature"] = f"sha256={signature}"
|
||||||
|
|
||||||
|
# SSRF protection: validate URL before sending
|
||||||
try:
|
try:
|
||||||
async with httpx.AsyncClient(timeout=webhook.timeout_seconds) as client:
|
_validate_webhook_url(webhook.url)
|
||||||
|
except ValueError as exc:
|
||||||
|
logger.warning("Webhook URL validation failed for %s: %s", webhook.url, exc)
|
||||||
|
return {"success": False, "status_code": None, "error": f"URL validation failed: {exc}"}
|
||||||
|
|
||||||
|
try:
|
||||||
|
async with httpx.AsyncClient(
|
||||||
|
timeout=webhook.timeout_seconds,
|
||||||
|
follow_redirects=False, # Prevent SSRF via redirect
|
||||||
|
) as client:
|
||||||
response = await client.post(
|
response = await client.post(
|
||||||
webhook.url,
|
webhook.url,
|
||||||
content=body_bytes,
|
content=body_bytes,
|
||||||
|
|||||||
+15
-8
@@ -34,8 +34,8 @@ services:
|
|||||||
PGDATA: /var/lib/postgresql/data/pgdata
|
PGDATA: /var/lib/postgresql/data/pgdata
|
||||||
volumes:
|
volumes:
|
||||||
- pgdata:/var/lib/postgresql/data
|
- pgdata:/var/lib/postgresql/data
|
||||||
ports:
|
# No exposed ports — only internal Docker network access
|
||||||
- "5432:5432" # local-only; remove for prod-like runs
|
# For local debugging, uncomment: ports: ["127.0.0.1:5432:5432"]
|
||||||
healthcheck:
|
healthcheck:
|
||||||
test: ["CMD-SHELL", "pg_isready -U ${POSTGRES_USER:-crm_user} -d ${POSTGRES_DB:-crm_db}"]
|
test: ["CMD-SHELL", "pg_isready -U ${POSTGRES_USER:-crm_user} -d ${POSTGRES_DB:-crm_db}"]
|
||||||
interval: 10s
|
interval: 10s
|
||||||
@@ -50,13 +50,13 @@ services:
|
|||||||
image: redis:7-alpine
|
image: redis:7-alpine
|
||||||
container_name: crm-redis
|
container_name: crm-redis
|
||||||
restart: unless-stopped
|
restart: unless-stopped
|
||||||
command: redis-server --requirepass ${REDIS_PASSWORD:-changeme}
|
command: redis-server --requirepass ${REDIS_PASSWORD:?REDIS_PASSWORD is required}
|
||||||
volumes:
|
volumes:
|
||||||
- redisdata:/data
|
- redisdata:/data
|
||||||
ports:
|
# No exposed ports — only internal Docker network access
|
||||||
- "6379:6379" # local-only; remove for prod-like runs
|
# For local debugging, uncomment: ports: ["127.0.0.1:6379:6379"]
|
||||||
healthcheck:
|
healthcheck:
|
||||||
test: ["CMD", "redis-cli", "-a", "${REDIS_PASSWORD:-changeme}", "ping"]
|
test: ["CMD", "redis-cli", "-a", "${REDIS_PASSWORD}", "ping"]
|
||||||
interval: 10s
|
interval: 10s
|
||||||
timeout: 5s
|
timeout: 5s
|
||||||
retries: 5
|
retries: 5
|
||||||
@@ -76,10 +76,14 @@ services:
|
|||||||
redis:
|
redis:
|
||||||
condition: service_healthy
|
condition: service_healthy
|
||||||
environment:
|
environment:
|
||||||
|
# App/worker uses crm_runtime (NOSUPERUSER, NOBYPASSRLS) — RLS enforced
|
||||||
DATABASE_URL: ${DATABASE_URL:?DATABASE_URL is required}
|
DATABASE_URL: ${DATABASE_URL:?DATABASE_URL is required}
|
||||||
REDIS_URL: ${REDIS_URL:-redis://:${REDIS_PASSWORD:-changeme}@redis:6379/0}
|
# Migration/DDL uses crm_user (owner, can bypass RLS for DDL)
|
||||||
|
MIGRATION_DATABASE_URL: ${MIGRATION_DATABASE_URL:-postgresql+asyncpg://${POSTGRES_USER:-crm_user}:${POSTGRES_PASSWORD}@postgres:5432/${POSTGRES_DB:-crm_db}}
|
||||||
|
REDIS_URL: ${REDIS_URL:-redis://:${REDIS_PASSWORD}@redis:6379/0}
|
||||||
SECRET_KEY: ${SECRET_KEY:?SECRET_KEY is required (min 32 chars)}
|
SECRET_KEY: ${SECRET_KEY:?SECRET_KEY is required (min 32 chars)}
|
||||||
CORS_ORIGINS: ${CORS_ORIGINS:-http://localhost:8000,http://localhost:5173}
|
CORS_ORIGINS: ${CORS_ORIGINS:-http://localhost:8000,http://localhost:5173}
|
||||||
|
FRONTEND_URL: ${FRONTEND_URL:-http://localhost:8000}
|
||||||
ENVIRONMENT: ${ENVIRONMENT:-production}
|
ENVIRONMENT: ${ENVIRONMENT:-production}
|
||||||
LOG_LEVEL: ${LOG_LEVEL:-INFO}
|
LOG_LEVEL: ${LOG_LEVEL:-INFO}
|
||||||
SESSION_COOKIE_SECURE: ${SESSION_COOKIE_SECURE:-true}
|
SESSION_COOKIE_SECURE: ${SESSION_COOKIE_SECURE:-true}
|
||||||
@@ -120,9 +124,12 @@ services:
|
|||||||
condition: service_healthy
|
condition: service_healthy
|
||||||
entrypoint: ["/app/worker.sh"]
|
entrypoint: ["/app/worker.sh"]
|
||||||
environment:
|
environment:
|
||||||
|
# Worker uses crm_runtime too — RLS enforced
|
||||||
DATABASE_URL: ${DATABASE_URL:?DATABASE_URL is required}
|
DATABASE_URL: ${DATABASE_URL:?DATABASE_URL is required}
|
||||||
REDIS_URL: ${REDIS_URL:-redis://:${REDIS_PASSWORD:-changeme}@redis:6379/0}
|
MIGRATION_DATABASE_URL: ${MIGRATION_DATABASE_URL:-postgresql+asyncpg://${POSTGRES_USER:-crm_user}:${POSTGRES_PASSWORD}@postgres:5432/${POSTGRES_DB:-crm_db}}
|
||||||
|
REDIS_URL: ${REDIS_URL:-redis://:${REDIS_PASSWORD}@redis:6379/0}
|
||||||
SECRET_KEY: ${SECRET_KEY:?SECRET_KEY is required (min 32 chars)}
|
SECRET_KEY: ${SECRET_KEY:?SECRET_KEY is required (min 32 chars)}
|
||||||
|
FRONTEND_URL: ${FRONTEND_URL:-http://localhost:8000}
|
||||||
ENVIRONMENT: ${ENVIRONMENT:-production}
|
ENVIRONMENT: ${ENVIRONMENT:-production}
|
||||||
LOG_LEVEL: ${LOG_LEVEL:-INFO}
|
LOG_LEVEL: ${LOG_LEVEL:-INFO}
|
||||||
SESSION_COOKIE_SECURE: ${SESSION_COOKIE_SECURE:-true}
|
SESSION_COOKIE_SECURE: ${SESSION_COOKIE_SECURE:-true}
|
||||||
|
|||||||
@@ -15,6 +15,8 @@ import { useAIUIControlStore } from '@/store/aiUIControlStore';
|
|||||||
import { PluginPage } from '@/components/plugins/PluginLoader';
|
import { PluginPage } from '@/components/plugins/PluginLoader';
|
||||||
import { useAuthStore } from '@/store/authStore';
|
import { useAuthStore } from '@/store/authStore';
|
||||||
import { CustomFieldRenderer } from '@/components/contacts/CustomFieldRenderer';
|
import { CustomFieldRenderer } from '@/components/contacts/CustomFieldRenderer';
|
||||||
|
import { TagBadge } from '@/components/tags/TagBadge';
|
||||||
|
import { EntityHistoryPanel } from '@/components/common/EntityHistoryPanel';
|
||||||
import { useCustomFields } from '@/api/customFields';
|
import { useCustomFields } from '@/api/customFields';
|
||||||
import {
|
import {
|
||||||
type UnifiedContact,
|
type UnifiedContact,
|
||||||
@@ -427,7 +429,9 @@ export function ContactDetail({ contact, loading, onEdit, onDeleted }: ContactDe
|
|||||||
<dd>
|
<dd>
|
||||||
{tags.length > 0 ? (
|
{tags.length > 0 ? (
|
||||||
<div className="flex flex-wrap gap-1 mt-1">
|
<div className="flex flex-wrap gap-1 mt-1">
|
||||||
{tags.map((tag) => <Badge key={tag} variant="secondary">{tag}</Badge>)}
|
{tags.map((tag) => (
|
||||||
|
<TagBadge key={tag} tag={{ name: tag, color: 'blue' }} size="sm" />
|
||||||
|
))}
|
||||||
</div>
|
</div>
|
||||||
) : (
|
) : (
|
||||||
<span className="text-sm text-secondary-400">—</span>
|
<span className="text-sm text-secondary-400">—</span>
|
||||||
@@ -453,6 +457,13 @@ export function ContactDetail({ contact, loading, onEdit, onDeleted }: ContactDe
|
|||||||
<HistoryViewer entityType="contact" entityId={contact.id} />
|
<HistoryViewer entityType="contact" entityId={contact.id} />
|
||||||
</Section>
|
</Section>
|
||||||
)}
|
)}
|
||||||
|
|
||||||
|
{/* Entity History Timeline */}
|
||||||
|
{contact.id && (
|
||||||
|
<Section title={t('history.timeline', 'Aktivitäts-Timeline')}>
|
||||||
|
<EntityHistoryPanel entityType="contact" entityId={contact.id} />
|
||||||
|
</Section>
|
||||||
|
)}
|
||||||
</div>
|
</div>
|
||||||
) : (
|
) : (
|
||||||
<div className="p-4">
|
<div className="p-4">
|
||||||
|
|||||||
@@ -13,6 +13,7 @@ import { WindowContainer } from '@/components/window/WindowContainer';
|
|||||||
import { ErrorBoundary } from '@/components/ErrorBoundary';
|
import { ErrorBoundary } from '@/components/ErrorBoundary';
|
||||||
import { WelcomeDialog } from '@/components/onboarding/WelcomeDialog';
|
import { WelcomeDialog } from '@/components/onboarding/WelcomeDialog';
|
||||||
import { OnboardingTour } from '@/components/onboarding/OnboardingTour';
|
import { OnboardingTour } from '@/components/onboarding/OnboardingTour';
|
||||||
|
import { useOnboardingStore } from '@/store/onboardingStore';
|
||||||
|
|
||||||
export function AppShell() {
|
export function AppShell() {
|
||||||
const location = useLocation();
|
const location = useLocation();
|
||||||
@@ -23,6 +24,11 @@ export function AppShell() {
|
|||||||
// AI UI Control — WebSocket-based UI control from AI agents (Phase 4)
|
// AI UI Control — WebSocket-based UI control from AI agents (Phase 4)
|
||||||
useAIUIControl();
|
useAIUIControl();
|
||||||
|
|
||||||
|
// Onboarding state — show WelcomeDialog on first login
|
||||||
|
const completed = useOnboardingStore((s) => s.completed);
|
||||||
|
const skipped = useOnboardingStore((s) => s.skipped);
|
||||||
|
const skip = useOnboardingStore((s) => s.skip);
|
||||||
|
|
||||||
// Hide message sidebar on the AI Assistant page itself
|
// Hide message sidebar on the AI Assistant page itself
|
||||||
const showMessageSidebar = !location.pathname.startsWith('/ai-assistant');
|
const showMessageSidebar = !location.pathname.startsWith('/ai-assistant');
|
||||||
|
|
||||||
@@ -52,7 +58,7 @@ export function AppShell() {
|
|||||||
<AIUIControlIndicator />
|
<AIUIControlIndicator />
|
||||||
<WindowContainer />
|
<WindowContainer />
|
||||||
<ToastContainer />
|
<ToastContainer />
|
||||||
<WelcomeDialog open={false} />
|
<WelcomeDialog open={!completed && !skipped} onClose={skip} />
|
||||||
<OnboardingTour />
|
<OnboardingTour />
|
||||||
</div>
|
</div>
|
||||||
);
|
);
|
||||||
|
|||||||
@@ -1,6 +1,40 @@
|
|||||||
import '@testing-library/jest-dom';
|
import '@testing-library/jest-dom';
|
||||||
import { vi, beforeEach, afterEach, beforeAll } from 'vitest';
|
import { vi, beforeEach, afterEach, beforeAll } from 'vitest';
|
||||||
|
import React from 'react';
|
||||||
import i18n from '@/i18n';
|
import i18n from '@/i18n';
|
||||||
|
import { QueryClient, QueryClientProvider } from '@tanstack/react-query';
|
||||||
|
import { render, renderHook } from '@testing-library/react';
|
||||||
|
|
||||||
|
// ── Global QueryClient for tests ──────────────────────────────────────────
|
||||||
|
// Many tests use hooks from @tanstack/react-query without explicitly wrapping
|
||||||
|
// in a QueryClientProvider. We monkey-patch render/renderHook to automatically
|
||||||
|
// wrap with the provider.
|
||||||
|
function getTestQueryClient() {
|
||||||
|
return new QueryClient({
|
||||||
|
defaultOptions: {
|
||||||
|
queries: { retry: false, staleTime: 0, gcTime: 0 },
|
||||||
|
mutations: { retry: false },
|
||||||
|
},
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
const originalRender = render;
|
||||||
|
const originalRenderHook = renderHook;
|
||||||
|
|
||||||
|
// Override global render to auto-wrap with QueryClientProvider
|
||||||
|
(globalThis as any).render = (ui: React.ReactElement, options?: any) => {
|
||||||
|
const client = getTestQueryClient();
|
||||||
|
const wrapped = React.createElement(QueryClientProvider, { client }, ui);
|
||||||
|
return originalRender(wrapped, options);
|
||||||
|
};
|
||||||
|
|
||||||
|
// Override global renderHook to auto-wrap with QueryClientProvider
|
||||||
|
(globalThis as any).renderHook = (hook: any, options?: any) => {
|
||||||
|
const client = getTestQueryClient();
|
||||||
|
const wrapper = ({ children }: { children: React.ReactNode }) =>
|
||||||
|
React.createElement(QueryClientProvider, { client }, children);
|
||||||
|
return originalRenderHook(hook, { wrapper, ...options });
|
||||||
|
};
|
||||||
|
|
||||||
// Mock matchMedia for jsdom
|
// Mock matchMedia for jsdom
|
||||||
Object.defineProperty(window, 'matchMedia', {
|
Object.defineProperty(window, 'matchMedia', {
|
||||||
|
|||||||
+14
-12
@@ -49,24 +49,25 @@ export function printElement(elementId: string): void {
|
|||||||
'}' +
|
'}' +
|
||||||
'</style>';
|
'</style>';
|
||||||
|
|
||||||
// Single document.write — no mixing with appendChild
|
// Use Blob URL instead of document.write (safer — no XSS risk)
|
||||||
printWindow.document.open();
|
const htmlContent =
|
||||||
printWindow.document.write(
|
|
||||||
'<!DOCTYPE html><html lang="de"><head><meta charset="UTF-8">' +
|
'<!DOCTYPE html><html lang="de"><head><meta charset="UTF-8">' +
|
||||||
'<title>Druckansicht</title>' +
|
'<title>Druckansicht</title>' +
|
||||||
stylesHtml +
|
stylesHtml +
|
||||||
printStyles +
|
printStyles +
|
||||||
'</head><body>' +
|
'</head><body>' +
|
||||||
clone.outerHTML +
|
clone.outerHTML +
|
||||||
'</body></html>',
|
'</body></html>';
|
||||||
);
|
const blob = new Blob([htmlContent], { type: 'text/html' });
|
||||||
printWindow.document.close();
|
const blobUrl = URL.createObjectURL(blob);
|
||||||
|
printWindow.location.href = blobUrl;
|
||||||
|
|
||||||
// Wait for stylesheets to load before printing
|
// Wait for stylesheets to load before printing
|
||||||
printWindow.onload = () => {
|
printWindow.onload = () => {
|
||||||
printWindow.focus();
|
printWindow.focus();
|
||||||
printWindow.print();
|
printWindow.print();
|
||||||
setTimeout(() => {
|
setTimeout(() => {
|
||||||
|
URL.revokeObjectURL(blobUrl);
|
||||||
printWindow.close();
|
printWindow.close();
|
||||||
}, 500);
|
}, 500);
|
||||||
};
|
};
|
||||||
@@ -122,23 +123,24 @@ export function exportToPDF(elementId: string, filename: string): void {
|
|||||||
'}' +
|
'}' +
|
||||||
'</style>';
|
'</style>';
|
||||||
|
|
||||||
// Single document.write — no mixing with appendChild
|
// Use Blob URL instead of document.write (safer — no XSS risk)
|
||||||
printWindow.document.open();
|
const htmlContent =
|
||||||
printWindow.document.write(
|
|
||||||
'<!DOCTYPE html><html lang="de"><head><meta charset="UTF-8">' +
|
'<!DOCTYPE html><html lang="de"><head><meta charset="UTF-8">' +
|
||||||
`<title>${filename}</title>` +
|
`<title>${filename}</title>` +
|
||||||
stylesHtml +
|
stylesHtml +
|
||||||
printStyles +
|
printStyles +
|
||||||
'</head><body>' +
|
'</head><body>' +
|
||||||
clone.outerHTML +
|
clone.outerHTML +
|
||||||
'</body></html>',
|
'</body></html>';
|
||||||
);
|
const blob = new Blob([htmlContent], { type: 'text/html' });
|
||||||
printWindow.document.close();
|
const blobUrl = URL.createObjectURL(blob);
|
||||||
|
printWindow.location.href = blobUrl;
|
||||||
|
|
||||||
printWindow.onload = () => {
|
printWindow.onload = () => {
|
||||||
printWindow.focus();
|
printWindow.focus();
|
||||||
printWindow.print();
|
printWindow.print();
|
||||||
setTimeout(() => {
|
setTimeout(() => {
|
||||||
|
URL.revokeObjectURL(blobUrl);
|
||||||
printWindow.close();
|
printWindow.close();
|
||||||
}, 500);
|
}, 500);
|
||||||
};
|
};
|
||||||
|
|||||||
+29
-4
@@ -3,20 +3,45 @@
|
|||||||
# prestart.sh — Container entrypoint for CRM API container
|
# prestart.sh — Container entrypoint for CRM API container
|
||||||
#
|
#
|
||||||
# Responsibilities:
|
# Responsibilities:
|
||||||
# 1. Run Alembic DB migrations (alembic upgrade head).
|
# 1. Run Alembic DB migrations (alembic upgrade head) using the owner user.
|
||||||
# 2. Start uvicorn as PID 1 (so signals like SIGTERM are forwarded correctly).
|
# 2. Set the crm_runtime password (for RLS-enforced app access).
|
||||||
|
# 3. Start uvicorn as PID 1 (so signals like SIGTERM are forwarded correctly).
|
||||||
#
|
#
|
||||||
# Notes:
|
# Notes:
|
||||||
# - `set -e` ensures the container crashes loudly if migrations fail.
|
# - `set -e` ensures the container crashes loudly if migrations fail.
|
||||||
# - The ARQ worker runs in a separate container (see worker.sh / docker-compose).
|
# - The ARQ worker runs in a separate container (see worker.sh / docker-compose).
|
||||||
|
# - Migrations use MIGRATION_DATABASE_URL (crm_user, owner, can bypass RLS).
|
||||||
|
# - The app uses DATABASE_URL (crm_runtime, NOSUPERUSER, NOBYPASSRLS).
|
||||||
# =============================================================================
|
# =============================================================================
|
||||||
|
|
||||||
set -e
|
set -e
|
||||||
|
|
||||||
echo "[prestart] $(date -u +%Y-%m-%dT%H:%M:%SZ) - Running alembic upgrade head..."
|
# Use MIGRATION_DATABASE_URL for alembic (falls back to DATABASE_URL for backwards compat)
|
||||||
alembic upgrade head
|
export ALEMBIC_DATABASE_URL="${MIGRATION_DATABASE_URL:-$DATABASE_URL}"
|
||||||
|
|
||||||
|
# Configure alembic to use the migration database URL
|
||||||
|
export ALEMBIC_DATABASE_URL
|
||||||
|
|
||||||
|
echo "[prestart] $(date -u +%Y-%m-%dT%H:%M:%SZ) - Running alembic upgrade head (owner user)..."
|
||||||
|
# Temporarily override DATABASE_URL for alembic
|
||||||
|
DATABASE_URL="$ALEMBIC_DATABASE_URL" alembic upgrade head
|
||||||
echo "[prestart] DB migrations completed successfully."
|
echo "[prestart] DB migrations completed successfully."
|
||||||
|
|
||||||
|
# Set crm_runtime password if RUNTIME_DB_PASSWORD is set
|
||||||
|
if [ -n "$RUNTIME_DB_PASSWORD" ]; then
|
||||||
|
echo "[prestart] Setting crm_runtime password..."
|
||||||
|
# Parse the migration DB URL to get psql connection params
|
||||||
|
PGHOST=postgres
|
||||||
|
PGUSER="${POSTGRES_USER:-crm_user}"
|
||||||
|
PGDATABASE="${POSTGRES_DB:-crm_db}"
|
||||||
|
PGPASSWORD="$POSTGRES_PASSWORD"
|
||||||
|
export PGHOST PGUSER PGDATABASE PGPASSWORD
|
||||||
|
psql -c "ALTER ROLE crm_runtime WITH LOGIN PASSWORD '${RUNTIME_DB_PASSWORD}' NOSUPERUSER NOBYPASSRLS;" 2>/dev/null || \
|
||||||
|
echo "[prestart] WARNING: Could not set crm_runtime password (role may not exist yet)"
|
||||||
|
unset PGPASSWORD
|
||||||
|
echo "[prestart] crm_runtime password set."
|
||||||
|
fi
|
||||||
|
|
||||||
echo "[prestart] Starting uvicorn on 0.0.0.0:8000 (workers=1)..."
|
echo "[prestart] Starting uvicorn on 0.0.0.0:8000 (workers=1)..."
|
||||||
exec uvicorn app.main:app \
|
exec uvicorn app.main:app \
|
||||||
--host 0.0.0.0 \
|
--host 0.0.0.0 \
|
||||||
|
|||||||
+32
-9
@@ -106,13 +106,29 @@ def _get_sync_engine():
|
|||||||
|
|
||||||
@pytest.fixture(scope="session", autouse=True)
|
@pytest.fixture(scope="session", autouse=True)
|
||||||
def db_setup():
|
def db_setup():
|
||||||
"""Drop and recreate all tables once per test session."""
|
"""Drop and recreate all tables once per test session.
|
||||||
|
|
||||||
|
Uses SET lock_timeout to prevent deadlocks when multiple test processes
|
||||||
|
try to DROP SCHEMA simultaneously. Falls back to TRUNCATE if DROP fails.
|
||||||
|
"""
|
||||||
sync_eng = _get_sync_engine()
|
sync_eng = _get_sync_engine()
|
||||||
with sync_eng.connect() as conn:
|
with sync_eng.connect() as conn:
|
||||||
# Drop all tables and types
|
# Set a short lock timeout to prevent deadlocks
|
||||||
conn.execute(text("DROP SCHEMA public CASCADE;"))
|
conn.execute(text("SET lock_timeout = '5s';"))
|
||||||
conn.execute(text("CREATE SCHEMA public;"))
|
try:
|
||||||
conn.execute(text("GRANT ALL ON SCHEMA public TO leocrm;"))
|
conn.execute(text("DROP SCHEMA IF EXISTS public CASCADE;"))
|
||||||
|
conn.execute(text("CREATE SCHEMA public;"))
|
||||||
|
conn.execute(text("GRANT ALL ON SCHEMA public TO leocrm;"))
|
||||||
|
except Exception:
|
||||||
|
# If DROP SCHEMA deadlocks, fall back to TRUNCATE all tables
|
||||||
|
conn.rollback()
|
||||||
|
conn.execute(text("SET lock_timeout = '5s';"))
|
||||||
|
conn.execute(text(
|
||||||
|
"DO $$ DECLARE r RECORD; BEGIN "
|
||||||
|
"FOR r IN (SELECT tablename FROM pg_tables WHERE schemaname='public') "
|
||||||
|
"LOOP EXECUTE 'TRUNCATE TABLE public.' || quote_ident(r.tablename) || ' CASCADE'; "
|
||||||
|
"END LOOP; END $$;"
|
||||||
|
))
|
||||||
conn.commit()
|
conn.commit()
|
||||||
sync_eng.dispose()
|
sync_eng.dispose()
|
||||||
|
|
||||||
@@ -126,12 +142,19 @@ def db_setup():
|
|||||||
asyncio.get_event_loop().run_until_complete(_create())
|
asyncio.get_event_loop().run_until_complete(_create())
|
||||||
yield
|
yield
|
||||||
|
|
||||||
# Cleanup after session
|
# Cleanup after session — use TRUNCATE instead of DROP to avoid deadlocks
|
||||||
sync_eng = _get_sync_engine()
|
sync_eng = _get_sync_engine()
|
||||||
with sync_eng.connect() as conn:
|
with sync_eng.connect() as conn:
|
||||||
conn.execute(text("DROP SCHEMA public CASCADE;"))
|
conn.execute(text("SET lock_timeout = '5s';"))
|
||||||
conn.execute(text("CREATE SCHEMA public;"))
|
try:
|
||||||
conn.execute(text("GRANT ALL ON SCHEMA public TO leocrm;"))
|
conn.execute(text(
|
||||||
|
"DO $$ DECLARE r RECORD; BEGIN "
|
||||||
|
"FOR r IN (SELECT tablename FROM pg_tables WHERE schemaname='public') "
|
||||||
|
"LOOP EXECUTE 'TRUNCATE TABLE public.' || quote_ident(r.tablename) || ' CASCADE'; "
|
||||||
|
"END LOOP; END $$;"
|
||||||
|
))
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
conn.commit()
|
conn.commit()
|
||||||
sync_eng.dispose()
|
sync_eng.dispose()
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user