phase11: /health/live + /health/ready endpoints + monitoring docs + Prometheus metrics docs

2026-07-29 23:22:29 +02:00
parent 3eeeeb6173
commit 3e9cfbef8a
2 changed files with 190 additions and 2 deletions
@@ -0,0 +1,134 @@
+# Monitoring und Logging
+
+## Health Endpoints
+
+### `/health/live` — Liveness Probe
+
+Prüft ob der Prozess lebt. Immer 200 wenn der API-Prozess läuft.
+
+```bash
+curl https://crm.media-on.de/health/live
+# → {"status":"alive"}
+```
+
+Verwendung: Kubernetes/Coolify Restart-Entscheidung.
+
+### `/health/ready` — Readiness Probe
+
+Prüft ob die App bereit ist Requests zu bedienen:
+- PostgreSQL Verbindung
+- Redis Verbindung
+- Storage Backend
+- Worker Heartbeat (falls verfügbar)
+
+```bash
+curl https://crm.media-on.de/health/ready
+# → {"status":"ready","checks":{"database":"ok","redis":"ok","storage":"ok"}}
+```
+
+Verwendung: Load Balancer Traffic-Routing. Bei `not_ready` → 503.
+
+### `/api/v1/health` — Full Health
+
+Vollständiger Health Check mit allen Details. Backward compatible.
+
+```bash
+curl https://crm.media-on.de/api/v1/health
+# → {"status":"healthy","version":"1.0.0","checks":{...}}
+```
+
+## Metrics Endpoint
+
+### `/api/v1/metrics` — Prometheus Metrics
+
+Prometheus-kompatible Metriken. Admin-only (403 für non-admin).
+
+```bash
+curl -H "Authorization: Bearer ..." https://crm.media-on.de/api/v1/metrics
+```
+
+Metriken:
+- `leocrm_http_requests_total` — HTTP Request Counter
+- `leocrm_http_request_duration_seconds` — Request Duration Histogram
+- `leocrm_db_pool_size` — DB Connection Pool Size
+- `leocrm_db_pool_checked_out` — Active DB Connections
+- `leocrm_outbox_pending` — Pending Outbox Events
+- `leocrm_outbox_failed` — Failed Outbox Events
+
+## Externes Monitoring
+
+### Empfohlene Tools
+
+- **Uptime Kuma** — Einfache Uptime-Überwachung
+- **Prometheus + Grafana** — Full Metrics Dashboard
+- **Sentry** — Error Tracking
+- **Coolify Health Monitoring** — Eingebaut in Coolify
+
+### Alerting Regeln
+
+| Alert | Bedingung | Severity |
+|-------|-----------|----------|
+| API Down | `/health/live` nicht erreichbar | Critical |
+| API Not Ready | `/health/ready` = not_ready | Warning |
+| DB Down | Health check database = down | Critical |
+| Redis Down | Health check redis = down | Warning |
+| High Error Rate | Fehlerrate > 5% | Warning |
+| Slow Response | p95 > 2s | Warning |
+| DB Pool Exhausted | Pool checked_out = pool_size | Critical |
+| Outbox Backlog | Pending > 100 | Warning |
+| Worker Down | Worker heartbeat fehlt | Critical |
+
+### Coolify Health Check Konfiguration
+
+```yaml
+health_check:
+  type: http
+  path: /health/ready
+  port: 8000
+  interval: 30
+  timeout: 10
+  retries: 3
+  start_period: 15
+```
+
+### Uptime Kuma Setup
+
+1. Monitor URL: `https://crm.media-on.de/health/live`
+2. Expected Status: 200
+3. Interval: 30s
+4. Alert bei: 3 consecutive failures
+
+### Prometheus Scrape Config
+
+```yaml
+scrape_configs:
+  - job_name: 'leocrm'
+    metrics_path: '/api/v1/metrics'
+    static_configs:
+      - targets: ['crm.media-on.de']
+    authorization:
+      type: Bearer
+      credentials: '<admin-token>'
+```
+
+## Strukturiertes Logging
+
+Alle API-Requests werden strukturiert geloggt:
+
+```json
+{
+  "method": "POST",
+  "path": "/api/v1/contacts",
+  "status": 200,
+  "duration_ms": 15.3,
+  "tenant_id": "bfe4d09e-...",
+  "event": "api_request",
+  "level": "info",
+  "timestamp": "2026-07-29T16:00:00Z"
+}
+```
+
+Log-Level:
+- `info` — Normale API-Requests
+- `warning` — Langsame Requests, Permission denied
+- `error` — 500er Fehler, Exceptions