# T10: Monitoring, Performance, Documentation & Environment Config — Implementation Briefing

## Task
Three modules in one task: (1) Monitoring & Alerting, (2) Performance, (3) Documentation.

## Acceptance Criteria (18 ACs)
### Monitoring (AC1-6)
1. GET /api/v1/health → 200 + JSON with status, checks.database, checks.redis, checks.storage, checks.worker
2. GET /api/v1/health mit DB down → 200 + status=degraded, checks.database.status=down
3. GET /api/v1/metrics → 200 + text/plain Prometheus format (admin only, 403 for non-admin)
4. Prometheus metrics include leocrm_http_requests_total, leocrm_db_pool_connections, leocrm_arq_jobs_total
5. Structured JSON log entry for API request: {timestamp, level, event, method, path, status, duration_ms, tenant_id}
6. Error log includes stacktrace and request context

### Performance (AC7-12)
7. scripts/seed_perf_data.py --count 200000 → creates 200k contacts in test DB
8. GET /api/v1/contacts?page=1&page_size=25 with 200k records → response time <500ms
9. GET /api/v1/contacts?search=Mueller with 200k records → response time <500ms
10. page_size > 100 → 422 (max page_size enforced)
11. CSV export >1000 records → ARQ background job started → notification on completion
12. Streaming CSV export: GET /api/v1/contacts/export?format=csv → text/csv stream (not buffered)

### Documentation (AC13-18)
13. README.md exists with Setup-Anleitung (dev + prod), API section, links to admin-guide
14. Swagger UI available at /api/v1/docs (FastAPI auto-gen)
15. docs/admin-guide.md exists with Deploy, Backup, Restore, Env-Vars, Troubleshooting sections
16. docs/api-overview.md exists with endpoint summary table
17. .env.example file exists with all required variables documented (database, redis, smtp, storage, secret_key)
18. Environment-specific config: dev, test, prod profiles documented in docs/admin-guide.md

## Existing Code References
- **Health endpoint:** app/routes/health.py (simple, needs extension)
- **Health test:** tests/test_health.py (basic 200 check)
- **Main app:** app/main.py (FastAPI app with CORS, CSRF middleware)
- **Config:** app/config.py (settings with pydantic-settings)
- **DB:** app/core/db.py (async engine)
- **Routes:** app/routes/ (auth, companies, contacts, etc.)
- **Contacts route:** app/routes/contacts.py (has search param, pagination)
- **Companies route:** app/routes/companies.py (has search, pagination, export)
- **README.md:** exists (basic, needs update with prod setup, API section, admin-guide link)
- **.env.example:** exists (good coverage, may need SMTP/storage additions)
- **docs/:** only requirements docs, needs admin-guide.md + api-overview.md
- **Docker:** docker-compose.yml + Dockerfile exist
- **Coolify:** COOLIFY_SETUP.md exists

## Files to Create
- `app/core/monitoring.py` — Health check extensions, Prometheus metrics, structured logging
- `app/routes/metrics.py` — Prometheus metrics endpoint (admin-only)
- `scripts/seed_perf_data.py` — Performance test data seeding script
- `scripts/check_indexes.py` — DB index verification script
- `tests/test_monitoring.py` — Monitoring tests (health, metrics, logging)
- `tests/test_performance.py` — Performance tests (pagination, export, page_size limit)
- `docs/admin-guide.md` — Admin guide (Deploy, Backup, Restore, Env-Vars, Troubleshooting)
- `docs/api-overview.md` — API endpoint summary

## Files to Modify
- `app/routes/health.py` — Extend health check with DB+Redis+Storage+Worker status
- `app/main.py` — Add metrics route, structured logging middleware, request timing
- `app/routes/contacts.py` — Enforce page_size max 100, add streaming CSV export
- `app/routes/companies.py` — Enforce page_size max 100, add streaming CSV export
- `app/config.py` — Add SMTP/storage config if missing
- `README.md` — Update with prod setup, API section, admin-guide link, env profiles
- `.env.example` — Add SMTP/storage/secret_key vars if missing
- `tests/test_health.py` — Update for extended health check
- `requirements.txt` — Add prometheus-client, structlog if needed

## Dependencies to Add (if not present)
- `prometheus-client>=0.20` (Prometheus metrics)
- `structlog>=24.0` (structured JSON logging)

## Test Spec
- Run: `cd /a0/usr/workdir/dev-projects/leocrm && python -m pytest tests/test_monitoring.py tests/test_performance.py tests/test_health.py -v --tb=short`
- Coverage: `python -m pytest tests/test_monitoring.py --cov=app/core/monitoring --cov-report=term-missing`
- Docs check: `test -f README.md && test -f docs/admin-guide.md && test -f docs/api-overview.md && echo 'Docs OK'`
- Coverage target: 80%
- Follow existing test pattern from tests/test_health.py or tests/test_companies.py

## Forbidden Patterns
- No blocking I/O in async health check — use async DB ping
- No credentials in logs or metrics
- No unbounded pagination — max 100 per page enforced
- No buffering large CSV exports — use StreamingResponse
- No hardcoded config — use app/config.py settings

## Estimated Size
- ~500 lines code (monitoring + scripts + docs)
- ~300+ lines tests