leocrm/docs/admin-guide.md

LeoCRM Admin Guide

Operations manual for LeoCRM administrators: deployment, backup, restore, environment configuration, and troubleshooting.

Table of Contents

1. Deployment
2. Environment Configuration
3. Environment Profiles
4. Backup
5. Restore
6. Monitoring
7. Troubleshooting

---

Deployment

Prerequisites

• Docker 24+ and Docker Compose v2
• PostgreSQL 15+ (or use the included Docker container)
• Redis 7+ (or use the included Docker container)
• A Coolify instance (for managed deployment) or a VPS with Docker

Docker Compose Deployment (Production)

1. Clone the repository:

   git clone <repo-url> leocrm
   cd leocrm
   

2. Copy and configure environment:

   cp .env.example .env
   # Edit .env — set DATABASE_URL, REDIS_URL, SECRET_KEY, CORS_ORIGINS
   nano .env
   

3. Start services:

   docker compose up -d
   

4. Run database migrations:

   docker compose exec api alembic upgrade head
   

5. Create the first admin user:

   docker compose exec api python -c \
     "from app.services.auth_service import bootstrap_first_user; import asyncio; asyncio.run(bootstrap_first_user('admin@example.com', 'SecurePassword123!', 'Admin'))"
   

6. Verify health:

   curl http://localhost:8000/api/v1/health
   

Coolify Deployment

See COOLIFY_SETUP.md for detailed Coolify deployment instructions.

Manual Deployment (without Docker)

1. Install Python 3.11+ and PostgreSQL 15+
2. Create a virtual environment: python3 -m venv .venv && source .venv/bin/activate
3. Install dependencies: pip install -r requirements.txt
4. Configure .env (see Environment Configuration)
5. Run migrations: alembic upgrade head
6. Start the server: uvicorn app.main:app --host 0.0.0.0 --port 8000 --workers 2
7. Start the ARQ worker: arq app.core.jobs.WorkerSettings

---

Environment Configuration

All configuration is managed via environment variables (loaded from .env).

Required Variables

Variable	Description	Example
DATABASE_URL	PostgreSQL async connection URL	postgresql+asyncpg://user:pass@host:5432/dbname
REDIS_URL	Redis connection URL	redis://localhost:6379/0
SECRET_KEY	Secret key for signing (≥32 chars)	Use python3 -c "import secrets; print(secrets.token_urlsafe(48))"
CORS_ORIGINS	Comma-separated allowed origins (no wildcards)	https://crm.example.com

Database Configuration

Variable	Default	Description
DB_POOL_SIZE	10	Connection pool size
DB_MAX_OVERFLOW	20	Max overflow connections
DB_ECHO	false	Echo SQL statements (debug only)

Redis Configuration

Variable	Default	Description
REDIS_URL	redis://localhost:6379/0	Redis connection URL
SESSION_TTL_SECONDS	28800	Session lifetime (8 hours)

SMTP / Email Configuration

Variable	Default	Description
SMTP_HOST	localhost	SMTP server hostname
SMTP_PORT	587	SMTP server port
SMTP_USERNAME	(none)	SMTP username (if auth required)
SMTP_PASSWORD	(none)	SMTP password (if auth required)
SMTP_FROM_EMAIL	noreply@leocrm.local	From email address
SMTP_USE_TLS	true	Use TLS for SMTP connection

Storage Configuration

Variable	Default	Description
STORAGE_PATH	/tmp	File storage path (DMS, uploads)

Security Configuration

Variable	Default	Description
BCRYPT_ROUNDS	12	Password hashing cost factor
SESSION_COOKIE_SECURE	false	Set true in production (HTTPS only)
SESSION_COOKIE_SAMESITE	strict	SameSite cookie policy
SESSION_COOKIE_HTTPONLY	true	HttpOnly cookie flag
PASSWORD_RESET_EXPIRY_HOURS	1	Password reset token lifetime

Rate Limiting

Variable	Default	Description
RATE_LIMIT_LOGIN_MAX	5	Max login attempts per window
RATE_LIMIT_LOGIN_WINDOW	900	Login window (seconds, 15 min)
RATE_LIMIT_GENERAL_MAX	60	General API rate limit
RATE_LIMIT_GENERAL_WINDOW	60	General window (seconds, 1 min)

---

Environment Profiles

LeoCRM supports three environment profiles via the ENVIRONMENT variable.

Development (ENVIRONMENT=development)

• Database: Local PostgreSQL or Docker container
• Redis: Local instance
• Logging: DEBUG level (verbose)
• CORS: http://localhost:5173,http://localhost:3000
• Session cookie secure: false
• DB Echo: false (set true for SQL debugging)
• Auto-reload: uvicorn app.main:app --reload --port 8000

Testing (ENVIRONMENT=testing)

• Database: leocrm_test database (separate from dev/prod)
• Redis: Local instance (flushed between tests)
• Logging: WARNING level (minimal output)
• Test fixtures: Auto-seeded tenants, users, and test data
• Run tests: pytest -v --tb=short

Production (ENVIRONMENT=production)

• Database: Managed PostgreSQL (e.g., Coolify managed DB)
• Redis: Separate Redis container or managed service
• Logging: INFO level (structured JSON via structlog)
• CORS: Production domain only (e.g., https://crm.example.com)
• Session cookie secure: true (HTTPS only)
• DB Echo: false
• Workers: uvicorn app.main:app --host 0.0.0.0 --port 8000 --workers 2
• ARQ Worker: arq app.core.jobs.WorkerSettings
• Secret key: Must be a secure random string (≥32 chars)

---

Backup

Database Backup

# Full database dump (recommended daily)
pg_dump -U leocrm -h localhost leocrm > backup_$(date +%Y%m%d).sql

# Compressed backup
pg_dump -U leocrm -h localhost leocrm | gzip > backup_$(date +%Y%m%d).sql.gz

Automated Backup (Cron)

Add to crontab for daily backup at 2 AM:

0 2 * * * pg_dump -U leocrm -h localhost leocrm | gzip > /backups/leocrm_$(date +\%Y\%m\%d).sql.gz

File Storage Backup

# Backup the storage directory
tar -czf storage_$(date +%Y%m%d).tar.gz /data/storage/

Redis Backup

# Save Redis snapshot
redis-cli SAVE
cp /var/lib/redis/dump.rdb /backups/redis_$(date +%Y%m%d).rdb

---

Restore

Database Restore

# Restore from SQL dump
psql -U leocrm -h localhost leocrm < backup_20260101.sql

# Restore from compressed backup
gunzip -c backup_20260101.sql.gz | psql -U leocrm -h localhost leocrm

File Storage Restore

# Restore storage directory
tar -xzf storage_20260101.tar.gz -C /data/

Redis Restore

# Stop Redis, replace dump file, start Redis
systemctl stop redis
cp /backups/redis_20260101.rdb /var/lib/redis/dump.rdb
systemctl start redis

---

Monitoring

Health Endpoint

curl http://localhost:8000/api/v1/health

Returns JSON with overall status and individual checks:

• database — PostgreSQL connectivity
• redis — Redis connectivity
• storage — Storage directory writability
• worker — ARQ worker queue status

Status values: healthy (all checks up) or degraded (one or more checks down).

Prometheus Metrics

# Requires admin authentication
curl -H "Cookie: leocrm_session=<session_id>" http://localhost:8000/api/v1/metrics

Available metrics:

• leocrm_http_requests_total — Total HTTP requests (by method, path, status)
• leocrm_http_request_duration_seconds — Request duration histogram
• leocrm_db_pool_connections — Database connection pool size
• leocrm_arq_jobs_total — Total ARQ background jobs (by function, status)

Structured Logging

LeoCRM uses structlog for structured JSON logging. Logs include:

• timestamp — ISO timestamp
• level — Log level (INFO, ERROR, etc.)
• event — Event name (e.g., api_request)
• method — HTTP method
• path — Request path
• status — HTTP status code
• duration_ms — Request duration in milliseconds
• tenant_id — Tenant identifier (when available)

Error logs additionally include:

• error — Error message
• traceback — Full stacktrace

Performance Testing

# Seed 200k contacts for performance testing
python scripts/seed_perf_data.py --count 200000

# Verify database indexes
python scripts/check_indexes.py

---

Troubleshooting

Database Connection Issues

Symptom: Health check reports database: down

1. Verify PostgreSQL is running: systemctl status postgresql
2. Check connection string in .env: DATABASE_URL=postgresql+asyncpg://...
3. Test connection: psql -U leocrm -h localhost leocrm
4. Check pool settings: DB_POOL_SIZE and DB_MAX_OVERFLOW

Redis Connection Issues

Symptom: Health check reports redis: down, sessions not persisting

1. Verify Redis is running: systemctl status redis or redis-cli ping
2. Check REDIS_URL in .env
3. Check Redis logs for errors

Storage Issues

Symptom: Health check reports storage: down, file uploads failing

1. Verify storage path exists: ls -la $STORAGE_PATH
2. Check write permissions: touch $STORAGE_PATH/test && rm $STORAGE_PATH/test
3. Update STORAGE_PATH in .env if needed

Worker Issues

Symptom: Background jobs not processing, health check reports worker: down

1. Verify ARQ worker is running: ps aux | grep arq
2. Start worker: arq app.core.jobs.WorkerSettings
3. Check Redis queue: redis-cli LLEN arq:queue

Performance Issues

Symptom: Slow API responses (>500ms)

1. Run python scripts/check_indexes.py — verify all indexes exist
2. Check database pool size: increase DB_POOL_SIZE if needed
3. Seed test data: python scripts/seed_perf_data.py --count 10000
4. Check Prometheus metrics at /api/v1/metrics for slow endpoints
5. Enable SQL echo (temporarily): set DB_ECHO=true in .env

Authentication Issues

Symptom: Login fails, 401 errors

1. Verify user exists and is active
2. Check session cookie name: SESSION_COOKIE_NAME in .env
3. Verify SESSION_COOKIE_SECURE is false in development (no HTTPS)
4. Check Redis for session data: redis-cli KEYS session:*

CORS Errors

Symptom: Browser console shows CORS errors

1. Check CORS_ORIGINS in .env — must include frontend URL
2. No wildcards allowed — explicit origins only
3. Restart server after changing .env

Migration Issues

Symptom: alembic upgrade head fails

1. Check database connectivity
2. Verify Alembic config: alembic.ini
3. Review migration files in alembic/versions/
4. Check current revision: alembic current
5. Reset (DESTRUCTIVE): alembic downgrade base && alembic upgrade head