Leopoldadmin/leocrm
1
0
Fork 0
Code Issues 363 Pull Requests Actions Packages Projects Releases 1 Wiki Activity

refactor(deploy): remove old multi-resource code, document single docker-compose workflow

Browse Source 
- Remove POSTGRES_COMPOSE, REDIS_COMPOSE templates (unused)
- Remove create_service(), create_api_application(), generate_worker_compose()
- Remove deploy_worker(), deploy_worker_only(), verify_worker_service()
- Remove resolve_worker_uuid() and all worker_uuid references
- Remove get_worker_envs(), get_api_envs(), get_postgres_envs(), get_redis_envs()
- Remove set_service_envs(), set_application_envs() (dead code)
- Remove _extract_deploy_uuid(), _wait_service_healthy() (only used by deploy_worker)
- Remove seed_admin_user() (only used by old deploy_full)
- Remove DB_HOST, REDIS_HOST, WORKER_UUID, WORKER_NAME config vars
- Remove --worker-only CLI arg
- Replace old deploy_full() with simple redeploy via /api/v1/deploy
- Update run_verification() to remove worker_uuid param
- Add KI workflow comment at top of deploy.py
- Update DEPLOY.md: single docker-compose stack workflow
- Update COOLIFY_SETUP.md: single docker-compose stack, remove 3-resource setup
- Update docs/INSTALL.md: automated --initial workflow

deploy.py: 1370 → 893 lines (-477 lines, -35%)
This commit is contained in:
Agent Zero
2026-08-06 01:22:09 +02:00
parent 85fcb90b32
commit 5b7d93cd0e
4 changed files with 400 additions and 877 deletions
Download Patch File Download Diff File Expand all files Collapse all files
COOLIFY_SETUP.md
+190 -266
View File
@@ -1,177 +1,195 @@
# Coolify Setup — CRM System v1.0
# Coolify Setup — LeoCRM
Production deployment guide for the **CRM System** to the Coolify PaaS instance
at `server.media-on.de` (server UUID `lw80w8scs4044gwcw084s00s4`).
Production deployment guide for LeoCRM to the Coolify PaaS instance
at `server.media-on.de`.
The deploy consists of **three Coolify resources** in the same project/environment:
## Architektur
1. A **PostgreSQL 16** database resource (one-click or Docker image).
2. The **crm-app** Application (Dockerfile build from a Git repository).
3. The **crm-worker** Application (same Dockerfile build, different entrypoint).
LeoCRM läuft als **einzelner docker-compose Stack** in einer Coolify Application.
Coolify liest die `docker-compose.yaml` aus dem Git-Repo und startet alle 4
Container (postgres, redis, crm_app, crm_worker) in einem gemeinsamen Stack.
The resources talk to each other over the internal Docker network. The app
is exposed publicly on `https://crm.media-on.de:443` (Let's Encrypt via Coolify).
The worker is not exposed publicly — it only needs Redis and PostgreSQL access.
```
Coolify Application (build_pack=dockercompose)
├── postgres (pgvector/pgvector:pg16)
├── redis (redis:7-alpine)
├── crm_app (FastAPI API Server, Port 8000)
└── crm_worker (ARQ Background Worker)
```
Alle Container teilen sich ein Docker-Netzwerk. Service-Namen funktionieren
als DNS-Namen (z.B. `postgres`, `redis`, `crm_app`, `crm_worker`).
**Keine separaten Coolify Services** für DB/Redis/Worker. Das funktioniert nicht,
weil Coolify jedem Service ein eigenes Netzwerk gibt und die Container sich
nicht per DNS erreichen können.
---
## 0. ⚠️ Critical domain-format gotcha
Coolify's per-application **Domain field must contain an explicit port** in the
URL. If you enter the domain without `:443`, Let's Encrypt certificate issuance
will silently fail and Traefik will not route traffic correctly.
```
✅ https://crm.media-on.de:443
❌ https://crm.media-on.de
❌ crm.media-on.de
```
> The same rule applies in the Coolify API: when calling
> `PATCH /api/v1/applications/{uuid}` you must set
> `{"domains": "https://crm.media-on.de:443"}` (note the `:443` suffix).
> This is a known bug-fix from earlier deployments — never drop the port.
---
## 1. Prerequisites
## 1. Voraussetzungen
- Coolify server reachable at `https://server.media-on.de`, API token created
in *Keys & Tokens → API tokens* (Bearer token, scope: `*`).
- The DNS **A record** for `crm.media-on.de` points to the public IP of the
Coolify server (Traefik will answer on `:443` and route by `Host` header).
- The CRM source code lives in a **Forgejo repository** that Coolify can
clone. Suggested location:
`https://forge.media-on.de/leopoldadmin/crm-system` (branch `master`).
> If the repo does not exist yet, create it and push the project:
> ```bash
> # One-time: create the repo via Forgejo API or UI
> git remote add origin https://leopoldadmin:<TOKEN>@forge.media-on.de/leopoldadmin/crm-system.git
> git push -u origin master
> ```
- You have the **internal host:port** of the Postgres resource that will be
provisioned in step 2 (Coolify will print it, e.g. `abc123-postgres:5432`).
- DNS **A record** for die App-Domain (z.B. `crm.media-on.de`) zeigt auf die
öffentliche IP des Coolify-Servers.
- LeoCRM source code in Forgejo repository:
`https://forgejo.media-on.de/Leopoldadmin/leocrm.git` (branch `main`).
- Ein **Private Deploy Key** in Coolify hinterlegt (für Git-Zugriff).
- Python 3.12+ mit `httpx` für das deploy script.
---
## 2. Resource A — PostgreSQL 16 database
## 2. Initial Deployment (automatisiert)
In the Coolify UI:
1. Go to **Databases → + Add**.
2. Choose **PostgreSQL 16** (Alpine).
3. Configuration:
- **Name**: `crm-postgres`
- **Database name**: `crm_db`
- **User**: `crm_user`
- **Password**: *(generate a strong one — see Secret generation below)*
- **Public accessibility**: **disabled** (only the crm-app talks to it)
4. Click **Deploy** and wait for status `running:healthy`.
5. Note the **internal host:port** Coolify exposes (typically
`<resource-uuid>-postgres:5432`). You will need it in step 3.
> **Alternative (API):**
> ```bash
> curl -X POST http://server.media-on.de/api/v1/databases \
> -H "Authorization: Bearer $COOLIFY_TOKEN" \
> -H "Content-Type: application/json" \
> -d '{"type":"postgresql","project_uuid":"...","environment_name":"production",
> "server_uuid":"lw80w8scs4044gwcw084s00s4",
> "name":"crm-postgres","postgres_user":"crm_user",
> "postgres_password":"<STRONG_PASSWORD>",
> "postgres_db":"crm_db","is_public":false}'
> ```
---
## 3. Resource B — crm-app (Dockerfile build)
In the Coolify UI:
1. **Projects → + Add Project** if you don't have one yet (e.g. `CRM`).
2. **Environment → + Add Environment** → name: `production`.
3. Inside that environment, **+ Add → Application → Public/Private Repository**.
4. Fill in:
- **Git repository**: `https://forge.media-on.de/leopoldadmin/crm-system`
- **Branch**: `master`
- **Build pack**: `Dockerfile`
- **Dockerfile location**: `Dockerfile` (default, repo root)
- **Port**: `8000`
5. Click **Deploy** once to let Coolify create the resource (it will fail to
start without environment variables — that's expected).
6. Note the **Application UUID** (visible in the URL or via
`GET /api/v1/applications`).
---
## 4. Environment variables (on the crm-app resource)
In **crm-app → Environment Variables**, set:
| Key | Value | Notes |
|-----|-------|-------|
| `DATABASE_URL` | `postgresql+asyncpg://crm_user:<PW>@<postgres-internal-host>:5432/crm_db` | Use the internal host from step 2 (e.g. `crm-postgres-xyz:5432`), **not** `localhost` and **not** the public DNS. |
| `AUTH_SECRET` | *see secret generation* | **MUST be ≥ 32 chars.** |
| `CORS_ORIGINS` | `https://crm.media-on.de:443` | Comma-separated, no wildcards, must match the domain where the browser actually loads the SPA. |
| `ENVIRONMENT` | `production` | |
| `LOG_LEVEL` | `INFO` | `DEBUG` only temporarily. |
| `BCRYPT_ROUNDS` | `12` | Aligned with `.env.example`. |
### Secret generation (run once, locally)
### 2.1 Umgebungsvariablen setzen
```bash
# AUTH_SECRET (min 32 chars, recommended 48+)
python -c "import secrets; print(secrets.token_urlsafe(48))"
# POSTGRES_PASSWORD (min 16 chars, recommended 24+)
python -c "import secrets; print(secrets.token_urlsafe(24))"
export COOLIFY_API_TOKEN="dein-token"
export APP_DOMAIN="https://crm.media-on.de"
export APP_NAME="leocrm"
export DB_PASSWORD="<sicheres-passwort>"
export REDIS_PASSWORD="<sicheres-passwort>"
export SECRET_KEY="<mindestens-32-zeichen>"
export COOLIFY_PROJECT_UUID="<project-uuid>"
export COOLIFY_SERVER_UUID="<server-uuid>"
export COOLIFY_PRIVATE_KEY_UUID="<private-key-uuid>"
export COOLIFY_ENVIRONMENT="production" # optional
export ADMIN_EMAIL="admin@media-on.de" # optional
export ADMIN_PASSWORD="Admin123!" # optional
```
**Never commit these values.** Coolify stores them encrypted at rest, but they
are still rendered in the UI to anyone with read access to the environment.
### 2.2 Deploy starten
> **Alternative (API — bulk update):**
> ```bash
> curl -X PATCH http://server.media-on.de/api/v1/applications/$APP_UUID/envs/bulk \
> -H "Authorization: Bearer $COOLIFY_TOKEN" \
> -H "Content-Type: application/json" \
> -d '{
> "data": [
> {"key":"DATABASE_URL", "value":"postgresql+asyncpg://crm_user:<PW>@<PG_HOST>:5432/crm_db"},
> {"key":"AUTH_SECRET", "value":"<TOKEN_URLSAFE_48>"},
> {"key":"CORS_ORIGINS", "value":"https://crm.media-on.de:443"},
> {"key":"ENVIRONMENT", "value":"production"},
> {"key":"LOG_LEVEL", "value":"INFO"},
> {"key":"BCRYPT_ROUNDS", "value":"12"}
> ]
> }'
> ```
```bash
python scripts/deploy.py --initial
```
Das Script führt einen **2-Phase Deploy** durch:
1. **Phase 1**: Application via `private-deploy-key` erstellen, build_pack auf
`dockercompose` setzen, ENV-Variablen setzen, erster Deploy **ohne Domain**.
Coolify liest `docker-compose.yaml` aus dem Git-Repo und baut alle Container.
2. **Phase 2**: `docker_compose_domains` setzen (für Traefik-Labels), dann
Redeploy. Jetzt ist die App unter der Domain erreichbar.
### 2.3 Warum 2-Phase Deploy?
Coolify muss zuerst die `docker-compose.yaml` aus dem Git-Repo lesen, um die
Service-Namen zu kennen. Erst dann kann `docker_compose_domains` korrekt
zugeordnet werden. Ein Deploy ohne vorherigen Read der Compose-Datei führt zu
fehlenden Traefik-Labels → 503 Fehler.
---
## 5. Configure the public domain (with port!)
## 3. Redeploy (bestehende Anwendung)
In **crm-app → Domains → + Add Domain**:
```bash
export COOLIFY_API_TOKEN="dein-token"
export APP_DOMAIN="https://crm.media-on.de"
export COOLIFY_APP_UUID="dx4pqdziu4uj6x9fxs1u5z0x" # optional
- **Domain**: `https://crm.media-on.de:443`
- ⚠️ **Port `:443` is mandatory.** See section 0.
- **Let's Encrypt**: **enabled** (default).
- Click **Save**. Coolify will issue the certificate and reload Traefik.
python scripts/deploy.py
```
> **Alternative (API):**
> ```bash
> curl -X PATCH http://server.media-on.de/api/v1/applications/$APP_UUID \
> -H "Authorization: Bearer $COOLIFY_TOKEN" \
> -H "Content-Type: application/json" \
> -d '{"domains": "https://crm.media-on.de:443"}'
> ```
Triggert `/api/v1/deploy` für die bestehende Coolify Application und wartet auf
Erfolg. Danach läuft automatisch die Verifikation (HTTP, Login, Alembic, RLS).
---
## 6. Healthcheck (Coolify side)
## 4. Verifikation
In **crm-app → Advanced → Healthcheck**:
```bash
python scripts/deploy.py --verify-only
```
Prüft:
- HTTP Health (`/api/v1/health`)
- Login (optional, wenn LOGIN_EMAIL/LOGIN_PASSWORD gesetzt)
- Alembic Migration Head (via SSH in den postgres Container)
- RLS-Tabellen-Anzahl (via SSH)
---
## 5. Wichtige Hinweise
### 5.1 Service-Namen mit Unterstrichen
In `docker-compose.yaml` müssen Service-Namen **Unterstriche** verwenden:
`crm_app`, `crm_worker` — nicht `crm-app`, `crm-worker`.
Coolify konvertiert Bindestriche zu Unterstrichen in `docker_compose_domains`.
Bei Bindestrichen in der Compose-Datei gibt es keinen Match → keine
Traefik-Labels → 503 Fehler.
### 5.2 docker-compose.yaml (nicht .yml)
Coolify sucht nach `docker-compose.yaml` (mit `.yaml`). Eine Datei namens
`docker-compose.yml` wird nicht gefunden.
### 5.3 Domain ohne :443
In `docker_compose_domains` darf die Domain **kein** `:443` am Ende haben:
```
✅ https://crm.media-on.de
❌ https://crm.media-on.de:443
```
Das `:443` führt zu leeren `Host()` Traefik-Labels.
### 5.4 Kein connect_to_docker_network
Innerhalb eines docker-compose Stacks kümmert sich Coolify selbst um das
Netzwerk. `connect_to_docker_network=True` ist nicht nötig und sollte nicht
gesetzt werden.
---
## 6. Mehrere Instanzen
```bash
# Test-Instanz
APP_NAME=leocrm-test APP_DOMAIN=https://crm-test.media-on.de \
python scripts/deploy.py --initial
# Produktions-Instanz
APP_NAME=leocrm APP_DOMAIN=https://crm.media-on.de \
python scripts/deploy.py --initial
```
Jede Instanz hat eigene DB, Redis, Container und Domain. Alle Parameter werden
aus `APP_NAME` und `APP_DOMAIN` abgeleitet.
---
## 7. Environment-Variablen in Coolify
Das `--initial` Script setzt automatisch folgende ENV-Variablen in Coolify:
| Key | Wert | Quelle |
|-----|------|--------|
| `POSTGRES_USER` | `crm_user` | Default |
| `POSTGRES_DB` | `crm_db` | Default |
| `DB_PASSWORD` | * | ENV |
| `REDIS_PASSWORD` | * | ENV |
| `SECRET_KEY` | * | ENV |
| `ENVIRONMENT` | `production` | Default |
| `LOG_LEVEL` | `INFO` | Default |
| `SESSION_COOKIE_SECURE` | `true` | Default |
| `STORAGE_PATH` | `/data/storage` | Default |
| `CORS_ORIGINS` | APP_DOMAIN | ENV |
| `FRONTEND_URL` | APP_DOMAIN | ENV |
| `APP_DOMAIN` | APP_DOMAIN | ENV |
| `ADMIN_EMAIL` | `admin@media-on.de` | ENV (optional) |
| `ADMIN_PASSWORD` | `Admin123!` | ENV (optional) |
Die `docker-compose.yaml` verwendet `${VARIABLE}` Syntax — Coolify substituiert
aus diesen ENV-Variablen.
---
## 8. Healthcheck
In **Coolify → Application → Advanced → Healthcheck**:
- **Healthcheck path**: `/api/v1/health`
- **Healthcheck method**: `GET`
@@ -180,138 +198,44 @@ In **crm-app → Advanced → Healthcheck**:
- **Healthcheck retries**: `3`
- **Healthcheck start period**: `15s`
> The Dockerfile's in-container `HEALTHCHECK` is the source of truth for
> Docker-level health. The Coolify/Traefik healthcheck is what drives
> automatic rollbacks and load-balancer routing. Set both, identically.
---
## 9. Going forward — Redeploys
- **Code change** → push to `main` auf Forgejo → `python scripts/deploy.py`
(oder Coolify UI → Deployments → Deploy).
- **Environment variable change** → Coolify UI (oder API `PATCH .../envs/bulk`)
→ Deploy (Coolify startet nicht automatisch bei ENV-Änderung neu).
- **Domain change** → API (`PATCH /api/v1/applications/{uuid}` mit
`docker_compose_domains`) — reproduzierbar, UI als Fallback.
---
## 7. Build & deploy
## 10. Troubleshooting
In the Coolify UI: **crm-app → Deployments → Deploy**.
**503 Fehler (Traefik):**
- Domain ohne `:443` in `docker_compose_domains`
- Service-Namen mit Unterstrichen in docker-compose.yaml
- `docker-compose.yaml` (nicht `.yml`)
Watch the build log. The first deploy will:
**Container können sich nicht erreichen (DNS):**
- Alles in einem docker-compose Stack (nicht separate Coolify Services)
- Kein `connect_to_docker_network` setzen
1. Clone the repo (branch `master`).
2. Build the multi-stage Dockerfile (≈ 1–2 min, depending on cache).
3. Start the container. `prestart.sh` runs `alembic upgrade head` against the
Postgres database.
4. Uvicorn binds to `0.0.0.0:8000` and starts serving.
A healthy deploy ends with the container status `running:healthy`.
> **Alternative (API):**
> ```bash
> curl -X POST http://server.media-on.de/api/v1/deploy \
> -H "Authorization: Bearer $COOLIFY_TOKEN" \
> -H "Content-Type: application/json" \
> -d "{\"uuid\":\"$APP_UUID\"}"
> ```
---
## 8. Verification
From anywhere with internet access:
**"Docker Compose file not found":**
- Datei heißt `docker-compose.yaml` (nicht `.yml`)
**Migration fehlgeschlagen:**
```bash
# 1. Root health (used by Docker HEALTHCHECK & Coolify healthcheck)
curl -fsSL -o /dev/null -w "%{http_code}\n" https://crm.media-on.de:443/health
# → 200
# 2. API v1 health (mounted under the versioned router)
curl -fsSL -o /dev/null -w "%{http_code}\n" https://crm.media-on.de:443/api/v1/health
# → 200
# 3. Frontend SPA (served by the static-files mount)
curl -fsSL -o /dev/null -w "%{http_code} %{content_type}\n" \
https://crm.media-on.de:443/index.html
# → 200 text/html
# 4. Interactive API docs
# Open in a browser: https://crm.media-on.de:443/docs
# Register a user via POST /api/v1/auth/register
# Login via POST /api/v1/auth/login → access_token
# Use the token as `Authorization: Bearer <access_token>` on protected routes
docker exec <postgres-container> psql -U crm_user -d crm_db -c "SELECT version_num FROM alembic_version"
docker exec <api-container> alembic upgrade head
```
If any of these return `502` / `503` / `504`:
- Check **crm-app → Logs** in Coolify (the UI is the only place with full
stdout/stderr, the API does not expose logs).
- Confirm the container is `running:healthy` (not `running:unhealthy`,
`exited`, or `starting`).
- Confirm the Postgres resource is `running:healthy` and the
`DATABASE_URL` host matches its internal DNS name.
For full incident response, see [`/a0/.a0/runbook-restore.md`](../../a0/runbook-restore.md).
---
## 9. Going forward — redeploys
- **Code change** → push to `master` on Forgejo → **Deployments → Deploy** in
Coolify. The Dockerfile layer-cache will reuse `pip install -r
requirements.txt` if `requirements.txt` is unchanged.
- **Environment variable change** → edit in Coolify UI (or `PATCH .../envs/bulk`
via API) → **Deploy** (Coolify does *not* auto-restart on ENV change alone).
- **Domain change** → use the API (`PATCH /api/v1/applications/{uuid}`) so it
is reproducible; the UI is a fallback only.
---
## 10. References
## 11. Referenzen
- Coolify v4 API — `/a0/usr/plugins/coolify_control/help/coolify-control/help.md`
- App architecture (Section 13 lockdown) — `/a0/.a0/02-architecture.md`
- Task graph (Phase 4d) — `/a0/.a0/03-task-graph.json`
- Restore runbook — `/a0/.a0/runbook-restore.md`
---
## 11. Resource C — crm-worker (Background Worker)
The crm-worker runs the ARQ background worker and scheduler in a separate
container, using the same Docker image as crm-app but with a different
entrypoint (`/app/worker.sh` instead of `/app/prestart.sh`).
### Setup in Coolify UI
1. In the same project/environment as crm-app, **+ Add → Application →
Public/Private Repository**.
2. Fill in:
- **Git repository**: same as crm-app (`https://forgejo.media-on.de/Leopoldadmin/leocrm.git`)
- **Branch**: `main`
- **Build pack**: `Dockerfile`
- **Dockerfile location**: `Dockerfile` (same image)
- **Port**: `8000` (not used, but Coolify requires a port)
- **Custom Entrypoint**: `/app/worker.sh`
3. Click **Deploy** once to create the resource.
4. Note the **Application UUID**.
### Environment variables (on the crm-worker resource)
Set the same variables as crm-app, except:
| Key | Value | Notes |
|-----|-------|-------|
| `DATABASE_URL` | same as crm-app | |
| `REDIS_URL` | same as crm-app | |
| `SECRET_KEY` | same as crm-app | |
| `ENVIRONMENT` | `production` | |
| `LOG_LEVEL` | `INFO` | |
| `STORAGE_PATH` | `/data/storage` | |
No domain is needed — the worker is not publicly accessible.
### Healthcheck (Coolify side)
- **Healthcheck path**: `/api/v1/health` (not used by worker, but Coolify requires one)
- Alternatively, use a custom healthcheck command:
`pgrep -f "arq app.core.worker.WorkerSettings" || exit 1`
### Scaling
To scale the worker horizontally, deploy multiple crm-worker instances.
Cron jobs use a Redis-based distributed lock (`SET NX` with TTL) so only
one replica executes each scheduled job.
- App architecture — `architecture.md`
- Deploy script — `scripts/deploy.py`
- Install guide — `docs/INSTALL.md`