leocrm/COOLIFY_SETUP.md

# Coolify Setup — CRM System v1.0

Production deployment guide for the **CRM System** to the Coolify PaaS instance
at `server.media-on.de` (server UUID `lw80w8scs4044gwcw084s00s4`).

The deploy consists of **three Coolify resources** in the same project/environment:

1. A **PostgreSQL 16** database resource (one-click or Docker image).
2. The **crm-app** Application (Dockerfile build from a Git repository).
3. The **crm-worker** Application (same Dockerfile build, different entrypoint).

The resources talk to each other over the internal Docker network. The app
is exposed publicly on `https://crm.media-on.de:443` (Let's Encrypt via Coolify).
The worker is not exposed publicly — it only needs Redis and PostgreSQL access.

---

## 0. ⚠️ Critical domain-format gotcha

Coolify's per-application **Domain field must contain an explicit port** in the
URL. If you enter the domain without `:443`, Let's Encrypt certificate issuance
will silently fail and Traefik will not route traffic correctly.

```
✅  https://crm.media-on.de:443
❌  https://crm.media-on.de
❌  crm.media-on.de
```

> The same rule applies in the Coolify API: when calling
> `PATCH /api/v1/applications/{uuid}` you must set
> `{"domains": "https://crm.media-on.de:443"}` (note the `:443` suffix).
> This is a known bug-fix from earlier deployments — never drop the port.

---

## 1. Prerequisites

- Coolify server reachable at `https://server.media-on.de`, API token created
  in *Keys & Tokens → API tokens* (Bearer token, scope: `*`).
- The DNS **A record** for `crm.media-on.de` points to the public IP of the
  Coolify server (Traefik will answer on `:443` and route by `Host` header).
- The CRM source code lives in a **Forgejo repository** that Coolify can
  clone. Suggested location:
  `https://forge.media-on.de/leopoldadmin/crm-system` (branch `master`).
  > If the repo does not exist yet, create it and push the project:
  > ```bash
  > # One-time: create the repo via Forgejo API or UI
  > git remote add origin https://leopoldadmin:<TOKEN>@forge.media-on.de/leopoldadmin/crm-system.git
  > git push -u origin master
  > ```
- You have the **internal host:port** of the Postgres resource that will be
  provisioned in step 2 (Coolify will print it, e.g. `abc123-postgres:5432`).

---

## 2. Resource A — PostgreSQL 16 database

In the Coolify UI:

1. Go to **Databases → + Add**.
2. Choose **PostgreSQL 16** (Alpine).
3. Configuration:
   - **Name**: `crm-postgres`
   - **Database name**: `crm_db`
   - **User**: `crm_user`
   - **Password**: *(generate a strong one — see Secret generation below)*
   - **Public accessibility**: **disabled** (only the crm-app talks to it)
4. Click **Deploy** and wait for status `running:healthy`.
5. Note the **internal host:port** Coolify exposes (typically
   `<resource-uuid>-postgres:5432`). You will need it in step 3.

> **Alternative (API):**
> ```bash
> curl -X POST http://server.media-on.de/api/v1/databases \
>   -H "Authorization: Bearer $COOLIFY_TOKEN" \
>   -H "Content-Type: application/json" \
>   -d '{"type":"postgresql","project_uuid":"...","environment_name":"production",
>        "server_uuid":"lw80w8scs4044gwcw084s00s4",
>        "name":"crm-postgres","postgres_user":"crm_user",
>        "postgres_password":"<STRONG_PASSWORD>",
>        "postgres_db":"crm_db","is_public":false}'
> ```

---

## 3. Resource B — crm-app (Dockerfile build)

In the Coolify UI:

1. **Projects → + Add Project** if you don't have one yet (e.g. `CRM`).
2. **Environment → + Add Environment** → name: `production`.
3. Inside that environment, **+ Add → Application → Public/Private Repository**.
4. Fill in:
   - **Git repository**: `https://forge.media-on.de/leopoldadmin/crm-system`
   - **Branch**: `master`
   - **Build pack**: `Dockerfile`
   - **Dockerfile location**: `Dockerfile` (default, repo root)
   - **Port**: `8000`
5. Click **Deploy** once to let Coolify create the resource (it will fail to
   start without environment variables — that's expected).
6. Note the **Application UUID** (visible in the URL or via
   `GET /api/v1/applications`).

---

## 4. Environment variables (on the crm-app resource)

In **crm-app → Environment Variables**, set:

| Key | Value | Notes |
|-----|-------|-------|
| `DATABASE_URL` | `postgresql+asyncpg://crm_user:<PW>@<postgres-internal-host>:5432/crm_db` | Use the internal host from step 2 (e.g. `crm-postgres-xyz:5432`), **not** `localhost` and **not** the public DNS. |
| `AUTH_SECRET` | *see secret generation* | **MUST be ≥ 32 chars.** |
| `CORS_ORIGINS` | `https://crm.media-on.de:443` | Comma-separated, no wildcards, must match the domain where the browser actually loads the SPA. |
| `ENVIRONMENT` | `production` | |
| `LOG_LEVEL` | `INFO` | `DEBUG` only temporarily. |
| `BCRYPT_ROUNDS` | `12` | Aligned with `.env.example`. |
| `JWT_ALGORITHM` | `HS256` | Aligned with `.env.example`. |
| `JWT_EXPIRY_HOURS` | `24` | Aligned with `.env.example`. |

### Secret generation (run once, locally)

```bash
# AUTH_SECRET (min 32 chars, recommended 48+)
python -c "import secrets; print(secrets.token_urlsafe(48))"

# POSTGRES_PASSWORD (min 16 chars, recommended 24+)
python -c "import secrets; print(secrets.token_urlsafe(24))"
```

**Never commit these values.** Coolify stores them encrypted at rest, but they
are still rendered in the UI to anyone with read access to the environment.

> **Alternative (API — bulk update):**
> ```bash
> curl -X PATCH http://server.media-on.de/api/v1/applications/$APP_UUID/envs/bulk \
>   -H "Authorization: Bearer $COOLIFY_TOKEN" \
>   -H "Content-Type: application/json" \
>   -d '{
>     "data": [
>       {"key":"DATABASE_URL",        "value":"postgresql+asyncpg://crm_user:<PW>@<PG_HOST>:5432/crm_db"},
>       {"key":"AUTH_SECRET",          "value":"<TOKEN_URLSAFE_48>"},
>       {"key":"CORS_ORIGINS",         "value":"https://crm.media-on.de:443"},
>       {"key":"ENVIRONMENT",          "value":"production"},
>       {"key":"LOG_LEVEL",            "value":"INFO"},
>       {"key":"BCRYPT_ROUNDS",        "value":"12"},
>       {"key":"JWT_ALGORITHM",        "value":"HS256"},
>       {"key":"JWT_EXPIRY_HOURS",     "value":"24"}
>     ]
>   }'
> ```

---

## 5. Configure the public domain (with port!)

In **crm-app → Domains → + Add Domain**:

- **Domain**: `https://crm.media-on.de:443`
  - ⚠️ **Port `:443` is mandatory.** See section 0.
- **Let's Encrypt**: **enabled** (default).
- Click **Save**. Coolify will issue the certificate and reload Traefik.

> **Alternative (API):**
> ```bash
> curl -X PATCH http://server.media-on.de/api/v1/applications/$APP_UUID \
>   -H "Authorization: Bearer $COOLIFY_TOKEN" \
>   -H "Content-Type: application/json" \
>   -d '{"domains": "https://crm.media-on.de:443"}'
> ```

---

## 6. Healthcheck (Coolify side)

In **crm-app → Advanced → Healthcheck**:

- **Healthcheck path**: `/api/v1/health`
- **Healthcheck method**: `GET`
- **Healthcheck interval**: `30s`
- **Healthcheck timeout**: `10s`
- **Healthcheck retries**: `3`
- **Healthcheck start period**: `15s`

> The Dockerfile's in-container `HEALTHCHECK` is the source of truth for
> Docker-level health. The Coolify/Traefik healthcheck is what drives
> automatic rollbacks and load-balancer routing. Set both, identically.

---

## 7. Build & deploy

In the Coolify UI: **crm-app → Deployments → Deploy**.

Watch the build log. The first deploy will:

1. Clone the repo (branch `master`).
2. Build the multi-stage Dockerfile (≈ 1–2 min, depending on cache).
3. Start the container. `prestart.sh` runs `alembic upgrade head` against the
   Postgres database.
4. Uvicorn binds to `0.0.0.0:8000` and starts serving.

A healthy deploy ends with the container status `running:healthy`.

> **Alternative (API):**
> ```bash
> curl -X POST http://server.media-on.de/api/v1/deploy \
>   -H "Authorization: Bearer $COOLIFY_TOKEN" \
>   -H "Content-Type: application/json" \
>   -d "{\"uuid\":\"$APP_UUID\"}"
> ```

---

## 8. Verification

From anywhere with internet access:

```bash
# 1. Root health (used by Docker HEALTHCHECK & Coolify healthcheck)
curl -fsSL -o /dev/null -w "%{http_code}\n" https://crm.media-on.de:443/health
# → 200

# 2. API v1 health (mounted under the versioned router)
curl -fsSL -o /dev/null -w "%{http_code}\n" https://crm.media-on.de:443/api/v1/health
# → 200

# 3. Frontend SPA (served by the static-files mount)
curl -fsSL -o /dev/null -w "%{http_code} %{content_type}\n" \
  https://crm.media-on.de:443/index.html
# → 200 text/html

# 4. Interactive API docs
# Open in a browser:  https://crm.media-on.de:443/docs
# Register a user via POST /api/v1/auth/register
# Login via POST /api/v1/auth/login → access_token
# Use the token as `Authorization: Bearer <access_token>` on protected routes
```

If any of these return `502` / `503` / `504`:

- Check **crm-app → Logs** in Coolify (the UI is the only place with full
  stdout/stderr, the API does not expose logs).
- Confirm the container is `running:healthy` (not `running:unhealthy`,
  `exited`, or `starting`).
- Confirm the Postgres resource is `running:healthy` and the
  `DATABASE_URL` host matches its internal DNS name.

For full incident response, see [`/a0/.a0/runbook-restore.md`](../../a0/runbook-restore.md).

---

## 9. Going forward — redeploys

- **Code change** → push to `master` on Forgejo → **Deployments → Deploy** in
  Coolify. The Dockerfile layer-cache will reuse `pip install -r
  requirements.txt` if `requirements.txt` is unchanged.
- **Environment variable change** → edit in Coolify UI (or `PATCH .../envs/bulk`
  via API) → **Deploy** (Coolify does *not* auto-restart on ENV change alone).
- **Domain change** → use the API (`PATCH /api/v1/applications/{uuid}`) so it
  is reproducible; the UI is a fallback only.

---

## 10. References

- Coolify v4 API — `/a0/usr/plugins/coolify_control/help/coolify-control/help.md`
- App architecture (Section 13 lockdown) — `/a0/.a0/02-architecture.md`
- Task graph (Phase 4d) — `/a0/.a0/03-task-graph.json`
- Restore runbook — `/a0/.a0/runbook-restore.md`

---

## 11. Resource C — crm-worker (Background Worker)

The crm-worker runs the ARQ background worker and scheduler in a separate
container, using the same Docker image as crm-app but with a different
entrypoint (`/app/worker.sh` instead of `/app/prestart.sh`).

### Setup in Coolify UI

1. In the same project/environment as crm-app, **+ Add → Application →
   Public/Private Repository**.
2. Fill in:
   - **Git repository**: same as crm-app (`https://forgejo.media-on.de/Leopoldadmin/leocrm.git`)
   - **Branch**: `main`
   - **Build pack**: `Dockerfile`
   - **Dockerfile location**: `Dockerfile` (same image)
   - **Port**: `8000` (not used, but Coolify requires a port)
   - **Custom Entrypoint**: `/app/worker.sh`
3. Click **Deploy** once to create the resource.
4. Note the **Application UUID**.

### Environment variables (on the crm-worker resource)

Set the same variables as crm-app, except:

| Key | Value | Notes |
|-----|-------|-------|
| `DATABASE_URL` | same as crm-app | |
| `REDIS_URL` | same as crm-app | |
| `SECRET_KEY` | same as crm-app | |
| `ENVIRONMENT` | `production` | |
| `LOG_LEVEL` | `INFO` | |
| `STORAGE_PATH` | `/data/storage` | |

No domain is needed — the worker is not publicly accessible.

### Healthcheck (Coolify side)

- **Healthcheck path**: `/api/v1/health` (not used by worker, but Coolify requires one)
- Alternatively, use a custom healthcheck command:
  `pgrep -f "arq app.core.worker.WorkerSettings" || exit 1`

### Scaling

To scale the worker horizontally, deploy multiple crm-worker instances.
Cron jobs use a Redis-based distributed lock (`SET NX` with TTL) so only
one replica executes each scheduled job.