customer-presentation/OPERATIONS.md

# Operations

Day-to-day commands for running project-bifrost.

## Managing invites

All commands run on the VPS as the `fenja` user.

```bash
# Add someone
sudo -u fenja node /opt/fenja/bin/invite.js add someone@example.com

# Remove someone (doesn't kill their active session — see below)
sudo -u fenja node /opt/fenja/bin/invite.js remove someone@example.com

# List everyone
sudo -u fenja node /opt/fenja/bin/invite.js list
```

Removing an invite stops a user from requesting *new* codes, but doesn't invalidate their existing session cookie (valid 30 days). To kick them out immediately, also run:

```bash
sudo sqlite3 /opt/fenja/data/fenja.sqlite \
  "DELETE FROM sessions WHERE email = 'someone@example.com';"
```

## Reading Join-CTA clicks

Every press of the final "Join Project Bifrost" CTA is logged to the `bifrost_joins` table. Use `bin/joins.js` to read it:

```bash
# Every click, newest first (id, email, session)
sudo -u fenja node /opt/fenja/bin/joins.js list

# One row per user, with click count + first/last timestamps
sudo -u fenja node /opt/fenja/bin/joins.js summary

# Full click history for a single user
sudo -u fenja node /opt/fenja/bin/joins.js for someone@example.com

# Totals — clicks + unique users
sudo -u fenja node /opt/fenja/bin/joins.js stats
```

One row is written per click (the schema uses auto-increment `id`, not email-as-PK), so re-clicks are preserved. For ad-hoc SQL:

```bash
sudo -u fenja sqlite3 /opt/fenja/data/fenja.sqlite \
  "SELECT email, datetime(clicked_at/1000,'unixepoch') FROM bifrost_joins ORDER BY clicked_at DESC;"
```

## Service control

```bash
sudo systemctl status fenja          # is it running?
sudo systemctl restart fenja         # restart (after config/code changes)
sudo journalctl -u fenja -f          # live log tail
sudo journalctl -u fenja -n 100      # last 100 lines
```

## Deploying code changes

The project lives on a Windows filesystem; deploys run from **WSL** (Ubuntu under Windows 11) because `rsync` and `ssh` come with it and behave identically to Linux. Never deploy from PowerShell directly — Windows path semantics + line endings cause subtle breakage on the VPS.

### One-time WSL setup

```bash
# Inside WSL (Ubuntu):
sudo apt update && sudo apt install -y rsync openssh-client

# Copy your SSH key into WSL's ~/.ssh (if it isn't already there).
# The Windows OpenSSH key lives at C:\Users\<you>\.ssh\ — WSL sees this as:
mkdir -p ~/.ssh
cp /mnt/c/Users/Arlin/.ssh/id_ed25519       ~/.ssh/
cp /mnt/c/Users/Arlin/.ssh/id_ed25519.pub   ~/.ssh/
chmod 700 ~/.ssh
chmod 600 ~/.ssh/id_ed25519

# Sanity check: you should land on the VPS without being prompted for a password.
ssh user@project-bifrost.fenja.ai "hostname"
```

> **File permissions gotcha**: SSH will refuse a key that's group-readable. If it prompts for a password despite a copied key, re-run `chmod 600 ~/.ssh/id_ed25519`.

### Every deploy

> **Destructive-rsync warning**: both rsync steps use `--delete`. Every `--delete` invocation below is paired with `--exclude data --exclude .env --exclude node_modules`; do not omit any of those excludes. The consequence of forgetting them is that `/opt/fenja/data` (SQLite DB + nightly backups) and/or `/opt/fenja/node_modules` gets wiped — the service fails to boot until you recreate them and re-run `npm ci`. Has happened once — April 2026.

```bash
# 1. Open WSL and cd into the repo via its WSL path. The Windows project
#    folder `C:\Users\Arlin\01 DEVELOPMENT\fenja-bifrost` is visible as:
cd "/mnt/c/Users/Arlin/01 DEVELOPMENT/fenja-bifrost"

# 2. Push the tree to a staging dir on the VPS. Notes dirs / secrets /
#    build artefacts are excluded so rsync --delete doesn't nuke things
#    on the server that aren't in your checkout.
rsync -avz --delete \
  --exclude node_modules --exclude data --exclude .env --exclude .git \
  --exclude 'CHANGES*.md' --exclude 'MERGE_NOTES.md' --exclude 'AUTH_SIMPLIFICATION_NOTES.md' \
  ./ user@project-bifrost.fenja.ai:/tmp/fenja-upload/

# 3. SSH in and promote the upload.
#    CRITICAL: the exclude list on the --delete rsync must include
#    `data`, `.env`, AND `node_modules` — all three are intentionally
#    absent from the upload, and without these excludes, --delete
#    wipes the server-side copies. Incident log: April 2026 (data),
#    April 2026 (node_modules, same deploy).
ssh user@project-bifrost.fenja.ai
sudo rsync -a --delete \
  --exclude data --exclude .env --exclude node_modules \
  /tmp/fenja-upload/ /opt/fenja/
sudo chown -R fenja:fenja /opt/fenja
rm -rf /tmp/fenja-upload

# 4. If package.json changed (new dep or version bump):
cd /opt/fenja
sudo -u fenja npm ci --omit=dev

# 5. Take a pre-change DB snapshot if the deploy includes a schema change
#    (new/renamed column, new table — check git diff of src/db.js first):
sudo -u fenja sqlite3 /opt/fenja/data/fenja.sqlite \
  ".backup /opt/fenja/data/pre-change-$(date +%F).sqlite"

# 6. Restart and watch the first ~20 log lines for a clean boot
sudo systemctl restart fenja
sudo journalctl -u fenja -n 20
```

Confirm `[bifrost] listening on 127.0.0.1:3000` appears in the logs. (There is no longer an `[mail] SMTP relay reachable` line — the mail stack was removed in the auth simplification.)

### WSL-specific pitfalls

| Symptom | Cause | Fix |
|---|---|---|
| `rsync: command not found` | WSL image lacks rsync | `sudo apt install rsync` |
| `Permission denied (publickey)` | SSH key missing in WSL, or wrong perms | Copy from `/mnt/c/Users/<you>/.ssh/` and `chmod 600` |
| `CRLF will be replaced by LF` warnings from git / scripts failing on server with `bad interpreter: ^M` | Windows line endings snuck in | In the repo: `git config core.autocrlf input`, then re-save the file. Shell scripts in `bin/` must be LF. |
| Very slow rsync | Running rsync against files on `/mnt/c/` is slower than a native WSL filesystem; acceptable for small deploys, painful for big ones | Fine for this repo (< 50MB). For large trees, clone into `~/repos/` inside WSL instead. |
| `ssh: Could not resolve hostname` | Corporate VPN or DNS quirks | Confirm with `ssh -v`; may need to switch network. |

## Editing env config

`/etc/fenja/env` is intentionally minimal — only `PORT`, `PUBLIC_ORIGIN`, and `NODE_ENV=production`. There are **no secrets** (no pepper, no SMTP, no mail-from): auth is email-only against the invite list, and the mail stack was removed.

```bash
sudo nano /etc/fenja/env
sudo systemctl restart fenja
```

Expected contents:

```
PORT=3000
PUBLIC_ORIGIN=https://project-bifrost.fenja.ai
NODE_ENV=production
```

If you ever add a real secret back (e.g. an analytics token), match the invariant in PROJECT.md: root:fenja, mode 640, never in the repo.

## Backups

Nightly cron at `/etc/cron.d/fenja-backup` snapshots the SQLite file to `/opt/fenja/data/backup-YYYY-MM-DD.sqlite`, keeping 14 days.

Manual snapshot:

```bash
sudo -u fenja sqlite3 /opt/fenja/data/fenja.sqlite \
  ".backup /opt/fenja/data/backup-manual-$(date +%F).sqlite"
```

Pull a backup to your laptop:

```bash
scp user@project-bifrost.fenja.ai:/opt/fenja/data/backup-YYYY-MM-DD.sqlite .
```

## Quick health checks

From **WSL** (bash) on your laptop:

```bash
curl -I https://project-bifrost.fenja.ai/                # expect 200
curl -I https://project-bifrost.fenja.ai/timeline.js     # expect 302 → /
curl -I https://project-bifrost.fenja.ai/api/bifrost-join -X POST   # expect 401 (auth-gated)
```

Or from **PowerShell** — use `curl.exe` (the bare `curl` alias is PowerShell's own `Invoke-WebRequest` with a different flag grammar):

```powershell
curl.exe -I https://project-bifrost.fenja.ai/              # expect 200
curl.exe -I https://project-bifrost.fenja.ai/timeline.js   # expect 302 → /
```

If either fails, check Nginx (`sudo systemctl status nginx`) and Node (`sudo systemctl status fenja`) on the VPS.

## Troubleshooting

| Symptom | First thing to check |
|---|---|
| Invited user can't log in | Confirm the email is actually on the invite list: `node bin/invite.js list`. Email is matched lowercase-trimmed. |
| 502 Bad Gateway | Node crashed — `systemctl status fenja` then `journalctl` |
| 504 Gateway Timeout | Node running but hung — `systemctl restart fenja` |
| Nginx config change broke something | `sudo nginx -t` will tell you exactly what |
| Session cookie shipped without `Secure` | `NODE_ENV=production` missing from `/etc/fenja/env`; add it and restart |

## File locations

```
/opt/fenja/                 code (owned by fenja:fenja)
/opt/fenja/data/            SQLite + nightly backups
/etc/fenja/env              secrets (root:fenja, 640)
/etc/systemd/system/fenja.service
/etc/nginx/sites-available/project-bifrost
/etc/nginx/sites-enabled/project-bifrost  (symlink)
/etc/letsencrypt/live/project-bifrost.fenja.ai/  TLS certs
/etc/cron.d/fenja-backup
```