customer-presentation/OPERATIONS.md

Operations

Day-to-day commands for running project-bifrost.

Managing invites

All commands run on the VPS as the fenja user.

# Add someone
sudo -u fenja node /opt/fenja/bin/invite.js add someone@example.com

# Remove someone (doesn't kill their active session — see below)
sudo -u fenja node /opt/fenja/bin/invite.js remove someone@example.com

# List everyone
sudo -u fenja node /opt/fenja/bin/invite.js list

Removing an invite stops a user from requesting new codes, but doesn't invalidate their existing session cookie (valid 30 days). To kick them out immediately, also run:

sudo sqlite3 /opt/fenja/data/fenja.sqlite \
  "DELETE FROM sessions WHERE email = 'someone@example.com';"

Admin web UI (hidden)

There's a small read-only admin page at /fenjaops — intentionally unlinked from anywhere in the site, and the path is obscure-by-choice so scripted probes of common admin paths (/admin, /wp-admin, etc.) miss. Access requires:

1. A valid session cookie (standard login), and
2. The user's invite row has is_admin = 1.

Anything else — logged-out, logged-in-but-not-admin, scripted probe — gets a plain 404, same as a missing URL. The existence of /fenjaops is not leaked. Internal code uses the word "admin" everywhere (files, middleware, CLI) — only the public URL path is obscured.

Grant / revoke admin via CLI only (there is no web mutation):

# Promote an existing invitee to admin
sudo -u fenja node /opt/fenja/bin/invite.js admin add someone@example.com

# Demote
sudo -u fenja node /opt/fenja/bin/invite.js admin remove someone@example.com

# Who's an admin right now?
sudo -u fenja node /opt/fenja/bin/invite.js admin list

The email must already be in invites — admin add doesn't create an invite. The page itself shows: stats, per-user join summary, invite list (with an Admin badge), and the raw click log. Read-only — edits still go through the CLI.

Reading Join-CTA clicks

Every press of the final "Join Project Bifrost" CTA is logged to the bifrost_joins table. Use bin/joins.js to read it:

# Every click, newest first (id, email, session)
sudo -u fenja node /opt/fenja/bin/joins.js list

# One row per user, with click count + first/last timestamps
sudo -u fenja node /opt/fenja/bin/joins.js summary

# Full click history for a single user
sudo -u fenja node /opt/fenja/bin/joins.js for someone@example.com

# Totals — clicks + unique users
sudo -u fenja node /opt/fenja/bin/joins.js stats

One row is written per click (the schema uses auto-increment id, not email-as-PK), so re-clicks are preserved. For ad-hoc SQL:

sudo -u fenja sqlite3 /opt/fenja/data/fenja.sqlite \
  "SELECT email, datetime(clicked_at/1000,'unixepoch') FROM bifrost_joins ORDER BY clicked_at DESC;"

Service control

sudo systemctl status fenja          # is it running?
sudo systemctl restart fenja         # restart (after config/code changes)
sudo journalctl -u fenja -f          # live log tail
sudo journalctl -u fenja -n 100      # last 100 lines

Deploying code changes

The project lives on a Windows filesystem; deploys run from WSL (Ubuntu under Windows 11) because rsync and ssh come with it and behave identically to Linux. Never deploy from PowerShell directly — Windows path semantics + line endings cause subtle breakage on the VPS.

One-time WSL setup

# Inside WSL (Ubuntu):
sudo apt update && sudo apt install -y rsync openssh-client

# Copy your SSH key into WSL's ~/.ssh (if it isn't already there).
# The Windows OpenSSH key lives at C:\Users\<you>\.ssh\ — WSL sees this as:
mkdir -p ~/.ssh
cp /mnt/c/Users/Arlin/.ssh/id_ed25519       ~/.ssh/
cp /mnt/c/Users/Arlin/.ssh/id_ed25519.pub   ~/.ssh/
chmod 700 ~/.ssh
chmod 600 ~/.ssh/id_ed25519

# Sanity check: you should land on the VPS without being prompted for a password.
ssh user@project-bifrost.fenja.ai "hostname"

File permissions gotcha: SSH will refuse a key that's group-readable. If it prompts for a password despite a copied key, re-run chmod 600 ~/.ssh/id_ed25519.

Every deploy

Destructive-rsync warning: both rsync steps use --delete. Every --delete invocation below is paired with --exclude data --exclude .env --exclude node_modules; do not omit any of those excludes. The consequence of forgetting them is that /opt/fenja/data (SQLite DB + nightly backups) and/or /opt/fenja/node_modules gets wiped — the service fails to boot until you recreate them and re-run npm ci. Has happened once — April 2026.

# 1. Open WSL and cd into the repo via its WSL path. The Windows project
#    folder `C:\Users\Arlin\01 DEVELOPMENT\fenja-bifrost` is visible as:
cd "/mnt/c/Users/Arlin/01 DEVELOPMENT/fenja-bifrost"

# 2. Push the tree to a staging dir on the VPS. Notes dirs / secrets /
#    build artefacts are excluded so rsync --delete doesn't nuke things
#    on the server that aren't in your checkout.
rsync -avz --delete \
  --exclude node_modules --exclude data --exclude .env --exclude .git \
  --exclude 'CHANGES*.md' --exclude 'MERGE_NOTES.md' --exclude 'AUTH_SIMPLIFICATION_NOTES.md' \
  ./ user@project-bifrost.fenja.ai:/tmp/fenja-upload/

# 3. SSH in and promote the upload.
#    CRITICAL: the exclude list on the --delete rsync must include
#    `data`, `.env`, AND `node_modules` — all three are intentionally
#    absent from the upload, and without these excludes, --delete
#    wipes the server-side copies. Incident log: April 2026 (data),
#    April 2026 (node_modules, same deploy).
ssh user@project-bifrost.fenja.ai
sudo rsync -a --delete \
  --exclude data --exclude .env --exclude node_modules \
  /tmp/fenja-upload/ /opt/fenja/
sudo chown -R fenja:fenja /opt/fenja
rm -rf /tmp/fenja-upload

# 4. If package.json changed (new dep or version bump):
cd /opt/fenja
sudo -u fenja npm ci --omit=dev

# 5. Take a pre-change DB snapshot if the deploy includes a schema change
#    (new/renamed column, new table — check git diff of src/db.js first):
sudo -u fenja sqlite3 /opt/fenja/data/fenja.sqlite \
  ".backup /opt/fenja/data/pre-change-$(date +%F).sqlite"

# 6. Restart and watch the first ~20 log lines for a clean boot
sudo systemctl restart fenja
sudo journalctl -u fenja -n 20

Confirm [bifrost] listening on 127.0.0.1:3000 appears in the logs. (There is no longer an [mail] SMTP relay reachable line — the mail stack was removed in the auth simplification.)

WSL-specific pitfalls

Symptom	Cause	Fix
rsync: command not found	WSL image lacks rsync	sudo apt install rsync
Permission denied (publickey)	SSH key missing in WSL, or wrong perms	Copy from /mnt/c/Users/<you>/.ssh/ and chmod 600
CRLF will be replaced by LF warnings from git / scripts failing on server with bad interpreter: ^M	Windows line endings snuck in	In the repo: git config core.autocrlf input, then re-save the file. Shell scripts in bin/ must be LF.
Very slow rsync	Running rsync against files on /mnt/c/ is slower than a native WSL filesystem; acceptable for small deploys, painful for big ones	Fine for this repo (< 50MB). For large trees, clone into ~/repos/ inside WSL instead.
ssh: Could not resolve hostname	Corporate VPN or DNS quirks	Confirm with ssh -v; may need to switch network.

Editing env config

/etc/fenja/env is intentionally minimal — only PORT, PUBLIC_ORIGIN, and NODE_ENV=production. There are no secrets (no pepper, no SMTP, no mail-from): auth is email-only against the invite list, and the mail stack was removed.

sudo nano /etc/fenja/env
sudo systemctl restart fenja

Expected contents:

PORT=3000
PUBLIC_ORIGIN=https://project-bifrost.fenja.ai
NODE_ENV=production

If you ever add a real secret back (e.g. an analytics token), match the invariant in PROJECT.md: root:fenja, mode 640, never in the repo.

Backups

Nightly cron at /etc/cron.d/fenja-backup snapshots the SQLite file to /opt/fenja/data/backup-YYYY-MM-DD.sqlite, keeping 14 days.

Manual snapshot:

sudo -u fenja sqlite3 /opt/fenja/data/fenja.sqlite \
  ".backup /opt/fenja/data/backup-manual-$(date +%F).sqlite"

Pull a backup to your laptop:

scp user@project-bifrost.fenja.ai:/opt/fenja/data/backup-YYYY-MM-DD.sqlite .

Quick health checks

From WSL (bash) on your laptop:

curl -I https://project-bifrost.fenja.ai/                # expect 200
curl -I https://project-bifrost.fenja.ai/timeline.js     # expect 302 → /
curl -I https://project-bifrost.fenja.ai/api/bifrost-join -X POST   # expect 401 (auth-gated)

Or from PowerShell — use curl.exe (the bare curl alias is PowerShell's own Invoke-WebRequest with a different flag grammar):

curl.exe -I https://project-bifrost.fenja.ai/              # expect 200
curl.exe -I https://project-bifrost.fenja.ai/timeline.js   # expect 302 → /

If either fails, check Nginx (sudo systemctl status nginx) and Node (sudo systemctl status fenja) on the VPS.

Troubleshooting

Symptom	First thing to check
Invited user can't log in	Confirm the email is actually on the invite list: node bin/invite.js list. Email is matched lowercase-trimmed.
502 Bad Gateway	Node crashed — systemctl status fenja then journalctl
504 Gateway Timeout	Node running but hung — systemctl restart fenja
Nginx config change broke something	sudo nginx -t will tell you exactly what
Session cookie shipped without Secure	NODE_ENV=production missing from /etc/fenja/env; add it and restart

File locations

/opt/fenja/                 code (owned by fenja:fenja)
/opt/fenja/data/            SQLite + nightly backups
/etc/fenja/env              secrets (root:fenja, 640)
/etc/systemd/system/fenja.service
/etc/nginx/sites-available/project-bifrost
/etc/nginx/sites-enabled/project-bifrost  (symlink)
/etc/letsencrypt/live/project-bifrost.fenja.ai/  TLS certs
/etc/cron.d/fenja-backup