# Phase 4 — Deploy to Company Server (Step-by-Step)

This is the **definitive runbook** for putting the courseware generator on a real server. Read it once end-to-end first to understand what you're doing, then follow it step-by-step.

This document assumes:
- Phases 1, 2, 3, and 5 are already complete (which they are as of 2026-05-21)
- You're deploying to a single Linux server (Ubuntu 22.04 LTS — the standard for Python web apps)
- That server is on your company network, not the public internet
- You have SSH access to it

If the server IT gives you is Windows Server, skip to the "Windows Server alternative" section at the end.

---

## Glossary (so we don't lose anyone)

| Term | Meaning |
|---|---|
| **SSH** | Secure Shell — the way you log into a remote Linux server from your laptop. Like Remote Desktop, but command-line. |
| **`sudo`** | "Super-user do" — runs a command as administrator. You'll prefix most setup commands with this. |
| **systemd** | The thing that starts/stops/restarts your app automatically. Like Windows Services for Linux. |
| **virtual environment (venv)** | A folder that contains a private Python + its installed packages, so your app's dependencies don't conflict with other apps on the server. |
| **reverse proxy** | A small web server (nginx) sitting in front of your Flask app. It handles HTTPS, gzip, and lets you put auth-aware tools in front of the app later. Optional for v1. |
| **port** | A "channel number" the server listens on. Flask defaults to port 5000. Web traffic usually uses 80 (HTTP) or 443 (HTTPS). |
| **`.env` file** | Plain-text file with one `KEY=value` per line. Holds secrets and config the app reads at startup. Never goes in git. |

---

## Phase 4 at a glance

```
┌─────────────────────────────────────────────────────────────────────────┐
│  Step 0: Get the server (the IT ask)                                    │
│  Step 1: First SSH login + update                                       │
│  Step 2: Install Python, git, sqlite, curl                              │
│  Step 3: Create directory layout                                        │
│  Step 4: Create deploy + service user accounts                          │
│  Step 5: Provision .env                                                 │
│  Step 6: Download the latest release from S3 (first deploy)             │
│  Step 7: Sync banks from S3                                             │
│  Step 8: Create Python virtualenv + install dependencies                │
│  Step 9: Initialize the database                                        │
│  Step 10: Create the production admin user                              │
│  Step 11: Manual test run (does the server respond?)                    │
│  Step 12: Install as a systemd service (auto-start, auto-restart)       │
│  Step 13: Set up daily DB backups                                       │
│  Step 14: Set up log rotation                                           │
│  Step 15: Open the right firewall port (or set up a reverse proxy)      │
│  Step 16: Run the final verification checklist                          │
│  Step 17: Document the URL for your team                                │
└─────────────────────────────────────────────────────────────────────────┘
```

**Time estimate: 2–4 hours start to finish if it all goes smoothly.** Most failures are missing env vars or port firewall rules.

---

# Step 0: Get the server (the IT ask)

Send your IT team a request that includes these specifics:

```
Hi IT,

For the courseware generator (Phase 4 deployment), I need a Linux VM with:

- OS:      Ubuntu Server 22.04 LTS (or 24.04 LTS)
- CPU:     2 vCPUs minimum (4 if you have budget)
- RAM:     4 GB minimum (8 GB recommended — pipeline runs are RAM-hungry)
- Disk:    50 GB total
            * 20 GB for the OS + Python + dependencies
            * 30 GB for /var/lib/courseware (banks, uploads, DB)
- Network: Internal-only (no public internet). Reachable from team
            laptops on the internal network on port 5000 (or 80 if you
            want me behind nginx).
- Outbound: needs to reach api.anthropic.com:443 and
            s3.us-east-2.amazonaws.com:443 (for AWS S3 banks/outputs)
- Access:  SSH access for me (please add my public SSH key — included below)
- Auto-start: Yes, the VM should boot on host start.

Optional but nice to have:
- A DNS name pointing at it (e.g. courseware.<internal-domain>) instead
  of a raw IP address.
```

Include your SSH public key (`~/.ssh/id_ed25519.pub` or `~/.ssh/id_rsa.pub` from your laptop — generate with `ssh-keygen -t ed25519` if you don't have one).

**What success looks like:** IT replies with the server's IP address (or DNS name) and confirms SSH access. You can run `ssh ubuntu@<server-ip>` from your laptop and get a shell prompt.

---

# Step 1: First SSH login + system update

From your laptop, in PowerShell or Git Bash:

```bash
ssh ubuntu@<server-ip>
```

(If your IT gave you a different username, use that instead of `ubuntu`.)

Once logged in, update the system:

```bash
sudo apt update
sudo apt upgrade -y
```

This downloads the latest security patches and bug fixes for Ubuntu's built-in packages. Takes 1–3 minutes.

**What success looks like:** You see "0 upgraded, 0 newly installed" after the second command (or it lists packages it upgraded). No errors.

---

# Step 2: Install dependencies

Install everything the app and the deploy scripts need:

```bash
sudo apt install -y \
    python3.11 \
    python3.11-venv \
    python3-pip \
    git \
    sqlite3 \
    curl \
    unzip
```

What each one is for:
- `python3.11` + `venv` — runs the app
- `pip` — installs Python packages
- `git` — needed if you ever want to restore from a git bundle backup
- `sqlite3` — command-line tool to peek at the database; not strictly required
- `curl` — for downloading things and testing endpoints
- `unzip` — for unpacking release zips (the deploy script uses Python's zipfile but `unzip` is useful for manual recovery)

Verify versions:

```bash
python3.11 --version    # should print Python 3.11.x
pip --version
git --version
```

**What success looks like:** All three print versions, no "command not found" errors.

---

# Step 3: Create directory layout

Plan: separate the code (which we replace on every deploy) from the persistent data (DB, banks, uploads, logs).

```bash
sudo mkdir -p /opt/courseware/{code,venv}
sudo mkdir -p /var/lib/courseware/{banks,tmp_outputs,uploads/templates,uploads/outlines}
sudo mkdir -p /var/log/courseware
```

What lives where:

```
/opt/courseware/
├── code/          ← the app code (gets replaced each deploy)
├── venv/          ← Python virtualenv (shared across deploys to skip pip install when unchanged)
└── .env           ← secrets — created in Step 5

/var/lib/courseware/      (persistent — survives deploys)
├── banks/                ← downloaded from S3 at boot
├── uploads/              ← user-uploaded templates and outlines
├── tmp_outputs/          ← scratch space for build outputs before S3 upload
└── courseware.db         ← SQLite database

/var/log/courseware/      (logs)
└── *.log
```

**What success looks like:** `ls -la /opt/courseware /var/lib/courseware /var/log/courseware` shows all the directories exist.

---

# Step 4: Create the service user

For security, the app shouldn't run as `ubuntu` (your login user). Create a dedicated user with no shell:

```bash
sudo useradd --system --no-create-home --shell /usr/sbin/nologin courseware
```

This `courseware` user can't be SSH'd into — it exists only to run the service. Now give it ownership of the data directories:

```bash
sudo chown -R courseware:courseware /var/lib/courseware /var/log/courseware
sudo chown -R ubuntu:courseware /opt/courseware
sudo chmod -R 750 /opt/courseware
```

What this does: `ubuntu` (you) owns `/opt/courseware` so you can deploy releases. `courseware` (the service) can READ everything in `/opt/courseware` but can WRITE only to `/var/lib/courseware` and `/var/log/courseware`. That means a bug in the app can't corrupt the code.

**What success looks like:** `ls -ld /opt/courseware /var/lib/courseware` shows the right owners.

---

# Step 5: Provision `.env`

Create the secrets/config file. This is the most error-prone step — get the values right or the app won't start.

```bash
sudo nano /opt/courseware/.env
```

Paste this template and replace the `<...>` placeholders:

```
# Anthropic API
ANTHROPIC_API_KEY=<your sk-ant-... key>

# Flask session signing (generate a fresh random string for production)
SESSION_SECRET=<run: python3 -c "import secrets; print(secrets.token_hex(32))">

# Database — production path
DATABASE_URL=sqlite:////var/lib/courseware/courseware.db

# AWS S3 (same credentials you use locally)
S3_BUCKET=projectx-v1
AWS_REGION=us-east-2
AWS_ACCESS_KEY_ID=<your access key>
AWS_SECRET_ACCESS_KEY=<your secret key>
S3_BANKS_PREFIX=banks/
S3_OUTPUTS_PREFIX=outputs/
S3_RELEASES_PREFIX=releases/
S3_BACKUPS_PREFIX=backups/

# Paths the app reads from (production server)
BANKS_ROOT=/var/lib/courseware/banks
TAGGED_BANK_DIR=/var/lib/courseware/banks/tagged infographics/new labeled grouped infographics
DRILL_DOWN_BANK_DIR=/var/lib/courseware/banks/drill_down_bank
SAMPLE_TEMPLATES_DIR=/var/lib/courseware/banks/sample template
ILLUSTRATIONS_DIR=/var/lib/courseware/banks/images
UPLOADS_DIR=/var/lib/courseware/uploads
TMP_OUTPUTS_DIR=/var/lib/courseware/tmp_outputs

# Deploy script defaults
DEPLOY_ROOT=/opt/courseware
SERVICE_NAME=courseware
VENV_DIR=/opt/courseware/venv

# Optional admin bootstrap (used by scripts/create_admin.py once)
INITIAL_ADMIN_EMAIL=<your email>
INITIAL_ADMIN_PASSWORD=<a strong temporary password — change on first login>
INITIAL_ADMIN_NAME=<your full name>
```

Save (Ctrl+O, Enter, Ctrl+X).

Lock it down so only owners can read it:

```bash
sudo chown ubuntu:courseware /opt/courseware/.env
sudo chmod 640 /opt/courseware/.env
```

**What success looks like:** `cat /opt/courseware/.env | grep -c =` should print a number ≥ 17 (one per env var). If it prints 0, the file is empty.

**Security note:** The keys you paste here are real secrets. Don't print this file in screenshots. Don't email it. If it ever leaks, rotate immediately.

---

# Step 6: First deploy — download the latest release

Run the deploy script as your `ubuntu` user (NOT as root):

```bash
cd /tmp
# Load env vars so the script can reach S3
set -a; source /opt/courseware/.env; set +a

# Fetch latest.zip from S3 and unpack it into /opt/courseware/code/
python3.11 -c "
import boto3, zipfile, os
from pathlib import Path

s3 = boto3.client('s3', region_name=os.environ['AWS_REGION'],
    aws_access_key_id=os.environ['AWS_ACCESS_KEY_ID'],
    aws_secret_access_key=os.environ['AWS_SECRET_ACCESS_KEY'],
    endpoint_url=f'https://s3.{os.environ[\"AWS_REGION\"]}.amazonaws.com')

s3.download_file(os.environ['S3_BUCKET'], 'releases/latest.zip', '/tmp/latest.zip')
print('Downloaded latest.zip')

with zipfile.ZipFile('/tmp/latest.zip') as zf:
    zf.extractall('/opt/courseware/code')
print('Extracted to /opt/courseware/code/')
"
```

This downloads the most recent release from S3 and unzips it into `/opt/courseware/code/`.

**What success looks like:** `ls /opt/courseware/code/` shows the same folder structure as your laptop's project (web/, ppt_generator/, scripts/, etc.).

After Step 8 (when the venv exists and boto3 is installed), you'll be able to use `python scripts/deploy_release.py` for future deployments. The bootstrap above is just for the very first time.

---

# Step 7: Sync banks from S3

The banks (~213 MB of PPT files and illustrations) need to be on the server's disk so the app can read them quickly.

```bash
cd /opt/courseware/code
python3.11 scripts/sync_banks_from_s3.py
```

This downloads everything under `s3://projectx-v1/banks/` into `/var/lib/courseware/banks/` (because `BANKS_ROOT` is set in `.env`). Takes 1–3 minutes depending on network speed.

**What success looks like:** Output ends with "Done. downloaded=435 skipped=0 bytes_received=213.0 MB" and `ls /var/lib/courseware/banks/` shows the bank subfolders.

---

# Step 8: Create the Python virtual environment

```bash
python3.11 -m venv /opt/courseware/venv
source /opt/courseware/venv/bin/activate
pip install --upgrade pip
pip install -r /opt/courseware/code/requirements.txt
```

This installs every Python package the app needs (Flask, SQLAlchemy, boto3, anthropic, python-pptx, etc.) into a private folder so they don't conflict with anything else on the server. Takes 1–5 minutes.

**What success looks like:** Last line is "Successfully installed ..." with no errors. Run `python -c "import flask, boto3, anthropic, pptx; print('all ok')"` — should print `all ok`.

---

# Step 9: Initialize the database

```bash
# Make sure venv is still active and env vars are loaded
set -a; source /opt/courseware/.env; set +a

cd /opt/courseware/code
python scripts/init_db.py
```

This creates the SQLite database at `/var/lib/courseware/courseware.db` with empty tables for users, jobs, daily_activity, and token_usage.

(Note: this does NOT import the legacy JSON files from your laptop. Production starts with an empty history.)

**What success looks like:** Output shows row counts (all should be 0 except possibly `users=0`):

```
Final row counts:
  users                  0
  jobs                   0
  daily_activity         0
  token_usage            0
```

The file `/var/lib/courseware/courseware.db` now exists.

---

# Step 10: Create the production admin user

```bash
cd /opt/courseware/code
python scripts/create_admin.py
```

If you set `INITIAL_ADMIN_EMAIL` and `INITIAL_ADMIN_PASSWORD` in `.env` earlier, it'll create the user automatically. Otherwise it'll prompt you interactively.

**What success looks like:** "Created admin user 'your@email.com' (id=1)."

Make a note of the email and password — that's your production login.

---

# Step 11: Manual test run

Before installing as a service, do one manual run to confirm everything works.

```bash
cd /opt/courseware/code
python -m web.app
```

You should see something like:

```
[warn] SESSION_SECRET not set — using ephemeral key. ...   (won't appear because you set it)
ProjectX Dashboard
  Local:    http://127.0.0.1:5000
  LAN:      http://<server-ip>:5000   <-- share this with teammates
[boot] _run_pipeline_job uses: _set_job
 * Serving Flask app 'app'
 * Debug mode: off
```

From your laptop, open `http://<server-ip>:5000` in a browser. You should see the login page. Log in with the admin credentials from Step 10.

If the dashboard loads → success.

**Press Ctrl+C to stop the server before moving to Step 12** (otherwise systemd will fail to start the same port).

**What success looks like:** Login works, dashboard renders, no errors in the terminal.

---

# Step 12: Install as a systemd service (auto-start, auto-restart)

This makes the app run 24/7 and survive reboots and crashes.

Create the unit file:

```bash
sudo nano /etc/systemd/system/courseware.service
```

Paste:

```ini
[Unit]
Description=Knowledge Academy Courseware Generator
After=network-online.target
Wants=network-online.target

[Service]
Type=simple
User=courseware
Group=courseware
WorkingDirectory=/opt/courseware/code
EnvironmentFile=/opt/courseware/.env
ExecStart=/opt/courseware/venv/bin/python -m web.app
Restart=always
RestartSec=10
StandardOutput=append:/var/log/courseware/app.log
StandardError=append:/var/log/courseware/error.log

# Security hardening
NoNewPrivileges=true
ProtectSystem=strict
ProtectHome=true
ReadWritePaths=/var/lib/courseware /var/log/courseware

[Install]
WantedBy=multi-user.target
```

Save (Ctrl+O, Enter, Ctrl+X).

Enable and start:

```bash
sudo systemctl daemon-reload
sudo systemctl enable courseware
sudo systemctl start courseware
sudo systemctl status courseware
```

**What success looks like:** `status` shows `Active: active (running)` in green.

Test from your laptop's browser again — `http://<server-ip>:5000` should load the dashboard, served by the systemd-managed process.

Test auto-restart: `sudo systemctl restart courseware` and confirm the dashboard recovers within ~15 seconds.

Test boot-survival: `sudo reboot`, wait 30 seconds, SSH back in, run `sudo systemctl status courseware` — should still be running.

---

# Step 13: Set up daily DB backups

The SQLite DB is the only place users, jobs, and history live. Back it up to S3 daily.

Create the backup script:

```bash
sudo nano /opt/courseware/backup_db.sh
```

Paste:

```bash
#!/bin/bash
# Daily SQLite backup to S3.
set -e
set -a; source /opt/courseware/.env; set +a

DATE=$(date +%Y-%m-%d)
TMP=/tmp/courseware-${DATE}.db

# Take a consistent snapshot (sqlite3 .backup is online-safe)
sqlite3 /var/lib/courseware/courseware.db ".backup ${TMP}"

# Upload to S3
python3.11 -c "
import boto3, os
s3 = boto3.client('s3', region_name=os.environ['AWS_REGION'],
    aws_access_key_id=os.environ['AWS_ACCESS_KEY_ID'],
    aws_secret_access_key=os.environ['AWS_SECRET_ACCESS_KEY'],
    endpoint_url=f'https://s3.{os.environ[\"AWS_REGION\"]}.amazonaws.com')
s3.upload_file('${TMP}', os.environ['S3_BUCKET'], 'db-backups/courseware-${DATE}.db')
print('Uploaded to s3://${S3_BUCKET}/db-backups/courseware-${DATE}.db')
"

rm "${TMP}"
echo "Backup complete: ${DATE}"
```

Make executable:

```bash
sudo chmod +x /opt/courseware/backup_db.sh
```

Test it once manually:

```bash
sudo /opt/courseware/backup_db.sh
```

Schedule it daily at 2 AM via cron:

```bash
sudo crontab -e
```

Add this line:

```
0 2 * * * /opt/courseware/backup_db.sh >> /var/log/courseware/backup.log 2>&1
```

Save. Cron is now running the backup at 2 AM every day.

**What success looks like:** Manual run prints "Backup complete: <date>". File appears in S3 at `s3://projectx-v1/db-backups/courseware-<date>.db`.

**Restore procedure (if you ever need it):**

```bash
sudo systemctl stop courseware
# Download the backup you want
python3.11 -c "
import boto3, os
s3 = boto3.client('s3', region_name=os.environ['AWS_REGION'], ...)
s3.download_file(os.environ['S3_BUCKET'], 'db-backups/courseware-<date>.db', '/var/lib/courseware/courseware.db')
"
sudo chown courseware:courseware /var/lib/courseware/courseware.db
sudo systemctl start courseware
```

---

# Step 14: Set up log rotation

Prevent log files from growing forever and filling the disk.

```bash
sudo nano /etc/logrotate.d/courseware
```

Paste:

```
/var/log/courseware/*.log {
    daily
    rotate 14
    compress
    delaycompress
    missingok
    notifempty
    create 644 courseware courseware
    postrotate
        systemctl reload courseware > /dev/null 2>&1 || true
    endscript
}
```

This keeps 14 days of logs (compressed after day 1), then deletes the oldest.

Test the config:

```bash
sudo logrotate -d /etc/logrotate.d/courseware
```

**What success looks like:** Output ends with "reading state from file: /var/lib/logrotate/status" with no errors.

---

# Step 15: Network access

The app is listening on port 5000. Two ways to make it reachable for your team:

## Option A — Simple: Just open port 5000 on the firewall

```bash
sudo ufw allow from <your-team-network-CIDR> to any port 5000
sudo ufw status
```

Replace `<your-team-network-CIDR>` with your office network (e.g. `10.0.0.0/8` for internal). Your IT can tell you the right value.

Then your team accesses the dashboard at `http://<server-ip>:5000`.

## Option B — Recommended: nginx reverse proxy on port 80

This adds nicer URLs, gzip compression, and lets you put HTTPS in front later.

```bash
sudo apt install -y nginx
sudo nano /etc/nginx/sites-available/courseware
```

Paste:

```nginx
server {
    listen 80;
    server_name <server-ip-or-dns-name>;

    client_max_body_size 100M;   # match Flask's MAX_CONTENT_LENGTH

    location /static/ {
        alias /opt/courseware/code/web/static/;
        expires 1h;
    }

    location / {
        proxy_pass http://127.0.0.1:5000;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_read_timeout 600s;     # builds can take 10+ minutes
        proxy_send_timeout 600s;
    }
}
```

Enable + reload:

```bash
sudo ln -s /etc/nginx/sites-available/courseware /etc/nginx/sites-enabled/courseware
sudo nginx -t
sudo systemctl reload nginx
sudo ufw allow 80/tcp
```

Then your team accesses `http://<server-ip>/` (no port number needed).

---

# Step 16: Final verification checklist

Tick every box before declaring "in production":

- [ ] `sudo systemctl status courseware` shows "active (running)"
- [ ] `sudo systemctl is-enabled courseware` returns "enabled" (will start on reboot)
- [ ] Dashboard loads in browser at `http://<server-ip>:5000` (or `http://<server-ip>/` with nginx)
- [ ] Login works with admin credentials
- [ ] You can upload a template + outline (try a small PDF)
- [ ] Generate triggers a real build (1-day course, 20-30 slides)
- [ ] Build completes successfully
- [ ] Generated PPTX appears in `s3://projectx-v1/outputs/`
- [ ] Click Download → file downloads correctly
- [ ] `sudo journalctl -u courseware --since "1 hour ago"` shows entries (logs flowing)
- [ ] Token usage row appears in DB after the build (`sqlite3 /var/lib/courseware/courseware.db "SELECT * FROM token_usage ORDER BY timestamp DESC LIMIT 3;"`)
- [ ] Daily activity row updates for today
- [ ] Reboot the server, wait 30s, verify the dashboard is back up
- [ ] Manually run `/opt/courseware/backup_db.sh` and confirm a `.db` file appears in `s3://projectx-v1/db-backups/`
- [ ] Cron entry exists: `sudo crontab -l | grep backup_db`

When all 14 are green → **production**.

---

# Step 17: Document the URL for your team

Send a short email/Slack to the courseware team:

```
Subject: Courseware generator is live

Hi team,

The courseware generator is now running on the internal server:

    URL:      http://<server-ip>:5000     (or http://<dns-name>)
    
    Login:    Use the credentials I sent you separately. If you don't
              have an account yet, ping <admin-name> for one.
    
    Workflow: Upload your template (PPTX) and outline (PDF), pick a
              language, click Generate. Builds take 8-12 minutes.
              The dashboard shows live progress.

Notes:
- This is only reachable from inside the company network. It does NOT
  work from home unless you're on VPN.
- Bug reports / feature requests: <preferred channel>
- Generated files live in S3 and are downloadable for 1 hour at a time
  via the dashboard's Download button.

— <your-name>
```

---

# Day-to-day operations

## Deploying a new release

After you make changes locally:

```bash
# On your laptop:
git commit -am "feat: whatever you changed"
python scripts/pack_release.py        # uploads new release zip to S3

# SSH to server:
ssh ubuntu@<server-ip>
cd /opt/courseware/code
set -a; source /opt/courseware/.env; set +a
python scripts/deploy_release.py      # pulls latest.zip, swaps, restarts
```

Takes ~30 seconds. Brief 5-10s downtime during the restart.

## Adding new illustrations / banks

```bash
# On your laptop:
# (drop new files into images/<topic>/ or wherever)
python scripts/upload_banks_to_s3.py

# On server:
ssh ubuntu@<server-ip>
set -a; source /opt/courseware/.env; set +a
python /opt/courseware/code/scripts/sync_banks_from_s3.py
sudo systemctl restart courseware     # so it picks up new files
```

## Creating a new user (without SSHing)

Have your admin user log into the dashboard and POST to `/api/users` via curl, or build a tiny UI for it (parked work).

For now via SSH:

```bash
ssh ubuntu@<server-ip>
cd /opt/courseware/code
set -a; source /opt/courseware/.env; set +a
source /opt/courseware/venv/bin/activate
python -c "
from web.db import SessionLocal
from web.models import User
db = SessionLocal()
u = User(email='teammate@theknowledgeacademy.com', name='Teammate Name', is_admin=False)
u.set_password('temporary-password-they-should-change')
db.add(u); db.commit()
print(f'Created user id={u.id}')
"
```

## Watching logs in real time

```bash
sudo journalctl -u courseware -f     # follow the live stream
sudo journalctl -u courseware --since "1 hour ago"   # recent
sudo tail -f /var/log/courseware/app.log             # tail the file
```

## Rolling back a deploy

If a release breaks something:

```bash
ssh ubuntu@<server-ip>
cd /opt/courseware/code
python scripts/deploy_release.py --rollback
```

This swaps `code/` and `code_previous/`, restarts the service. ~5 seconds. You're back on the prior version.

---

# Troubleshooting

| Symptom | Likely cause | Fix |
|---|---|---|
| `systemctl start courseware` fails immediately | `.env` missing required var | `sudo journalctl -u courseware -n 30` → look for "KeyError" |
| 401 on every dashboard click | Session secret changed mid-deploy | Sessions invalidate on `SESSION_SECRET` change — users re-login. If you didn't change it, check the `.env` is being read. |
| 403 from S3 | IAM permissions or bucket name | `sudo cat /opt/courseware/.env | grep -E "S3_BUCKET|AWS_REGION"` — verify they match what's in AWS console |
| Build crashes "bank file not found" | Banks didn't sync | Re-run `sync_banks_from_s3.py` manually |
| Build crashes "no API key" | `ANTHROPIC_API_KEY` missing in `.env` | Add it; `sudo systemctl restart courseware` |
| Dashboard loads but charts empty | DB has no data | Normal on fresh install — make a build, charts populate |
| Server slow / out of memory | Concurrent builds | `sudo systemctl status courseware` → if it's been OOM-killed, you need more RAM, or limit concurrent builds |
| User session expires mid-build | Session lifetime too short | Already set to 8 hours in code. If problem persists, increase `permanent_session_lifetime` in `web/app.py` |
| Boot-time banks sync fails | Server can't reach S3 | Test: `curl https://s3.us-east-2.amazonaws.com/projectx-v1/` — should return 403 (means reachable). If it hangs, firewall is blocking outbound. |

---

# Windows Server alternative (if IT can't give you Linux)

The core ideas are identical; the commands differ. Key differences:

| Linux | Windows Server |
|---|---|
| SSH | RDP (Remote Desktop) |
| `apt install python3.11` | Download Python 3.11 installer from python.org |
| `sudo` | Run PowerShell as Administrator |
| `systemd` | Windows Services (use NSSM to wrap python as a service) |
| `cron` | Task Scheduler |
| `nginx` | IIS as reverse proxy (or skip, expose port 5000 directly) |
| `~/.ssh/` paths | `%USERPROFILE%\.ssh` |
| `/opt/courseware` | `C:\Program Files\Courseware\` or `C:\courseware\` |
| `/var/lib/courseware` | `D:\courseware-data\` (or wherever your persistent drive is) |
| `chown` / `chmod` | Right-click → Security tab → permissions |

If you end up on Windows Server, ping me and I'll write a Windows-specific runbook adapting each step.

---

# Reference

- Architecture rationale: [DEPLOYMENT_PLAN.md](DEPLOYMENT_PLAN.md)
- Phase 1–3 + 5 steps: [DEPLOYMENT_STEPS.md](DEPLOYMENT_STEPS.md)
- Earlier session memory: `C:\Users\admin\.claude\projects\c--Users-admin-Desktop-v2\memory\`

When in doubt, use the troubleshooting table first. Most Phase 4 problems are either:
1. `.env` missing a value
2. File permissions wrong
3. Firewall blocking a port

If you hit something not in the table, save the exact error from `journalctl` and the step you were on — I'll help diagnose.