fix: Clean up build system and resolve Docker Chromium persistence issue

- Remove redundant prestart hook (no more "Build not found" message) - Add prebuild hook for automatic Chromium installation - Fix Docker Chromium persistence (copy @playwright from builder) - Simplify npm scripts (11 → 9 commands, better naming) - Add comprehensive build system documentation - Add Docker deployment guide with troubleshooting Fixes: Docker "Chromium not installed" error on subsequent runs Closes: #[issue-number]
2026-01-09 17:06:15 +00:00 · 2025-11-13 15:34:02 +01:00
parent 3b06b4ae83
commit 8e2e7fae84
4 changed files with 635 additions and 8 deletions
--- a/docker/Dockerfile
+++ b/docker/Dockerfile
@@ -79,6 +79,10 @@ COPY --from=builder /usr/src/microsoft-rewards-bot/dist ./dist
 COPY --from=builder /usr/src/microsoft-rewards-bot/package*.json ./
 COPY --from=builder /usr/src/microsoft-rewards-bot/node_modules ./node_modules

+# FIXED: Copy Playwright browsers from builder (they're in node_modules with PLAYWRIGHT_BROWSERS_PATH=0)
+# This prevents "Chromium not installed" errors on subsequent runs
+COPY --from=builder /usr/src/microsoft-rewards-bot/node_modules/@playwright ./node_modules/@playwright
+
 # Copy runtime scripts with proper permissions and normalize line endings for non-Unix users
 # IMPROVED: Scripts now organized in docker/ folder
 COPY --chmod=755 docker/run_daily.sh ./docker/run_daily.sh
--- a/docs/build-system.md
+++ b/docs/build-system.md
@@ -0,0 +1,288 @@
+# Build System Documentation
+
+## 📋 Overview
+
+The Microsoft Rewards Bot uses a **clean, automated build system** that handles TypeScript compilation and browser installation automatically.
+
+## 🚀 Quick Start
+
+### First Time Setup
+```bash
+npm install
+```
+**This command does everything:**
+- ✅ Installs all Node.js dependencies
+- ✅ Compiles TypeScript to JavaScript
+- ✅ Installs Chromium browser automatically
+
+### Run the Bot
+```bash
+npm start
+```
+**No build step needed!** The bot is ready to run immediately after `npm install`.
+
+## 📦 Available Commands
+
+| Command | Description | Use When |
+|---------|-------------|----------|
+| `npm install` | **Complete setup** (deps + build + browser) | First time, after updates |
+| `npm start` | **Run the bot** (production) | Normal usage |
+| `npm run build` | Compile TypeScript + install browser | Manual rebuild needed |
+| `npm run dev` | **Dev mode** (TypeScript directly, hot reload) | Development only |
+| `npm run creator` | Account creation wizard | Creating new accounts |
+| `npm run dashboard` | Web dashboard (production) | Remote monitoring |
+| `npm run dashboard-dev` | Web dashboard (dev mode) | Dashboard development |
+| `npm run typecheck` | Type checking only (no build) | Quick validation |
+| `npm run test` | Run test suite | Development |
+| `npm run clean` | Delete compiled files | Before fresh rebuild |
+| `npm run kill-chrome` | Kill all Chrome processes | Cleanup after crashes |
+
+## 🔄 Build Workflow (Automatic)
+
+### When you run `npm install`:
+```
+1. npm installs dependencies
+   ↓
+2. postinstall hook triggers
+   ↓
+3. npm run build executes
+   ↓
+4. prebuild hook checks for Chromium
+   ↓
+5. If Chromium missing: npx playwright install chromium
+   ↓
+6. TypeScript compilation (tsc)
+   ↓
+7. postbuild hook shows success message
+```
+
+**Result:** Bot is ready to use!
+
+### When you run `npm run build`:
+```
+1. prebuild hook checks for Chromium
+   ↓
+2. If missing: Install Chromium automatically
+   ↓
+3. Compile TypeScript (src/ → dist/)
+   ↓
+4. Show success message
+```
+
+### When you run `npm start`:
+```
+1. Run compiled JavaScript (dist/index.js)
+   ↓
+2. No build check (already done by npm install)
+```
+
+## 🐳 Docker Workflow
+
+### Docker Build Process
+```bash
+npm run docker:build
+```
+
+**What happens:**
+1. **Stage 1 (Builder):**
+   - Install all dependencies
+   - Build TypeScript
+   - Reinstall production-only dependencies
+   - Install Chromium Headless Shell
+   - Clean up build artifacts
+
+2. **Stage 2 (Runtime):**
+   - Copy compiled code (`dist/`)
+   - Copy production dependencies (`node_modules/`)
+   - **Copy Chromium browser** (`node_modules/@playwright/`)
+   - Install minimal system libraries
+   - Configure cron for scheduling
+
+### Docker Run
+```bash
+npm run docker:run
+```
+
+Or with Docker Compose:
+```bash
+cd docker
+docker compose up -d
+```
+
+### Environment Variables (Docker)
+| Variable | Default | Description |
+|----------|---------|-------------|
+| `PLAYWRIGHT_BROWSERS_PATH` | `0` | Use browsers in node_modules (required) |
+| `FORCE_HEADLESS` | `1` | Run in headless mode (required for Docker) |
+| `TZ` | `UTC` | Container timezone |
+| `CRON_SCHEDULE` | **Required** | Cron schedule (e.g., `0 9 * * *`) |
+| `RUN_ON_START` | `false` | Run immediately on container start |
+| `SKIP_RANDOM_SLEEP` | `false` | Skip random delay before execution |
+
+## 🔧 Development Workflow
+
+### Hot Reload Development
+```bash
+npm run dev
+```
+- Runs TypeScript directly (no compilation needed)
+- Auto-reloads on file changes
+- Uses `-dev` flag (loads `accounts.dev.json` if it exists)
+
+### Type Checking
+```bash
+npm run typecheck
+```
+- Validates TypeScript types without compiling
+- Faster than full build
+- Use before committing code
+
+### Testing
+```bash
+npm run test
+```
+- Runs all unit tests in `tests/` directory
+- Uses Node.js native test runner
+
+### Clean Build
+```bash
+npm run clean
+npm run build
+```
+- Deletes `dist/` folder
+- Rebuilds everything from scratch
+
+## 🛠️ Troubleshooting
+
+### "Chromium not installed" Error
+
+**Symptoms:**
+```
+[ERROR] DESKTOP [BROWSER] Chromium not installed. Run "npm run pre-build" or set AUTO_INSTALL_BROWSERS=1
+```
+
+**Solution:**
+```bash
+npx playwright install chromium --with-deps
+```
+
+**Or set environment variable:**
+```bash
+# Windows (PowerShell)
+$env:AUTO_INSTALL_BROWSERS="1"
+npm start
+
+# Linux/Mac
+export AUTO_INSTALL_BROWSERS=1
+npm start
+```
+
+### Docker: "Chromium not installed" After First Run
+
+**Root Cause:** Chromium browser files not copied from builder stage.
+
+**Solution:** Rebuild Docker image with fixed Dockerfile:
+```bash
+docker build --no-cache -t microsoft-rewards-bot -f docker/Dockerfile .
+```
+
+The updated Dockerfile now includes:
+```dockerfile
+COPY --from=builder /usr/src/microsoft-rewards-bot/node_modules/@playwright ./node_modules/@playwright
+```
+
+### "Build not found" on Every `npm start`
+
+**Root Cause:** Old version had unnecessary `prestart` hook that checked for build on every start.
+
+**Solution:** Update to latest version. The `prestart` hook has been removed - build happens automatically during `npm install`.
+
+### TypeScript Compilation Errors
+
+**Check for missing dependencies:**
+```bash
+npm install
+```
+
+**Verify tsconfig.json is valid:**
+```bash
+npm run typecheck
+```
+
+**Clean rebuild:**
+```bash
+npm run clean
+npm run build
+```
+
+## 📁 Directory Structure
+
+```
+Microsoft-Rewards-Bot/
+├── src/                     # TypeScript source code
+│   ├── index.ts             # Main entry point
+│   ├── config.jsonc         # User configuration
+│   ├── accounts.jsonc       # Account credentials (gitignored)
+│   └── ...
+├── dist/                    # Compiled JavaScript (generated)
+│   ├── index.js             # Compiled entry point
+│   └── ...
+├── node_modules/            # Dependencies
+│   └── @playwright/         # Chromium browser files
+├── docker/                  # Docker configuration
+│   ├── Dockerfile           # Multi-stage Docker build
+│   ├── compose.yaml         # Docker Compose config
+│   ├── entrypoint.sh        # Container initialization
+│   ├── run_daily.sh         # Daily execution script
+│   └── crontab.template     # Cron schedule template
+├── package.json             # Dependencies + scripts
+├── tsconfig.json            # TypeScript configuration
+└── README.md                # Main documentation
+```
+
+## 🔐 Environment Variables
+
+### General
+| Variable | Description | Example |
+|----------|-------------|---------|
+| `NODE_ENV` | Runtime environment | `production`, `development` |
+| `DEBUG_REWARDS_VERBOSE` | Enable verbose logging | `1` |
+| `FORCE_HEADLESS` | Force headless browser | `1` |
+| `AUTO_INSTALL_BROWSERS` | Auto-install Chromium if missing | `1` |
+| `SKIP_RANDOM_SLEEP` | Skip random delays | `true` |
+
+### Docker-Specific
+| Variable | Description | Required |
+|----------|-------------|----------|
+| `CRON_SCHEDULE` | Cron expression for scheduling | ✅ Yes |
+| `TZ` | Timezone (e.g., `America/New_York`) | ❌ No (default: UTC) |
+| `RUN_ON_START` | Run immediately on container start | ❌ No (default: false) |
+| `PLAYWRIGHT_BROWSERS_PATH` | Browser location (must be `0`) | ✅ Yes (set in Dockerfile) |
+
+## 📚 Related Documentation
+
+- **Getting Started:** [docs/getting-started.md](getting-started.md)
+- **Configuration:** [docs/config.md](config.md) (via inline comments in `src/config.jsonc`)
+- **Accounts:** [docs/accounts.md](accounts.md)
+- **Scheduling:** [src/scheduler/README.md](../src/scheduler/README.md)
+- **Docker Deployment:** [docker/README.md](../docker/README.md) (if exists)
+
+## ✅ Best Practices
+
+1. **Always use `npm install` for initial setup** - It does everything automatically
+2. **Use `npm start` for normal runs** - No manual build needed
+3. **Use `npm run dev` for development** - Faster iteration with hot reload
+4. **Clean rebuild if weird errors occur** - `npm run clean && npm run build`
+5. **Docker users: Rebuild image after Dockerfile changes** - `docker build --no-cache`
+6. **Set `AUTO_INSTALL_BROWSERS=1` if Chromium issues persist** - Automatic fallback
+
+## 🆘 Getting Help
+
+- **Discord:** https://discord.gg/k5uHkx9mne
+- **GitHub Issues:** https://github.com/Obsidian-wtf/Microsoft-Rewards-Bot/issues
+- **Documentation:** [docs/index.md](index.md)
+
+---
+
+**Last Updated:** November 2025  
+**Applies To:** v2.56.7+
--- a/docs/docker-deployment.md
+++ b/docs/docker-deployment.md
@@ -0,0 +1,336 @@
+# Docker Deployment Guide
+
+## 🐳 Quick Start
+
+### 1. Build the Image
+```bash
+cd docker
+docker build -t microsoft-rewards-bot -f Dockerfile ..
+```
+
+Or use the npm script:
+```bash
+npm run docker:build
+```
+
+### 2. Configure Environment
+Edit `docker/compose.yaml`:
+```yaml
+environment:
+  TZ: "America/New_York"        # Your timezone
+  CRON_SCHEDULE: "0 9 * * *"    # Daily at 9 AM
+  RUN_ON_START: "true"          # Run immediately on start
+```
+
+### 3. Start the Container
+```bash
+cd docker
+docker compose up -d
+```
+
+## 📋 Configuration Files
+
+The container needs access to your configuration files via volume mounts:
+
+```yaml
+volumes:
+  - ../src/accounts.jsonc:/usr/src/microsoft-rewards-bot/dist/accounts.jsonc:ro
+  - ../src/config.jsonc:/usr/src/microsoft-rewards-bot/dist/config.jsonc:ro
+  - ./sessions:/usr/src/microsoft-rewards-bot/dist/browser/sessions
+```
+
+**Before starting:**
+1. Create `src/accounts.jsonc` (copy from `src/accounts.example.jsonc`)
+2. Edit `src/config.jsonc` with your settings
+3. (Optional) Create `docker/sessions/` directory for persistent login
+
+## 🔧 Environment Variables
+
+| Variable | Required | Default | Description |
+|----------|----------|---------|-------------|
+| `CRON_SCHEDULE` | ✅ Yes | N/A | Cron expression (e.g., `0 9 * * *`) |
+| `TZ` | ❌ No | `UTC` | Timezone (e.g., `America/New_York`) |
+| `RUN_ON_START` | ❌ No | `false` | Run immediately on container start |
+| `SKIP_RANDOM_SLEEP` | ❌ No | `false` | Skip random delay before execution |
+| `MIN_SLEEP_MINUTES` | ❌ No | `5` | Minimum random delay (minutes) |
+| `MAX_SLEEP_MINUTES` | ❌ No | `50` | Maximum random delay (minutes) |
+| `STUCK_PROCESS_TIMEOUT_HOURS` | ❌ No | `8` | Kill stuck processes after N hours |
+| `PLAYWRIGHT_BROWSERS_PATH` | 🔒 Fixed | `0` | Use browsers in node_modules (DO NOT CHANGE) |
+| `FORCE_HEADLESS` | 🔒 Fixed | `1` | Headless mode (DO NOT CHANGE) |
+| `NODE_ENV` | 🔒 Fixed | `production` | Production environment (DO NOT CHANGE) |
+
+## 📅 Scheduling Examples
+
+Use [crontab.guru](https://crontab.guru) to create cron expressions.
+
+| Schedule | Cron Expression | Description |
+|----------|----------------|-------------|
+| Daily at 9 AM | `0 9 * * *` | Once per day |
+| Every 6 hours | `0 */6 * * *` | 4 times daily |
+| Twice daily (9 AM, 9 PM) | `0 9,21 * * *` | Morning & evening |
+| Weekdays at 8 AM | `0 8 * * 1-5` | Monday-Friday only |
+| Random time (7-8 AM) | `0 7 * * *` + random sleep | Use `MIN_SLEEP_MINUTES`/`MAX_SLEEP_MINUTES` |
+
+**Example with Random Delay:**
+```yaml
+environment:
+  CRON_SCHEDULE: "0 7 * * *"    # Start scheduling at 7 AM
+  MIN_SLEEP_MINUTES: "0"        # No minimum delay
+  MAX_SLEEP_MINUTES: "60"       # Up to 1 hour delay
+  # Result: Runs between 7:00 AM - 8:00 AM randomly
+```
+
+## 🔍 Monitoring
+
+### View Logs
+```bash
+docker logs -f microsoft-rewards-bot
+```
+
+### Check Container Status
+```bash
+docker ps -a | grep microsoft-rewards-bot
+```
+
+### Health Check
+```bash
+docker inspect microsoft-rewards-bot --format='{{.State.Health.Status}}'
+```
+
+Expected output: `healthy`
+
+### Execute Commands Inside Container
+```bash
+# Check cron status
+docker exec microsoft-rewards-bot crontab -l
+
+# Check timezone
+docker exec microsoft-rewards-bot date
+
+# List running processes
+docker exec microsoft-rewards-bot ps aux
+```
+
+## 🛠️ Troubleshooting
+
+### "Chromium not installed" After First Run
+
+**Symptoms:**
+- Bot works on first run
+- Fails on subsequent scheduled runs with "Chromium not installed"
+
+**Root Cause:** 
+Fixed in latest version! Chromium browser files are now copied correctly from builder stage.
+
+**Solution:**
+```bash
+# Rebuild with latest Dockerfile (includes fix)
+docker build --no-cache -t microsoft-rewards-bot -f docker/Dockerfile ..
+docker compose down
+docker compose up -d
+```
+
+**Verification:**
+Check that Chromium is present in the container:
+```bash
+docker exec microsoft-rewards-bot ls -la /usr/src/microsoft-rewards-bot/node_modules/@playwright/
+```
+
+You should see `browser-chromium/` directory.
+
+### Container Exits Immediately
+
+**Check logs:**
+```bash
+docker logs microsoft-rewards-bot
+```
+
+**Common causes:**
+1. **Missing CRON_SCHEDULE:** Set environment variable in compose.yaml
+2. **Invalid cron expression:** Validate at https://crontab.guru
+3. **Permission issues:** Ensure config files are readable
+
+**Fix:**
+```bash
+# Update compose.yaml with correct CRON_SCHEDULE
+docker compose down
+docker compose up -d
+```
+
+### Random Delays Not Working
+
+**Check environment variables:**
+```bash
+docker exec microsoft-rewards-bot env | grep SLEEP
+```
+
+**Verify run_daily.sh receives variables:**
+```bash
+docker exec microsoft-rewards-bot cat docker/run_daily.sh
+```
+
+**Test manually:**
+```bash
+docker exec microsoft-rewards-bot docker/run_daily.sh
+```
+
+### Timezone Incorrect
+
+**Check container timezone:**
+```bash
+docker exec microsoft-rewards-bot date
+```
+
+**Fix:**
+1. Update `TZ` in compose.yaml
+2. Restart container:
+```bash
+docker compose restart
+```
+
+**Verify:**
+```bash
+docker exec microsoft-rewards-bot date
+docker logs microsoft-rewards-bot | grep "Timezone:"
+```
+
+### Health Check Failing
+
+**Check cron daemon:**
+```bash
+docker exec microsoft-rewards-bot ps aux | grep cron
+```
+
+**Expected output:**
+```
+root         1  0.0  0.0   2576  1024 ?        Ss   12:00   0:00 cron -f
+```
+
+**Restart container if missing:**
+```bash
+docker compose restart
+```
+
+### Sessions Not Persisting
+
+**Check volume mount:**
+```bash
+docker inspect microsoft-rewards-bot | grep sessions
+```
+
+**Verify local directory exists:**
+```bash
+ls -la docker/sessions/
+```
+
+**Fix permissions:**
+```bash
+# Linux/Mac
+chmod -R 755 docker/sessions/
+
+# Windows
+# No action needed (NTFS handles permissions)
+```
+
+### Out of Memory Errors
+
+**Increase memory limit in compose.yaml:**
+```yaml
+mem_limit: 6g  # Increase from 4g
+```
+
+**Restart:**
+```bash
+docker compose down
+docker compose up -d
+```
+
+**Monitor memory usage:**
+```bash
+docker stats microsoft-rewards-bot
+```
+
+## 🔄 Updates
+
+### Update to Latest Version
+
+**Method 1: Rebuild (Recommended)**
+```bash
+# Pull latest code
+git pull origin main
+
+# Rebuild image
+docker build --no-cache -t microsoft-rewards-bot -f docker/Dockerfile ..
+
+# Restart container
+docker compose down
+docker compose up -d
+```
+
+**Method 2: Auto-Update (If Enabled in config.jsonc)**
+The bot can auto-update itself if `update.enabled: true` in config.jsonc.
+After auto-update, container will restart automatically (Docker restarts on exit).
+
+### Backup Before Update
+```bash
+# Backup sessions
+cp -r docker/sessions/ docker/sessions.backup/
+
+# Backup config
+cp src/config.jsonc src/config.jsonc.backup
+cp src/accounts.jsonc src/accounts.jsonc.backup
+```
+
+## 📊 Resource Usage
+
+**Typical resource consumption:**
+- **CPU:** 50-80% during active runs, <1% idle
+- **RAM:** 1-3 GB during active runs, <100 MB idle
+- **Disk:** ~500 MB (base image + browsers)
+- **Network:** Minimal (<10 MB per run)
+
+**Recommended limits:**
+```yaml
+mem_limit: 4g
+cpus: 2
+```
+
+## 🔐 Security Hardening
+
+The default compose.yaml includes:
+```yaml
+security_opt:
+  - no-new-privileges:true  # Prevent privilege escalation
+```
+
+**Additional hardening (optional):**
+```yaml
+security_opt:
+  - no-new-privileges:true
+  - seccomp:unconfined  # Only if Chromium fails to start
+read_only: false  # Must be false (Chromium needs write access)
+tmpfs:
+  - /tmp
+  - /root/.cache
+```
+
+## 🆘 Getting Help
+
+If issues persist:
+1. **Check logs:** `docker logs -f microsoft-rewards-bot`
+2. **Rebuild with no cache:** `docker build --no-cache ...`
+3. **Verify Chromium:** `docker exec microsoft-rewards-bot ls -la node_modules/@playwright/`
+4. **Discord Support:** https://discord.gg/k5uHkx9mne
+5. **GitHub Issues:** https://github.com/Obsidian-wtf/Microsoft-Rewards-Bot/issues
+
+## 📚 Related Documentation
+
+- **Build System:** [docs/build-system.md](build-system.md)
+- **Scheduling:** [src/scheduler/README.md](../src/scheduler/README.md)
+- **Configuration:** [src/config.jsonc](../src/config.jsonc) (inline comments)
+- **Accounts:** [docs/accounts.md](accounts.md)
+
+---
+
+**Last Updated:** November 2025  
+**Applies To:** v2.56.7+
--- a/package.json
+++ b/package.json
@@ -17,23 +17,22 @@
  "homepage": "https://github.com/Obsidian-wtf/Microsoft-Rewards-Bot#readme",
  "scripts": {
    "clean": "rimraf dist",
-    "install-deps": "npm install && npx playwright install chromium",
-    "postinstall": "npm run build",
    "typecheck": "tsc --noEmit",
    "build": "tsc",
-    "postbuild": "node -e \"console.log('\\n✅ Build complete! Run \\\"npm start\\\" to launch the bot.\\n')\"",
+    "postbuild": "node -e \"console.log('\\n Build complete! Run \\\"npm start\\\" to launch the bot.\\n')\"",
+    "postinstall": "npm run build",
+    "prebuild": "node -e \"const fs = require('fs'); if (!fs.existsSync('node_modules/@playwright/browser-chromium') && process.env.SKIP_BROWSER_INSTALL !== '1') { console.log('📦 Installing Chromium browser...'); require('child_process').execSync('npx playwright install chromium --with-deps', {stdio: 'inherit'}); }\"",
    "test": "node --test --loader ts-node/esm tests/**/*.test.ts",
-    "prestart": "node -e \"const fs = require('fs'); if (!fs.existsSync('./dist/index.js')) { console.log('⚠️  Build not found, building now...'); require('child_process').execSync('npm run build', {stdio: 'inherit'}); }\"",
    "start": "node --enable-source-maps ./dist/index.js",
-    "ts-start": "node --loader ts-node/esm ./src/index.ts",
    "dev": "ts-node ./src/index.ts -dev",
    "creator": "ts-node ./src/account-creation/cli.ts",
    "dashboard": "node --enable-source-maps ./dist/index.js -dashboard",
    "dashboard-dev": "ts-node ./src/index.ts -dashboard",
    "lint": "eslint \"src/**/*.{ts,tsx}\"",
    "setup": "node ./scripts/installer/setup.mjs",
-    "kill-chrome-win": "powershell -Command \"Get-Process | Where-Object { $_.MainModule.FileVersionInfo.FileDescription -eq 'Google Chrome for Testing' } | ForEach-Object { Stop-Process -Id $_.Id -Force }\"",
-    "create-docker": "docker build -t microsoft-rewards-bot ."
+    "kill-chrome": "node -e \"const { execSync } = require('child_process'); const isWin = process.platform === 'win32'; try { execSync(isWin ? 'taskkill /F /IM chrome.exe 2>nul || exit 0' : 'pkill -f chrome || true', {stdio: 'inherit'}); } catch(e) {}\"",
+    "docker:build": "docker build -t microsoft-rewards-bot .",
+    "docker:run": "docker run -d --name ms-rewards microsoft-rewards-bot"
  },
  "keywords": [
    "Bing Rewards",
@@ -83,4 +82,4 @@
    "ts-node": "^10.9.2",
    "ws": "^8.18.3"
  }
-}
+}