HT-013 — PM2 Process Manager (Nova-Alpha)

Purpose

PM2 is the process manager that keeps Nova's WSL-side services alive, restarts them on crash, and brings them back after reboots. This document is the single source of truth for PM2 on Nova-Alpha.

If you're ever confused about what's running, why something stopped, or how to make a new WSL-side service survive power loss — this is your first stop.

Scope

PM2 manages WSL-side services ONLY. This includes:

• Node.js apps (Next.js, Express, etc.)
• Python scripts (Discord bots, agents)
• Any Linux-side executable (Docker CLI scripts, shell scripts, etc.)

PM2 does NOT manage Windows-side services. For Windows services such as the Nova Hardware API (C# / .NET), see HT-014 — Windows Service Management.

This is by design, not a limitation. See "Dual Ignition Architecture" below.

Dual Ignition Architecture

Nova-Alpha runs two parallel ecosystems on the same physical machine (Trunks). They talk to each other across a network bridge, but each has its own startup/lifecycle management.

┌─────────────────────────────────────────────┐
│  WINDOWS SIDE (Trunks host OS)              │
│                                             │
│  • .NET / C# apps (hardware access)        │
│  • LibreHardwareMonitor (needs Win APIs)   │
│  • Future: Immich, game servers, etc.      │
│                                             │
│  Ignition: Windows Services / Task Sched.  │
│  See HT-014                                 │
└─────────────────────────────────────────────┘
                    ↑↓
         (172.26.x.x bridge)
                    ↑↓
┌─────────────────────────────────────────────┐
│  WSL / UBUNTU SIDE                          │
│                                             │
│  • Node.js apps (OPS dashboard)            │
│  • Python apps (Discord bots)              │
│  • Docker containers (future)              │
│  • Kali tools (coming soon)                │
│                                             │
│  Ignition: PM2                              │
│  See HT-013 (this doc)                      │
└─────────────────────────────────────────────┘

Mental model: Two engines. Same machine. Different ignition keys.

PM2 = Node/Python ignition. Windows Services = .NET/hardware ignition.

What PM2 Is

PM2 stands for Process Manager 2. It is a Node.js-based process manager that runs Node, Python, or any other executable as a managed background service.

Key capabilities:

• Runs apps in the background (detached from terminal)
• Auto-restarts apps when they crash
• Auto-restarts apps when the machine reboots (with setup)
• Centralized logging
• Resource monitoring (CPU, memory per process)
• Zero-downtime reloads
• Cluster mode for load balancing

PM2 was originally built for the Node.js ecosystem but handles Python, shell scripts, and any command-line executable.

Where PM2 Runs on Nova-Alpha

PM2 is installed inside WSL (Ubuntu) on Nova-Alpha. It does not run in Windows.

• Installation path: /usr/lib/node_modules/pm2/
• Binary: /usr/bin/pm2
• User data: ~/.pm2/

To reach PM2, always enter WSL first:

wsl
# or from Windows Terminal Ubuntu profile

PowerShell cannot run pm2 commands directly unless you set up aliases (see HT-011).

Installation (For Reference)

PM2 was installed on Nova-Alpha using:

sudo npm install -g pm2

To verify PM2 is installed:

pm2 --version

Core Commands

Starting Apps

Start a Node.js app (script file):

pm2 start app.js --name my-app

Start an npm script:

pm2 start npm --name atlantis-ops -- run start

Start a Python script:

pm2 start bot.py --name ozzy-bot --interpreter python3

Start with environment variables:

pm2 start app.js --name my-app --env production

Viewing Status

pm2 list                    # pretty table of all processes
pm2 status                  # same thing
pm2 show atlantis-ops       # detailed info on one process
pm2 jlist                   # JSON output (useful for dashboards)

Controlling Apps

pm2 restart atlantis-ops    # restart a process
pm2 stop atlantis-ops       # stop (stays in list)
pm2 delete atlantis-ops     # remove from list entirely
pm2 restart all             # restart every process
pm2 stop all                # stop every process

Logs

pm2 logs                            # live tail all logs
pm2 logs atlantis-ops               # tail one app
pm2 logs atlantis-ops --lines 100   # last 100 lines
pm2 flush                           # clear all logs
pm2 flush atlantis-ops              # clear one app's logs

Monitoring

pm2 monit                   # live CPU/RAM dashboard (full-screen)

Persistence — Surviving Reboots and Power Loss

This is the most important section. Without persistence setup, every reboot or power loss wipes PM2's process list.

Note: This only handles WSL-side services. The Windows Hardware API has its own auto-start mechanism documented in HT-014.

Step 1 — Generate Boot Script (inside WSL)

Run once:

pm2 startup systemd

PM2 will output a command that looks like:

sudo env PATH=$PATH:/usr/bin /usr/lib/node_modules/pm2/bin/pm2 startup systemd -u shane --hp /home/shane

Copy and run that command exactly as shown. This registers PM2 with systemd so it starts when WSL starts.

Step 2 — Save Current Process List

After starting all the apps you want to persist:

pm2 save

This writes the current list to ~/.pm2/dump.pm2. PM2 will resurrect these apps automatically on boot.

Step 3 — WSL Auto-Start on Windows Boot (Nova-Alpha Specific)

This is a two-layer problem on Nova-Alpha:

1. WSL must start when Windows boots (WSL is off by default)
2. PM2 must resurrect saved processes once WSL is running (handled by Step 1 + Step 2 above)

To solve layer 1, choose one:

Option A — Windows Task Scheduler (recommended):

1. Press Win + R, type taskschd.msc, press Enter
2. Create Basic Task → Name: Start WSL on Boot
3. Trigger: When I log on
4. Action: Start a program
5. Program: wsl.exe
6. Arguments: -d Ubuntu -u shane

Because systemd + PM2 are already configured inside WSL, starting WSL automatically resurrects everything PM2 had saved.

Option B — Startup Shortcut:

Create a .bat file at:

C:\Users\Shane\AppData\Roaming\Microsoft\Windows\Start Menu\Programs\Startup\nova-wsl.bat

Contents:

@echo off
wsl -d Ubuntu -u shane

Testing Persistence

After setup, reboot the machine. When you log in and open WSL:

pm2 list

All previously saved apps should be running.

Current Nova-Alpha Process Inventory

Managed by PM2 (WSL side)

Managed by Windows (NOT PM2)

Expected PM2 Additions

Common Workflows

Adding a New WSL-Side Service

cd ~/nova/agents/ozzy-bot
pm2 start ozzy_bot.py --name ozzy-bot --interpreter python3
pm2 save                          # persist the new addition

Updating a Service After Code Changes

cd ~/nova/atlantis-ops
git pull
pm2 restart atlantis-ops

Troubleshooting a Crashed Service

pm2 list                          # check status column
pm2 logs atlantis-ops --lines 50  # see why it crashed
pm2 restart atlantis-ops          # try restarting
pm2 show atlantis-ops             # detailed info including restart count

Zero-Downtime Reload

pm2 reload atlantis-ops           # reload without dropping connections

Integration With Command Center

PM2 exposes a JSON API via pm2 jlist. The Atlantis OPS Command Center can consume this to show live service status alongside hardware telemetry.

Example Node.js integration pattern for the Next.js dashboard:

const { exec } = require('child_process');

exec('pm2 jlist', (err, stdout) => {
  if (err) return;
  const processes = JSON.parse(stdout);
  // pass to dashboard
});

This enables a "Services" card on the Command Center showing green/red status for every managed PM2 process.

Note: For Windows-managed services (hardware API), the dashboard queries them directly over HTTP at 172.26.x.x:5002. Each ignition system reports independently.

Troubleshooting

pm2: command not found

PM2 is installed in WSL only. Either enter WSL first with wsl, or set up PowerShell aliases (see HT-011).

Services Did Not Restart After Reboot

Check that both persistence layers are completed:

systemctl status pm2-shane        # systemd integration active?
cat ~/.pm2/dump.pm2               # saved process list exists?

Also confirm the Windows auto-start (Task Scheduler or Startup .bat) is actually firing. If WSL never starts, PM2 never runs.

"Why Isn't PM2 Restarting the Hardware API?"

It never will. The hardware API is a Windows .NET process and is outside PM2's scope. See HT-014 for Windows auto-start setup.

Memory Keeps Growing

Some apps leak memory over time. PM2 can auto-restart based on memory threshold:

pm2 start app.js --name my-app --max-memory-restart 300M

Service Restarts in a Loop

The restart counter (↺ column in pm2 list) will keep climbing. Check logs:

pm2 logs my-app --lines 100

Stop the loop while you debug:

pm2 stop my-app

Related Documents

• HT-011 — Nova-Alpha Quick Access (Terminal profiles, PowerShell aliases, shortcuts)
• HT-012 — Nova Workspace Structure (folder taxonomy, Git/SSH setup)
• HT-014 — Windows Service Management (for .NET / Hardware API / non-WSL services)
• HT-015 — WSL ↔ Windows Bridge (cross-ecosystem networking)
• SOP-004 — Nova-Alpha Desktop Build (comprehensive setup)
• INC-001 — AAL v2 Upgrade Failure (backup/rollback reference)

Revision History

End of HT-013 — PM2 Process Manager (Nova-Alpha)