SOP 003 Pre Upgrade Backup Rollback

SOP-003

Pre-Upgrade Backup & Rollback Procedure

Mandatory gate before any AAL or platform version upgrade

1. DOCUMENT INFORMATION

SOP ID SOP-003 Version 1.0

Created March 17, 2026 Last March 17, Updated 2026

Author Ozzy (Claude --- Atlantis AI) Approved Shane By

Category Change Management / Version Status Active Control

Applies To All Atlantis ITS team members
performing platform upgrades

Related INC-001 (AAL1.5 Rollback
Docs Incident), HT-001, TO-001

2. PURPOSE

This SOP defines the mandatory backup and verification steps required before initiating any platform version upgrade within the Atlantis ITS environment. It ensures that a clean, restorable snapshot exists prior to any upgrade attempt, enabling fast rollback with minimal downtime if the upgrade fails.

This procedure was formalized following INC-001, in which an incomplete backup of AAL1.5 resulted in approximately 5.5 hours of manual rebuild effort during a failed AAL2 upgrade.

3. SCOPE

This SOP applies to ALL of the following upgrade scenarios:

  • AAL version increments (e.g., AAL1.5 → AAL2, AAL2 → AAL3)

  • Major dependency or framework upgrades (Node.js, n8n, Docker base images)

  • Platform or environment migrations (e.g., new Cloudflare tunnel, new Docker network)

  • Any change that modifies the core platform state and cannot be easily undone

4. PREREQUISITES

Before executing this SOP, confirm the following are available:

  • Access to D:\Data\AtlantisITS\ (local working directory)

  • Git installed and authenticated to the Atlantis GitHub repository

  • Sufficient disk space for a ZIP snapshot (minimum 2x current project size)

  • VS Code or terminal access for Git operations

  • Backups folder exists at E:\Data\AtlantisITS\backups\ (create if missing)

5. STEP-BY-STEP PROCEDURE

+--------+------------------------------------------------------------------+ | # | Action | +:======:+==================================================================+ | 1 | Verify current state is clean and committed | | | | | | Run \'git status\' to confirm no uncommitted changes exist. | | | Stash or commit all pending work before proceeding. | +--------+------------------------------------------------------------------+ | | git status git add -A && git commit -m \"Pre-upgrade commit --- | | | AAL[X.X] stable\" | +--------+------------------------------------------------------------------+ | 2 | Tag the stable version in Git | | | | | | Create an annotated tag marking the current stable state. This | | | tag becomes your rollback anchor point. | +--------+------------------------------------------------------------------+ | | git tag -a AAL[X.X]-stable -m \"Stable AAL[X.X] before | | | upgrade to AAL[Y.Y]\" git push origin AAL[X.X]-stable | +--------+------------------------------------------------------------------+ | 3 | Create a local ZIP snapshot | | | | | | Copy and ZIP the entire project folder to the backups directory. | | | Name it with the AAL version and date for easy identification. | +--------+------------------------------------------------------------------+ | | Destination: | | | D:\Data\AtlantisITS\backups\AAL[X.X]-stable-YYYYMMDD.zip | | | Tip: Right-click project folder > Send to > Compressed | | | (zipped) folder | +--------+------------------------------------------------------------------+ | 4 | Verify the tag and snapshot | | | | | | Confirm the tag was pushed to GitHub and the ZIP file exists in | | | the backups folder before proceeding. | +--------+------------------------------------------------------------------+ | | git tag --list (confirm tag appears) Verify ZIP exists in | | | D:\Data\AtlantisITS\backups\ before continuing | +--------+------------------------------------------------------------------+ | 5 | Document the upgrade intent | | | | | | Log the planned upgrade in the team\'s changelog or Atlantis AI | | | memory file. Note what version you are moving from and to, and | | | why. | +--------+------------------------------------------------------------------+ | 6 | Proceed with upgrade | | | | | | Only after Steps 1--5 are confirmed complete should the upgrade | | | be initiated. If any step cannot be completed, DO NOT proceed. | +--------+------------------------------------------------------------------+

6. ROLLBACK PROCEDURE (IF UPGRADE FAILS)

If the upgrade fails at any point, follow these steps to restore the stable state:

+--------+------------------------------------------------------------------+ | # | Rollback Action | +:======:+==================================================================+ | A | Restore from Git tag | | | | | | Check out the stable tag to restore the codebase to the | | | pre-upgrade state. | +--------+------------------------------------------------------------------+ | | git checkout AAL[X.X]-stable git checkout -b | | | rollback-AAL[X.X] git push origin rollback-AAL[X.X] --force | +--------+------------------------------------------------------------------+ | B | Restore from ZIP (if Git unavailable) | | | | | | Extract the ZIP snapshot from D:\Data\AtlantisITS\backups\ | | | and overwrite the project directory. | +--------+------------------------------------------------------------------+ | C | Verify restored state | | | | | | Run git status and test the platform to confirm stable operation | | | before notifying the team. | +--------+------------------------------------------------------------------+ | D | File an Incident Report | | | | | | Document the failed upgrade as an INC-XXX record and add it to | | | the helpdesk / incident log. | +--------+------------------------------------------------------------------+

7. PRE-UPGRADE CHECKLIST (QUICK REFERENCE)

Done Checklist Item

☐ All changes committed --- git status is clean

☐ Git tag created and pushed to GitHub (AAL[X.X]-stable)

☐ ZIP snapshot saved to D:\Data\AtlantisITS\backups\

☐ Tag and ZIP both verified to exist

☐ Upgrade intent logged in changelog or AI memory file

☐ Team notified of planned upgrade window

✓ All items above confirmed --- SAFE TO PROCEED WITH UPGRADE

8. IMPORTANT NOTES & WARNINGS

  **⚠     Do NOT skip this SOP. The INC-001 incident (5.5hr rebuild)

WARNING** was caused by proceeding without a complete backup. This checklist is a mandatory gate, not optional guidance.

ℹ NOTE Git tags are lightweight and fast. There is no acceptable reason to skip Step 2. If you are unsure of the correct tag name, ask Ozzy before proceeding.

✓ TIP Name your ZIP with the date: AAL1.5-stable-20260317.zip. This makes it immediately clear when it was taken and allows multiple snapshots to coexist in the backups folder.

9. REVISION HISTORY

Version Date Author Notes

1.0 Mar 17, 2026 Ozzy / Shane Initial release --- drafted following INC-001 AAL1.5 rollback incident

1.1 Mar 17, 2026 Ozzy / Shane Updated backup path to E:\Backups\Projects\ (Toshiba Canvio) --- corrected from D:\Data\AtlantisITS\backups\

--- End of SOP-003 ---