SOP-008 — AI Production Risk Review (NIST AI RMF + OWASP AI/LLM + CIS Benchmark)

1. Purpose

This Standard Operating Procedure defines the Atlantis ITS review process for moving AI-enabled applications, agents, automations, and public-facing workflows into production.

It combines two complementary frameworks:

• NIST AI Risk Management Framework (AI RMF) — governance, lifecycle risk, trustworthiness, accountability, and documentation
• OWASP AI/LLM security guidance — technical vulnerabilities, adversarial testing, attack surface review, and mitigations

NIST tells Atlantis what risks to govern. OWASP tells Atlantis how attackers may break the system.

This SOP exists to prevent Atlantis ITS from shipping AI systems that are useful but unsafe, poorly governed, vulnerable to prompt injection, leaky with data, overly autonomous, missing a human approval gate, or unclear in ownership.

Version 0.2 adds Jimmy’s governance updates:

• Human-in-the-loop (HITL) requirements
• Long-term memory / RAG hygiene
• Automated secrets scanning
• Excessive agency review
• Kill Switch protocol
• Environment separation
• Incident-owner alert path

Version 0.4 adds CIS Ubuntu 24.04 LTS Benchmark integration:

• CIS Benchmark infrastructure baseline added as a third framework pillar (Section 4.3)
• Infrastructure Security Baseline section added (Section 16.3, 16.4, 16.5)
• CIS audit script reference added (~/nova/cis_wsl2_audit.sh)
• VPS pre-deployment CIS requirement added (Section 16.4)
• atlantis-lab sandbox CIS container baseline added (Section 16.5)
• Agent tech watch sandbox rules updated to reference CIS hardening (Section 16.5)
• OpenClaw CVE-2026-25253 "ClawBleed" (CVSS 8.8) flagged in OWASP supply chain review (Section 12)
• Production readiness checklist updated with CIS audit requirement (Section 19)
• Related documents updated with CIS Benchmark reference (Section 22)

• Multi-user isolation review added to Intake Template (Section 8)
• Kill Switch KPI tightened to < 5 minutes in Nova Router checklist (Section 14.1)
• Docker hardening checklist added to Environment Separation section (Section 16)
• Rate Limiting / Abuse Prevention section added (Section 14.4)
• Privacy fix: full last name removed from approval table (Section 24)

2. Scope

In scope:

• Public Atlantis apps moved to GitHub AtlantisITS
• Apps deployed to Vercel, Cloudflare, VPS, or public domains
• Nova Router / Nova Bus routes promoted beyond local testing
• n8n workflows handling real leads, customer data, travel inquiries, contractor requests, or alerts
• AI chat intake pages, including Home Help / Nova chat intake
• LLM-powered summaries, recommendations, lead scoring, content generation, or agent-chain decisions
• Apps using external AI APIs, OpenRouter, local Ollama models, or n8n AI workflows
• Production-facing workflows connected to Google Sheets, CRM, Telegram, Discord, email, or other notification systems
• AI agents with ability to execute code, write files, update memory, send public emails, post externally, delete records, or write to external databases
• Systems using shared RAG, long-term memory, vector stores, or persistent AI context

Out of scope:

• Private experiments with no customer data and no public exposure
• One-off local scripts that do not connect to customer, lead, or production systems
• Purely static marketing pages with no AI, forms, customer data, or automation
• Full legal/compliance audit by external counsel
• Formal certification against NIST, ISO, SOC 2, or other third-party audit frameworks

3. Ownership

4. Framework Summary

NIST AI RMF

NIST AI RMF is the governance and lifecycle backbone.

It focuses on:

• Govern
• Map
• Measure
• Manage

Atlantis uses NIST AI RMF to answer:

• Who owns this system?
• What risks exist?
• What data is used?
• What documentation exists?
• What monitoring is required?
• What happens when the system fails?
• What is the expected impact on customers, users, operators, and the business?
• Where must a human review or approve the AI action?

OWASP AI/LLM Security

OWASP is the hands-on security and adversarial testing layer.

Atlantis uses OWASP guidance to review:

• Prompt injection
• Data leakage
• Insecure output handling
• Model or data poisoning
• Overreliance on LLM output
• Supply chain risk
• Sensitive information disclosure
• Agent/tool misuse
• Authentication and authorization gaps
• Excessive agency
• Logging and monitoring blind spots

4.3 CIS Ubuntu 24.04 LTS Benchmark

CIS Benchmark is the infrastructure hardening baseline.

Atlantis uses CIS Level 1 to enforce:

• Filesystem security (kernel modules, partition options)
• Service minimization (disable unused services)
• Network parameter hardening (sysctl settings)
• Host-based firewall (UFW active, default deny inbound)
• SSH hardening (no root login, MaxAuthTries ≤ 4)
• Password policy (PASS_MAX_DAYS, PASS_MIN_DAYS, PASS_WARN_AGE)
• Audit logging (auditd, rsyslog)
• File permission integrity (/etc/passwd, /etc/shadow, /etc/group)
• Account hygiene (no UID 0 duplicates, no empty passwords)

Audit script: ~/nova/cis_wsl2_audit.sh Minimum score: 80% pass rate (excluding WSL2 skips) before production Re-audit required: after any major system change, kernel update, or new service deployment

NIST tells Atlantis what risks to govern. OWASP tells Atlantis how attackers may break the system. CIS tells Atlantis how to harden the infrastructure they run on.

5. Trigger Conditions

Run this SOP before production release when any of the following are true:

• An app is moved to the public AtlantisITS GitHub organization
• An app is deployed to a public domain or public subdomain
• A Nova route is exposed to production traffic
• A workflow handles real leads, customers, contractor requests, travel inquiries, or business data
• A form, chatbot, or intake system sends data into n8n, Sheets, CRM, Discord, Telegram, or email
• LLM output influences user-facing responses, routing, lead qualification, scoring, summaries, or decisions
• A workflow uses API keys, secrets, credentials, or external AI services
• A system becomes part of the revenue path
• An agent gains tool access that can execute code, delete files, modify records, send emails, post publicly, write to memory, or write to external systems
• A system uses shared RAG, long-term memory, persistent context, or vector storage
• A repo is moved from private/internal Forgejo to public GitHub

If unsure, run the review.

6. Architecture

App / Page / Agent / Workflow
        ↓
SOP-008 Intake
        ↓
NIST AI RMF Review
        ↓
OWASP AI/LLM Security Review
        ↓
Atlantis Architecture Review
        ↓
Privacy / Public Content Review
        ↓
HITL + Kill Switch Review
        ↓
Risk Rating + Mitigation Plan
        ↓
Sign-off Gate
        ↓
Production Release
        ↓
Post-launch Monitoring

For Nova/Atlantis systems:

Public App / QR / Chat Intake
        ↓
Nova Router / Nova Bus
        ↓
Agent or Lead Engine
        ↓
n8n Workflow
        ↓
Sheets / CRM / Alerting
        ↓
Human Review / Follow-up / Monitoring / Incident Path

7. Prerequisites

Before starting SOP-008, collect:

• App or workflow name
• Repo path and branch
• Owner / lead agent
• Production target URL or internal route
• Data handled
• Authentication method
• Secrets used
• n8n workflow names or IDs
• Nova route names, if applicable
• Cloudflare / Vercel / VPS deployment notes
• Known risks or prior incidents
• Whether the system affects revenue, client trust, or public reputation
• Whether the system has any agent/tool permissions
• Whether the system writes to memory, RAG, Sheets, CRM, files, or external systems
• Whether a human approval gate exists
• Whether a Kill Switch exists and has been tested
• Whether the repo has been secrets-scanned

8. Intake Template

Create or complete the following intake block:

# SOP-008 Intake

## System
- Name:
- Type: app / agent / workflow / route / page / automation
- Repo:
- Branch:
- Owner:
- Production URL:
- Internal URL:
- Current status: draft / staging / production candidate / production

## Business Context
- Purpose:
- Audience:
- Revenue impact:
- Customer or lead impact:
- Related funnel:
- Multi-user system? yes/no
- User isolation model: (e.g., namespace, database-level, agent-level, none)

## Data
- Data collected:
- Data stored:
- Data transmitted:
- Sensitive data present? yes/no
- Customer/client data present? yes/no

## AI Usage
- AI model(s):
- Local or external:
- Prompting involved:
- Agent/tool access:
- Human approval required:
- Shared memory / RAG / vector store involved? yes/no

## Agent Agency
- Can the agent execute code? yes/no
- Can the agent write files? yes/no
- Can the agent update memory/RAG? yes/no
- Can the agent send email/messages? yes/no
- Can the agent post publicly? yes/no
- Can the agent delete or modify records? yes/no
- Can the agent move money, trigger billing, or affect revenue? yes/no

## Integrations
- Nova routes:
- n8n workflows:
- External APIs:
- Alert channels:
- Storage destinations:

## Controls
- Auth method:
- HITL gate:
- Kill Switch path:
- Incident owner:
- Logging path:
- Rollback path:
- Secrets scan completed? yes/no
- Rate limiting / abuse prevention implemented? yes/no

## Known Risks
- Known security concerns:
- Known privacy concerns:
- Known reliability concerns:
- Known unresolved TODOs:

9. NIST AI RMF Review

9.1 Govern

Required output:

Govern result:
- Status:
- Required fixes:
- Owner:

9.2 Map

Required output:

Map result:
- Status:
- Key risks:
- Required fixes:

9.3 Measure

Required output:

Measure result:
- Status:
- Tests run:
- Evidence:

9.4 Manage

Required output:

Manage result:
- Status:
- Risk rating:
- Go / no-go:

10. Human-in-the-Loop (HITL) Requirements

HITL is required for high-impact or irreversible actions.

10.1 HITL Required When

A human must review or approve before the AI/agent can:

• Delete files, records, rows, tickets, issues, repos, or workflows
• Send public emails or external customer messages
• Post publicly to social media, websites, Chronicle, Briefing, or public feeds
• Modify long-term memory, RAG, production prompts, or shared context
• Affect billing, pricing, payments, refunds, commissions, or revenue records
• Route real customer/lead data to a third party
• Trigger contractor/customer outreach
• Execute code that changes production infrastructure or data
• Change credentials, auth rules, Cloudflare settings, Vercel settings, or firewall behavior

10.2 HITL Review Questions

Required output:

HITL result:
- Required? yes/no
- Reviewer:
- Approval path:
- Launch blocker? yes/no

11. Memory / RAG Hygiene Review

Run this review for any system that reads from or writes to:

• ATLANTIS_AI_MEMORY.md
• updates.md
• Persona files
• Prompt files
• RAG stores
• Vector databases
• Shared agent memory
• Google Sheets or docs used as persistent context

11.1 Memory Hygiene Checks

Required output:

Memory hygiene result:
- Status:
- Highest risk:
- Required mitigations:

12. OWASP AI/LLM Security Review

Use this section for adversarial review.

Required output:

OWASP result:
- Highest risk:
- Required mitigations:
- Blocker? yes/no

13. Automated Secrets Scanning

Before moving any repo to public GitHub or production deployment, perform automated secrets scanning.

13.1 Minimum Requirement

Run at least one secrets scan before release.

Suggested tools:

• TruffleHog
• Gitleaks
• GitHub secret scanning, if available
• Manual grep for common secret patterns as fallback

13.2 Example Commands

# TruffleHog example
trufflehog git file://$PWD --only-verified

# Gitleaks example
gitleaks detect --source . --redact

# Lightweight fallback checks
grep -R "OPENAI_API_KEY\|ANTHROPIC_API_KEY\|BRAVE_API_KEY\|N8N\|TOKEN\|SECRET\|PASSWORD" .   --exclude-dir=node_modules   --exclude-dir=.git

13.3 Secrets Scan Checklist

Required output:

Secrets scan result:
- Tool used:
- Result:
- Secrets found? yes/no
- Rotations required:

14. Atlantis-Specific Review

14.1 Nova Router / Nova Bus

14.2 n8n Workflow

14.3 Public App / Page

14.4 Abuse Prevention & Rate Limiting

Required output:

Abuse prevention result:
- Rate limiting: yes/no
- Method:
- Honeypot: yes/no
- Launch blocker? yes/no

14.5 GitHub / Repo

15. Kill Switch Protocol

Every production AI system, route, workflow, or public intake path must have a documented Kill Switch.

The Kill Switch must be simple enough to execute under stress.

15.1 Kill Switch Options

Acceptable Kill Switches include:

• Disable route in Nova Router registry
• Disable n8n workflow
• Rotate or revoke API key
• Disable Cloudflare route / Access policy
• Remove Vercel production domain
• Set feature flag to disabled
• PM2 stop/restart for local production-like services
• Block webhook path at reverse proxy
• Temporarily replace public page with maintenance message

15.2 Kill Switch Checklist

Required output:

Kill Switch result:
- Owner:
- Method:
- Tested? yes/no
- Launch blocker? yes/no

15.3 Minimum Kill Switch Record

## Kill Switch Record
- System:
- Owner:
- Disable method:
- Command / UI path:
- Alert channel:
- Re-enable method:
- Last tested:

16. Environment Separation & Docker Hardening

Production systems must not use development credentials, test data stores, staging alert channels, or unhardened containers unless explicitly documented.

16.1 Environment Separation

16.2 Docker Hardening

Run this checklist for any system deployed in a Docker container.

Required output:

Environment & Docker hardening result:
- Environment separation status:
- Docker hardening status: (N/A if not containerized)
- Required fixes:

16.3 CIS Ubuntu 24.04 LTS Infrastructure Baseline

All Atlantis production infrastructure (Trunks WSL2, Hetzner VPS, Docker hosts) must meet CIS Ubuntu 24.04 LTS Level 1 benchmark before any production service is deployed.

Audit script: ~/nova/cis_wsl2_audit.sh

CIS Level 1 minimum requirements before production:

Required output:

CIS baseline result:
- System audited:
- Audit date:
- Score:
- FAIL items:
- WARN items:
- Remediation status:
- Launch blocker? yes/no

16.4 VPS Pre-Deployment CIS Requirement

Before any Atlantis service (Forgejo, n8n, NTFY, Prometheus, Grafana, Portainer) is deployed to Hetzner or Hostinger VPS:

Sequence: CIS audit PASS → UFW configured → SSH hardened → then deploy services. Never the other way.

16.5 atlantis-lab Sandbox CIS Container Baseline

All agent tool experiments (Freebuff, UI-TARS, OpenHuman, Hermes, OpenClaw, and any future agent tech watch candidates) must run inside a CIS-hardened container environment in atlantis-lab.

Special rule for UI-TARS and any desktop-control agent: UI-TARS and similar tools that can control mouse and keyboard must run in a fully isolated WSL profile or VM with no access to the host Windows session, no saved Windows credentials, and no access to production dashboards or browser profiles. Shortest leash on the property.

Required output:

atlantis-lab sandbox result:
- Tool tested:
- Container isolation verified: yes/no
- Credentials in scope: yes/no (must be no)
- Production access in scope: yes/no (must be no)
- Findings documented: yes/no
- Ready for SOP-008 promotion review: yes/no

Use the highest applicable level.

Required output:

Final risk rating:
- Rating:
- Reason:
- Launch decision:
- Required fixes:
- Follow-up owner:

18. Sign-Off

No system covered by this SOP should move to production until required sign-offs are complete or explicitly waived by Shane with documented reason.

19. Production Readiness Checklist

- [ ] Intake completed
- [ ] NIST Govern reviewed
- [ ] NIST Map reviewed
- [ ] NIST Measure reviewed
- [ ] NIST Manage reviewed
- [ ] HITL requirements reviewed
- [ ] Memory / RAG hygiene reviewed
- [ ] OWASP AI/LLM review completed
- [ ] Automated secrets scan completed
- [ ] Atlantis-specific review completed
- [ ] Environment separation reviewed
- [ ] Docker hardening reviewed (if containerized)
- [ ] CIS Level 1 audit run on target infrastructure (score ≥ 80%)
- [ ] CIS audit log saved to ~/nova/logs/ and atlantis-docs/security/
- [ ] Rate limiting / abuse prevention reviewed
- [ ] Kill Switch documented and tested (< 5 min)
- [ ] Privacy / OPSEC review completed
- [ ] Risk rating assigned
- [ ] Required mitigations completed or assigned
- [ ] Monitoring path defined
- [ ] Rollback path defined
- [ ] Sign-off captured

20. Example: Nova Chat Intake Page

Use this example when reviewing the Home Help chat intake.

System

• Name: Nova Home Help Chat Intake
• Type: public chat intake / lead capture
• Domain: atlantisits.co/home-help
• Data: name, phone, email optional, service need, location, urgency, message
• Flow: public page → Nova Router → lead-engine → n8n → Sheets/CRM/alerts

NIST concerns

• Data collection must be disclosed clearly
• User must understand this is an intake, not a guaranteed contractor assignment
• Human follow-up path must be clear
• Lead routing and storage must be documented
• Human review should occur before contractor handoff
• Incident owner must be assigned before public release
• Kill Switch must disable the intake route or n8n workflow quickly

OWASP concerns

• Prompt injection attempting to override Nova behavior
• Malicious payloads in message fields
• Data leakage in Telegram/Discord alert templates
• Overreliance on automated classification
• Insecure output handling if user text is reflected into alerts or sheets
• Scout-layer or signal-ingestion abuse if lead source is later connected to social/search input
• Persistent memory poisoning if intake content is saved to long-term memory

Required mitigations

• Input validation
• Message length limit
• Field normalization
• Alert template fallback text
• No internal hostnames or debug output
• Human review before contractor handoff
• Logging without secrets
• Secrets scan before public GitHub deployment
• Kill Switch documented and tested
• Production workflow separated from dev/test keys and Sheets

21. Troubleshooting

22. Related Documents

• NIST AI RMF — AI risk management framework
• OWASP Top 10 for LLM Applications — LLM security risks and mitigations
• OWASP ML Security Top 10 — machine learning security risks
• CIS Ubuntu Linux 24.04 LTS Benchmark v1.0.0 — infrastructure hardening baseline
• ~/nova/cis_wsl2_audit.sh — Atlantis CIS Level 1 audit script (Trunks WSL2 + VPS)
• SOP-005 — Forgejo Deployment
• SOP-006 — Credential Rotation
• SOP-007 — Chronicle Build Procedure
• SOP-010 — Gemma 4 E4B Local Model Deployment
• atlantis-lab/SECURITY.md — sandbox rules for agent tech watch experiments
• core/PRIVACY-PROTOCOL.md — Atlantis privacy and OPSEC rules
• core/PRIORITY-RULES.md — Atlantis priority system
• atlantis-memory/updates.md — current tactical state
• ATLANTIS_AI_MEMORY.md — long-term memory

23. Revision History

24. Approval

End of SOP-008 — AI Production Risk Review (NIST AI RMF + OWASP AI/LLM + CIS Benchmark)