Guide/Ops Runbook

Ops Runbook

QA checklist, autonomy dial strategy, feature roadmap, and integration sync plan.

INTERNALv1.0 — Apr 1, 2026

1. V1 QA Checklist

Verify each item before declaring v1 functional. Open the factory at factory.asapai.net and walk through.

2. The Autonomy Dial

How to progressively automate the factory from expert #1 (verify everything) to expert #N (only touch gates).

Expert #1 (Samuel)CURRENT

Everything is Human Gate. Verify every output. Build trust in the system.

Deep Research

Expert Recon

MasteryBook Sync

Framework Creator

Demo Compiler

Rubric Builder

All Extractors

Gap Analyzer

Clone Compiler

Lead Magnet

Onboarding

Clone Tester

Orchestrator

Success criteria for dial-up: Samuel pipeline scores 85%+ on clone-tester. All gate approvals have positive feedback. No rework cycles on any block.

Expert #2NEXT

Research and recon run automatically. Extractors run automatically. Human gates on framework + audit + test only.

Deep Research

Expert Recon

MasteryBook Sync

Framework Creator

Demo Compiler

Rubric Builder

All Extractors

Gap Analyzer

Clone Compiler

Lead Magnet

Onboarding

Clone Tester

Orchestrator

Human touches: 3 (framework approval, gap review, test sign-off). Time: ~45 min vs ~2 hours.

Expert #5+TARGET

Nearly fully autonomous. Human only reviews framework design and final test. Everything else runs end-to-end.

Deep Research

Expert Recon

MasteryBook Sync

Framework Creator

Demo Compiler

Rubric Builder

All Extractors

Gap Analyzer

Clone Compiler

Lead Magnet

Onboarding

Clone Tester

Orchestrator

Human touches: 2 (framework guided review, test sign-off). Time: ~20 min. Framework Creator is "Guided" not "Gate" — it runs automatically but flags for optional review.

How to Change Autonomy Levels

Right now: autonomy levels are defined in lib/block-guides.ts (display) and the block template's execution.human_gate field (behavior).

V1: Change the template JSON and create a new run. V2 (build next): Per-run autonomy overrides in the workbench UI. Click the autonomy badge → toggle between levels → saves to run state.

3. Feature Roadmap

What to build next, in priority order. Core structure is down — everything below is additive.

P0TODAY

QA walkthrough on live Samuel run

Walk through the QA checklist above. Fix anything broken.

ANTHROPIC_API_KEY on Vercel

Add env var so chat proxy works in production.

Verify update-state persistence

Chat, file uploads, and form inputs must survive page refresh.

P1THIS WEEK

Per-run autonomy overrides

UI toggle on each block to change autonomy level per run. Saves to run state, doesn't change global template.

Block execution from workbench

"Run Block" button that calls the skill via Claude Code / API. Currently blocks are executed externally.

Gemini image generation in build phase

Add GEMINI_API_KEY. Build blocks can generate infographics, social cards, resource images from extraction data.

NotebookLM Reimagined podcast generation

After extraction, auto-generate podcast episodes from framework structures. Button in outputs tab.

MasteryOS sync endpoint

API route that pushes extraction outputs (KFs, resources, frameworks, experiences) to MasteryOS. Sumit integration point.

P2NEXT SPRINT

Create run from UI

Form on home page: enter expert name, upload initial materials, select template → creates run.

Real-time run updates

WebSocket or SSE for live block status changes. No more 30s polling.

Multi-model routing

OpenRouter integration. Different blocks use different models (Sonnet for chat, Opus for extraction, Gemini for images).

Artifact diff view

When a block re-runs, show diff between old and new output.

Team permissions

Role-based access: operator (run blocks), reviewer (approve gates), viewer (read-only).

P3V2 VISION

Agentic execution

Blocks trigger autonomously via FORGE VPS agent. Full Auto blocks execute without any human presence.

Template marketplace

Multiple factory templates: Expert Clone, Course Builder, Content Engine. Each gets the workbench pattern.

Live clone preview

Embedded Probiotic chat in the outputs tab. Test the clone without leaving the factory.

Feedback loop

Post-deployment: user conversations feed back into extraction → clone improves over time.

4. MasteryOS Sync Plan

How the standalone extraction factory connects to the platform where experts live.

The Problem

The factory produces structured extraction outputs (system prompt, knowledge files, frameworks, resources, offers, design system). MasteryOS is where the expert's clone lives and users interact with it. Currently these are disconnected — outputs are manually copied.

The Sync

system-prompt.md→MasteryOS Expert Profile → System Prompt field

knowledge-file-*.md→MasteryOS Knowledge Files (RAG documents)

frameworks.json→MasteryOS Prompts / Experiences (tool configs)

resources.json→MasteryOS Resources library

offers.json→MasteryOS CTA / Upgrade configuration

design-tokens.json→MasteryOS Expert Theme

onboarding-config.json→MasteryOS Onboarding Flow

Sumit Handoff Notes

What we need from Sumit: API endpoint(s) on MasteryOS that accept extraction outputs and populate an expert's profile. Ideally one endpoint per artifact type, or a batch endpoint that takes the full extraction bundle.

What we provide: Structured JSON from each extractor, plus the compiled system prompt and knowledge files. All outputs have consistent schemas defined in lib/types.ts.

Reference code: Probiotic repos (probiotic-Front--JDM-use,probiotic-back--JDM-use) show how the old system consumed these artifacts. MasteryOS should follow the same pattern but with its own data model.

Integration point: The factory's boarding-orchestrator block (Phase 8) will call the MasteryOS sync API as its final step. The workbench outputs tab will show a "Sync to MasteryOS" button.

Sequence

1Factory produces extraction outputs → stored in Supabase

2Boarding orchestrator calls MasteryOS sync API → creates/updates expert profile

3MasteryOS receives KFs, resources, frameworks, prompts, experiences, theme

4Expert's clone goes live on MasteryOS with all content pre-populated

5Users interact with clone on MasteryOS → feedback loop back to factory (v2)

5. Environment Variables

What needs to be set on Vercel for full functionality.

Variable	Status	Used By
NEXT_PUBLIC_SUPABASE_URL	✅ Set	All API routes
SUPABASE_SERVICE_ROLE_KEY	✅ Set	All API routes
ANTHROPIC_API_KEY	⚠️ Needed	Chat proxy (/api/chat)
OPENROUTER_API_KEY	🔜 P1	Multi-model routing
PPLX_API_KEY	🔜 P1	Deep research block
GEMINI_API_KEY	🔜 P1	Image/infographic generation