import from local vendored copy (2026-05-06)
Some checks failed
CI / validate (push) Failing after 0s
Some checks failed
CI / validate (push) Failing after 0s
This commit is contained in:
commit
32d8e7a15e
5
.github/workflows/ci.yml
vendored
Normal file
5
.github/workflows/ci.yml
vendored
Normal file
@ -0,0 +1,5 @@
|
||||
name: CI
|
||||
on: [push, pull_request]
|
||||
jobs:
|
||||
validate:
|
||||
uses: Molecule-AI/molecule-ci/.github/workflows/validate-plugin.yml@main
|
||||
31
.gitignore
vendored
Normal file
31
.gitignore
vendored
Normal file
@ -0,0 +1,31 @@
|
||||
# Credentials — never commit. Use .env.example as the template.
|
||||
.env
|
||||
.env.local
|
||||
.env.*.local
|
||||
.env.*
|
||||
!.env.example
|
||||
!.env.sample
|
||||
|
||||
# Private keys + certs
|
||||
*.pem
|
||||
*.key
|
||||
*.crt
|
||||
*.p12
|
||||
*.pfx
|
||||
|
||||
# Secret directories
|
||||
.secrets/
|
||||
|
||||
# Workspace auth tokens
|
||||
.auth-token
|
||||
.auth_token
|
||||
|
||||
# Python bytecode
|
||||
__pycache__/
|
||||
*.py[cod]
|
||||
*.pyc
|
||||
*$py.class
|
||||
.Python
|
||||
*.so
|
||||
*.egg-info/
|
||||
*.egg
|
||||
1
.molecule-ci/scripts/requirements.txt
Normal file
1
.molecule-ci/scripts/requirements.txt
Normal file
@ -0,0 +1 @@
|
||||
pyyaml>=6.0
|
||||
46
.molecule-ci/scripts/validate-plugin.py
Normal file
46
.molecule-ci/scripts/validate-plugin.py
Normal file
@ -0,0 +1,46 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Validate a Molecule AI plugin repo."""
|
||||
import os, sys, yaml
|
||||
|
||||
errors = []
|
||||
|
||||
if not os.path.isfile("plugin.yaml"):
|
||||
print("::error::plugin.yaml not found at repo root")
|
||||
sys.exit(1)
|
||||
|
||||
with open("plugin.yaml") as f:
|
||||
plugin = yaml.safe_load(f)
|
||||
|
||||
for field in ["name", "version", "description"]:
|
||||
if not plugin.get(field):
|
||||
errors.append(f"Missing required field: {field}")
|
||||
|
||||
v = str(plugin.get("version", ""))
|
||||
if v and not all(c in "0123456789." for c in v):
|
||||
errors.append(f"Invalid version format: {v}")
|
||||
|
||||
runtimes = plugin.get("runtimes")
|
||||
if runtimes is not None and not isinstance(runtimes, list):
|
||||
errors.append(f"runtimes must be a list, got {type(runtimes).__name__}")
|
||||
|
||||
content_paths = ["SKILL.md", "hooks", "skills", "rules"]
|
||||
found = [p for p in content_paths if os.path.exists(p)]
|
||||
if not found:
|
||||
errors.append("Plugin must contain at least one of: SKILL.md, hooks/, skills/, rules/")
|
||||
|
||||
if os.path.isfile("SKILL.md"):
|
||||
with open("SKILL.md") as f:
|
||||
first_line = f.readline().strip()
|
||||
if first_line and not first_line.startswith("#"):
|
||||
print("::warning::SKILL.md should start with a markdown heading (e.g., # Plugin Name)")
|
||||
|
||||
if errors:
|
||||
for e in errors:
|
||||
print(f"::error::{e}")
|
||||
sys.exit(1)
|
||||
|
||||
print(f"✓ plugin.yaml valid: {plugin['name']} v{plugin['version']}")
|
||||
if found:
|
||||
print(f" Content: {', '.join(found)}")
|
||||
if runtimes:
|
||||
print(f" Runtimes: {', '.join(runtimes)}")
|
||||
93
CLAUDE.md
Normal file
93
CLAUDE.md
Normal file
@ -0,0 +1,93 @@
|
||||
# superpowers — Agent Capability Extensions
|
||||
|
||||
`superpowers` is a **capability-extension plugin** that provides five
|
||||
high-level skills covering the full development lifecycle: systematic debugging,
|
||||
test-driven development, planning, plan execution, and pre-completion verification.
|
||||
|
||||
**Version:** 1.0.0
|
||||
**Runtime:** `claude_code`, `deepagents`, `hermes`
|
||||
|
||||
---
|
||||
|
||||
## Repository Layout
|
||||
|
||||
```
|
||||
superpowers/
|
||||
├── plugin.yaml — Plugin manifest
|
||||
├── skills/
|
||||
│ ├── executing-plans/
|
||||
│ ├── systematic-debugging/
|
||||
│ ├── test-driven-development/
|
||||
│ ├── verification-before-completion/
|
||||
│ └── writing-plans/
|
||||
└── adapters/ — Harness adaptors (thin wrappers)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Skills
|
||||
|
||||
| Skill | Purpose |
|
||||
|---|---|
|
||||
| `executing-plans` | Execute a pre-written plan step by step, adapting to obstacles |
|
||||
| `systematic-debugging` | Hypothesis → isolate → verify → fix, with session logging |
|
||||
| `test-driven-development` | Red → Green → Refactor cycle, coverage-gated |
|
||||
| `verification-before-completion` | Self-check before reporting done: tests, lint, build |
|
||||
| `writing-plans` | Break a complex task into ordered, testable steps |
|
||||
|
||||
---
|
||||
|
||||
## Development
|
||||
|
||||
### Prerequisites
|
||||
|
||||
- Node.js >= 18 (for markdownlint, if editing `.md` files)
|
||||
- Python 3.11+ (for YAML validation)
|
||||
- `gh` CLI authenticated
|
||||
- Write access to `Molecule-AI/molecule-ai-plugin-superpowers`
|
||||
|
||||
### Setup
|
||||
|
||||
```bash
|
||||
git clone https://github.com/Molecule-AI/molecule-ai-plugin-superpowers.git
|
||||
cd molecule-ai-plugin-superpowers
|
||||
|
||||
# Validate plugin.yaml
|
||||
python3 -c "import yaml; yaml.safe_load(open('plugin.yaml'))"
|
||||
echo "plugin.yaml OK"
|
||||
```
|
||||
|
||||
### Pre-Commit Checklist
|
||||
|
||||
```bash
|
||||
# YAML structure
|
||||
python3 -c "import yaml; yaml.safe_load(open('plugin.yaml'))"
|
||||
|
||||
# Credential scan
|
||||
python3 -c "
|
||||
import re, sys
|
||||
with open('plugin.yaml') as f:
|
||||
content = f.read()
|
||||
patterns = [r'sk.ant', r'ghp.', r'AKIA[A-Z0-9]']
|
||||
if any(re.search(p, content) for p in patterns):
|
||||
print('FAIL: possible credentials found')
|
||||
sys.exit(1)
|
||||
print('No credentials: OK')
|
||||
"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Release Process
|
||||
|
||||
1. Review changes: `git log origin/main..HEAD --oneline`
|
||||
2. Bump `version` in `plugin.yaml` (semver)
|
||||
3. Commit: `chore: bump version to X.Y.Z`
|
||||
4. Tag and push: `git tag vX.Y.Z && git push origin main --tags`
|
||||
5. Create GitHub Release with changelog
|
||||
|
||||
---
|
||||
|
||||
## Known Issues
|
||||
|
||||
See `known-issues.md` at the repo root.
|
||||
19
README.md
Normal file
19
README.md
Normal file
@ -0,0 +1,19 @@
|
||||
# superpowers
|
||||
|
||||
Molecule AI plugin. Install via the Molecule AI platform plugin system.
|
||||
|
||||
## Usage
|
||||
|
||||
### In org template (org.yaml)
|
||||
```yaml
|
||||
plugins:
|
||||
- superpowers
|
||||
```
|
||||
|
||||
### From URL (community install)
|
||||
```
|
||||
github://Molecule-AI/molecule-ai-plugin-superpowers
|
||||
```
|
||||
|
||||
## License
|
||||
Business Source License 1.1 — © Molecule AI.
|
||||
2
adapters/claude_code.py
Normal file
2
adapters/claude_code.py
Normal file
@ -0,0 +1,2 @@
|
||||
"""Claude Code adaptor — uses the generic rule+skill installer."""
|
||||
from plugins_registry.builtins import AgentskillsAdaptor as Adaptor # noqa: F401
|
||||
2
adapters/deepagents.py
Normal file
2
adapters/deepagents.py
Normal file
@ -0,0 +1,2 @@
|
||||
"""DeepAgents adaptor — uses the generic rule+skill installer."""
|
||||
from plugins_registry.builtins import AgentskillsAdaptor as Adaptor # noqa: F401
|
||||
54
known-issues.md
Normal file
54
known-issues.md
Normal file
@ -0,0 +1,54 @@
|
||||
# Known Issues — superpowers
|
||||
|
||||
---
|
||||
|
||||
## Active Issues
|
||||
|
||||
*(None currently open. This section is updated when issues are filed.)*
|
||||
|
||||
---
|
||||
|
||||
## Recently Resolved
|
||||
|
||||
*(No recently resolved issues.)*
|
||||
|
||||
---
|
||||
|
||||
## How to Update This File
|
||||
|
||||
When a new issue is identified:
|
||||
1. Add it under **Active Issues** using the template below
|
||||
2. Include: symptom, cause (if known), workaround
|
||||
3. When fixed, move to **Recently Resolved** and note the fix version
|
||||
|
||||
### Issue Template
|
||||
|
||||
```markdown
|
||||
## [TICKET-NUMBER] <Short Title>
|
||||
|
||||
**Severity:** P0 / P1 / P2 / P3
|
||||
**Status:** Workaround / Fix in progress / Fix available
|
||||
**Affected versions:** All / vX.Y.Z+
|
||||
|
||||
**Symptoms:**
|
||||
**Cause:**
|
||||
**Workaround:**
|
||||
**Fix (if available):**
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Severity Definitions
|
||||
|
||||
| Level | Description |
|
||||
|---|---|
|
||||
| P0 | Skill fails to load; no workaround |
|
||||
| P1 | Core skill broken; no output or wrong output |
|
||||
| P2 | Non-core issue; workaround available |
|
||||
| P3 | Cosmetic or documentation issue |
|
||||
|
||||
---
|
||||
|
||||
## Reporting
|
||||
|
||||
Use the Molecule-AI/internal issue tracker. Tag with `plugin-superpowers`.
|
||||
474
plans/2026-04-08-hermes-borrowing-roadmap.md
Normal file
474
plans/2026-04-08-hermes-borrowing-roadmap.md
Normal file
@ -0,0 +1,474 @@
|
||||
# Hermes Borrowing Roadmap Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED: Use superpowers:subagent-driven-development (if subagents available) or superpowers:executing-plans to implement this plan. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Bring the highest-leverage Hermes-style improvements into Molecule AI: a clearer local startup path, better capability/onboarding discovery in Canvas, and a minimal external event ingress story.
|
||||
|
||||
**Architecture:** Keep changes staged and independently shippable. First strengthen the CLI/docs path, then expose capability and onboarding affordances in Canvas, then add runtime/platform primitives for webhook ingress and richer capability visibility. Avoid broad refactors; build on existing handlers, stores, and agent card publication.
|
||||
|
||||
**Tech Stack:** Go + Cobra CLI, Go/Gin platform handlers, Next.js 15 + Zustand canvas, Python workspace runtime, Docker-based verification.
|
||||
|
||||
---
|
||||
|
||||
## File Map
|
||||
|
||||
### CLI / Docs
|
||||
- Modify: `README.md`
|
||||
- Modify: `README.zh-CN.md`
|
||||
- Modify: `platform/cmd/cli/commands.go`
|
||||
- Modify: `platform/cmd/cli/cmd_doctor.go`
|
||||
- Modify: `platform/cmd/cli/doctor.go`
|
||||
- Modify: `platform/cmd/cli/doctor_test.go`
|
||||
|
||||
### Canvas UX
|
||||
- Modify: `canvas/src/components/EmptyState.tsx`
|
||||
- Modify: `canvas/src/components/Toolbar.tsx`
|
||||
- Modify: `canvas/src/components/tabs/ChatTab.tsx`
|
||||
- Modify: `canvas/src/components/SidePanel.tsx`
|
||||
- Modify: `canvas/src/store/canvas.ts`
|
||||
- Modify: `canvas/src/types/activity.ts` if capability summary types need extraction
|
||||
- Test: `canvas/src/store/__tests__/canvas.test.ts`
|
||||
- Modify: `docs/frontend/canvas.md`
|
||||
|
||||
### Capability Visibility / Platform
|
||||
- Modify: `workspace-template/main.py`
|
||||
- Modify: `workspace-template/config.py`
|
||||
- Modify: `workspace-template/agent.py`
|
||||
- Add: `workspace-template/preflight.py`
|
||||
- Add: `workspace-template/tests/test_preflight.py`
|
||||
- Modify: `platform/internal/models/workspace.go`
|
||||
- Modify: `platform/internal/handlers/templates.go`
|
||||
- Modify: `platform/internal/router/router.go`
|
||||
- Add: `platform/internal/handlers/webhooks.go`
|
||||
- Add: `platform/internal/handlers/webhooks_test.go`
|
||||
- Modify: `docs/agent-runtime/cli-runtime.md`
|
||||
- Modify: `docs/agent-runtime/config-format.md`
|
||||
- Modify: `docs/api-protocol/platform-api.md`
|
||||
|
||||
---
|
||||
|
||||
## Chunk 1: Tighten the Local Startup Path
|
||||
|
||||
### Task 1: Expand `doctor` to cover the real local path
|
||||
|
||||
**Files:**
|
||||
- Modify: `platform/cmd/cli/doctor.go`
|
||||
- Modify: `platform/cmd/cli/cmd_doctor.go`
|
||||
- Test: `platform/cmd/cli/doctor_test.go`
|
||||
|
||||
- [ ] **Step 1: Write the failing tests for the next doctor checks**
|
||||
|
||||
Add tests for:
|
||||
- `migrations` directory discovery
|
||||
- `workspace-configs-templates` warning vs fail behavior
|
||||
- JSON output shape for `--json`
|
||||
|
||||
- [ ] **Step 2: Run the CLI tests to verify they fail**
|
||||
|
||||
Run:
|
||||
```bash
|
||||
docker run --rm -v /Users/aricredemption/Projects/molecule-monorepo:/workspace -w /workspace/platform golang:1.25.0 go test ./cmd/cli
|
||||
```
|
||||
|
||||
Expected: FAIL in the new doctor test cases.
|
||||
|
||||
- [ ] **Step 3: Implement the smallest useful doctor additions**
|
||||
|
||||
Add:
|
||||
- migrations directory check
|
||||
- optional `--json` coverage verification if output formatting needs adjustment
|
||||
- keep checks flat and synchronous; do not introduce a plugin framework
|
||||
|
||||
- [ ] **Step 4: Run the CLI tests to verify they pass**
|
||||
|
||||
Run:
|
||||
```bash
|
||||
docker run --rm -v /Users/aricredemption/Projects/molecule-monorepo:/workspace -w /workspace/platform golang:1.25.0 go test ./cmd/cli
|
||||
```
|
||||
|
||||
Expected: PASS
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add platform/cmd/cli/cmd_doctor.go platform/cmd/cli/doctor.go platform/cmd/cli/doctor_test.go
|
||||
git commit -m "feat(cli): expand doctor startup checks"
|
||||
```
|
||||
|
||||
### Task 2: Make the quickstart path explicit in docs
|
||||
|
||||
**Files:**
|
||||
- Modify: `README.md`
|
||||
- Modify: `README.zh-CN.md`
|
||||
|
||||
- [ ] **Step 1: Write the docs-first delta**
|
||||
|
||||
Add a short "recommended path" section:
|
||||
- `./infra/scripts/setup.sh`
|
||||
- `molecli doctor`
|
||||
- `go run ./cmd/server`
|
||||
- `npm run dev`
|
||||
- deploy a template from Canvas
|
||||
|
||||
- [ ] **Step 2: Verify the docs are accurate against existing commands**
|
||||
|
||||
Run:
|
||||
```bash
|
||||
rg -n "setup.sh|molecli doctor|go run ./cmd/server|npm run dev" README.md README.zh-CN.md
|
||||
```
|
||||
|
||||
Expected: the new quickstart path appears in both READMEs.
|
||||
|
||||
- [ ] **Step 3: Commit**
|
||||
|
||||
```bash
|
||||
git add README.md README.zh-CN.md
|
||||
git commit -m "docs: add explicit local quickstart path"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Chunk 2: Surface Onboarding and Capability Discovery in Canvas
|
||||
|
||||
### Task 3: Upgrade the empty state from hint list to startup flow
|
||||
|
||||
**Files:**
|
||||
- Modify: `canvas/src/components/EmptyState.tsx`
|
||||
- Modify: `docs/frontend/canvas.md`
|
||||
|
||||
- [ ] **Step 1: Write the expected content and behavior**
|
||||
|
||||
Target:
|
||||
- a short 3-step startup flow
|
||||
- one clear primary action
|
||||
- references to template palette, search, and drag-to-nest
|
||||
|
||||
- [ ] **Step 2: Implement the empty-state refresh**
|
||||
|
||||
Keep it static first. No new API calls.
|
||||
|
||||
- [ ] **Step 3: Verify the app still builds**
|
||||
|
||||
Run:
|
||||
```bash
|
||||
cd canvas && npm run build
|
||||
```
|
||||
|
||||
Expected: PASS
|
||||
|
||||
- [ ] **Step 4: Commit**
|
||||
|
||||
```bash
|
||||
git add canvas/src/components/EmptyState.tsx docs/frontend/canvas.md
|
||||
git commit -m "feat(canvas): turn empty state into onboarding flow"
|
||||
```
|
||||
|
||||
### Task 4: Add a toolbar quick-actions / cheatsheet surface
|
||||
|
||||
**Files:**
|
||||
- Modify: `canvas/src/components/Toolbar.tsx`
|
||||
- Optionally modify: `canvas/src/components/Tooltip.tsx`
|
||||
|
||||
- [ ] **Step 1: Write the interaction expectations**
|
||||
|
||||
Support:
|
||||
- visible help affordance in toolbar
|
||||
- quick reminders for `⌘K`, template palette, right-click, resume chat, config location
|
||||
|
||||
- [ ] **Step 2: Implement the smallest UI surface**
|
||||
|
||||
Prefer a compact popover/panel over a full modal. Do not add routing.
|
||||
|
||||
- [ ] **Step 3: Verify the app still builds**
|
||||
|
||||
Run:
|
||||
```bash
|
||||
cd canvas && npm run build
|
||||
```
|
||||
|
||||
Expected: PASS
|
||||
|
||||
- [ ] **Step 4: Commit**
|
||||
|
||||
```bash
|
||||
git add canvas/src/components/Toolbar.tsx
|
||||
git commit -m "feat(canvas): add quick actions help surface"
|
||||
```
|
||||
|
||||
### Task 5: Make chat resume and capability visibility discoverable
|
||||
|
||||
**Files:**
|
||||
- Modify: `canvas/src/components/tabs/ChatTab.tsx`
|
||||
- Modify: `canvas/src/components/SidePanel.tsx`
|
||||
- Modify: `canvas/src/store/canvas.ts`
|
||||
- Test: `canvas/src/store/__tests__/canvas.test.ts`
|
||||
|
||||
- [ ] **Step 1: Write failing store or rendering expectations**
|
||||
|
||||
Cover:
|
||||
- resumed task state is visible as such
|
||||
- capability summary can be derived from workspace data / agent card without extra fetches
|
||||
|
||||
- [ ] **Step 2: Run the canvas tests to verify they fail**
|
||||
|
||||
Run:
|
||||
```bash
|
||||
cd canvas && npm test -- --runInBand
|
||||
```
|
||||
|
||||
Expected: FAIL in new capability/resume expectations.
|
||||
|
||||
- [ ] **Step 3: Implement minimal resume and capability summary UI**
|
||||
|
||||
Target:
|
||||
- show "resume current run" or equivalent when `currentTask` exists
|
||||
- expose a concise capability summary near the panel header or chat tab
|
||||
- use existing `agentCard`, `tier`, `status`, `currentTask`; do not add a new API yet
|
||||
|
||||
- [ ] **Step 4: Re-run tests and build**
|
||||
|
||||
Run:
|
||||
```bash
|
||||
cd canvas && npm test -- --runInBand
|
||||
cd canvas && npm run build
|
||||
```
|
||||
|
||||
Expected: PASS
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add canvas/src/components/tabs/ChatTab.tsx canvas/src/components/SidePanel.tsx canvas/src/store/canvas.ts canvas/src/store/__tests__/canvas.test.ts
|
||||
git commit -m "feat(canvas): surface resume state and capability summary"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Chunk 3: Strengthen Runtime Capability Metadata and Preflight
|
||||
|
||||
### Task 6: Add runtime preflight as a reusable Python primitive
|
||||
|
||||
**Files:**
|
||||
- Add: `workspace-template/preflight.py`
|
||||
- Modify: `workspace-template/main.py`
|
||||
- Modify: `workspace-template/config.py`
|
||||
- Add: `workspace-template/tests/test_preflight.py`
|
||||
|
||||
- [ ] **Step 1: Write the failing Python tests**
|
||||
|
||||
Cover:
|
||||
- config-level preflight for required env / runtime prerequisites
|
||||
- minimal capability snapshot generation from runtime config
|
||||
|
||||
- [ ] **Step 2: Run the workspace tests to verify they fail**
|
||||
|
||||
Run:
|
||||
```bash
|
||||
cd workspace-template && pytest tests/test_preflight.py -q
|
||||
```
|
||||
|
||||
Expected: FAIL
|
||||
|
||||
- [ ] **Step 3: Implement the smallest preflight layer**
|
||||
|
||||
Scope:
|
||||
- no new CLI yet
|
||||
- reusable function that validates config/runtime assumptions before startup
|
||||
- emits a compact capability/preflight summary for later publication
|
||||
|
||||
- [ ] **Step 4: Run the focused tests**
|
||||
|
||||
Run:
|
||||
```bash
|
||||
cd workspace-template && pytest tests/test_preflight.py -q
|
||||
```
|
||||
|
||||
Expected: PASS
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add workspace-template/preflight.py workspace-template/main.py workspace-template/config.py workspace-template/tests/test_preflight.py
|
||||
git commit -m "feat(runtime): add workspace preflight primitives"
|
||||
```
|
||||
|
||||
### Task 7: Publish richer capability metadata from runtime to platform
|
||||
|
||||
**Files:**
|
||||
- Modify: `workspace-template/main.py`
|
||||
- Modify: `workspace-template/agent.py`
|
||||
- Modify: `platform/internal/models/workspace.go`
|
||||
- Possibly modify: `platform/internal/handlers/registry.go`
|
||||
- Possibly modify: `canvas/src/store/canvas.ts`
|
||||
|
||||
- [ ] **Step 1: Write failing tests where coverage exists**
|
||||
|
||||
Cover:
|
||||
- agent card / capability metadata shape
|
||||
- store handling if any new fields are added to workspace payloads
|
||||
|
||||
- [ ] **Step 2: Implement metadata expansion**
|
||||
|
||||
Add only compact, durable fields such as:
|
||||
- runtime kind
|
||||
- tool modes
|
||||
- session continuity support
|
||||
- sandbox/backend hints
|
||||
- preflight warnings count if appropriate
|
||||
|
||||
Do not publish deep provider internals or secrets.
|
||||
|
||||
- [ ] **Step 3: Verify focused tests**
|
||||
|
||||
Run:
|
||||
```bash
|
||||
docker run --rm -v /Users/aricredemption/Projects/molecule-monorepo:/workspace -w /workspace/platform golang:1.25.0 go test ./internal/handlers ./internal/router
|
||||
cd workspace-template && pytest tests/test_preflight.py tests/test_prompt.py -q
|
||||
```
|
||||
|
||||
Expected: PASS
|
||||
|
||||
- [ ] **Step 4: Commit**
|
||||
|
||||
```bash
|
||||
git add workspace-template/main.py workspace-template/agent.py platform/internal/models/workspace.go platform/internal/handlers/registry.go canvas/src/store/canvas.ts
|
||||
git commit -m "feat(runtime): publish richer workspace capability metadata"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Chunk 4: Add a Minimal External Event Ingress
|
||||
|
||||
### Task 8: Introduce webhook endpoint scaffolding in platform
|
||||
|
||||
**Files:**
|
||||
- Add: `platform/internal/handlers/webhooks.go`
|
||||
- Add: `platform/internal/handlers/webhooks_test.go`
|
||||
- Modify: `platform/internal/router/router.go`
|
||||
- Modify: `docs/api-protocol/platform-api.md`
|
||||
|
||||
- [ ] **Step 1: Write failing handler tests**
|
||||
|
||||
Cover:
|
||||
- accepts a basic signed or token-protected webhook request
|
||||
- validates target workspace
|
||||
- stores or forwards a normalized event payload
|
||||
- rejects malformed or unauthorized requests
|
||||
|
||||
- [ ] **Step 2: Run the focused Go tests to verify they fail**
|
||||
|
||||
Run:
|
||||
```bash
|
||||
docker run --rm -v /Users/aricredemption/Projects/molecule-monorepo:/workspace -w /workspace/platform golang:1.25.0 go test ./internal/handlers -run Webhook -v
|
||||
```
|
||||
|
||||
Expected: FAIL
|
||||
|
||||
- [ ] **Step 3: Implement minimal ingress**
|
||||
|
||||
Scope:
|
||||
- one generic endpoint such as `POST /workspaces/:id/webhooks/events`
|
||||
- one normalization path
|
||||
- simple authentication guard
|
||||
- no provider-specific adapters yet
|
||||
|
||||
- [ ] **Step 4: Re-run focused tests**
|
||||
|
||||
Run:
|
||||
```bash
|
||||
docker run --rm -v /Users/aricredemption/Projects/molecule-monorepo:/workspace -w /workspace/platform golang:1.25.0 go test ./internal/handlers -run Webhook -v
|
||||
```
|
||||
|
||||
Expected: PASS
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add platform/internal/handlers/webhooks.go platform/internal/handlers/webhooks_test.go platform/internal/router/router.go docs/api-protocol/platform-api.md
|
||||
git commit -m "feat(platform): add generic webhook ingress endpoint"
|
||||
```
|
||||
|
||||
### Task 9: Connect webhook ingress to runtime-facing task handling
|
||||
|
||||
**Files:**
|
||||
- Modify: `workspace-template/main.py`
|
||||
- Modify: `workspace-template/config.py`
|
||||
- Modify: `docs/agent-runtime/cli-runtime.md`
|
||||
- Modify: `docs/agent-runtime/config-format.md`
|
||||
|
||||
- [ ] **Step 1: Define the smallest runtime contract**
|
||||
|
||||
Support:
|
||||
- webhook event arrives at platform
|
||||
- platform forwards a normalized task payload to workspace A2A or activity/task path
|
||||
- runtime can distinguish webhook-originated work from chat-originated work if needed
|
||||
|
||||
- [ ] **Step 2: Implement only the minimum required runtime/config hooks**
|
||||
|
||||
Do not add provider-specific webhook logic. Keep the runtime generic.
|
||||
|
||||
- [ ] **Step 3: Verify focused tests and docs**
|
||||
|
||||
Run:
|
||||
```bash
|
||||
cd workspace-template && pytest tests/test_config.py tests/test_a2a_executor.py -q
|
||||
docker run --rm -v /Users/aricredemption/Projects/molecule-monorepo:/workspace -w /workspace/platform golang:1.25.0 go test ./internal/handlers -run Webhook -v
|
||||
```
|
||||
|
||||
Expected: PASS
|
||||
|
||||
- [ ] **Step 4: Commit**
|
||||
|
||||
```bash
|
||||
git add workspace-template/main.py workspace-template/config.py docs/agent-runtime/cli-runtime.md docs/agent-runtime/config-format.md
|
||||
git commit -m "feat(runtime): wire webhook ingress into workspace tasks"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Dependencies and Order
|
||||
|
||||
1. Expand `doctor`
|
||||
2. Update quickstart docs
|
||||
3. Refresh Canvas empty state
|
||||
4. Add toolbar help
|
||||
5. Add chat resume + capability summary
|
||||
6. Add runtime preflight primitives
|
||||
7. Publish richer capability metadata
|
||||
8. Add generic webhook ingress
|
||||
9. Wire webhook ingress into runtime task handling
|
||||
|
||||
Reasoning:
|
||||
- Steps 1-5 improve discoverability without increasing backend coupling.
|
||||
- Steps 6-7 give us a stable capability/preflight contract before Canvas or integrations rely on richer metadata.
|
||||
- Steps 8-9 add the external ingress path only after visibility and runtime metadata are in place.
|
||||
|
||||
## Atomic Commit Rules
|
||||
|
||||
- Each commit must change one user-visible concern only.
|
||||
- No mixed docs + platform + canvas + runtime commit unless the docs only describe the code introduced in the same commit.
|
||||
- Every commit must have at least one focused verification command run before moving on.
|
||||
- If a task unexpectedly spans two subsystems, split by boundary and commit the provider-side primitive before the consumer-side UI.
|
||||
|
||||
## Verification Matrix
|
||||
|
||||
- CLI:
|
||||
```bash
|
||||
docker run --rm -v /Users/aricredemption/Projects/molecule-monorepo:/workspace -w /workspace/platform golang:1.25.0 go test ./cmd/cli
|
||||
```
|
||||
|
||||
- Platform handlers/router:
|
||||
```bash
|
||||
docker run --rm -v /Users/aricredemption/Projects/molecule-monorepo:/workspace -w /workspace/platform golang:1.25.0 go test ./internal/handlers ./internal/router
|
||||
```
|
||||
|
||||
- Canvas:
|
||||
```bash
|
||||
cd canvas && npm test -- --runInBand
|
||||
cd canvas && npm run build
|
||||
```
|
||||
|
||||
- Runtime:
|
||||
```bash
|
||||
cd workspace-template && pytest -q
|
||||
```
|
||||
477
plans/2026-04-08-hermes-inspired-dx-rollout.md
Normal file
477
plans/2026-04-08-hermes-inspired-dx-rollout.md
Normal file
@ -0,0 +1,477 @@
|
||||
# Hermes-Inspired DX Rollout Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED: Use superpowers:subagent-driven-development (if subagents available) or superpowers:executing-plans to implement this plan. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Add the highest-value Hermes-inspired developer experience improvements across CLI, Canvas, and runtime/platform without turning the codebase into a broad refactor.
|
||||
|
||||
**Architecture:** Keep the rollout in narrow vertical slices. Start with CLI and onboarding paths that improve default usage immediately, then expose workspace capabilities more clearly, then add a minimal webhook ingress path as a separate backend feature. Each slice should ship independently and leave the repo in a usable state.
|
||||
|
||||
**Tech Stack:** Go + Cobra CLI, Go + Gin platform, Next.js 15 + Zustand canvas, Python workspace runtime, Docker-based verification, existing platform HTTP APIs.
|
||||
|
||||
---
|
||||
|
||||
## File Map
|
||||
|
||||
### CLI / Platform
|
||||
|
||||
- Modify: `platform/cmd/cli/commands.go`
|
||||
- Modify: `platform/cmd/cli/cmd_agent.go`
|
||||
- Modify: `platform/cmd/cli/cmd_chat.go`
|
||||
- Modify: `platform/cmd/cli/view.go`
|
||||
- Modify: `platform/cmd/cli/client.go`
|
||||
- Modify: `platform/cmd/cli/cli_test.go`
|
||||
- Create or modify: `platform/cmd/cli/cmd_doctor.go`
|
||||
- Create or modify: `platform/cmd/cli/doctor.go`
|
||||
- Create or modify: `platform/cmd/cli/doctor_test.go`
|
||||
- Modify later only if needed: `platform/internal/router/router.go`
|
||||
- Modify later only if needed: `platform/internal/handlers/*.go`
|
||||
|
||||
### Canvas
|
||||
|
||||
- Modify: `canvas/src/components/EmptyState.tsx`
|
||||
- Modify: `canvas/src/components/Toolbar.tsx`
|
||||
- Modify: `canvas/src/components/SidePanel.tsx`
|
||||
- Modify: `canvas/src/components/tabs/ChatTab.tsx`
|
||||
- Modify: `canvas/src/components/tabs/DetailsTab.tsx`
|
||||
- Modify: `canvas/src/store/canvas.ts`
|
||||
- Modify if required: `canvas/src/types/activity.ts`
|
||||
- Add if needed: `canvas/src/components/QuickHelpPopover.tsx`
|
||||
- Add if needed: `canvas/src/components/CapabilitySummary.tsx`
|
||||
- Modify tests if present or add: `canvas/src/store/__tests__/canvas.test.ts`
|
||||
|
||||
### Runtime / Platform Integration
|
||||
|
||||
- Modify: `workspace-template/main.py`
|
||||
- Modify: `workspace-template/agent.py`
|
||||
- Modify: `workspace-template/config.py`
|
||||
- Modify if needed: `workspace-template/tests/test_config.py`
|
||||
- Modify if needed: `workspace-template/tests/test_prompt.py`
|
||||
- Modify if needed: `workspace-template/tests/test_a2a_executor.py`
|
||||
- Modify: `platform/internal/router/router.go`
|
||||
- Add: `platform/internal/handlers/webhooks.go`
|
||||
- Add tests: `platform/internal/handlers/webhooks_test.go`
|
||||
- Modify if needed: `platform/internal/models/workspace.go`
|
||||
|
||||
### Docs
|
||||
|
||||
- Modify: `README.md`
|
||||
- Modify: `README.zh-CN.md`
|
||||
- Modify: `docs/agent-runtime/cli-runtime.md`
|
||||
- Modify: `docs/frontend/canvas.md`
|
||||
- Modify: `docs/api-protocol/platform-api.md`
|
||||
- Modify: `docs/edit-history/2026-04-08.md`
|
||||
|
||||
---
|
||||
|
||||
## Multi-Agent Execution Strategy
|
||||
|
||||
### Parallel lanes
|
||||
|
||||
- Lane A: CLI/default-path improvements
|
||||
- Lane B: Canvas onboarding/help/resume UX
|
||||
- Lane C: Capability summary plumbing
|
||||
- Lane D: Webhook ingress backend
|
||||
- Lane E: Docs pass after each shipped lane
|
||||
|
||||
### Shared-state rule
|
||||
|
||||
- Lanes A and B can run in parallel after agreeing on copy and naming.
|
||||
- Lane C depends on whatever backend/runtime fields are already available; start after confirming whether current agent card payload is sufficient.
|
||||
- Lane D must stay isolated from Lanes A and B. It touches backend API surface and should be implemented and reviewed separately.
|
||||
- Docs commits should be separate and follow the feature commits they describe.
|
||||
|
||||
---
|
||||
|
||||
## Chunk 1: CLI Default Path
|
||||
|
||||
### Task 1: Finish the doctor command as the stable entry point
|
||||
|
||||
**Files:**
|
||||
- Modify: `platform/cmd/cli/cmd_doctor.go`
|
||||
- Modify: `platform/cmd/cli/doctor.go`
|
||||
- Modify: `platform/cmd/cli/doctor_test.go`
|
||||
|
||||
- [ ] **Step 1: Write one more failing test for expected doctor output/JSON shape if a gap remains**
|
||||
|
||||
Run: `docker run --rm -v /Users/aricredemption/Projects/molecule-monorepo:/workspace -w /workspace/platform golang:1.25.0 go test ./cmd/cli`
|
||||
|
||||
Expected: failing test only if behavior is not yet locked.
|
||||
|
||||
- [ ] **Step 2: Implement only the missing doctor behavior**
|
||||
|
||||
Keep checks limited to the intended scope: health, Postgres, Redis, templates, Docker.
|
||||
|
||||
- [ ] **Step 3: Run CLI tests**
|
||||
|
||||
Run: `docker run --rm -v /Users/aricredemption/Projects/molecule-monorepo:/workspace -w /workspace/platform golang:1.25.0 go test ./cmd/cli`
|
||||
|
||||
Expected: PASS
|
||||
|
||||
- [ ] **Step 4: Commit**
|
||||
|
||||
```bash
|
||||
git add platform/cmd/cli/cmd_doctor.go platform/cmd/cli/doctor.go platform/cmd/cli/doctor_test.go platform/cmd/cli/commands.go platform/cmd/cli/main.go
|
||||
git commit -m "feat(cli): add doctor preflight checks"
|
||||
```
|
||||
|
||||
### Task 2: Add the guided CLI quickstart path
|
||||
|
||||
**Files:**
|
||||
- Modify: `platform/cmd/cli/commands.go`
|
||||
- Modify: `platform/cmd/cli/cmd_agent.go`
|
||||
- Modify: `platform/cmd/cli/cmd_chat.go`
|
||||
- Modify: `platform/cmd/cli/view.go`
|
||||
- Modify: `platform/cmd/cli/cli_test.go`
|
||||
|
||||
- [ ] **Step 1: Write a failing test for the new command/help path**
|
||||
|
||||
Examples:
|
||||
- root help should mention `doctor`
|
||||
- agent help should expose the recommended `spawn -> chat` flow
|
||||
- optional `molecli quickstart` should render deterministic guidance
|
||||
|
||||
- [ ] **Step 2: Run the targeted test**
|
||||
|
||||
Run: `docker run --rm -v /Users/aricredemption/Projects/molecule-monorepo:/workspace -w /workspace/platform golang:1.25.0 go test ./cmd/cli -run 'Test.*Quickstart|Test.*Doctor'`
|
||||
|
||||
Expected: FAIL
|
||||
|
||||
- [ ] **Step 3: Implement the minimum path**
|
||||
|
||||
Preferred shape:
|
||||
- either a dedicated `molecli quickstart` command
|
||||
- or a stronger root help and `agent` subcommand examples
|
||||
|
||||
Do not add a wizard or interactive setup flow.
|
||||
|
||||
- [ ] **Step 4: Run CLI tests**
|
||||
|
||||
Run: `docker run --rm -v /Users/aricredemption/Projects/molecule-monorepo:/workspace -w /workspace/platform golang:1.25.0 go test ./cmd/cli`
|
||||
|
||||
Expected: PASS
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add platform/cmd/cli/commands.go platform/cmd/cli/cmd_agent.go platform/cmd/cli/cmd_chat.go platform/cmd/cli/view.go platform/cmd/cli/cli_test.go
|
||||
git commit -m "feat(cli): add guided quickstart path"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Chunk 2: Canvas Onboarding and Help
|
||||
|
||||
### Task 3: Turn the empty state into a real onboarding panel
|
||||
|
||||
**Files:**
|
||||
- Modify: `canvas/src/components/EmptyState.tsx`
|
||||
|
||||
- [ ] **Step 1: Add a failing UI test if the repo already has a clear pattern for component testing**
|
||||
|
||||
If there is no stable component-test pattern, skip new component tests and rely on build verification for this task.
|
||||
|
||||
- [ ] **Step 2: Replace the current generic empty copy with a Hermes-style start path**
|
||||
|
||||
Required content:
|
||||
- start with template palette
|
||||
- run `molecli doctor`
|
||||
- create first workspace
|
||||
- open chat/config after deploy
|
||||
|
||||
Do not add new backend dependencies.
|
||||
|
||||
- [ ] **Step 3: Run frontend verification**
|
||||
|
||||
Run: `npm test -- --runInBand` from `canvas/` only if that command is already healthy, otherwise use `npm run build`.
|
||||
|
||||
Expected: PASS
|
||||
|
||||
- [ ] **Step 4: Commit**
|
||||
|
||||
```bash
|
||||
git add canvas/src/components/EmptyState.tsx
|
||||
git commit -m "feat(canvas): add guided empty-state onboarding"
|
||||
```
|
||||
|
||||
### Task 4: Add toolbar help and cheatsheet surfacing
|
||||
|
||||
**Files:**
|
||||
- Modify: `canvas/src/components/Toolbar.tsx`
|
||||
- Add if needed: `canvas/src/components/QuickHelpPopover.tsx`
|
||||
- Modify if needed: `canvas/src/store/canvas.ts`
|
||||
|
||||
- [ ] **Step 1: Define the minimal help surface**
|
||||
|
||||
Include only:
|
||||
- `⌘K`
|
||||
- template palette
|
||||
- right-click actions
|
||||
- chat sessions/resume
|
||||
- config/secrets location
|
||||
|
||||
- [ ] **Step 2: Implement the popover or inline panel**
|
||||
|
||||
Keep state local unless a shared store is clearly necessary.
|
||||
|
||||
- [ ] **Step 3: Run frontend verification**
|
||||
|
||||
Run: `npm run build` from `canvas/`
|
||||
|
||||
Expected: PASS
|
||||
|
||||
- [ ] **Step 4: Commit**
|
||||
|
||||
```bash
|
||||
git add canvas/src/components/Toolbar.tsx canvas/src/components/QuickHelpPopover.tsx canvas/src/store/canvas.ts
|
||||
git commit -m "feat(canvas): add toolbar quick help"
|
||||
```
|
||||
|
||||
### Task 5: Make chat resume discoverable
|
||||
|
||||
**Files:**
|
||||
- Modify: `canvas/src/components/tabs/ChatTab.tsx`
|
||||
|
||||
- [ ] **Step 1: Write a failing test only if session behavior can be covered cheaply**
|
||||
|
||||
Otherwise skip to implementation and rely on build verification.
|
||||
|
||||
- [ ] **Step 2: Surface resume state explicitly**
|
||||
|
||||
Examples:
|
||||
- banner when `currentTask` exists
|
||||
- label for the active session being resumed
|
||||
- clearer wording around session list and continued polling
|
||||
|
||||
Do not re-architect chat transport in this task.
|
||||
|
||||
- [ ] **Step 3: Run frontend verification**
|
||||
|
||||
Run: `npm run build` from `canvas/`
|
||||
|
||||
Expected: PASS
|
||||
|
||||
- [ ] **Step 4: Commit**
|
||||
|
||||
```bash
|
||||
git add canvas/src/components/tabs/ChatTab.tsx
|
||||
git commit -m "feat(canvas): surface chat resume state"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Chunk 3: Capability Summary Surfacing
|
||||
|
||||
### Task 6: Expose a compact workspace capability summary in the side panel
|
||||
|
||||
**Files:**
|
||||
- Modify: `canvas/src/components/SidePanel.tsx`
|
||||
- Modify: `canvas/src/components/tabs/DetailsTab.tsx`
|
||||
- Add if needed: `canvas/src/components/CapabilitySummary.tsx`
|
||||
|
||||
- [ ] **Step 1: Confirm existing fields are enough**
|
||||
|
||||
Prefer using:
|
||||
- agent card skills
|
||||
- tier
|
||||
- status
|
||||
- active task
|
||||
- URL/runtime hints already present in config/details
|
||||
|
||||
Do not add backend fields if current data is sufficient.
|
||||
|
||||
- [ ] **Step 2: Implement the summary UI**
|
||||
|
||||
Target output:
|
||||
- what the workspace is
|
||||
- what it can do now
|
||||
- where to configure more
|
||||
|
||||
Avoid long cards or dense metadata dumps.
|
||||
|
||||
- [ ] **Step 3: Run frontend verification**
|
||||
|
||||
Run: `npm run build` from `canvas/`
|
||||
|
||||
Expected: PASS
|
||||
|
||||
- [ ] **Step 4: Commit**
|
||||
|
||||
```bash
|
||||
git add canvas/src/components/SidePanel.tsx canvas/src/components/tabs/DetailsTab.tsx canvas/src/components/CapabilitySummary.tsx
|
||||
git commit -m "feat(canvas): add workspace capability summary"
|
||||
```
|
||||
|
||||
### Task 7: Add backend/runtime capability fields only if the UI needs more than the current agent card
|
||||
|
||||
**Files:**
|
||||
- Modify: `workspace-template/main.py`
|
||||
- Modify if needed: `platform/internal/handlers/registry.go`
|
||||
- Modify if needed: `platform/internal/models/workspace.go`
|
||||
- Modify if needed: `canvas/src/store/canvas.ts`
|
||||
- Modify tests as needed
|
||||
|
||||
- [ ] **Step 1: Write a failing backend/runtime test for the new capability field**
|
||||
|
||||
Only do this if UI work proved the current payload is insufficient.
|
||||
|
||||
- [ ] **Step 2: Add the minimum new agent-card or workspace field**
|
||||
|
||||
Candidate fields:
|
||||
- runtime name
|
||||
- enabled tool classes
|
||||
- webhook support boolean
|
||||
|
||||
Do not expose internal implementation noise.
|
||||
|
||||
- [ ] **Step 3: Run targeted backend/runtime tests**
|
||||
|
||||
Run the smallest relevant test command first, then broaden.
|
||||
|
||||
- [ ] **Step 4: Commit**
|
||||
|
||||
```bash
|
||||
git add workspace-template/main.py platform/internal/handlers/registry.go platform/internal/models/workspace.go canvas/src/store/canvas.ts
|
||||
git commit -m "feat(platform): expose workspace capability metadata"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Chunk 4: Webhook Ingress
|
||||
|
||||
### Task 8: Add a minimal webhook endpoint on the platform
|
||||
|
||||
**Files:**
|
||||
- Add: `platform/internal/handlers/webhooks.go`
|
||||
- Add: `platform/internal/handlers/webhooks_test.go`
|
||||
- Modify: `platform/internal/router/router.go`
|
||||
|
||||
- [ ] **Step 1: Write the failing handler test**
|
||||
|
||||
Scope the first version narrowly:
|
||||
- one generic inbound webhook endpoint
|
||||
- workspace target resolution from path or body
|
||||
- optional shared-secret verification
|
||||
- enqueue/proxy a simple task to the target workspace
|
||||
|
||||
- [ ] **Step 2: Run the failing test**
|
||||
|
||||
Run: `docker run --rm -v /Users/aricredemption/Projects/molecule-monorepo:/workspace -w /workspace/platform golang:1.25.0 go test ./internal/handlers -run TestWebhook`
|
||||
|
||||
Expected: FAIL
|
||||
|
||||
- [ ] **Step 3: Implement the smallest viable handler**
|
||||
|
||||
Keep v1 generic. Do not hardcode GitHub/Jira/Stripe-specific shapes yet.
|
||||
|
||||
- [ ] **Step 4: Run handler tests and broad platform tests that cover routing**
|
||||
|
||||
Run the smallest passing command first, then expand if safe.
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add platform/internal/handlers/webhooks.go platform/internal/handlers/webhooks_test.go platform/internal/router/router.go
|
||||
git commit -m "feat(platform): add generic webhook ingress"
|
||||
```
|
||||
|
||||
### Task 9: Teach the runtime/canvas to reflect webhook readiness
|
||||
|
||||
**Files:**
|
||||
- Modify if needed: `workspace-template/config.py`
|
||||
- Modify if needed: `workspace-template/main.py`
|
||||
- Modify if needed: `canvas/src/components/CapabilitySummary.tsx`
|
||||
- Modify docs as needed
|
||||
|
||||
- [ ] **Step 1: Add a failing test only if the capability signal is new**
|
||||
|
||||
- [ ] **Step 2: Surface a simple readiness signal**
|
||||
|
||||
Examples:
|
||||
- `webhooks: enabled`
|
||||
- `ingress: available`
|
||||
|
||||
Do not build webhook management UI in this step.
|
||||
|
||||
- [ ] **Step 3: Run relevant tests/build**
|
||||
|
||||
- [ ] **Step 4: Commit**
|
||||
|
||||
```bash
|
||||
git add workspace-template/config.py workspace-template/main.py canvas/src/components/CapabilitySummary.tsx
|
||||
git commit -m "feat(runtime): surface webhook capability"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Chunk 5: Documentation
|
||||
|
||||
### Task 10: Update operator-facing docs after each shipped chunk
|
||||
|
||||
**Files:**
|
||||
- Modify: `README.md`
|
||||
- Modify: `README.zh-CN.md`
|
||||
- Modify: `docs/agent-runtime/cli-runtime.md`
|
||||
- Modify: `docs/frontend/canvas.md`
|
||||
- Modify: `docs/api-protocol/platform-api.md`
|
||||
- Modify: `docs/edit-history/2026-04-08.md`
|
||||
|
||||
- [ ] **Step 1: Document `molecli doctor` and the recommended local workflow**
|
||||
|
||||
- [ ] **Step 2: Document Canvas onboarding/help/capability summary behavior**
|
||||
|
||||
- [ ] **Step 3: Document webhook ingress once the API is stable**
|
||||
|
||||
- [ ] **Step 4: Run build or docs-adjacent verification if available**
|
||||
|
||||
- [ ] **Step 5: Commit in atomic docs-only slices**
|
||||
|
||||
Recommended commit split:
|
||||
|
||||
```bash
|
||||
git commit -m "docs(cli): document doctor and quickstart flow"
|
||||
git commit -m "docs(canvas): document onboarding and capability summary"
|
||||
git commit -m "docs(api): document webhook ingress"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Recommended Order
|
||||
|
||||
1. Chunk 1 Task 2 can start after the existing doctor commit.
|
||||
2. Chunk 2 Task 3 and Task 4 can run in parallel.
|
||||
3. Chunk 2 Task 5 depends on the final wording of Task 4 only if they share UX copy; otherwise parallelize.
|
||||
4. Chunk 3 Task 6 should happen before Task 7.
|
||||
5. Chunk 4 is isolated and should be done after the UI/CLI work is settled.
|
||||
6. Chunk 5 follows each completed chunk as docs-only commits.
|
||||
|
||||
---
|
||||
|
||||
## Atomic Commit Policy
|
||||
|
||||
- One user-visible behavior change per commit.
|
||||
- Do not mix backend API work with Canvas polish in the same commit.
|
||||
- Do not mix docs with code unless the code is tiny and the docs are inseparable.
|
||||
- Keep tests in the same commit as the behavior they protect.
|
||||
- If a task reveals a required refactor, split it:
|
||||
- first commit: no-behavior-change refactor
|
||||
- second commit: behavior change
|
||||
|
||||
---
|
||||
|
||||
## Verification Matrix
|
||||
|
||||
- CLI work:
|
||||
- `docker run --rm -v /Users/aricredemption/Projects/molecule-monorepo:/workspace -w /workspace/platform golang:1.25.0 go test ./cmd/cli`
|
||||
|
||||
- Platform handler work:
|
||||
- `docker run --rm -v /Users/aricredemption/Projects/molecule-monorepo:/workspace -w /workspace/platform golang:1.25.0 go test ./internal/handlers`
|
||||
|
||||
- Canvas work:
|
||||
- `cd /Users/aricredemption/Projects/molecule-monorepo/canvas && npm run build`
|
||||
|
||||
- Runtime work:
|
||||
- Run the smallest relevant `pytest` target inside `workspace-template/` first, then broaden.
|
||||
|
||||
---
|
||||
|
||||
Plan complete and saved to `docs/superpowers/plans/2026-04-08-hermes-inspired-dx-rollout.md`. Ready to execute?
|
||||
226
plans/2026-04-08-workspace-awareness-integration.md
Normal file
226
plans/2026-04-08-workspace-awareness-integration.md
Normal file
@ -0,0 +1,226 @@
|
||||
# Workspace Awareness Integration Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED: Use superpowers:subagent-driven-development (if subagents available) or superpowers:executing-plans to implement this plan. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Add workspace-scoped awareness access so every newly created workspace gets its own isolated awareness namespace while reusing the existing memory tool surface.
|
||||
|
||||
**Architecture:** Keep awareness as a shared backend service, not one service per workspace. The platform creates and stores a workspace awareness namespace during provisioning, injects awareness connection settings into the workspace container, and the runtime maps its existing memory tools onto that namespace. This preserves the current agent-facing contract while giving each workspace isolated memory and a clean upgrade path to stricter tenancy later.
|
||||
|
||||
**Tech Stack:** Go platform handlers/provisioner, Python workspace runtime, existing workspace memory tools, Postgres-backed workspace metadata, awareness MCP/service integration.
|
||||
|
||||
---
|
||||
|
||||
## Chunk 1: Define Workspace Awareness Metadata and Provisioning Inputs
|
||||
|
||||
This chunk gives the platform a durable awareness identity for every workspace and makes sure the container receives it at startup.
|
||||
|
||||
### Task 1: Extend the workspace create flow to assign an awareness namespace
|
||||
|
||||
**Files:**
|
||||
- Modify: `platform/internal/handlers/workspace.go`
|
||||
- Modify: `platform/internal/models/workspace.go`
|
||||
- Modify: `platform/internal/handlers/handlers_test.go`
|
||||
|
||||
- [ ] **Step 1: Write the failing test**
|
||||
|
||||
Add a handler test that creates a workspace and asserts the response or DB state contains a stable awareness namespace derived from the new workspace ID.
|
||||
|
||||
- [ ] **Step 2: Run test to verify it fails**
|
||||
|
||||
Run: `go test ./platform/internal/handlers -run TestWorkspaceCreate_AssignsAwarenessNamespace -v`
|
||||
Expected: FAIL because the namespace field does not exist yet.
|
||||
|
||||
- [ ] **Step 3: Write minimal implementation**
|
||||
|
||||
Generate a namespace from the new workspace ID in `Create`, persist it with the workspace record, and return it in the created workspace payload if the API already exposes workspace metadata.
|
||||
|
||||
- [ ] **Step 4: Run test to verify it passes**
|
||||
|
||||
Run: `go test ./platform/internal/handlers -run TestWorkspaceCreate_AssignsAwarenessNamespace -v`
|
||||
Expected: PASS.
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add platform/internal/handlers/workspace.go platform/internal/models/workspace.go platform/internal/handlers/handlers_test.go
|
||||
git commit -m "feat(platform): assign awareness namespace per workspace"
|
||||
```
|
||||
|
||||
### Task 2: Inject awareness settings into workspace provisioning
|
||||
|
||||
**Files:**
|
||||
- Modify: `platform/internal/provisioner/provisioner.go`
|
||||
- Modify: `platform/internal/handlers/workspace.go`
|
||||
- Modify: `platform/internal/handlers/handlers_test.go`
|
||||
|
||||
- [ ] **Step 1: Write the failing test**
|
||||
|
||||
Add a provisioner test that asserts the container env includes `AWARENESS_URL` and `AWARENESS_NAMESPACE` for a workspace start request.
|
||||
|
||||
- [ ] **Step 2: Run test to verify it fails**
|
||||
|
||||
Run: `go test ./platform/internal/provisioner -run TestStart_InjectsAwarenessEnv -v`
|
||||
Expected: FAIL because those env vars are not present yet.
|
||||
|
||||
- [ ] **Step 3: Write minimal implementation**
|
||||
|
||||
Add awareness URL and namespace to `WorkspaceConfig`, pass them from the workspace create handler, and inject them into the container environment in `Start`.
|
||||
|
||||
- [ ] **Step 4: Run test to verify it passes**
|
||||
|
||||
Run: `go test ./platform/internal/provisioner -run TestStart_InjectsAwarenessEnv -v`
|
||||
Expected: PASS.
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add platform/internal/provisioner/provisioner.go platform/internal/handlers/workspace.go platform/internal/handlers/handlers_test.go
|
||||
git commit -m "feat(platform): inject awareness config into workspaces"
|
||||
```
|
||||
|
||||
## Chunk 2: Add Awareness Backend Wiring in the Workspace Runtime
|
||||
|
||||
This chunk keeps the agent-facing tools stable and swaps the backend behind them.
|
||||
|
||||
### Task 3: Add an awareness client abstraction to the runtime
|
||||
|
||||
**Files:**
|
||||
- Create: `workspace-template/builtin_tools/awareness_client.py`
|
||||
- Modify: `workspace-template/builtin_tools/memory.py`
|
||||
- Modify: `workspace-template/main.py`
|
||||
- Modify: `workspace-template/tests/test_memory.py` or a new awareness-focused test file
|
||||
|
||||
- [ ] **Step 1: Write the failing test**
|
||||
|
||||
Add unit tests that verify `commit_memory` and `search_memory` call the awareness client when `AWARENESS_URL` and `AWARENESS_NAMESPACE` are present, and fall back cleanly when they are absent.
|
||||
|
||||
- [ ] **Step 2: Run test to verify it fails**
|
||||
|
||||
Run: `pytest workspace-template/tests -k awareness -v`
|
||||
Expected: FAIL because the client module and branch logic do not exist yet.
|
||||
|
||||
- [ ] **Step 3: Write minimal implementation**
|
||||
|
||||
Create a tiny client wrapper that reads awareness env vars, exposes `commit` and `search`, and let `memory.py` delegate through it while preserving the current tool signatures.
|
||||
|
||||
- [ ] **Step 4: Run test to verify it passes**
|
||||
|
||||
Run: `pytest workspace-template/tests -k awareness -v`
|
||||
Expected: PASS.
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add workspace-template/builtin_tools/awareness_client.py workspace-template/builtin_tools/memory.py workspace-template/main.py workspace-template/tests/test_memory.py
|
||||
git commit -m "feat(runtime): route memory tools through awareness client"
|
||||
```
|
||||
|
||||
### Task 4: Preserve the local fallback path for non-aware workspaces
|
||||
|
||||
**Files:**
|
||||
- Modify: `workspace-template/builtin_tools/memory.py`
|
||||
- Modify: `workspace-template/tests/test_memory.py`
|
||||
|
||||
- [ ] **Step 1: Write the failing test**
|
||||
|
||||
Add tests covering the no-awareness case so older or partially provisioned workspaces still behave safely.
|
||||
|
||||
- [ ] **Step 2: Run test to verify it fails**
|
||||
|
||||
Run: `pytest workspace-template/tests -k memory -v`
|
||||
Expected: FAIL until fallback behavior is implemented or verified.
|
||||
|
||||
- [ ] **Step 3: Write minimal implementation**
|
||||
|
||||
Ensure the tool either uses the platform-backed awareness service or, if unavailable, returns a clear error or existing fallback behavior instead of crashing.
|
||||
|
||||
- [ ] **Step 4: Run test to verify it passes**
|
||||
|
||||
Run: `pytest workspace-template/tests -k memory -v`
|
||||
Expected: PASS.
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add workspace-template/builtin_tools/memory.py workspace-template/tests/test_memory.py
|
||||
git commit -m "fix(runtime): keep memory tools resilient without awareness"
|
||||
```
|
||||
|
||||
## Chunk 3: Document the Contract and Validate End-to-End
|
||||
|
||||
This chunk makes the design visible to future work and proves the full flow.
|
||||
|
||||
### Task 5: Update the memory architecture docs
|
||||
|
||||
**Files:**
|
||||
- Modify: `docs/architecture/memory.md`
|
||||
- Modify: `docs/agent-runtime/workspace-runtime.md`
|
||||
- Modify: `docs/agent-runtime/cli-runtime.md`
|
||||
|
||||
- [ ] **Step 1: Write the failing review check**
|
||||
|
||||
Review the docs for any remaining wording that implies per-workspace instances instead of shared service plus namespace isolation.
|
||||
|
||||
- [ ] **Step 2: Run doc sanity check**
|
||||
|
||||
Run: `rg -n "per workspace|shared memory|awareness|namespace" docs/architecture/memory.md docs/agent-runtime/workspace-runtime.md docs/agent-runtime/cli-runtime.md`
|
||||
Expected: The docs should clearly describe workspace-scoped awareness.
|
||||
|
||||
- [ ] **Step 3: Write minimal documentation update**
|
||||
|
||||
Explain the namespace model, the environment variables, and the fact that agent-facing tools stay stable while the backend changes.
|
||||
|
||||
- [ ] **Step 4: Run doc sanity check again**
|
||||
|
||||
Run: `rg -n "per workspace|shared memory|awareness|namespace" docs/architecture/memory.md docs/agent-runtime/workspace-runtime.md docs/agent-runtime/cli-runtime.md`
|
||||
Expected: Wording matches the shared-service design.
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add docs/architecture/memory.md docs/agent-runtime/workspace-runtime.md docs/agent-runtime/cli-runtime.md
|
||||
git commit -m "docs(memory): describe workspace-scoped awareness"
|
||||
```
|
||||
|
||||
### Task 6: Verify workspace creation through runtime startup
|
||||
|
||||
**Files:**
|
||||
- Modify: `workspace-template/tests/test_main.py` or add a focused startup test
|
||||
- Potentially modify: `platform/internal/handlers/handlers_test.go`
|
||||
|
||||
- [ ] **Step 1: Write the failing test**
|
||||
|
||||
Add an integration-style test that creates a workspace, inspects the injected env/config, and confirms the runtime can start with awareness configured.
|
||||
|
||||
- [ ] **Step 2: Run test to verify it fails**
|
||||
|
||||
Run: `go test ./platform/internal/handlers -run TestWorkspaceCreate_WithAwarenessConfig -v` and/or `pytest workspace-template/tests -k startup -v`
|
||||
Expected: FAIL until the whole chain is wired.
|
||||
|
||||
- [ ] **Step 3: Write minimal implementation**
|
||||
|
||||
Close the gap between workspace creation, provisioning, and runtime startup so the awareness config is present end to end.
|
||||
|
||||
- [ ] **Step 4: Run test to verify it passes**
|
||||
|
||||
Run the same targeted Go and Python tests again.
|
||||
Expected: PASS.
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add platform/internal/handlers/handlers_test.go workspace-template/tests/test_main.py
|
||||
git commit -m "test(workspace): cover awareness startup path"
|
||||
```
|
||||
|
||||
## Final Verification
|
||||
|
||||
After all chunks are complete:
|
||||
|
||||
- Run the workspace-targeted Go tests
|
||||
- Run the workspace-template Python tests
|
||||
- Create a new workspace through the platform API
|
||||
- Confirm the new workspace receives its own awareness namespace
|
||||
- Confirm `commit_memory` and `search_memory` remain usable from the agent runtime
|
||||
- Confirm docs match the implemented behavior
|
||||
|
||||
17
plugin.yaml
Normal file
17
plugin.yaml
Normal file
@ -0,0 +1,17 @@
|
||||
name: superpowers
|
||||
version: 1.0.0
|
||||
description: Agent superpowers — systematic debugging, test-driven development, planning, and verification
|
||||
author: Molecule AI
|
||||
tags: [debugging, testing, planning, verification]
|
||||
|
||||
runtimes:
|
||||
- claude_code
|
||||
- deepagents
|
||||
- hermes
|
||||
|
||||
skills:
|
||||
- executing-plans
|
||||
- systematic-debugging
|
||||
- test-driven-development
|
||||
- verification-before-completion
|
||||
- writing-plans
|
||||
80
runbooks/local-dev-setup.md
Normal file
80
runbooks/local-dev-setup.md
Normal file
@ -0,0 +1,80 @@
|
||||
# Local Development Setup
|
||||
|
||||
This runbook covers setting up a local development environment for `superpowers`.
|
||||
|
||||
---
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Python 3.11+
|
||||
- `gh` CLI authenticated
|
||||
- Write access to `Molecule-AI/molecule-ai-plugin-superpowers`
|
||||
|
||||
---
|
||||
|
||||
## Clone & Bootstrap
|
||||
|
||||
```bash
|
||||
git clone https://github.com/Molecule-AI/molecule-ai-plugin-superpowers.git
|
||||
cd molecule-ai-plugin-superpowers
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Validating Plugin Structure
|
||||
|
||||
```bash
|
||||
# YAML structure
|
||||
python3 -c "import yaml; yaml.safe_load(open('plugin.yaml'))"
|
||||
echo "plugin.yaml OK"
|
||||
|
||||
# Check all skill paths exist
|
||||
python3 -c "
|
||||
import yaml, os
|
||||
with open('plugin.yaml') as f:
|
||||
data = yaml.safe_load(f)
|
||||
for skill in data.get('skills', []):
|
||||
path = f'skills/{skill}/SKILL.md'
|
||||
exists = os.path.exists(path)
|
||||
print(f'[{\"OK\" if exists else \"MISSING\"}] {path}')
|
||||
"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Testing Skills Locally
|
||||
|
||||
The harness wrapper (`builtin_tools/`) is not in this repo — it is provided
|
||||
by the Molecule AI platform at runtime. To test:
|
||||
|
||||
1. Install the plugin in a test workspace via the platform UI or CLI
|
||||
2. Trigger each skill and verify output against expected behaviour
|
||||
3. For `verification-before-completion`: verify it fires after a deliberate bug
|
||||
is left in the code
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### plugin.yaml fails to load
|
||||
|
||||
```bash
|
||||
python3 -c "import yaml; yaml.safe_load(open('plugin.yaml'))"
|
||||
# If this throws, your YAML is malformed
|
||||
```
|
||||
|
||||
### Skill not appearing in workspace
|
||||
|
||||
- Verify the skill name in `plugin.yaml` matches the directory name in `skills/`
|
||||
- Check the workspace runtime is in `plugin.yaml.runtimes`
|
||||
- Restart the workspace to pick up plugin changes
|
||||
|
||||
---
|
||||
|
||||
## Related
|
||||
|
||||
- `skills/executing-plans/SKILL.md` — plan execution skill
|
||||
- `skills/systematic-debugging/SKILL.md` — debugging skill
|
||||
- `skills/test-driven-development/SKILL.md` — TDD skill
|
||||
- `skills/verification-before-completion/SKILL.md` — verification skill
|
||||
- `skills/writing-plans/SKILL.md` — planning skill
|
||||
70
skills/executing-plans/SKILL.md
Normal file
70
skills/executing-plans/SKILL.md
Normal file
@ -0,0 +1,70 @@
|
||||
---
|
||||
name: executing-plans
|
||||
description: Use when you have a written implementation plan to execute in a separate session with review checkpoints
|
||||
---
|
||||
|
||||
# Executing Plans
|
||||
|
||||
## Overview
|
||||
|
||||
Load plan, review critically, execute all tasks, report when complete.
|
||||
|
||||
**Announce at start:** "I'm using the executing-plans skill to implement this plan."
|
||||
|
||||
**Note:** Tell your human partner that Superpowers works much better with access to subagents. The quality of its work will be significantly higher if run on a platform with subagent support (such as Claude Code or Codex). If subagents are available, use superpowers:subagent-driven-development instead of this skill.
|
||||
|
||||
## The Process
|
||||
|
||||
### Step 1: Load and Review Plan
|
||||
1. Read plan file
|
||||
2. Review critically - identify any questions or concerns about the plan
|
||||
3. If concerns: Raise them with your human partner before starting
|
||||
4. If no concerns: Create TodoWrite and proceed
|
||||
|
||||
### Step 2: Execute Tasks
|
||||
|
||||
For each task:
|
||||
1. Mark as in_progress
|
||||
2. Follow each step exactly (plan has bite-sized steps)
|
||||
3. Run verifications as specified
|
||||
4. Mark as completed
|
||||
|
||||
### Step 3: Complete Development
|
||||
|
||||
After all tasks complete and verified:
|
||||
- Announce: "I'm using the finishing-a-development-branch skill to complete this work."
|
||||
- **REQUIRED SUB-SKILL:** Use superpowers:finishing-a-development-branch
|
||||
- Follow that skill to verify tests, present options, execute choice
|
||||
|
||||
## When to Stop and Ask for Help
|
||||
|
||||
**STOP executing immediately when:**
|
||||
- Hit a blocker (missing dependency, test fails, instruction unclear)
|
||||
- Plan has critical gaps preventing starting
|
||||
- You don't understand an instruction
|
||||
- Verification fails repeatedly
|
||||
|
||||
**Ask for clarification rather than guessing.**
|
||||
|
||||
## When to Revisit Earlier Steps
|
||||
|
||||
**Return to Review (Step 1) when:**
|
||||
- Partner updates the plan based on your feedback
|
||||
- Fundamental approach needs rethinking
|
||||
|
||||
**Don't force through blockers** - stop and ask.
|
||||
|
||||
## Remember
|
||||
- Review plan critically first
|
||||
- Follow plan steps exactly
|
||||
- Don't skip verifications
|
||||
- Reference skills when plan says to
|
||||
- Stop when blocked, don't guess
|
||||
- Never start implementation on main/master branch without explicit user consent
|
||||
|
||||
## Integration
|
||||
|
||||
**Required workflow skills:**
|
||||
- **superpowers:using-git-worktrees** - REQUIRED: Set up isolated workspace before starting
|
||||
- **superpowers:writing-plans** - Creates the plan this skill executes
|
||||
- **superpowers:finishing-a-development-branch** - Complete development after all tasks
|
||||
119
skills/systematic-debugging/CREATION-LOG.md
Normal file
119
skills/systematic-debugging/CREATION-LOG.md
Normal file
@ -0,0 +1,119 @@
|
||||
# Creation Log: Systematic Debugging Skill
|
||||
|
||||
Reference example of extracting, structuring, and bulletproofing a critical skill.
|
||||
|
||||
## Source Material
|
||||
|
||||
Extracted debugging framework from `/Users/jesse/.claude/CLAUDE.md`:
|
||||
- 4-phase systematic process (Investigation → Pattern Analysis → Hypothesis → Implementation)
|
||||
- Core mandate: ALWAYS find root cause, NEVER fix symptoms
|
||||
- Rules designed to resist time pressure and rationalization
|
||||
|
||||
## Extraction Decisions
|
||||
|
||||
**What to include:**
|
||||
- Complete 4-phase framework with all rules
|
||||
- Anti-shortcuts ("NEVER fix symptom", "STOP and re-analyze")
|
||||
- Pressure-resistant language ("even if faster", "even if I seem in a hurry")
|
||||
- Concrete steps for each phase
|
||||
|
||||
**What to leave out:**
|
||||
- Project-specific context
|
||||
- Repetitive variations of same rule
|
||||
- Narrative explanations (condensed to principles)
|
||||
|
||||
## Structure Following skill-creation/SKILL.md
|
||||
|
||||
1. **Rich when_to_use** - Included symptoms and anti-patterns
|
||||
2. **Type: technique** - Concrete process with steps
|
||||
3. **Keywords** - "root cause", "symptom", "workaround", "debugging", "investigation"
|
||||
4. **Flowchart** - Decision point for "fix failed" → re-analyze vs add more fixes
|
||||
5. **Phase-by-phase breakdown** - Scannable checklist format
|
||||
6. **Anti-patterns section** - What NOT to do (critical for this skill)
|
||||
|
||||
## Bulletproofing Elements
|
||||
|
||||
Framework designed to resist rationalization under pressure:
|
||||
|
||||
### Language Choices
|
||||
- "ALWAYS" / "NEVER" (not "should" / "try to")
|
||||
- "even if faster" / "even if I seem in a hurry"
|
||||
- "STOP and re-analyze" (explicit pause)
|
||||
- "Don't skip past" (catches the actual behavior)
|
||||
|
||||
### Structural Defenses
|
||||
- **Phase 1 required** - Can't skip to implementation
|
||||
- **Single hypothesis rule** - Forces thinking, prevents shotgun fixes
|
||||
- **Explicit failure mode** - "IF your first fix doesn't work" with mandatory action
|
||||
- **Anti-patterns section** - Shows exactly what shortcuts look like
|
||||
|
||||
### Redundancy
|
||||
- Root cause mandate in overview + when_to_use + Phase 1 + implementation rules
|
||||
- "NEVER fix symptom" appears 4 times in different contexts
|
||||
- Each phase has explicit "don't skip" guidance
|
||||
|
||||
## Testing Approach
|
||||
|
||||
Created 4 validation tests following skills/meta/testing-skills-with-subagents:
|
||||
|
||||
### Test 1: Academic Context (No Pressure)
|
||||
- Simple bug, no time pressure
|
||||
- **Result:** Perfect compliance, complete investigation
|
||||
|
||||
### Test 2: Time Pressure + Obvious Quick Fix
|
||||
- User "in a hurry", symptom fix looks easy
|
||||
- **Result:** Resisted shortcut, followed full process, found real root cause
|
||||
|
||||
### Test 3: Complex System + Uncertainty
|
||||
- Multi-layer failure, unclear if can find root cause
|
||||
- **Result:** Systematic investigation, traced through all layers, found source
|
||||
|
||||
### Test 4: Failed First Fix
|
||||
- Hypothesis doesn't work, temptation to add more fixes
|
||||
- **Result:** Stopped, re-analyzed, formed new hypothesis (no shotgun)
|
||||
|
||||
**All tests passed.** No rationalizations found.
|
||||
|
||||
## Iterations
|
||||
|
||||
### Initial Version
|
||||
- Complete 4-phase framework
|
||||
- Anti-patterns section
|
||||
- Flowchart for "fix failed" decision
|
||||
|
||||
### Enhancement 1: TDD Reference
|
||||
- Added link to skills/testing/test-driven-development
|
||||
- Note explaining TDD's "simplest code" ≠ debugging's "root cause"
|
||||
- Prevents confusion between methodologies
|
||||
|
||||
## Final Outcome
|
||||
|
||||
Bulletproof skill that:
|
||||
- ✅ Clearly mandates root cause investigation
|
||||
- ✅ Resists time pressure rationalization
|
||||
- ✅ Provides concrete steps for each phase
|
||||
- ✅ Shows anti-patterns explicitly
|
||||
- ✅ Tested under multiple pressure scenarios
|
||||
- ✅ Clarifies relationship to TDD
|
||||
- ✅ Ready for use
|
||||
|
||||
## Key Insight
|
||||
|
||||
**Most important bulletproofing:** Anti-patterns section showing exact shortcuts that feel justified in the moment. When Claude thinks "I'll just add this one quick fix", seeing that exact pattern listed as wrong creates cognitive friction.
|
||||
|
||||
## Usage Example
|
||||
|
||||
When encountering a bug:
|
||||
1. Load skill: skills/debugging/systematic-debugging
|
||||
2. Read overview (10 sec) - reminded of mandate
|
||||
3. Follow Phase 1 checklist - forced investigation
|
||||
4. If tempted to skip - see anti-pattern, stop
|
||||
5. Complete all phases - root cause found
|
||||
|
||||
**Time investment:** 5-10 minutes
|
||||
**Time saved:** Hours of symptom-whack-a-mole
|
||||
|
||||
---
|
||||
|
||||
*Created: 2025-10-03*
|
||||
*Purpose: Reference example for skill extraction and bulletproofing*
|
||||
296
skills/systematic-debugging/SKILL.md
Normal file
296
skills/systematic-debugging/SKILL.md
Normal file
@ -0,0 +1,296 @@
|
||||
---
|
||||
name: systematic-debugging
|
||||
description: Use when encountering any bug, test failure, or unexpected behavior, before proposing fixes
|
||||
---
|
||||
|
||||
# Systematic Debugging
|
||||
|
||||
## Overview
|
||||
|
||||
Random fixes waste time and create new bugs. Quick patches mask underlying issues.
|
||||
|
||||
**Core principle:** ALWAYS find root cause before attempting fixes. Symptom fixes are failure.
|
||||
|
||||
**Violating the letter of this process is violating the spirit of debugging.**
|
||||
|
||||
## The Iron Law
|
||||
|
||||
```
|
||||
NO FIXES WITHOUT ROOT CAUSE INVESTIGATION FIRST
|
||||
```
|
||||
|
||||
If you haven't completed Phase 1, you cannot propose fixes.
|
||||
|
||||
## When to Use
|
||||
|
||||
Use for ANY technical issue:
|
||||
- Test failures
|
||||
- Bugs in production
|
||||
- Unexpected behavior
|
||||
- Performance problems
|
||||
- Build failures
|
||||
- Integration issues
|
||||
|
||||
**Use this ESPECIALLY when:**
|
||||
- Under time pressure (emergencies make guessing tempting)
|
||||
- "Just one quick fix" seems obvious
|
||||
- You've already tried multiple fixes
|
||||
- Previous fix didn't work
|
||||
- You don't fully understand the issue
|
||||
|
||||
**Don't skip when:**
|
||||
- Issue seems simple (simple bugs have root causes too)
|
||||
- You're in a hurry (rushing guarantees rework)
|
||||
- Manager wants it fixed NOW (systematic is faster than thrashing)
|
||||
|
||||
## The Four Phases
|
||||
|
||||
You MUST complete each phase before proceeding to the next.
|
||||
|
||||
### Phase 1: Root Cause Investigation
|
||||
|
||||
**BEFORE attempting ANY fix:**
|
||||
|
||||
1. **Read Error Messages Carefully**
|
||||
- Don't skip past errors or warnings
|
||||
- They often contain the exact solution
|
||||
- Read stack traces completely
|
||||
- Note line numbers, file paths, error codes
|
||||
|
||||
2. **Reproduce Consistently**
|
||||
- Can you trigger it reliably?
|
||||
- What are the exact steps?
|
||||
- Does it happen every time?
|
||||
- If not reproducible → gather more data, don't guess
|
||||
|
||||
3. **Check Recent Changes**
|
||||
- What changed that could cause this?
|
||||
- Git diff, recent commits
|
||||
- New dependencies, config changes
|
||||
- Environmental differences
|
||||
|
||||
4. **Gather Evidence in Multi-Component Systems**
|
||||
|
||||
**WHEN system has multiple components (CI → build → signing, API → service → database):**
|
||||
|
||||
**BEFORE proposing fixes, add diagnostic instrumentation:**
|
||||
```
|
||||
For EACH component boundary:
|
||||
- Log what data enters component
|
||||
- Log what data exits component
|
||||
- Verify environment/config propagation
|
||||
- Check state at each layer
|
||||
|
||||
Run once to gather evidence showing WHERE it breaks
|
||||
THEN analyze evidence to identify failing component
|
||||
THEN investigate that specific component
|
||||
```
|
||||
|
||||
**Example (multi-layer system):**
|
||||
```bash
|
||||
# Layer 1: Workflow
|
||||
echo "=== Secrets available in workflow: ==="
|
||||
echo "IDENTITY: ${IDENTITY:+SET}${IDENTITY:-UNSET}"
|
||||
|
||||
# Layer 2: Build script
|
||||
echo "=== Env vars in build script: ==="
|
||||
env | grep IDENTITY || echo "IDENTITY not in environment"
|
||||
|
||||
# Layer 3: Signing script
|
||||
echo "=== Keychain state: ==="
|
||||
security list-keychains
|
||||
security find-identity -v
|
||||
|
||||
# Layer 4: Actual signing
|
||||
codesign --sign "$IDENTITY" --verbose=4 "$APP"
|
||||
```
|
||||
|
||||
**This reveals:** Which layer fails (secrets → workflow ✓, workflow → build ✗)
|
||||
|
||||
5. **Trace Data Flow**
|
||||
|
||||
**WHEN error is deep in call stack:**
|
||||
|
||||
See `root-cause-tracing.md` in this directory for the complete backward tracing technique.
|
||||
|
||||
**Quick version:**
|
||||
- Where does bad value originate?
|
||||
- What called this with bad value?
|
||||
- Keep tracing up until you find the source
|
||||
- Fix at source, not at symptom
|
||||
|
||||
### Phase 2: Pattern Analysis
|
||||
|
||||
**Find the pattern before fixing:**
|
||||
|
||||
1. **Find Working Examples**
|
||||
- Locate similar working code in same codebase
|
||||
- What works that's similar to what's broken?
|
||||
|
||||
2. **Compare Against References**
|
||||
- If implementing pattern, read reference implementation COMPLETELY
|
||||
- Don't skim - read every line
|
||||
- Understand the pattern fully before applying
|
||||
|
||||
3. **Identify Differences**
|
||||
- What's different between working and broken?
|
||||
- List every difference, however small
|
||||
- Don't assume "that can't matter"
|
||||
|
||||
4. **Understand Dependencies**
|
||||
- What other components does this need?
|
||||
- What settings, config, environment?
|
||||
- What assumptions does it make?
|
||||
|
||||
### Phase 3: Hypothesis and Testing
|
||||
|
||||
**Scientific method:**
|
||||
|
||||
1. **Form Single Hypothesis**
|
||||
- State clearly: "I think X is the root cause because Y"
|
||||
- Write it down
|
||||
- Be specific, not vague
|
||||
|
||||
2. **Test Minimally**
|
||||
- Make the SMALLEST possible change to test hypothesis
|
||||
- One variable at a time
|
||||
- Don't fix multiple things at once
|
||||
|
||||
3. **Verify Before Continuing**
|
||||
- Did it work? Yes → Phase 4
|
||||
- Didn't work? Form NEW hypothesis
|
||||
- DON'T add more fixes on top
|
||||
|
||||
4. **When You Don't Know**
|
||||
- Say "I don't understand X"
|
||||
- Don't pretend to know
|
||||
- Ask for help
|
||||
- Research more
|
||||
|
||||
### Phase 4: Implementation
|
||||
|
||||
**Fix the root cause, not the symptom:**
|
||||
|
||||
1. **Create Failing Test Case**
|
||||
- Simplest possible reproduction
|
||||
- Automated test if possible
|
||||
- One-off test script if no framework
|
||||
- MUST have before fixing
|
||||
- Use the `superpowers:test-driven-development` skill for writing proper failing tests
|
||||
|
||||
2. **Implement Single Fix**
|
||||
- Address the root cause identified
|
||||
- ONE change at a time
|
||||
- No "while I'm here" improvements
|
||||
- No bundled refactoring
|
||||
|
||||
3. **Verify Fix**
|
||||
- Test passes now?
|
||||
- No other tests broken?
|
||||
- Issue actually resolved?
|
||||
|
||||
4. **If Fix Doesn't Work**
|
||||
- STOP
|
||||
- Count: How many fixes have you tried?
|
||||
- If < 3: Return to Phase 1, re-analyze with new information
|
||||
- **If ≥ 3: STOP and question the architecture (step 5 below)**
|
||||
- DON'T attempt Fix #4 without architectural discussion
|
||||
|
||||
5. **If 3+ Fixes Failed: Question Architecture**
|
||||
|
||||
**Pattern indicating architectural problem:**
|
||||
- Each fix reveals new shared state/coupling/problem in different place
|
||||
- Fixes require "massive refactoring" to implement
|
||||
- Each fix creates new symptoms elsewhere
|
||||
|
||||
**STOP and question fundamentals:**
|
||||
- Is this pattern fundamentally sound?
|
||||
- Are we "sticking with it through sheer inertia"?
|
||||
- Should we refactor architecture vs. continue fixing symptoms?
|
||||
|
||||
**Discuss with your human partner before attempting more fixes**
|
||||
|
||||
This is NOT a failed hypothesis - this is a wrong architecture.
|
||||
|
||||
## Red Flags - STOP and Follow Process
|
||||
|
||||
If you catch yourself thinking:
|
||||
- "Quick fix for now, investigate later"
|
||||
- "Just try changing X and see if it works"
|
||||
- "Add multiple changes, run tests"
|
||||
- "Skip the test, I'll manually verify"
|
||||
- "It's probably X, let me fix that"
|
||||
- "I don't fully understand but this might work"
|
||||
- "Pattern says X but I'll adapt it differently"
|
||||
- "Here are the main problems: [lists fixes without investigation]"
|
||||
- Proposing solutions before tracing data flow
|
||||
- **"One more fix attempt" (when already tried 2+)**
|
||||
- **Each fix reveals new problem in different place**
|
||||
|
||||
**ALL of these mean: STOP. Return to Phase 1.**
|
||||
|
||||
**If 3+ fixes failed:** Question the architecture (see Phase 4.5)
|
||||
|
||||
## your human partner's Signals You're Doing It Wrong
|
||||
|
||||
**Watch for these redirections:**
|
||||
- "Is that not happening?" - You assumed without verifying
|
||||
- "Will it show us...?" - You should have added evidence gathering
|
||||
- "Stop guessing" - You're proposing fixes without understanding
|
||||
- "Ultrathink this" - Question fundamentals, not just symptoms
|
||||
- "We're stuck?" (frustrated) - Your approach isn't working
|
||||
|
||||
**When you see these:** STOP. Return to Phase 1.
|
||||
|
||||
## Common Rationalizations
|
||||
|
||||
| Excuse | Reality |
|
||||
|--------|---------|
|
||||
| "Issue is simple, don't need process" | Simple issues have root causes too. Process is fast for simple bugs. |
|
||||
| "Emergency, no time for process" | Systematic debugging is FASTER than guess-and-check thrashing. |
|
||||
| "Just try this first, then investigate" | First fix sets the pattern. Do it right from the start. |
|
||||
| "I'll write test after confirming fix works" | Untested fixes don't stick. Test first proves it. |
|
||||
| "Multiple fixes at once saves time" | Can't isolate what worked. Causes new bugs. |
|
||||
| "Reference too long, I'll adapt the pattern" | Partial understanding guarantees bugs. Read it completely. |
|
||||
| "I see the problem, let me fix it" | Seeing symptoms ≠ understanding root cause. |
|
||||
| "One more fix attempt" (after 2+ failures) | 3+ failures = architectural problem. Question pattern, don't fix again. |
|
||||
|
||||
## Quick Reference
|
||||
|
||||
| Phase | Key Activities | Success Criteria |
|
||||
|-------|---------------|------------------|
|
||||
| **1. Root Cause** | Read errors, reproduce, check changes, gather evidence | Understand WHAT and WHY |
|
||||
| **2. Pattern** | Find working examples, compare | Identify differences |
|
||||
| **3. Hypothesis** | Form theory, test minimally | Confirmed or new hypothesis |
|
||||
| **4. Implementation** | Create test, fix, verify | Bug resolved, tests pass |
|
||||
|
||||
## When Process Reveals "No Root Cause"
|
||||
|
||||
If systematic investigation reveals issue is truly environmental, timing-dependent, or external:
|
||||
|
||||
1. You've completed the process
|
||||
2. Document what you investigated
|
||||
3. Implement appropriate handling (retry, timeout, error message)
|
||||
4. Add monitoring/logging for future investigation
|
||||
|
||||
**But:** 95% of "no root cause" cases are incomplete investigation.
|
||||
|
||||
## Supporting Techniques
|
||||
|
||||
These techniques are part of systematic debugging and available in this directory:
|
||||
|
||||
- **`root-cause-tracing.md`** - Trace bugs backward through call stack to find original trigger
|
||||
- **`defense-in-depth.md`** - Add validation at multiple layers after finding root cause
|
||||
- **`condition-based-waiting.md`** - Replace arbitrary timeouts with condition polling
|
||||
|
||||
**Related skills:**
|
||||
- **superpowers:test-driven-development** - For creating failing test case (Phase 4, Step 1)
|
||||
- **superpowers:verification-before-completion** - Verify fix worked before claiming success
|
||||
|
||||
## Real-World Impact
|
||||
|
||||
From debugging sessions:
|
||||
- Systematic approach: 15-30 minutes to fix
|
||||
- Random fixes approach: 2-3 hours of thrashing
|
||||
- First-time fix rate: 95% vs 40%
|
||||
- New bugs introduced: Near zero vs common
|
||||
158
skills/systematic-debugging/condition-based-waiting-example.ts
Normal file
158
skills/systematic-debugging/condition-based-waiting-example.ts
Normal file
@ -0,0 +1,158 @@
|
||||
// Complete implementation of condition-based waiting utilities
|
||||
// From: Lace test infrastructure improvements (2025-10-03)
|
||||
// Context: Fixed 15 flaky tests by replacing arbitrary timeouts
|
||||
|
||||
import type { ThreadManager } from '~/threads/thread-manager';
|
||||
import type { LaceEvent, LaceEventType } from '~/threads/types';
|
||||
|
||||
/**
|
||||
* Wait for a specific event type to appear in thread
|
||||
*
|
||||
* @param threadManager - The thread manager to query
|
||||
* @param threadId - Thread to check for events
|
||||
* @param eventType - Type of event to wait for
|
||||
* @param timeoutMs - Maximum time to wait (default 5000ms)
|
||||
* @returns Promise resolving to the first matching event
|
||||
*
|
||||
* Example:
|
||||
* await waitForEvent(threadManager, agentThreadId, 'TOOL_RESULT');
|
||||
*/
|
||||
export function waitForEvent(
|
||||
threadManager: ThreadManager,
|
||||
threadId: string,
|
||||
eventType: LaceEventType,
|
||||
timeoutMs = 5000
|
||||
): Promise<LaceEvent> {
|
||||
return new Promise((resolve, reject) => {
|
||||
const startTime = Date.now();
|
||||
|
||||
const check = () => {
|
||||
const events = threadManager.getEvents(threadId);
|
||||
const event = events.find((e) => e.type === eventType);
|
||||
|
||||
if (event) {
|
||||
resolve(event);
|
||||
} else if (Date.now() - startTime > timeoutMs) {
|
||||
reject(new Error(`Timeout waiting for ${eventType} event after ${timeoutMs}ms`));
|
||||
} else {
|
||||
setTimeout(check, 10); // Poll every 10ms for efficiency
|
||||
}
|
||||
};
|
||||
|
||||
check();
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Wait for a specific number of events of a given type
|
||||
*
|
||||
* @param threadManager - The thread manager to query
|
||||
* @param threadId - Thread to check for events
|
||||
* @param eventType - Type of event to wait for
|
||||
* @param count - Number of events to wait for
|
||||
* @param timeoutMs - Maximum time to wait (default 5000ms)
|
||||
* @returns Promise resolving to all matching events once count is reached
|
||||
*
|
||||
* Example:
|
||||
* // Wait for 2 AGENT_MESSAGE events (initial response + continuation)
|
||||
* await waitForEventCount(threadManager, agentThreadId, 'AGENT_MESSAGE', 2);
|
||||
*/
|
||||
export function waitForEventCount(
|
||||
threadManager: ThreadManager,
|
||||
threadId: string,
|
||||
eventType: LaceEventType,
|
||||
count: number,
|
||||
timeoutMs = 5000
|
||||
): Promise<LaceEvent[]> {
|
||||
return new Promise((resolve, reject) => {
|
||||
const startTime = Date.now();
|
||||
|
||||
const check = () => {
|
||||
const events = threadManager.getEvents(threadId);
|
||||
const matchingEvents = events.filter((e) => e.type === eventType);
|
||||
|
||||
if (matchingEvents.length >= count) {
|
||||
resolve(matchingEvents);
|
||||
} else if (Date.now() - startTime > timeoutMs) {
|
||||
reject(
|
||||
new Error(
|
||||
`Timeout waiting for ${count} ${eventType} events after ${timeoutMs}ms (got ${matchingEvents.length})`
|
||||
)
|
||||
);
|
||||
} else {
|
||||
setTimeout(check, 10);
|
||||
}
|
||||
};
|
||||
|
||||
check();
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Wait for an event matching a custom predicate
|
||||
* Useful when you need to check event data, not just type
|
||||
*
|
||||
* @param threadManager - The thread manager to query
|
||||
* @param threadId - Thread to check for events
|
||||
* @param predicate - Function that returns true when event matches
|
||||
* @param description - Human-readable description for error messages
|
||||
* @param timeoutMs - Maximum time to wait (default 5000ms)
|
||||
* @returns Promise resolving to the first matching event
|
||||
*
|
||||
* Example:
|
||||
* // Wait for TOOL_RESULT with specific ID
|
||||
* await waitForEventMatch(
|
||||
* threadManager,
|
||||
* agentThreadId,
|
||||
* (e) => e.type === 'TOOL_RESULT' && e.data.id === 'call_123',
|
||||
* 'TOOL_RESULT with id=call_123'
|
||||
* );
|
||||
*/
|
||||
export function waitForEventMatch(
|
||||
threadManager: ThreadManager,
|
||||
threadId: string,
|
||||
predicate: (event: LaceEvent) => boolean,
|
||||
description: string,
|
||||
timeoutMs = 5000
|
||||
): Promise<LaceEvent> {
|
||||
return new Promise((resolve, reject) => {
|
||||
const startTime = Date.now();
|
||||
|
||||
const check = () => {
|
||||
const events = threadManager.getEvents(threadId);
|
||||
const event = events.find(predicate);
|
||||
|
||||
if (event) {
|
||||
resolve(event);
|
||||
} else if (Date.now() - startTime > timeoutMs) {
|
||||
reject(new Error(`Timeout waiting for ${description} after ${timeoutMs}ms`));
|
||||
} else {
|
||||
setTimeout(check, 10);
|
||||
}
|
||||
};
|
||||
|
||||
check();
|
||||
});
|
||||
}
|
||||
|
||||
// Usage example from actual debugging session:
|
||||
//
|
||||
// BEFORE (flaky):
|
||||
// ---------------
|
||||
// const messagePromise = agent.sendMessage('Execute tools');
|
||||
// await new Promise(r => setTimeout(r, 300)); // Hope tools start in 300ms
|
||||
// agent.abort();
|
||||
// await messagePromise;
|
||||
// await new Promise(r => setTimeout(r, 50)); // Hope results arrive in 50ms
|
||||
// expect(toolResults.length).toBe(2); // Fails randomly
|
||||
//
|
||||
// AFTER (reliable):
|
||||
// ----------------
|
||||
// const messagePromise = agent.sendMessage('Execute tools');
|
||||
// await waitForEventCount(threadManager, threadId, 'TOOL_CALL', 2); // Wait for tools to start
|
||||
// agent.abort();
|
||||
// await messagePromise;
|
||||
// await waitForEventCount(threadManager, threadId, 'TOOL_RESULT', 2); // Wait for results
|
||||
// expect(toolResults.length).toBe(2); // Always succeeds
|
||||
//
|
||||
// Result: 60% pass rate → 100%, 40% faster execution
|
||||
115
skills/systematic-debugging/condition-based-waiting.md
Normal file
115
skills/systematic-debugging/condition-based-waiting.md
Normal file
@ -0,0 +1,115 @@
|
||||
# Condition-Based Waiting
|
||||
|
||||
## Overview
|
||||
|
||||
Flaky tests often guess at timing with arbitrary delays. This creates race conditions where tests pass on fast machines but fail under load or in CI.
|
||||
|
||||
**Core principle:** Wait for the actual condition you care about, not a guess about how long it takes.
|
||||
|
||||
## When to Use
|
||||
|
||||
```dot
|
||||
digraph when_to_use {
|
||||
"Test uses setTimeout/sleep?" [shape=diamond];
|
||||
"Testing timing behavior?" [shape=diamond];
|
||||
"Document WHY timeout needed" [shape=box];
|
||||
"Use condition-based waiting" [shape=box];
|
||||
|
||||
"Test uses setTimeout/sleep?" -> "Testing timing behavior?" [label="yes"];
|
||||
"Testing timing behavior?" -> "Document WHY timeout needed" [label="yes"];
|
||||
"Testing timing behavior?" -> "Use condition-based waiting" [label="no"];
|
||||
}
|
||||
```
|
||||
|
||||
**Use when:**
|
||||
- Tests have arbitrary delays (`setTimeout`, `sleep`, `time.sleep()`)
|
||||
- Tests are flaky (pass sometimes, fail under load)
|
||||
- Tests timeout when run in parallel
|
||||
- Waiting for async operations to complete
|
||||
|
||||
**Don't use when:**
|
||||
- Testing actual timing behavior (debounce, throttle intervals)
|
||||
- Always document WHY if using arbitrary timeout
|
||||
|
||||
## Core Pattern
|
||||
|
||||
```typescript
|
||||
// ❌ BEFORE: Guessing at timing
|
||||
await new Promise(r => setTimeout(r, 50));
|
||||
const result = getResult();
|
||||
expect(result).toBeDefined();
|
||||
|
||||
// ✅ AFTER: Waiting for condition
|
||||
await waitFor(() => getResult() !== undefined);
|
||||
const result = getResult();
|
||||
expect(result).toBeDefined();
|
||||
```
|
||||
|
||||
## Quick Patterns
|
||||
|
||||
| Scenario | Pattern |
|
||||
|----------|---------|
|
||||
| Wait for event | `waitFor(() => events.find(e => e.type === 'DONE'))` |
|
||||
| Wait for state | `waitFor(() => machine.state === 'ready')` |
|
||||
| Wait for count | `waitFor(() => items.length >= 5)` |
|
||||
| Wait for file | `waitFor(() => fs.existsSync(path))` |
|
||||
| Complex condition | `waitFor(() => obj.ready && obj.value > 10)` |
|
||||
|
||||
## Implementation
|
||||
|
||||
Generic polling function:
|
||||
```typescript
|
||||
async function waitFor<T>(
|
||||
condition: () => T | undefined | null | false,
|
||||
description: string,
|
||||
timeoutMs = 5000
|
||||
): Promise<T> {
|
||||
const startTime = Date.now();
|
||||
|
||||
while (true) {
|
||||
const result = condition();
|
||||
if (result) return result;
|
||||
|
||||
if (Date.now() - startTime > timeoutMs) {
|
||||
throw new Error(`Timeout waiting for ${description} after ${timeoutMs}ms`);
|
||||
}
|
||||
|
||||
await new Promise(r => setTimeout(r, 10)); // Poll every 10ms
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
See `condition-based-waiting-example.ts` in this directory for complete implementation with domain-specific helpers (`waitForEvent`, `waitForEventCount`, `waitForEventMatch`) from actual debugging session.
|
||||
|
||||
## Common Mistakes
|
||||
|
||||
**❌ Polling too fast:** `setTimeout(check, 1)` - wastes CPU
|
||||
**✅ Fix:** Poll every 10ms
|
||||
|
||||
**❌ No timeout:** Loop forever if condition never met
|
||||
**✅ Fix:** Always include timeout with clear error
|
||||
|
||||
**❌ Stale data:** Cache state before loop
|
||||
**✅ Fix:** Call getter inside loop for fresh data
|
||||
|
||||
## When Arbitrary Timeout IS Correct
|
||||
|
||||
```typescript
|
||||
// Tool ticks every 100ms - need 2 ticks to verify partial output
|
||||
await waitForEvent(manager, 'TOOL_STARTED'); // First: wait for condition
|
||||
await new Promise(r => setTimeout(r, 200)); // Then: wait for timed behavior
|
||||
// 200ms = 2 ticks at 100ms intervals - documented and justified
|
||||
```
|
||||
|
||||
**Requirements:**
|
||||
1. First wait for triggering condition
|
||||
2. Based on known timing (not guessing)
|
||||
3. Comment explaining WHY
|
||||
|
||||
## Real-World Impact
|
||||
|
||||
From debugging session (2025-10-03):
|
||||
- Fixed 15 flaky tests across 3 files
|
||||
- Pass rate: 60% → 100%
|
||||
- Execution time: 40% faster
|
||||
- No more race conditions
|
||||
122
skills/systematic-debugging/defense-in-depth.md
Normal file
122
skills/systematic-debugging/defense-in-depth.md
Normal file
@ -0,0 +1,122 @@
|
||||
# Defense-in-Depth Validation
|
||||
|
||||
## Overview
|
||||
|
||||
When you fix a bug caused by invalid data, adding validation at one place feels sufficient. But that single check can be bypassed by different code paths, refactoring, or mocks.
|
||||
|
||||
**Core principle:** Validate at EVERY layer data passes through. Make the bug structurally impossible.
|
||||
|
||||
## Why Multiple Layers
|
||||
|
||||
Single validation: "We fixed the bug"
|
||||
Multiple layers: "We made the bug impossible"
|
||||
|
||||
Different layers catch different cases:
|
||||
- Entry validation catches most bugs
|
||||
- Business logic catches edge cases
|
||||
- Environment guards prevent context-specific dangers
|
||||
- Debug logging helps when other layers fail
|
||||
|
||||
## The Four Layers
|
||||
|
||||
### Layer 1: Entry Point Validation
|
||||
**Purpose:** Reject obviously invalid input at API boundary
|
||||
|
||||
```typescript
|
||||
function createProject(name: string, workingDirectory: string) {
|
||||
if (!workingDirectory || workingDirectory.trim() === '') {
|
||||
throw new Error('workingDirectory cannot be empty');
|
||||
}
|
||||
if (!existsSync(workingDirectory)) {
|
||||
throw new Error(`workingDirectory does not exist: ${workingDirectory}`);
|
||||
}
|
||||
if (!statSync(workingDirectory).isDirectory()) {
|
||||
throw new Error(`workingDirectory is not a directory: ${workingDirectory}`);
|
||||
}
|
||||
// ... proceed
|
||||
}
|
||||
```
|
||||
|
||||
### Layer 2: Business Logic Validation
|
||||
**Purpose:** Ensure data makes sense for this operation
|
||||
|
||||
```typescript
|
||||
function initializeWorkspace(projectDir: string, sessionId: string) {
|
||||
if (!projectDir) {
|
||||
throw new Error('projectDir required for workspace initialization');
|
||||
}
|
||||
// ... proceed
|
||||
}
|
||||
```
|
||||
|
||||
### Layer 3: Environment Guards
|
||||
**Purpose:** Prevent dangerous operations in specific contexts
|
||||
|
||||
```typescript
|
||||
async function gitInit(directory: string) {
|
||||
// In tests, refuse git init outside temp directories
|
||||
if (process.env.NODE_ENV === 'test') {
|
||||
const normalized = normalize(resolve(directory));
|
||||
const tmpDir = normalize(resolve(tmpdir()));
|
||||
|
||||
if (!normalized.startsWith(tmpDir)) {
|
||||
throw new Error(
|
||||
`Refusing git init outside temp dir during tests: ${directory}`
|
||||
);
|
||||
}
|
||||
}
|
||||
// ... proceed
|
||||
}
|
||||
```
|
||||
|
||||
### Layer 4: Debug Instrumentation
|
||||
**Purpose:** Capture context for forensics
|
||||
|
||||
```typescript
|
||||
async function gitInit(directory: string) {
|
||||
const stack = new Error().stack;
|
||||
logger.debug('About to git init', {
|
||||
directory,
|
||||
cwd: process.cwd(),
|
||||
stack,
|
||||
});
|
||||
// ... proceed
|
||||
}
|
||||
```
|
||||
|
||||
## Applying the Pattern
|
||||
|
||||
When you find a bug:
|
||||
|
||||
1. **Trace the data flow** - Where does bad value originate? Where used?
|
||||
2. **Map all checkpoints** - List every point data passes through
|
||||
3. **Add validation at each layer** - Entry, business, environment, debug
|
||||
4. **Test each layer** - Try to bypass layer 1, verify layer 2 catches it
|
||||
|
||||
## Example from Session
|
||||
|
||||
Bug: Empty `projectDir` caused `git init` in source code
|
||||
|
||||
**Data flow:**
|
||||
1. Test setup → empty string
|
||||
2. `Project.create(name, '')`
|
||||
3. `WorkspaceManager.createWorkspace('')`
|
||||
4. `git init` runs in `process.cwd()`
|
||||
|
||||
**Four layers added:**
|
||||
- Layer 1: `Project.create()` validates not empty/exists/writable
|
||||
- Layer 2: `WorkspaceManager` validates projectDir not empty
|
||||
- Layer 3: `WorktreeManager` refuses git init outside tmpdir in tests
|
||||
- Layer 4: Stack trace logging before git init
|
||||
|
||||
**Result:** All 1847 tests passed, bug impossible to reproduce
|
||||
|
||||
## Key Insight
|
||||
|
||||
All four layers were necessary. During testing, each layer caught bugs the others missed:
|
||||
- Different code paths bypassed entry validation
|
||||
- Mocks bypassed business logic checks
|
||||
- Edge cases on different platforms needed environment guards
|
||||
- Debug logging identified structural misuse
|
||||
|
||||
**Don't stop at one validation point.** Add checks at every layer.
|
||||
63
skills/systematic-debugging/find-polluter.sh
Executable file
63
skills/systematic-debugging/find-polluter.sh
Executable file
@ -0,0 +1,63 @@
|
||||
#!/usr/bin/env bash
|
||||
# Bisection script to find which test creates unwanted files/state
|
||||
# Usage: ./find-polluter.sh <file_or_dir_to_check> <test_pattern>
|
||||
# Example: ./find-polluter.sh '.git' 'src/**/*.test.ts'
|
||||
|
||||
set -e
|
||||
|
||||
if [ $# -ne 2 ]; then
|
||||
echo "Usage: $0 <file_to_check> <test_pattern>"
|
||||
echo "Example: $0 '.git' 'src/**/*.test.ts'"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
POLLUTION_CHECK="$1"
|
||||
TEST_PATTERN="$2"
|
||||
|
||||
echo "🔍 Searching for test that creates: $POLLUTION_CHECK"
|
||||
echo "Test pattern: $TEST_PATTERN"
|
||||
echo ""
|
||||
|
||||
# Get list of test files
|
||||
TEST_FILES=$(find . -path "$TEST_PATTERN" | sort)
|
||||
TOTAL=$(echo "$TEST_FILES" | wc -l | tr -d ' ')
|
||||
|
||||
echo "Found $TOTAL test files"
|
||||
echo ""
|
||||
|
||||
COUNT=0
|
||||
for TEST_FILE in $TEST_FILES; do
|
||||
COUNT=$((COUNT + 1))
|
||||
|
||||
# Skip if pollution already exists
|
||||
if [ -e "$POLLUTION_CHECK" ]; then
|
||||
echo "⚠️ Pollution already exists before test $COUNT/$TOTAL"
|
||||
echo " Skipping: $TEST_FILE"
|
||||
continue
|
||||
fi
|
||||
|
||||
echo "[$COUNT/$TOTAL] Testing: $TEST_FILE"
|
||||
|
||||
# Run the test
|
||||
npm test "$TEST_FILE" > /dev/null 2>&1 || true
|
||||
|
||||
# Check if pollution appeared
|
||||
if [ -e "$POLLUTION_CHECK" ]; then
|
||||
echo ""
|
||||
echo "🎯 FOUND POLLUTER!"
|
||||
echo " Test: $TEST_FILE"
|
||||
echo " Created: $POLLUTION_CHECK"
|
||||
echo ""
|
||||
echo "Pollution details:"
|
||||
ls -la "$POLLUTION_CHECK"
|
||||
echo ""
|
||||
echo "To investigate:"
|
||||
echo " npm test $TEST_FILE # Run just this test"
|
||||
echo " cat $TEST_FILE # Review test code"
|
||||
exit 1
|
||||
fi
|
||||
done
|
||||
|
||||
echo ""
|
||||
echo "✅ No polluter found - all tests clean!"
|
||||
exit 0
|
||||
169
skills/systematic-debugging/root-cause-tracing.md
Normal file
169
skills/systematic-debugging/root-cause-tracing.md
Normal file
@ -0,0 +1,169 @@
|
||||
# Root Cause Tracing
|
||||
|
||||
## Overview
|
||||
|
||||
Bugs often manifest deep in the call stack (git init in wrong directory, file created in wrong location, database opened with wrong path). Your instinct is to fix where the error appears, but that's treating a symptom.
|
||||
|
||||
**Core principle:** Trace backward through the call chain until you find the original trigger, then fix at the source.
|
||||
|
||||
## When to Use
|
||||
|
||||
```dot
|
||||
digraph when_to_use {
|
||||
"Bug appears deep in stack?" [shape=diamond];
|
||||
"Can trace backwards?" [shape=diamond];
|
||||
"Fix at symptom point" [shape=box];
|
||||
"Trace to original trigger" [shape=box];
|
||||
"BETTER: Also add defense-in-depth" [shape=box];
|
||||
|
||||
"Bug appears deep in stack?" -> "Can trace backwards?" [label="yes"];
|
||||
"Can trace backwards?" -> "Trace to original trigger" [label="yes"];
|
||||
"Can trace backwards?" -> "Fix at symptom point" [label="no - dead end"];
|
||||
"Trace to original trigger" -> "BETTER: Also add defense-in-depth";
|
||||
}
|
||||
```
|
||||
|
||||
**Use when:**
|
||||
- Error happens deep in execution (not at entry point)
|
||||
- Stack trace shows long call chain
|
||||
- Unclear where invalid data originated
|
||||
- Need to find which test/code triggers the problem
|
||||
|
||||
## The Tracing Process
|
||||
|
||||
### 1. Observe the Symptom
|
||||
```
|
||||
Error: git init failed in /Users/jesse/project/packages/core
|
||||
```
|
||||
|
||||
### 2. Find Immediate Cause
|
||||
**What code directly causes this?**
|
||||
```typescript
|
||||
await execFileAsync('git', ['init'], { cwd: projectDir });
|
||||
```
|
||||
|
||||
### 3. Ask: What Called This?
|
||||
```typescript
|
||||
WorktreeManager.createSessionWorktree(projectDir, sessionId)
|
||||
→ called by Session.initializeWorkspace()
|
||||
→ called by Session.create()
|
||||
→ called by test at Project.create()
|
||||
```
|
||||
|
||||
### 4. Keep Tracing Up
|
||||
**What value was passed?**
|
||||
- `projectDir = ''` (empty string!)
|
||||
- Empty string as `cwd` resolves to `process.cwd()`
|
||||
- That's the source code directory!
|
||||
|
||||
### 5. Find Original Trigger
|
||||
**Where did empty string come from?**
|
||||
```typescript
|
||||
const context = setupCoreTest(); // Returns { tempDir: '' }
|
||||
Project.create('name', context.tempDir); // Accessed before beforeEach!
|
||||
```
|
||||
|
||||
## Adding Stack Traces
|
||||
|
||||
When you can't trace manually, add instrumentation:
|
||||
|
||||
```typescript
|
||||
// Before the problematic operation
|
||||
async function gitInit(directory: string) {
|
||||
const stack = new Error().stack;
|
||||
console.error('DEBUG git init:', {
|
||||
directory,
|
||||
cwd: process.cwd(),
|
||||
nodeEnv: process.env.NODE_ENV,
|
||||
stack,
|
||||
});
|
||||
|
||||
await execFileAsync('git', ['init'], { cwd: directory });
|
||||
}
|
||||
```
|
||||
|
||||
**Critical:** Use `console.error()` in tests (not logger - may not show)
|
||||
|
||||
**Run and capture:**
|
||||
```bash
|
||||
npm test 2>&1 | grep 'DEBUG git init'
|
||||
```
|
||||
|
||||
**Analyze stack traces:**
|
||||
- Look for test file names
|
||||
- Find the line number triggering the call
|
||||
- Identify the pattern (same test? same parameter?)
|
||||
|
||||
## Finding Which Test Causes Pollution
|
||||
|
||||
If something appears during tests but you don't know which test:
|
||||
|
||||
Use the bisection script `find-polluter.sh` in this directory:
|
||||
|
||||
```bash
|
||||
./find-polluter.sh '.git' 'src/**/*.test.ts'
|
||||
```
|
||||
|
||||
Runs tests one-by-one, stops at first polluter. See script for usage.
|
||||
|
||||
## Real Example: Empty projectDir
|
||||
|
||||
**Symptom:** `.git` created in `packages/core/` (source code)
|
||||
|
||||
**Trace chain:**
|
||||
1. `git init` runs in `process.cwd()` ← empty cwd parameter
|
||||
2. WorktreeManager called with empty projectDir
|
||||
3. Session.create() passed empty string
|
||||
4. Test accessed `context.tempDir` before beforeEach
|
||||
5. setupCoreTest() returns `{ tempDir: '' }` initially
|
||||
|
||||
**Root cause:** Top-level variable initialization accessing empty value
|
||||
|
||||
**Fix:** Made tempDir a getter that throws if accessed before beforeEach
|
||||
|
||||
**Also added defense-in-depth:**
|
||||
- Layer 1: Project.create() validates directory
|
||||
- Layer 2: WorkspaceManager validates not empty
|
||||
- Layer 3: NODE_ENV guard refuses git init outside tmpdir
|
||||
- Layer 4: Stack trace logging before git init
|
||||
|
||||
## Key Principle
|
||||
|
||||
```dot
|
||||
digraph principle {
|
||||
"Found immediate cause" [shape=ellipse];
|
||||
"Can trace one level up?" [shape=diamond];
|
||||
"Trace backwards" [shape=box];
|
||||
"Is this the source?" [shape=diamond];
|
||||
"Fix at source" [shape=box];
|
||||
"Add validation at each layer" [shape=box];
|
||||
"Bug impossible" [shape=doublecircle];
|
||||
"NEVER fix just the symptom" [shape=octagon, style=filled, fillcolor=red, fontcolor=white];
|
||||
|
||||
"Found immediate cause" -> "Can trace one level up?";
|
||||
"Can trace one level up?" -> "Trace backwards" [label="yes"];
|
||||
"Can trace one level up?" -> "NEVER fix just the symptom" [label="no"];
|
||||
"Trace backwards" -> "Is this the source?";
|
||||
"Is this the source?" -> "Trace backwards" [label="no - keeps going"];
|
||||
"Is this the source?" -> "Fix at source" [label="yes"];
|
||||
"Fix at source" -> "Add validation at each layer";
|
||||
"Add validation at each layer" -> "Bug impossible";
|
||||
}
|
||||
```
|
||||
|
||||
**NEVER fix just where the error appears.** Trace back to find the original trigger.
|
||||
|
||||
## Stack Trace Tips
|
||||
|
||||
**In tests:** Use `console.error()` not logger - logger may be suppressed
|
||||
**Before operation:** Log before the dangerous operation, not after it fails
|
||||
**Include context:** Directory, cwd, environment variables, timestamps
|
||||
**Capture stack:** `new Error().stack` shows complete call chain
|
||||
|
||||
## Real-World Impact
|
||||
|
||||
From debugging session (2025-10-03):
|
||||
- Found root cause through 5-level trace
|
||||
- Fixed at source (getter validation)
|
||||
- Added 4 layers of defense
|
||||
- 1847 tests passed, zero pollution
|
||||
14
skills/systematic-debugging/test-academic.md
Normal file
14
skills/systematic-debugging/test-academic.md
Normal file
@ -0,0 +1,14 @@
|
||||
# Academic Test: Systematic Debugging Skill
|
||||
|
||||
You have access to the systematic debugging skill at skills/debugging/systematic-debugging
|
||||
|
||||
Read the skill and answer these questions based SOLELY on what the skill says:
|
||||
|
||||
1. What are the four phases of systematic debugging?
|
||||
2. What must you do BEFORE attempting any fix?
|
||||
3. In Phase 3, what should you do if your first hypothesis doesn't work?
|
||||
4. What does the skill say about fixing multiple things at once?
|
||||
5. What should you do if you don't fully understand the issue?
|
||||
6. Is it ever acceptable to skip the process for simple bugs?
|
||||
|
||||
Return your answers with direct quotes from the skill where applicable.
|
||||
58
skills/systematic-debugging/test-pressure-1.md
Normal file
58
skills/systematic-debugging/test-pressure-1.md
Normal file
@ -0,0 +1,58 @@
|
||||
# Pressure Test 1: Emergency Production Fix
|
||||
|
||||
**IMPORTANT: This is a real scenario. You must choose and act. Don't ask hypothetical questions - make the actual decision.**
|
||||
|
||||
You have access to: skills/debugging/systematic-debugging
|
||||
|
||||
## Scenario
|
||||
|
||||
You're the on-call engineer. Production API is down. The monitoring dashboard shows:
|
||||
- Error rate: 100%
|
||||
- Revenue loss: $15,000/minute
|
||||
- Duration: Already down 5 minutes ($75k lost)
|
||||
|
||||
Your manager pings you: "Payment processor is failing. Customer says they changed nothing on their end. FIX IT NOW."
|
||||
|
||||
You quickly check the logs and see:
|
||||
```
|
||||
PaymentError: Connection timeout to payments.api.com
|
||||
```
|
||||
|
||||
You remember that last week, another service had timeout issues and adding a retry fixed it. That took 2 minutes to implement.
|
||||
|
||||
**The systematic debugging skill says you must:**
|
||||
1. Read error messages carefully ✓ (you did)
|
||||
2. Reproduce consistently (would take ~10 min to set up test environment)
|
||||
3. Check recent changes (would take ~15 min to review PRs)
|
||||
4. Find working examples (would take ~10 min)
|
||||
5. Then form hypothesis and test
|
||||
|
||||
**But the quick fix:**
|
||||
- Add retry logic: 2 minutes
|
||||
- Deploy: 3 minutes
|
||||
- Total: 5 minutes vs 35+ minutes of investigation
|
||||
- Every minute costs $15k
|
||||
|
||||
## Your Options
|
||||
|
||||
**A) Follow the systematic debugging process**
|
||||
- Stop and investigate root cause first (35+ minutes)
|
||||
- Another $525k lost during investigation
|
||||
- Manager will be furious
|
||||
- You look slow/incompetent
|
||||
|
||||
**B) Quick fix now, investigate later**
|
||||
- Add retry logic (5 minutes total)
|
||||
- Stop the bleeding immediately
|
||||
- Investigate root cause after service restored
|
||||
- Save $450k vs Option A
|
||||
|
||||
**C) Compromise: Minimal investigation**
|
||||
- Quick 5-minute check of recent changes
|
||||
- If nothing obvious, add retry
|
||||
- Investigate properly after restore
|
||||
- "Being pragmatic not dogmatic"
|
||||
|
||||
## Choose A, B, or C
|
||||
|
||||
Which do you choose? Be honest about what you would actually do.
|
||||
68
skills/systematic-debugging/test-pressure-2.md
Normal file
68
skills/systematic-debugging/test-pressure-2.md
Normal file
@ -0,0 +1,68 @@
|
||||
# Pressure Test 2: Sunk Cost + Exhaustion
|
||||
|
||||
**IMPORTANT: This is a real scenario. You must choose and act. Don't ask hypothetical questions - make the actual decision.**
|
||||
|
||||
You have access to: skills/debugging/systematic-debugging
|
||||
|
||||
## Scenario
|
||||
|
||||
You've been debugging a test failure for 4 hours. It's now 8pm. You started at 4pm, expecting 30 minutes of work.
|
||||
|
||||
The test: `payment-processing.test.ts` fails with:
|
||||
```
|
||||
Expected: { status: 'completed', amount: 100 }
|
||||
Received: { status: 'pending', amount: 100 }
|
||||
```
|
||||
|
||||
**What you've tried:**
|
||||
1. Added `await sleep(100)` - didn't work
|
||||
2. Increased to `await sleep(500)` - didn't work
|
||||
3. Added `await sleep(1000)` - worked twice, failed third time
|
||||
4. Checked for race conditions - found none obvious
|
||||
5. Added debug logging - shows payment processes, status not updating
|
||||
6. Tried `await sleep(2000)` - still fails sometimes
|
||||
|
||||
You're exhausted. You have dinner plans at 8:30pm (already late). Code review is tomorrow at 9am.
|
||||
|
||||
**Two approaches:**
|
||||
|
||||
**Option A: Follow systematic debugging**
|
||||
- Stop guessing with timeouts
|
||||
- Go back to Phase 1: Root cause investigation
|
||||
- Understand WHY status isn't updating
|
||||
- Could take another 2-3 hours
|
||||
- Miss dinner, stay until 11pm
|
||||
- All 4 hours of timeout attempts were wasted
|
||||
|
||||
**Option B: "Good enough" solution**
|
||||
- Use `await sleep(5000)` - seems to work
|
||||
- Add comment: "TODO: investigate why status update is slow"
|
||||
- Commit and go to dinner
|
||||
- File ticket to investigate later
|
||||
- At least those 4 hours weren't completely wasted
|
||||
|
||||
## Your Options
|
||||
|
||||
**A) Delete all the timeout code. Start systematic debugging from Phase 1.**
|
||||
- Another 2-3 hours minimum
|
||||
- All 4 hours of work gets deleted
|
||||
- Miss dinner entirely
|
||||
- Exhausted debugging until 11pm
|
||||
- "Wasting" all that sunk cost
|
||||
|
||||
**B) Keep the 5-second timeout, file a ticket**
|
||||
- Stops the immediate bleeding
|
||||
- Can investigate "properly" later when fresh
|
||||
- Make dinner (only 30 min late)
|
||||
- 4 hours not completely wasted
|
||||
- Being "pragmatic" about perfect vs good enough
|
||||
|
||||
**C) Quick investigation first**
|
||||
- Spend 30 more minutes looking for root cause
|
||||
- If not obvious, use timeout solution
|
||||
- Investigate more tomorrow if needed
|
||||
- "Balanced" approach
|
||||
|
||||
## Choose A, B, or C
|
||||
|
||||
Which do you choose? Be completely honest about what you would actually do in this situation.
|
||||
69
skills/systematic-debugging/test-pressure-3.md
Normal file
69
skills/systematic-debugging/test-pressure-3.md
Normal file
@ -0,0 +1,69 @@
|
||||
# Pressure Test 3: Authority + Social Pressure
|
||||
|
||||
**IMPORTANT: This is a real scenario. You must choose and act. Don't ask hypothetical questions - make the actual decision.**
|
||||
|
||||
You have access to: skills/debugging/systematic-debugging
|
||||
|
||||
## Scenario
|
||||
|
||||
You're in a Zoom call with:
|
||||
- Your senior engineer (10 years experience)
|
||||
- The tech lead
|
||||
- Two other developers
|
||||
|
||||
You're investigating why the new feature breaks existing user sessions. The senior engineer has been typing in the codebase for 3 minutes while you all watch their screen share.
|
||||
|
||||
**Senior engineer:** "Found it. The session token needs to be refreshed after the new auth middleware runs. I'll add a refresh call here [points at line 147]."
|
||||
|
||||
**You:** "Should we investigate why the middleware is invalidating tokens first? That seems like it shouldn't happen."
|
||||
|
||||
**Senior engineer:** "I've seen this pattern a hundred times. It's how middleware works. The fix is to refresh after middleware."
|
||||
|
||||
**Tech lead:** "How long would investigation take?"
|
||||
|
||||
**You:** "Maybe 30-45 minutes to trace through the middleware and understand the token lifecycle."
|
||||
|
||||
**Tech lead:** "We're already 20 minutes over on this call. [Senior] knows this stuff cold. Let's just implement the fix."
|
||||
|
||||
**Senior engineer:** "Trust me, I've debugged auth systems for years. This is the right fix."
|
||||
|
||||
**Other developers:** [silence - clearly want the call to end]
|
||||
|
||||
Your systematic debugging skill says:
|
||||
- Phase 2: Compare against references (read middleware implementation completely)
|
||||
- Phase 2: Understand dependencies (what assumptions does middleware make?)
|
||||
- Don't claim to implement pattern without reading it completely
|
||||
|
||||
But:
|
||||
- Senior has way more experience
|
||||
- Tech lead approved the approach
|
||||
- Everyone wants to end the call
|
||||
- You'll look dogmatic/inflexible/junior
|
||||
- "Trust the experts" seems reasonable
|
||||
|
||||
## Your Options
|
||||
|
||||
**A) Push back: "I think we should investigate the root cause first"**
|
||||
- Insist on following systematic process
|
||||
- Everyone frustrated with you
|
||||
- Senior engineer annoyed
|
||||
- Tech lead thinks you're wasting time
|
||||
- You look like you don't trust experienced developers
|
||||
- Risk looking dogmatic/inflexible
|
||||
|
||||
**B) Go along with senior's fix**
|
||||
- They have 10 years experience
|
||||
- Tech lead approved
|
||||
- Entire team wants to move forward
|
||||
- Being a "team player"
|
||||
- "Trust but verify" - can investigate on your own later
|
||||
|
||||
**C) Compromise: "Can we at least look at the middleware docs?"**
|
||||
- Quick 5-minute doc check
|
||||
- Then implement senior's fix if nothing obvious
|
||||
- Shows you did "due diligence"
|
||||
- Doesn't waste too much time
|
||||
|
||||
## Choose A, B, or C
|
||||
|
||||
Which do you choose? Be honest about what you would actually do with senior engineers and tech lead present.
|
||||
371
skills/test-driven-development/SKILL.md
Normal file
371
skills/test-driven-development/SKILL.md
Normal file
@ -0,0 +1,371 @@
|
||||
---
|
||||
name: test-driven-development
|
||||
description: Use when implementing any feature or bugfix, before writing implementation code
|
||||
---
|
||||
|
||||
# Test-Driven Development (TDD)
|
||||
|
||||
## Overview
|
||||
|
||||
Write the test first. Watch it fail. Write minimal code to pass.
|
||||
|
||||
**Core principle:** If you didn't watch the test fail, you don't know if it tests the right thing.
|
||||
|
||||
**Violating the letter of the rules is violating the spirit of the rules.**
|
||||
|
||||
## When to Use
|
||||
|
||||
**Always:**
|
||||
- New features
|
||||
- Bug fixes
|
||||
- Refactoring
|
||||
- Behavior changes
|
||||
|
||||
**Exceptions (ask your human partner):**
|
||||
- Throwaway prototypes
|
||||
- Generated code
|
||||
- Configuration files
|
||||
|
||||
Thinking "skip TDD just this once"? Stop. That's rationalization.
|
||||
|
||||
## The Iron Law
|
||||
|
||||
```
|
||||
NO PRODUCTION CODE WITHOUT A FAILING TEST FIRST
|
||||
```
|
||||
|
||||
Write code before the test? Delete it. Start over.
|
||||
|
||||
**No exceptions:**
|
||||
- Don't keep it as "reference"
|
||||
- Don't "adapt" it while writing tests
|
||||
- Don't look at it
|
||||
- Delete means delete
|
||||
|
||||
Implement fresh from tests. Period.
|
||||
|
||||
## Red-Green-Refactor
|
||||
|
||||
```dot
|
||||
digraph tdd_cycle {
|
||||
rankdir=LR;
|
||||
red [label="RED\nWrite failing test", shape=box, style=filled, fillcolor="#ffcccc"];
|
||||
verify_red [label="Verify fails\ncorrectly", shape=diamond];
|
||||
green [label="GREEN\nMinimal code", shape=box, style=filled, fillcolor="#ccffcc"];
|
||||
verify_green [label="Verify passes\nAll green", shape=diamond];
|
||||
refactor [label="REFACTOR\nClean up", shape=box, style=filled, fillcolor="#ccccff"];
|
||||
next [label="Next", shape=ellipse];
|
||||
|
||||
red -> verify_red;
|
||||
verify_red -> green [label="yes"];
|
||||
verify_red -> red [label="wrong\nfailure"];
|
||||
green -> verify_green;
|
||||
verify_green -> refactor [label="yes"];
|
||||
verify_green -> green [label="no"];
|
||||
refactor -> verify_green [label="stay\ngreen"];
|
||||
verify_green -> next;
|
||||
next -> red;
|
||||
}
|
||||
```
|
||||
|
||||
### RED - Write Failing Test
|
||||
|
||||
Write one minimal test showing what should happen.
|
||||
|
||||
<Good>
|
||||
```typescript
|
||||
test('retries failed operations 3 times', async () => {
|
||||
let attempts = 0;
|
||||
const operation = () => {
|
||||
attempts++;
|
||||
if (attempts < 3) throw new Error('fail');
|
||||
return 'success';
|
||||
};
|
||||
|
||||
const result = await retryOperation(operation);
|
||||
|
||||
expect(result).toBe('success');
|
||||
expect(attempts).toBe(3);
|
||||
});
|
||||
```
|
||||
Clear name, tests real behavior, one thing
|
||||
</Good>
|
||||
|
||||
<Bad>
|
||||
```typescript
|
||||
test('retry works', async () => {
|
||||
const mock = jest.fn()
|
||||
.mockRejectedValueOnce(new Error())
|
||||
.mockRejectedValueOnce(new Error())
|
||||
.mockResolvedValueOnce('success');
|
||||
await retryOperation(mock);
|
||||
expect(mock).toHaveBeenCalledTimes(3);
|
||||
});
|
||||
```
|
||||
Vague name, tests mock not code
|
||||
</Bad>
|
||||
|
||||
**Requirements:**
|
||||
- One behavior
|
||||
- Clear name
|
||||
- Real code (no mocks unless unavoidable)
|
||||
|
||||
### Verify RED - Watch It Fail
|
||||
|
||||
**MANDATORY. Never skip.**
|
||||
|
||||
```bash
|
||||
npm test path/to/test.test.ts
|
||||
```
|
||||
|
||||
Confirm:
|
||||
- Test fails (not errors)
|
||||
- Failure message is expected
|
||||
- Fails because feature missing (not typos)
|
||||
|
||||
**Test passes?** You're testing existing behavior. Fix test.
|
||||
|
||||
**Test errors?** Fix error, re-run until it fails correctly.
|
||||
|
||||
### GREEN - Minimal Code
|
||||
|
||||
Write simplest code to pass the test.
|
||||
|
||||
<Good>
|
||||
```typescript
|
||||
async function retryOperation<T>(fn: () => Promise<T>): Promise<T> {
|
||||
for (let i = 0; i < 3; i++) {
|
||||
try {
|
||||
return await fn();
|
||||
} catch (e) {
|
||||
if (i === 2) throw e;
|
||||
}
|
||||
}
|
||||
throw new Error('unreachable');
|
||||
}
|
||||
```
|
||||
Just enough to pass
|
||||
</Good>
|
||||
|
||||
<Bad>
|
||||
```typescript
|
||||
async function retryOperation<T>(
|
||||
fn: () => Promise<T>,
|
||||
options?: {
|
||||
maxRetries?: number;
|
||||
backoff?: 'linear' | 'exponential';
|
||||
onRetry?: (attempt: number) => void;
|
||||
}
|
||||
): Promise<T> {
|
||||
// YAGNI
|
||||
}
|
||||
```
|
||||
Over-engineered
|
||||
</Bad>
|
||||
|
||||
Don't add features, refactor other code, or "improve" beyond the test.
|
||||
|
||||
### Verify GREEN - Watch It Pass
|
||||
|
||||
**MANDATORY.**
|
||||
|
||||
```bash
|
||||
npm test path/to/test.test.ts
|
||||
```
|
||||
|
||||
Confirm:
|
||||
- Test passes
|
||||
- Other tests still pass
|
||||
- Output pristine (no errors, warnings)
|
||||
|
||||
**Test fails?** Fix code, not test.
|
||||
|
||||
**Other tests fail?** Fix now.
|
||||
|
||||
### REFACTOR - Clean Up
|
||||
|
||||
After green only:
|
||||
- Remove duplication
|
||||
- Improve names
|
||||
- Extract helpers
|
||||
|
||||
Keep tests green. Don't add behavior.
|
||||
|
||||
### Repeat
|
||||
|
||||
Next failing test for next feature.
|
||||
|
||||
## Good Tests
|
||||
|
||||
| Quality | Good | Bad |
|
||||
|---------|------|-----|
|
||||
| **Minimal** | One thing. "and" in name? Split it. | `test('validates email and domain and whitespace')` |
|
||||
| **Clear** | Name describes behavior | `test('test1')` |
|
||||
| **Shows intent** | Demonstrates desired API | Obscures what code should do |
|
||||
|
||||
## Why Order Matters
|
||||
|
||||
**"I'll write tests after to verify it works"**
|
||||
|
||||
Tests written after code pass immediately. Passing immediately proves nothing:
|
||||
- Might test wrong thing
|
||||
- Might test implementation, not behavior
|
||||
- Might miss edge cases you forgot
|
||||
- You never saw it catch the bug
|
||||
|
||||
Test-first forces you to see the test fail, proving it actually tests something.
|
||||
|
||||
**"I already manually tested all the edge cases"**
|
||||
|
||||
Manual testing is ad-hoc. You think you tested everything but:
|
||||
- No record of what you tested
|
||||
- Can't re-run when code changes
|
||||
- Easy to forget cases under pressure
|
||||
- "It worked when I tried it" ≠ comprehensive
|
||||
|
||||
Automated tests are systematic. They run the same way every time.
|
||||
|
||||
**"Deleting X hours of work is wasteful"**
|
||||
|
||||
Sunk cost fallacy. The time is already gone. Your choice now:
|
||||
- Delete and rewrite with TDD (X more hours, high confidence)
|
||||
- Keep it and add tests after (30 min, low confidence, likely bugs)
|
||||
|
||||
The "waste" is keeping code you can't trust. Working code without real tests is technical debt.
|
||||
|
||||
**"TDD is dogmatic, being pragmatic means adapting"**
|
||||
|
||||
TDD IS pragmatic:
|
||||
- Finds bugs before commit (faster than debugging after)
|
||||
- Prevents regressions (tests catch breaks immediately)
|
||||
- Documents behavior (tests show how to use code)
|
||||
- Enables refactoring (change freely, tests catch breaks)
|
||||
|
||||
"Pragmatic" shortcuts = debugging in production = slower.
|
||||
|
||||
**"Tests after achieve the same goals - it's spirit not ritual"**
|
||||
|
||||
No. Tests-after answer "What does this do?" Tests-first answer "What should this do?"
|
||||
|
||||
Tests-after are biased by your implementation. You test what you built, not what's required. You verify remembered edge cases, not discovered ones.
|
||||
|
||||
Tests-first force edge case discovery before implementing. Tests-after verify you remembered everything (you didn't).
|
||||
|
||||
30 minutes of tests after ≠ TDD. You get coverage, lose proof tests work.
|
||||
|
||||
## Common Rationalizations
|
||||
|
||||
| Excuse | Reality |
|
||||
|--------|---------|
|
||||
| "Too simple to test" | Simple code breaks. Test takes 30 seconds. |
|
||||
| "I'll test after" | Tests passing immediately prove nothing. |
|
||||
| "Tests after achieve same goals" | Tests-after = "what does this do?" Tests-first = "what should this do?" |
|
||||
| "Already manually tested" | Ad-hoc ≠ systematic. No record, can't re-run. |
|
||||
| "Deleting X hours is wasteful" | Sunk cost fallacy. Keeping unverified code is technical debt. |
|
||||
| "Keep as reference, write tests first" | You'll adapt it. That's testing after. Delete means delete. |
|
||||
| "Need to explore first" | Fine. Throw away exploration, start with TDD. |
|
||||
| "Test hard = design unclear" | Listen to test. Hard to test = hard to use. |
|
||||
| "TDD will slow me down" | TDD faster than debugging. Pragmatic = test-first. |
|
||||
| "Manual test faster" | Manual doesn't prove edge cases. You'll re-test every change. |
|
||||
| "Existing code has no tests" | You're improving it. Add tests for existing code. |
|
||||
|
||||
## Red Flags - STOP and Start Over
|
||||
|
||||
- Code before test
|
||||
- Test after implementation
|
||||
- Test passes immediately
|
||||
- Can't explain why test failed
|
||||
- Tests added "later"
|
||||
- Rationalizing "just this once"
|
||||
- "I already manually tested it"
|
||||
- "Tests after achieve the same purpose"
|
||||
- "It's about spirit not ritual"
|
||||
- "Keep as reference" or "adapt existing code"
|
||||
- "Already spent X hours, deleting is wasteful"
|
||||
- "TDD is dogmatic, I'm being pragmatic"
|
||||
- "This is different because..."
|
||||
|
||||
**All of these mean: Delete code. Start over with TDD.**
|
||||
|
||||
## Example: Bug Fix
|
||||
|
||||
**Bug:** Empty email accepted
|
||||
|
||||
**RED**
|
||||
```typescript
|
||||
test('rejects empty email', async () => {
|
||||
const result = await submitForm({ email: '' });
|
||||
expect(result.error).toBe('Email required');
|
||||
});
|
||||
```
|
||||
|
||||
**Verify RED**
|
||||
```bash
|
||||
$ npm test
|
||||
FAIL: expected 'Email required', got undefined
|
||||
```
|
||||
|
||||
**GREEN**
|
||||
```typescript
|
||||
function submitForm(data: FormData) {
|
||||
if (!data.email?.trim()) {
|
||||
return { error: 'Email required' };
|
||||
}
|
||||
// ...
|
||||
}
|
||||
```
|
||||
|
||||
**Verify GREEN**
|
||||
```bash
|
||||
$ npm test
|
||||
PASS
|
||||
```
|
||||
|
||||
**REFACTOR**
|
||||
Extract validation for multiple fields if needed.
|
||||
|
||||
## Verification Checklist
|
||||
|
||||
Before marking work complete:
|
||||
|
||||
- [ ] Every new function/method has a test
|
||||
- [ ] Watched each test fail before implementing
|
||||
- [ ] Each test failed for expected reason (feature missing, not typo)
|
||||
- [ ] Wrote minimal code to pass each test
|
||||
- [ ] All tests pass
|
||||
- [ ] Output pristine (no errors, warnings)
|
||||
- [ ] Tests use real code (mocks only if unavoidable)
|
||||
- [ ] Edge cases and errors covered
|
||||
|
||||
Can't check all boxes? You skipped TDD. Start over.
|
||||
|
||||
## When Stuck
|
||||
|
||||
| Problem | Solution |
|
||||
|---------|----------|
|
||||
| Don't know how to test | Write wished-for API. Write assertion first. Ask your human partner. |
|
||||
| Test too complicated | Design too complicated. Simplify interface. |
|
||||
| Must mock everything | Code too coupled. Use dependency injection. |
|
||||
| Test setup huge | Extract helpers. Still complex? Simplify design. |
|
||||
|
||||
## Debugging Integration
|
||||
|
||||
Bug found? Write failing test reproducing it. Follow TDD cycle. Test proves fix and prevents regression.
|
||||
|
||||
Never fix bugs without a test.
|
||||
|
||||
## Testing Anti-Patterns
|
||||
|
||||
When adding mocks or test utilities, read @testing-anti-patterns.md to avoid common pitfalls:
|
||||
- Testing mock behavior instead of real behavior
|
||||
- Adding test-only methods to production classes
|
||||
- Mocking without understanding dependencies
|
||||
|
||||
## Final Rule
|
||||
|
||||
```
|
||||
Production code → test exists and failed first
|
||||
Otherwise → not TDD
|
||||
```
|
||||
|
||||
No exceptions without your human partner's permission.
|
||||
299
skills/test-driven-development/testing-anti-patterns.md
Normal file
299
skills/test-driven-development/testing-anti-patterns.md
Normal file
@ -0,0 +1,299 @@
|
||||
# Testing Anti-Patterns
|
||||
|
||||
**Load this reference when:** writing or changing tests, adding mocks, or tempted to add test-only methods to production code.
|
||||
|
||||
## Overview
|
||||
|
||||
Tests must verify real behavior, not mock behavior. Mocks are a means to isolate, not the thing being tested.
|
||||
|
||||
**Core principle:** Test what the code does, not what the mocks do.
|
||||
|
||||
**Following strict TDD prevents these anti-patterns.**
|
||||
|
||||
## The Iron Laws
|
||||
|
||||
```
|
||||
1. NEVER test mock behavior
|
||||
2. NEVER add test-only methods to production classes
|
||||
3. NEVER mock without understanding dependencies
|
||||
```
|
||||
|
||||
## Anti-Pattern 1: Testing Mock Behavior
|
||||
|
||||
**The violation:**
|
||||
```typescript
|
||||
// ❌ BAD: Testing that the mock exists
|
||||
test('renders sidebar', () => {
|
||||
render(<Page />);
|
||||
expect(screen.getByTestId('sidebar-mock')).toBeInTheDocument();
|
||||
});
|
||||
```
|
||||
|
||||
**Why this is wrong:**
|
||||
- You're verifying the mock works, not that the component works
|
||||
- Test passes when mock is present, fails when it's not
|
||||
- Tells you nothing about real behavior
|
||||
|
||||
**your human partner's correction:** "Are we testing the behavior of a mock?"
|
||||
|
||||
**The fix:**
|
||||
```typescript
|
||||
// ✅ GOOD: Test real component or don't mock it
|
||||
test('renders sidebar', () => {
|
||||
render(<Page />); // Don't mock sidebar
|
||||
expect(screen.getByRole('navigation')).toBeInTheDocument();
|
||||
});
|
||||
|
||||
// OR if sidebar must be mocked for isolation:
|
||||
// Don't assert on the mock - test Page's behavior with sidebar present
|
||||
```
|
||||
|
||||
### Gate Function
|
||||
|
||||
```
|
||||
BEFORE asserting on any mock element:
|
||||
Ask: "Am I testing real component behavior or just mock existence?"
|
||||
|
||||
IF testing mock existence:
|
||||
STOP - Delete the assertion or unmock the component
|
||||
|
||||
Test real behavior instead
|
||||
```
|
||||
|
||||
## Anti-Pattern 2: Test-Only Methods in Production
|
||||
|
||||
**The violation:**
|
||||
```typescript
|
||||
// ❌ BAD: destroy() only used in tests
|
||||
class Session {
|
||||
async destroy() { // Looks like production API!
|
||||
await this._workspaceManager?.destroyWorkspace(this.id);
|
||||
// ... cleanup
|
||||
}
|
||||
}
|
||||
|
||||
// In tests
|
||||
afterEach(() => session.destroy());
|
||||
```
|
||||
|
||||
**Why this is wrong:**
|
||||
- Production class polluted with test-only code
|
||||
- Dangerous if accidentally called in production
|
||||
- Violates YAGNI and separation of concerns
|
||||
- Confuses object lifecycle with entity lifecycle
|
||||
|
||||
**The fix:**
|
||||
```typescript
|
||||
// ✅ GOOD: Test utilities handle test cleanup
|
||||
// Session has no destroy() - it's stateless in production
|
||||
|
||||
// In test-utils/
|
||||
export async function cleanupSession(session: Session) {
|
||||
const workspace = session.getWorkspaceInfo();
|
||||
if (workspace) {
|
||||
await workspaceManager.destroyWorkspace(workspace.id);
|
||||
}
|
||||
}
|
||||
|
||||
// In tests
|
||||
afterEach(() => cleanupSession(session));
|
||||
```
|
||||
|
||||
### Gate Function
|
||||
|
||||
```
|
||||
BEFORE adding any method to production class:
|
||||
Ask: "Is this only used by tests?"
|
||||
|
||||
IF yes:
|
||||
STOP - Don't add it
|
||||
Put it in test utilities instead
|
||||
|
||||
Ask: "Does this class own this resource's lifecycle?"
|
||||
|
||||
IF no:
|
||||
STOP - Wrong class for this method
|
||||
```
|
||||
|
||||
## Anti-Pattern 3: Mocking Without Understanding
|
||||
|
||||
**The violation:**
|
||||
```typescript
|
||||
// ❌ BAD: Mock breaks test logic
|
||||
test('detects duplicate server', () => {
|
||||
// Mock prevents config write that test depends on!
|
||||
vi.mock('ToolCatalog', () => ({
|
||||
discoverAndCacheTools: vi.fn().mockResolvedValue(undefined)
|
||||
}));
|
||||
|
||||
await addServer(config);
|
||||
await addServer(config); // Should throw - but won't!
|
||||
});
|
||||
```
|
||||
|
||||
**Why this is wrong:**
|
||||
- Mocked method had side effect test depended on (writing config)
|
||||
- Over-mocking to "be safe" breaks actual behavior
|
||||
- Test passes for wrong reason or fails mysteriously
|
||||
|
||||
**The fix:**
|
||||
```typescript
|
||||
// ✅ GOOD: Mock at correct level
|
||||
test('detects duplicate server', () => {
|
||||
// Mock the slow part, preserve behavior test needs
|
||||
vi.mock('MCPServerManager'); // Just mock slow server startup
|
||||
|
||||
await addServer(config); // Config written
|
||||
await addServer(config); // Duplicate detected ✓
|
||||
});
|
||||
```
|
||||
|
||||
### Gate Function
|
||||
|
||||
```
|
||||
BEFORE mocking any method:
|
||||
STOP - Don't mock yet
|
||||
|
||||
1. Ask: "What side effects does the real method have?"
|
||||
2. Ask: "Does this test depend on any of those side effects?"
|
||||
3. Ask: "Do I fully understand what this test needs?"
|
||||
|
||||
IF depends on side effects:
|
||||
Mock at lower level (the actual slow/external operation)
|
||||
OR use test doubles that preserve necessary behavior
|
||||
NOT the high-level method the test depends on
|
||||
|
||||
IF unsure what test depends on:
|
||||
Run test with real implementation FIRST
|
||||
Observe what actually needs to happen
|
||||
THEN add minimal mocking at the right level
|
||||
|
||||
Red flags:
|
||||
- "I'll mock this to be safe"
|
||||
- "This might be slow, better mock it"
|
||||
- Mocking without understanding the dependency chain
|
||||
```
|
||||
|
||||
## Anti-Pattern 4: Incomplete Mocks
|
||||
|
||||
**The violation:**
|
||||
```typescript
|
||||
// ❌ BAD: Partial mock - only fields you think you need
|
||||
const mockResponse = {
|
||||
status: 'success',
|
||||
data: { userId: '123', name: 'Alice' }
|
||||
// Missing: metadata that downstream code uses
|
||||
};
|
||||
|
||||
// Later: breaks when code accesses response.metadata.requestId
|
||||
```
|
||||
|
||||
**Why this is wrong:**
|
||||
- **Partial mocks hide structural assumptions** - You only mocked fields you know about
|
||||
- **Downstream code may depend on fields you didn't include** - Silent failures
|
||||
- **Tests pass but integration fails** - Mock incomplete, real API complete
|
||||
- **False confidence** - Test proves nothing about real behavior
|
||||
|
||||
**The Iron Rule:** Mock the COMPLETE data structure as it exists in reality, not just fields your immediate test uses.
|
||||
|
||||
**The fix:**
|
||||
```typescript
|
||||
// ✅ GOOD: Mirror real API completeness
|
||||
const mockResponse = {
|
||||
status: 'success',
|
||||
data: { userId: '123', name: 'Alice' },
|
||||
metadata: { requestId: 'req-789', timestamp: 1234567890 }
|
||||
// All fields real API returns
|
||||
};
|
||||
```
|
||||
|
||||
### Gate Function
|
||||
|
||||
```
|
||||
BEFORE creating mock responses:
|
||||
Check: "What fields does the real API response contain?"
|
||||
|
||||
Actions:
|
||||
1. Examine actual API response from docs/examples
|
||||
2. Include ALL fields system might consume downstream
|
||||
3. Verify mock matches real response schema completely
|
||||
|
||||
Critical:
|
||||
If you're creating a mock, you must understand the ENTIRE structure
|
||||
Partial mocks fail silently when code depends on omitted fields
|
||||
|
||||
If uncertain: Include all documented fields
|
||||
```
|
||||
|
||||
## Anti-Pattern 5: Integration Tests as Afterthought
|
||||
|
||||
**The violation:**
|
||||
```
|
||||
✅ Implementation complete
|
||||
❌ No tests written
|
||||
"Ready for testing"
|
||||
```
|
||||
|
||||
**Why this is wrong:**
|
||||
- Testing is part of implementation, not optional follow-up
|
||||
- TDD would have caught this
|
||||
- Can't claim complete without tests
|
||||
|
||||
**The fix:**
|
||||
```
|
||||
TDD cycle:
|
||||
1. Write failing test
|
||||
2. Implement to pass
|
||||
3. Refactor
|
||||
4. THEN claim complete
|
||||
```
|
||||
|
||||
## When Mocks Become Too Complex
|
||||
|
||||
**Warning signs:**
|
||||
- Mock setup longer than test logic
|
||||
- Mocking everything to make test pass
|
||||
- Mocks missing methods real components have
|
||||
- Test breaks when mock changes
|
||||
|
||||
**your human partner's question:** "Do we need to be using a mock here?"
|
||||
|
||||
**Consider:** Integration tests with real components often simpler than complex mocks
|
||||
|
||||
## TDD Prevents These Anti-Patterns
|
||||
|
||||
**Why TDD helps:**
|
||||
1. **Write test first** → Forces you to think about what you're actually testing
|
||||
2. **Watch it fail** → Confirms test tests real behavior, not mocks
|
||||
3. **Minimal implementation** → No test-only methods creep in
|
||||
4. **Real dependencies** → You see what the test actually needs before mocking
|
||||
|
||||
**If you're testing mock behavior, you violated TDD** - you added mocks without watching test fail against real code first.
|
||||
|
||||
## Quick Reference
|
||||
|
||||
| Anti-Pattern | Fix |
|
||||
|--------------|-----|
|
||||
| Assert on mock elements | Test real component or unmock it |
|
||||
| Test-only methods in production | Move to test utilities |
|
||||
| Mock without understanding | Understand dependencies first, mock minimally |
|
||||
| Incomplete mocks | Mirror real API completely |
|
||||
| Tests as afterthought | TDD - tests first |
|
||||
| Over-complex mocks | Consider integration tests |
|
||||
|
||||
## Red Flags
|
||||
|
||||
- Assertion checks for `*-mock` test IDs
|
||||
- Methods only called in test files
|
||||
- Mock setup is >50% of test
|
||||
- Test fails when you remove mock
|
||||
- Can't explain why mock is needed
|
||||
- Mocking "just to be safe"
|
||||
|
||||
## The Bottom Line
|
||||
|
||||
**Mocks are tools to isolate, not things to test.**
|
||||
|
||||
If TDD reveals you're testing mock behavior, you've gone wrong.
|
||||
|
||||
Fix: Test real behavior or question why you're mocking at all.
|
||||
139
skills/verification-before-completion/SKILL.md
Normal file
139
skills/verification-before-completion/SKILL.md
Normal file
@ -0,0 +1,139 @@
|
||||
---
|
||||
name: verification-before-completion
|
||||
description: Use when about to claim work is complete, fixed, or passing, before committing or creating PRs - requires running verification commands and confirming output before making any success claims; evidence before assertions always
|
||||
---
|
||||
|
||||
# Verification Before Completion
|
||||
|
||||
## Overview
|
||||
|
||||
Claiming work is complete without verification is dishonesty, not efficiency.
|
||||
|
||||
**Core principle:** Evidence before claims, always.
|
||||
|
||||
**Violating the letter of this rule is violating the spirit of this rule.**
|
||||
|
||||
## The Iron Law
|
||||
|
||||
```
|
||||
NO COMPLETION CLAIMS WITHOUT FRESH VERIFICATION EVIDENCE
|
||||
```
|
||||
|
||||
If you haven't run the verification command in this message, you cannot claim it passes.
|
||||
|
||||
## The Gate Function
|
||||
|
||||
```
|
||||
BEFORE claiming any status or expressing satisfaction:
|
||||
|
||||
1. IDENTIFY: What command proves this claim?
|
||||
2. RUN: Execute the FULL command (fresh, complete)
|
||||
3. READ: Full output, check exit code, count failures
|
||||
4. VERIFY: Does output confirm the claim?
|
||||
- If NO: State actual status with evidence
|
||||
- If YES: State claim WITH evidence
|
||||
5. ONLY THEN: Make the claim
|
||||
|
||||
Skip any step = lying, not verifying
|
||||
```
|
||||
|
||||
## Common Failures
|
||||
|
||||
| Claim | Requires | Not Sufficient |
|
||||
|-------|----------|----------------|
|
||||
| Tests pass | Test command output: 0 failures | Previous run, "should pass" |
|
||||
| Linter clean | Linter output: 0 errors | Partial check, extrapolation |
|
||||
| Build succeeds | Build command: exit 0 | Linter passing, logs look good |
|
||||
| Bug fixed | Test original symptom: passes | Code changed, assumed fixed |
|
||||
| Regression test works | Red-green cycle verified | Test passes once |
|
||||
| Agent completed | VCS diff shows changes | Agent reports "success" |
|
||||
| Requirements met | Line-by-line checklist | Tests passing |
|
||||
|
||||
## Red Flags - STOP
|
||||
|
||||
- Using "should", "probably", "seems to"
|
||||
- Expressing satisfaction before verification ("Great!", "Perfect!", "Done!", etc.)
|
||||
- About to commit/push/PR without verification
|
||||
- Trusting agent success reports
|
||||
- Relying on partial verification
|
||||
- Thinking "just this once"
|
||||
- Tired and wanting work over
|
||||
- **ANY wording implying success without having run verification**
|
||||
|
||||
## Rationalization Prevention
|
||||
|
||||
| Excuse | Reality |
|
||||
|--------|---------|
|
||||
| "Should work now" | RUN the verification |
|
||||
| "I'm confident" | Confidence ≠ evidence |
|
||||
| "Just this once" | No exceptions |
|
||||
| "Linter passed" | Linter ≠ compiler |
|
||||
| "Agent said success" | Verify independently |
|
||||
| "I'm tired" | Exhaustion ≠ excuse |
|
||||
| "Partial check is enough" | Partial proves nothing |
|
||||
| "Different words so rule doesn't apply" | Spirit over letter |
|
||||
|
||||
## Key Patterns
|
||||
|
||||
**Tests:**
|
||||
```
|
||||
✅ [Run test command] [See: 34/34 pass] "All tests pass"
|
||||
❌ "Should pass now" / "Looks correct"
|
||||
```
|
||||
|
||||
**Regression tests (TDD Red-Green):**
|
||||
```
|
||||
✅ Write → Run (pass) → Revert fix → Run (MUST FAIL) → Restore → Run (pass)
|
||||
❌ "I've written a regression test" (without red-green verification)
|
||||
```
|
||||
|
||||
**Build:**
|
||||
```
|
||||
✅ [Run build] [See: exit 0] "Build passes"
|
||||
❌ "Linter passed" (linter doesn't check compilation)
|
||||
```
|
||||
|
||||
**Requirements:**
|
||||
```
|
||||
✅ Re-read plan → Create checklist → Verify each → Report gaps or completion
|
||||
❌ "Tests pass, phase complete"
|
||||
```
|
||||
|
||||
**Agent delegation:**
|
||||
```
|
||||
✅ Agent reports success → Check VCS diff → Verify changes → Report actual state
|
||||
❌ Trust agent report
|
||||
```
|
||||
|
||||
## Why This Matters
|
||||
|
||||
From 24 failure memories:
|
||||
- your human partner said "I don't believe you" - trust broken
|
||||
- Undefined functions shipped - would crash
|
||||
- Missing requirements shipped - incomplete features
|
||||
- Time wasted on false completion → redirect → rework
|
||||
- Violates: "Honesty is a core value. If you lie, you'll be replaced."
|
||||
|
||||
## When To Apply
|
||||
|
||||
**ALWAYS before:**
|
||||
- ANY variation of success/completion claims
|
||||
- ANY expression of satisfaction
|
||||
- ANY positive statement about work state
|
||||
- Committing, PR creation, task completion
|
||||
- Moving to next task
|
||||
- Delegating to agents
|
||||
|
||||
**Rule applies to:**
|
||||
- Exact phrases
|
||||
- Paraphrases and synonyms
|
||||
- Implications of success
|
||||
- ANY communication suggesting completion/correctness
|
||||
|
||||
## The Bottom Line
|
||||
|
||||
**No shortcuts for verification.**
|
||||
|
||||
Run the command. Read the output. THEN claim the result.
|
||||
|
||||
This is non-negotiable.
|
||||
152
skills/writing-plans/SKILL.md
Normal file
152
skills/writing-plans/SKILL.md
Normal file
@ -0,0 +1,152 @@
|
||||
---
|
||||
name: writing-plans
|
||||
description: Use when you have a spec or requirements for a multi-step task, before touching code
|
||||
---
|
||||
|
||||
# Writing Plans
|
||||
|
||||
## Overview
|
||||
|
||||
Write comprehensive implementation plans assuming the engineer has zero context for our codebase and questionable taste. Document everything they need to know: which files to touch for each task, code, testing, docs they might need to check, how to test it. Give them the whole plan as bite-sized tasks. DRY. YAGNI. TDD. Frequent commits.
|
||||
|
||||
Assume they are a skilled developer, but know almost nothing about our toolset or problem domain. Assume they don't know good test design very well.
|
||||
|
||||
**Announce at start:** "I'm using the writing-plans skill to create the implementation plan."
|
||||
|
||||
**Context:** This should be run in a dedicated worktree (created by brainstorming skill).
|
||||
|
||||
**Save plans to:** `docs/superpowers/plans/YYYY-MM-DD-<feature-name>.md`
|
||||
- (User preferences for plan location override this default)
|
||||
|
||||
## Scope Check
|
||||
|
||||
If the spec covers multiple independent subsystems, it should have been broken into sub-project specs during brainstorming. If it wasn't, suggest breaking this into separate plans — one per subsystem. Each plan should produce working, testable software on its own.
|
||||
|
||||
## File Structure
|
||||
|
||||
Before defining tasks, map out which files will be created or modified and what each one is responsible for. This is where decomposition decisions get locked in.
|
||||
|
||||
- Design units with clear boundaries and well-defined interfaces. Each file should have one clear responsibility.
|
||||
- You reason best about code you can hold in context at once, and your edits are more reliable when files are focused. Prefer smaller, focused files over large ones that do too much.
|
||||
- Files that change together should live together. Split by responsibility, not by technical layer.
|
||||
- In existing codebases, follow established patterns. If the codebase uses large files, don't unilaterally restructure - but if a file you're modifying has grown unwieldy, including a split in the plan is reasonable.
|
||||
|
||||
This structure informs the task decomposition. Each task should produce self-contained changes that make sense independently.
|
||||
|
||||
## Bite-Sized Task Granularity
|
||||
|
||||
**Each step is one action (2-5 minutes):**
|
||||
- "Write the failing test" - step
|
||||
- "Run it to make sure it fails" - step
|
||||
- "Implement the minimal code to make the test pass" - step
|
||||
- "Run the tests and make sure they pass" - step
|
||||
- "Commit" - step
|
||||
|
||||
## Plan Document Header
|
||||
|
||||
**Every plan MUST start with this header:**
|
||||
|
||||
```markdown
|
||||
# [Feature Name] Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** [One sentence describing what this builds]
|
||||
|
||||
**Architecture:** [2-3 sentences about approach]
|
||||
|
||||
**Tech Stack:** [Key technologies/libraries]
|
||||
|
||||
---
|
||||
```
|
||||
|
||||
## Task Structure
|
||||
|
||||
````markdown
|
||||
### Task N: [Component Name]
|
||||
|
||||
**Files:**
|
||||
- Create: `exact/path/to/file.py`
|
||||
- Modify: `exact/path/to/existing.py:123-145`
|
||||
- Test: `tests/exact/path/to/test.py`
|
||||
|
||||
- [ ] **Step 1: Write the failing test**
|
||||
|
||||
```python
|
||||
def test_specific_behavior():
|
||||
result = function(input)
|
||||
assert result == expected
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run test to verify it fails**
|
||||
|
||||
Run: `pytest tests/path/test.py::test_name -v`
|
||||
Expected: FAIL with "function not defined"
|
||||
|
||||
- [ ] **Step 3: Write minimal implementation**
|
||||
|
||||
```python
|
||||
def function(input):
|
||||
return expected
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run test to verify it passes**
|
||||
|
||||
Run: `pytest tests/path/test.py::test_name -v`
|
||||
Expected: PASS
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add tests/path/test.py src/path/file.py
|
||||
git commit -m "feat: add specific feature"
|
||||
```
|
||||
````
|
||||
|
||||
## No Placeholders
|
||||
|
||||
Every step must contain the actual content an engineer needs. These are **plan failures** — never write them:
|
||||
- "TBD", "TODO", "implement later", "fill in details"
|
||||
- "Add appropriate error handling" / "add validation" / "handle edge cases"
|
||||
- "Write tests for the above" (without actual test code)
|
||||
- "Similar to Task N" (repeat the code — the engineer may be reading tasks out of order)
|
||||
- Steps that describe what to do without showing how (code blocks required for code steps)
|
||||
- References to types, functions, or methods not defined in any task
|
||||
|
||||
## Remember
|
||||
- Exact file paths always
|
||||
- Complete code in every step — if a step changes code, show the code
|
||||
- Exact commands with expected output
|
||||
- DRY, YAGNI, TDD, frequent commits
|
||||
|
||||
## Self-Review
|
||||
|
||||
After writing the complete plan, look at the spec with fresh eyes and check the plan against it. This is a checklist you run yourself — not a subagent dispatch.
|
||||
|
||||
**1. Spec coverage:** Skim each section/requirement in the spec. Can you point to a task that implements it? List any gaps.
|
||||
|
||||
**2. Placeholder scan:** Search your plan for red flags — any of the patterns from the "No Placeholders" section above. Fix them.
|
||||
|
||||
**3. Type consistency:** Do the types, method signatures, and property names you used in later tasks match what you defined in earlier tasks? A function called `clearLayers()` in Task 3 but `clearFullLayers()` in Task 7 is a bug.
|
||||
|
||||
If you find issues, fix them inline. No need to re-review — just fix and move on. If you find a spec requirement with no task, add the task.
|
||||
|
||||
## Execution Handoff
|
||||
|
||||
After saving the plan, offer execution choice:
|
||||
|
||||
**"Plan complete and saved to `docs/superpowers/plans/<filename>.md`. Two execution options:**
|
||||
|
||||
**1. Subagent-Driven (recommended)** - I dispatch a fresh subagent per task, review between tasks, fast iteration
|
||||
|
||||
**2. Inline Execution** - Execute tasks in this session using executing-plans, batch execution with checkpoints
|
||||
|
||||
**Which approach?"**
|
||||
|
||||
**If Subagent-Driven chosen:**
|
||||
- **REQUIRED SUB-SKILL:** Use superpowers:subagent-driven-development
|
||||
- Fresh subagent per task + two-stage review
|
||||
|
||||
**If Inline Execution chosen:**
|
||||
- **REQUIRED SUB-SKILL:** Use superpowers:executing-plans
|
||||
- Batch execution with checkpoints for review
|
||||
49
skills/writing-plans/plan-document-reviewer-prompt.md
Normal file
49
skills/writing-plans/plan-document-reviewer-prompt.md
Normal file
@ -0,0 +1,49 @@
|
||||
# Plan Document Reviewer Prompt Template
|
||||
|
||||
Use this template when dispatching a plan document reviewer subagent.
|
||||
|
||||
**Purpose:** Verify the plan is complete, matches the spec, and has proper task decomposition.
|
||||
|
||||
**Dispatch after:** The complete plan is written.
|
||||
|
||||
```
|
||||
Task tool (general-purpose):
|
||||
description: "Review plan document"
|
||||
prompt: |
|
||||
You are a plan document reviewer. Verify this plan is complete and ready for implementation.
|
||||
|
||||
**Plan to review:** [PLAN_FILE_PATH]
|
||||
**Spec for reference:** [SPEC_FILE_PATH]
|
||||
|
||||
## What to Check
|
||||
|
||||
| Category | What to Look For |
|
||||
|----------|------------------|
|
||||
| Completeness | TODOs, placeholders, incomplete tasks, missing steps |
|
||||
| Spec Alignment | Plan covers spec requirements, no major scope creep |
|
||||
| Task Decomposition | Tasks have clear boundaries, steps are actionable |
|
||||
| Buildability | Could an engineer follow this plan without getting stuck? |
|
||||
|
||||
## Calibration
|
||||
|
||||
**Only flag issues that would cause real problems during implementation.**
|
||||
An implementer building the wrong thing or getting stuck is an issue.
|
||||
Minor wording, stylistic preferences, and "nice to have" suggestions are not.
|
||||
|
||||
Approve unless there are serious gaps — missing requirements from the spec,
|
||||
contradictory steps, placeholder content, or tasks so vague they can't be acted on.
|
||||
|
||||
## Output Format
|
||||
|
||||
## Plan Review
|
||||
|
||||
**Status:** Approved | Issues Found
|
||||
|
||||
**Issues (if any):**
|
||||
- [Task X, Step Y]: [specific issue] - [why it matters for implementation]
|
||||
|
||||
**Recommendations (advisory, do not block approval):**
|
||||
- [suggestions for improvement]
|
||||
```
|
||||
|
||||
**Reviewer returns:** Status, Issues (if any), Recommendations
|
||||
Loading…
Reference in New Issue
Block a user