147 lines
5.3 KiB
Markdown
147 lines
5.3 KiB
Markdown
---
|
|
name: ultraqa
|
|
description: "[OMX] QA cycling workflow - test, verify, fix, repeat until goal met"
|
|
---
|
|
|
|
# UltraQA Skill
|
|
|
|
[ULTRAQA ACTIVATED - AUTONOMOUS QA CYCLING]
|
|
|
|
## Overview
|
|
|
|
## GPT-5.4 Guidance Alignment
|
|
|
|
- Default to concise, evidence-dense progress and completion reporting unless the user or risk level requires more detail.
|
|
- Treat newer user task updates as local overrides for the active workflow branch while preserving earlier non-conflicting constraints.
|
|
- If correctness depends on additional inspection, retrieval, execution, or verification, keep using the relevant tools until the QA cycle is grounded.
|
|
- Continue through clear, low-risk, reversible next steps automatically; ask only when the next step is materially branching, destructive, or preference-dependent.
|
|
|
|
You are now in **ULTRAQA** mode - an autonomous QA cycling workflow that runs until your quality goal is met.
|
|
|
|
**Cycle**: qa-tester → architect verification → fix → repeat
|
|
|
|
## Goal Parsing
|
|
|
|
Parse the goal from arguments. Supported formats:
|
|
|
|
| Invocation | Goal Type | What to Check |
|
|
|------------|-----------|---------------|
|
|
| `/ultraqa --tests` | tests | All test suites pass |
|
|
| `/ultraqa --build` | build | Build succeeds with exit 0 |
|
|
| `/ultraqa --lint` | lint | No lint errors |
|
|
| `/ultraqa --typecheck` | typecheck | No TypeScript errors |
|
|
| `/ultraqa --custom "pattern"` | custom | Custom success pattern in output |
|
|
|
|
If no structured goal provided, interpret the argument as a custom goal.
|
|
|
|
## Cycle Workflow
|
|
|
|
### Cycle N (Max 5)
|
|
|
|
1. **RUN QA**: Execute verification based on goal type
|
|
- `--tests`: Run the project's test command
|
|
- `--build`: Run the project's build command
|
|
- `--lint`: Run the project's lint command
|
|
- `--typecheck`: Run the project's type check command
|
|
- `--custom`: Run appropriate command and check for pattern
|
|
- `--interactive`: Use qa-tester for interactive CLI/service testing:
|
|
```
|
|
delegate(role="qa-tester", tier="STANDARD", task="TEST:
|
|
Goal: [describe what to verify]
|
|
Service: [how to start]
|
|
Test cases: [specific scenarios to verify]")
|
|
```
|
|
|
|
2. **CHECK RESULT**: Did the goal pass?
|
|
- **YES** → Exit with success message
|
|
- **NO** → Continue to step 3
|
|
|
|
3. **ARCHITECT DIAGNOSIS**: Spawn architect to analyze failure
|
|
```
|
|
delegate(role="architect", tier="THOROUGH", task="DIAGNOSE FAILURE:
|
|
Goal: [goal type]
|
|
Output: [test/build output]
|
|
Provide root cause and specific fix recommendations.")
|
|
```
|
|
|
|
4. **FIX ISSUES**: Apply architect's recommendations
|
|
```
|
|
delegate(role="executor", tier="STANDARD", task="FIX:
|
|
Issue: [architect diagnosis]
|
|
Files: [affected files]
|
|
Apply the fix precisely as recommended.")
|
|
```
|
|
|
|
5. **REPEAT**: Go back to step 1
|
|
|
|
## Exit Conditions
|
|
|
|
| Condition | Action |
|
|
|-----------|--------|
|
|
| **Goal Met** | Exit with success: "ULTRAQA COMPLETE: Goal met after N cycles" |
|
|
| **Cycle 5 Reached** | Exit with diagnosis: "ULTRAQA STOPPED: Max cycles. Diagnosis: ..." |
|
|
| **Same Failure 3x** | Exit early: "ULTRAQA STOPPED: Same failure detected 3 times. Root cause: ..." |
|
|
| **Environment Error** | Exit: "ULTRAQA ERROR: [tmux/port/dependency issue]" |
|
|
|
|
## Observability
|
|
|
|
Output progress each cycle:
|
|
```
|
|
[ULTRAQA Cycle 1/5] Running tests...
|
|
[ULTRAQA Cycle 1/5] FAILED - 3 tests failing
|
|
[ULTRAQA Cycle 1/5] Architect diagnosing...
|
|
[ULTRAQA Cycle 1/5] Fixing: auth.test.ts - missing mock
|
|
[ULTRAQA Cycle 2/5] Running tests...
|
|
[ULTRAQA Cycle 2/5] PASSED - All 47 tests pass
|
|
[ULTRAQA COMPLETE] Goal met after 2 cycles
|
|
```
|
|
|
|
## State Tracking
|
|
|
|
Use `omx_state` MCP tools for UltraQA lifecycle state.
|
|
|
|
- **On start**:
|
|
`state_write({mode: "ultraqa", active: true, current_phase: "qa", iteration: 1, started_at: "<now>"})`
|
|
- **On each cycle**:
|
|
`state_write({mode: "ultraqa", current_phase: "qa", iteration: <cycle>})`
|
|
- **On diagnose/fix transitions**:
|
|
`state_write({mode: "ultraqa", current_phase: "diagnose"})`
|
|
`state_write({mode: "ultraqa", current_phase: "fix"})`
|
|
- **On completion**:
|
|
`state_write({mode: "ultraqa", active: false, current_phase: "complete", completed_at: "<now>"})`
|
|
- **For resume detection**:
|
|
`state_read({mode: "ultraqa"})`
|
|
|
|
|
|
## Scenario Examples
|
|
|
|
**Good:** The user says `continue` after the workflow already has a clear next step. Continue the current branch of work instead of restarting or re-asking the same question.
|
|
|
|
**Good:** The user changes only the output shape or downstream delivery step (for example `make a PR`). Preserve earlier non-conflicting workflow constraints and apply the update locally.
|
|
|
|
**Bad:** The user says `continue`, and the workflow restarts discovery or stops before the missing verification/evidence is gathered.
|
|
|
|
## Cancellation
|
|
|
|
User can cancel with `/cancel` which clears the state file.
|
|
|
|
## Important Rules
|
|
|
|
1. **PARALLEL when possible** - Run diagnosis while preparing potential fixes
|
|
2. **TRACK failures** - Record each failure to detect patterns
|
|
3. **EARLY EXIT on pattern** - 3x same failure = stop and surface
|
|
4. **CLEAR OUTPUT** - User should always know current cycle and status
|
|
5. **CLEAN UP** - Clear state file on completion or cancellation
|
|
|
|
## STATE CLEANUP ON COMPLETION
|
|
|
|
When goal is met OR max cycles reached OR exiting early, run `$cancel` or call:
|
|
|
|
`state_clear({mode: "ultraqa"})`
|
|
|
|
Use MCP state cleanup rather than deleting files directly.
|
|
|
|
---
|
|
|
|
Begin ULTRAQA cycling now. Parse the goal and start cycle 1.
|