skills/moos-ivp-eval-mission-builder/SKILL.md

---
name: moos-ivp-eval-mission-builder
description: "Build or repair one self-evaluating MOOS-IvP mission by adding an evaluation layer to an ordinary mission: headless startup, pMissionEval grading, results.txt, uMayFinish/xlaunch completion, zlaunch automation, scoped teardown, and single-scenario validation."
---

# MOOS-IvP Eval Mission Builder

## Overview

Use this skill for one self-evaluating mission folder: a normal MOOS-IvP mission
with an added single-run grading contract. The mission should still be readable
and runnable by a person, but it must also run headlessly, decide pass/fail
inside the mission, write `results.txt`, and finish through the shared
`xlaunch.sh` / `uMayFinish` path.

For ordinary mission layout, use `moos-ivp-mission-builder` first. For multi-case
matrices, patch sweeps, parallel runs, or expected-vs-actual aggregation, use
`moos-ivp-harness-builder`. For post-run `.alog` evidence, use
`moos-alog-analysis`.

## Core Rules

- Start from an ordinary mission that already launches cleanly. Prefer the
  `moos-ivp-mission-builder` baselines or an existing nearby mission family.
- Add only the evaluation plumbing needed for one scenario:
  optional `pAutoPoke`, optional `uTimerScript`, `pMissionEval`,
  `results.txt`, and a thin `zlaunch.sh`.
- Keep `launch.sh` human-facing. It may accept `--xlaunched`, `--nogui`, and
  port overrides, but it should not contain case loops or result aggregation.
- Keep `zlaunch.sh` thin: parse automation arguments, truncate `results.txt`,
  call shared `xlaunch.sh`, validate that `results.txt` contains `grade=`, then
  apply project-local scoped cleanup.
- Let `xlaunch.sh --max_time=<secs>` own `uMayFinish` and the timed wait/stop
  contract. Do not duplicate that lifecycle in mission-local wrappers.
- Do not synthesize `grade=` or write the final result row from `zlaunch.sh`,
  `launch.sh`, or target-file parsing. `pMissionEval` must own the verdict and
  write `results.txt`; wrappers may only truncate, launch, wait, validate
  presence of `grade=`, and clean up.
- For cleanup backstops, copy `assets/moos_scoped_teardown.sh` into the target
  project as `<project-root>/scripts/moos_scoped_teardown.sh` if it does not
  already exist. Reuse an existing project-root helper unless it is clearly
  stale or incompatible.
- Prefer `pAutoPoke` to seed deploy and evaluation variables in moving
  missions. Unit-style evals may use `uTimerScript` or the app under test for
  readiness when there is no vehicle/deploy lifecycle. Do not put pass/fail
  logic in `pAutoPoke`.
- Use `pMissionEval` as the primary verdict owner. Prefer mission-level booleans
  or simple scalar checks over harness-side parsing of raw MOOS traffic.
- Prefer event-driven `pMissionEval` leads: evaluate when the mission-owned
  completion event occurs. Use `uMayFinish` through `xlaunch.sh --max_time` as
  the outer infrastructure ceiling. Use a time-driven evaluation-window lead
  only when non-completion is an expected mission outcome that should produce
  mission-owned `grade=fail`.
- Multiple `lead_condition` lines in the same aspect are allowed, but they are
  ANDed: all must be true before pass/fail conditions are evaluated. A
  `lead_condition` after pass/fail conditions starts the next ordered aspect.
  A single `lead_condition` may use textual `or` when each operand is
  parenthesized, for example `(EVENT_A = true) or (EVENT_B = true)`. Do not use
  `||`; it is not a supported `LogicCondition` operator.
- Treat `BHV_ERROR_SEEN=false` as a normal safety/integrity pass condition.
  Treat `BHV_WARNING` as advisory development evidence by default: inspect and
  investigate it with appcasts or `.alog` tools, but do not add a sticky
  `BHV_WARNING_SEEN` mailflag, result column, or pass condition unless the
  scenario is explicitly warning-intolerant and the warning signal is known to be
  stable rather than transient/retracted.
- Keep `results.txt` scalar and parseable. The only hard schema requirement is
  `grade=<pass|fail>`; fields such as `form=`, `eval=`, `timeout=`, domain
  facts, and `mhash=` are recommended evidence, not a mandatory metric set.
- `mission_mod` is optional mission-owned provenance. Use it only when one
  mission folder intentionally supports multiple named standalone modes. Omit
  it from single-scenario eval missions, and do not use it to represent harness
  cases.
- If a vehicle-local variable is graded shoreside, bridge it explicitly through
  the vehicle broker and shoreside broker.
- For GUI-capable eval missions, keep normal operator buttons available. Do not
  force appcast/realmcast viewer modes unless the evaluation scenario needs it.
- Do not add `--case`, `--jobs`, temp mission copies, per-case port blocks, or
  expected-vs-actual aggregation here. Those belong to the harness builder.

## Workflow

1. Confirm the base mission launches and generates targets.
2. Identify the smallest mission-owned pass/fail signal.
   - unit-style app variable
   - behavior end flag
   - arrival/collision/encounter outcome
   - load/process/host info signal
3. Add evaluation state to the relevant `.bhv` or app config.
   - When adapting an ordinary waypoint mission, make the graded behavior
     finite, such as `repeat = 0`, or add an explicit completion flag. A
     repeating operator survey is usually not a valid eval completion signal.
4. Bridge graded vehicle-local variables to shoreside when needed.
5. Add `pAutoPoke` or an equivalent explicit initializer for deploy and
   evaluation variables.
6. Add `pMissionEval` with simple lead condition(s), clear pass conditions,
   `result_flag = MISSION_EVALUATED = true`, and `report_file = results.txt`.
7. Add or update `zlaunch.sh` to set a mission-appropriate `MAX_TIME` default,
   accept `--max_time=<secs>` as an override, and forward the final value to
   `xlaunch.sh --max_time=<secs>`.
8. Add or update `README.md` with scenario, grading signal, and run commands.
9. Validate target generation, then run the headless cycle and inspect
   `results.txt`.

## Reference Use

- Read `references/eval-mission-style.md` for boundaries and file layout.
- Read `references/evaluator-apps.md` before wiring `pAutoPoke` or
  `pMissionEval`.
- Read `references/scenario-and-grading.md` before grading obstacles, contacts,
  moving/integration outcomes, or structured payloads.
- Read `references/zlaunch-xlaunch.md` before editing automation wrappers.
- Read `references/validation.md` before reporting an eval mission as done.
- Copy `assets/eval-single-vehicle/` when a concrete minimal moving example is
  useful.
- Copy `assets/moos_scoped_teardown.sh` into the target project as
  `<project-root>/scripts/moos_scoped_teardown.sh` when the project does not
  already have an equivalent root-scoped helper.
- Run `scripts/static_check_eval_mission.sh <mission-dir>` for a quick
  structural check.
- Run `scripts/live_check_eval_mission.sh <mission-dir> --port_base=<free-base>`
  for bundled-example or high-trust validation when MOOS-IvP runtime tools are
  available.
- Treat live-check teardown failure as a test failure, show the teardown error,
  and preserve the temporary workdir for diagnosis.

## Validation Checklist

- `./launch.sh --just_make --nogui <warp>` succeeds.
- Generated targets contain `pMissionEval`, explicit initialization
  (`pAutoPoke`, `uTimerScript`, or an app-owned producer), and any evaluator
  apps needed for reported columns such as `pMissionHash`.
- If `pMissionHash` is used for `mhash=` evidence, keep it headless-only by
  default; GUI targets should not launch both `pMissionHash` and
  `pMarineViewer` unless the overlapping pMarineViewer hash feature is
  deliberately disabled.
- Generated targets include bridged graded variables if the verdict depends on
  vehicle-local posts.
- `./zlaunch.sh --just_make <warp>` succeeds when `xlaunch.sh` is on `PATH`.
- Headless `./zlaunch.sh --max_time=<secs> <warp>` exits cleanly.
- `results.txt` contains one parseable result line with `grade=`.
- Runtime warnings are investigated during validation; only stable,
  scenario-relevant warning metrics are surfaced in `results.txt`.
- High-trust checks use `scripts/live_check_eval_mission.sh` or equivalent to
  verify result rows, surface warning evidence, and detect leftover listeners on
  scoped ports.
- No mission wrapper uses global `ktm`, `pkill`, or unrelated cleanup.
- Eval wrappers use `<project-root>/scripts/moos_scoped_teardown.sh` as a scoped
  backstop after `xlaunch.sh`; they do not use global `ktm`, `pkill`, or broad
  process discovery.
- GUI runs retain normal operator controls unless the user requested a
  headless-only mission.