← NeoSigmaCONTENT HISTORY

Update to NeoSigma

Snapshot Nov 17, 2026 · 00:13 UTC · version 1.7.3

Collection source: downloaded plugin package.

WHAT CHANGED

First saved snapshot

No earlier snapshot is available to establish a change.

Save NeoSigma and check future changes in Following.

Up to 3 plugins free · no card required.

Compare saved observations

Download comparison JSON
Full technical diff · 0 changed fields
Full snapshot data
{
  "description": "Imports a codebase's existing quality checks into NeoSigma as engine-backed verifier YAML. Activates on requests like \"import our verifiers into NeoSigma\", \"set up verifiers from my codebase\", \"add our evals to NeoSigma\", \"onboard our quality checks to NeoSigma\", or \"turn our eval suite into NeoSigma verifiers\". Requires the NeoSigma MCP to be connected. Do NOT use for analyzing traces already in NeoSigma or for integrating the tracing SDK (use integrate-sdk).",
  "included_files": [],
  "name": "import-verifiers",
  "skill_md_contents": "---\nname: import-verifiers\ndescription: Imports a codebase's existing quality checks into NeoSigma as engine-backed verifier YAML. Activates on requests like \"import our verifiers into NeoSigma\", \"set up verifiers from my codebase\", \"add our evals to NeoSigma\", \"onboard our quality checks to NeoSigma\", or \"turn our eval suite into NeoSigma verifiers\". Requires the NeoSigma MCP to be connected. Do NOT use for analyzing traces already in NeoSigma or for integrating the tracing SDK (use integrate-sdk).\n---\n\n# Import verifiers into NeoSigma\n\nFind checks that evaluate an agent's output, translate the trace-evaluable ones\ninto NeoSigma's repository authoring format, and apply the repository's complete\nverifier set with the `sync_verifiers` MCP tool.\n\nEach imported verifier is a YAML file under `.neosigma/verifiers/`. Its filename\nis its stable slug. The file contains a self-contained prompt evaluated against\none production trace, with no access to source code or runtime state.\n\n## Core principles\n\n1. **Import only trace-evaluable checks.** A verifier sees one production trace\n   plus its prompt. Skip exact-value assertions, latency thresholds, status\n   codes, internal variables, and external state absent from the trace.\n2. **Translate intent, never code.** State what to check and what makes the trace\n   pass or fail. Do not reference source files, functions, variables, or\n   implementation details.\n3. **Write one condition per file.** Use one short slug and one self-contained\n   prompt. Do not embed numeric scales, rubric levels, aggregation, or weights.\n4. **Preserve the complete repository-owned set.** Read existing\n   `.neosigma/config.yaml` and `.neosigma/verifiers/*.yaml` files first. Keep\n   existing files unless the user asked to remove them. A sync disables a file\n   previously synced from this repository when that file is omitted.\n5. **Keep provenance truthful and work durable.** Call `sync_verifiers` only\n   after the generated files exist in the commit passed as `sourceCommit`.\n   Before writing files, obtain explicit authorization to commit them in the\n   same turn. If the user does not authorize a commit, stop before writing and\n   ask for authorization; uncommitted clone changes do not survive turns.\n6. **Keep the complete set within 200 files.** The sync API cannot apply a set\n   with more than 200 verifier files, and splitting a set across calls disables\n   omitted files. Stop before writing when the existing files plus the selected\n   imports would exceed 200, and report that the repository set is unsupported.\n\n## Authoring format\n\nCreate `.neosigma/config.yaml` when it does not exist:\n\n```yaml\nscope: org\ndefaults:\n  scope: turn\n  timeout_seconds: 120\n  output:\n    type: assertion\n    pass_is_good: true\n```\n\nWrite each verifier as `.neosigma/verifiers/<slug>.yaml`:\n\n```yaml\nprompt: |\n  Check whether the response answers the user's request.\n  Pass when it directly resolves the request with relevant information.\n  Fail when it avoids, misunderstands, or leaves the request unresolved.\n```\n\nUse lowercase kebab-case slugs. Do not add `version`, `model`, or a top-level\nslug to an individual verifier file. Shared defaults belong in\n`.neosigma/config.yaml`.\n\n## Workflow\n\n1. **Inspect the existing set.** Read `.neosigma/config.yaml` and every file\n   directly under `.neosigma/verifiers/`. Record the existing slugs so the\n   import does not overwrite or omit them.\n2. **Find candidate checks.** Search for model-graded evals, scorers, graders,\n   rubric prompts, and qualitative assertions. Common locations include\n   `evals/`, `evaluations/`, and `tests/`, plus files named `*eval*`,\n   `*scorer*`, or `*grader*`.\n3. **Classify each check.** Import it when one trace contains enough evidence to\n   evaluate its intent. Otherwise skip it and record a one-line reason.\n4. **Preflight the set size.** Count the existing files plus the selected new\n   verifier files. If the complete set would exceed 200 files, stop before\n   writing and report the unsupported case. Never split the set across calls.\n5. **Confirm commit authorization.** Before writing, confirm that the user has\n   explicitly authorized committing the generated files. If not, stop and ask\n   for authorization. Do not leave uncommitted generated files for a later turn.\n6. **Write the files.** Create or preserve the config, then add one verifier YAML\n   file per imported condition. Never delete an existing file merely because it\n   was not discovered during this scan.\n7. **Validate the set.** Confirm every path is directly under\n   `.neosigma/verifiers/`, every filename ends in `.yaml`, the YAML parses,\n   and every prompt states both pass and fail behavior.\n8. **Commit the files.** Use the repository's normal workflow in the same turn,\n   then resolve `sourceRepo` from the origin remote and `sourceCommit` from the\n   commit containing the generated files.\n9. **Apply once.** Call `sync_verifiers` with:\n   - `sourceRepo`: the repository's `owner/name`.\n   - `sourceCommit`: the 7-40 character commit SHA containing the files.\n   - `config`: the complete text of `.neosigma/config.yaml`.\n   - `files`: every `.neosigma/verifiers/*.yaml` file, with paths relative\n     to `.neosigma/`, such as `verifiers/answers-the-question.yaml`.\n10. **Report the result.** List imported and skipped checks, the sync counts, and\n   every returned authoring error. Do not report success unless the sync call\n   succeeds.\n\n## Output format\n\nReport:\n\n- Imported verifiers as `<slug>` plus a one-line description.\n- Skipped checks plus a one-line reason.\n- Any uncertain check that needs confirmation.\n- The `created`, `updated`, `unchanged`, and `disabled` counts returned\n  by `sync_verifiers`, or the exact authoring errors.\n\n## Examples\n\n**Trace-evaluable checks.** A support agent suite checks whether responses answer\nthe question, stay grounded in supplied context, remain concise, and avoid\nexposing personal information. Write four verifier files, preserve every\nexisting file, and validate the set. First obtain explicit commit authorization;\nif it is not granted, stop before writing. Once authorized, write, validate, and\ncommit the files in the same turn, then send the complete set in one\n`sync_verifiers` call.\n\n**Mechanical checks.** A test asserts an exact response ID, latency below 500\nmilliseconds, and an internal cache size. Skip all three because one trace\ncannot reproduce those conditions reliably.\n\n**External state.** A refund-policy check depends on manager approval. Skip it\nwhen approval is not visible in the trace. When the trace contains an explicit\napproval event, write the prompt against that visible evidence and flag the\ntranslation for review.\n"
}

SHA-256 of public snapshot: f6eb6a1f700db088529b5fa96737ab4dc8e3919e495abdf6f4e99427b3a3142c