references/component-contract-audit.md

# Biohub ESM component contract audit

Audited 2026-07-13 and reconciled with the official Biohub tutorial workflows on 2026-07-14. This inventory covers the complete plugin, including components that do not call a Biohub HTTP endpoint. The detailed eighteen-operation HTTP matrix is in [`api-contract-audit.md`](api-contract-audit.md).

## Runtime and workflow components

| Component | Authoritative contract | Plugin boundary checked |
| --- | --- | --- |
| Plugin manifest and activation | Codex plugin manifest/skill conventions and the packaged starter prompts | Manifest paths, Scientific Research categorization, read/write/interactive capabilities, icon assets, skill discovery, natural-language activation, and absence of sequence literals in marketplace prompts |
| Router | Biohub model capabilities plus explicit plugin planning heuristics | Scientific task-to-provider routing, ESMC SAE extraction versus Atlas catalog lookup, 32-item scale-out heuristic labeling, and separation of ESMC, ESMFold2, Atlas, binder, Modal, and self-hosted work |
| Credential setup and preflight | Biohub Bearer authentication, Modal environment/profile resolution, Hugging Face public access, and Codex secure-secret behavior | Status-only output, environment/Keychain precedence, no value disclosure or mutation, Atlas no-auth status, and preservation of existing credentials |
| Managed HTTP transport | All nine current Biohub managed request pages | HTTPS origin pinning, `/api/v1/{operation}` path allowlist, Bearer header, JSON request/response mode, response-size/depth limits, timeouts, and no implicit retry of indeterminate inference |
| Managed ESMC validation | Current `encode` and `logits` request schemas plus pinned `Biohub/esm` tokenizer behavior | Track/token envelopes, mask handling, model IDs, output selection, hidden-layer bounds, SAE IDs, concern flag, and exact mutation request construction |
| ESMC mutation analysis | Pinned tokenizer and the documented masked-context log-likelihood-ratio definition | One masked residue, one-based biological numbering versus zero-based tensor index, natural-log score derivation, raw log probabilities, exact one-request behavior, and non-predictive interpretation |
| Tutorial-informed ESMC analysis | Official mutation-landscape and SAE-interpretation notebooks at the pinned source revision | Per-position masked contexts, entropy bits, 19 genuine substitutions, exact SAE dictionary binding, BOS/EOS removal, ranking semantics, and alignment-aware structure mapping |
| Managed ESMFold2 validation | Current `fold` and `fold_all_atom` request schemas, pinned SDK input types, and the official all-atom/MSA tutorial | Sequence/MSA and all-atom alternatives, hosted parameter bounds, model-specific Fast behavior, paired-MSA header preservation, nullable entity IDs, modifications, ligands, conditioning references, and requested output flags |
| Managed response materialization | Pinned `Biohub/esm` response objects and captured managed responses; the public request pages publish no response schema | Optional `data` envelope, confidence shapes/scales, PDB conversion for `/fold`, mmCIF conversion for `/fold_all_atom`, exact SDK revision verification, and incomplete provenance on conversion/schema failure |
| Atlas JSON and image client | Complete Atlas reference, OpenAPI, concepts, FAQ, changelog, Swagger page, and all worked examples | All nine published operations, request bounds/defaults, complete required response schemas, alpha-schema diagnostics, PNG/ZIP validation, and public/no-auth routing |
| Atlas durable batch lifecycle | Atlas batch prose plus OpenAPI job schemas | Synchronous versus asynchronous completion, atomic state, indeterminate submission handling, bounded polling, idempotent cancellation, expired jobs, signed-URL non-persistence, safe download hosts, resume integrity, and ZIP publication |
| Atlas anonymous bulk-data guidance | Biohub Atlas get-started and dataset documentation | Public S3/no-key boundary, attribution, resumability, and separation from managed inference |
| Generic Modal jobs | Current Modal Python SDK behavior used by the adapter | Function lookup, spawn/call-ID persistence, poll/gather/cancel/resume state transitions, partial failures, deadlines, and credential preflight without token disclosure |
| ESMFold2 Modal smoke | Official Modal ESMFold2 example plus pinned Biohub source/model revisions | Bounded one-input integration smoke, explicit GPU/function contract, immutable source and model revisions, and smoke-versus-scientific-validation labeling |
| Binder-design smoke and campaign | Official Modal binder-design example at the pinned commit | Reviewed source checksums, all checkpoint pins, one-seed/batch-one smoke bound, durable call IDs, campaign scale guidance, critic/selection provenance, and no managed-API routing claim |
| Hugging Face/self-hosted guidance | Pinned Biohub model cards and pinned `Biohub/esm`/Transformers source | Exact revisions, direct-VCS installation, Transformers override, installed-revision verification, public-versus-optional Hub token behavior, and separation of hosted versus local parameter contracts |
| Starter contracts | Packaged starter JSON, pinned RCSB 1PGA identifiers/sequence, and per-workflow execution validators | Thin natural prompts, internal sequence/digest binding, exact route and request, one primary provider call with at most one conditional Atlas presentation read, required artifacts, presentation action, and duplicate source/JSON synchronization |
| Provenance and diagnostics | Plugin provenance schema, source attribution, redaction policy, and provider response limits | Input/model/revision binding, checksums, timestamps, provider-call outcome, confidence metadata, atomic publication, complete recoverable raw JSON, bounded human-readable diagnostics, and secret-safe failure messages |
| Structure/image presentation | Codex Molecular Structure Viewer host contract and packaged viewer handoff | Automatic structure open after a successful fold, primary-object verification, B-factor/pLDDT coloring request, Atlas structure-or-thumbnail fallback, inline SVG/image display, and separation of viewer failure from inference success |
| Packaging, installation, and release | Codex plugin authoring validator and exact marketplace source/cache layout | Self-contained installed files, source/cache distinction, validation from source and installed target, exact PR-head mirroring, and preservation of locally configured secrets during replacement |

## Disposition and verification

The managed field audit covers every top-level and nested request field rendered for all nine managed operations, including the five intentionally unexposed ESM3 operations. The Atlas audit covers every published operation, response family, worked-example behavior, OpenAPI discrepancy, and synchronous/asynchronous batch branch. The table below records the final disposition of every plugin component; a residual is a declared host, upstream, evidence, or scope boundary, not an unrecorded API guess.

| Component | Audit result and concrete fixes | Verification evidence | Residual or source limitation |
| --- | --- | --- | --- |
| Plugin manifest and activation | **Pass with host qualification pending.** The manifest uses the Scientific Research category and declares interactive/read/write capabilities for its actual file/network behavior, exposes one outcome-rich prompt from each pinned official tutorial with no sequence literal, permits implicit invocation only through the router, and embeds the official supplied Biohub PNG byte-for-byte in the declared icon with the Biohub brand color. | `test_manifest.py`, `test_activation.py`, `test_tutorial_use_cases.py`, the plugin authoring validator, and the embedded PNG SHA-256 check. | The deterministic classifier is a regression fixture, not the Codex host router. A clean installed-host activation trace and an actual composer-chip render remain release evidence. |
| Router | **Pass.** ESMC, ESMFold2, Atlas, binder, Modal, and self-hosted routes are separated; SAE activation extraction routes to ESMC while the existing feature catalog routes to Atlas; Fast versus full/MSA behavior is explicit; the 32-item scale-out threshold is labeled as plugin policy rather than a provider quota. | `test_routing.py`, `test_activation.py`, `test_tutorial_use_cases.py`, `skills/biohub-esm/references/routing.md`, and starter route-contract tests. | Modal execution requires a user-owned deployed Function, and self-hosted execution requires user-owned compute; the plugin supplies routing/control contracts, not those deployments. |
| Credential setup and preflight | **Pass with a host boundary.** Environment precedence, read-only macOS Keychain lookup, status-only output, one-shot setup recovery, credential preservation, Atlas no-auth, and optional Hugging Face auth are enforced. | `test_security_validation.py`, `test_managed_preflight.py`, and `biohub_esm.py preflight`. | The host owns any masked credential sheet. `.modal.toml` presence proves local profile configuration only, not live authentication; the first authenticated Modal SDK operation is authoritative. |
| Managed HTTP transport | **Pass.** The client pins the Biohub origin and four exposed paths, sends Bearer JSON, applies time/byte/JSON budgets, performs no implicit managed retry, preserves retry metadata, and rejects redirects without following them or attempting to consume an unreliable redirect body. | `test_http_contracts.py`, `test_diagnostics.py`, and managed CLI failure tests. | The public managed pages publish no success schema, error schema, or status table. Provider errors therefore retain bounded redacted messages rather than claiming a public error-object contract. Generic local JSON input remains caller-trusted before semantic validation and is not subject to the provider-response byte budget. |
| Managed ESMC validation | **Pass.** `encode` and `logits` use the documented `inputs.sequence` envelopes; ESMC model IDs, token envelopes, output flags, hidden-layer ranges, all five SAE IDs, concern flag, JSON mode, and the 300M SAE normalization discrepancy are validated. | `test_managed_preflight.py`, `test_managed.py`, `test_managed_cli.py`, and [`managed-api-field-audit.md`](managed-api-field-audit.md). | Token IDs and model-layer bounds are pinned-SDK/tokenizer facts because the public request page omits them. ESM3 tracks and binary return mode are intentionally not exposed. |
| ESMC mutation analysis | **Pass.** The workflow verifies the stated wild type, separates one-based residue numbering from the BOS-offset tensor index, masks exactly one token, makes one no-retry logits request, computes full-vocabulary natural-log log-softmax and alternate-minus-wild-type LLR, and retains raw probabilities plus SVG/JSON provenance. | `test_esmc_mutation.py`, mutation sections of `test_managed_cli.py`, and the starter execution contract. | The score is not a fitness, stability, activity, binding, or function prediction. Inline SVG display is host behavior, and sequence-file reads are not independently byte-capped before sequence-length validation. |
| Tutorial-informed ESMC analysis | **Pass at the workflow-contract layer.** Explicit tutorial reproduction pins the notebook revision, one masked context per position, separately named full-vocabulary entropy in bits, 19 genuine substitution alternates, canonical `SAEConfig.models`, exact 6B/16K Atlas dictionary compatibility, BOS/EOS removal, ranking semantics, and alignment-aware structure mapping. The complete 259-context landscape runs on the managed API as concurrent bounded calls through the host-pinned client, matching the quickstart's documented pattern; the plugin ships no ESMC Modal function, so routing it to Modal pointed at a deployment the user would have to build. | `test_tutorial_use_cases.py`, `test_activation.py`, `examples/tutorial-use-cases.json`, and `skills/esmc/references/analysis.md`. | The dedicated CLI remains single-substitution only, and generic managed JSON preserves rather than decodes SAE tensors. Full landscapes and decoded SAE visualization use the pinned SDK route until a separately tested artifact helper is added. |
| Managed ESMFold2 validation | **Pass.** Both `/fold` and `/fold_all_atom` validate the full published sequence/MSA and all-atom alternatives, folding controls, output flags, nullable managed IDs, modifications, ligands, bonds, pockets, and distogram conditioning. Pinned-SDK MSA headers are preserved only for all-atom per-chain MSAs where official paired-MSA semantics apply; top-level single-chain requests remove them. The RNA null-MSA compatibility field is also removed, and Fast rejects every MSA. | `test_managed_preflight.py`, `test_security_validation.py`, `test_tutorial_use_cases.py`, `test_managed.py`, `test_confidence_contract.py`, and the complete fold sections of [`managed-api-field-audit.md`](managed-api-field-audit.md). | The public field tables omit MSA headers and response schemas, while the exact SDK serializer and official paired-MSA tutorial include them. Other recorded discrepancies cover `lm_mask_pct`, pair-chain iPTM, nullable IDs, and distogram children. |
| Managed response materialization | **Pass with an output-mode boundary.** Optional `data` envelopes, requested outputs, confidence shapes/scales, atom37 coordinates, and the pinned `MolecularComplex` JSON state are validated in every output mode. Every biological `/fold` residue requires finite N, CA, and C coordinates; only an explicit `|` position on a full sequence track may be an all-null row. Pinned-SDK PDB/mmCIF conversion, raw-first recovery, incomplete provenance, and `result.json` for all exposed endpoints are implemented. | `test_managed.py`, `test_confidence_contract.py`, `test_managed_cli.py`, and `test_diagnostics.py`. | Full raw/result/provenance/structure materialization requires `--output-dir`; stdout-only mode still validates the same provider-independent structure schema before emitting normalized JSON. Public managed response fields remain unpublished, so serializers are tied to the verified ESM SDK revision. |
| Atlas JSON and image client | **Pass; original defect fixed.** All nine published operations are implemented. `sequence_length` is canonical, complete decoded alpha responses are retained before nested projection validation, protein feature results are bound to requested in-catalog indices/order or the top-K ceiling, ranked results are not discarded on one malformed child, and PNG/ZIP media are validated. Protein lookup sends `fold_on_miss=false` by default; an explicit opt-in performs a non-folding lookup first and stops unless Atlas returns the actual MD5-bound sequence in the supported alphabet at no more than 699 residues. Length metadata alone is insufficient. | `test_http_contracts.py`, `test_cli.py`, `test_scientific_correctness.py`, and [`api-contract-audit.md`](api-contract-audit.md). | Atlas is alpha and may drift again. The 800-residue similarity limit is an explicitly conservative plugin policy versus the OpenAPI 2,048 limit; the discrepancy is preserved in documentation. |
| Atlas durable batch lifecycle | **Pass.** Synchronous ZIP and asynchronous JSON responses, pre-call state, locks, submission ambiguity, Retry-After, bounded polling, idempotent cancellation, terminal/expired jobs, signed-URL non-persistence, safe hosts, ranged resume, checksums, state recovery, and atomic publication are covered. | `test_cli_batch_reliability.py`, Atlas batch cases in `test_cli.py` and `test_http_contracts.py`. | Live provider retention and signed-URL lifetime are external. The local contract reacquires URLs and refuses blind resubmission when acceptance is indeterminate. |
| Atlas anonymous bulk-data guidance | **Pass.** Anonymous `s3://esm-protein-atlas/v1/`, CC BY 4.0 attribution, resumability, and its separation from managed inference are explicit. | `skills/esm-atlas/references/bulk-data.md`, source-citation tests, and routing tests. | This is operational guidance, not a bundled S3 mirroring client; transfer tooling and storage capacity remain user-owned. |
| Generic Modal jobs | **Pass; reliability gaps fixed.** The adapter pins Modal SDK 1.5.2, the credential-bound non-secret workspace, the explicit environment, and a deployed Function version in schema 1.5 state. Before each spawn/gather/cancel action it resolves `Workspace.from_context().hydrate()`, requires the verified name to match the reviewed workspace, and then compares the complete saved identity before resolving or acting on a call ID. The store applies byte, depth, node, aggregate-text, individual-token, and numeric-token limits before decoding; then it fully traverses and schema-validates state. It records indeterminate spawn, supports partial gather/cancel/resume, distinguishes retryable Modal control-plane failures from terminal deserialized user-code exceptions, and caps one shard at 128 jobs, 16 MiB input, 512 KiB result, 32 result levels, 50,000 result nodes, and 72 MiB state. | The focused Modal job suite, Modal parser/input-bound tests in `test_cli.py`, an isolated official 1.5.2-wheel signature/exception-hierarchy check, and `skills/esmfold2-binder-design/references/modal.md`. | `.modal.toml` is configuration evidence, not an auth probe, and profile/token contents never enter state. Returned artifact records are provider declarations until bytes are separately retrieved and checksum-verified; the plugin now says this explicitly. |
| ESMFold2 Modal smoke | **Pass as an integration smoke.** The harness fail-closes unless the local Modal CLI and imported SDK are exactly 1.5.2, records that version, pins ESM, Transformers, model revision, GPU/function parameters, one input, deterministic settings, and mmCIF/metric output. Its hidden execution token is an internal defense-in-depth check, never user-facing spend authorization; paid execution still requires separate explicit current-turn confirmation. Managed SDK live smokes also force one wire attempt per recorded call, and every smoke atomically claims a nonexistent evidence directory. | The focused live-smoke and scientific-correctness suites plus the pinned smoke script. | It is development-only, requires external Modal/GPU access and payment readiness, and is not scientific validation or the deployed Function used by every user. Authentication, payment, current pricing, and explicit spend approval remain external gates. |
| Binder-design smoke and campaign | **Pass within experimental scope.** Official example source/checksums, ESM and five checkpoint revisions, one-seed/batch-one smoke bounds, durable Modal identity, campaign metrics, critic provenance, and wet-lab boundaries are explicit. | `test_binder_smoke.py`, `test_modal_jobs.py`, and binder campaign/reference documents. | The checkpoints/workflow are experimental. The plugin plans and controls a user-owned campaign; it does not expose a managed binder endpoint or independently verify remote artifact bytes. |
| Hugging Face/self-hosted guidance | **Pass as source/model-pinned guidance.** Direct-VCS ESM and forced Transformers fork pins, all model commit pins, installed PEP 610 revision verification, public-weight token behavior, and hosted-versus-local parameter differences are recorded. | `test_scientific_correctness.py`, provenance pin tests, `verify-install`, and `references/source-pins.md`. | The plugin does not bundle weights, a general local inference runtime, or a complete dependency lock. Broad transitive dependency ranges plus Python, accelerator, driver, and hardware state remain user-owned/live-smoke concerns, so source/model pinning is not presented as bitwise environment reproducibility. |
| Starter contracts | **Pass structurally; clean-host execution gate pending.** Three focused GB1 regression prompts bind exact route/request/call count/artifacts/presentation without sequence literals. The ESMC, ESMFold2, and Atlas no-provider qualification plans are independently oracle-validated exact-line arrays in the shipped starter contract; qualification must read those bytes from the installed tree before literal transcription. Three separate page-facing defaults are owned by the tutorial contract and disclose their pinned target/construct, route, item and call counts, artifacts, and available cost before separate explicit current-turn confirmation. Selecting any prompt is not provider authorization. The activation fixture lives in the shipped `examples/` surface, and a tests-omitted runtime-bundle regression validates both prompt families. | `validate_starter_examples.py`, `test_starter_examples.py`, `test_starter_execution_contracts.py`, `test_tutorial_use_cases.py`, `test_clean_install_e2e.py`, `test_combined_demo.py`, and `test_package_combined_evidence.py`. | `qualificationStatus` intentionally remains `pending-clean-host-qualification`. The current isolated qualification harness is read-only, network-disabled, and inherits no scientific credentials, so its eventual evidence can prove installation, activation, and planning behavior rather than provider execution or live presentation. |
| Provenance and diagnostics | **Pass; publication regressions fixed.** Input/model/revision binding, redaction, checksums, confidence metadata, provider-call outcomes, bounded diagnostics, complete decoded Atlas evidence, and incomplete failure state are covered. Exact pretty JSON serialization now finishes in an anonymous spooled file before a named temp; Linux `renameat2` and macOS `renamex_np(RENAME_EXCL)` consume no-replace temps atomically without unsafe path cleanup. | `test_provenance.py`, `test_diagnostics.py`, `test_managed_cli.py`, plus batch reliability tests. | Schema validation of a remote artifact record is not byte verification. On filesystems lacking either native no-replace rename, the hardlink fallback deliberately preserves its source residue rather than risk unlinking a raced replacement. |
| Structure/image presentation | **Pass at the plugin handoff layer.** The viewer contract maps returned `viewerSessionId` to control-call `sessionId`, verifies the primary object, requests B-factor pLDDT coloring, reconciles mutation timeout with state, and treats viewer failure separately. Combined-demo mapping uses ATOM C-alpha records, preserves residue/chain identity, ignores HETATM calcium, disables hidden SDK retries, and avoids novelty overclaims. | `test_combined_demo.py`, `test_package_combined_evidence.py`, viewer handoff tests, and the Playwright viewer harness. | Viewer/image display is best-effort host integration. A clean installed-host trace for the exact Fast starter opening `prediction.pdb` and applying B-factor coloring is still required; open success alone does not prove the later color mutation. |
| Packaging, installation, and release | **Pass in source; release handoff verification required.** Source layout, manifest, marketplace entry, icon, skills, examples, and references pass the authoring validator; source tests and starter validator are green. | The source suite, starter validator, plugin validator, Ruff check/format, and diff checks are the required exact-head evidence; record their current results rather than freezing totals in this contract. | Installation identity is an external release-state fact, not a source contract. The release handoff must record commit/push, exact PR-head reinstall without touching credentials, source/cache comparison, installed-target validation, and the remaining clean-host presentation qualification. Version `0.2.4` is the patch increment for the reviewed first-run, starter-execution, managed-fold recovery, and renderer-neutral handoff fixes so version-aware marketplace and local installs can detect the corrected bytes; internal deployment separately requires a distribution entry and merge. |

Environment-dependent skips may include pinned-ESM-SDK materialization checks when that optional SDK is absent and the platform-dependent `RLIMIT_AS` resource-limit check. They are not API-contract failures; exact skip details belong in the current test evidence rather than this source contract. The exact SDK path is separately revision-pinned and covered by install/live-smoke gates.

## Evidence levels

The audit keeps these evidence classes separate:

- **Public API documented:** request fields, defaults, bounds, authentication, and Atlas response schemas that appear in the current public references.
- **Official SDK or pinned source derived:** managed response objects, local model builders, tokenizer behavior, Modal example code, and artifact serializers not specified by the managed request pages.
- **Captured response tested:** provider JSON details and confidence shapes seen in checked-in fixtures or a fresh public Atlas probe.
- **Plugin policy:** explicit model selection, conservative search length, exactly-one-request starters, Fast/no-MSA rejection, atomic output directories, and routing thresholds. These are labeled as plugin constraints rather than provider limits.
- **Host integration:** secure credential collection and structure-viewer actions. The plugin can request these behaviors but does not implement the host UI itself.

## Intentional scope exclusions

The plugin does not expose managed `decode`, `generate`, `generate_tensor`, `forward_and_sample`, or `inverse_fold`; those operations are ESM3-oriented and outside this ESMC/ESMFold2 plugin surface. Listing and auditing their current contracts does not imply they are implemented. Atlas, by contrast, has client coverage for every published operation.

The plugin also does not ship a hosted MCP service, a custom molecular viewer, an untrained predictive head, or a managed binder-design endpoint. Modal and Hugging Face paths execute pinned public-weight workflows using their own authentication and compute boundaries.