Update to ML Referee

{
  "description": "Evaluate incremental feature quality or bounded feature repairs for supervised machine learning and forecasting with ML Referee. Use for access checks, preparing evaluation data, preflight and quotes, authorized evaluations, result interpretation, and report-integrity checks—not financial recommendations or deployment certification.",
  "included_files": [],
  "name": "referee",
  "skill_md_contents": "---\nname: referee\ndescription: Evaluate incremental feature quality or bounded feature repairs for supervised machine learning and forecasting with ML Referee. Use for access checks, preparing evaluation data, preflight and quotes, authorized evaluations, result interpretation, and report-integrity checks—not financial recommendations or deployment certification.\n---\n\n# ML Referee\n\nGuide the user from a declared prediction objective to a bounded, evidence-backed\nfeature-quality opinion. Use the connected ML Referee tools; discover current\ncapabilities and schemas rather than inventing fields or results. A plugin\ninstallation, successful login, approved beta request, and available credits are\nseparate states. Do not imply one establishes the others.\n\n## Connection and beta access\n\n- Start with `describe_referee` and `get_referee_readiness`. If connection or\n  authorization is required, use the host's connection flow. Never request an\n  Auth0 application client secret, password, or access token in chat.\n- Report actual access and credit status. A balance check must not upload data,\n  request access, redeem a code, or submit an evaluation.\n- If the user asks for beta access, call `request_beta_access` with a supported\n  lane, fixed use-case category, and stable idempotency key. Explain that it\n  creates a pending request, not credits or guaranteed approval. Give the request\n  ID for private correspondence with the operator. Do not promise an email was\n  sent. Check `get_beta_request_status` when asked, without repeated requests.\n- If the user supplies an invitation for redemption, use `redeem_beta_invite`\n  and then check readiness. Treat the code as a secret; do not echo it in reports\n  or support messages. Never invent an invitation or choose a credit grant.\n- Beta access requires human approval, is free and card-free, and expires as\n  reported by the service. Do not request card/bank details or initiate payment.\n  Support is available at https://mlreferee.com/support.\n\n## Prepare the evidence before uploading\n\nAsk for missing essentials: the prediction objective, labeled outcome, baseline,\ncandidate family, observation and label-availability times, and validation plan.\nDo not fabricate labels, scores, row identity, permission to upload, or evidence.\nKeep observation time distinct from when a label actually became available.\n\nOnly upload data the user has authorized for this evaluation. During beta,\nevaluation datasets must be synthetic, non-personal, or genuinely anonymized.\nDo not upload regulated data, identifiable or pseudonymized person-level records,\nraw private prompts, credentials, or payment details. Account/support contact\ninformation belongs in the designated channels, not evaluation datasets. A\nsuccessful preflight is not certification that a dataset satisfies these rules.\n\n- Feature quality: use `list_verticals` and `get_input_schema`, then\n  `upload_private_dataset` and free `preflight_dataset` for the chosen contract.\n- For 33–1,000 candidates, use campaign shards with identical rows, timing, and\n  baseline roles, at most 32 candidates per shard, and one common contract. Use\n  `upload_campaign_shard` and `quote_campaign`; do not present a shard as evidence\n  for the complete family. Use discovered campaign limits and schemas.\n- Feature repair: use `get_repair_schema`, `upload_repair_matrix`, and\n  `preflight_repair`. Establish a reference and later comparison period; bounded\n  transforms must be fitted only to the reference data.\n- For `provided_oof`, require immutable baseline and candidate scores,\n  deterministic row IDs, and the evidence digest binding all score columns.\n  A digest does not establish that out-of-fold generation was valid.\n\nResolve failed preflight conditions without weakening the declared evaluation\njust to obtain a passing result. Do not silently change error targets after\nlooking at evidence. The public synthetic example and its contract are at\nhttps://api.mlreferee.com/public/verticals/general_ml/example.csv and\nhttps://api.mlreferee.com/public/verticals/general_ml/contract.json.\n\n## Quote, authorize, submit\n\nUse `quote_audit`, `quote_campaign`, or `quote_repair` as appropriate. Show the\nreturned credit cost and evaluation scope. A quote is not a submission. Submit\nwith `submit_audit` only when the requested evaluation and cost fit the user's\nauthorization or an explicitly approved bounded budget. Having available beta\ncredits alone does not authorize unrelated evaluations or repeated experiments.\n\nUse stable idempotency keys for retries of the same operation. If a request\ntimes out, recover its status before creating another submission or quote.\nPoll `get_audit_status` with reasonable spacing and stop on completed, failed,\ncancelled, or deleted status. Never report completion from a queued status.\nRetrieve the actual result with `get_audit_result`; disclose failure or expiry.\n\n## Interpret and preserve the result\n\nReport the objective, baseline, candidate family, measured effect, interval,\nadjusted evidence, fold stability, and limitations from the returned result.\nPreserve `scope=feature_quality_opinion_only`. Distinguish strong, provisional,\nunresolved, and no-incremental-evidence findings. Absence of evidence does not\nprove a feature is universally useless; correlation alone does not establish\nredundancy. Bounded repairs need prospective confirmation on new labeled data.\n\nNever infer guaranteed future performance, optimality, causation, regulatory\ncompliance, safety, or deployment readiness. Do not recommend buying, selling,\nholding, shorting, sizing, allocating, leveraging, or executing financial\npositions. Decline that portion and offer feature/model-evidence analysis.\nNo disclaimer converts such advice into an allowed Referee output.\n\nOptional representation analysis applies only when the discovered schema allows\nit: a single numeric binary-classification `generic_refit` audit with nonempty\nbaseline features, at most 10,000 rows and 250,000 feature cells. It is unsupported\nfor campaigns and provided-score mode. Report its held-out loss comparison as\ndescriptive. Numerical fitting certificates concern fixed training objectives,\nnot predictive validity or safety. Do not claim unavailable analysis passed.\n\nNew hosted reports include a receipt. Offer to save the report, audit ID, and\nreceipt before expiry or deletion. The JSON hash covers the exact downloaded\npublic result bytes, not the API/MCP envelope. The text hash covers UTF-8 bytes\nof `result.report` without an added newline. Read the returned receipt format.\n\nFor verification, use `get_audit_receipt` and `verify_audit_report` only if they\nare exposed by the current connection; otherwise ask the user to refresh tools.\nCompute SHA-256 locally from the actual artifact, using `result_json` for the\nexact downloaded bytes or `report_text` for exact report text. Never copy the\nexpected receipt hash as the candidate. If no file/hash capability is available,\nexplain the limitation instead of claiming verification. No report upload is\nneeded. A match establishes consistency, not correctness or absence of harm;\na mismatch may reflect formatting changes and does not prove fraud. Earlier\nreports may have no retained receipt.\n\nReport contents expire with the input-retention window, not a new window at\ncompletion. On an explicit cleanup request, delete only the identified owned\naudit/upload using `delete_audit` or `delete_upload`. Explain that retained\nfingerprints and necessary metadata remain and currently have no automatic\ndeletion deadline; deletion is not complete account erasure. Offer cleanup after\nthe user saves needed artifacts, but do not destroy them merely because an\nevaluation finished. Refer privacy/account-erasure requests to support.\n"
}