{"id":56114,"plugin_id":"plugins_6abf4e8b66808191af07692fd78a9110","kind":"skill","collection_source":"plugin_package","comparison_source":null,"observed_at":"2026-10-26T18:06:39.443Z","digest":"7f8c5325eea3b887c95b71ad4016c2802ba14ce10d62c8ad4d153f42baf8176b","against":null,"payload":{"description":"Migrate hardcoded values, hyperparameters, and configuration values in ML code to Valohai parameters. Use this skill when a user wants to make their ML scripts configurable through Valohai, expose hyperparameters for tuning, externalize configuration values, convert hardcoded values to command-line arguments, or add parameter definitions to valohai.yaml. Triggers on mentions of parameters, hyperparameters, configuration, argparse, configurable training, or Valohai parameter migration.","included_files":[],"name":"valohai-migrate-parameters","skill_md_contents":"---\nname: valohai-migrate-parameters\ndescription: Migrate hardcoded values, hyperparameters, and configuration values in ML code to Valohai parameters. Use this skill when a user wants to make their ML scripts configurable through Valohai, expose hyperparameters for tuning, externalize configuration values, convert hardcoded values to command-line arguments, or add parameter definitions to valohai.yaml. Triggers on mentions of parameters, hyperparameters, configuration, argparse, configurable training, or Valohai parameter migration.\nlicense: Apache-2.0\nmetadata:\n  author: valohai\n  version: \"1.0\"\n  compatibility: Requires Python. Designed for Claude Code or similar AI coding agents.\n---\n\n# Valohai Parameter Migration\n\nMigrate hardcoded values in ML code to Valohai-managed parameters. Parameters cover both **hyperparameters** (learning rate, epochs, batch size) and **configuration values** (model name, output format, feature flags, thresholds, file paths). This makes jobs configurable without code changes, enables experiment tracking, hyperparameter sweeps, and reproducibility.\n\n## Philosophy\n\nValohai follows a **YAML-over-SDK** approach. Your ML code should not be entangled with the platform. Parameters are declared in `valohai.yaml` and injected as command-line arguments. Your code uses standard Python `argparse` to receive them.\n\n## Step-by-Step Instructions\n\n### 1. Identify Hardcoded Values\n\nScan the user's ML code for hardcoded values that should be configurable. Parameters are not just for hyperparameters — any value that a user might want to change between runs belongs here:\n\n- **Training hyperparameters**: learning rate, batch size, epochs, dropout rate, weight decay\n- **Model architecture**: hidden layers, units per layer, activation functions, model type/name\n- **Data processing**: train/test split ratio, image size, sequence length, number of workers\n- **Optimization**: optimizer type, scheduler type, warmup steps, gradient clipping\n- **Configuration values**: output format, confidence thresholds, feature flags, model name/path, random seed, number of threads/workers, log level\n- **Any magic numbers** in training loops, model definitions, or scripts\n\n### 2. Add argparse to the Python Code\n\nConvert hardcoded values to command-line arguments using Python's standard `argparse`:\n\n```python\nimport argparse\n\nparser = argparse.ArgumentParser()\nparser.add_argument(\"--learning_rate\", type=float, default=0.001)\nparser.add_argument(\"--epochs\", type=int, default=10)\nparser.add_argument(\"--batch_size\", type=int, default=32)\nparser.add_argument(\"--optimizer\", type=str, default=\"adam\")\nparser.add_argument(\"--use_augmentation\", action=\"store_true\")\nargs = parser.parse_args()\n```\n\nThen replace every hardcoded reference with the parsed argument:\n\n```python\n# Before\nlearning_rate = 0.001\n\n# After\nlearning_rate = args.learning_rate\n```\n\n### 3. Declare Parameters in valohai.yaml\n\nAdd a `parameters` section to the step definition:\n\n```yaml\n- step:\n    name: train-model\n    image: tensorflow/tensorflow:2.6.0\n    command:\n      - pip install -r requirements.txt\n      - python train.py {parameters}\n    parameters:\n      - name: learning_rate\n        type: float\n        default: 0.001\n      - name: epochs\n        type: integer\n        default: 10\n      - name: batch_size\n        type: integer\n        default: 32\n      - name: optimizer\n        type: string\n        default: adam\n      - name: use_augmentation\n        type: flag\n        default: false\n```\n\n### Command Interpolation: How Parameters Get Passed\n\nValohai supports several placeholders in the `command` field. Understanding the difference is critical to avoid argument parsing errors.\n\n**SYNTAX WARNING**: The delimiter is always a **colon** (`:`), never brackets or parentheses. `{parameter-value:name}` is correct. `{parameter-value[name]}` and `{parameter(name)}` are WRONG and will silently fail.\n\n#### `{parameters}` - Preferred approach\n\nExpands to ALL parameters with their `pass-as` format (default: `--name=value`). This is the safest and most common approach:\n\n```yaml\ncommand:\n  - python train.py {parameters}\n# Expands to: python train.py --learning_rate=0.001 --epochs=10 --optimizer=adam\n```\n\n#### `{parameter:name}` - Single parameter with pass-as format\n\nExpands to the full `--name=value` formatted string for one parameter. Useful when parameters need to go in specific positions:\n\n```yaml\ncommand:\n  - python train.py --model /valohai/inputs/model/*.onnx {parameter:conf} {parameter:iou}\n# Expands to: python train.py --model /valohai/inputs/model/*.onnx --conf=0.5 --iou=0.7\n```\n\n#### `{parameter-value:name}` - Raw value only\n\nExpands to just the raw value, without the `--name=` prefix. Use this when you need to embed the value into a custom argument format:\n\n```yaml\ncommand:\n  - python train.py --threshold {parameter-value:threshold}\n# Expands to: python train.py --threshold 0.5\n```\n\n#### Common mistake: using `{parameter:name}` where `{parameter-value:name}` is needed\n\n```yaml\n# WRONG - double-passes the flag name, causes \"expected one argument\" errors\ncommand:\n  - python main.py --conf={parameter:conf}\n# Expands to: python main.py --conf=--conf=0.5   <-- BROKEN\n\n# CORRECT - use {parameter-value:name} for raw values\ncommand:\n  - python main.py --conf={parameter-value:conf}\n# Expands to: python main.py --conf=0.5\n\n# BEST - just use {parameter:name} on its own or {parameters} for all\ncommand:\n  - python main.py {parameter:conf} {parameter:iou}\n# Expands to: python main.py --conf=0.5 --iou=0.7\n```\n\n**Rule of thumb**: Use `{parameters}` to pass all parameters. Use `{parameter:name}` when you need specific parameters in specific positions. Only use `{parameter-value:name}` when embedding a value into a custom format. Never combine `--flag={parameter:name}` - that double-wraps the argument. **Always use colons** — `{parameter-value:name}` not `{parameter-value[name]}`.\n\n### 4. Parameter Types Reference\n\n| Type      | YAML              | Python argparse           | CLI Example          | Notes |\n|-----------|--------------------|---------------------------|----------------------|-------|\n| Integer   | `type: integer`    | `type=int`                | `--epochs=100`       | Whole numbers |\n| Float     | `type: float`      | `type=float`              | `--lr=0.001`         | Decimal numbers |\n| String    | `type: string`     | `type=str`                | `--model=resnet`     | Text values |\n| Flag      | `type: flag`       | `action='store_true'`     | `--augment`          | When false, nothing is passed |\n\n### 5. Advanced Parameter Options\n\n#### Constrained Ranges\n\n```yaml\nparameters:\n  - name: learning_rate\n    type: float\n    default: 0.001\n    min: 0.0001\n    max: 0.1\n  - name: batch_size\n    type: integer\n    default: 32\n    min: 8\n    max: 512\n```\n\n#### Choices (Dropdown in UI)\n\n```yaml\nparameters:\n  - name: optimizer\n    type: string\n    default: adam\n    choices:\n      - adam\n      - sgd\n      - rmsprop\n      - adamw\n```\n\n#### Optional Parameters\n\n```yaml\nparameters:\n  - name: pretrained_model\n    type: string\n    optional: true\n```\n\n#### Flag with Boolean Pass-Through\n\nBy default, flags only pass the flag name when true and pass nothing when false. For explicit true/false:\n\n```yaml\nparameters:\n  - name: verbose\n    type: flag\n    default: false\n    pass-true-as: --verbose=True\n    pass-false-as: --verbose=False\n```\n\n#### Custom Pass Format\n\n```yaml\nparameters:\n  - name: seed\n    type: integer\n    default: 42\n    pass-as: --random-seed={v}\n```\n\n#### Multiple Values\n\n```yaml\nparameters:\n  - name: layers\n    type: integer\n    default: 128\n    multiple: separate\n    multiple-separator: \",\"\n```\n\n### 6. Alternative: Config File Approach\n\nIf the code cannot use argparse (e.g., it reads a config file), Valohai also creates config files automatically:\n\n```python\nimport json\n\n# Valohai creates this file automatically from the parameter values\nwith open(\"/valohai/config/parameters.json\") as f:\n    params = json.load(f)\n    learning_rate = params[\"learning_rate\"]\n    epochs = params[\"epochs\"]\n```\n\nOr with YAML:\n\n```python\nimport yaml\n\nwith open(\"/valohai/config/parameters.yaml\") as f:\n    params = yaml.safe_load(f)\n```\n\n## Common Patterns\n\n### PyTorch Training Script\n\n```python\nimport argparse\n\nparser = argparse.ArgumentParser()\nparser.add_argument(\"--epochs\", type=int, default=10)\nparser.add_argument(\"--learning_rate\", type=float, default=0.001)\nparser.add_argument(\"--batch_size\", type=int, default=32)\nparser.add_argument(\"--weight_decay\", type=float, default=1e-5)\nparser.add_argument(\"--optimizer\", type=str, default=\"adam\", choices=[\"adam\", \"sgd\", \"adamw\"])\nargs = parser.parse_args()\n\n# Use args throughout your training code\noptimizer = torch.optim.Adam(model.parameters(), lr=args.learning_rate, weight_decay=args.weight_decay)\ntrain_loader = DataLoader(dataset, batch_size=args.batch_size)\n\nfor epoch in range(args.epochs):\n    train(model, train_loader, optimizer)\n```\n\n### TensorFlow/Keras Training Script\n\n```python\nimport argparse\n\nparser = argparse.ArgumentParser()\nparser.add_argument(\"--epochs\", type=int, default=50)\nparser.add_argument(\"--learning_rate\", type=float, default=0.001)\nparser.add_argument(\"--batch_size\", type=int, default=64)\nparser.add_argument(\"--dropout_rate\", type=float, default=0.5)\nargs = parser.parse_args()\n\nmodel.compile(optimizer=tf.keras.optimizers.Adam(learning_rate=args.learning_rate), ...)\nmodel.fit(x_train, y_train, epochs=args.epochs, batch_size=args.batch_size)\n```\n\n## Validate the YAML\n\n**IMPORTANT**: After adding parameters to `valohai.yaml`, always run `vh lint` to validate the configuration before attempting to run:\n\n```shell\nvh lint\n```\n\nThis catches common issues like invalid parameter types, missing fields, bad indentation, and duplicate names. Fix any errors before proceeding.\n\n## Overriding Parameters at Runtime\n\nOnce parameters are defined, users can override them without changing code:\n\n```shell\n# Via CLI\nvh execution run train-model --epochs=100 --learning_rate=0.0001 --adhoc\n\n# Parameters also appear in the Valohai web UI for easy changes\n```\n\n## Edge Cases\n\n- If code already uses argparse, just add the matching `parameters` section to `valohai.yaml` - no code changes needed\n- Parameter names in YAML must match the argparse argument names (without the `--` prefix)\n- Use underscores in parameter names for consistency (both `--learning_rate` and `--learning-rate` work)\n- Flag parameters: when false, nothing is passed to the command line (the argparse default handles it)\n- The `{parameters}` placeholder is optional - if omitted, Valohai appends parameters to the last command\n"},"changes":[],"summary":"First saved snapshot. No earlier version is available for comparison.","summary_kind":"deterministic","summary_metadata":{}}