Valohai Skills

View saved version →

---
name: valohai-design-pipelines
description: Analyze ML project structure and design Valohai pipelines to orchestrate multi-step workflows. Use this skill when a user wants to create a Valohai pipeline, connect multiple ML steps into an automated workflow, identify pipeline opportunities in their codebase, design data flow between preprocessing/training/evaluation/inference steps, or add conditional logic and parallel execution to pipelines. Triggers on mentions of pipelines, workflows, DAGs, orchestration, multi-step ML, or connecting Valohai steps.
license: Apache-2.0
metadata:
  author: valohai
  version: "1.0"
  compatibility: Requires Python ML projects. Designed for Claude Code or similar AI coding agents.
---

# Design Valohai Pipelines

Analyze ML project structure to identify pipeline opportunities and design multi-step Valohai pipelines that orchestrate preprocessing, training, evaluation, and deployment workflows.

## What Pipelines Do

A Valohai pipeline connects individual steps into an automated workflow:
- **Nodes** = Individual steps (preprocessing, training, evaluation, inference, deployment)
- **Edges** = Data flow between steps (outputs become inputs, metadata becomes parameters)

Valohai automatically executes nodes in the right order, passes data between steps, runs independent nodes in parallel, and tracks complete data lineage.

## Step-by-Step: Analyze the Project

### 1. Identify Pipeline Stages

Examine the ML project to find distinct stages. Common patterns:

**Standard ML Pipeline:**
```
Raw Data → Preprocessing → Training → Evaluation → (Deployment)
```

**With Feature Engineering:**
```
Raw Data → Feature Engineering → Train/Val/Test Split → Training → Evaluation
```

**With Hyperparameter Optimization:**
```
Preprocessing → HPO Sweep (Task) → Select Best → Final Training → Evaluation
```

**Multi-Model Pipeline:**
```
                    ┌→ Train Model A ─┐
Preprocessing ──────┤                  ├→ Ensemble → Evaluation
                    └→ Train Model B ─┘
```

**Full Production Pipeline:**
```
Data Ingestion → Preprocessing → Training → Evaluation → Model Validation → Deployment → Smoke Test
```

### 2. Map Data Dependencies

For each stage, identify:
- **What files does it need?** (input data, models, configs)
- **What files does it produce?** (processed data, trained models, metrics, reports)
- **What metrics does it produce?** (accuracy, loss, F1 that downstream steps might need)
- **What parameters should flow between steps?** (best learning rate, selected features)

### 3. Identify Parallelization Opportunities

Steps that don't depend on each other can run in parallel:
- Training multiple model variants
- Processing different data splits
- Running different evaluation metrics
- A/B training with different hyperparameters

## Step-by-Step: Create the Pipeline YAML

### Design Principle: Every Step Should Run Independently

**IMPORTANT**: Whenever possible, give every input a `default` value so each step can be tested and debugged on its own with `vh execution run step-name --adhoc`. Pipeline edges override these defaults at runtime.

Look for default values in the existing code, README, documentation, config files, or data paths already referenced in the project. Only add a default if you can find a real, meaningful value. Do not invent placeholder URLs or dummy paths - leave the input without a default if no real value is available.

### 1. Ensure Steps Are Defined First

Each pipeline node references an existing step. Define all steps before the pipeline:

```yaml
- step:
    name: preprocess-data
    image: python:3.11
    command:
      - pip install -r requirements.txt
      - python preprocess.py {parameters}
    parameters:
      - name: test_split
        type: float
        default: 0.2
    inputs:
      - name: raw-data
        default: s3://bucket/raw-data/

- step:
    name: train-model
    image: pytorch/pytorch:2.1.0-cuda12.1-cudnn8-runtime
    command:
      - pip insta

Preview of the first 4,000 characters. Read full instructions →

View saved version →

---
name: valohai-migrate-data
description: Migrate data loading and model saving in ML code to use Valohai's input/output system. Use this skill when a user wants to configure data inputs from cloud storage (S3, Azure Blob, GCS), save model outputs to Valohai, replace hardcoded file paths, remove boto3/cloud SDK code, or set up the Valohai file I/O system. Triggers on mentions of inputs, outputs, data loading, model saving, S3, cloud storage, file paths, or Valohai data migration.
license: Apache-2.0
metadata:
  author: valohai
  version: "1.0"
  compatibility: Requires Python. Designed for Claude Code or similar AI coding agents.
---

# Valohai Data I/O Migration

Migrate data loading and model saving to use Valohai's managed input/output system. This eliminates cloud SDK boilerplate, handles authentication automatically, and enables full data lineage tracking.

## Philosophy

Valohai separates data access from code. Inputs are declared in `valohai.yaml` with cloud storage URLs. Valohai downloads them to `/valohai/inputs/{name}/` before your code runs. Outputs are saved to `/valohai/outputs/` and automatically uploaded. No boto3, no credentials in code, no download logic.

## Step-by-Step: Inputs

### 1. Identify Data Loading Code

Look for patterns like:
- `boto3.client('s3')` / `s3.download_file()`
- `from google.cloud import storage`
- `azure.storage.blob`
- `requests.get()` for downloading datasets
- `wget` or `curl` in shell commands
- Hardcoded local paths like `/data/train/`, `./datasets/`
- `pd.read_csv("path/to/data.csv")`
- `torch.load("model.pth")` for pretrained models

### Design Principle: Always Set Default Values for Inputs

**IMPORTANT**: Whenever possible, give every input a `default` value so the step can run independently without a pipeline. This lets users test individual steps with `vh execution run step-name --adhoc`. Pipeline edges override defaults at runtime.

Look for default values in the existing code, README, documentation, config files, or data paths already referenced in the project. Only add a default if you can find a real, meaningful value. Do not invent placeholder URLs or dummy paths.

### 2. Define Inputs in valohai.yaml

```yaml
- step:
    name: train-model
    image: tensorflow/tensorflow:2.6.0
    command:
      - pip install -r requirements.txt
      - python train.py
    inputs:
      - name: training-data
        default: s3://my-bucket/datasets/train.csv
      - name: pretrained-model
        default: datum://production-model-latest
        filename: model.h5
```

#### Input Options

```yaml
inputs:
  # Single file
  - name: config
    default: s3://bucket/config.json

  # Multiple files with wildcard
  - name: images
    default:
      - s3://bucket/images/*.jpg
      - s3://bucket/images/*.png

  # Multiple cloud sources
  - name: data
    default:
      - s3://aws-bucket/data.csv
      - azure://container/data.csv
      - gs://gcs-bucket/data.csv

  # Rename on download
  - name: model
    default: s3://bucket/models/v42-best.h5
    filename: model.h5

  # Preserve directory structure
  - name: dataset
    default: s3://bucket/dataset/**/*.json
    keep-directories: suffix

  # Optional input (can be empty)
  - name: checkpoint
    optional: true

  # Reference a Valohai datum (output from another execution)
  - name: pretrained
    default: datum://my-model-alias
```

#### keep-directories Options

- `none` (default): All files flat in `/valohai/inputs/{name}/`
- `suffix`: Keeps path after the wildcard root
- `full`: Keeps the full storage path

### 3. Update Python Code

Replace cloud SDK / download code with simple local file reads:

#### Before (with boto3)

```python
import boto3

s3 = boto3.client('s3', aws_access_key_id=KEY, aws_secret_access_key=SECRET)
s3.download_file('my-bucket', 'datasets/train.csv', '/tmp/train.csv')
df = pd.read_csv('/tmp/train.csv')
```

#### After (Valohai)

```python
import pandas as pd

# Files are already downloaded to /valohai/inputs/
df = 

Preview of the first 4,000 characters. Read full instructions →

View saved version →

---
name: valohai-migrate-metrics
description: Add metrics and metadata tracking to ML code for Valohai. Use this skill when a user wants to track training metrics (loss, accuracy, F1, etc.), log experiment metadata, enable real-time metric visualization, or compare experiments in Valohai. Triggers on mentions of metrics, metadata, experiment tracking, logging accuracy/loss, or Valohai metric migration.
license: Apache-2.0
metadata:
  author: valohai
  version: "1.0"
  compatibility: Requires Python. Designed for Claude Code or similar AI coding agents.
---

# Valohai Metrics/Metadata Migration

Add metrics tracking to ML code so Valohai automatically captures, visualizes, and enables comparison across experiments. No special libraries required - just print JSON to stdout.

## Philosophy

Valohai captures metrics by detecting JSON printed to stdout during execution. This is deliberately simple and framework-agnostic. No SDK imports, no decorators, no special API calls. Just `print(json.dumps({...}))`.

## Step-by-Step Instructions

### 1. Identify Metrics to Track

Scan the user's ML code for values worth tracking:

- **Training metrics**: loss, accuracy, precision, recall, F1 score, AUC-ROC
- **Training dynamics**: learning rate (if scheduled), gradient norm, batch processing time
- **Validation metrics**: val_loss, val_accuracy, val_f1 (per epoch or interval)
- **Resource metrics**: GPU utilization, memory usage, throughput (samples/sec)
- **Final results**: best model score, total training time, convergence epoch
- **Custom KPIs**: any domain-specific metric the user cares about

### 2. Add JSON Printing to Code

The core pattern is simple - print a JSON dictionary to stdout:

```python
import json

# Log metrics at any point in your code
print(json.dumps({"accuracy": 0.92, "loss": 0.08}))
```

**CRITICAL: Group all metrics from the same moment into a single `json.dumps()` call.** Each `print(json.dumps(...))` creates one metadata event with one timestamp. If you print metrics separately, Valohai treats them as disconnected events and they cannot be correlated.

```python
# WRONG - 4 disconnected events, can't be correlated or plotted together
print(json.dumps({"inference_time_s": 0.45}))
print(json.dumps({"num_detections": 6}))
print(json.dumps({"confidence_threshold": 0.25}))
print(json.dumps({"iou_threshold": 0.7}))

# CORRECT - 1 event, all metrics linked together
print(json.dumps({
    "inference_time_s": 0.45,
    "num_detections": 6,
    "confidence_threshold": 0.25,
    "iou_threshold": 0.7,
}))
```

Same rule applies to training loops - one epoch = one `json.dumps()`:

```python
# WRONG
print(json.dumps({"epoch": epoch}))
print(json.dumps({"train_loss": train_loss}))
print(json.dumps({"val_accuracy": val_acc}))

# CORRECT
print(json.dumps({
    "epoch": epoch,
    "train_loss": train_loss,
    "val_accuracy": val_acc,
}))
```

Valohai automatically:
- Captures every JSON line printed to stdout
- Adds a UTC timestamp
- Makes values searchable, sortable, and plottable
- Enables real-time visualization during execution

### 3. Common Integration Patterns

#### Training Loop (Most Common)

```python
import json

for epoch in range(epochs):
    train_loss = train_one_epoch(model, train_loader, optimizer)
    val_loss, val_acc = validate(model, val_loader)

    print(json.dumps({
        "epoch": epoch,
        "train_loss": train_loss,
        "val_loss": val_loss,
        "val_accuracy": val_acc,
    }))
```

#### Batch-Level Logging

```python
import json

for epoch in range(epochs):
    for batch_idx, (data, target) in enumerate(train_loader):
        loss = train_step(model, data, target, optimizer)

        if batch_idx % 100 == 0:  # Log every N batches to stay under 50 events/sec
            print(json.dumps({
                "epoch": epoch,
                "batch": batch_idx,
                "loss": loss.item(),
            }))
```

#### Multiple Phases with Context

```python
import json

# Training phase
for e

Preview of the first 4,000 characters. Read full instructions →

View saved version →

---
name: valohai-migrate-parameters
description: Migrate hardcoded values, hyperparameters, and configuration values in ML code to Valohai parameters. Use this skill when a user wants to make their ML scripts configurable through Valohai, expose hyperparameters for tuning, externalize configuration values, convert hardcoded values to command-line arguments, or add parameter definitions to valohai.yaml. Triggers on mentions of parameters, hyperparameters, configuration, argparse, configurable training, or Valohai parameter migration.
license: Apache-2.0
metadata:
  author: valohai
  version: "1.0"
  compatibility: Requires Python. Designed for Claude Code or similar AI coding agents.
---

# Valohai Parameter Migration

Migrate hardcoded values in ML code to Valohai-managed parameters. Parameters cover both **hyperparameters** (learning rate, epochs, batch size) and **configuration values** (model name, output format, feature flags, thresholds, file paths). This makes jobs configurable without code changes, enables experiment tracking, hyperparameter sweeps, and reproducibility.

## Philosophy

Valohai follows a **YAML-over-SDK** approach. Your ML code should not be entangled with the platform. Parameters are declared in `valohai.yaml` and injected as command-line arguments. Your code uses standard Python `argparse` to receive them.

## Step-by-Step Instructions

### 1. Identify Hardcoded Values

Scan the user's ML code for hardcoded values that should be configurable. Parameters are not just for hyperparameters — any value that a user might want to change between runs belongs here:

- **Training hyperparameters**: learning rate, batch size, epochs, dropout rate, weight decay
- **Model architecture**: hidden layers, units per layer, activation functions, model type/name
- **Data processing**: train/test split ratio, image size, sequence length, number of workers
- **Optimization**: optimizer type, scheduler type, warmup steps, gradient clipping
- **Configuration values**: output format, confidence thresholds, feature flags, model name/path, random seed, number of threads/workers, log level
- **Any magic numbers** in training loops, model definitions, or scripts

### 2. Add argparse to the Python Code

Convert hardcoded values to command-line arguments using Python's standard `argparse`:

```python
import argparse

parser = argparse.ArgumentParser()
parser.add_argument("--learning_rate", type=float, default=0.001)
parser.add_argument("--epochs", type=int, default=10)
parser.add_argument("--batch_size", type=int, default=32)
parser.add_argument("--optimizer", type=str, default="adam")
parser.add_argument("--use_augmentation", action="store_true")
args = parser.parse_args()
```

Then replace every hardcoded reference with the parsed argument:

```python
# Before
learning_rate = 0.001

# After
learning_rate = args.learning_rate
```

### 3. Declare Parameters in valohai.yaml

Add a `parameters` section to the step definition:

```yaml
- step:
    name: train-model
    image: tensorflow/tensorflow:2.6.0
    command:
      - pip install -r requirements.txt
      - python train.py {parameters}
    parameters:
      - name: learning_rate
        type: float
        default: 0.001
      - name: epochs
        type: integer
        default: 10
      - name: batch_size
        type: integer
        default: 32
      - name: optimizer
        type: string
        default: adam
      - name: use_augmentation
        type: flag
        default: false
```

### Command Interpolation: How Parameters Get Passed

Valohai supports several placeholders in the `command` field. Understanding the difference is critical to avoid argument parsing errors.

**SYNTAX WARNING**: The delimiter is always a **colon** (`:`), never brackets or parentheses. `{parameter-value:name}` is correct. `{parameter-value[name]}` and `{parameter(name)}` are WRONG and will silently fail.

#### `{parameters}` - Preferred approach

Expands to ALL parameters with their `pass-as`

Preview of the first 4,000 characters. Read full instructions →

View saved version →

---
name: valohai-project-run
description: Create a Valohai project, link it to a local directory, and run executions or pipelines using the Valohai CLI. Use this skill when a user wants to set up a new Valohai project, link an existing project, run a step execution, run a pipeline, watch execution logs, or manage their Valohai project via the CLI. Triggers on mentions of vh project create, vh execution run, vh pipeline run, running on Valohai, or Valohai CLI commands.
license: Apache-2.0
metadata:
  author: valohai
  version: "1.0"
  compatibility: Requires valohai-cli installed (pip install valohai-cli). Designed for Claude Code or similar AI coding agents.
---

# Valohai Project Setup and Execution

Set up a Valohai project and run ML steps using the Valohai CLI (`vh`). This covers project creation, linking, running executions, running pipelines, and monitoring jobs.

## Prerequisites

```shell
# Install the CLI
pip install valohai-cli

# Login (interactive)
vh login

# Login with token (for SSO or CI/CD)
vh login --token YOUR_TOKEN

# Login to self-hosted instance
vh login --host https://your-company.valohai.io --token YOUR_TOKEN
```

## Step-by-Step: Project Setup

### 1. Create a New Project

```shell
# Create and link to current directory
vh project create --name my-ml-project

# Create with description
vh project create --name my-ml-project --description "Image classification pipeline"

# Create under an organization (CASE SENSITIVE - must match exactly)
vh project create --name my-ml-project --owner my-org

# Create under an organization team (CASE SENSITIVE)
vh project create --name my-ml-project --owner my-org:ml-team

# Create without linking
vh project create --name my-ml-project --no-link
```

This creates the project on Valohai and links the current directory to it.

### 2. Link to an Existing Project

```shell
# Interactive selection from your projects
vh project link

# Or specify directly
vh project link --project my-org/my-ml-project
```

### 3. Check Project Status

```shell
# See current project link
vh project status

# List all your projects
vh project list

# Open project in browser
vh project open
```

## Step-by-Step: Running Executions

### 1. Run a Step (Ad-hoc)

During development, use `--adhoc` to run local code without Git commits:

```shell
# Basic run with local code
vh execution run train-model --adhoc

# Run and watch logs in real-time
vh execution run train-model --adhoc --watch

# Run with parameter overrides
vh execution run train-model --adhoc --epochs=100 --learning_rate=0.0001

# Run with input overrides
vh execution run train-model --adhoc --training-data=s3://bucket/new-data.csv

# Run with specific Docker image override
vh execution run train-model --adhoc --image pytorch/pytorch:2.2.0-cuda12.1-cudnn8-runtime

# Run with environment override
vh execution run train-model --adhoc --environment aws-eu-west-1-p3-2xlarge

# Run with tags
vh execution run train-model --adhoc --tag experiment-v2 --tag baseline

# Run with title
vh execution run train-model --adhoc --title "Baseline experiment with new data"

# Run with environment variables
vh execution run train-model --adhoc --var WANDB_MODE=disabled

# Combine multiple options
vh execution run train-model --adhoc --watch \
  --epochs=100 \
  --learning_rate=0.0001 \
  --training-data=s3://bucket/v2/train.csv \
  --tag baseline-v2
```

### 2. Run from a Git Commit

When code is committed and pushed:

```shell
# Run from latest fetched commit
vh execution run train-model

# Run from specific commit
vh execution run train-model --commit abc123

# Run from a branch/tag
vh execution run train-model --commit main
```

### 3. Monitor Executions

```shell
# List recent executions
vh execution list
vh execution list --count 10

# Stream logs from an execution
vh execution logs <counter-or-id> --stream

# Watch a running execution
vh execution watch <counter-or-id>

# Get execution details
vh execution info <counter-or-id>

# Open execution in bro

Preview of the first 4,000 characters. Read full instructions →

View saved version →

---
name: valohai-yaml-step
description: Create valohai.yaml step definitions for ML projects. Use this skill when a user wants to define a new step in valohai.yaml, configure Docker images, set up commands, define parameters/inputs for a step, or create a complete valohai.yaml from scratch for their ML project. Triggers on mentions of valohai.yaml, step definition, YAML configuration, Docker image selection, or creating Valohai steps.
license: Apache-2.0
metadata:
  author: valohai
  version: "1.0"
  compatibility: Requires Python. Designed for Claude Code or similar AI coding agents.
---

# Create Valohai YAML Steps

Create and configure `valohai.yaml` step definitions that describe how ML code should run on Valohai. A step defines the Docker image, commands, parameters, and inputs for an execution.

## Philosophy

The `valohai.yaml` file is the single source of truth for your ML workflows. It lives in the repository root and is version-controlled alongside your code. Your code stays framework-agnostic and portable - all platform configuration is in this file.

## Step-by-Step Instructions

### Design Principle: Steps Should Run Independently

**IMPORTANT**: Every step should be runnable on its own, without relying on a pipeline to provide its inputs. Whenever possible, set a `default` value for inputs so users can test and debug individual steps via `vh execution run step-name --adhoc` without needing to run the full pipeline first. Pipeline edges override these defaults at runtime.

Look for default values in the existing code, README, documentation, config files, or data paths already referenced in the project. Only add a default if you can find a real, meaningful value. Do not invent placeholder URLs or dummy paths.

### 1. Analyze the Project

Before creating the YAML, understand the project:

- **What scripts exist?** (train.py, preprocess.py, evaluate.py, infer.py)
- **What framework?** (PyTorch, TensorFlow, scikit-learn, XGBoost, HuggingFace, etc.)
- **What Python version?** (Check pyproject.toml, setup.py, .python-version)
- **What dependencies?** (requirements.txt, pyproject.toml, conda environment)
- **Does it need GPU?** (CUDA imports, torch.cuda, tf.config)
- **What are the inputs?** (datasets, pretrained models, config files)
- **What are the outputs?** (trained models, metrics, plots, predictions)
- **What are the configurable values?** (hyperparameters, paths, flags)

### 2. Choose the Docker Image

Select an appropriate base image:

| Framework | CPU Image | GPU Image |
|-----------|-----------|-----------|
| Python only | `python:3.11` | N/A |
| TensorFlow | `tensorflow/tensorflow:2.15.0` | `tensorflow/tensorflow:2.15.0-gpu` |
| PyTorch | `pytorch/pytorch:2.1.0-cuda12.1-cudnn8-runtime` | Same (includes CUDA) |
| HuggingFace | `huggingface/transformers-pytorch-gpu:latest` | Same |
| scikit-learn | `python:3.11` | N/A |
| R | `r-base:4.3.0` | N/A |
| Custom | Your own Docker image from a registry | Your own Docker image |

**Guidelines:**
- The Docker image is the execution environment, not your code container
- Don't include your code or data in the image
- Pin version tags for reproducibility (avoid `latest`)
- If using GPU, choose an image with the right CUDA version

### 3. Install Dependencies in Commands

**IMPORTANT**: The Docker image is a clean base environment. Your project's dependencies (from `requirements.txt`, `pyproject.toml`, `environment.yml`, etc.) are NOT pre-installed. You must install them as part of the step commands BEFORE running your script, otherwise you'll get `ModuleNotFoundError`.

Look for dependency files in the project:

| File Found | Install Command |
|------------|----------------|
| `requirements.txt` | `pip install -r requirements.txt` |
| `pyproject.toml` (with pip) | `pip install .` or `pip install -e .` |
| `pyproject.toml` (with uv) | `pip install uv && uv pip install --system .` |
| `setup.py` | `pip install .` |
| `environment.yml` (conda) | `conda env update -n base

Preview of the first 4,000 characters. Read full instructions →