{"id":53703,"plugin_id":"plugins_6ac012b82680819195f8a26cbe32c246","kind":"skill","collection_source":"plugin_package","comparison_source":null,"observed_at":"2026-10-24T13:54:52.636Z","digest":"42dc62d1b330cc56d9ae28e815a1cc71832d35c5890e3a673cc14585602391e5","against":null,"payload":{"description":"Use the domino-data Python SDK (dominodatalab-data) for programmatic data access in Domino. Covers DataSourceClient for SQL queries and object storage, DatasetClient for dataset files, TrainingSets for ML data versioning, Feature Store, and VectorDB (Pinecone) integration. Use when querying data sources, downloading datasets, managing training sets, or working with vector databases in Domino.","included_files":[{"relative_path":"DATA-SOURCES.md","size_in_bytes":4210},{"relative_path":"DATASETS.md","size_in_bytes":4313},{"relative_path":"TRAINING-SETS.md","size_in_bytes":5715},{"relative_path":"VECTORDB.md","size_in_bytes":5193}],"name":"domino-data-sdk","skill_md_contents":"---\nname: domino-data-sdk\ndescription: Use the domino-data Python SDK (dominodatalab-data) for programmatic data access in Domino. Covers DataSourceClient for SQL queries and object storage, DatasetClient for dataset files, TrainingSets for ML data versioning, Feature Store, and VectorDB (Pinecone) integration. Use when querying data sources, downloading datasets, managing training sets, or working with vector databases in Domino.\n---\n\n# Domino Data SDK Skill\n\nThis skill provides comprehensive knowledge for working with the `domino-data` Python SDK (`dominodatalab-data`) - the official library for Domino's Access Data features.\n\n## Installation\n\n```bash\n# Via pip\npip install -U dominodatalab-data\n\n# Via Poetry\npoetry add dominodatalab-data\n\n# In Domino environment (requirements.txt)\ndominodatalab-data>=6.0.0\n```\n\n## Key Components\n\n| Module | Purpose |\n|--------|---------|\n| `DataSourceClient` | Query SQL databases and access object stores |\n| `DatasetClient` | Read files from Domino Datasets |\n| `TrainingSets` | Version and manage ML training data |\n| `Feature Store` | Manage ML features with Git integration |\n| `VectorDB` | Pinecone vector database integration |\n\n## Related Documentation\n\n- [DATA-SOURCES.md](./DATA-SOURCES.md) - SQL queries and object storage\n- [DATASETS.md](./DATASETS.md) - Dataset file operations\n- [TRAINING-SETS.md](./TRAINING-SETS.md) - Training data versioning\n- [VECTORDB.md](./VECTORDB.md) - Pinecone integration\n\n## Quick Start\n\n### Query a Data Source\n\n```python\nfrom domino_data.data_sources import DataSourceClient\n\n# Initialize client (auto-configured in Domino)\nclient = DataSourceClient()\n\n# Get a data source by name\nds = client.get_datasource(\"my-redshift-db\")\n\n# Execute SQL query\nresult = ds.query(\"SELECT * FROM customers WHERE region = 'US'\")\n\n# Convert to pandas DataFrame\ndf = result.to_pandas()\n\n# Or save to parquet\nresult.to_parquet(\"output.parquet\")\n```\n\n### Access Object Storage\n\n```python\nfrom domino_data.data_sources import DataSourceClient\n\nclient = DataSourceClient()\nds = client.get_datasource(\"my-s3-bucket\")\n\n# List objects\nobjects = ds.list_objects(prefix=\"data/\", page_size=100)\n\n# Download a file\nds.download_file(\"data/input.csv\", \"local_input.csv\")\n\n# Upload a file\nds.put(\"data/output.csv\", open(\"results.csv\", \"rb\").read())\n\n# Get signed URL\nurl = ds.get_key_url(\"data/file.csv\", is_read_write=False)\n```\n\n### Read from Datasets\n\n```python\nfrom domino_data.datasets import DatasetClient\n\nclient = DatasetClient()\n\n# Get dataset by name\ndataset = client.get_dataset(\"training-data\")\n\n# List files\nfiles = dataset.list_files(prefix=\"images/\")\n\n# Download file\ndataset.download(\"model.pkl\", \"local_model.pkl\", max_workers=4)\n\n# Get file content as bytes\ncontent = dataset.get(\"config.json\")\n```\n\n### Training Sets\n\n```python\nfrom domino_data.training_sets import (\n    create_training_set_version,\n    get_training_set,\n    list_training_sets\n)\nimport pandas as pd\n\n# Create training set version from DataFrame\ndf = pd.DataFrame({\n    \"id\": [1, 2, 3],\n    \"feature_a\": [0.1, 0.2, 0.3],\n    \"label\": [1, 0, 1]\n})\n\nversion = create_training_set_version(\n    training_set_name=\"customer-churn\",\n    df=df,\n    key_columns=[\"id\"],\n    description=\"Initial training data\"\n)\n\n# Get training set\nts = get_training_set(\"customer-churn\")\n\n# List all training sets\nall_sets = list_training_sets()\n```\n\n### Vector Database (Pinecone)\n\n```python\nfrom domino_data.vectordb import (\n    domino_pinecone3x_init_params,\n    domino_pinecone3x_index_params\n)\nfrom pinecone import Pinecone\n\n# Initialize Pinecone client with Domino credentials\ninit_params = domino_pinecone3x_init_params(\"my-pinecone-ds\")\npc = Pinecone(**init_params)\n\n# Get index parameters\nindex_params = domino_pinecone3x_index_params(\"my-pinecone-ds\", \"embeddings\")\nindex = pc.Index(**index_params)\n\n# Query vectors\nresults = index.query(\n    vector=[0.1, 0.2, 0.3, ...],\n    top_k=10,\n    include_metadata=True\n)\n```\n\n## Authentication\n\nThe library auto-configures authentication inside Domino workspaces and jobs using injected environment variables:\n\n```python\n# Environment variables used automatically inside Domino:\n# DOMINO_TOKEN_FILE - Token file location (preferred, short-lived)\n# DOMINO_API_PROXY - API proxy URL\n# DOMINO_DATA_API_GATEWAY - Data API gateway (default: http://127.0.0.1:8766)\n#\n# DOMINO_USER_API_KEY - Legacy API key (deprecated, will be removed)\n```\n\nFor external use (e.g., CI/CD outside a Domino execution):\n\n> **Note:** `DOMINO_USER_API_KEY` is deprecated and will be removed in a future Domino release. Prefer running data-SDK code from inside a Domino workspace or job where token-based auth is injected automatically.\n\n```python\nimport os\nos.environ[\"DOMINO_USER_API_KEY\"] = \"your-api-key\"  # deprecated\nos.environ[\"DOMINO_API_HOST\"] = \"https://your-domino.com\"\n\nfrom domino_data.data_sources import DataSourceClient\nclient = DataSourceClient()\n```\n\n## Error Handling\n\n```python\nfrom domino_data.data_sources import DominoError, UnauthenticatedError\n\ntry:\n    result = ds.query(\"SELECT * FROM table\")\nexcept UnauthenticatedError:\n    print(\"Authentication failed - check API key\")\nexcept DominoError as e:\n    print(f\"Domino error: {e}\")\n```\n\n## Best Practices\n\n1. **Use within Domino**: Auth is automatic in workspaces/jobs\n2. **Parallel downloads**: Use `max_workers` for large files\n3. **Pagination**: Use `page_size` when listing many objects\n4. **Training Sets**: Version your training data for reproducibility\n5. **Connection reuse**: Reuse client instances when possible\n\n## Package Info\n\n- **PyPI**: `dominodatalab-data`\n- **GitHub**: https://github.com/dominodatalab/domino-data\n- **License**: Apache 2.0\n- **Python**: 3.8+\n"},"changes":[],"summary":"First saved snapshot. No earlier version is available for comparison.","summary_kind":"deterministic","summary_metadata":{}}