JoinLayer

View saved version →

---
name: joinlayer-pipelines
description: Create, validate, preview, run, monitor, and diagnose JoinLayer data pipelines through the JoinLayer MCP server. Use for PostgreSQL/MySQL replication, Kafka JSON or Avro streams and Schema Registry, ClickHouse finite sources and append targets, BigQuery targets, mappings, filters, calculated fields, lookup enrichment, scheduled syncs, realtime CDC/backfill, workspace capacity, and failed-run diagnosis. Never use it to collect or transmit connection credentials in conversation.
---

# JoinLayer Pipelines

Operate JoinLayer through its MCP tools while preserving workspace permissions, credentials, checkpoints, and target data.

## Understand The Product Model

JoinLayer has five durable layers. Keep them separate in reasoning and reports:

- **Connections** are stored, encrypted source, target, or lookup endpoints. MCP exposes safe metadata and capabilities, never credentials.
- **Pipeline drafts** describe routes, schemas, mappings, transforms, enrichment, scheduling, and target write behavior. A draft is not a run.
- **Validation and preview** prove structural readiness and show representative output without starting data movement.
- **Runs** execute one-time, scheduled, realtime, or realtime-with-backfill work and carry progress, checkpoints, worker state, recovery, errors, and dead letters.
- **Governance** combines the user's current workspace role, OAuth scopes, exact run approvals or narrow automation permissions, workspace capacity, and attributed activity. The narrowest boundary wins.

For the first nontrivial task in a new session, or whenever the available product surface is unclear, read [product-capabilities.md](references/product-capabilities.md) before selecting tools. It contains the complete MCP tool map, task playbooks, state model, and error recovery contract.

## Start Here In Every New Session

Do not wait for the user to explain JoinLayer's API and do not explore the repository. The connected `joinlayer` MCP server is the authoritative product interface.

For the first JoinLayer request in a session:

1. Confirm that the `joinlayer` MCP server and its tools are available. If the client has not connected yet, use the configured MCP URL and let the client discover JoinLayer OAuth and open browser consent. Never ask the user to create, paste, reveal, or carry an agent token. If the client cannot perform MCP OAuth, stop and explain that an OAuth-capable MCP connection is required. Do not substitute shell commands, direct HTTP calls, database access, or guessed product behavior.
2. Call `get_workspace_context`. State the workspace name/ID, agent identity, role, and granted scopes. Stop if the workspace is not the one the user expects.
3. Call `get_workspace_capacity`. Report whether new work can start and name every returned blocker. Do not collapse independent limits into one balance.
4. For an inspection request, call `list_connections` and `list_pipelines`, then summarize current resources without changing anything.
5. For a creation request, continue with connector discovery, stored connection discovery, and schema discovery before drafting a pipeline.

For Codex CLI when the `joinlayer` MCP server is not configured, use the deployment URL declared in `agents/openai.yaml`:

```bash
codex mcp add joinlayer --url https://mcp.joinlayer.app/mcp
```

Current Codex releases may start browser OAuth immediately from `mcp add` and may initially request every capability advertised by the authorization server. JoinLayer preselects read permissions only; operating, editing, and execution permissions require an explicit click. For the safe first-session inspection, leave only `workspace:read`, `usage:read`, `connections:read`, and `pipelines:read` checked. JoinLayer grants the exact checked subset. If the server is configured but not authenticated, or when reconnecting after logout, use:

```bash
codex mcp login joinlayer --scopes workspace:read,usage:read,connections:read,pipel

Preview of the first 4,000 characters. Read full instructions →

Referenced files: 6 — file references are metadata, not package contents.