skills/temporal-ops/references/ops/cloud-certs.md

# Temporal Cloud -- mTLS Certificate Management

Quick-reference for generating, uploading, filtering, and rotating mTLS certificates on Temporal Cloud using `tcld`.

> For diagnosing certificate errors (x509 failures, TLS handshake errors), see the triage file `../triage/certificates.md`.

---

## Namespace ID format

All `--namespace` flags accept a **Namespace ID** in the format `<namespace_name>.<account_suffix>` (e.g., `your-namespace.a1b2c`).

If `--namespace` is omitted, the value of the environment variable `$TEMPORAL_CLOUD_NAMESPACE` is used.

---

## 1. Generating certificates with tcld

### 1a. Generate a CA certificate

```bash
tcld generate-certificates certificate-authority-certificate \
    --organization <value> \
    --validity-period <duration> \
    --ca-certificate-file <path>.pem \
    --ca-key-file <path>.key
```

Alias for the subcommand: `ca`

| Flag | Alias | Purpose |
|---|---|---|
| `--organization` | `--org` | Organization name for the certificate |
| `--validity-period` | `-d` | Duration in `d/h` format (e.g. `30d10h`) |
| `--ca-certificate-file` | `--ca-cert` | Output path for the `.pem` CA certificate |
| `--ca-key-file` | `--ca-key` | Output path for the `.key` private key |
| `--rsa-algorithm` | `--rsa` | Use 4096-bit RSA instead of ECDSA P-384 (disabled by default) |

Default key algorithm: ECDSA P-384.

A CA certificate generated by tcld has a maximum duration of 1 year (`-d 1y`) and a minimum duration of 7 days.

**Shorthand example** (from docs):

```bash
tcld gen ca --org temporal -d 1y --ca-cert ca.pem --ca-key ca.key
```

### 1b. Generate an end-entity (leaf) certificate

```bash
tcld generate-certificates end-entity-certificate \
    --organization <value> \
    --validity-period <duration> \
    --ca-certificate-file <ca>.pem \
    --ca-key-file <ca>.key \
    --certificate-file <leaf>.pem \
    --key-file <leaf>.key
```

Alias for the subcommand: `leaf`

| Flag | Alias | Purpose |
|---|---|---|
| `--organization` | `--org` | Organization name |
| `--organization-unit` | _(none)_ | Optional OU name |
| `--common-name` | _(none)_ | Optional common name |
| `--validity-period` | `-d` | Duration in `d/h` format |
| `--ca-certificate-file` | `--ca-cert` | Path to the signing CA `.pem` |
| `--ca-key-file` | `--ca-key` | Path to the signing CA `.key` |
| `--certificate-file` | `--cert` | Output path for the leaf `.pem` |
| `--key-file` | `--key` | Output path for the leaf `.key` |

End-entity certificate must expire before its root CA certificate.

**Shorthand example** (from docs):

```bash
tcld gen leaf --org temporal -d 364d --ca-cert ca.pem --ca-key ca.key --cert client.pem --key client.key
```

---

## 2. CA certificate requirements

CA certificates uploaded to Temporal Cloud must meet all of the following:

- X.509v3
- Each certificate must be a root certificate or issued by another certificate in the bundle
- Must include `CA: true`
- Cannot be a well-known CA (e.g. DigiCert, Let's Encrypt) unless certificate filters are also specified
- Signing algorithm: RSA or ECDSA with SHA-256 or stronger (SHA-1 and MD5 rejected)
- Cannot be generated with a passphrase
- Bundle limit: up to 16 CA certificates, max 32 KB payload before base64 encoding

End-entity certificates must include `CA: false` and Digital Signature key usage.

Each certificate in the chain (from end-entity to root) must have a unique Distinguished Name. Distinguished Names are not case sensitive.

---

## 3. Uploading CA certificates to a Namespace

### 3a. Add a CA certificate (appends to existing)

```bash
tcld namespace accepted-client-ca add \
    --namespace <namespace_name>.<account_suffix> \
    --ca-certificate-file <path>
```

Alias: `a`

| Flag | Alias | Notes |
|---|---|---|
| `--ca-certificate` | `-c` | Base64-encoded CA certificate PEM string |
| `--ca-certificate-file` | `-f` | Path to CA certificate PEM file |
| `--namespace` | `-n` | Namespace ID |
| `--resource-version` | `-v` | ETag; uses latest if omitted |
| `--request-id` | `-r` | Request identifier for async op |

If both `--ca-certificate` and `--ca-certificate-file` are specified, only `--ca-certificate` is used.

### 3b. Set CA certificates (replaces all existing)

```bash
tcld namespace accepted-client-ca set \
    --namespace <namespace_name>.<account_suffix> \
    --ca-certificate-file <path>
```

Alias: `s`

Same flags as `add` (`--ca-certificate` / `--ca-certificate-file`, `--namespace`, `--resource-version`, `--request-id`).

### 3c. List current CA certificates

```bash
tcld namespace accepted-client-ca list \
    --namespace <namespace_name>.<account_suffix>
```

Alias: `l`

### 3d. Remove a CA certificate

```bash
tcld namespace accepted-client-ca remove \
    --namespace <namespace_name>.<account_suffix> \
    --ca-certificate-file <path>
```

Alias: `r`

Removal can target by certificate content or fingerprint:

| Flag | Alias | Notes |
|---|---|---|
| `--ca-certificate` | `-c` | Base64-encoded PEM string |
| `--ca-certificate-file` | `-f` | Path to PEM file |
| `--ca-certificate-fingerprint` | `--fp` | Certificate fingerprint (takes precedence; if set, cert/file flags are ignored) |
| `--all` | _(none)_ | Remove all CA certificates; cannot be combined with the cert/file/fingerprint flags; blocked when auth method is `mtls` or `api_key_or_mtls` |

---

## 4. Certificate filters

Certificate filters restrict which end-entity certificates can connect, based on DN fields.

A filter can include any combination (at least one) of:

- `commonName`
- `organization`
- `organizationalUnit`
- `subjectAlternativeName`

Maximum 25 certificate filters per Namespace.

Matching is case-insensitive. A single `*` wildcard is allowed at the beginning or end (but not both) of a value. A bare `*` is not valid.

If a well-known CA certificate is configured, you cannot clear certificate filters.

JSON format for filter definitions:

```json
{ "filters": [ { "commonName": "test1" } ] }
```

### 4a. Import (set) certificate filters

Replaces any existing filters with the ones provided.

```bash
tcld namespace certificate-filters import \
    --namespace <namespace_name>.<account_suffix> \
    --certificate-filter-file <path>
```

Alias: `imp`

| Flag | Alias | Notes |
|---|---|---|
| `--certificate-filter-file` | `--file`, `-f` | Path to JSON file |
| `--certificate-filter-input` | `--input`, `-i` | Inline JSON string |

If both `--certificate-filter-file` and `--certificate-filter-input` are specified, the command returns an error.

### 4b. Add certificate filters (appends)

```bash
tcld namespace certificate-filters add \
    --namespace <namespace_name>.<account_suffix> \
    --certificate-filter-file <path>
```

| Flag | Alias | Notes |
|---|---|---|
| `--certificate-filter-file` | `-f`, `--file` | Path to JSON file |
| `--certificate-filter-input` | `-i`, `--input` | Inline JSON string |

### 4c. Export (view) current certificate filters

```bash
tcld namespace certificate-filters export \
    --namespace <namespace_name>.<account_suffix> \
    --certificate-filter-file <output_path>
```

Alias: `exp`

### 4d. Clear all certificate filters

```bash
tcld namespace certificate-filters clear \
    --namespace <namespace_name>.<account_suffix>
```

Alias: `c`

Caution: clearing filters allows any client certificate that chains up to a configured CA certificate to connect.

---

## 5. Certificate rotation procedure (zero-downtime)

This is the rollover process documented for both UI and tcld.

### Using tcld

1. Create a single PEM file containing both old and new CA certificates concatenated:

   ```
   -----BEGIN CERTIFICATE-----
   ... old CA cert ...
   -----END CERTIFICATE-----
   -----BEGIN CERTIFICATE-----
   ... new CA cert ...
   -----END CERTIFICATE-----
   ```

2. Upload the bundle:

   ```bash
   tcld namespace accepted-client-ca set --ca-certificate-file <path>
   ```

3. Monitor traffic to the old certificate until it ceases.

4. Create a file containing only the new CA certificate.

5. Run `set` again with the new-only file to remove the old CA:

   ```bash
   tcld namespace accepted-client-ca set --ca-certificate-file <path>
   ```

### Using the Cloud UI

1. Navigate to **Namespaces** > select Namespace > **Edit** > **Authentication**.
2. Scroll past the existing certificate's `-----END CERTIFICATE-----` and paste the new PEM block on the next line.
3. **Save**.
4. Wait until all Workers are using the new certificate.
5. Return to **Edit** > **Authentication**, delete the old certificate, and **Save**.

---

## 6. Namespace auth method: mTLS vs API keys

When creating a Namespace, `--auth-method` selects the authentication mode.

Valid values: `mtls`, `api_key`, `restricted`, `api_key_or_mtls`

- `mtls` (default): requires `--ca-certificate` or `--ca-certificate-file`
- `api_key`: no certificate flags needed
- `api_key_or_mtls`: accepts both authentication methods simultaneously (requires flexible auth to be enabled; contact [Temporal Support](https://docs.temporal.io/cloud/support#ticketing) to enable it)

```bash
tcld namespace create \
    --namespace <namespace_name>.<account_suffix> \
    --region us-east-1 \
    --auth-method api_key
```

Creating with mTLS:

```bash
tcld namespace create \
    --namespace <namespace_name>.<account_suffix> \
    --region us-east-1 \
    --ca-certificate-file ca.pem
```

Certificate filters can optionally be set at create time via `--certificate-filter-file` or `--certificate-filter-input`.

---

## 7. Handling a compromised end-entity certificate

Temporal does not support or check certificate revocation lists (CRLs).

Recommended approach: use short-lived end-entity certificates so a compromised one expires quickly.

If immediate action is needed:

1. If using certificate filters, set filters to block the compromised certificate.
2. Otherwise, generate a new CA certificate.
3. Deploy the new CA alongside the existing one (so existing end-entity certs continue working).
4. Regenerate all end-entity certificates from the new CA.
5. Remove the old (compromised) CA certificate from the Namespace.
6. Monitor audit logs for unauthorized access.

---

## 8. Configuring clients with end-entity certificates

Use the `temporal` CLI with these flags:

```bash
temporal <command> <subcommand> \
    --tls-ca-path <path_to_server_CA_certificate> \
    --tls-cert-path <path_to_x509_certificate> \
    --tls-key-path <path_to_private_certificate_key> \
    --tls-server-name <override_for_target_TLS_server_name>
```

SDK-specific connection guides:

- Go: `/develop/go/client/temporal-client#connect-to-temporal-cloud`
- Java: `/develop/java/client/temporal-client#connect-to-temporal-cloud`
- Python: `/develop/python/client/temporal-client#connect-to-temporal-cloud`
- TypeScript: `/develop/typescript/client/temporal-client#connect-to-temporal-cloud`
- .NET: `/develop/dotnet/client/temporal-client#connect-to-temporal-cloud`
- PHP: `/develop/php/client/temporal-client#connect-to-a-dev-cluster`

For Java SDK: convert the private key from PKCS1 to PKCS8 format:

```bash
openssl pkcs8 -topk8 -inform PEM -outform PEM -in <key>.key -out <key>.pkcs8.key -nocrypt
```

---

## 9. Expiration notifications

Temporal Cloud sends email notifications 15 days before certificate expiration.

An expired root CA invalidates all downstream certificates. An expired end-entity certificate prevents Temporal Clients from connecting or starting Workflow Executions. Workers with expired certs will either stall indefinitely or cause Workflow timeouts.