Update to CrowdStrike Falcon Foundry

{
  "name": "e2e-testing",
  "description": "End-to-end testing for Falcon Foundry apps using Playwright and @crowdstrike/foundry-playwright. TRIGGER when user asks to \"add e2e tests\", \"add playwright tests\", \"write end-to-end tests\", \"test my app\", or mentions \"e2e\", \"playwright\", or \"end-to-end\" in the context of testing a Foundry app. DO NOT TRIGGER during normal app creation, UI development, or function development. This skill is opt-in; not all apps need e2e tests.",
  "included_files": [
    {
      "relative_path": "references/debugging-with-mcp.md",
      "size_in_bytes": 5012
    }
  ],
  "skill_md_contents": "---\nname: e2e-testing\ndescription: End-to-end testing for Falcon Foundry apps using Playwright and @crowdstrike/foundry-playwright. TRIGGER when user asks to \"add e2e tests\", \"add playwright tests\", \"write end-to-end tests\", \"test my app\", or mentions \"e2e\", \"playwright\", or \"end-to-end\" in the context of testing a Foundry app. DO NOT TRIGGER during normal app creation, UI development, or function development. This skill is opt-in; not all apps need e2e tests.\nversion: 1.5.0\nupdated: 2026-08-19\ntags: [foundry, e2e, playwright, testing]\nauthor: CrowdStrike\nlicense: MIT\ncompatibility: Claude Code >=1.0\nmetadata:\n  category: testing\n---\n\n# Foundry E2E Testing\n\nEnd-to-end testing for Falcon Foundry apps using [Playwright](https://playwright.dev/) and the [`@crowdstrike/foundry-playwright`](https://github.com/CrowdStrike/foundry-playwright) library.\n\nThe library provides authentication, app install/uninstall, page objects, and configuration so each app only writes its app-specific tests.\n\n> **Part of a suite.** If `development-workflow` has not already run, and this is a new app or its first capability, load the `development-workflow` skill first — it owns the CLI prerequisite check, scaffolding order, and manifest coordination.\n\n## Quick Start\n\n### 1. Create the `e2e/` directory\n\n```\nmy-foundry-app/\n├── e2e/\n│   ├── .env                    # Local credentials (git-ignored)\n│   ├── .env.sample             # Template for other developers\n│   ├── .gitignore\n│   ├── package.json\n│   ├── playwright.config.ts\n│   └── tests/\n│       └── foundry.spec.ts\n├── manifest.yml\n└── ...\n```\n\n### 2. `package.json`\n\nNode.js LTS is recommended.\n\n```json\n{\n  \"name\": \"playwright-foundry\",\n  \"version\": \"1.0.0\",\n  \"scripts\": {\n    \"test\": \"npx playwright test\",\n    \"test:ui\": \"npx playwright test --ui\",\n    \"test:debug\": \"npx playwright test --debug\",\n    \"test:verbose\": \"DEBUG=true npx playwright test --reporter=list\"\n  },\n  \"type\": \"commonjs\",\n  \"devDependencies\": {\n    \"@crowdstrike/foundry-playwright\": \"0.5.0\",\n    \"@types/node\": \"25.6.0\"\n  }\n}\n```\n\n**Always pin exact versions** — never use `\"latest\"`, `\"^\"`, or `\"~\"`. Check npm for the current version of each package.\n\nThe library brings `@playwright/test`, `@dotenvx/dotenvx`, and `otpauth` as transitive dependencies. No need to install them separately.\n\n### 3. `.env`\n\n```sh\nFALCON_USERNAME=your.email@company.com\nFALCON_PASSWORD=your-password\nFALCON_AUTH_SECRET=your-totp-secret\nFALCON_BASE_URL=https://falcon.us-2.crowdstrike.com\nAPP_NAME=your-app-name\n```\n\n**Convention for sample apps:** Set `APP_NAME` to match the manifest `name` field, which should match the repo name (e.g., `foundry-sample-functions-python`). This avoids spaces in names and simplifies CI. This is a convention, not a hard requirement.\n\n### 4. `playwright.config.ts`\n\n```typescript\nimport { defineFoundryConfig } from '@crowdstrike/foundry-playwright';\n\nexport default defineFoundryConfig();\n```\n\nThis gives you the standard 4-project pipeline automatically:\n1. **setup**: authenticate and save session state\n2. **app-install**: install the app via App Catalog\n3. **chromium**: run your tests\n4. **app-uninstall**: clean up after tests\n\n### 5. `.gitignore`\n\n```\nnode_modules/\nplaywright/.auth/\nplaywright-report/\ntest-results/\n.env\n```\n\n### 6. Install and run\n\n```bash\ncd e2e\nnpm install\nnpx playwright install chromium --with-deps\nnpm test\n```\n\n## Writing Tests\n\n### Available page objects\n\nThe library provides these page objects:\n\n| Class | Purpose |\n|-------|---------|\n| `WorkflowsPage` | Search, open, execute, and verify Falcon Fusion SOAR workflows |\n| `DetectionExtensionPage` | Navigate to Endpoint Detections, expand extensions, return iframe FrameLocator |\n| `HostManagementPage` | Navigate to host management, retrieve host IDs |\n| `AppCatalogPage` | Install, uninstall, and navigate to apps |\n| `AppBuilderPage` | Disable workflow provisioning before install |\n| `AppManagerPage` | Find and navigate to apps in App Manager |\n| `FoundryHomePage` | Navigate to Falcon Foundry home |\n\n### Fixtures pattern\n\nCreate `src/fixtures.ts` to wire up page objects as Playwright fixtures. **Only import what your tests actually use** — don't define unused fixtures:\n\n```typescript\nimport { test as baseTest } from '@playwright/test';\nimport { DetectionExtensionPage, WorkflowsPage } from '@crowdstrike/foundry-playwright';\n\ntype FoundryFixtures = {\n  detectionExtensionPage: DetectionExtensionPage;\n  workflowsPage: WorkflowsPage;\n};\n\nexport const test = baseTest.extend<FoundryFixtures>({\n  detectionExtensionPage: async ({ page }, use) => { await use(new DetectionExtensionPage(page)); },\n  workflowsPage: async ({ page }, use) => { await use(new WorkflowsPage(page)); },\n});\n\nexport { expect } from '@playwright/test';\n```\n\nPlaywright fixtures are lazy (only instantiated when a test requests them), so unused fixtures don't hurt performance — but they add confusion and dead code. Add fixtures as you add tests that need them.\n\n### Example test: workflows\n\n```typescript\nimport { test } from '../src/fixtures';\n\ntest.describe.configure({ mode: 'serial' });\n\ntest('should execute workflow', async ({ workflowsPage }) => {\n  test.setTimeout(180000);\n  await workflowsPage.navigateToWorkflows();\n  await workflowsPage.executeAndVerifyWorkflow('My Workflow Name');\n  await workflowsPage.verifyWorkflowExecutionCompleted();\n});\n\ntest('should execute workflow with input', async ({ workflowsPage, hostManagementPage }) => {\n  test.setTimeout(180000);\n  const hostId = await hostManagementPage.getFirstHostId();\n  if (!hostId) { test.skip(true, 'No hosts available'); return; }\n\n  await workflowsPage.navigateToWorkflows();\n  await workflowsPage.executeAndVerifyWorkflow('Host Details Workflow', {\n    inputs: { 'Host ID': hostId },\n  });\n  await workflowsPage.verifyWorkflowExecutionCompleted();\n});\n```\n\n`executeAndVerifyWorkflow()` handles search, execution trigger, and initial verification. `verifyWorkflowExecutionCompleted()` opens the execution detail view in a new tab and polls until the status leaves \"In Progress\" — it fails the test if the execution reports \"Failed\" and times out after 120s by default. For render-only checks (e.g., ServiceNow workflows without credentials), use `verifyWorkflowRenders()`.\n\n### Example test: UI extensions\n\n```typescript\nimport { test, expect } from '../src/fixtures';\n\ntest('should render extension', async ({ detectionExtensionPage }) => {\n  const frame = await detectionExtensionPage.openExtension('hello');\n  await expect(frame.getByText(/My App Title/i)).toBeVisible({ timeout: 10000 });\n});\n```\n\n`openExtension()` navigates to Endpoint Detections, opens the first detection, scrolls to the named extension button, expands it, and returns the iframe FrameLocator.\n\n## Apps with Configuration Screens\n\nIf your app has API integration settings during install (e.g., ServiceNow credentials), the default install will fail because the Install button stays disabled until fields are filled.\n\n### 1. Add integration credentials to `.env` and `.env.sample`\n\n```sh\n# .env.sample — commit this as a template\nSERVICENOW_INSTANCE_URL=https://dev123456.service-now.com\nSERVICENOW_USERNAME=your-servicenow-username\nSERVICENOW_PASSWORD=your-servicenow-password\n\n# .env — local values, git-ignored\nSERVICENOW_INSTANCE_URL=https://dev99999.service-now.com\nSERVICENOW_USERNAME=admin\nSERVICENOW_PASSWORD=s3cret\n```\n\n### 2. Create a custom `tests/app-install.setup.ts`\n\n```typescript\nimport { test as setup } from '@playwright/test';\nimport { AppCatalogPage, config } from '@crowdstrike/foundry-playwright';\n\nsetup('install app', async ({ page }) => {\n  const catalog = new AppCatalogPage(page);\n\n  const instanceUrl = process.env.SERVICENOW_INSTANCE_URL;\n  const username = process.env.SERVICENOW_USERNAME;\n  const password = process.env.SERVICENOW_PASSWORD;\n  if (!instanceUrl || !username || !password) {\n    throw new Error('Missing required ServiceNow env vars: SERVICENOW_INSTANCE_URL, SERVICENOW_USERNAME, SERVICENOW_PASSWORD');\n  }\n\n  await catalog.installApp(config.appName, {\n    configureSettings: async (page) => {\n      await page.getByRole('textbox', { name: 'Name', exact: true }).fill('ServiceNow Integration');\n      await page.getByRole('textbox', { name: 'Instance' }).fill(instanceUrl);\n      await page.getByRole('textbox', { name: 'Username' }).fill(username);\n      await page.getByRole('textbox', { name: 'Password' }).fill(password);\n    },\n  });\n});\n```\n\nThe library loads `.env` automatically (via `@dotenvx/dotenvx`) so `process.env` values are available without extra setup. In CI, set these as GitHub Actions secrets instead.\n\n### 3. Point the config at the custom install\n\n```typescript\nexport default defineFoundryConfig({\n  appInstallDir: './tests',\n});\n```\n\n**How to discover field names:** Use Playwright MCP to take a snapshot of the install page and inspect the form fields. See [debugging-with-mcp.md](references/debugging-with-mcp.md).\n\n### Disabling Workflow Provisioning\n\nApps with workflows that require valid API credentials will fail during install if you provide fake credentials for the configuration screen. Some workflows are also long-running (e.g., Anomali ThreatStream ingestion) and shouldn't be provisioned during testing because they'll start automatically. Use `AppBuilderPage.disableWorkflowProvisioning()` before install to skip provisioning:\n\n```typescript\nimport { test as setup } from '@playwright/test';\nimport { AppBuilderPage, AppCatalogPage, config } from '@crowdstrike/foundry-playwright';\n\nsetup('install app', async ({ page }) => {\n  setup.setTimeout(300000);\n  const appBuilder = new AppBuilderPage(page);\n  await appBuilder.disableWorkflowProvisioning(config.appName);\n\n  const catalog = new AppCatalogPage(page);\n  await catalog.installApp(config.appName);\n});\n```\n\nThis navigates to the App Builder, finds the app, toggles off workflow provisioning for each workflow, and saves. Call it before `installApp()` in your custom `app-install.setup.ts`.\n\n### Multi-Screen Configuration Wizards\n\nSome apps have settings spread across multiple screens (e.g., Workday with 4 screens, SailPoint with 3). Navigate between screens using the \"Next setting\" button:\n\n```typescript\nimport { test as setup } from '@playwright/test';\nimport { AppBuilderPage, AppCatalogPage, config } from '@crowdstrike/foundry-playwright';\n\nsetup('install app', async ({ page }) => {\n  setup.setTimeout(300000);\n  const appBuilder = new AppBuilderPage(page);\n  await appBuilder.disableWorkflowProvisioning(config.appName);\n\n  const catalog = new AppCatalogPage(page);\n\n  await catalog.installApp(config.appName, {\n    configureSettings: async (page) => {\n      const nextButton = page.getByRole('button', { name: 'Next setting' });\n\n      // Screen 1: First API integration settings\n      await page.getByRole('textbox', { name: 'Name' }).fill('My Integration');\n      await page.getByRole('textbox', { name: 'Host' }).fill('https://api.example.com');\n      await nextButton.click();\n      await page.waitForLoadState('networkidle').catch(() => {});\n\n      // Screen 2: Authentication settings\n      await page.getByRole('textbox', { name: 'Client ID' }).fill(process.env.CLIENT_ID!);\n      await page.getByRole('textbox', { name: 'Client Secret' }).fill(process.env.CLIENT_SECRET!);\n      await nextButton.click();\n      await page.waitForLoadState('networkidle').catch(() => {});\n\n      // Screen 3: Additional settings (last screen — \"Install app\" button is visible here)\n      await page.getByRole('textbox', { name: 'Tenant ID' }).fill(process.env.TENANT_ID!);\n    },\n  });\n});\n```\n\nThe `waitForLoadState('networkidle').catch(() => {})` after each \"Next setting\" click gives the next screen time to load. The `.catch(() => {})` prevents failures if the page is already idle.\n\n## Custom Page Objects\n\nFor app-specific UI that the library doesn't cover, extend `BasePage`:\n\n```typescript\nimport { Page, expect } from '@playwright/test';\nimport { BasePage, AppCatalogPage, config } from '@crowdstrike/foundry-playwright';\n\nexport class MyAppPage extends BasePage {\n  constructor(page: Page) {\n    super(page, 'MyAppPage'); // display name for logging only\n  }\n\n  protected getPagePath(): string {\n    throw new Error('Direct path navigation not supported. Use navigateToInstalledApp() instead.');\n  }\n\n  protected async verifyPageLoaded(): Promise<void> {\n    await this.page.locator('h1').filter({ hasText: /My App/i }).waitFor();\n  }\n\n  async navigateToInstalledApp(): Promise<void> {\n    return this.withTiming(async () => {\n      const catalog = new AppCatalogPage(this.page);\n      await catalog.navigateToInstalledApp(config.appName);\n      await this.verifyPageLoaded();\n    }, 'Navigate to installed app');\n  }\n\n  async verifyAppContent(): Promise<void> {\n    return this.withTiming(async () => {\n      const iframe = this.page.frameLocator('iframe[name=\"portal\"]');\n      const heading = iframe.getByRole('heading', { name: /My App/i });\n      await expect(heading).toBeVisible({ timeout: 10000 });\n    }, 'Verify app content');\n  }\n}\n```\n\nFoundry app page URLs contain deployment-specific IDs that change on every deploy, so never hardcode paths. Use `AppCatalogPage.navigateToInstalledApp()` to navigate via the App Catalog menu instead.\n\n`BasePage` gives you `smartClick()`, `withTiming()`, `navigateToPath()`, `elementExists()`, a `logger`, and a `waiter` (SmartWaiter instance).\n\nAdd the custom page to your fixtures alongside library page objects.\n\n## Debugging with Playwright MCP\n\nPlaywright MCP is invaluable for writing and debugging e2e tests. See [references/debugging-with-mcp.md](references/debugging-with-mcp.md) for detailed patterns.\n\n**Key principle:** When using Playwright MCP interactively, authentication is manual. Navigate to the Falcon login page, then tell the user: *\"Please log in to Falcon in the browser, then let me know when you're ready.\"* Do not attempt automated TOTP login through MCP.\n\n## CI with GitHub Actions\n\nThe sample apps use a shared `e2e.yml` workflow pattern. Rather than duplicating it here (where it would go stale), reference the canonical implementation:\n\n**Reference:** [`foundry-sample-functions-python/.github/workflows/e2e.yml`](https://github.com/CrowdStrike/foundry-sample-functions-python/blob/main/.github/workflows/e2e.yml)\n\n### Key CI concepts\n\n1. **Concurrency serialization**: Only one e2e run per repo at a time (prevents deployment collisions)\n2. **Unique app names**: CI generates a unique name from repo + actor + timestamp to avoid conflicts\n3. **Manifest ID stripping**: `yq -i 'del(.. | select(has(\"id\")).id) | del(.. | select(has(\"app_id\")).app_id)' manifest.yml`\n4. **Deploy → wait → release → test → cleanup**: The workflow deploys, polls for success, releases, runs tests, then always deletes the app\n5. **App cleanup**: `foundry apps delete -f` runs in an `always()` step so apps are cleaned up even on failure\n\n### Required GitHub secrets\n\n| Secret | Purpose |\n|--------|---------|\n| `FOUNDRY_API_CLIENT_ID` | Foundry CLI authentication |\n| `FOUNDRY_API_CLIENT_SECRET` | Foundry CLI authentication |\n| `FOUNDRY_CID` | CrowdStrike Customer ID |\n| `FOUNDRY_CLOUD_REGION` | Cloud region (us-1, us-2, us-3, eu-1) |\n| `FALCON_USERNAME` | Falcon console login for Playwright |\n| `FALCON_PASSWORD` | Falcon console password |\n| `FALCON_AUTH_SECRET` | TOTP secret for 2FA |\n\n### CI-specific behavior\n\nThe library detects `CI=true` and adjusts:\n- Higher timeouts (60s default vs 45s local)\n- Retries enabled (2 retries vs 0 local)\n- `.env` loading skipped (credentials come from environment)\n\n## `defineFoundryConfig()` Options\n\n| Option | Type | Default |\n|--------|------|---------|\n| `testDir` | `string` | `'./tests'` |\n| `appInstallDir` | `string` | Library built-in (override for apps with config screens) |\n| `timeout` | `number` | 60s (CI) / 45s (local) |\n| `retries` | `number` | 2 (CI) / 0 (local) |\n| `reporter` | `string` | `'list'` |\n| `use` | `object` | Merged with defaults (`testIdAttribute: 'data-test-selector'`) |\n| `projects` | `array` | Replaces the default 4-project pipeline if provided |\n\n### Sidebar navigation flakiness\n\nThe Falcon sidebar menu re-renders during navigation, which can detach DOM elements mid-click and cause intermittent timeouts. This affects any test that navigates through the sidebar (detections, workflows, host management, etc.). The library's navigation methods include retry logic for this, but with `retries: 0` locally, a single re-render failure will fail the test.\n\nFor apps that navigate through the sidebar, set `retries: 2` to handle this reliably:\n\n```typescript\nexport default defineFoundryConfig({ retries: 2 });\n```\n\n## Common Pitfalls\n\n| Problem | Cause | Fix |\n|---------|-------|-----|\n| \"Could not find app in catalog\" | `APP_NAME` doesn't match the manifest `name` | Ensure `.env` APP_NAME matches `manifest.yml` name field |\n| Install button stays disabled | App has config screens (API integrations) | Create custom `app-install.setup.ts` with `configureSettings()` |\n| \"collection existed previously but was deleted\" | Stale IDs in manifest from a previous deployment | Strip all IDs: `yq -i 'del(.. \\| select(has(\"id\")).id) \\| del(.. \\| select(has(\"app_id\")).app_id)' manifest.yml` |\n| Tests fail on first run after deploy | Release not propagated yet | Wait 15-30s after release before running tests; CI workflow includes a sleep |\n| Tests pass locally but fail in CI | Different timeout/retry settings | Check `CI=true` is set; library auto-adjusts timeouts |\n| Login fails with account lockout | Running `npm test` in multiple apps simultaneously | Run tests for one app at a time. Concurrent login attempts against the same Falcon account trigger rate-limiting and temporarily lock the account. |\n\n## Reference Implementations\n\nAll [CrowdStrike/foundry-sample-*](https://github.com/CrowdStrike?q=foundry-sample) apps have e2e tests using this library:\n\n| App | Highlights |\n|-----|------------|\n| [foundry-sample-functions-python](https://github.com/CrowdStrike/foundry-sample-functions-python/tree/main/e2e) | `configureSettings()`, workflows, extension, host details |\n| [foundry-sample-mitre](https://github.com/CrowdStrike/foundry-sample-mitre/tree/main/e2e) | Custom page objects (`MitreChartPage`), parallel tests |\n| [foundry-sample-anomali-threatstream](https://github.com/CrowdStrike/foundry-sample-anomali-threatstream/tree/main/e2e) | API integration config |\n| [foundry-sample-category-blocking](https://github.com/CrowdStrike/foundry-sample-category-blocking/tree/main/e2e) | UI page testing |\n| [foundry-sample-charlotte-toolkit](https://github.com/CrowdStrike/foundry-sample-charlotte-toolkit/tree/main/e2e) | Charlotte AI toolkit |\n| [foundry-sample-collections-toolkit](https://github.com/CrowdStrike/foundry-sample-collections-toolkit/tree/main/e2e) | Collection CRUD |\n| [foundry-sample-detection-translation](https://github.com/CrowdStrike/foundry-sample-detection-translation/tree/main/e2e) | Detection extension |\n| [foundry-sample-foundryjs-demo](https://github.com/CrowdStrike/foundry-sample-foundryjs-demo/tree/main/e2e) | Foundry-JS demo |\n| [foundry-sample-idp-notifications](https://github.com/CrowdStrike/foundry-sample-idp-notifications/tree/main/e2e) | IDP notifications |\n| [foundry-sample-insider-risk-sailpoint](https://github.com/CrowdStrike/foundry-sample-insider-risk-sailpoint/tree/main/e2e) | SailPoint integration |\n| [foundry-sample-insider-risk-workday](https://github.com/CrowdStrike/foundry-sample-insider-risk-workday/tree/main/e2e) | Workday integration |\n| [foundry-sample-logscale](https://github.com/CrowdStrike/foundry-sample-logscale/tree/main/e2e) | LogScale data ingestion |\n| [foundry-sample-ngsiem-importer](https://github.com/CrowdStrike/foundry-sample-ngsiem-importer/tree/main/e2e) | Next-Gen SIEM importer |\n| [foundry-sample-openrouter-toolkit](https://github.com/CrowdStrike/foundry-sample-openrouter-toolkit/tree/main/e2e) | OpenRouter AI toolkit |\n| [foundry-sample-rapid-response](https://github.com/CrowdStrike/foundry-sample-rapid-response/tree/main/e2e) | Rapid response |\n| [foundry-sample-scalable-rtr](https://github.com/CrowdStrike/foundry-sample-scalable-rtr/tree/main/e2e) | Scalable RTR |\n| [foundry-sample-servicenow-idp](https://github.com/CrowdStrike/foundry-sample-servicenow-idp/tree/main/e2e) | ServiceNow IDP |\n| [foundry-sample-servicenow-itsm](https://github.com/CrowdStrike/foundry-sample-servicenow-itsm/tree/main/e2e) | ServiceNow ITSM |\n| [foundry-sample-threat-intel](https://github.com/CrowdStrike/foundry-sample-threat-intel/tree/main/e2e) | Threat intelligence |\n| [foundry-sample-zscaler-internet-access](https://github.com/CrowdStrike/foundry-sample-zscaler-internet-access/tree/main/e2e) | Zscaler integration |\n\n## Integration with Other Skills\n\n- **development-workflow**: E2E testing is delegated from the orchestrator when users request it\n- **ui-development**: Tests verify UI pages and extensions render correctly in the Falcon console\n- **workflows-development**: Tests verify workflow execution and completion\n- **debugging-workflows**: Use for deploy/release issues that block testing\n"
}