TPC Benchmarks

Description

From the marketplace listing

See whether an AI coding agent can actually integrate a developer API, SDK or tool from its documentation, measured by The Prompting Company. Ask about a product or a category, such as search APIs or transactional email, and get measured results in chat: completion rate, passed, failed and blocked runs, average successful run time, and an overall score from 0 to 100 that averages an experience score (graded on Install, Auth, Execute, Docs and Current) and an agent readiness score. Name a task, like sending a transactional email or verifying a sending domain, and products are ranked by their results on that task. Tasks a product does not offer through its public API are marked not supported, with the documentation that shows it. Ask to see the results and get interactive cards: a comparison table, a weekly overall score chart, a passed, failed and blocked bar for each product, the last test date and a link to the full benchmark page on promptingcompany.com. Every answer is complete inside ChatGPT; opening a benchmark page is optional. Covers products with 3+ benchmark runs on the public TPC leaderboard; grows as benchmarks run. Products not covered return no results; you can ask the app to request a benchmark, which the TPC team reviews. No sign-in, no account changes, no credentials; the only thing the app writes is a benchmark request you ask for. Results describe how AI coding agents performed on the days they were tested and are not a guarantee of how a product will work for you. See the privacy policy for what the app stores.