MCP server

ZeroWidth Caliper Official

ai.zerowidth/caliper

Checked 26 Oct 2026, 08:29 UTC · Connected to it and listed its tools

Build datasets and rubrics, then run evals that score your AI in Caliper.

Website

At a glance

Type

MCP server

Publisher

zerowidth.ai (verified by the MCP Registry)

Status

Working at last check

Tools

45

Token cost

About 17,884 tokens per request

Sets up in

Claude Code, Claude Desktop, Codex, Gemini CLI, Cursor, VS Code, Antigravity, ChatGPT, OpenAI API, Grok

Last checked

26 Oct 2026, 08:29 UTC

Connect it to your agent

Add to CursorAdd to VS CodeVS Code Insiders

Opens the app, which shows the server and asks you to confirm. For other apps, copy the setup below.

Claude Code

claude mcp add --transport http caliper https://api.zerowidth.ai/mcp/x/caliper

Claude Desktop

Settings → Connectors → Add custom connector, and paste the URL.

Codex (config.toml)

[mcp_servers.caliper]
url = "https://api.zerowidth.ai/mcp/x/caliper"

Gemini CLI

{"mcpServers":{"caliper":{"httpUrl":"https://api.zerowidth.ai/mcp/x/caliper"}}}

Cursor

{"mcpServers":{"caliper":{"url":"https://api.zerowidth.ai/mcp/x/caliper"}}}

VS Code

{"servers":{"caliper":{"type":"http","url":"https://api.zerowidth.ai/mcp/x/caliper"}}}

Antigravity

agy mcp add caliper https://api.zerowidth.ai/mcp/x/caliper

ChatGPT

Settings → Apps & Connectors → Advanced settings: turn on Developer mode, then Create a connector and paste the URL. Which plans can do this varies.

OpenAI API (Responses tools)

{"type":"mcp","server_label":"caliper","server_url":"https://api.zerowidth.ai/mcp/x/caliper","require_approval":"always"}

Grok (xAI API tools)

{"type":"mcp","server_label":"caliper","server_url":"https://api.zerowidth.ai/mcp/x/caliper"}

Tools

• caliper_datasets_add_items ~847 tokens
  Appends items to an existing dataset — use this to grow coverage (new edge cases, scenarios from a completed interview) instead of creating a parallel dataset. Same four shapes as caliper_datasets_create (Q&A, chat, sequence, simulated). Existing items and their ratings are untouched.
• caliper_datasets_create ~1226 tokens
  Creates a dataset of test items — THE FIRST STEP of setting up evaluation for a flow. Four item shapes: Q&A (input + optional expectedOutput, the golden answer); CHAT (messages: the conversation so far as role/content user and assistant messages you write, ending on a user message, + expectedRespons
• caliper_datasets_delete Deletes data · ~243 tokens
  Deletes a dataset. Evals bound to it stop being runnable (their binding shows datasetOk: false), and reviews/specs over its items lose their source — check caliper_evals_list for evals that reference it and say so before proposing. May return `needs_confirmation`.
• caliper_datasets_generate ~1010 tokens
  Creates a NEW dataset of model-written items — Q&A by default, or scripted sequences / simulated people via `shape` — pass flowId and the generator reads the flow's prompt, mode, and schema to write realistic cases for THAT flow; `description` adds guidance (or stands alone when there's no flow). Us
• caliper_datasets_generate_items ~750 tokens
  Appends model-written items to an EXISTING dataset, in the style of what's already there (existing items are the few-shot examples; the output shape matches theirs unless overridden). Use to widen coverage when the user says 'more like these' or 'add edge cases'; write them by hand with caliper_data
• caliper_datasets_get ~302 tokens
  One dataset with a page of its items — read this BEFORE editing items (caliper_datasets_update_item needs the item id) and before extending coverage, so you don't add cases that already exist. Items come `limit` at a time (default 25) from `offset`; `total` is the full count. Long fields are cut at
• caliper_datasets_list ~129 tokens
  Lists datasets in the active workspace. Returns summaries (id, name, item count); items are not included.
• caliper_datasets_remove_item ~237 tokens
  Drops one item. Ratings and run results keyed to it are orphaned (kept in history, gone from the dataset). A dataset must keep at least one item. May return `needs_confirmation`.
• caliper_datasets_update ~366 tokens
  Changes a dataset's title, description, or visibility. Pass only what changes; items are untouched (use caliper_datasets_add_items / caliper_datasets_update_item / caliper_datasets_remove_item for those). May return `needs_confirmation`.
• caliper_datasets_update_item ~894 tokens
  Rewrites one item's content, keeping its id (ratings and eval results keyed to it stay attached). The item's KIND can't change — a Q&A item stays Q&A, a chat item stays chat, a sequence stays a sequence — so pass the same shape you read from caliper_datasets_get. Raw items, and chat items captured w
• caliper_evals_create ~828 tokens
  Binds a dataset + rubric to something under test as a repeatable eval — THE LAST SETUP STEP before scoring. Two targets: a Workbench flow (pass flowId; Caliper runs inference itself, then caliper_evals_run scores it), or 'external' (target: 'external'; the user's own model runs elsewhere and their s
• caliper_evals_delete Deletes data · ~200 tokens
  Deletes an eval and stops its schedule. Run history is kept but no longer reachable from the eval. May return `needs_confirmation`.
• caliper_evals_get ~242 tokens
  One eval: what it targets (Workbench flow + stage, or external), its dataset and rubric ids, the rubric criteria it scores with (the snapshot taken at creation), schedule + runOnPublish + regression settings, run counts, latest score, and `binding` health (datasetOk / flowOk false = a run would fail
• caliper_evals_list ~142 tokens
  Lists evals in the active workspace with their target config (which Workbench flow, which dataset/rubric). Use to find the evalId for caliper_evals_run.
• caliper_evals_run ~423 tokens
  Queues a new run of an eval — inference over the dataset, then LLM-judge scoring. THE VERIFY STEP of the improvement loop: after an approved workbench_flows_edit_text, run the eval again and report the score delta vs the previous run. Runs take a while — but you're brought back into THIS conversatio
• caliper_evals_runs_cancel ~306 tokens
  Stops a run that is still PENDING / RUNNING / SCORING — the brake on a run that's spending more than expected or was started by mistake. The queue stops at once; an item already handed to the flow finishes on its own timeout. The run settles as FAILED with a 'cancelled' reason and keeps the items it
• caliper_evals_runs_get ~398 tokens
  One run: status, overall score, and by default the ten lowest-scoring items with their per-criterion scores and the judge's reasoning (`items: all` for every item, `none` for totals only). THE ANSWER to 'why did the score drop' — read this, then name the criterion that slipped and quote the reasonin
• caliper_evals_runs_item_execution ~360 tokens
  What the flow actually DID on one run item: every step in order (what it said, which tools it called with what arguments, what came back) and the final answer, plus status, duration, and cost. Read this when a low score needs explaining beyond the judge's reasoning — a wrong tool call or an empty to
• caliper_evals_runs_list ~234 tokens
  Recent runs for one eval, newest first: status (PENDING/RUNNING/SCORING/DONE/FAILED), overall score once DONE, label, and timestamps. THE CHECK-BACK for caliper_evals_run: when the user asks how the run went, read this — cite the run id and score, and compare against the PREVIOUS run's score for the
• caliper_evals_update ~798 tokens
  Changes an eval's title, description, visibility, schedule (interval + time anchor), runOnPublish (queue a run whenever the flow publishes), regressionThreshold (score drop vs the previous run that triggers an alert; null = off), or scenarioId (the scenario runs use). Pass only what changes. The dat
• caliper_flow_performance ~290 tokens
  How a Workbench flow is ACTUALLY doing, with receipts: every Caliper eval targeting the flow, recent runs with scores, the latest run decomposed into per-criterion averages, the score delta vs the previous run, and the worst-scoring items WITH the judge's reasoning. Use this BEFORE claiming a flow w
• caliper_rubrics_create ~653 tokens
  Creates the scoring rubric an eval's LLM judge uses — 1-10 criteria, each scored on a numeric scale (default 1-5), judged pass/fail (kind pass_fail), or checked in code with no judge (kind check: contains, not_contains, matches a regex, valid_json, max_chars, equals_expected). Write criteria about t
• caliper_rubrics_delete Deletes data · ~205 tokens
  Deletes a rubric. Existing evals keep their snapshot of it and keep running; nothing new can bind to it. May return `needs_confirmation`.
• caliper_rubrics_get ~200 tokens
  One rubric with its criteria — each criterion's name, what the judge looks for, and the score scale. Read this to explain a score (which criterion slipped and what it asks for) or before caliper_rubrics_update. Get the rubricId from caliper_rubrics_list or an eval's rubricId.
• caliper_rubrics_list ~176 tokens
  Lists the rubrics in the active workspace (id, name, description, criterion count). Check here BEFORE caliper_rubrics_create — reuse an existing rubric's id in caliper_evals_create when one already scores the same job. Read a rubric's criteria with caliper_rubrics_get.
• caliper_rubrics_update ~753 tokens
  Changes a rubric's title, description, visibility, or replaces its criteria wholesale (pass the FULL list — criteria get fresh ids). Evals snapshot the rubric when they're created, so an existing eval keeps scoring with the criteria it started with; say so, and offer to create a new eval when the cr
• caliper_source_feeds_list ~197 tokens
  Datasets this source keeps adding to as conversations arrive: which dataset, for what (review, eval, spec), the pick it matches, sampling, how many have landed, and whether it's still running or why it stopped.
• caliper_source_feeds_stop ~249 tokens
  Stops a source from adding new conversations to a dataset. What it already added stays. Get the feedId from caliper_source_feeds_list. May return `needs_confirmation`.
• caliper_sources_list ~207 tokens
  Apps sending their agent's traces to Caliper — from their own code, OpenTelemetry, or a published Workbench flow — busiest first: name, how it sends, traces in the last 14 days (per day), and the tools its agent calls most with the share of traces using each. Start here when the user asks how their
• caliper_sources_over_time ~310 tokens
  One source day by day: traces, failures, tool calls, lookups that found nothing, tokens, cost, and speed (meanMs; p50/p95 as 'answered within' bucket edges); each tool with its calls and the share of each day's traces that used it; the documents lookups landed on most; models used. Use it to answer
• caliper_sources_to_dataset ~622 tokens
  Saves the conversations behind a pick (a day, a tool, a document, failures — the same narrowing as caliper_traces_list) into a new dataset (newDatasetName) or an existing one (datasetId), newest first up to `limit`. `purpose` decides what each item keeps: review = the agent's reply, tool calls inclu
• caliper_starters_install ~233 tokens
  Copies one starter into the workspace as a new dataset and a new rubric the user owns and can edit. Returns both ids; the next step is caliper_evals_create binding them to the flow (or target 'external'). Needs caliper:datasets:write and caliper:evals:write.
• caliper_starters_list ~202 tokens
  The starter sets Caliper ships: prompt injection, system-prompt leaking, over-refusal, personal-data handling, bias under ambiguity. Each is a small original dataset (single messages AND multi-turn build-ups: false memory, fabricated earlier turns, slow escalation) paired with an anchored rubric. Su
• caliper_traces_get ~256 tokens
  One trace: what came in, what went out, and every step in order — model calls (model, tokens, cost, reasoning), tool calls (arguments and result), lookups (query and documents with scores), anything else — each with timing and status. Long values are cut at 2,000 characters. Use it to explain why on
• caliper_traces_list ~337 tokens
  The traces behind a point on a source's chart, newest first, 50 a page: the question and answer (cut short), tools called, time, tokens, cost, status. Narrow by day, tool, document, or failures; page with `before` (the nextBefore from the last page). Open one with caliper_traces_get.
• comments_create Sends messages · ~403 tokens
  Posts a comment on a workspace entity — a new thread, or a reply when rootId is given. Use it to leave findings where the discussion already lives (an eval result on the flow being debated, a summary on a long thread). Mention people via mentionedUserIds (from workspace member ids) to ring their not
• comments_list ~287 tokens
  Lists the comment threads on one workspace entity (open first, then resolved) with authors and timestamps. Read this before weighing in on contested work — the threads are where disagreement lives before it becomes a decision.
• comments_resolve ~208 tokens
  Sets a comment thread's resolved state (rootId = the thread's root comment id). Resolve ONLY when the human asked or the thread's question is demonstrably settled — and say what settled it in a reply first. Reopening is for new evidence.
• entity_tags_browse Browses the web · ~241 tokens
  Without a tag: every tag in use across the workspace with how many entities carry it, most-used first — the vocabulary the team already organizes by. With a tag: everything filed under it across every tool, each with its kind, id, title, and path. Use it to reuse existing labels instead of inventing
• entity_tags_get ~340 tokens
  Returns the tags on a batch of entities of one kind — the labels galleries organize by. Ids come from the kind's list/get tool or from search_workspace. Use it before entity_tags_set so you replace the full set knowingly, and to answer 'what is this filed under'. Entities the user can't see are omit
• entity_tags_set ~409 tokens
  Replaces the FULL tag set on one entity (an empty list clears it). Read the current tags with entity_tags_get first and pass the merged list — this is not additive. Tags are lowercase letters, numbers, spaces, and hyphens; prefer labels already in use (entity_tags_browse) so the workspace's vocabula
• get_doc ~131 tokens
  Fetch the full Markdown body of a specific docs page by its slug. Use this after `search_docs` when the user needs the complete content of a page. No authentication required.
• list_docs ~128 tokens
  Enumerate all available docs pages, optionally filtered by product (e.g. 'compass', 'legal', 'overview'). Use this to discover what slugs exist before calling `get_doc`. No authentication required.
• search_docs ~208 tokens
  Search ZeroWidth product documentation. Returns matching pages with title, slug, public URL, and a query-relevant snippet. Use this when the user asks about a ZeroWidth product (Compass, Workbench, Caliper, Prism, Ledger, Napkin, the assistant), an API behavior, or a policy. No authentication requir
• search_workspace ~704 tokens
  Finds anything in the workspace in one call — uploaded files (searching the text inside them), folders, Workbench flows, shims, knowledge bases and tasks, Compass pages and changes, Caliper datasets, evals, rubrics, reviews, specs and sources, Ledger entries, Napkin sketches, decks, docs, sheets, di

Browse

• Website monitoring
• GitHub and code hosting
• Calling other AI models

Related MCP servers

• ZeroWidth Workbench
  Build, run and publish Workbench flows, tasks, shims and knowledge bases.
• ZeroWidth
  Search, write and run work across Compass, Workbench, Caliper, Ledger, Prism and Napkin.
• ZeroWidth Compass
  Read and update your Compass map: pages, changes, opportunities and interviews.
• AI PM Lab
  Learn how AI products work: lessons, model cost comparisons, real experiments, PM guides.
• Rubrkit
  MCP-native AI evaluation: rubric audits, eval suites, and proof reports for AI/LLM output.
• Apis Io
  Discover, rate and compare APIs and providers in the APIs.io catalog; manage a listing you own.
• CueFrame
  Turn footage into finished video: reframing, captions behind the subject, motion graphics, renders.
• ZeroWidth Ledger
  Log decisions with expectations, add evidence, and track metrics in Ledger.

Badge for your README

[![AptLynx](https://aptlynx.net/badge/ai.zerowidth/caliper.svg)](https://aptlynx.net/servers/ai.zerowidth/caliper)

Listing text is written by the maker; AptLynx lists tools and does not vouch for them. Find another tool · Browse by topic · For agents