MCP server

LLM Eval Harness

io.github.razilzul/llm-eval-harness

Checked 26 Oct 2026, 08:31 UTC · Checked against OSV.dev for malware and vulnerabilities; never run by us

Run a golden set and gate LLM model swaps on accuracy, with latency and cost per request.

Source

At a glance

Type

MCP server

Publisher

Not verified

Status

Runs on your computer

Sets up in

Claude Code, Claude Desktop, Codex, Gemini CLI, Cursor, VS Code

Last checked

26 Oct 2026, 08:31 UTC

Connect it to your agent

Add to CursorAdd to VS CodeVS Code Insiders

Opens the app, which shows the server and asks you to confirm. It then runs the package on your computer with your permissions. For other apps, copy the setup below.

Claude Code

claude mcp add llm-eval-harness -- docker run -i --rm ghcr.io/razilzul/llm-eval-harness:1.0.0

Claude Desktop

{"mcpServers":{"llm-eval-harness":{"command":"docker","args":["run","-i","--rm","ghcr.io/razilzul/llm-eval-harness:1.0.0"]}}}

Codex (config.toml)

[mcp_servers.llm-eval-harness]
command = "docker"
args = ["run","-i","--rm","ghcr.io/razilzul/llm-eval-harness:1.0.0"]

Gemini CLI

{"mcpServers":{"llm-eval-harness":{"command":"docker","args":["run","-i","--rm","ghcr.io/razilzul/llm-eval-harness:1.0.0"]}}}

Cursor

{"mcpServers":{"llm-eval-harness":{"command":"docker","args":["run","-i","--rm","ghcr.io/razilzul/llm-eval-harness:1.0.0"]}}}

VS Code

{"servers":{"llm-eval-harness":{"type":"stdio","command":"docker","args":["run","-i","--rm","ghcr.io/razilzul/llm-eval-harness:1.0.0"]}}}

Runs on your computer with your permissions: check the package and its source before installing.

Browse

• GitHub tools
• GitHub and code hosting
• Calling other AI models

Related MCP servers

• Nonobench
  An open-source benchmark of how well LLMs solve nonogram puzzles, from 5x5 to 20x20.
• Agent Broker
  23 MCP tools: compliance, verification, messaging, booking, US contracts. 15 need no key.
• Jean — Molecular Biology & Genomics
  Validated molecular biology tools: cloning, primers, CRISPR, Sanger, alignment, qPCR, phylogenetics
• AI PM Lab
  Learn how AI products work: lessons, model cost comparisons, real experiments, PM guides.
• Rubrkit
  MCP-native AI evaluation: rubric audits, eval suites, and proof reports for AI/LLM output.
• ScoreCompute
  38 MCP tools: Rust computation, Discovery, composition, counterexample scans and signed checks.
• TheProtocol — Sovereign AI Agent Platform
  Sovereign AI agent economy: identity, payments, governance, exchanges. 400+ tools, ?mode=gateway.
• Moretes Architect MCP
  Evidence-first architecture workbench: 48 deterministic tools, 10 architect personas, any cloud

Badge for your README

[![AptLynx](https://aptlynx.net/badge/io.github.razilzul/llm-eval-harness.svg)](https://aptlynx.net/servers/io.github.razilzul/llm-eval-harness)

Listing text is written by the maker; AptLynx lists tools and does not vouch for them. Find another tool · Browse by topic · For agents