Agent tools / Agent evaluations
Score exact and normalized answer matches Compare caller-supplied actual and expected values for every case. Both fields are required; this deterministic comparison does not judge semantic correctness.
Status: implemented. Runtime: browser-or-local. Version: 1.0.0.
Keep deterministic scores separate from model judgments; show rubric, model version, evidence and uncertainty.
Supported profile This tool accepts the schema below. Its result includes data and versioned runtime metadata. Read warnings in returned data for algorithm coverage and assumptions.
Input schema {
"type": "object",
"properties": {
"cases": {
"type": "array",
"maxItems": 2000,
"items": {
"type": "object",
"properties": {
"actual": {},
"expected": {}
},
"required": [
"actual",
"expected"
]
}
},
"normalize": {
"type": "boolean"
}
},
"required": [
"cases"
],
"additionalProperties": false
} Output schema {
"$schema": "http://json-schema.org/draft-07/schema#",
"type": "object",
"properties": {
"data": {
"type": "object",
"properties": {
"scores": {
"type": "array",
"items": {
"type": "object",
"properties": {
"passed": {
"type": "boolean"
}
},
"required": [
"passed"
],
"additionalProperties": false,
"maxProperties": 2000
},
"maxItems": 2000
},
"accuracy": {
"anyOf": [
{
"type": "number",
"minimum": 0,
"maximum": 1
},
{
"type": "null"
}
]
}
},
"required": [
"scores",
"accuracy"
],
"additionalProperties": false,
"maxProperties": 2000
},
"metadata": {
"type": "object",
"properties": {
"tool": {
"const": "eval.exact_match"
},
"version": {
"const": "1.0.0"
},
"runtime": {
"type": "string",
"enum": [
"local",
"browser",
"server"
]
},
"elapsedMs": {
"type": "integer",
"minimum": 0,
"maximum": 9007199254740991
},
"retained": {
"const": false
}
},
"required": [
"tool",
"version",
"runtime",
"retained"
],
"additionalProperties": false,
"maxProperties": 2000
}
},
"required": [
"data",
"metadata"
],
"additionalProperties": false,
"$defs": {
"json": {
"anyOf": [
{
"type": "null"
},
{
"type": "boolean"
},
{
"type": "number"
},
{
"type": "string",
"maxLength": 200000
},
{
"type": "array",
"items": {
"$ref": "#/$defs/json"
},
"maxItems": 2000
},
{
"type": "object",
"maxProperties": 2000,
"additionalProperties": {
"$ref": "#/$defs/json"
}
}
]
}
}
} Example arguments {
"cases": [
{
"actual": " A ",
"expected": "a"
}
],
"normalize": true
} Local MCP tool name: eval__exact_match. Browser and local execution work without a key.
Hosted REST and MCP execution are disabled in this release. Run the example in the browser console below, or use the local MCP server from this repository.
Privacy and limits 256 KiB arguments, 512 KiB output, 20 nesting levels and 2,000 array entries. No input/output logging in this layer. Browser/local utilities do not submit inputs remotely. Optional network operations disclose destination requests. Caller-supplied policies and guardrail findings never grant execution privileges.
Parse and validate JSON json.parseRuns locally Validate JSON against a schema json.schemaRuns locally Convert JSON, CSV, YAML and XML json.convertRuns locally Compare JSON values json.diffRuns locally Query JSON with JSONPath json.queryRuns locally Repair malformed JSON with an edit log json.repairRuns locally Generate types from sample JSON json.typesRuns locally Profile CSV data data.profileRuns locally Join tabular datasets data.joinRuns locally Build a pivot table data.pivotRuns locally Normalize Unicode and whitespace text.normalizeRuns locally Compare text text.diffRuns locally Deduplicate lines text.deduplicateRuns locally Generate a URL slug text.slugRuns locally Extract URLs text.urlsRuns locally Convert HTML to plain text text.htmlRuns locally Convert Markdown to sanitized HTML text.markdownRuns locally Count tokens with a named tokenizer tokens.countRuns locally Estimate inference cost tokens.costRuns locally Pack context within a token budget context.packRuns locally Format supplied document citations document.citationsRuns locally Hash UTF-8 text file.checksumRuns locally Decode bounded text bytes file.encodingRuns locally Convert line endings file.line_endingsRuns locally Evaluate a bounded mathematical expression math.evaluateRuns locally Convert physical units units.convertRuns locally Compute descriptive statistics statistics.describeRuns locally Calculate a confidence interval statistics.intervalRuns locally Calculate date differences date.differenceRuns locally Calculate business days date.business_daysRuns locally Convert timezones date.timezoneRuns locally Explain a cron schedule date.cronRuns locally Calculate an agent deadline budget date.deadlineRuns locally Verify arithmetic claims math.verifyRuns locally Detect supported sensitive-data patterns privacy.detectRuns locally Redact supported sensitive text privacy.redactRuns locally Mask structured sensitive values privacy.maskRuns locally Redact timestamped transcripts privacy.transcriptRuns locally Scan for credential patterns privacy.secretsRuns locally Sanitize an agent log privacy.log_cleanRuns locally Create dataset-scoped pseudonyms privacy.pseudonymsRuns locally Evaluate a proposed data transfer privacy.egressRuns locally Minimize a payload by policy privacy.minimizeRuns locally Check an output for supplied canaries and secrets privacy.output_leakRuns locally Render a literal prompt template prompt.templateRuns locally Label trusted and untrusted prompt sections prompt.boundariesRuns locally Flag suspicious instruction patterns prompt.injection_scanRuns locally Find hidden or off-screen HTML instructions prompt.hidden_textRuns locally Inspect confusables and bidi controls prompt.unicode_riskRuns locally Inspect bounded encoded text prompt.encoded_payloadRuns locally Lint a tool description for instruction-like content prompt.tool_descriptionRuns locally Compare versioned tool manifests prompt.manifest_diffRuns locally Check messages against provider-independent role rules prompt.model_rulesRuns locally Generate approved injection test fixtures prompt.attack_fixtureRuns locally Preview a proposed policy decision policy.evaluateRuns locally Check an allowed tool set policy.tool_allowlistRuns locally Inspect lexical path scope policy.path_scopeRuns locally Validate an outbound domain against a grant policy.domain_scopeRuns locally Classify declared action effects policy.side_effectsRuns locally Prepare a reviewable action request approval.prepareRuns locally Preview a bounded write operation policy.dry_runRuns locally Compute a bounded retry schedule run.retryRuns locally Validate a structured execution trace run.trace_validateRuns locally Detect repeated unproductive calls run.loop_detectRuns locally Normalize tool errors for recovery run.error_normalizeRuns locally Lint an OpenAPI document api.openapi_lintRuns locally Detect breaking API contract changes api.compatibility_diffRuns locally Convert cURL into a fetch example api.curl_fetchRuns locally Generate a bounded schema example api.schema_exampleRuns locally Validate a request against a contract api.request_validateRuns locally Validate an API response contract api.response_validateRuns locally Lint an MCP tool manifest mcp.manifest_lintRuns locally Inspect JWT structure without trusting it api.jwt_inspectRuns locally Generate seeded synthetic records test.synthetic_recordsRuns locally Validate chat JSONL dataset rows test.jsonl_datasetRuns locally Evaluate regex cases with a deadline test.regex_casesRuns locally Generate schema boundary cases test.schema_edgesRuns locally Check numeric-array properties test.property_casesRuns locally Compare a result to a golden fixture test.golden_compareRuns locally Describe a synthetic fault fixture test.fault_fixtureRuns locally Find exact and normalized dataset duplicates test.dataset_duplicatesRuns locally Check train and test overlap test.split_leakageRuns locally Compare supplied artifact measurements test.artifact_invariantsRuns locally Score exact and normalized answer matches eval.exact_matchRuns locally Score tool selection eval.tool_choiceRuns locally Score tool arguments eval.argumentsRuns locally Score workflow completion constraints eval.trajectoryRuns locally Evaluate refusal and over-refusal eval.refusalRuns locally Score approved injection tests eval.injection_resilienceRuns locally Compare two agent configurations eval.paired_runsRuns locally Build a reproducible evaluation report eval.reportRuns locally Split text into bounded chunks rag.chunkRuns locally Compare supplied embedding vectors rag.semantic_compareRuns locally Extract frequency-based keyphrases rag.keyphrasesRuns locally Search supplied source records rag.retrieveRuns locally Rerank candidates by lexical overlap rag.rerankRuns locally Check that cited passages exist rag.citation_checkRuns locally Measure source coverage for listed claims rag.coverageRuns locally Check source age against a task policy rag.freshnessRuns locally Deduplicate retrieved sources rag.source_dedupeRuns locally Evaluate robots rules for a named crawler seo.robotsRuns locally Inspect sitemap URLs seo.sitemapRuns locally Validate page metadata seo.metaRuns locally Audit reciprocal hreflang seo.hreflangRuns locally Inspect JSON-LD blocks seo.structured_dataRuns locally Lint llms.txt and linked Markdown seo.llms_lintRuns locally Build campaign tracking URLs seo.utmRuns locally Generate canonical sitemap content seo.sitemap_buildRuns locally Check color contrast a11y.contrastRuns locally Inspect HTML heading order a11y.headingsRuns locally Check form label associations a11y.labelsRuns locally Generate a semantically labelled table a11y.tableRuns locally Suggest high-contrast palette fallbacks a11y.paletteRuns locally Run a bounded static accessibility audit a11y.static_auditRuns locally Audit a recorded keyboard flow a11y.keyboard_traceRuns locally Validate an artifact link bundle publish.link_integrityRuns locally Prepare a text bundle manifest publish.bundleRuns locally Test a semantic version range dependency.semverRuns locally Inspect a Content Security Policy security.cspRuns locally Compare dependency lockfiles dependency.lock_diffRuns locally Generate an inventory SBOM dependency.sbomRuns locally Inspect declared license allowlists dependency.licensesRuns locally Verify subresource integrity security.sri_verifyRuns locally Normalize static-analysis findings security.sarifRuns locally Scan a supplied code diff for configured risks security.diff_scanRuns locally Convert subtitle formats media.subtitle_convertRuns locally Validate subtitle timestamps media.subtitle_validateRuns locally Shift subtitle timestamps media.subtitle_shiftRuns locally Validate a bounded workflow graph workflow.validateRuns locally Preview a typed workflow workflow.planRuns locally Partition items into bounded batches workflow.batchRuns locally Plan remaining workflow steps workflow.resumeRuns locally Prepare a reviewable webhook delivery integration.webhook_prepareRuns locally Build a structured email draft integration.email_draftRuns locally Build a calendar-event draft integration.calendar_draftRuns locally Compare supplied connector scope lists integration.connector_scopesRuns locally Compare caller-supplied actual and expected values for every case. Both fields are required; this deterministic comparison does not judge semantic correctness.
Keep deterministic scores separate from model judgments; show rubric, model version, evidence and uncertainty.
Contract and limitations
Arguments (JSON) Run locally Cancel Reset example
Result Run a tool to see its structured result. Let a browser agent use this console Enable WebMCP when your browser supports it. Inputs run locally and results remain visible here.
Enable browser agent tools
Full integration guide