JJevHunt
LaunchesProjectsTasksCategoriesCollectionsGuidesFAQEvidenceUpdatesSubmit
JevHunt / Tasks / Evaluation & benchmarks

Evaluation & benchmarks — page 13

Evaluation harnesses, benchmarks and graders. Jev scores or chooses between candidate outputs with an explicit rubric.

320 active projects match this task, ordered by traction.

← Previous Page 13 of 14 Next →
J

jev-hft

cristiancolon/jev-hft
★ 0

Research pipeline testing whether TypeSafe's Jev (via Vercel AI Gateway) can judge news and market data fast enough to matter. Bitcoin and US stocks, paper trading only.

VerifiedJev applicationTypeScriptupdated 2026-09-25
Details →GitHub ↗Documented ↗
J

jev-ja-eval

uesgugikouhei-oss/jev-ja-eval
★ 0

Evaluate Jev (TypeSafe AI) on Japanese customer-inquiry data: 150-item dataset + comparison script (Jev / LLM / rule-based)

VerifiedResearchPythonupdated 2026-09-19
Details →GitHub ↗Documented ↗
J

jev-lab

dairui1/jev-lab
★ 0

Experiments with TypeSafe's Jev: triage benchmark vs LLM, and a source study of jev-ultrafast vs Cline's jev-browser

VerifiedResearchHTMLupdated 2026-09-19
Details →GitHub ↗Documented ↗
J

jev-latam-lead-triage

integralmarketingmx/jev-latam-lead-triage
★ 0

Triage de leads de WhatsApp/CRM con Jev (TypeSafe AI): ruteo por confianza, plantilla n8n y benchmark en español. Sin dependencias. No afiliado.

VerifiedResearchPythonupdated 2026-09-19
Details →GitHub ↗Documented ↗
J

jev-test

souvikr/jev-test
★ 0

Test harness + benchmark for TypeSafe's Jev decision model (noul/choice/score) via OpenRouter's Decisions API

VerifiedResearchPythonupdated 2026-09-19
Details →GitHub ↗Documented ↗
J

jev-tetris-benchmark

planstack-ai/jev-tetris-benchmark
★ 0

Reproducible Tetris decision benchmark comparing TypeSafe Jev with Claude Haiku

VerifiedResearchTypeScriptupdated 2026-09-19
Details →GitHub ↗Documented ↗
J

judge

Illusion47586/judge
★ 0

Probabilistic judgment. Deterministic execution.

Jev applicationTypeScriptupdated 2026-09-19
Details →GitHub ↗Documented ↗
J

judge-jev

alexhawat/judge-jev
★ 0

Jev-native LLM output judge kit (skill + agents + core)

Jev applicationPythonupdated 2026-09-22
Details →GitHub ↗Documented ↗
L

live-rubric

jose-troche/live-rubric
★ 0

A writing editor that re-scores 15 typed rubric dimensions on every typing pause — one Jev System One call per pause, ~$0.000004 each.

VerifiedJev applicationTypeScriptupdated 2026-09-20
Details →GitHub ↗Documented ↗
M

meridian-demo

MoRohn/meridian-demo
★ 0

Meridian | AI context-intake and contraction risk & compliance micro-app with live TypeSafe AI vs OpenAI evaluation

VerifiedResearchTypeScriptupdated 2026-09-21
Details →GitHub ↗Documented ↗
P

pi-jev-skill-bench

iamdin/pi-jev-skill-bench
★ 0

Benchmark for Pi skill routing: BM25 vs TypeSafe Jev across roster sizes 50–500

VerifiedResearchTypeScriptupdated 2026-09-20
Details →GitHub ↗Documented ↗
S

system-one-benchmark

laidick/system-one-benchmark
★ 0

No project description available.

Local alternativeupdated 2026-09-19
Details →GitHub ↗Documented ↗
T

typed-judge-kit

Ascurse/typed-judge-kit
★ 0

Typed questions to a model, verdict in code, thresholds from your labels

VerifiedJev applicationPythonupdated 2026-09-22
Details →GitHub ↗Documented ↗
G

git-judge-jev

davidrydberg/git-judge-jev
★ 0

GitHub Action for agent-written PRs: one comment that says what the PR really does, which hunks need a human, why, and what to verify. About a cent per PR, on TypeSafe Jev.

VerifiedJev applicationTypeScriptupdated 2026-09-20
Details →GitHub ↗Documented ↗
J

jev-horingssvar-eval

EmilLindfors/jev-horingssvar-eval
★ 0

Early-access test of TypeSafe's Jev on 24 Norwegian hearing responses, next to DeepSeek V4.1 Flash

VerifiedResearchPythonupdated 2026-09-18
Details →GitHub ↗Documented ↗
J

jev-swap

HomenShum/jev-swap
★ 0

Claude Code skill: swap System 2 LLM pipeline components for System 1 TypeSafe Jev decisions via investigation, live three-arm eval, fallback, and an independent judge

VerifiedResearchPythonupdated 2026-09-18
Details →GitHub ↗Documented ↗
L

local-skill-router

rockjoel/local-skill-router
★ 0

Local-first skill router for coding agents. Inspired by TypeSafe Jev; optional swappable judge (not an official Jev product).

VerifiedIntegrationPythonupdated 2026-09-24
Details →GitHub ↗Documented ↗
P

plexus

nshkrdotcom/plexus
★ 0

High-concurrency Elixir actor runtime for large-scale semantic graphs, search swarms, and batched model evaluation.

VerifiedResearchElixirupdated 2026-09-19
Details →GitHub ↗Documented ↗
S

smoking-extraction-benchmark

vclic/smoking-extraction-benchmark
★ 0

Synthetic smoking-history extraction benchmark comparing TypeSafe Jev and OpenAI structured outputs, with reproducible accuracy, cost, and latency results.

VerifiedResearchPythonupdated 2026-09-18
Details →GitHub ↗Documented ↗
T

typesafe-package

AutomationAnywhere/typesafe-package
★ 0

Automation Anywhere custom package wrapping the TypeSafe (Jev) API for typed yes/no, categorical, and scored-scale text evaluation.

VerifiedResearchJavaupdated 2026-09-24
Details →GitHub ↗Documented ↗
J

jev-gamebenchmark

aieo-product/jev-gamebenchmark
★ 0

Sandbox & benchmark: optimize how you ask Jev (TypeSafe System One) to play falling-block puzzle games, head-to-head against LLMs

VerifiedResearchPythonupdated 2026-09-18
Details →GitHub ↗Documented ↗
J

jev-judge-bench

cmartinez9/jev-judge-bench
★ 0

Binary LLM-judge bench — compare Jev (TypeSafe System One) against a frontier LLM judge on speed, cost, and agreement with human labels.

VerifiedJev applicationupdated 2026-09-18
Details →GitHub ↗Documented ↗
L

llm-routing-verification-bench

flintfromthebasement/llm-routing-verification-bench
★ 0

Portable benchmark for LLM tier routing, factual-claim verification, and safe interim messages

VerifiedResearchJavaScriptupdated 2026-09-29
Details →GitHub ↗Documented ↗
T

typesafe-ai_smoke-test

TyrellD1/typesafe-ai_smoke-test
★ 0

Smoke test: route prompts to a work or life database with TypeSafe AI (Jev), 30-case eval

VerifiedResearchHTMLupdated 2026-09-17
Details →GitHub ↗Documented ↗
← Previous Page 13 of 14 Next →

Related tasks

Browser automationDrive Chrome or Playwright with typed decisionsComputer & desktop useControl apps, screens and the mouseAgent routing & tool choicePick the next tool, model or branchClassification & labellingSort text, tickets and intentReranking & relevanceReorder search and retrieval resultsRAG & retrievalGround answers in your own data
Independent Jev ecosystem directoryPrivacyTermsSecurityEnglish简体中文繁體中文日本語한국어EspañolFrançaisDeutschPortuguêsРусскийहिन्दीBahasa IndonesiaTiếng ViệtTürkçeItaliano