Evaluation & benchmarks

RAG-Fusion generates query variants and merges retrieval rankings with Reciprocal Rank Fusion to find material a single query may miss. Jev is an optional reranker in the evaluation pipeline; its experiments require JEV_API_KEY. The original demo remains vector-only. Compare retrieval strategies on NFCorpus with the evaluation harness; the hybrid pipeline combines BM25, vector search, fusion and reranking.

Rizzo Flow runs open decision models locally, returning choices, probabilities, rubric scores and numbers through a Jev-compatible HTTP interface. It reproduces Jev’s interface pattern, not its proprietary architecture or training; the project is independent. Serve a selected model with llama.cpp and point TypeSafe-compatible application code at the local endpoint.

The collection supplies skills for record triage, evidence checks, rubric evaluation and choosing legal actions. Follow the installation guide, choosing real Jev through TypeSafe or OpenRouter, or an explicitly labeled agent simulation.

Awesome JEV is a gallery of System One research, open models, applications and independent evaluations, with links to code and papers. Use the official primitives and known-failure links for orientation, then follow model or evaluation entries to their original sources.

pg-jev filters, ranks and classifies PostgreSQL rows using plain-language SQL conditions. Batched Jev questions judge rows; SQL functions expose boolean, probability, choice and rubric-score results. Requires PostgreSQL 14–17, plpython3u, superuser access and a TypeSafe key.

Valen scores supplied candidates against text, images or video and returns decision probabilities without generating answer tokens. This Jev-inspired implementation uses a Qwen3.5 backbone and a shared decision head. Run inference or evaluation with both the Valen checkpoint and its Qwen3.5-2B base model.

System One Adapter implements the typesafe_sdk evaluation interface using other providers’ LLM APIs. It enables comparisons with TypeSafe while preserving typed answers and adding retry, latency and normalization diagnostics. Configure a provider, model and probability or discrete answer mode; native structured outputs are optional.

System One Connector exposes typed decisions to coding agents through its evaluate command and MCP integration. It connects to Jev or compatible self-hosted models such as CLM and Laya. Define Choice, Noul or Score questions; agent setup and batch evaluation are provided.

A candidate-scoring head returns distributions for gates, routes and scores without decoding prose or JSON answers. The README compares a benchmark-trained AgentJev specialist with Jev 1.13.0 evaluated zero-shot; these use different training conditions.

The repository offers open decision models for Jev-style Choice, Score and Noul workflows, including separate Verdict and Verdict 2.0 models. Serve webgpu-demo with Python’s HTTP server and open the documented browser playground; model artifacts and evaluation receipts are linked separately.

fingers combines several web fingerprint databases and can use Jev to review rule matches, deduplicate findings and refine versions. Install the Go library and enable the judge with EnableJudge and a TypeSafe key, or use the CLI’s jev-key option.

JevBench v1 - a benchmark for Jev-class typed decision models: smart, cheap, fast, reliable, open.

Leaderboard that puts Jev and 14 LLMs through 108 four-option multiple-choice questions about the Convex backend platform, comparing accuracy, latency, and cost, with a per-answer explorer showing Jev probabilities.

Agentic RL 中文零基础教程（33 章）：从概念到 GRPO 实战，含 TRL 最小可跑示例；26–33 章附一套可运行的三方判别模型实证工程（encoder vs LLM-LoRA vs 规则基线）。第 25 章讲清 Jev / TypeSafe System One 与 RL 的能力边界 | Chinese Agentic RL tutorial (33 chapters) + a reproducible discriminative-model benchmark

Non-autoregressive decision engine on ModernBERT (151M) with calibrated uncertainty (RLCD), TypeSafe AI Jev benchmark audit, and in-browser WebGPU playground

Personal-assistant agent built on Vercel's eve with 100 mocked tools, measuring how many steps it takes when Jev picks the tool versus the LLM.

Using Jev as an evaluator.

An open-source JEV-powered workbench for streaming data selection, quality evaluation, automatic LoRA training and held-out model evaluation.

JevGym is an open-source platform designed to benchmark and facilitate better probabilistic estimation in Jev-alike models

TypeSafe decisions for Pi: batched evaluation tool, terminal playground, and typed API for extension authors

Typed JSON inference with DiffusionGemma, with Every and Jev benchmark results

A database-agnostic Agentic GraphRAG framework using swappable System One models (local Laya / cloud Jev). A plug-and-play intelligence layer featuring a complete 4-phase pipeline, continuous evaluation and custom A* traversal for any graph database.

This is a LLM Gateway that mimics typesafe ai structured output. Like an imposter Jev.

Jev-powered, rule-based grader for text files. Runs every rule against every line in parallel. No skimming, no missed lines.