Research and benchmarks — page 16

An independent benchmark of TypeSafe's Jev on 868 real decisions from a public repo, with mechanical (non-model-generated) labels.

Jev (TypeSafe AI) vs LLMs: benchmark + demo web ao vivo comparando latencia, custo e concordancia

Learning TypeSafe AI's Jev evaluation model fast by building a /play request router for a DJ chatbot

LLM-as-a-judge evaluation harness: a confidence-gated cascade (fast typed judge → Claude) that scores LLM outputs and gates CI on a regression diff. TypeScript, real recorded transcripts, real benchmark numbers.

Eval harness for TypeSafe Jev decision models: golden-dataset evaluation, calibration metrics, and schema-guarantee tests for typed decisions (Choice / Score / Noul)

Standalone TypeSafe Jev evaluations on 2026 Korean CSAT and SAT Practice Test #11

Local playground for decision models in the shape of TypeSafe's Jev, running the open openjev replication (Qwen3.5 as an NLI cross-encoder). CPU-only, no API key.

Measures whether TypeSafe's Jev decision model flips actions on identical replays near shipped thresholds across 12,705 calls. Replication data, harness, and independent verification.

Shadow-mode eval of TypeSafe Jev as an AIOps triage router vs Qwen3.8-27B and Claude Sonnet 5

Pluggable, observable RAG MCP server — hybrid retrieval (BM25+dense+RRF), BEIR-benchmarked rerankers (BGE / TypeSafe Jev / LLM), Streamlit dashboard + Langfuse, multimodal ingestion

Ultra-fast, sub-100ms API & webhook guardrail and triage gateway powered by TypeSafe AI System One (Jev). Parallel 7-dimension speculative evaluation, deterministic policy router, zero-dependency SQLite audit trail, and automated outbound dispatch.

S1Rank: Can a System-One decision model (TypeSafe Jev) rerank? Benchmark, raw responses, and paper.

Strands agents that make small local LLMs (Qwen3 0.6B, MiniCPM5 2B, Qwen3.5 4B) answer like Jev, a System 1 model: typed yes/no, choice and score answers with probabilities. Benchmarked against Jev on the jevals suite, via logprob readout and written-out JSON probabilities.

Deterministic evaluation harness for LLM agents, with a calibrated judge (Jev) that can act as a CI gate.

results from iterative benchmark runs

Jev is actually not a traditional LLM, it doesn’t generate text. It’s what the TypeSafe AI team calls a System One model: 📖 System One models are a class of AI models built to make fast, structured decisions that software can use directly. A System One model evaluates a state and returns typed answers and probabilities.

Evidence-aware semiconductor alert triage research demo with TypeSafe Jev, policy guards, and reproducible evaluation

JEV evaluation (FM-JEV-01): replay-only, advisory-only evaluation of Foreman-style supervision with TypeSafe Jev. Measures how much safety comes from the model versus a deterministic evidence gate. No worker authority.

This repository contains a GitHub issue classifier built on Jev, TypeSafe AI's System One model. It labels every new issue with typed values and calibrated confidence in milliseconds, labelling what it is sure about and escalating what it is not. Three guardrail layers guard every write, and a frozen eval suite gates each deploy.

Alert triage with Jev (TypeSafe System One): proof of concept and benchmark against a rules-only baseline

Clean-room CLI for TypeSafe System One / Jev Decision Contracts, search, and benchmarks.

typesafe.ai Jev(jev-1.13.0) benchmarked against the 2026 수능 국어영역 — JSON passages/questions + scored results

Production evaluation and gateway tooling for Jev and System One-compatible decision models

Decision Arena: TypeSafe's Jev vs open-source Laya playing highway-env, Snake and Blackjack with zero training, plus benchmarks and a Claude Code watchdog