ASSAY-001: independent, pre-registered verification of TypeSafe Jev's calibration and type-safety claims. Split verdict, published in full.
Stars measure the whole repository, including work unrelated to Jev.
Documented records the linked documentation or source. JevHunt has not independently run or benchmarked this project.
**Verdict:** on CLINC150, Jev's chosen-option probabilities were calibrated (ECE 0.0204); on Banking77 they were not (ECE 0.0936, systematically overconfident). Across 8,576 responses there were zero type errors. Full write-up: https://donttrustme.ai/assay-001.html
Evidence commit: b7f71053a88025a5af80af00801c5ce725c9547d
Discovered through: github-search.
How we review →