What a Jev search layer should decide
A Jev search layer sits after retrieval and before generation. Give it a bounded set of passages, files or lines and ask the relevance or answerability question your application can act on. Keep the search index, source identifiers and final response in deterministic code.
Retrieve candidates before judging them
A search engine, lexical index, embedding retriever or repository scan supplies candidate material. Jev can then judge those candidates against the user's question. If the relevant passage never reaches the candidate set, a better reranker cannot recover it.
query -> candidate retrieval -> Jev relevance/evidence checks
-> selected passages with source IDs -> answer model or reviewThe official RAG passage cookbook checks relevance, usable evidence, contradictory premises and attempts to instruct the answering model. Its design separates supporting material from conflicting evidence before a language model writes the answer.
Which Jevgrep project do you mean?
dzhng/jevgrep is a CLI for finding relevant code and source context by asking what the code does. Other repositories also use the name Jevgrep, including tools for logs or different code-search workflows. Match the owner and package name before installing one.
For code search, inspect whether results preserve paths, source excerpts and line references, how the candidate scan works, and which files leave the machine for a model call. A tool that runs locally may still use a hosted inference provider.
Ask about relevance and answerability separately
A passage can discuss the right topic without answering the question. A Choice can select a preferred candidate from a supplied set, but it still selects an option when every candidate is weak. Add an appropriate fallback or separate evidence check rather than treating the top-ranked result as proof.
The official reranking and line-by-line search examples show related decision patterns. Use source IDs in the state so your code can map a judgment back to the original material without asking the model to invent a citation.
Assemble context with provenance
Retain the source location, revision or retrieval timestamp alongside each selected passage. Keep contradictory evidence visible when it changes the answer. Separate retrieved content from instructions and test passages designed to redirect the system.
If the selected evidence is insufficient, return a review or insufficient-evidence outcome. A model can confidently select among bad candidates; a confident answer does not repair missing sources. JevRAG is one community project exploring decision boundaries such as evidence sufficiency and context selection.
The same pattern beyond documents
Emoji discovery, paper filtering and semantic log selection use a similar shortlist-and-judge pattern, but their ground truth differs. A relevant paper may not contain a requested result; an emoji match is a preference judgment; a log flag can trigger an operational action. Define the outcome for each task rather than carrying thresholds between them.
Evaluate the pipeline in layers
- Retrieval recall: did the candidate set contain an acceptable source?
- Relevance quality: did the judgment promote the right candidates?
- Answerability: did unanswerable questions avoid unsupported answers?
- Grounding: did the final response match the retained evidence?
- Operational cost: how much did retrieval, judgment, generation and retries consume?
Test fresh questions, ambiguous wording and conflicting sources. Keep the candidate pool constant when comparing rerankers, and record model and rubric versions. That makes a useful comparison possible without attributing every improvement or failure to the decision model.