Jev computer use, visual experiments and game agents

How a browser or desktop loop works

environment -> text/structured observation + legal actions
            -> typed Jev decision -> controller executes
            -> new observation -> next decision

The official model documentation describes text input. A browser can provide DOM or accessibility information; desktop software can use OCR or another perception component. Those representations, their freshness and the supplied action set determine what the decision model can judge.

Jev Ultrafast documents indexed operations and elements, with a separate LLM generating text when an operation needs typing. typesafe-computer-use is another community implementation with platform-specific capture and control. Inspect the current repository before assuming a platform or permission is supported.

Jev Visual is an independent model experiment

Jev Visual uses a Qwen multimodal model with MLX on Apple Silicon. Its README explicitly says it does not reproduce TypeSafe's proprietary architecture, training, calibration or serving system. Its image input belongs to that local implementation.

When comparing visual demonstrations, identify the actual perception model, decision backend and controller. Otherwise an observed improvement can be attributed to the wrong component. A local vision model also has different hardware and numerical behavior from a hosted text endpoint.

Voice control still needs a transcription stage

A voice browser or desktop assistant needs speech recognition to turn audio into text before using the documented Jev API. A language model or deterministic formatter may produce output text, and a speech system may read that text aloud. Jev's role can be to choose an action or judge intent within that chain.

Evaluate transcription mistakes, action selection and successful execution separately. A correct decision about an incorrect transcript is still a failed user task.

Doom, Pokemon, Snake and Mario experiments

jev-doom-agent documents a Chocolate Doom WebAssembly environment, structured state and a controller translating selected tactical actions into game controls. Its comparison distinguishes a connected Jev policy from an offline policy.

jev-plays-pokemon-red describes reading emulator memory and presenting legal moves as candidates. Other game projects explore Snake, Mario and related environments. Check whether a demo uses the actual engine, a simplified simulator, a prerecorded run or an explicitly labeled mock.

Compare agents on the same starting states and controllers. Record completion rate, invalid actions, recovery behavior and decision-to-action latency. A recorded success or a dramatic speed claim does not describe every level, state or game.

Robot experiments add a control and physics layer

EmbodiedJev and Jev LIBERO describe simulated observation, decision and execution workflows. Their environment, perception and motion control are distinct from the typed model judgment.

For a robotics experiment, inspect observation quality, action constraints, stopping conditions and the fidelity of the simulator. Simulation outcomes do not establish physical-world reliability. Keep motion and environment constraints in the controller rather than asking a model score to enforce them.