# Judix — Real-time evals for agents and RAG

Catch the wrong turn before your user does.

By Anuj Ojha. Status: Live · open source. Page: https://anujojha.com/work/judix/

## What it is

Judix scores AI agents and RAG systems while they run, step by step, so you find out an agent is going wrong in the turn it is taking, not in next week's report.

## The problem

Logs tell you what happened. Offline evals tell you about last week. Neither tells you that the agent is going wrong right now, and by the time you read either one, a user has already had the bad answer.

## How it works

A deterministic engine written in Rust scores every agent step and every RAG claim with zero model calls, so each score costs $0. It makes two passes over every step; a model is only asked afterwards, to explain the why. There are two front doors to one engine: a CLI for local runs and a streaming HTTP endpoint for live systems.

## The proof

It ships with its own worst case on the page. An agent told to book a table for two on Friday, avoiding downtown, searched downtown three times, then announced it would try downtown again. Judix scored that run 15 out of 100, capped at 59 by a critical failure, and flagged it across four separate steps.

## Where it is now

Live and open source at v0.1.0, with a public playground and no signup.

## Numbers

- $0: per score — zero model calls anywhere in the scoring path
- 4: steps flagged in that run
- 2: front doors — CLI and streaming HTTP — over one engine
- v0.1.0: current release · open source, no signup, live playground

## Said plainly

- 15 / 100: its own worst case, published on its own page: the downtown run, capped at 59 by a critical fail

Stack: Rust · CLI + HTTP

## Questions

**What is Judix?**
Real-time evaluation for AI agents and RAG. It scores every agent step and every RAG claim while the agent runs, so you find out an agent is going wrong in the turn it is taking.

**How can a score cost $0?**
The scoring engine is deterministic and written in Rust, with zero model calls anywhere in the scoring path. A model is only asked afterwards, to explain the why.

**How is Judix different from offline evals?**
Logs tell you what happened and offline evals tell you about last week. Judix scores the turn the agent is taking right now.

**How do I use it?**
Through a CLI for local runs or a streaming HTTP endpoint for live systems. It is open source, with a public playground and no signup.

Links: [Site](https://judix-8piu.onrender.com), [Repo](https://github.com/Anuj7411/judix)