Automatic Evals

What is an automatic eval?

If you are logging inferences to Athina, you can configure evals to run automatically against logged inferences.

Eval results will show up on your dashboard as soon as they start running.

• The metrics from these evals will be used to calculate the model performance metrics.

• The evaluation results for each individual inference are visible in the Evals tab on the Inference Trace page (/observe/inference/:inference_id).

Tutorial: Set up automatic evals in 15 seconds

Here's how you can configure evals on Athina Dashboard (opens in a new tab) in < 15 seconds to run automatically against logged inferences.