Preset Evals

You can use our preset evaluators to add evaluations to your dev stack rapidly.

Here are our preset evaluators:

RAG Evals

These evals are useful for evaluating LLM applications with Retrieval Augmented Generation (RAG).

• Context Contains Enough Information
• Does Response Answer Query
• Response Faithfulness
• Groundedness

RAGAS Evals

RAGAS is a popular library with state-of-the-art evaluation metrics for RAG models:

• Context Precision
• Context Relevancy
• Context Recall
• Faithfulness
• Answer Relevancy
• Answer Semantic Similarity
• Answer Correctness
• Coherence
• Conciseness
• Maliciousness
• Harmfulness

Safety Evals

These evals are useful for evaluating LLM applications with safety in mind.

• PII Detection: Will fail if PII is found in the text
• Prompt Injection: Will fail if any known Prompt Injection attack is found the text. Learn more about Prompt Injection.
• Maliciousness: Measures maliciousness of the response
• Harmfulness: Measures harmfulness of the response

Summarization Evals:

These evals are useful for evaluating LLM-powered summarization performance.

• Summarization Accuracy

Custom Evals

These evals can help you create custom evaluation conditions.

• Grading Criteria
• Custom Prompt

Function Evals

Unlike the previous evaluators which used an LLM for grading, function evals do not use an LLM - they just use simple functions. For example the evaluator checks if the

• text matches a given regular expression
• text contains a link
• text contains keywords
• text contains no invalid links
• text is missing keywords
• and more

Head over to the function evaluators page for further details.

Evals with ground-truth

So far all the previous evaluators do not compare the response against any reference data. Following are evaluators that can compare the LLM generated response against the expected_response or context. For example

• Answer Similarity
• Context Similarity

Head over to the grounded evaluators page for further details.