Model Evaluation
Authorship and approval
[Written with AI] [Approved by a human]
Updated
28.10.2026
Model evaluation (evals) is the systematic measurement of whether an AI model or built-in AI application works as intended: a set of questions with expected outcomes, evaluation rules, and a final score. Instead of "it seems to work," you get "passed 52 out of 52 tests."
0.05→0.62
human alignment
52/52
chat tests
31.1%
papers on evaluation (wk 43)
Types of Evaluation
- Public benchmarks: the same tasks for all models. Good for comparisons, poor at reflecting your specific use case.
- Custom tests: questions from your domain and rules on what constitutes a good answer. These tell you whether the chatbot is ready for your customers.
- Red teaming: hundreds of attacks to find where the application breaks.
- Human alignment: whether the automated evaluator scores the way an expert would.
How does our lab do it?
- Chat tests with hard rules (promptfoo): questions about knowledge, abuse, attacks, and conversation. Rules are verified by code, without an LLM-judge, making the results reproducible: 52/52 (27.10.2026).
- Red team: 512 single-message attacks and 36 multi-turn conversations, zero leaks.
- Chat threshold from measurement: 30 in-domain and 6 out-of-domain questions established a threshold of 0.764.
- Vector model selection via measurement: 18 questions, relevant text in the top five 18 out of 18 times.
- Human-aligned source review: we compared our automated daily top-text selector with 15 human evaluations. Old weights yielded 0.05 alignment (practically random, favoring paper abstracts). After weight adjustments: 0.62.
How to get started yourself?
- Gather 20–30 real customer questions and 5–10 cases where the chatbot should refuse.
- For each, write a rule: must include a source, must state specific information, cannot include a link.
- Run tests before every change to the prompt, model, or knowledge base.
- Add red teaming before deployment, not after the first incident.
Related terms
Prompt injection · AI Hallucinations · AI Sycophancy · RAG
Questions and answers
What is the difference between a benchmark and an evaluation?
A benchmark is a ready-made, public test suite. Evaluation is a broader process where a benchmark may be just one component alongside custom tests.
Can evaluation be performed by another AI model?
Yes (known as LLM-as-a-judge), but such a judge makes mistakes too. What can be checked with code, check with code.
How many tests are enough?
To start, 30–50 well-chosen cases matter more than 1,000 random ones. It is more important to run them on every change.
Sources
- promptfoo: documentation for LLM application testing and red teaming
- Zheng et al.: Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena, arXiv:2306.05685
- Lab measurements: Reviewer alignment with human evaluation (23.09.2026), chat tests (21–28.10.2026)
Next