How do you judge whether a model answers well: evals for beginners

This article belongs to module 2 (Using a model) of the learning track. Anyone who starts building applications around large language models quickly discovers a persistent bottleneck: how do you objectively determine whether the generated answers are actually of high quality? Where traditional software delivers binary outcomes (the code compiles or fails), language models work with probability distributions and variable phrasing. A prompt change that fixes one specific problem can unknowingly break three other use cases.

We call the systematic testing of language model outputs evaluations, abbreviated in the field to evals. Without automated evaluations, prompt engineering remains a matter of guesswork. In this article, we cover the full methodology behind evaluations: from simple deterministic asserts to deploying advanced model judges (LLM-as-a-judge). We discuss how to build reliable test sets, which statistical metrics are relevant, and how to avoid common measurement errors.

Why manual testing ('vibe checks') fails in production

When developers write a new prompt or switch to a different base model, they often test it by manually entering three to five examples into a chat window. If the answer looks convincing, polite, and grammatically correct, the temptation arises to push the change straight to production. In practice, this approach is known as the vibe check.

The fundamental problem with manual checking is that language models operate in an infinite input space. A model can perform excellently on friendly, straightforward questions, but derail completely as soon as a user gives contradictory instructions, enters missing parameters, or uses a different dialect. In addition, human fatigue quickly sets in during manual inspection: subtle errors in date formats, missing legal disclaimers, or hallucinated facts are easily overlooked after fifty checks.

Moreover, manual testing makes regression testing impossible. In classic software development, a test suite (such as unit tests and integration tests) ensures that new functionality doesn't break existing code. With LLM applications, you need exactly the same safety net. Without a fixed evaluation suite, you can never be sure that tightening the system instruction doesn't cause a regression in earlier scenarios.

The three levels of evaluation

To set up a scalable evaluation strategy, we categorize tests based on their complexity, execution speed, and cost. We distinguish three complementary levels:

A robust evaluation pipeline never relies on just one of these layers. The trick is to use cheap, deterministic rules as the first filter and only bring in a heavier language model as a judge for more complex quality criteria.

Level 1: Deterministic and heuristic checks

Deterministic checks require no artificial intelligence; they are traditional pieces of program code that check whether the output meets hard structural conditions. This is the most cost-efficient way to immediately reject faulty generations.

Think of concrete validations such as:

• JSON and schema conformity: Can the generated text be parsed as JSON without errors, and does the object contain all required fields with the correct data types?
• Presence of required elements: Does the output contain necessary sentences, such as a warning or a specific reference number?
• Blocklists (Negative matches): Does the response contain forbidden words, competitor names, or leaks of internal system prompts?
• Length and formatting limits: Does the summary stay strictly under a maximum number of words or paragraphs?

Below is an example of a simple Python evaluation script that checks whether a generated customer service response meets strict basic rules:

import json
import re

def evalueer_deterministisch(output_tekst: str) -> dict:
  resultaten = {
    "is_valide_json": False,
    "bevat_ticket_id": False,
    "geen_verboden_termen": True,
    "lengte_binnen_marge": False
  }
  
  # 1. Valideer JSON-structuur
  try:
    data = json.loads(output_tekst)
    resultaten["is_valide_json"] = True
    
    # 2. Controleer op ticket-ID patroon (bijv. NL-12345)
    bericht = data.get("antwoord", "")
    if re.search(r"NL-\d{5}", bericht):
      resultaten["bevat_ticket_id"] = True
      
    # 3. Zwarte lijst controle
    verboden = ["interne database", "geheim", "systeemprompt"]
    if any(woord in bericht.lower() for woord in verboden):
      resultaten["geen_verboden_termen"] = False
      
    # 4. Lengtecontrole van het antwoord (tussen 20 en 200 woorden)
    woord_aantal = len(bericht.split())
    if 20 <= woord_aantal <= 200:
      resultaten["lengte_binnen_marge"] = True
      
  except json.JSONDecodeError:
    resultaten["is_valide_json"] = False

  return resultaten

When a model fails one of these basic checks, you don't need to incur further costs on deeper semantic inspection. The response can be immediately flagged as invalid.

Level 2: Semantic evaluation and embedding distances

Not every answer can be captured by an exact regex. When a model answers a customer's question, there are hundreds of ways to correctly phrase the same message. At this stage, we look at semantic similarity.

Historically, n-gram metrics such as BLEU (Bilingual Evaluation Understudy) and ROUGE (Recall-Oriented Understudy for Gisting Evaluation) were used for this. These metrics count how many words or word combinations from the reference text appear exactly in the generated text. While useful for translations, they fall short for creative answers: two sentences with exactly the same meaning but completely different synonyms score dismally low on ROUGE.

The modern approach makes use of vector embeddings. Here, we convert both the model's answer and a pre-established 'golden answer' into a high-dimensional vector. We then calculate the cosine similarity between the two vectors. If the similarity is above a certain threshold value (for example 0.88), we consider the meaning equivalent. Watch out for the pitfall: a negation ("The store is not open on Sunday" versus "The store is open on Sunday") has a high cosine similarity at the embedding level, while the actual meaning is diametrically opposed.

Level 3: LLM-as-a-Judge

For complex judgments — such as factual correctness, empathy, logical coherence, or instruction fidelity — a language model with advanced reasoning capabilities is itself the best judge. We call this principle LLM-as-a-Judge.

Here, we feed an independent judging model (the 'judge') with the original user input, the generated answer, any context documents, and an explicit scoring rubric. The judge is instructed to first reason step by step and only afterward render a structured verdict (such as a score from 1 to 5 or a binary 'PASS'/'FAIL').

Below is an example of an effective judging prompt for factual accuracy:

Je bent een neutrale en strikte examinator die antwoorden van een AI-systeem beoordeelt.

[BRONCONTEXT]
{context}

[GEBRUIKERSVRAAG]
{vraag}

[GEGENEREERD ANTWOORD]
{antwoord}

Beoordeel of het gegenereerde antwoord feitelijk volledig wordt ondersteund door de broncontext.
Volg strikt deze stappen:
1. Identificeer alle feitelijke beweringen in het gegenereerde antwoord.
2. Controleer voor elke bewering of deze expliciet terug te vinden is in de broncontext.
3. Noteer eventuele tegenstrijdigheden of niet-ondersteunde aannames (hallucinaties).
4. Geef je eindoordeel in JSON-formaat.

Formaat:
{
  "redenering": "Stapsgewijze analyse...",
  "bevat_hallucinaties": true/false,
  "score": 1 tot 5,
  "oordeel": "PASS of FAIL"
}

In the guide on becoming an AI agent engineer in 2026 you'll discover how automated evaluations play a central role in building robust software systems and autonomous agents.

Assembling a golden test set

Without a representative test dataset (often called the golden dataset ) evaluations are worthless. A good test set reflects the actual diversity of production traffic and contains at least a hundred to five hundred examples.

A balanced test set consists of four pillars:

• Standard cases (Happy path): Common, clear questions for which an unambiguous answer exists.
• Edge cases: Questions with missing information, extremely long texts, or ambiguous phrasing.
• Contradictory and malicious input (Adversarial): Attempts at prompt injection, inappropriate requests, or requests that fall outside the application's domain.
• Language-specific characteristics: Specific to Dutch-language applications: input with compound words, abbreviations, typos, and regional terminology.

Let's look at a concrete Dutch case: a customer service assistant for a Dutch health insurer. Instead of generic examples, you test situations such as:

Statistical metrics and quantification

Once your evaluation set runs across hundreds of examples, you analyze the aggregated results with concrete metrics. Depending on the application, you use different metrics:

1. Pass Rate (% of tests passed): The percentage of test cases that meet all binary conditions. This is the most common indicator in continuous integration (CI/CD) pipelines.

2. Precision, Recall, and F1 score: Crucial for classification tasks or information extraction. Precision measures how many of the extracted entities were correct. Recall measures how many of all the entities present were actually identified.

3. Pass@k: This metric is widely used in code generation and reasoning tasks. If we have the model generate $k$ answers for the same question, what is the probability that at least one of them is fully correct? This provides insight into the model's generative potential under different sampling settings.

4. Win rate in A/B comparison: When you weigh two prompts or two models directly against each other, you let a judge determine which response is superior. Read the guide on A/B testing of prompts to see how you systematically pit two prompt variants against each other in a live test environment.

Pitfalls and blind spots in automated evals

Although LLM-as-a-judge is powerful, it introduces its own structural measurement errors. Anyone who doesn't account for this ends up optimizing for a skewed reality. The most important pitfalls are:

• Position bias: When a model has to compare two answers (A and B), it often has a strong preference for the answer presented first. You solve this by running every comparison twice with the order swapped (A/B and B/A) and only awarding a win if the model remains consistent.
• Length bias (Verbosity bias): Language models systematically rate longer, more elaborate answers as higher quality than concise ones, even when the extra text is pure padding. You prevent this by explicitly stating in the rubric that conciseness and directness are rewarded.
• Self-preference (Self-enhancement bias): Models from a particular vendor often give a higher score to text generated by a model from the same family. Preferably use a different type of model as the judge than the model you're testing.

Integration into the development cycle

Evaluations only deliver real value once they form a fixed part of the development cycle. This means that with every change to the prompt, the context input, or the underlying model, the evaluation suite is run automatically.

By setting threshold values (for example: a pull request may only be merged if the pass rate is at least 95% and no single safety check fails), you create reproducibility. This transforms prompt development from an unpredictable tinkering process into a full-fledged software engineering discipline.