Skip to main content
Version: 3.0.3

Evaluator basics

Evaluators evaluate the test results that you submit from your generative AI pipelines and score them on a scale that you specify.

Creating evaluators

Create an evaluator by selecting "Evaluators" underneath the relevant pipeline and then selecting "New evaluator".

Create new evaluator

Evaluator scoring

Evaluators must define a scoring method. Gentrace provides two different ways to define scoring.

Enum scoring

In enum scoring, the evaluator output must map options to percentage values, which are used to calculate the final score of evaluations. All enum options must have a percentage mapping. We compute the final score of a series of evaluations by using the associated percentage.

Evaluator enum scoring

Evaluator enum template

Percentage scoring

In percentage scoring, the evaluator output must output a value between 0 and 1, inclusive.

Evaluator types

Gentrace has three evaluator types. Read more about them at the links below.