Evaluator basics
Evaluators evaluate the test results that you submit from your generative AI pipelines and score them on a scale that you specify.
Creating evaluators
Create an evaluator by selecting "Evaluators" underneath the relevant pipeline and then selecting "New evaluator".
Evaluator scoring
Evaluators must define a scoring method. Gentrace provides two different ways to define scoring.
Enum scoring
In enum scoring, the evaluator output must map options to percentage values, which are used to calculate the final score of evaluations. All enum options must have a percentage mapping. We compute the final score of a series of evaluations by using the associated percentage.
Percentage scoring
In percentage scoring, the evaluator output must output a value between 0 and 1, inclusive.
Evaluator types
Gentrace has three evaluator types. Read more about them at the links below.