Evaluator and processor templates are saved templates that can be shared amongst pipelines, users, and teams.
Manage them on the templates page and then use them when creating evaluators.
Creating evaluator evaluators - from existing
Evaluator templates are created from existing evaluators by clicking the
Save as template button on the evaluator page.
Creating evaluator templates - from scratch
Evaluator templates are created from scratch on the evaluator templates page.
Editing evaluator templates
Evaluator templates can be edited on the templates page. Hover over the evaluator template and click the "Edit" button that appears.
Processor templates can only be edited in the context of an attached evaluator template.
Creating processor templates
Processor templates are created whenever an evaluator template is created with a processor, including from an existing evaluator or from scratch.
Processor templates cannot be created outside of the context of evaluator template creation at this time.
Evaluator and processor templates can be deleted on the templates page. Hover over the evaluator template and click the "Delete" button that appears.
Using evaluator templates
Once created, evaluator templates show up under a new "Saved" section in the evaluator creation dialog.
Creating an evaluator from a template creates a new evaluator with the same configuration as the template. The new evaluator is not linked to the template, so changes to the template do not affect the evaluator.
Using processor templates
Once created, processor templates show up under a new "Template" section when selecting from processors during evaluator creation / edit.
When the evaluator is saved with the template processor, a copy of the processor template is saved as a new processor into the pipeline.
The new processor is not linked to the template, so changes to the template do not affect the processor.
Repeated usages of the same processor template are deduplicated into a single processor as long as the processor has the same content as the template.