Run evaluations against a dataset using evalDataset()
and eval_dataset()
eval_dataset()
and TypeScript evalDataset()
functions run a series of evaluations against a dataset using a provided interaction()
function.
These functions must be called within the context of an experiment()
and automatically create individual test spans for each test case in the dataset.
init()
. If you have an existing OpenTelemetry setup or need custom configuration, see the manual setup guide.None
values in the corresponding positions.
interaction()
.TestInputs
structure from above.
This is useful when you want to use test cases from the Gentrace API via the test cases SDK or directly defining them in-line.
ERROR
status when exceptions occurinit()
with a valid API key. The SDK automatically configures OpenTelemetry for you. For custom OpenTelemetry setups, see the manual setup guideexperiment()
functiondata
function must return an array of test casesinit()
- Initialize the Gentrace SDKinteraction()
- Instrument AI functions for tracing within experimentsexperiment()
- Create experiment context for dataset evaluationseval()
/ evalOnce()
- Run individual test cases within an experimenttraced()
- Alternative approach for tracing functions