Skip to main content
The test cases SDK provides programmatic access to Gentrace test cases. While commonly used with evalDataset() for batch evaluations, test cases can also be accessed and managed independently.

​
Basic usage

import { init, testCases } from 'gentrace';

init({
  apiKey: process.env.GENTRACE_API_KEY,
});

// List test cases from a dataset
const testCasesList = await testCases.list({
  datasetId: 'your-dataset-id',
});

// Access the test cases
for (const testCase of testCasesList.data) {
  console.log(testCase.name);
  console.log(testCase.inputs);
}

​
Overview

The SDK is built by Stainless and provides type-safe access to Gentrace entities. The testCases object exposes methods to list, create, update, and delete test cases.

​
Test case structure

Each test case contains:
​
id
string
Unique identifier
​
name
string
Human-readable name for the test case
​
inputs
object
required
Dictionary/object containing the input data for your AI function
​
expectedOutputs
object
Optional expected outputs for validation
​
datasetId
string
required
UUID of the dataset this test case belongs to
​
createdAt
string
required
Creation timestamp
​
updatedAt
string
required
Last update timestamp

​
Resource methods

​
Create a test case

const testCase = await testCases.create({
  datasetId: 'your-dataset-id',
  inputs: { query: 'What is AI?' },
  name: 'Basic AI question',
  expectedOutputs: { answer: 'Artificial Intelligence is...' }, // optional
});

​
Retrieve a test case

const testCase = await testCases.retrieve('test-case-id');
console.log(testCase.inputs);

​
Delete a test case

await testCases.delete('test-case-id');

​
List with filters

// Filter by pipeline
const testCasesList = await testCases.list({
  pipelineId: 'pipeline-id',
  // or use pipelineSlug: 'pipeline-slug'
});

​
Common usage with evalDataset()

Test cases are frequently used with evalDataset() for running batch evaluations: 
await evalDataset({
  data: async () => {
    const testCasesList = await testCases.list({
      datasetId: DATASET_ID,
    });
    return testCasesList.data;
  },
  interaction: yourAIFunction, // See interaction() docs
});
The interaction parameter should be a function wrapped with interaction() for proper OpenTelemetry tracing within experiments.

​
See also

  • evalDataset() - Common usage pattern for batch evaluations
  • Datasets - Managing datasets that contain test cases