> ## Documentation Index
> Fetch the complete documentation index at: https://next.gentrace.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Test Cases

> Programmatic access to Gentrace test cases

The test cases SDK provides programmatic access to Gentrace test cases. While commonly used with [`evalDataset()`](../evaluation/dataset-tests) for batch evaluations, test cases can also be accessed and managed independently.

## Basic usage

<CodeGroup dropdown>
  ```typescript theme={null}
  import { init, testCases } from 'gentrace';

  init({
    apiKey: process.env.GENTRACE_API_KEY,
  });

  // List test cases from a dataset
  const testCasesList = await testCases.list({
    datasetId: 'your-dataset-id',
  });

  // Access the test cases
  for (const testCase of testCasesList.data) {
    console.log(testCase.name);
    console.log(testCase.inputs);
  }
  ```

  ```python theme={null}
  import os
  from gentrace import init, test_cases_async

  init(api_key=os.environ["GENTRACE_API_KEY"])

  # List test cases from a dataset
  test_case_list = await test_cases_async.list(
      dataset_id="your-dataset-id"
  )

  # Access the test cases
  for test_case in test_case_list.data:
      print(test_case.name)
      print(test_case.inputs)
  ```
</CodeGroup>

## Overview

The SDK is built by [Stainless](https://stainless.com) and provides type-safe access to Gentrace entities. The `testCases` object exposes methods to list, create, update, and delete test cases.

## Test case structure

Each test case contains:

<ResponseField name="id" type="string">
  Unique identifier
</ResponseField>

<ResponseField name="name" type="string">
  Human-readable name for the test case
</ResponseField>

<ResponseField name="inputs" type="object" required>
  Dictionary/object containing the input data for your AI function
</ResponseField>

<ResponseField name="expectedOutputs" type="object">
  Optional expected outputs for validation
</ResponseField>

<ResponseField name="datasetId" type="string" required>
  UUID of the dataset this test case belongs to
</ResponseField>

<ResponseField name="createdAt" type="string" required>
  Creation timestamp
</ResponseField>

<ResponseField name="updatedAt" type="string" required>
  Last update timestamp
</ResponseField>

## Resource methods

### Create a test case

<CodeGroup dropdown>
  ```typescript theme={null}
  const testCase = await testCases.create({
    datasetId: 'your-dataset-id',
    inputs: { query: 'What is AI?' },
    name: 'Basic AI question',
    expectedOutputs: { answer: 'Artificial Intelligence is...' }, // optional
  });
  ```

  ```python theme={null}
  test_case = await test_cases.create(
      dataset_id="your-dataset-id",
      inputs={"query": "What is AI?"},
      name="Basic AI question",
      expected_outputs={"answer": "Artificial Intelligence is..."}  # optional
  )
  ```
</CodeGroup>

### Retrieve a test case

<CodeGroup dropdown>
  ```typescript theme={null}
  const testCase = await testCases.retrieve('test-case-id');
  console.log(testCase.inputs);
  ```

  ```python theme={null}
  test_case = await test_cases.retrieve("test-case-id")
  print(test_case.inputs)
  ```
</CodeGroup>

### Delete a test case

<CodeGroup dropdown>
  ```typescript theme={null}
  await testCases.delete('test-case-id');
  ```

  ```python theme={null}
  await test_cases.delete("test-case-id")
  ```
</CodeGroup>

### List with filters

<CodeGroup dropdown>
  ```typescript theme={null}
  // Filter by pipeline
  const testCasesList = await testCases.list({
    pipelineId: 'pipeline-id',
    // or use pipelineSlug: 'pipeline-slug'
  });
  ```

  ```python theme={null}
  # Filter by pipeline
  test_case_list = await test_cases_async.list(
      pipeline_id="pipeline-id",
      # or use pipeline_slug="pipeline-slug"
  )
  ```
</CodeGroup>

## Common usage with evalDataset()

Test cases are frequently used with [`evalDataset()`](../evaluation/dataset-tests) for running batch evaluations:

<CodeGroup dropdown>
  ```typescript theme={null}
  await evalDataset({
    data: async () => {
      const testCasesList = await testCases.list({
        datasetId: DATASET_ID,
      });
      return testCasesList.data;
    },
    interaction: yourAIFunction, // See interaction() docs
  });
  ```

  ```python theme={null}
  async def fetch_test_cases():
      test_case_list = await test_cases_async.list(dataset_id=DATASET_ID)
      return test_case_list.data

  await eval_dataset(
      data=fetch_test_cases,
      interaction=your_ai_function, # See interaction() docs
  )
  ```
</CodeGroup>

The `interaction` parameter should be a function wrapped with [`interaction()`](../tracing/interactions) for proper OpenTelemetry tracing within experiments.

## See also

* [`evalDataset()`](../evaluation/dataset-tests) - Common usage pattern for batch evaluations
* [Datasets](./datasets) - Managing datasets that contain test cases