Experiments

🛑Alpha

OpenTelemetry support is currently in alpha and may undergo significant changes.

Experiments

The experiments SDK provides programmatic access to Gentrace experiments. While typically created via the experiment() function, experiments can also be accessed and managed directly through the SDK.

Overview

The SDK is built by Stainless and provides type-safe access to Gentrace entities. The experiments object exposes methods to create, retrieve, update, and list experiments.

Basic usage

TypeScript

typescript
import { init, experiments } from 'gentrace';
init({
  apiKey: process.env.GENTRACE_API_KEY,
});
// List experiments
const experimentList = await experiments.list();
// Access the experiments
for (const experiment of experimentList.data) {
  console.log(experiment.name);
  console.log(experiment.metadata);
}

Python

python
import os
from gentrace import init, experiments
init(api_key=os.environ["GENTRACE_API_KEY"])
# List experiments
experiment_list = await experiments.list()
# Access the experiments
for experiment in experiment_list.data:
    print(experiment.name)
    print(experiment.metadata)

Experiment structure

Each experiment contains:

• id: Unique identifier (UUID)
• name: Optional friendly name for the experiment
• pipelineId: UUID of the associated pipeline
• metadata: Optional metadata object for custom data
• createdAt: Creation timestamp
• updatedAt: Last update timestamp

Resource methods

Create an experiment

TypeScript

typescript
const experiment = await experiments.create({
  pipelineId: 'pipeline-uuid',
  name: 'Prompt optimization experiment',
  metadata: {
    temperature: 0.7,
    model: 'gpt-4o',
  }
});

Python

python
experiment = await experiments.create(
    pipeline_id="pipeline-uuid",
    name="Prompt optimization experiment",
    metadata={
        "temperature": 0.7,
        "model": "gpt-4o",
    }
)

Retrieve an experiment

TypeScript

typescript
const experiment = await experiments.retrieve('experiment-id');
console.log(experiment.pipelineId);

Python

python
experiment = await experiments.retrieve("experiment-id")
print(experiment.pipeline_id)

Update an experiment

TypeScript

typescript
const experiment = await experiments.update('experiment-id', {
  name: 'Updated experiment name',
  metadata: { version: 2 },
  status: 'EVALUATING',
});

Python

python
experiment = await experiments.update(
    "experiment-id",
    name="Updated experiment name",
    metadata={"version": 2},
    status="EVALUATING",
)

List with filters

TypeScript

typescript
// Filter by pipeline
const experimentList = await experiments.list({
  pipelineId: 'pipeline-id',
});

Python

python
# Filter by pipeline
experiment_list = await experiments.list(
    pipeline_id="pipeline-id",
)

Common usage with experiment()

Experiments are typically created automatically when using the experiment() function:

TypeScript

typescript
import { experiment } from 'gentrace';
// This automatically creates an experiment
experiment(PIPELINE_ID, async () => {
  // Your evaluation code here
});

Python

python
from gentrace import experiment
# This automatically creates an experiment
@experiment(pipeline_id=PIPELINE_ID)
async def my_experiment():
    # Your evaluation code here
    pass

See also

• Experiments API Reference - Full API documentation
• experiment() function - High-level function for running experiments
• evalDataset() - Running evaluations within experiments
• Pipelines - Managing pipelines that contain experiments