OpenTelemetry SDK setup

🛑Alpha

OpenTelemetry support is currently in alpha and may undergo significant changes.

This guide shows how to set up the OpenTelemetry SDK to work with Gentrace's instrumentation features (interaction, traced, experiment, eval, evalDataset). OpenTelemetry integration is required for these features to function correctly.

Why OpenTelemetry setup is required

Without proper OpenTelemetry configuration, you'll encounter the following error when trying to use Gentrace features:

This error indicates that:

• OpenTelemetry SDK (TracerProvider) does not appear to be configured
• Gentrace tracing features may not record data for:
  • @interaction
  • @eval
  • @traced
  • eval_dataset()

To resolve this, follow the setup instructions below.

Prerequisites

• A Gentrace API key (available from your Gentrace dashboard)
• Node.js 20+ (for TypeScript/JavaScript) or Python 3.8+ (for Python)

Installation

TypeScript/JavaScript

Modern package managers should automatically install the OpenTelemetry dependencies when you install gentrace. If they weren't installed automatically, you can install them manually:

bash
# Or pnpm i / yarn add
npm install @opentelemetry/sdk-node @opentelemetry/exporter-trace-otlp-http @opentelemetry/semantic-conventions @opentelemetry/context-async-hooks

Python

The OpenTelemetry SDK is included as a dependency when you install the Gentrace Python SDK:

bash
pip install --pre gentrace-py

If you need to install OpenTelemetry packages separately:

bash
pip install opentelemetry-sdk opentelemetry-exporter-otlp-proto-http opentelemetry-instrumentation opentelemetry-processor-baggage

Basic setup

TypeScript/JavaScript

Create a setup file (e.g., otel-setup.ts or otel-setup.js) and import it at the very beginning of your application:

typescript
// otel-setup.ts
import { NodeSDK } from '@opentelemetry/sdk-node';
import { OTLPTraceExporter } from '@opentelemetry/exporter-trace-otlp-http';
import { Resource } from '@opentelemetry/resources';
import { ATTR_SERVICE_NAME } from '@opentelemetry/semantic-conventions';
import { AsyncLocalStorageContextManager } from '@opentelemetry/context-async-hooks';
import { GentraceSampler, GentraceSpanProcessor } from 'gentrace';
const GENTRACE_API_KEY = process.env.GENTRACE_API_KEY || '';
if (!GENTRACE_API_KEY) {
  throw new Error('GENTRACE_API_KEY must be set');
}
const sdk = new NodeSDK({
  resource: new Resource({
    [ATTR_SERVICE_NAME]: 'your-generative-ai-product',
  }),
  traceExporter: new OTLPTraceExporter({
    url: 'https://gentrace.ai/api/otel/v1/traces',
    headers: {
      Authorization: `Bearer ${GENTRACE_API_KEY}`,
    },
  }),
  sampler: new GentraceSampler(),
  spanProcessors: [
    new GentraceSpanProcessor()
  ],
  contextManager: (new AsyncLocalStorageContextManager()).enable()
});
sdk.start();
console.log('OpenTelemetry SDK started, exporting traces to Gentrace.');
// Ensures spans get flushed before the runtime exits
process.on('beforeExit', async () => {
  await sdk.shutdown();
});
// Ensures spans get flushed when the runtime is asked to terminate
process.on('SIGTERM', async () => {
  await sdk.shutdown();
});

Then import this setup file at the very beginning of your main application file:

typescript
// main.ts or index.ts
import './otel-setup'; // Must be first import
import { init } from 'gentrace';
// Initialize Gentrace
init({
  apiKey: process.env.GENTRACE_API_KEY,
});
// Rest of your application code...

Python

Set up OpenTelemetry before initializing Gentrace:

python
# otel_setup.py
from opentelemetry.sdk.resources import Resource
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import SimpleSpanProcessor
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
from opentelemetry import trace
from gentrace import GentraceSampler, GentraceSpanProcessor
import os
# In virtually all cases, you should use https://gentrace.ai/api as the base URL
GENTRACE_BASE_URL = os.environ.get('GENTRACE_BASE_URL', 'https://gentrace.ai/api')
GENTRACE_API_KEY = os.environ['GENTRACE_API_KEY']
resource = Resource.create({
    "service.name": "my-gentrace-app"
})
provider = TracerProvider(
    resource=resource,
    sampler=GentraceSampler()  # Use GentraceSampler for selective tracing
)
trace.set_tracer_provider(provider)
exporter = OTLPSpanExporter(
    endpoint=f"{GENTRACE_BASE_URL}/otel/v1/traces",
    headers={
        "Authorization": f"Bearer {GENTRACE_API_KEY}"
    },
)
# Add GentraceSpanProcessor to propagate baggage to span attributes
provider.add_span_processor(GentraceSpanProcessor())
provider.add_span_processor(SimpleSpanProcessor(exporter))
print("OpenTelemetry SDK started – spans will be sent to Gentrace.")

Then import this setup in your main application:

python
# main.py
import os
import otel_setup  # Must be imported first
from gentrace import init
# Initialize Gentrace
init(
    api_key=os.environ["GENTRACE_API_KEY"],
    # Optional for self-hosted deployments: base_url=os.environ.get("GENTRACE_BASE_URL", "https://gentrace.ai/api")
)
# Rest of your application code...

Key components

GentraceSampler

The GentraceSampler is a custom OpenTelemetry sampler that selectively samples spans based on the presence of a gentrace.sample attribute. This helps reduce the volume of telemetry data by only sending relevant spans to Gentrace.

How it works:

• Checks for the gentrace.sample key in OpenTelemetry Baggage or as a span attribute
• If gentrace.sample is set to "true", the span will be sampled and exported to Gentrace
• Otherwise, the span will be dropped and not exported

This is particularly useful for filtering out spans that are not relevant to Gentrace tracing, reducing noise and data volume.

GentraceSpanProcessor

The GentraceSpanProcessor is a specialized span processor that simplifies the process of converting OpenTelemetry baggage values to span attributes, specifically for Gentrace integration.

Usage:

TypeScript/JavaScript

typescript
import { NodeSDK } from '@opentelemetry/sdk-node';
import { GentraceSpanProcessor } from 'gentrace';
const sdk = new NodeSDK({
  // ... other configuration
  spanProcessors: [
    new GentraceSpanProcessor(),
    // ... other processors
  ]
});

Python

python
from opentelemetry.sdk.trace import TracerProvider
from gentrace import GentraceSpanProcessor
provider = TracerProvider()
provider.add_span_processor(GentraceSpanProcessor())

How it works:

• When a span starts, checks for the gentrace.sample key in the current OpenTelemetry Baggage
• If found, extracts this value and adds it as an attribute to the span
• Ensures that the sampling attribute is propagated correctly to all spans that need to be tracked by Gentrace

Why it's needed:

• The OpenTelemetry Collector can only filter and route spans based on span attributes, not baggage
• This processor eliminates the need for manual attribute management or additional processors
• Works seamlessly with GentraceSampler to ensure proper trace routing

Using this component alone provides optimal control over which spans are sent to Gentrace.

Advanced configuration

Self-hosted deployments

If you're using a self-hosted Gentrace deployment, update the endpoint URL:

TypeScript/JavaScript

typescript
const GENTRACE_BASE_URL = process.env.GENTRACE_BASE_URL || 'https://gentrace.ai/api';
const sdk = new NodeSDK({
  // ... other configuration
  traceExporter: new OTLPTraceExporter({
    url: `${GENTRACE_BASE_URL}/otel/v1/traces`,
    headers: {
      Authorization: `Bearer ${GENTRACE_API_KEY}`,
    },
  }),
});

Python

python
GENTRACE_BASE_URL = os.environ.get('GENTRACE_BASE_URL', 'https://gentrace.ai/api')
exporter = OTLPSpanExporter(
    endpoint=f"{GENTRACE_BASE_URL}/otel/v1/traces",
    headers={
        "Authorization": f"Bearer {GENTRACE_API_KEY}"
    },
)

Custom service name

Set a descriptive service name to identify your application in traces:

TypeScript/JavaScript

typescript
import { NodeSDK } from '@opentelemetry/sdk-node';
import { Resource } from '@opentelemetry/resources';
import { ATTR_SERVICE_NAME } from '@opentelemetry/semantic-conventions';
const sdk = new NodeSDK({
  resource: new Resource({
    [ATTR_SERVICE_NAME]: 'my-ai-chatbot', // Replace with your service name
  }),
  // ... other configuration
});

Python

python
from opentelemetry.sdk.resources import Resource
resource = Resource.create({
    "service.name": "my-ai-chatbot"  # Replace with your service name
})

Verification

To verify that your OpenTelemetry setup is working correctly:

1. Check console output: You should see a message indicating that the OpenTelemetry SDK has started
2. Run a simple interaction: Create a basic interaction and check if traces appear in your Gentrace dashboard
3. Monitor for errors: Watch for any OpenTelemetry-related errors in your application logs

TypeScript/JavaScript

typescript
import { interaction } from 'gentrace';
const testInteraction = interaction(
  'test-interaction',
  async (input: string) => {
    console.log(`Processing: ${input}`);
    return `Processed: ${input}`;
  },
  { pipelineId: 'your-pipeline-id' }
);
// Test the interaction
testInteraction('Hello, World!').then(result => {
  console.log('Result:', result);
  console.log('Check your Gentrace dashboard for the trace!');
});

Python

python
from gentrace import interaction
@interaction(pipeline_id="your-pipeline-id")
async def test_interaction(input_text: str) -> str:
    print(f"Processing: {input_text}")
    return f"Processed: {input_text}"
# Test the interaction
import asyncio
async def main():
    result = await test_interaction("Hello, World!")
    print(f"Result: {result}")
    print("Check your Gentrace dashboard for the trace!")
asyncio.run(main())

Troubleshooting

Common issues

1. No traces appearing in Gentrace

   • Verify your API key is correct
   • Check that the OpenTelemetry setup is imported before any Gentrace code
   • Ensure the GentraceSampler and GentraceSpanProcessor are properly configured

2. Import order issues

   • The OpenTelemetry setup must be imported before any other modules that use tracing

3. Environment variables not set

   • Ensure GENTRACE_API_KEY is set in your environment
   • For self-hosted deployments, verify GENTRACE_BASE_URL is correct

4. Spans not being sampled

   • Check that the gentrace.sample attribute is being set correctly
   • Verify that Gentrace SDK helpers (interaction, traced) are being used properly

Next steps

Once you have OpenTelemetry set up, you can:

• Use interaction() to trace your AI functions
• Create experiment() contexts for testing
• Run evaluations with eval() and evalDataset()
• Add custom tracing with traced()

For more advanced OpenTelemetry configurations, see the Setup with OpenTelemetry guide.