Context basics

You can attach context with our SDKs to attach related information to your generative output.

Usage

You can add context using both our simple and advanced SDKs.

Supported keys

Some keys only work on certain types of AI generations. For example, user IDs can only be associated to an entire generation, whereas metadata can be assigned directly on particular steps within a generation. Visit the detailed page for each context type to learn more about its semantics.

User IDsuserId

Associate user IDs from your own service with particular AI generations

ThreadingpreviousRunId

Associate a series of AI generations together (e.g. chat bots)

Render formatrender

Render a key in the outputs payload with a custom format. Currently, we can only render HTML at this time.

Other metadata

Attach arbitrary key/value information to your observed AI data.

Simple SDK

Every simple invocation allows you to specify a gentrace parameter, where you can supply specific context key/value pairs. Here's an example from our OpenAI simple plugin.

TypeScript

typescript
import { init } from "@gentrace/core";
import { OpenAI } from "@gentrace/openai";
 
init({
  apiKey: process.env.GENTRACE_API_KEY,
});
 
const openai = new OpenAI({
  apiKey: process.env.OPENAI_KEY,
});
 
await openai.chat.completions.create({
  messages: [
    {
      role: "user",
      contentTemplate: "Hello! My name is {{ name }}. Write a brief essay about Maine.",
      contentInputs: { name: "Vivek" },
    },
  ],
  model: "gpt-3.5-turbo",
  pipelineSlug: "simple-pipeline",
  stream: true,
 
  // Context is specified with the `gentrace` key. In this case, the user ID will be 
  // associated with this generation.
  gentrace: {
    userId: "TWFuIGlzIGRpc3Rpbmd1aXNoZW",
  },
});

Python

python
import openai
import gentrace
import os
gentrace.configure_openai()
openai.api_key = os.getenv("OPENAI_KEY")
openai.ChatCompletion.create(
    messages=[
        {
            "role": "user",
            "contentTemplate": "Hello! My name is {{ name }}. Write a brief essay about Maine.",
            "contentInputs": {"name": "Vivek"},
        }
    ],
    model="gpt-3.5-turbo",
    pipeline_slug="example-completion",
    # Context is specified with the `gentrace` key. In this case, the user ID will be 
    # associated with this generation.
    gentrace={
        "userId": "TWFuIGlzIGRpc3Rpbmd1aXNoZW",
    },
)

Advanced SDK

You can specify context when you initialize a PipelineRun instance with the pipeline.start() function.

TypeScript

typescript
import { Pipeline } from "@gentrace/core";
import { initPlugin } from "@gentrace/openai";
 
const plugin = await initPlugin({
  apiKey: process.env.OPENAI_KEY,
});
 
const pipeline = new Pipeline({
  slug: "advanced-pipeline",
  plugins: {
    openai: plugin,
  },
});
 
// Context is specified directly as an object. This context is associated with all
// steps in the generative pipeline.
const runner = pipeline.start({
  userId: "TWFuIGlzIGRpc3Rpbmd1aXNoZW",
});
 
const openai = runner.openai;
 
const chatCompletionResponse = await openai.chat.completions.create({
  messages: [{ role: "user", content: "Hello!" }],
  model: "gpt-3.5-turbo",
  stream: true,
});
 

Python

python
import gentrace
PIPELINE_SLUG = "compose"
pipeline = gentrace.Pipeline(
    PIPELINE_SLUG,
    openai_config={
        "api_key": process.env.OPENAI_KEY,
    },
)
runner = pipeline.start({
  "userId": "TWFuIGlzIGRpc3Rpbmd1aXNoZW",
})
openai = runner.get_openai()
result = openai.ChatCompletion.create(
  messages=[{ "role": "user", "content": "Hello!" }],
  model="gpt-3.5-turbo",
  stream=True
)

You can also associate context with any individual step with the gentrace parameter.

TypeScript

typescript
import { Pipeline } from "@gentrace/core";
import { initPlugin } from "@gentrace/openai";
 
const plugin = await initPlugin({
  apiKey: process.env.OPENAI_KEY,
});
 
const pipeline = new Pipeline({
  slug: "advanced-pipeline",
  plugins: {
    openai: plugin,
  },
});
 
const runner = pipeline.start();
 
const openai = runner.openai;
 
const chatCompletionResponse = await openai.chat.completions.create({
  messages: [{ role: "user", content: "Hello!" }],
  model: "gpt-3.5-turbo",
  stream: true,
 
  // Context is applied directly to this AI step 
  gentrace: {
    userId: "TWFuIGlzIGRpc3Rpbmd1aXNoZW"
  }
});

Python

python
import gentrace
gentrace.init(
    api_key=os.getenv("GENTRACE_API_KEY"),
)
gentrace.configure_openai()
openai.api_key = os.getenv("OPENAI_KEY")
result = openai.ChatCompletion.create(
    pipeline_slug="testing-chat-completion-value",
    messages=[
        {
            "role": "user",
            "contentTemplate": "Hello {{ name }}! What's your name?",
            "contentInputs": {"name": "John"},
        }
    ],
    model="gpt-3.5-turbo",
    gentrace={
        "userId": "TWFuIGlzIGRpc3Rpbmd1aXNoZW",
    },
)

Certain step methods like measure() and checkpoint() require that context is explicitly passed in a context key.

TypeScript

typescript
import { Pipeline } from "@gentrace/core";
import { initPlugin } from "@gentrace/openai";
 
const plugin = await initPlugin({
  apiKey: process.env.OPENAI_KEY,
});
 
const pipeline = new Pipeline({
  slug: "advanced-pipeline",
  plugins: {
    openai: plugin,
  },
});
 
const runner = pipeline.start();
 
const outputs = await runner.measure(
  (pageDescription) => {
    // ... Omitted logic for creating HTML from provided inputs
    return {
      koalaHtml: htmlOutput
    };
  },
  ["Create HTML for a site about koalas"],
  {
    context: {
      render: {
        type: "html",
        key: "koalaHtml",
      },
    },
  },
);
 
await runner.submit();

Python

python
import gentrace
def generate_html(description):
  # ... Generate HTML
  return {
    "koalaHtml": htmlOutput
  }
PIPELINE_SLUG = "compose"
pipeline = gentrace.Pipeline(
    PIPELINE_SLUG,
    openai_config={
        "api_key": process.env.OPENAI_KEY,
    },
)
runner = pipeline.start()
output = runner.measure(
  generate_html,
  description="Create HTML for a site about koalas",
  step_info={
    "context": {
      "render": {
        "type": "html",
        "key": "koalaHtml"
      }
    }
  },
)
runner.submit()