PipelineRun class - OpenAI handler

TypeScript

Python

The PipelineRun class instance exposes an OpenAI handler that simplifies capturing generative output in Gentrace.

For a guided walkthrough of the Gentrace OpenAI integration, visit our quickstart here.

Usage

When the PipelineRun instance is created, you can create a handle in two different ways to communicate with OpenAI.

Simple

typescript

Creating a simple handle is the easiest way to get started.

typescript
const openai = new OpenAI({
  apiKey: process.env.OPENAI_KEY,
})
 
const chatCompletionResponse = await openai.chat.completions.create({
  messages: [{ role: "user", content: "Hello! What's the capital of Maine?" }],
  model: "gpt-3.5-turbo",
  stream: true,
});

python

Creating a simple handle is the easiest way to get started. By invoking gentrace.configure_openai(), we configure the Gentrace SDK to automatically capture telemetry from OpenAI calls.

python
import os
import gentrace
import openai
openai.api_key = os.getenv("OPENAI_KEY")
gentrace.init(
    api_key=os.getenv("GENTRACE_API_KEY"),
)
gentrace.configure_openai()
result = openai.ChatCompletion.create(
    pipeline_slug="example-pipeline",
    messages=[
        {
            "role": "user",
            "content": "Hello! What's the capital of Maine?",
        }
    ],
    model="gpt-3.5-turbo",
)

Advanced

The more advanced way involves creating a handle that is bound to a PipelineRun instance. This allows you to attach multiple steps to a single PipelineRun instance.

typescript

typescript
const runner = pipeline.start();
 
const openai = runner.openai;
 
const chatCompletionResponse = await openai.chat.completions.create({
  messages: [{ role: "user", content: "Hello! What's the capital of Maine?" }],
  model: "gpt-3.5-turbo",
  stream: true,
});
 
await runner.submit();

python

python
import gentrace
gentrace.init(
    api_key=os.getenv("GENTRACE_API_KEY"),
)
pipeline = gentrace.Pipeline(
    "example-pipeline",
    openai_config={
        "api_key": os.getenv("OPENAI_KEY"),
    },
)
pipeline.setup()
runner = pipeline.start()
openai = runner.get_openai()
result = openai.ChatCompletion.create(
    messages=[{"role": "user", "content": "Hello! What's the capital of Maine?"}],
    model="gpt-3.5-turbo",
)

Chat completion

typescript

The openai.chat.completions.create() method is a wrapper around the equivalent OpenAI Node.JS chat completion API.

Arguments

The original openai.chat.completions.create() method parameters are supported. The below key/value pairs overwrite or augment the defaults.

pipelineSlug?

If you're using the Simple SDK, you can specify the pipeline slug here. If you're using the Advanced SDK, this is set in the Pipeline constructor and must be omitted.

messages

An array of messages that will be fed to the model. The array must contain at least one message. The main different between this and the original API is that the messages array optionally allows templated values in each element with the contentTemplate and contentInputs keys.

typescript
const runner = pipeline.start();
 
const openai = runner.openai;
 
const chatCompletionResponse = await openai.chat.completions.create({
  messages: [
    {
      role: "user",
      contentTemplate: "Hello {{ name }}!",
      contentInputs: { name: "Vivek" },
    }
  ],
  model: "gpt-3.5-turbo",
  stream: true,
});

gentrace?

This object contains Gentrace context. Learn about context here.

Return value

The return value is a Promise that resolves to the original OpenAI response. If you're using the simple SDK, the response is augmented with a pipelineRunId that references the run.

pipelineRunId

The pipelineRunId is a unique identifier for the run. It can be used to retrieve the run later.

python

caution

We will be updating our Python SDK to support the latest 1.x.x Python SDK version. You can learn more about the new interface here.

The openai.ChatCompletion.create() method is a wrapper around the equivalent OpenAI Python chat completion API.

Arguments

The original openai.ChatCompletion.create() method parameters are supported. The below key/value pairs overwrite or augment the defaults.

pipeline_slug?

If you're using the Simple SDK, you can specify the pipeline slug here. If you're using the Advanced SDK, this is set in the Pipeline constructor and must be omitted.

messages

An array of messages that will be fed to the model. The array must contain at least one message. The main different between this and the original API is that the messages array optionally allows templated values in each element with the contentTemplate and contentInputs keys.

python
import gentrace
gentrace.init(
    api_key=os.getenv("GENTRACE_API_KEY"),
)
pipeline = gentrace.Pipeline(
    "example-pipeline",
    openai_config={
        "api_key": os.getenv("OPENAI_KEY"),
    },
)
pipeline.setup()
runner = pipeline.start()
openai = runner.get_openai()
result = openai.ChatCompletion.create(
    messages=[{"role": "user", "content": "Hello! What's the capital of Maine?"}],
    model="gpt-3.5-turbo",
)

gentrace?

This object contains Gentrace context. Learn about context here.

Return value

The return value is a Promise that resolves to the original OpenAI response. If you're using the simple SDK, the response is augmented with a pipelineRunId that references the run.

pipelineRunId

The pipelineRunId is a unique identifier for the run. It can be used to retrieve the run later.

Completion

typescript

The openai.completions.create() method is a wrapper around the equivalent OpenAI Node.JS completion API.

Arguments

The original openai.completions.create() method parameters are supported. The below key/value pairs overwrite or augment the defaults.

pipelineSlug?

If you're using the Simple SDK, you can specify the pipeline slug here. If you're using the Advanced SDK, this is set in the Pipeline constructor and must be omitted.

promptTemplate

A template string that will be fed to the model. The string optionally allows templated values with the promptInputs key. We use Mustache syntax for templating.

promptInputs

An object containing values that will be used to replace the template values in promptTemplate.

typescript
const runner = pipeline.start();
 
const openai = runner.openai;
 
const completionResponse = await openai.completions.create({
  model: "text-davinci-003",
  promptTemplate: "Write a brief summary of the history of {{ company }}: ",
  promptInputs: {
    company: "Google",
  },
});
 
await runner.submit();

gentrace?

This object contains Gentrace context. Learn about context here.

Return value

The return value is a Promise that resolves to the original OpenAI response. If you're using the simple SDK, the response is augmented with a pipelineRunId that references the run.

pipelineRunId

The pipelineRunId is a unique identifier for the run. It can be used to retrieve the run later.

python

caution

We will be updating our Python SDK to support the latest 1.x.x Python SDK version. You can learn more about the new interface here.

The original openai.Completion.create() method parameters are supported. The below key/value pairs overwrite or augment the defaults.

pipeline_slug?

If you're using the Simple SDK, you can specify the pipeline slug here. If you're using the Advanced SDK, this is set in the Pipeline constructor and must be omitted.

prompt_template

A template string that will be fed to the model. The string optionally allows templated values with the prompt_inputs key. We use Mustache syntax for templating.

prompt_inputs

An object containing values that will be used to replace the template values in prompt_template.

python
import gentrace
gentrace.init(
    api_key=os.getenv("GENTRACE_API_KEY"),
)
pipeline = gentrace.Pipeline(
    "example-pipeline",
    openai_config={
        "api_key": os.getenv("OPENAI_KEY"),
    },
)
pipeline.setup()
runner = pipeline.start()
openai = runner.get_openai()
result = openai.Completion.create(
    prompt_template="Hello! What's the capital of {{ location }}?",
    prompt_inputs={"location": "Maine"},
    model="text-davinci-003",
)

gentrace?

This object contains Gentrace context. Learn about context here.

Return value

The return value is a Promise that resolves to the original OpenAI response. If you're using the simple SDK, the response is augmented with a pipelineRunId that references the run.

pipelineRunId

The pipelineRunId is a unique identifier for the run. It can be used to retrieve the run later.

Embedding

typescript

The openai.embeddings.create() method is a wrapper around the equivalent OpenAI Node.JS embedding API.

Arguments

The original openai.embeddings.create() method parameters are supported. The below key/value pairs overwrite or augment the defaults.

pipelineSlug?

If you're using the Simple SDK, you can specify the pipeline slug here. If you're using the Advanced SDK, this is set in the Pipeline constructor and must be omitted.

typescript
const runner = pipeline.start();
 
const openai = runner.openai;
 
const embeddingResponse = await openai.embeddings.create({
  model: "text-embedding-ada-002",
  input: "The capital of Maine is Augusta",
});
 
await runner.submit();

gentrace?

This object contains Gentrace context. Learn about context here.

Return value

The return value is a Promise that resolves to the original OpenAI response. If you're using the simple SDK, the response is augmented with a pipelineRunId that references the run.

pipelineRunId

The pipelineRunId is a unique identifier for the run. It can be used to retrieve the run later.

python

caution

We will be updating our Python SDK to support the latest 1.x.x Python SDK version. You can learn more about the new interface here.

The openai.Embedding.create() method is a wrapper around the equivalent OpenAI Python embedding API.

Arguments

The original openai.Embedding.create() method parameters are supported. The below key/value pairs overwrite or augment the defaults.

pipeline_slug?

If you're using the Simple SDK, you can specify the pipeline slug here. If you're using the Advanced SDK, this is set in the Pipeline constructor and must be omitted.

python
import gentrace
gentrace.init(
    api_key=os.getenv("GENTRACE_API_KEY"),
)
pipeline = gentrace.Pipeline(
    "example-pipeline",
    openai_config={
        "api_key": os.getenv("OPENAI_KEY"),
    },
)
pipeline.setup()
runner = pipeline.start()
openai = runner.get_openai()
result = openai.Embedding.create(
    model="text-davinci-003",
    input="The capital of Maine is Augusta",
)

gentrace?

This object contains Gentrace context. Learn about context here.

Return value

The return value is a Promise that resolves to the original OpenAI response. If you're using the simple SDK, the response is augmented with a pipelineRunId that references the run.

pipelineRunId

The pipelineRunId is a unique identifier for the run. It can be used to retrieve the run later.