Core tracer

When first starting out with Gentrace, we recommend that users use our provider wrappers for OpenAI and Pinecone to simplify the process of instrumenting their code.

However, if you want to trace arbitrary code (e.g. external API or database calls), we provide a tracer in our base SDKs.

Using measure() to trace arbitrary code

Let's say we have code to create an embedding in OpenAI that is instrumented by Gentrace using the OpenAI provider wrapper.

Original code

TypeScript

typescript
import { init, Pipeline } from "@gentrace/core";
import { initPlugin as initOpenAIPlugin } from "@gentrace/openai";
 
// This function globally initializes Gentrace with the appropriate 
// credentials. Constructors like Pipeline() will transparently use
// these credentials to authenticate with Gentrace.
init({
  apiKey: process.env.GENTRACE_API_KEY
});
 
const openaiPlugin = await initOpenAIPlugin({
  apiKey: process.env.OPENAI_KEY,
});
 
const pipeline = new Pipeline({
  slug: "create-embedding",
  plugins: {
    openai: openaiPlugin,
    pinecone: pineconePlugin
  },
});
 
const runner = pipeline.start()
 
const openai = runner.openai;
 
const embeddingResponse = await openai.embeddings.create({
  model: 'text-embedding-ada-002',
  input: "Human, software engineer, Gentrace: Vivek",
});
 
await runner.submit();

Python

python
import gentrace
import os
gentrace.init(
  api_key=os.getenv("GENTRACE_API_KEY")
)
pipeline = gentrace.Pipeline(
  slug="create-embedding",
  openai_config={
    "api_key": os.getenv("OPENAI_KEY"),
  }
)
runner = pipeline.start()
openai = runner.get_openai()
embedding_response = openai.Embedding.create(
  model="text-embedding-ada-002",
  input="Human, software engineer, Gentrace: Vivek"
);
runner.submit()

Adjusted code

We now must modify this code to include a database call to retrieve a user's name before creating our embedding.

typescript

typescript
import { init, Pipeline } from "@gentrace/core";
import { initPlugin as initOpenAIPlugin } from "@gentrace/openai";
import { User } from "../models";
 
// This function globally initializes Gentrace with the appropriate 
// credentials. Constructors like Pipeline() will transparently use
// these credentials to authenticate with Gentrace.
init({
  apiKey: process.env.GENTRACE_API_KEY
});
 
const openaiPlugin = await initOpenAIPlugin({
  apiKey: process.env.OPENAI_KEY,
});
 
const pipeline = new Pipeline({
  slug: "create-embedding",
  plugins: {
    openai: openaiPlugin,
    pinecone: pineconePlugin
  },
});
 
const runner = pipeline.start()
 
const openai = runner.openai;
 
const userId = "550e8400-e29b-41d4-a716-446655440000";
const gentraceOrgId = "c9a646d3-9c61-4cb7-bfcd-ee2522c8f633";
 
const name = await runner.measure(
  async (userId, gentraceOrgId) => {
    // Custom application logic
    const user = await User.findOne({
      where: {
        id: userId,
        orgId: gentraceOrgId,
      }
    })
    return user.name
  },
  // Inputs must be passed via a dependency array, so Gentrace
  // can properly trace the data.
  [userId, gentraceOrgId],
);
 
 
const embeddingResponse = await openai.embeddings.create({
  model: 'text-embedding-ada-002',
  input: `Human, software engineer, Gentrace: ${name}`,
});
 
await runner.submit();

python

python
import gentrace
import os
from app.models import User
gentrace.init(
    api_key=os.getenv("GENTRACE_API_KEY")
)
pipeline = gentrace.Pipeline(
    slug="create-embedding",
    openai_config={
      "api_key": os.getenv("OPENAI_KEY"),
    }
)
runner = pipeline.start()
user_id = "550e8400-e29b-41d4-a716-446655440000"
org_id = "c9a646d3-9c61-4cb7-bfcd-ee2522c8f633"
name = runner.measure(
    # Custom application logic
    lambda user_id, org_id: User.find_one({
        "id": user_id,
        "orgId": org_id,
    }),
    user_id=user_id, 
    org_id=org_id
)
openai = runner.get_openai()
embedding_response = openai.Embedding.create(
    model="text-embedding-ada-002",
    input="Human, software engineer, Gentrace: " + name
);
runner.submit()

In the above code example, the measure() function allows us to trace arbitrary code. If you navigate to the Gentrace UI, you will see that measured database call is now included in the trace.

Using checkpoint() to trace arbitrary code

If you want more control over when to start and stop a trace, you can use the checkpoint() function.

Rather than wrapping your desired logic in a measure() callback, you specify a checkpoint() invocation. The invocation then constructs a duration relative to the last time checkpoint() was invoked. You must also manually specify the exact inputs/outputs for that checkpoint.

Adjusted code

typescript

typescript
import { init, Pipeline } from "@gentrace/core";
import { initPlugin as initOpenAIPlugin } from "@gentrace/openai";
import { User } from "../models";
 
// This function globally initializes Gentrace with the appropriate 
// credentials. Constructors like Pipeline() will transparently use
// these credentials to authenticate with Gentrace.
init({
  apiKey: process.env.GENTRACE_API_KEY
});
 
const openaiPlugin = await initOpenAIPlugin({
  apiKey: process.env.OPENAI_KEY,
});
 
const pipeline = new Pipeline({
  slug: "create-embedding",
  plugins: {
    openai: openaiPlugin,
    pinecone: pineconePlugin
  },
});
 
const runner = pipeline.start()
 
const openai = runner.openai;
 
const userId = "550e8400-e29b-41d4-a716-446655440000";
const gentraceOrgId = "c9a646d3-9c61-4cb7-bfcd-ee2522c8f633";
 
// Custom application logic
const user = await User.findOne({
  where: {
    id: userId,
    orgId: gentraceOrgId,
  }
});
 
const name = user.name;
 
runner.checkpoint({
  inputs: { userId, orgId: gentraceOrgId },
  outputs: { name }
});
 
const embeddingResponse = await openai.embeddings.create({
  model: 'text-embedding-ada-002',
  input: `Human, software engineer, Gentrace: ${name}!`,
});
 
await runner.submit();

python

python
import gentrace
import os
from app.models import User
gentrace.init(
    api_key=os.getenv("GENTRACE_API_KEY")
)
pipeline = gentrace.Pipeline(
    slug="create-embedding",
    openai_config={
        "api_key": os.getenv("OPENAI_KEY"),
    }
)
runner = pipeline.start()
user_id = "550e8400-e29b-41d4-a716-446655440000"
org_id = "c9a646d3-9c61-4cb7-bfcd-ee2522c8f633"
# Custom application logic
user = User.find_one({
    "id": user_id,
    "orgId": org_id,
}),
name = user.name
runner.checkpoint({
    "inputs": {
        "user_id": user_id,
        "org_id": org_id,
    },
    "outputs": {
        "value": name
    }
})
openai = runner.get_openai()
embedding_response = openai.Embedding.create(
    model="text-embedding-ada-002",
    input="Human, software engineer, Gentrace: " + name
);
runner.submit()

Evaluation

Gentrace also provides a way to evaluate generative pipelines that use measure() or checkpoint() invocations. You can read more about that here.

Deeper understanding

Measure

SDK reference for the measure() function

Checkpoint

SDK reference for the checkpoint() function