PipelineRun class - measure() / ameasure()

TypeScript

Python

The PipelineRun class captures the output for a generative invocation. measure() is a method under this class.

measure() tracks the inputs, outputs and duration it takes to run a step in a generative pipeline.

Examples

typescript

typescript
import { Pipeline } from '@gentrace/core';
import { User } from '../models';
 
const pipeline = new Pipeline({
  slug: 'example-pipeline',
});
 
const runner = pipeline.start();
 
const userId = '550e8400-e29b-41d4-a716-446655440000';
const gentraceOrgId = 'c9a646d3-9c61-4cb7-bfcd-ee2522c8f633';
 
const name = await runner.measure(
  async (userId, gentraceOrgId) => {
    // Custom application logic
    const user = await User.findOne({
      where: {
        id: userId,
        orgId: gentraceOrgId,
      },
    });
    return user.name;
  },
  // Inputs must be passed via a dependency array, so Gentrace
  // can properly trace the data.
  [userId, gentraceOrgId],
);

python

python
import gentrace
import os
# TODO: replace with your own function that you want to measure
from db_utils import get_organization_usernames
pipeline = gentrace.Pipeline(
    slug="example-pipeline",
    openai_config={
        "api_key": os.environ["OPENAI_API_KEY"],
    }
)
runner = pipeline.start()
usernames = runner.measure(
    get_organization_usernames,
    org_id=organization_id
)
openai = runner.get_openai()
initial_draft_response = openai.chat.completions.create(
    messages=[
        {
            "role": "system",
            "content": f"Write an email on behalf of {sender} to the members of organization {organization_name}. These are the names of the members: {usernames.join(", ")}"
        },
    ],
    model="gpt-3.5-turbo"
)
initial_draft = initial_draft_response.choices[0].message.content
runner.submit()
return [initial_draft, runner]

Arguments

func: (...inputs: any[]) => Promise<any>

The asynchronous function to measure. The function can take any number of parameters.

inputs: any[]

These are the arguments to the function passed to measure().

typescript

stepInfo: object

python

step_info

An object containing additional information about the step. The most important key/value pair in the stepInfo is the context key, which attaches more metadata to the step.

Read about the typing for the context key here.

Return value

The return value of the function passed to measure().

Python-only

Feature only available in Python

ameasure()

python

The PipelineRun class in Python also provides an ameasure() method that supports asyncio for asynchronous operations. This method is similar to measure() but is designed to work with asynchronous functions.

Example

python
import gentrace
import os
import asyncio
# TODO: replace with your own function that you want to measure
from db_utils import get_organization_usernames
async def main():
    pipeline = gentrace.Pipeline(
        slug="example-pipeline",
        openai_config={
            "api_key": os.environ["OPENAI_API_KEY"],
        }
    )
    runner = pipeline.start()
    usernames = await runner.ameasure(
        get_organization_usernames,
        org_id=organization_id
    )
    openai = runner.get_openai()
    initial_draft_response = await openai.chat.completions.create(
        messages=[
            {
                "role": "system",
                "content": f"Write an email on behalf of {sender} to the members of organization {organization_name}. These are the names of the members: {', '.join(usernames)}"
            },
        ],
        model="gpt-3.5-turbo"
    )
    initial_draft = initial_draft_response.choices[0].message.content
    await runner.submit()
    return [initial_draft, runner]
# Run the async function
result = asyncio.run(main())