Rivet Integration

Rivet is a visual programming environment for generative AI. Gentrace provides a Rivet plugin that allows users to associate a Rivet graph with a Gentrace pipeline.

Package version

The Rivet Gentrace plugin is available in Rivet v1.2.0.

Testing Rivet graphs

Installation

Install Rivet from their Github releases page. Once installed, enable the Gentrace plugin and add your API key in Rivet's plugin interface.

If you don't have a Gentrace API key, create one here.

Once the Gentrace plugin is installed, two buttons show up next to the "Run" button in a Rivet graph view.

Associating a Gentrace pipeline

The "Change Gentrace Pipeline" button associates a Gentrace pipeline with a Rivet graph.

Running Gentrace tests

The "Run Gentrace tests" graph pulls and runs test cases defined in the Gentrace pipeline through the Rivet graph.

To make this more concrete, let's say you define 100 example test cases for a Gentrace pipeline. Each test case has the following schema.

typescript
type EmailTestCase = {
  "query": string;
  "sender": string;
  "receiver": string;
};

The plugin will pull all test cases and invoke the Rivet graph once per case. During each invocation, each key-value pair from a test case maps onto a Graph Input Rivet node with a matching ID.

With the TestCase schema above, three Graph Input Rivet nodes are required to properly run the graph.

Viewing test results

Once the Gentrace plugin finishes running all test cases through the Rivet graph, the plugin will show a toast notification with a link to the results.

Tracing Rivet graph executions

Gentrace provides an NPM library that allows users to trace Rivet graphs.

caution

Since the Rivet runtime is only available for JavaScript environments, we only provide a Node.js library.

Installation

bash
# 🚧 Execute only one, depending on your package manager. The core and Rivet integration 
# libraries are both required.
npm i @gentrace/core @gentrace/rivet-node
yarn add @gentrace/core @gentrace/rivet-node
pnpm i @gentrace/core @gentrace/rivet-node

The @gentrace/rivet-node is a near drop-in replacement for the @ironclad/rivet-node library. We will detail the key differences below.

Usage

If you define a Rivet project in a chat.rivet-project file, you can execute the following code to track a graph's execution in Gentrace.

In this case, we run the Play 24 graph.

typescript
import { init } from "@gentrace/core";
import { runGraphInFile } from "@gentrace/rivet-node";
import { env } from "process";
 
init({
  apiKey: env.GENTRACE_API_KEY
});
 
const GENTRACE_PIPELINE_SLUG = "rivet-trace";
 
const { outputs, pipelineRunId, pipelineRun } = await runGraphInFile(
  env.RIVET_PROJECT_FILE!, // Location: /rivet/projects/chat.rivet-project
  {
    graph: env.RIVET_GRAPH_ID,
    openAiKey: env.OPENAI_KEY,
    inputs: {
      messages: {
        type: "chat-message[]",
        value: [
          {
            type: "user",
            message: "5 5 1 6",
          },
          {
            type: "user",
            message:"I should have provided you some numbers in my previous message. Output these numbers, separated by commas. For example, \"5, 5, 3, 4.\""
          },
        ],
      },
    },
  },
  GENTRACE_PIPELINE_SLUG,
);
 
console.log("Outputs: ", outputs);
 

Interface differences

Parameters

The main difference between the @gentrace/rivet-node and @ironclad/rivet-node libraries is that the former requires an additional pipeline slug parameter. This slug associates the Rivet graph with a Gentrace pipeline.

typescript
const { outputs, pipelineRunId, pipelineRun } = await runGraphInFile(
  env.RIVET_PROJECT_FILE!, // Location: /rivet/projects/chat.rivet-project
  {
    graph: env.RIVET_GRAPH_ID,
    openAiKey: env.OPENAI_KEY,
    inputs: {
      messages: {
        type: "chat-message[]",
        value: [
          {
            type: "user",
            message: "5 5 1 6",
          },
          {
            type: "user",
            message:"I should have provided you some numbers in my previous message. Output these numbers, separated by commas. For example, \"5, 5, 3, 4.\""
          },
        ],
      },
    },
  },
  GENTRACE_PIPELINE_SLUG,
);
 
console.log("Outputs: ", outputs);

Return value

Old shape of the data.

typescript
// 🛑 Former
type OldRunGraphReturnValueType = Record<string, DataValue>;

The modified runGraphInFile function now returns an object with the following shape.

typescript
import { type DataValue } from "@ironclad/rivet-node";
import { PipelineRun } from "@gentrace/core";
// ✅ New
type NewRunGraphReturnValueType = {
  outputs: Record<string, DataValue>;
  pipelineRunId: string;
  pipelineRun: PipelineRun;
};

typescript
const { outputs, pipelineRunId, pipelineRun } = await runGraphInFile(
  env.RIVET_PROJECT_FILE!, // Location: /rivet/projects/chat.rivet-project
  {
    graph: env.RIVET_GRAPH_ID,
    openAiKey: env.OPENAI_KEY,
    inputs: {
      messages: {
        type: "chat-message[]",
        value: [
          {
            type: "user",
            message: "5 5 1 6",
          },
          {
            type: "user",
            message:"I should have provided you some numbers in my previous message. Output these numbers, separated by commas. For example, \"5, 5, 3, 4.\""
          },
        ],
      },
    },
  },
  GENTRACE_PIPELINE_SLUG,
);
 
console.log("Return value: ", outputs, pipelineRunId, pipelineRun);

The outputs are the same as before. The pipelineRunId is the ID of the pipeline run in Gentrace. The pipelineRun is the full pipeline run object which captures the intermediate steps in the Rivet graph invocation. You can learn more about the pipeline run object here.

Trace UI

When you execute the code above, you will see a trace in the Gentrace UI.

The timeline trace displays a visual representation of the Rivet graph execution. The trace is interactive and allows you to inspect the inputs, outputs, and metadata of each node.