Skip to main content
Version: 2.0.0

Rivet Integration for Evaluate

Rivet is a visual programming environment for generative AI. Gentrace provides a Rivet plugin that allows users to associate a Rivet graph with a Gentrace [pipeline] (pipeline/basics).

Package version​

The Rivet Gentrace plugin is available in Rivet v1.2.0.

Installation​

Install Rivet from their Github releases page. Once installed, enable the Gentrace plugin and add your API key in Rivet's plugin interface.

If you don't have a Gentrace API key, create one here.

Enable the Gentrace plugin

Add Gentrace API key

Usage​

Once the Gentrace plugin is installed, two buttons show up next to the "Run" button in a Rivet graph view.

Associating a Gentrace pipeline​

The "Change Gentrace Pipeline" button associates a Gentrace pipeline with a Rivet graph.

Rivet change Gentrace pipeline

Running Gentrace tests​

The "Run Gentrace tests" graph pulls and runs test cases defined in the Gentrace pipeline through the Rivet graph.

To make this more concrete, let's say you define 100 example test cases for a Gentrace pipeline. Each test case has the following schema.

typescript
type EmailTestCase = {
  "query": string;
  "sender": string;
  "receiver": string;
};

The plugin will pull all test cases and invoke the Rivet graph once per case. During each invocation, each key-value pair from a test case maps onto a Graph Input Rivet node with a matching ID.

With the TestCase schema above, three Graph Input Rivet nodes are required to properly run the graph.

Rivet graph input nodes

Viewing test results​

Once the Gentrace plugin finishes running all test cases through the Rivet graph, the plugin will show a toast notification with a link to the results.