Skip to main content
Version: 4.6.19

Pinecone

This guide covers our Pinecone integration. Gentrace monitors your logged Pinecone requests and tracks speed, cost, and aggregate statistics.

Installation​

This package is the simplest way to interact with our service in your application.

bash
# Execute only one, depending on your package manager
npm i @gentrace/core
yarn add @gentrace/core
pnpm i @gentrace/core

If you want to use our provider SDK handlers, you must install our associated plugin SDKs. These SDKs have a direct dependency on the officially supported SDK for their respective providers. We type match the official SDK whenever possible.

shell
# For Pinecone v1 (the new version)
npm install @gentrace/pinecone@v1
# For Pinecone v0 (the older version)
npm install @gentrace/pinecone@v0

These NPM packages will only work with Node.JS versions >= 16.16.0.

🛑Server-side only

Please only use this library on the server-side. Using it on the client-side will reveal your API key.

Simple SDK​

Requires @gentrace/pinecone@v1 or @gentrace/pinecone@v0

This section requires Gentrace's official Pinecone plugin. The plugin version matches the major version of the official Pinecone Node.JS SDK.

Pinecone v0 deprecated

As of September 7th, 2023, the Pinecone Node v0 (0.X.X) SDK has been deprecated. We recommend that you update your code to v1.

We still support both Pinecone SDK versions as separate plugins (@gentrace/pinecone@v1 and @gentrace/pinecone@v0).

We designed our SDKs to preserve the original interface to Pinecone's client library. Here's how you instantiate the Pinecone client with Gentrace.

typescript
import { init } from "@gentrace/core";
import { Pinecone } from "@gentrace/pinecone";
 
// This function globally initializes Gentrace with the appropriate
// credentials. Constructors like PineconeClient() will transparently use
// these credentials to authenticate with Gentrace.
init({
apiKey: process.env.GENTRACE_API_KEY!
});
 
const pinecone = new Pinecone({
apiKey: process.env.PINECONE_API_KEY!,
environment: process.env.PINECONE_ENVIRONMENT!,
});

The PineconeClient class has an identical interface to the equivalent class in Pinecone's official SDK.

Here's example code that queries for similar vectors in Pinecone.

typescript
import { init } from "@gentrace/core";
import { Pinecone } from "@gentrace/pinecone";
import { DEFAULT_VECTOR } from "../utils";
 
// This function globally initializes Gentrace with the appropriate
// credentials. Constructors like PineconeClient() will transparently use
// these credentials to authenticate with Gentrace.
init({
apiKey: process.env.GENTRACE_API_KEY!
});
 
const pinecone = new Pinecone({
apiKey: process.env.PINECONE_API_KEY!,
environment: process.env.PINECONE_ENVIRONMENT!,
});
 
async function queryPineconeIndex() {
const index = await pinecone.Index("openai-trec");
const queryResponse = await index.query({
includeValues: true,
topK: 3,
vector: DEFAULT_VECTOR,
});
 
console.log("queryResponse", queryResponse);
}
 
queryPineconeIndex();
 

You should provide a Pipeline slug as a request parameter to any supported method that you want to instrument. This ID associates Pinecone invocations to that identifier on our service. If you omit the slug, we will not track telemetry for that invocation.

The PipelineRun ID provided by the Pinecone Index methods is from Gentrace. Our SDK provides this for you to uniquely associate feedback with AI generated content. If you do not provide a Pipeline slug, the methods will not return a PipelineRun ID.

Telemetry support​

We automatically capture analytics only from methods within an Index() class instance. We plan to support other methods upon request.

  • Index()
    • fetch()
    • update()
    • query()
    • upsert()
    • delete()

Advanced usage​

The SDKs described above are designed for creating single invocations to one provider like OpenAI or Pinecone. We also provide abstractions for chaining multiple invocations together into a single pipeline.

Creating Pipeline and PipelineRuns​

To declare a Pipeline, you must define the configuration (including API keys) for Gentrace and the services you intend to monitor.

typescript
import { init, Pipeline } from "@gentrace/core";
import { initPlugin as initOpenAIPlugin } from "@gentrace/openai";
// This function globally initializes Gentrace with the appropriate
// credentials. Constructors like Pipeline() will transparently use
// these credentials to authenticate with Gentrace.
init({
apiKey: process.env.GENTRACE_API_KEY
});
const openaiPlugin = await initOpenAIPlugin({
apiKey: process.env.OPENAI_KEY,
});
const pineconePlugin = await initPineconePlugin({
apiKey: process.env.PINECONE_API_KEY,
environment: process.env.PINECONE_ENVIRONMENT,
});
const pipeline = new Pipeline({
slug: "searchCompanyKnowledge",
plugins: {
openai: openaiPlugin,
pinecone: pineconePlugin
},
});
Global Pipeline

We designed the Pipeline class to specify the static, global configuration of a pipeline. Then, we expect users to use this global Pipeline reference to create additional PipelineRun instances via pipeline.start(). More on that below.

To create a PipelineRun, invoke the following code. The returned runner allows you to interact with providers like Pinecone.

typescript
const runner = await pipeline.start();

To access a handle on a supported provider like OpenAI or Pinecone, invoke the following code.

typescript
// For OpenAI
const openAi = runner.openai;
// For Pinecone (relevant for this guide)
const pinecone = runner.pinecone;

You can then access methods for these external services on the handlers. These clients are nearly API-compatible with their equivalent official SDKs. There are a few key differences we’ll get into later when we cover each provider in detail.

Submission​

Once you've invoked all the requests you need, you can submit this data to our external provider with the following code. This functionality asynchronously sends the PipelineRun data to our servers and returns a PipelineRun ID that you can send to your client.

typescript
const { pipelineRunId } = await runner.submit()

If you want to wait for the result of the submission request, you can invoke the following.

typescript
const { pipelineRunId } = await runner.submit({
waitForServer: true
})
// This will block until the request returns

The PipelineRun ID is used to associate user feedback with the generated content. It is important to pass this ID to your client application so that you can effectively link user feedback to the corresponding AI-generated content. To facilitate this association, you can use the browser Feedback SDK.

Advanced SDK​

Requires @gentrace/pinecone@v1 or @gentrace/pinecone@v0

This section requires Gentrace's official Pinecone plugin. The plugin version matches the major version of the official Pinecone Node.JS SDK.

Our package provides a type-match for the Pinecone Node.JS SDK. To get an instrumented version of the Pinecone SDK, simply invoke the following code.

typescript
const pinecone = pipeline.pinecone;
const vectorResponse = await pinecone.Index('main').query(vectorQuery);
const snippets = // process vectorResponse

You can then invoke functions against the resulting handle that match the official SDK.

Note that in the Simple SDK, you had to specify a pipelineSlug for your invocations. If you're using the Pipeline object, the Pipeline slug is declared explicitly in the Pipeline object constructor. Similarly, the result of an invocation will not return a PipelineRun ID.

Configuration​

To configure Gentrace's Node.JS SDK with Pinecone, you must initialize a plugin using the initPlugin() method exported from every Gentrace plugin. Then, pass the same parameter object that you would pass to the PineconeClient constructor as the first parameter to initPlugin().

typescript
import { init, Pipeline } from "@gentrace/core";
import { initPlugin } from "@gentrace/pinecone";
// This function globally initializes Gentrace with the appropriate
// credentials. Constructors like Pipeline() will transparently use
// these credentials to authenticate with Gentrace.
init({
apiKey: process.env.GENTRACE_API_KEY
});
const pineconePlugin = await initPlugin({
apiKey: process.env.PINECONE_API_KEY,
environment: process.env.PINECONE_ENVIRONMENT
});
const pipeline = new Pipeline({
slug: "searchCompanyKnowledge",
// ... Other configuration
plugins: {
pinecone: pineconePlugin
}
});

Telemetry support​

We automatically capture analytics only from methods within an Index() class instance. We plan to support other methods upon request.

  • Index()
    • fetch()
    • update()
    • query()
    • upsert()
    • delete()

Full example​

Here's a full example of a PipelineRun invocation with multiple calls to OpenAI (docs here) and Pinecone.

typescript
export async function generateKnowledgeResponse(input: string) {
const runner = pipeline.start();
// Near type matches for the respective clients
const openai = runner.openai;
const pinecone = runner.pinecone;
const embeddingResponse = await openai.embeddings.create({
model: 'text-embedding-ada-002',
input,
});
const vectorQuery = // process embedding response
// getPinecone() returns a near type match for the Pinecone client
const vectorResponse = await pinecone.Index('main').query(vectorQuery);
const snippets = // process vectorResponse
const response = await openai.completions.create({
model: 'text-davinci-003',
temperature: 0.7,
// We do modify OpenAI in one way, splitting prompt into template and inputs
// this allows us to monitor metrics changes due to the template
promptTemplate: `Context:\n\n{{ snippets }}\n\n{{ input}}`,
promptInputs: {
input,
snippets,
},
});
// Data is submitted asynchronously
await runner.submit();
return response;
}