Observe quickstart (Pinecone)

This guide covers how to get started with Gentrace Observe for Pinecone. Observe monitors your logged requests and tracks speed, cost and aggregate statistics.

The other critical part of our product is Gentrace Evaluate, which automates grading your generative pipeline on a variety of dimensions. We will not cover Evaluate in this guide. Visit here to get started with Evaluate.

Installation

This package is the simplest way to interact with our service in your application.

TypeScript

bash
# Execute only one, depending on your package manager
npm i @gentrace/core
yarn add @gentrace/core
pnpm i @gentrace/core

Python

bash
# Execute only one, depending on your package manager
pip install gentrace-py
poetry add gentrace-py

typescript

If you want to use our provider SDK handlers, you must install our associated plugin SDKs. These SDKs have a direct dependency on the officially supported SDK for their respective providers. We type match the official SDK whenever possible.

shell
# For Pinecone v1 (the new version)
npm install @gentrace/pinecone@v1
# For Pinecone v0 (the older version)
npm install @gentrace/pinecone@v0

These NPM packages will only work with Node.JS versions >= 16.16.0.

🛑Server-side only

Please only use this library on the server-side. Using it on the client-side will reveal your API key.

python

This PyPI package will only work with Python versions >= 3.7.1.

Simple SDK

typescript

Requires @gentrace/pinecone@v1 or @gentrace/pinecone@v0

This section requires Gentrace's official Pinecone plugin. The plugin version matches the major version of the official Pinecone Node.JS SDK.

Pinecone v0 deprecated

As of September 7th, 2023, the Pinecone Node v0 (0.X.X) SDK has been deprecated. We recommend that you update your code to v1.

We still support both Pinecone SDK versions as separate plugins (@gentrace/pinecone@v1 and @gentrace/pinecone@v0).

We designed our SDKs to preserve the original interface to Pinecone's client library. Here's how you instantiate the Pinecone client with Gentrace.

typescript
import { init } from "@gentrace/core";
import { Pinecone } from "@gentrace/pinecone";
 
// This function globally initializes Gentrace with the appropriate 
// credentials. Constructors like PineconeClient() will transparently use
// these credentials to authenticate with Gentrace.
init({
  apiKey: process.env.GENTRACE_API_KEY!
});
 
const pinecone = new Pinecone({
  apiKey: process.env.PINECONE_API_KEY!,
});

The PineconeClient class has an identical interface to the equivalent class in Pinecone's official SDK.

Here's example code that queries for similar vectors in Pinecone.

typescript
import { init } from "@gentrace/core";
import { Pinecone } from "@gentrace/pinecone";
import { DEFAULT_VECTOR } from "../utils";
 
// This function globally initializes Gentrace with the appropriate 
// credentials. Constructors like PineconeClient() will transparently use
// these credentials to authenticate with Gentrace.
init({
  apiKey: process.env.GENTRACE_API_KEY!
});
 
const pinecone = new Pinecone({
  apiKey: process.env.PINECONE_API_KEY!,
});
 
async function queryPineconeIndex() {
  const index = await pinecone.Index("openai-trec");
  
  const queryResponse = await index.query({
    includeValues: true,
    topK: 3,
    vector: DEFAULT_VECTOR,
  });
 
  console.log("queryResponse", queryResponse);
}
 
queryPineconeIndex();
 

You should provide a Pipeline slug as a request parameter to any supported method that you want to instrument. This ID associates Pinecone invocations to that identifier on our service. If you omit the slug, we will not track telemetry for that invocation.

The PipelineRun ID provided by the Pinecone Index methods is from Gentrace. Our SDK provides this for you to uniquely associate feedback with AI generated content. If you do not provide a Pipeline slug, the methods will not return a PipelineRun ID.

python

Install official Pinecone SDK

Our library requires the Pinecone SDK to be installed to work. You can install their official package with pip install pinecone-client or poetry add pinecone-client.

We designed our SDKs to entirely preserve the original interface to Pinecone's client library. You can simply insert the following lines of code before your Pinecone invocation.

python
import os
import gentrace
gentrace.init(
  api_key=os.getenv("GENTRACE_API_KEY")
)
gentrace.configure_pinecone()

The gentrace.configure_pinecone() function configures the Pinecone SDK automatically tracks invocations to Pinecone and asynchronously forwards information to our service. Our SDK will not increase request latency to Pinecone.

Here's example code that queries for similar vectors in Pinecone.

python
import os
import gentrace
import pinecone
gentrace.init(
  api_key=os.getenv("GENTRACE_API_KEY")
)
gentrace.configure_pinecone()
pinecone.init(
    api_key=os.getenv("PINECONE_API_KEY"),
)
index = pinecone.Index("openai-trec")
query_result = index.query(
    top_k=10, vector=DEFAULT_VECTOR, pipeline_slug="self-contained-pinecone-query"
)
gentrace.flush()
print("Result: ", query_result["pipelineRunId"])

You should provide a Pipeline slug as a request parameter to any supported method that you want to instrument. This ID associates Pinecone invocations to that identifier on our service. If you omit the slug, we will not track telemetry for that invocation.

The PipelineRun ID provided by the Pinecone Index methods is from Gentrace. Our SDK provides this for you to uniquely associate feedback with AI generated content. If you do not provide a Pipeline slug, the methods will not return a PipelineRun ID.

Telemetry support

We automatically capture analytics only from methods within an Index() class instance. We plan to support other methods upon request.

typescript

• Index()
  • fetch()
  • update()
  • query()
  • upsert()
  • delete()

python

• Index()
  • fetch()
  • update()
  • query()
  • upsert()
  • delete()

Advanced usage

The SDKs described above are designed for creating single invocations to one provider like OpenAI or Pinecone. We also provide abstractions for chaining multiple invocations together into a single pipeline.

Creating Pipeline and PipelineRuns

To declare a Pipeline, you must define the configuration (including API keys) for Gentrace and the services you intend to monitor.

typescript

typescript
import { init, Pipeline } from "@gentrace/core";
import { initPlugin as initOpenAIPlugin } from "@gentrace/openai";
// This function globally initializes Gentrace with the appropriate 
// credentials. Constructors like Pipeline() will transparently use
// these credentials to authenticate with Gentrace.
init({
  apiKey: process.env.GENTRACE_API_KEY
});
const openaiPlugin = await initOpenAIPlugin({
  apiKey: process.env.OPENAI_KEY,
});
const pineconePlugin = await initPineconePlugin({
  apiKey: process.env.PINECONE_API_KEY,
});
const pipeline = new Pipeline({
  slug: "searchCompanyKnowledge",
  plugins: {
    openai: openaiPlugin,
    pinecone: pineconePlugin
  },
});

python

python
import gentrace
import os
pipeline = gentrace.Pipeline(
  "write-email",
  os.getenv("GENTRACE_API_KEY"),
  openai_config={
    "api_key": os.getenv("OPENAI_KEY"),
  },
  pinecone_config={
   	"api_key": os.getenv("PINECONE_API_KEY"), 
   	"environment": os.getenv("PINECONE_ENVIRONMENT")
  }
)
# Some providers like Pinecone perform a network request to bootstrap their package.
# We perform all global setup for any configured providers here.
pipeline.setup()

Global Pipeline

We designed the Pipeline class to specify the static, global configuration of a pipeline. Then, we expect users to use this global Pipeline reference to create additional PipelineRun instances via pipeline.start(). More on that below.

To create a PipelineRun, invoke the following code. The returned runner allows you to interact with providers like Pinecone.

typescript

typescript
const runner = await pipeline.start();

python

python
runner = pipeline.start()

To access a handle on a supported provider like OpenAI or Pinecone, invoke the following code.

typescript

typescript
// For OpenAI
const openAi = runner.openai;
// For Pinecone (relevant for this guide)
const pinecone = runner.pinecone;

python

python
# For OpenAI
open_ai = runner.get_openai()
# For Pinecone (relevant for ths guide)
pinecone = runner.get_pinecone()

You can then access methods for these external services on the handlers. These clients are nearly API-compatible with their equivalent official SDKs. There are a few key differences we’ll get into later when we cover each provider in detail.

Submission

Once you've invoked all the requests you need, you can submit this data to our external provider with the following code. This functionality asynchronously sends the PipelineRun data to our servers and returns a PipelineRun ID that you can send to your client.

typescript

typescript
const { pipelineRunId } = await runner.submit()

If you want to wait for the result of the submission request, you can invoke the following.

typescript
const { pipelineRunId } = await runner.submit({
  waitForServer: true
})
// This will block until the request returns

python

python
info = runner.submit()
pipeline_run_id = info["pipelineRunId"]

If you want to wait for the result of the submission request, you can invoke the following.

python
info = runner.submit(wait_for_server=True)
# This will block until the request returns

If you want to wait for the submission request using Python's asynchronous primitives, you can invoke the following.

python
import asyncio
async def main():
  # Runner setup
	info = await runner.asubmit()
  

The PipelineRun ID is used to associate user feedback with the generated content. It is important to pass this ID to your client application so that you can effectively link user feedback to the corresponding AI-generated content. To facilitate this association, you can use the browser Feedback SDK.

Advanced SDK

typescript

Requires @gentrace/pinecone@v1 or @gentrace/pinecone@v0

This section requires Gentrace's official Pinecone plugin. The plugin version matches the major version of the official Pinecone Node.JS SDK.

Our package provides a type-match for the Pinecone Node.JS SDK. To get an instrumented version of the Pinecone SDK, simply invoke the following code.

typescript
const pinecone = pipeline.pinecone;
const vectorResponse = await pinecone.Index('main').query(vectorQuery);
const snippets = // process vectorResponse

You can then invoke functions against the resulting handle that match the official SDK.

Note that in the Simple SDK, you had to specify a pipelineSlug for your invocations. If you're using the Pipeline object, the Pipeline slug is declared explicitly in the Pipeline object constructor. Similarly, the result of an invocation will not return a PipelineRun ID.

python

Install official Pinecone SDK

Our library requires the Pinecone SDK to be installed to work. You can install their official package with pip install pinecone-client or poetry add pinecone-client.

Our package provides a type-match for the Pinecone Python SDK. To get an instrumented version of the Pinecone SDK, simply invoke the following code.

python
pinecone = pipeline.get_pinecone()
vectorResponse = pinecone.Index('main').query(vectorQuery)
snippets = # process vectorResponse

You can then invoke functions against the resulting handle that match the official SDK.

Note that in the Simple SDK, you had to specify a pipeline_slug for your invocations. If you're using the Pipeline object, the Pipeline slug is declared explicitly in the Pipeline object constructor. Similarly, the result of an invocation will not return a PipelineRun ID.

Configuration

typescript

To configure Gentrace's Node.JS SDK with Pinecone, you must initialize a plugin using the initPlugin() method exported from every Gentrace plugin. Then, pass the same parameter object that you would pass to the PineconeClient constructor as the first parameter to initPlugin().

typescript
import { init, Pipeline }  from "@gentrace/core";
import { initPlugin } from "@gentrace/pinecone";
// This function globally initializes Gentrace with the appropriate 
// credentials. Constructors like Pipeline() will transparently use
// these credentials to authenticate with Gentrace.
init({
  apiKey: process.env.GENTRACE_API_KEY
});
const pineconePlugin = await initPlugin({
	apiKey: process.env.PINECONE_API_KEY,
});
const pipeline = new Pipeline({
  slug: "searchCompanyKnowledge",
  // ... Other configuration
  plugins: {
   	pinecone: pineconePlugin 
  }
});

python

To configure Gentrace's Python SDK with Pinecone, pass the Pinecone configuration to the Pipeline constructor as the pinecone_config keyword parameter.

python
import gentrace
import os
gentrace.init(
  api_key=os.getenv("GENTRACE_API_KEY")
)
pipeline = gentrace.Pipeline(
  slug="write-email",
  pinecone_config={
    "api_key": os.getenv("PINECONE_API_KEY"),
    "environment": os.getenv("PINECONE_ENVIRONMENT"),
  }, 
)

Telemetry support

We automatically capture analytics only from methods within an Index() class instance. We plan to support other methods upon request.

typescript

• Index()
  • fetch()
  • update()
  • query()
  • upsert()
  • delete()

python

• Index()
  • fetch()
  • update()
  • query()
  • upsert()
  • delete()

Full example

Here's a full example of a PipelineRun invocation with multiple calls to OpenAI (quickstart here) and Pinecone.

typescript

typescript
export async function generateKnowledgeResponse(input: string) {
  const runner = pipeline.start();
  
  // Near type matches for the respective clients
  const openai = runner.openai;
  const pinecone = runner.pinecone;
  
  const embeddingResponse = await openai.embeddings.create({
    model: 'text-embedding-ada-002',
    input,
  });
  
  const vectorQuery = // process embedding response
  
  // getPinecone() returns a near type match for the Pinecone client
  const vectorResponse = await pinecone.Index('main').query(vectorQuery);
  const snippets = // process vectorResponse
  
  const response = await openai.completions.create({
    model: 'text-davinci-003',
    temperature: 0.7,
    // We do modify OpenAI in one way, splitting prompt into template and inputs
    // this allows us to monitor metrics changes due to the template
    promptTemplate: `Context:\n\n{{ snippets }}\n\n{{ input}}`,
    promptInputs: {
        input,
        snippets,
    },
  });
  
  // Data is submitted asynchronously
  await runner.submit();
  
  return response;
}

python

python
async def generate_knowledge_response(input: str):
    runner = pipeline.start()
    
    # Near type matches for the respective clients
    openai = runner.get_open_ai()
    pinecone = runner.get_pinecone()
    
    # Create embedding using OpenAI client
    embedding_response = openai.Embedding.create (
        model='text-embedding-ada-002',
        input=input
    )
    
    vector_query = ... # process embedding response
    
    vector_response = pinecone.Index('main').query(vector_query)
    snippets = ... # process vector_response
    
    # Create completion using OpenAI client
    response = openai.Completion.create(
        model='text-davinci-003',
        temperature=0.7,
        prompt_template='Context:\n\n{{ snippets }}\n\n{{ input}}',
        prompt_inputs={
            'input': input,
            'snippets': snippets
        }
    )
    
    runner.submit()
    
    return response