Adding images to test cases (for multi-modal)

If you're evaluating multi-modal pipelines, you can upload images to Gentrace and use them as inputs to your test cases.

Uploading files

If your pipeline depends on file inputs (e.g. images, PDFs), you can upload your files to our object storage. We then return an authenticated URL to add to your test case inputs.

TypeScript

python
import os
import gentrace
from dotenv import load_dotenv
load_dotenv()
gentrace.init(
    api_key=os.getenv("GENTRACE_API_KEY"), host="http://localhost:3000/api/v1"
)
GENTRACE_PIPELINE_SLUG = "main"
with open("/home/user/gentrace-icon.png", "rb") as f:
    # This SDK method receives a file handle and returns an authenticated URL
    url = gentrace.upload_file(f)
    print("Gentrace file URL: ", url)
    gentrace.create_test_case(
      	# Pipeline slug
        GENTRACE_PIPELINE_SLUG,
        {
            "name": 'Gentrace Icon',
            'inputs': {
              	# Any Gentrace-uploaded URL will be rendered in a pretty format in the UI
                'iconUrl': url,
            },
            "expectedOutputs": {
                "value": "Gentrace logo"
            }
        }
    )

When viewing the test cases with Gentrace file URLs, we detect image extensions and render them directly in the UI.

Uploading file content bytes

If your file does not have a presence on the filesystem, you can directly upload the bytes to Gentrace. This approach requires you to name your file.

Python only

Uploading file content bytes is only supported in Python at this time.

Python

python
import gentrace
gentrace.init(
    api_key=os.getenv("GENTRACE_API_KEY")
)
with open("examples/files/gentrace-icon.png", "rb") as f:
    # Remember to specify the file extension! The Gentrace UI relies on the file
    # extension to render file contents correctly.
    url = gentrace.upload_bytes("gentrace-icon.png", f.read())
print("Uploaded URL: ", url)
gentrace.create_test_case(
    "main",
    {
        "name": 'Gentrace Icon',
        'inputs': {
            'imageUrl': url,
        },
        "expectedOutputs": {
            "value": "Gentrace logo"
        }
    }
)

Python

typescript
import { init, uploadBuffer, createTestCase } from "@gentrace/core";
import fs from "fs/promises";
 
init({
  apiKey: process.env.GENTRACE_API_KEY,
});
 
async function upload() {
  const buffer = await fs.readFile("./icon.png");
  // This SDK method receives a file name + buffer and returns an authenticated URL
  const url = await uploadBuffer("icon.png", buffer);
  
  const caseId = await createTestCase({
    pipelineSlug: "main",
    name: "Gentrace Icon",
    inputs: {
      // Any Gentrace-uploaded URL will be rendered in a pretty format in the UI
      iconUrl: url
    },
    expectedOutputs: {
      value: "Gentrace Icon"
    },
  });
 
  console.log("Case ID", caseId);
}
 
upload();
 

When viewing the test cases with Gentrace file URLs, we detect image extensions and render them directly in the UI.

Retrieving files

All uploaded files require authentication by an API key. When pulling test cases for a Gentrace pipeline, you need to construct an authenticated HTTP request to download the files associated with each case.

Here's a script that:

• Pulls test cases for a Gentrace pipeline
• Downloads the Gentrace-uploaded images from the input URL
• Runs the image data through our AI business logic
• Submits the outputs for grading

TypeScript

python
import os
from urllib.parse import urlparse
import gentrace
import requests
from dotenv import load_dotenv
# Import your AI pipeline
from ai.pipelines import image_to_words
GENTRACE_PIPELINE_SLUG = "main"
load_dotenv()
gentrace.init(
    api_key=os.getenv("GENTRACE_API_KEY")
)
cases = gentrace.get_test_cases(pipeline_slug=GENTRACE_PIPELINE_SLUG)
outputs = []
for case in cases:
    image_url = case.get("inputs").get("imageUrl")
    # Image URLs are authenticated. You must provide an API key as the bearer token.
    headers = {
        'Authorization': 'Bearer {}'.format(os.getenv("GENTRACE_API_KEY"))
    }
    response = requests.get(image_url, headers=headers)
    
    # Run the AI pipeline on the raw image content
    image_description = image_to_words(response.content)
    outputs.append({
      "value": image_description
    })
response = gentrace.submit_test_result(GENTRACE_PIPELINE_SLUG, cases, outputs)
print(response["resultId"])
  

Python

typescript
import { init, submitTestResult, getTestCases } from "@gentrace/core";
import { imageToWords } from "../api/pipelines";
 
const GENTRACE_PIPELINE_SLUG = "main";
const GENTRACE_API_KEY = process.env.GENTRACE_API_KEY;
 
init({
  apiKey: GENTRACE_API_KEY,
});
 
async function runTest() {
  const cases = await getTestCases(GENTRACE_PIPELINE_SLUG);
 
  const outputs: Record<string, any>[] = [];
 
  for (const testCase of cases) {
    const url = testCase.inputs.imageUrl;
    const response = await fetch(url, {
      method: "GET",
      headers: {
        Authorization: `Bearer ${GENTRACE_API_KEY}`,
      },
    });
 
    const blob = await response.blob();
    const words = await imageToWords(blob);
    outputs.push({
      value: words,
    });
  }
 
  const response = await submitTestResult(
    GENTRACE_PIPELINE_SLUG,
    cases,
    outputs,
  );
 
  console.log("Result ID:", response.resultId);
}
 
runTest();