Test results - Submit

TypeScript

The submitTestResult() function submits a test result for grading by the configured evaluators.

Python

The submit_test_result() function submits a test result for grading by the configured evaluators.

Arguments

typescript

pipelineSlug

The slug of the pipeline for which test results are being submitted. This string uniquely identifies the pipeline within the system.

testCases

An array of TestCase objects. Each TestCase object contains the details of a test case, including inputs, expected outputs, and other metadata related to the test case. View the types list in the "Get" section for the full property breakdown of a TestCase object.

outputs

An array of objects where each object represents the output of a test case through the pipeline. The outputs should be in the same order as the test cases provided.

python

pipeline_slug

The slug of the pipeline for which test results are being submitted. This string uniquely identifies the pipeline within the system.

test_cases

An array of TestCase objects. Each TestCase object contains the details of a test case, including inputs, expected outputs, and other metadata related to the test case. View the types list in the "Get" section for the full property breakdown of a TestCase object.

outputs_list

An array of objects where each object represents the output of a test case through the pipeline. The outputs should be in the same order as the test cases provided.

Return value

typescript

resultInfo

This endpoint returns a simple object with the test result ID as a UUID string. Here's an example response structure.

json
{
  "resultId": "FACB6642-4725-4FAE-9323-634E72533C89"
}

You can then use this ID to retrieve the test result using the getTestResult() function or check the status with the getTestResultStatus() function.

python

result_info

This endpoint returns a simple object with the test result ID as a UUID string. Here's an example response structure.

json
{
  "resultId": "FACB6642-4725-4FAE-9323-634E72533C89"
}

You can then use this ID to retrieve the test result using the get_test_result() function or check the status with the get_test_result_status() function.

Example

typescript

typescript
import { init, getTestCases, submitTestResult } from "@gentrace/core";
import { createAiOutput } from "../pipelines"; // TODO: replace with your own pipeline
 
const MY_PIPELINE_SLUG = "my-pipeline";
 
const testCases = await getTestCases(MY_PIPELINE_SLUG);
 
const outputs = testCases.map((testCase) => {
  return {
    output: createAiOutput(testCase.inputs),
  }
});
 
const resultInfo = await submitTestResult(MY_PIPELINE_SLUG, testCases, outputs);
 
console.log("Test result ID:", resultInfo.resultId);

python

python
import os
import gentrace
from app.pipelines import create_ai_output # TODO: replace with your own pipeline
gentrace.init(
    api_key=os.getenv("GENTRACE_API_KEY"),
)
PIPELINE_SLUG = "example-pipeline"
cases = gentrace.get_test_cases(pipeline_slug=PIPELINE_SLUG)
outputs_list = []
for case in cases:
    outputs_list.append({
        "value": create_ai_output(case["inputs"])
    })
result = gentrace.submit_test_result(
    PIPELINE_SLUG, test_cases=cases, outputs_list=outputs_list
)
print("Test result ID:", result["resultId"])