Test results - Submit

TypeScript

The submitTestResult() function submits a test result for grading by the configured evaluators.

Python

The submit_test_result() function submits a test result for grading by the configured evaluators.

Example

typescript

typescript
import { init, getTestCases, submitTestResult } from "@gentrace/core";
import { createAiOutput } from "../pipelines"; // TODO: replace with your own pipeline
 
const MY_PIPELINE_SLUG = "my-pipeline";
 
const testCases = await getTestCases(MY_PIPELINE_SLUG);
 
const outputs = testCases.map((testCase) => {
  return {
    output: createAiOutput(testCase.inputs),
  }
});
 
const resultInfo = await submitTestResult(MY_PIPELINE_SLUG, testCases, outputs);
 
console.log("Test result ID:", resultInfo.resultId);

python

python
import os
import gentrace
from app.pipelines import create_ai_output # TODO: replace with your own pipeline
gentrace.init(
    api_key=os.getenv("GENTRACE_API_KEY"),
)
PIPELINE_SLUG = "example-pipeline"
cases = gentrace.get_test_cases(pipeline_slug=PIPELINE_SLUG)
outputs_list = []
for case in cases:
    outputs_list.append({
        "value": create_ai_output(case["inputs"])
    })
result = gentrace.submit_test_result(
    PIPELINE_SLUG, test_cases=cases, outputs_list=outputs_list
)
print("Test result ID:", result["resultId"])

Arguments

typescript

pipelineSlug: string

testCases: TestCase[]

View the types list in the Test Results → Get page for the full property breakdown of a TestCase object.

outputs: object[]

An array of objects where each object represents the output of a test case through the pipeline. The outputs should be in the same order as the test cases provided.

context?: { "name": String, "metadata": MetadataValueObject }

Context object where you can specify the name of the test result and any metadata you want to attach to it.

python

pipeline_slug: str

test_cases: List[TestCase]

View the types list in the Test Results → Get page for the full property breakdown of a TestCase object.

outputs_list: List[dict]

An array of objects where each object represents the output of a test case through the pipeline. The outputs should be in the same order as the test cases provided.

context: { "metadata": MetadataValueObject }

result_name: Optional[str]

Return value

typescript

resultInfo: { resultId: string }

json
{
  "resultId": "FACB6642-4725-4FAE-9323-634E72533C89"
}

You can then use this ID to retrieve the test result using the getTestResult() function or check the status with the getTestResultStatus() function.

python

result_info: TypedDict("ResultInfo", {"resultId": str})

json
{
  "resultId": "FACB6642-4725-4FAE-9323-634E72533C89"
}

You can then use this ID to retrieve the test result using the get_test_result() function or check the status with the get_test_result_status() function.

Types

🛠️ MetadataValueObject

type: string

{ [key: string]: any }