CodeBasedEvaluatorDefinition interface

Code-based evaluator definition using python code

Properties

blob_uri	The blob URI for the evaluator storage
code_text	Inline code text for the evaluator
entry_point	The entry point Python file name for the uploaded evaluator code (e.g. 'answer_length_evaluator.py')
image_tag	The container image tag to use for evaluator code execution
type	The type discriminator, always 'code'.

data_schema	The JSON schema (Draft 2020-12) for the evaluator's input data. This includes parameters like type, properties, required.
init_parameters	The JSON schema (Draft 2020-12) for the evaluator's input parameters. This includes parameters like type, properties, required.
metrics	List of output metrics produced by this evaluator

The blob URI for the evaluator storage

blob_uri?: string

string

Inline code text for the evaluator

code_text?: string

string

The entry point Python file name for the uploaded evaluator code (e.g. 'answer_length_evaluator.py')

entry_point?: string

string

The container image tag to use for evaluator code execution

image_tag?: string

string

The type discriminator, always 'code'.

type: "code"

"code"

The JSON schema (Draft 2020-12) for the evaluator's input data. This includes parameters like type, properties, required.

data_schema?: Record<string, unknown>

Record<string, unknown>

Inherited From EvaluatorDefinition.data_schema

The JSON schema (Draft 2020-12) for the evaluator's input parameters. This includes parameters like type, properties, required.

init_parameters?: Record<string, unknown>

Record<string, unknown>

Inherited From EvaluatorDefinition.init_parameters

List of output metrics produced by this evaluator

metrics?: Record<string, EvaluatorMetric>

Record<string, EvaluatorMetric>

Inherited From EvaluatorDefinition.metrics