Decision Models
Answer choice, yes/no, and score questions with probabilities instead of text.
A decision model doesn't generate text. It reads a state, such as a support ticket or a JSON record, and answers typed questions about it with probabilities. Use decision models to:
- Classify text into one of your labels
- Check whether a statement about the input is true
- Rate the input on an ordered scale
DeepFellow serves decision models through the POST /v1/systemone endpoint. The endpoint isn't part of the OpenAI API, so the OpenAI client libraries have no method for it. Send the request with any HTTP client.
Install a Model
Before you call the endpoint, install a decision model in DeepFellow Infra. Three services serve decision models:
- ollama – decision models from the
ollamamodel catalog, such asnimble,nimble-v3,tev1,clef, andclef-flash. - ollama-external – decision models on a separate Ollama instance. DeepFellow Infra treats a synced model as a decision model when Ollama reports the
decisioncapability for it, or when the catalog lists the model as a decision model. - ollama-cloud – decision models hosted on Ollama Cloud, such as
clef:27b. DeepFellow Infra treats a model as a decision model when ollama.com reports thedecisioncapability for it. The service offers only the models that Ollama Cloud hosts, so local-only decision models, such asclef-flash, aren't available on it.
Decision models have the model type decision and serve only /v1/systemone. A chat request for a decision model fails. To install a decision model that isn't in the catalog, add it as a custom model with the decision model type.
Decision models on the ollama and ollama-external services need Ollama 0.35.0 or newer. The default image of the ollama service meets this requirement.
An older Ollama runs in an ollama service with an older Docker image version, or on an outdated external server of an ollama-external service. On an older Ollama, installing a decision model fails with 400 and the following message: "Decision models need Ollama 0.35.0 or newer; this Ollama is OLLAMA_VERSION." The ollama-external service also skips decision models when it syncs.
DeepFellow Infra reads the version of an external Ollama when it installs the ollama-external service and when it starts. After you update the external Ollama, restart DeepFellow Infra.
In a mesh, update every DeepFellow Infra node before any node has a decision model. A node on an older version rejects the whole model list of a node that reports a decision model, so that node won't join the mesh.
To check that an installed decision model works, click "Test" on its model card. DeepFellow Infra sends the model one choice question and shows the option it chose. For more information, see Testing Models.
Making a Decision Request
To ask a decision model a question, send the model name, the state, and a map of named questions:
curl -X 'POST' \
'https://deepfellow-server-host/v1/systemone' \
-H "Authorization: Bearer DEEPFELLOW_PROJECT_API_KEY" \
-H 'Content-Type: application/json' \
-d '{
"model": "nimble",
"state": "Our checkout has returned 500 errors since 9am.",
"questions": {
"label": {
"type": "choice",
"instructions": "Which label fits this ticket?",
"criteria": {
"billing": "Payments and refunds",
"bug": "Software errors",
"account": "Login and account access"
}
}
}
}'The response has one answer for each question, keyed by the question name:
{
"model": "nimble",
"answers": {
"label": {
"type": "choice",
"choice": "bug",
"probabilities": {
"billing": 0.0125,
"bug": 0.9781,
"account": 0.0093
},
"confidence": 0.8906
}
},
"usage": {
"input_tokens": 174,
"output_tokens": 1
}
}The endpoint doesn't stream. It returns the whole response at once.
Request Fields
| Field | Required | Description |
|---|---|---|
model | Yes | The decision model to use. The model must be allowed for your project. |
state | Yes | The input that every question is about: a string, a JSON object, or a JSON array. Some models accept null. |
questions | Yes | An object that maps a question name to a question. The model answers each question on its own, so answers aren't passed between questions. |
images | No | Base64-encoded images shared by all questions. Only a vision decision model accepts images. |
keep_alive | No | How long the model stays loaded after the request, as a duration string such as 5m or a number of seconds. |
DeepFellow Server passes fields it doesn't recognize to the model unchanged.
Question Types
Every question has a type and optional instructions, which accept the same content as state. Some models require instructions, and others reject them.
Choice
A choice question picks one option from the options you define. Set criteria to an object that maps each option key to a description, or to null for a bare label.
The answer contains the following fields:
choice: the option with the highest probabilityprobabilities: the probability of each option, summing to 1confidence: how strongly one option dominates, as described in Confidence
Yes/No
A noul question asks whether a statement about the state is true. criteria is optional. To describe the outcomes, set it to an object with "false" and "true" keys.
The answer's noul field is the probability that the statement is true, from 0 to 1. It isn't a Boolean, so pick a threshold that suits your use case:
{
"type": "noul",
"noul": 0.71
}Score
A score question rates the state on an ordered scale. Set criteria to an array of level descriptions, from the lowest level at index 0 to the highest.
The answer contains the following fields:
score: the probability-weighted average of the level indices, for example2.36on a scale from0to3legend: the description of each level, keyed by the level indexprobabilities: the probability of each level, keyed by the level indexconfidence: how strongly one level dominates, as described in Confidence
score falls between levels and isn't normalized to the range from 0 to 1.
For example, a score question with the criteria ["trivial", "minor", "major", "critical"] returns:
{
"type": "score",
"score": 2.36,
"legend": {"0": "trivial", "1": "minor", "2": "major", "3": "critical"},
"probabilities": {"0": 0.01, "1": 0.02, "2": 0.58, "3": 0.39},
"confidence": 0.58
}Confidence
confidence measures how concentrated the probabilities are, from 0 to 1. A value of 0 means every candidate is equally likely, and a value near 1 means one candidate dominates. It isn't a calibrated measure of accuracy: an answer with probabilities of 0.51 and 0.49 isn't a confident answer, even though one option wins.
Errors
| Status | Cause |
|---|---|
400 | The request is missing model, state, or questions, a question has an unknown type, or a choice or score question has no criteria. DeepFellow Server rejects the request before it reaches the model. |
401 | The request has no valid API key. |
403 | The model isn't allowed for your project. |
500 | DeepFellow Infra rejected the API key of DeepFellow Server. |
DeepFellow Server returns every other error from DeepFellow Infra unchanged, with its status code and body. For example, the model rejects a question with too many options with a 400. Limits such as the number of questions, options, or score levels depend on the model.
If DeepFellow Infra doesn't support decision models yet, the endpoint returns 405. If the model isn't installed in DeepFellow Infra, the endpoint returns 404.
Ollama Limits
Ollama enforces the following limits on a decision request:
- 1 to 64 questions
- 2 to 255 options in a
choicequestion, depending on the model - 2 to 10 levels in a
scorequestion - At most 64 KiB of text and question definitions, not counting images, and at most 32 MiB for the whole request
- Images only for a vision decision model, such as
clef
A request over the size limits fails with 413. The state, the questions, and the images must fit the context window of the loaded model, because Ollama never truncates them. On the ollama service, a decision model runs with the context length of the service. The context length that you set when you install a decision model has no effect.
Monitoring
DeepFellow Server records the token usage of every successful decision request in the gen_ai_client_token_usage metric and on the request's trace span. When tracing is enabled, the span also carries the request's state in the body.state attribute. PII anonymization doesn't apply to decision requests, so body.state includes any personal data that the state contains. For more information, see Observability.
For more information, see API Reference.
We use cookies on our website. We use them to ensure proper functioning of the site and, if you agree, for purposes such as analytics, marketing, and targeting ads.