Create a fine-tuning pipeline
Create a pipeline that periodically fine-tunes base_model on Q&A generated from the project's vector stores.
Each run trains a new adapter version, scores it against the version being served, and serves it
under the pipeline's alias only if it scores better. The alias can only be used by this project.
In: header
Header Parameters
Organization Id from OpenAi compatible endpoint. Required to organization access or project access (with OpenAI-Project header) when logged as User or Admin User.
Project Id from OpenAi compatible endpoint. Required to project access for organization api key or User / Admin User access with OpenAI-Organization
Pipeline name; its alias (the served model name) is derived from it.
1 <= length <= 64Hugging Face ID of the base model to fine-tune.
1 <= lengthWhere a pipeline's training data comes from.
When a pipeline runs on its own.
{}Hyperparameters for fine-tuning.
{"n_epochs":"auto","batch_size":"auto","learning_rate_multiplier":"auto"}Quantization during training.
Fine-tuning worker of type infra the pipeline trains on; defaults to the registered one.
How a promoted version replaces the one being served.
"blue_green""blue_green" | "replace"When a candidate replaces the served version.
{"min_delta":0.2}Response Body
curl -X POST "https://deepfellow-server-host/v1/fine_tuning/pipelines" \ -H "OpenAI-Organization: string" \ -H "OpenAI-Project: string" \ -H "Content-Type: application/json" \ -d '{ "name": "string", "base_model": "string", "source": { "vector_store_ids": [ "5eb7cf5a86d9755df3a6c593" ] } }'{
"id": "5eb7cf5a86d9755df3a6c593",
"object": "fine_tuning.pipeline",
"project_id": "5eb7cf5a86d9755df3a6c593",
"name": "string",
"alias": "string",
"base_model": "string",
"source": {
"vector_store_ids": [
"5eb7cf5a86d9755df3a6c593"
]
},
"schedule": {
"interval_hours": 1
},
"hyperparameters": {
"n_epochs": "auto",
"batch_size": "auto",
"learning_rate_multiplier": "auto"
},
"quantization": "4bit",
"worker_id": "5eb7cf5a86d9755df3a6c593",
"serving_strategy": "blue_green",
"promotion": {
"min_delta": 0.2
},
"enabled": true,
"active_run_id": "5eb7cf5a86d9755df3a6c593",
"next_run_at": 0,
"eval_set": {
"file_id": "5eb7cf5a86d9755df3a6c593",
"chunk_hashes": [
"string"
],
"size": 0,
"version": 1,
"built_at": 0
},
"champion": {
"run_id": "5eb7cf5a86d9755df3a6c593",
"version": 0,
"adapter_id": "string",
"score": 0,
"eval_config_hash": "string",
"eval_set_version": 0,
"predictions_file_id": "5eb7cf5a86d9755df3a6c593",
"infra_model_id": "string"
},
"created_at": 0
}{
"detail": [
{
"loc": [
"string"
],
"msg": "string",
"type": "string",
"input": null,
"ctx": {}
}
]
}We use cookies on our website. We use them to ensure proper functioning of the site and, if you agree, for purposes such as analytics, marketing, and targeting ads.