Metadata Expansion

Metadata Expansion extracts fields you define, such as author, title, or any custom field, from a batch of previously uploaded documents. For each document, it reads embedded file properties first (PDF, DOCX, and PPTX core properties), then falls back to an LLM for whatever fields are still missing.

Jobs run in the background via Nightshift, so you can queue a batch and poll for results instead of waiting on the request.

Create a Job

Upload your files first with POST /v1/files (see Upload Files), then queue metadata extraction for them:

curl -X 'POST' \
  'https://deepfellow-server-host/v1/metadata/expand' \
  -H "Authorization: Bearer DEEPFELLOW-PROJECT-API-KEY" \
  -H 'Content-Type: application/json' \
  -d '{
  "document_ids": ["FILE_ID_1", "FILE_ID_2"],
  "fields": [
    {"name": "author"},
    {"name": "title", "description": "The document title as it appears on the cover page."}
  ]
}'

import requests

response = requests.post(
    "https://deepfellow-server-host/v1/metadata/expand",
    headers={"Authorization": "Bearer DEEPFELLOW-PROJECT-API-KEY"},
    json={
        "document_ids": ["FILE_ID_1", "FILE_ID_2"],
        "fields": [
            {"name": "author"},
            {"name": "title", "description": "The document title as it appears on the cover page."},
        ],
    },
)

print(response.json())

const response = await fetch('https://deepfellow-server-host/v1/metadata/expand', {
    method: 'POST',
    headers: {
        Authorization: 'Bearer DEEPFELLOW-PROJECT-API-KEY',
        'Content-Type': 'application/json'
    },
    body: JSON.stringify({
        document_ids: ['FILE_ID_1', 'FILE_ID_2'],
        fields: [
            { name: 'author' },
            { name: 'title', description: 'The document title as it appears on the cover page.' }
        ]
    })
});

const data = await response.json();
console.log(data);

Response:

{
    "id": "68da445c5186deb8bca2bde9",
    "project_id": "68da445c5186deb8bca2bdea",
    "status": "pending",
    "fields": [
        {"name": "author", "description": null},
        {"name": "title", "description": "The document title as it appears on the cover page."}
    ],
    "documents": [
        {"document_id": "FILE_ID_1", "status": "pending", "extracted": null, "error": null, "attempts": 0},
        {"document_id": "FILE_ID_2", "status": "pending", "extracted": null, "error": null, "attempts": 0}
    ],
    "created_at": 1735689600,
    "finished_at": null
}

Each field in fields needs a unique name. description is optional and guides the LLM fallback when a field is not present as embedded metadata; it has no effect on which embedded properties get read.

Creating a job returns 400 Bad Request if any document_ids entry doesn't exist or doesn't belong to your project, and 503 Service Unavailable if Nightshift is enabled but not currently connected to RabbitMQ.

Describe Images before Extraction

Pass image_processing_mode: "description" to have a vision model describe the images in a document before the LLM fallback runs on it, matching the option available on Vector Store file ingestion. This only affects documents that need the LLM fallback; it has no effect on fields already found as embedded metadata. The default is "ignore".

Check Job Status

curl -X 'GET' \
  'https://deepfellow-server-host/v1/metadata/jobs/JOB_ID' \
  -H "Authorization: Bearer DEEPFELLOW-PROJECT-API-KEY"

The job's status is one of pending, processing, completed, failed, paused, or cancelled. completed covers partial failures too, so check each document's own status for details; failed is only set when every document in the job failed.

Once a document completes, its extracted field holds the requested metadata, keyed by field name:

{
    "document_id": "FILE_ID_1",
    "status": "completed",
    "extracted": {"author": "Jane Doe", "title": "Quarterly Report"},
    "error": null,
    "attempts": 1
}

Extracted metadata is also written back onto the file itself, so GET /v1/files/FILE_ID returns it under extracted_metadata without needing the job ID. A later job for the same document overwrites its previous value.

Pause, Resume, Retry, and Cancel

• POST /v1/metadata/jobs/{job_id}/pause stops a job's documents from advancing further. Pausing an already-completed, failed, or cancelled job returns 409 Conflict.
• POST /v1/metadata/jobs/{job_id}/resume re-queues a paused job's pending documents. Resuming a job that is not paused returns 409 Conflict.
• POST /v1/metadata/jobs/{job_id}/retry resets the job's failed documents to pending and re-queues them, each with a fresh attempt budget. A job with no failed documents, or a cancelled job, returns 409 Conflict.
• POST /v1/metadata/jobs/{job_id}/cancel permanently stops a job's documents from advancing, for example after starting the same batch twice by mistake. Unlike pausing, cancelling cannot be undone. Cancelling an already-completed, failed, or cancelled job returns 409 Conflict.

All four return the updated job, or 404 Not Found if job_id doesn't exist.

See Nightshift Background Jobs for how these interact with retry and dispatch when Nightshift is enabled or disabled.