← Back to index Logout

Processing Pipeline

POST /api/v1/file/:file_uuid/process

Auth: Required Scope: file-level

Trigger the processing pipeline for a registered file. Creates a monitor job that the worker picks up and processes sequentially. Returns immediately with the job info—processing runs asynchronously in the background.

Request Parameters

Field Type Required Default Description
processors string[] No all Specific processors to run: ["cut","asr","asrx","yolo","ocr","face","pose","visual_chunk","story","5w1h"]
rules string[] No all Rule names to apply (currently unused)

Example

# Run all processors
curl -s -X POST "$API/api/v1/file/$FILE_UUID/process" \
  -H "Content-Type: application/json" \
  -H "X-API-Key: $KEY" -d '{}'

# Run specific processors only
curl -s -X POST "$API/api/v1/file/$FILE_UUID/process" \
  -H "Content-Type: application/json" \
  -H "X-API-Key: $KEY" \
  -d '{"processors": ["asr", "face", "yolo"]}'

Response (200)

{
  "success": true,
  "job_id": 42,
  "file_uuid": "3a6c1865...",
  "status": "processing",
  "pids": [12345, 12346],
  "message": "Processing triggered for video.mp4"
}
Field Type Description
success boolean Always true on 200
job_id integer Monitor job ID (for job tracking)
file_uuid string 32-char hex UUID of the file
status string "processing"
pids integer[] Process IDs of started processors
message string Human-readable status

Error Responses

HTTP When
404 File UUID not found
401 Missing or invalid API key

GET /api/v1/file/:file_uuid/probe

Auth: Required Scope: file-level

Get ffprobe metadata for a registered file. Returns video/audio stream info, codec details, duration, resolution, and frame rate.

Example

curl -s "$API/api/v1/file/$FILE_UUID/probe" -H "X-API-Key: $KEY"

Response (200)

{
  "file_uuid": "3a6c1865...",
  "file_name": "video.mp4",
  "file_size": 794863677,
  "duration": 120.5,
  "width": 1920,
  "height": 1080,
  "fps": 24.0,
  "total_frames": 2892,
  "cached": true,
  "format": {
    "filename": "/path/to/video.mp4",
    "format_name": "mov,mp4,m4a,3gp",
    "duration": "120.5",
    "size": "12345678",
    "bit_rate": "819200"
  },
  "streams": [
    {
      "index": 0,
      "codec_name": "h264",
      "codec_type": "video",
      "width": 1920,
      "height": 1080,
      "r_frame_rate": "24/1",
      "duration": "120.5"
    }
  ]
}
Field Type Description
file_uuid string 32-char hex UUID
file_name string File name
file_size integer File size in bytes (from filesystem)
duration float Duration in seconds
width integer Video width in pixels
height integer Video height in pixels
fps float Frames per second
total_frames integer Estimated total frames
cached boolean True if result was from cached probe JSON
format object Container format info (ffprobe format section)
streams array Array of stream info objects

POST /api/v1/progress/:file_uuid

Auth: Required Scope: file-level

Get real-time processing progress for a file via Redis pub/sub. Includes per-processor status, current/total frames, ETA, and system resource stats.

Note: This endpoint uses POST method, not GET. The progress data is stored in Redis as a hash, and POST is used to retrieve the latest state.

Pipeline Order

Order Processor Dependencies Description
1 cut Scene detection
2 asr cut Speech-to-text (per scene)
3 asrx asr Speaker diarization
4 yolo Object detection
5 ocr Text recognition
6 face Face detection & embedding
7 pose Pose estimation
8 visual_chunk yolo Visual scene chunks
9 story asr, asrx, cut, yolo, face Scene summaries (template)
10 5w1h story 5W1H analysis (Gemma4 LLM)

All processors except story and 5w1h run concurrently when their dependencies are met. Story and 5W1H run sequentially after their prerequisites.

Example

curl -s -X POST "$API/api/v1/progress/$FILE_UUID" -H "X-API-Key: $KEY" | jq '{overall_progress, processors: [.processors[] | {name, status}]}'

Response (200)

{
  "file_uuid": "3a6c1865...",
  "overall_progress": 71,
  "cpu_percent": 45.2,
  "gpu_percent": 30.1,
  "memory_percent": 62.4,
  "processors": [
    {"processor_type": "asr", "status": "complete", "progress": 100},
    {"processor_type": "yolo", "status": "running", "progress": 65},
    {"processor_type": "face", "status": "pending", "progress": 0}
  ]
}
Field Type Description
file_uuid string 32-char hex UUID
overall_progress integer Overall progress percentage (0–100)
processors array Per-processor status list
processors[].processor_type string Processor name (asr, cut, yolo, etc.)
processors[].status string "pending", "running", "complete", or "failed"
processors[].progress integer Per-processor progress (0–100)
processors[].eta_seconds integer Estimated seconds remaining (running processors)
processors[].current integer Current frame count
processors[].total integer Total frame count
cpu_percent float Current CPU usage
gpu_percent float Current GPU utilization
memory_percent float Current memory usage

GET /api/v1/jobs

Auth: Required Scope: system-level

List all processing jobs (monitor jobs) in the system. Shows job status, which file each job is processing, and current processor info.

Example

curl -s "$API/api/v1/jobs" -H "X-API-Key: $KEY" | jq '{count, jobs: [.jobs[] | {uuid, status}]}'

Response (200)

{
  "jobs": [
    {
      "id": 42,
      "uuid": "3a6c1865...",
      "status": "running",
      "current_processor": "yolo",
      "created_at": "2026-05-16T12:00:00Z",
      "started_at": "2026-05-16T12:01:00Z"
    }
  ],
  "count": 15,
  "page": 1,
  "page_size": 20
}
Field Type Description
jobs array Array of job info objects
jobs[].id integer Job ID
jobs[].uuid string File UUID being processed
jobs[].status string "pending", "running", "completed", "failed"
jobs[].current_processor string Currently active processor, or null
count integer Total job count
page integer Current page number
page_size integer Jobs per page

GET /api/v1/file/:file_uuid/processor-counts

Auth: Required Scope: file-level

Get counts of processor JSON output files. See 15_tkg.md for full documentation.


Pipeline Steps (Manual)

These endpoints execute individual pipeline steps. They are typically called by the worker automatically, but can be invoked manually for debugging or re-processing.

POST /api/v1/file/:file_uuid/store-asrx

Auth: Required Scope: file-level

Store ASRX diarization results as chunk records in the database. Converts ASRX segments into searchable chunk entries.

Example

curl -s -X POST "$API/api/v1/file/$FILE_UUID/store-asrx" \
  -H "X-API-Key: $KEY"

Response (200)

{
  "success": true,
  "message": "ASRX chunks stored",
  "file_uuid": "3a6c1865..."
}

POST /api/v1/file/:file_uuid/rule1

Auth: Required Scope: file-level

Execute Rule 1 pipeline step. Applies rule-based chunking to create structured chunk records from processor outputs.

Example

curl -s -X POST "$API/api/v1/file/$FILE_UUID/rule1" \
  -H "X-API-Key: $KEY"

Response (200)

{
  "success": true,
  "message": "Rule 1 complete: 45 chunks",
  "file_uuid": "3a6c1865...",
  "chunks": 45
}
Field Type Description
success boolean Always true on 200
message string Human-readable completion message
file_uuid string 32-char hex UUID
chunks integer Number of chunks produced

POST /api/v1/file/:file_uuid/vectorize

Auth: Required Scope: file-level

Generate vector embeddings for all chunks of a file and store them in Qdrant for semantic search.

Example

curl -s -X POST "$API/api/v1/file/$FILE_UUID/vectorize" \
  -H "X-API-Key: $KEY"

Response (200)

{
  "success": true,
  "message": "Vectorization complete",
  "file_uuid": "3a6c1865..."
}

POST /api/v1/file/:file_uuid/phase1

Auth: Required Scope: file-level

Execute Phase 1 of the post-processing pipeline. Combines store-asrx, rule1, and vectorize into a single step.

Example

curl -s -X POST "$API/api/v1/file/$FILE_UUID/phase1" \
  -H "X-API-Key: $KEY"

Response (200)

{
  "success": true,
  "message": "Phase 1 complete",
  "file_uuid": "3a6c1865..."
}

POST /api/v1/file/:file_uuid/complete

Auth: Required Scope: file-level

Mark a video as fully processed. Updates the video status to completed and finalizes all pipeline state.

Example

curl -s -X POST "$API/api/v1/file/$FILE_UUID/complete" \
  -H "X-API-Key: $KEY"

Response (200)

{
  "success": true,
  "message": "Video marked as completed",
  "file_uuid": "3a6c1865..."
}

Pipeline Step Order

  process (trigger)
    │
    ├─→ cut, yolo, ocr, face, pose, asrx (parallel processors)
    │
    ├─→ store-asrx  (store diarization as chunks)
    │
    ├─→ rule1       (rule-based chunking)
    │
    ├─→ vectorize   (embed chunks to Qdrant)
    │
    └─→ complete    (mark done)

Phase 1 (/phase1) combines store-asrx + rule1 + vectorize into one call.


Updated: 2026-06-20 12:00:00