← Back to index Logout

Processing Pipeline

POST /api/v1/file/:file_uuid/process

Auth: Required Scope: file-level

Trigger the processing pipeline for a registered file. Creates a monitor job that the worker picks up and processes sequentially. Returns immediately with the job info—processing runs asynchronously in the background.

Request Parameters

Field Type Required Default Description
processors string[] No all Specific processors to run: ["cut","asr","asrx","yolo","ocr","face","pose","visual_chunk","story","5w1h"]
rules string[] No all Rule names to apply (currently unused)

Example

# Run all processors
curl -s -X POST "$API/api/v1/file/$FILE_UUID/process" \
  -H "Content-Type: application/json" \
  -H "X-API-Key: $KEY" -d '{}'

# Run specific processors only
curl -s -X POST "$API/api/v1/file/$FILE_UUID/process" \
  -H "Content-Type: application/json" \
  -H "X-API-Key: $KEY" \
  -d '{"processors": ["asr", "face", "yolo"]}'

Response (200)

{
  "success": true,
  "job_id": 42,
  "file_uuid": "3a6c1865...",
  "status": "processing",
  "pids": [12345, 12346],
  "message": "Processing triggered for video.mp4"
}
Field Type Description
success boolean Always true on 200
job_id integer Monitor job ID (for job tracking)
file_uuid string 32-char hex UUID of the file
status string "queued" — file enters the FIFO queue
pids integer[] Process IDs of started processors (empty for queued)
message string Human-readable status

Error Responses

HTTP When
404 File UUID not found
401 Missing or invalid API key

GET /api/v1/file/:file_uuid/probe

Auth: Required Scope: file-level

Get ffprobe metadata for a registered file. Returns video/audio stream info, codec details, duration, resolution, and frame rate.

Example

curl -s "$API/api/v1/file/$FILE_UUID/probe" -H "X-API-Key: $KEY"

Response (200)

{
  "file_uuid": "3a6c1865...",
  "file_name": "video.mp4",
  "file_size": 794863677,
  "duration": 120.5,
  "width": 1920,
  "height": 1080,
  "fps": 24.0,
  "total_frames": 2892,
  "cached": true,
  "format": {
    "filename": "/path/to/video.mp4",
    "format_name": "mov,mp4,m4a,3gp",
    "duration": "120.5",
    "size": "12345678",
    "bit_rate": "819200"
  },
  "streams": [
    {
      "index": 0,
      "codec_name": "h264",
      "codec_type": "video",
      "width": 1920,
      "height": 1080,
      "r_frame_rate": "24/1",
      "duration": "120.5"
    }
  ]
}
Field Type Description
file_uuid string 32-char hex UUID
file_name string File name
file_size integer File size in bytes (from filesystem)
duration float Duration in seconds
width integer Video width in pixels
height integer Video height in pixels
fps float Frames per second
total_frames integer Estimated total frames
cached boolean True if result was from cached probe JSON
format object Container format info (ffprobe format section)
streams array Array of stream info objects

POST /api/v1/progress/:file_uuid

Auth: Required Scope: file-level

Get real-time processing progress for a file via Redis pub/sub. Includes per-processor status, current/total frames, ETA, and system resource stats.

Note: This endpoint uses POST method, not GET. The progress data is stored in Redis as a hash, and POST is used to retrieve the latest state.

Pipeline Order

Order Processor Dependencies Description
1 cut Scene detection
2 asr cut Speech-to-text (per scene)
3 asrx asr Speaker diarization
4 yolo Object detection
5 ocr Text recognition
6 face Face detection & embedding
7 pose Pose estimation
8 visual_chunk yolo Visual scene chunks
9 story asr, asrx, cut, yolo, face Scene summaries (template)
10 5w1h story 5W1H analysis (Gemma4 LLM)

All processors except story and 5w1h run concurrently when their dependencies are met. Story and 5W1H run sequentially after their prerequisites.

Example

curl -s -X POST "$API/api/v1/progress/$FILE_UUID" -H "X-API-Key: $KEY" | jq '{overall_progress, processors: [.processors[] | {name, status}]}'

Response (200)

{
  "file_uuid": "3a6c1865...",
  "overall_progress": 71,
  "cpu_percent": 45.2,
  "gpu_percent": 30.1,
  "memory_percent": 62.4,
  "processors": [
    {"processor_type": "asr", "status": "complete", "progress": 100},
    {"processor_type": "yolo", "status": "running", "progress": 65},
    {"processor_type": "face", "status": "pending", "progress": 0}
  ]
}
Field Type Description
file_uuid string 32-char hex UUID
overall_progress integer Overall progress percentage (0–100)
processors array Per-processor status list
processors[].processor_type string Processor name (asr, cut, yolo, etc.)
processors[].status string "pending", "running", "complete", or "failed"
processors[].progress integer Per-processor progress (0–100)
processors[].eta_seconds integer Estimated seconds remaining (running processors)
processors[].current integer Current frame count
processors[].total integer Total frame count
cpu_percent float Current CPU usage
gpu_percent float Current GPU utilization
memory_percent float Current memory usage

GET /api/v1/jobs

Auth: Required Scope: system-level

List all processing jobs (monitor jobs) in the system. Shows job status, which file each job is processing, and current processor info.

Example

curl -s "$API/api/v1/jobs" -H "X-API-Key: $KEY" | jq '{count, jobs: [.jobs[] | {uuid, status}]}'

Response (200)

{
  "jobs": [
    {
      "id": 42,
      "uuid": "3a6c1865...",
      "status": "running",
      "current_processor": "yolo",
      "created_at": "2026-05-16T12:00:00Z",
      "started_at": "2026-05-16T12:01:00Z"
    }
  ],
  "count": 15,
  "page": 1,
  "page_size": 20
}
Field Type Description
jobs array Array of job info objects
jobs[].id integer Job ID
jobs[].uuid string File UUID being processed
jobs[].status string "pending", "running", "completed", "failed"
jobs[].current_processor string Currently active processor, or null
count integer Total job count
page integer Current page number
page_size integer Jobs per page

GET /api/v1/job/:uuid

Auth: Required Scope: file-level

Get detailed information about a specific processing job, including its queue position.

Response (200)

{
  "id": 51,
  "uuid": "c36f35685177c981aa139b66bbbccc5b",
  "status": "queued",
  "current_processor": null,
  "progress_current": 0,
  "progress_total": 0,
  "processors": [],
  "created_at": "2026-06-22 23:08:48.497018",
  "started_at": null,
  "updated_at": null,
  "queue_position": 3
}
Field Type Description
id integer Monitor job ID
uuid string File UUID
status string "pending", "queued", "running", "completed", "failed"
current_processor string Currently active processor, or null
progress_current integer Current progress count
progress_total integer Total progress count
processors array Processor list
created_at string Job creation timestamp
started_at string Processing start timestamp, or null
updated_at string Last update timestamp, or null
queue_position integer Position in FIFO queue (null if not pending/queued)

Status Lifecycle

register ──→ pending
                  
            trigger (POST /process)
                  
              queued ←── queue_position counts jobs ahead
                  
            worker picks up
                  
            processing
                  
         ┌────────┴────────┐
                           
    completed             failed
         
    checkin ──→ indexed
    checkout ──→ checked_out
Status Meaning
pending File registered, not yet triggered
queued Triggered, waiting for worker in FIFO queue
processing Worker actively processing
completed All processors finished successfully
failed One or more essential processors failed
indexed Post-processing checkin complete
checked_out User checked out the file

Queue order is FIFO (created_at ASC). The GET /api/v1/job/:uuid endpoint returns queue_position showing how many jobs are ahead.

Frontend Status Mapping

When displaying file status in the frontend list (e.g. after GET /api/v1/files/scan), map the status field as follows:

DB Status Status Label Filter: 待處理 Filter: 處理中 Count: pendingCount Count: processingCount
unregistered 未註冊 No No No No
registered 待處理 Yes No Yes No
pending 待處理 Yes No Yes No
queued 排隊中 Yes Yes Yes Yes
processing 處理中 No Yes No Yes
completed 已完成 No No No No
failed 處理失敗 No No No No
indexed 已入庫 No No No No

queued 的特殊處理: - statusLabel → 顯示「排隊中」,加 ms-badge-warn 樣式(黃色) - filterPending → 應包含 queued,讓它在「待處理」filter 可見 - pendingCount + processingCount → 兩者都應包含 queued,因它既是「待處理」也是「正在排隊」 - 在 refreshAllStatus / loadFiles 中,如果檔案狀態是 queued,應顯示簡單的排隊訊息(無需 polling progress) - 當 worker pickup 後,狀態會變為 processing,此時 refreshAllStatus 會自動偵測到並開始 polling progress - 也可以提供一個「queue_position」顯示:呼叫 GET /api/v1/job/:uuid 取得排在第幾位


GET /api/v1/file/:file_uuid/processor-counts

Auth: Required Scope: file-level

Get counts of processor JSON output files. See 15_tkg.md for full documentation.


Pipeline Steps (Manual)

These endpoints execute individual pipeline steps. They are typically called by the worker automatically, but can be invoked manually for debugging or re-processing.

POST /api/v1/file/:file_uuid/store-asrx

Auth: Required Scope: file-level

Store ASRX diarization results as chunk records in the database. Converts ASRX segments into searchable chunk entries.

Example

curl -s -X POST "$API/api/v1/file/$FILE_UUID/store-asrx" \
  -H "X-API-Key: $KEY"

Response (200)

{
  "success": true,
  "message": "ASRX chunks stored",
  "file_uuid": "3a6c1865..."
}

POST /api/v1/file/:file_uuid/rule1

Auth: Required Scope: file-level

Execute Rule 1 pipeline step. Applies rule-based chunking to create structured chunk records from processor outputs.

Example

curl -s -X POST "$API/api/v1/file/$FILE_UUID/rule1" \
  -H "X-API-Key: $KEY"

Response (200)

{
  "success": true,
  "message": "Rule 1 complete: 45 chunks",
  "file_uuid": "3a6c1865...",
  "chunks": 45
}
Field Type Description
success boolean Always true on 200
message string Human-readable completion message
file_uuid string 32-char hex UUID
chunks integer Number of chunks produced

POST /api/v1/file/:file_uuid/vectorize

Auth: Required Scope: file-level

Generate vector embeddings for all chunks of a file and store them in Qdrant for semantic search.

Example

curl -s -X POST "$API/api/v1/file/$FILE_UUID/vectorize" \
  -H "X-API-Key: $KEY"

Response (200)

{
  "success": true,
  "message": "Vectorization complete",
  "file_uuid": "3a6c1865..."
}

POST /api/v1/file/:file_uuid/phase1

Auth: Required Scope: file-level

Execute Phase 1 of the post-processing pipeline. Combines store-asrx, rule1, and vectorize into a single step.

Example

curl -s -X POST "$API/api/v1/file/$FILE_UUID/phase1" \
  -H "X-API-Key: $KEY"

Response (200)

{
  "success": true,
  "message": "Phase 1 complete",
  "file_uuid": "3a6c1865..."
}

POST /api/v1/file/:file_uuid/complete

Auth: Required Scope: file-level

Mark a video as fully processed. Updates the video status to completed and finalizes all pipeline state.

Example

curl -s -X POST "$API/api/v1/file/$FILE_UUID/complete" \
  -H "X-API-Key: $KEY"

Response (200)

{
  "success": true,
  "message": "Video marked as completed",
  "file_uuid": "3a6c1865..."
}

Pipeline Step Order

  process (trigger)
    │
    ├─→ cut, yolo, ocr, face, pose, asrx (parallel processors)
    │
    ├─→ store-asrx  (store diarization as chunks)
    │
    ├─→ rule1       (rule-based chunking)
    │
    ├─→ vectorize   (embed chunks to Qdrant)
    │
    └─→ complete    (mark done)

Phase 1 (/phase1) combines store-asrx + rule1 + vectorize into one call.


Updated: 2026-06-23 — Added queued status, FIFO queue order, queue_position in job detail, frontend status mapping table