admin/momentry_core
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: WASM-based doc viewer (pulldown-cmark)

Browse Source
This commit is contained in:
Accusys
2026-05-18 10:07:38 +08:00
parent 78ba6f3d3d
commit 6452ac5af2
21 changed files with 2828 additions and 7 deletions
Download Patch File Download Diff File Expand all files Collapse all files

280
docs_v1.0/doc_wasm/modules/01_auth.md Normal file
View File

@@ -0,0 +1,280 @@
<!-- module: auth -->
<!-- description: Authentication — login, logout, JWT, session cookie, API key -->
<!-- depends: -->
## Base URL
| Environment | URL | Purpose |
|-------------|-----|---------|
| Production | `http://localhost:3002` | Production deployment |
| External (M5) | `https://m5api.momentry.ddns.net` | Remote access |
## Variables
All examples in this documentation use these environment variables:
```bash
API="http://localhost:3002"
KEY="your-api-key-here"
```
## Authentication
All endpoints under `/api/v1/*` require authentication.
The following endpoints are public (no auth needed):
- `GET /health`
- `POST /api/v1/auth/login`
- `POST /api/v1/auth/logout`
### Three Authentication Modes
The system supports three authentication methods, checked in **priority order** by the middleware:
```
Middleware priority:
1. Session Cookie (Portal/browser)
2. JWT Bearer (API clients, CLI)
3. API Key Header (legacy compatibility)
4. API Key Query Param (?api_key=)
```
| Mode | Transport | Expiry | Scope | Best for |
|------|-----------|--------|-------|----------|
| **Session Cookie** | `Cookie: session_id=<session_id>` | 24h | per-browser session | Portal (browser) |
| **JWT** | `Authorization: Bearer <token>` | 1h | per-login token | API clients, CLI, scripts |
| **API Key** | `X-API-Key: <key>` | 90d | fixed key for automation | Legacy scripts, WordPress |
---
### Login
**Default accounts & API keys:**
| Username | Password | API Key | Role |
|----------|----------|---------|------|
| `admin` | `admin` | — | admin |
| `demo` | `demo` | `muser_demo_key_32chars_abcdef1234567890` | user |
The demo API key is set via `MOMENTRY_DEMO_API_KEY` env var and can be used in place of JWT for marcom integrations:
```bash
# Using API key instead of JWT
curl -s "$API/api/v1/files/scan" -H "X-API-Key: muser_demo_key_32chars_abcdef1234567890"
```
```bash
# Login as admin
curl -s -X POST "$API/api/v1/auth/login" \
-H "Content-Type: application/json" \
-d '{"username": "admin", "password": "admin"}'
# Login as demo user
curl -s -X POST "$API/api/v1/auth/login" \
-H "Content-Type: application/json" \
-d '{"username": "demo", "password": "demo"}'
```
#### Success Response
```json
{
"success": true,
"jwt": "eyJhbGciOiJIUzI1NiIs...",
"api_key": "muser_...",
"user": {
"username": "admin",
"role": "admin"
},
"expires_at": "2026-05-18T13:00:00Z"
}
```
| Field | Type | Description |
|-------|------|-------------|
| `jwt` | string | JWT access token. Use as `Authorization: Bearer <jwt>`. Expires in 1 hour. |
| `api_key` | string | Legacy API key. Use as `X-API-Key: <key>`. Good for 90 days. |
| `user.username` | string | Username |
| `user.role` | string | Role: `admin`, `user`, or `readonly` |
| `expires_at` | string | ISO8601 timestamp of JWT expiration |
The login endpoint also sets a `Set-Cookie` header for browser-based clients:
```
Set-Cookie: session_id=<session_id>; Path=/; HttpOnly; SameSite=Strict; Max-Age=86400
```
#### Error Response (401)
```json
{
"success": false,
"message": "Invalid username or password"
}
```
---
### Using JWT
JWT is preferred for API clients (CLI scripts, WordPress). It is validated by the middleware without a database lookup (stateless).
```bash
# Login and capture JWT
JWT=$(curl -s -X POST "$API/api/v1/auth/login" \
-H "Content-Type: application/json" \
-d '{"username":"admin","password":"admin"}' | python3 -c "import json,sys;print(json.load(sys.stdin)['jwt'])")
# Use JWT for all subsequent requests
curl -H "Authorization: Bearer $JWT" "$API/api/v1/files/scan"
curl -H "Authorization: Bearer $JWT" "$API/api/v1/resource/tmdb"
```
JWT is short-lived (1 hour). When it expires, request a new one via login.
---
### Using Session Cookie (Browser)
Browser-based clients (Portal) get a session cookie automatically after login. The browser sends the cookie with every request—no manual header needed.
```bash
# Login captures the session cookie from Set-Cookie header
curl -v -X POST "$API/api/v1/auth/login" \
-H "Content-Type: application/json" \
-d '{"username":"admin","password":"admin"}' 2>&1 | grep "Set-Cookie"
# Browser automatically sends: Cookie: session_id=<session_id>
# No manual header needed for subsequent requests
```
The session cookie is HttpOnly (not accessible from JavaScript) and SameSite=Strict (protected against CSRF).
---
### Using Legacy API Key
```bash
curl -H "X-API-Key: $KEY" "$API/api/v1/files/scan"
# Also accepted via Bearer header (non-JWT format) or query parameter:
curl -H "Authorization: Bearer $KEY" "$API/api/v1/files/scan"
curl "$API/api/v1/files/scan?api_key=$KEY"
```
API keys are validated via SHA256 hash lookup in the database. They are long-lived (90 days) and intended for automation.
### Obtaining an API Key (CLI)
```bash
momentry api-key create "My API Key" --key-type user
```
---
### Logout
```bash
# Logout using the session cookie (browser)
curl -X POST "$API/api/v1/auth/logout" \
-H "Cookie: session_id=<uuid>"
```
#### What logout does
| Auth mode | Effect |
|-----------|--------|
| **Session Cookie** | Session deleted from database. Same cookie returns 401 on subsequent requests. |
| **JWT** | JWT remains valid until expiry. (JWT is stateless — logout adds JWT to a blacklist only if API key mode is used.) |
| **API Key** | API key remains valid. (Legacy keys are shared across sessions — revoking would break other clients.) |
#### Example: full session lifecycle
```bash
# 1. Login
SESSION_ID=$(curl -s -D - -X POST "$API/api/v1/auth/login" \
-H "Content-Type: application/json" \
-d '{"username":"admin","password":"admin"}' | grep "Set-Cookie" | sed 's/.*session_id=\([^;]*\).*/\1/')
# 2. Use session (works)
curl -s -o /dev/null -w "HTTP %{http_code}\n" "$API/api/v1/resource/tmdb" \
-H "Cookie: session_id=$SESSION_ID"
# → HTTP 200
# 3. Logout
curl -s -X POST "$API/api/v1/auth/logout" \
-H "Cookie: session_id=$SESSION_ID"
# → {"success": true}
# 4. Use session again (rejected)
curl -s -o /dev/null -w "HTTP %{http_code}\n" "$API/api/v1/resource/tmdb" \
-H "Cookie: session_id=$SESSION_ID"
# → HTTP 401
```
---
### Authentication Flow Summary
```
Login Request
│
▼
┌──────────────────┐
│ 1. Check users │ ← users table (argon2 password verify)
│ table │
└──────┬───────────┘
│
┌───┴───┐
│ match │
└───┬───┘
│
▼
┌──────────────────┐
│ 2. Create JWT │ ← 1h expiry, signed with JWT_SECRET
├──────────────────┤
│ 3. Create │ ← 24h expiry, stored in sessions table
│ session │
├──────────────────┤
│ 4. Set-Cookie │ ← HttpOnly, SameSite=Strict, Path=/
├──────────────────┤
│ 5. Return │ ← JWT + api_key + user info to client
└──────────────────┘
```
```
Protected Request
│
▼
┌──────────────────────┐
│ Middleware checks: │
│ │
│ 1. Cookie session? │ → DB lookup session → get api_key → verify
│ │
│ 2. JWT Bearer? │ → verify JWT signature → decode claims
│ │
│ 3. X-API-Key? │ → SHA256 hash → DB lookup → verify
│ │
│ 4. ?api_key=? │ → same as #3
│ │
│ 5. None → 401 │
└──────────────────────┘
```
---
### Error Responses
| HTTP | When |
|------|------|
| `401` | Missing or invalid authentication |
| `401` | Session expired or logged out |
| `401` | JWT expired |
| `401` | API key revoked or inactive |
---
### Related
- `POST /api/v1/resource/tmdb/check` — test authentication + TMDb API connectivity
- `GET /health/detailed` — view auth status (integrations section)

147
docs_v1.0/doc_wasm/modules/02_health.md Normal file
View File

@@ -0,0 +1,147 @@
<!-- module: health -->
<!-- description: Health check endpoints -->
<!-- depends: 01_auth -->
## Health Check
### `GET /health`
**Auth**: Public
**Scope**: system-level
Returns basic server health status — used by load balancers and monitoring.
#### Example
```bash
curl "$API/health" | jq '{status, version}'
```
#### Response (200)
```json
{
"status": "ok",
"version": "1.0.0",
"build_git_hash": "3a6c1865",
"build_timestamp": "2026-05-16T13:38:15Z",
"uptime_ms": 3015
}
```
| Field | Type | Description |
|-------|------|-------------|
| `status` | string | `ok` or `degraded` |
| `version` | string | Semver version |
| `build_git_hash` | string | Git commit hash |
| `build_timestamp` | string | Binary build time |
| `uptime_ms` | integer | Milliseconds since server start |
---
### `GET /health/detailed`
**Auth**: Required
**Scope**: system-level
Returns full system health including each service status, resource utilization, pipeline readiness, schema migration status, identity file sync status, and external integrations.
> Requires authentication (JWT, session cookie, or API key). The basic `/health` endpoint remains public for load balancer checks.
#### Example
```bash
curl "$API/health/detailed" | jq '{status, services, resources: {cpu: .resources.cpu_used_percent, memory: .resources.memory_used_percent}}'
```
#### Response (200)
```json
{
"status": "ok",
"version": "1.0.0",
"services": {
"postgres": {"status": "ok", "latency_ms": 3},
"redis": {"status": "ok", "latency_ms": 1},
"qdrant": {"status": "ok", "latency_ms": 5}
},
"resources": {
"cpu_used_percent": 12.5,
"memory_available_mb": 32768,
"memory_used_percent": 31.7
},
"pipeline": {
"scripts_ready": true,
"scripts_count": 345,
"processors": {
"asr": true,
"yolo": true,
"face": true,
"pose": true,
"ocr": true,
"cut": true,
"scene": true,
"asrx": true,
"visual_chunk": true
},
"models_ready": true,
"models_count": 42,
"scripts_integrity": {"matched": 332, "total": 345, "ok": false},
"ffmpeg": true
},
"schema": {
"table_exists": true,
"applied": [{"filename": "migrate_add_users_table.sql"}],
"required": [],
"ok": true
},
"identities": {
"directory_exists": true,
"files_count": 3481,
"index_ok": true,
"db_count": 3481,
"synced": true
},
"integrations": {
"tmdb": {
"api_key_configured": false,
"enabled": false,
"api_reachable": null
}
}
}
```
#### Response Fields
| Field | Type | Description |
|-------|------|-------------|
| `status` | string | `ok` if all essential services healthy |
| `services` | object | Per-service status (postgres, redis, qdrant) |
| `services.*.status` | string | `ok`, `error`, or `degraded` |
| `services.*.latency_ms` | int | Response time in milliseconds |
| `resources` | object | CPU, memory usage |
| `pipeline.scripts_ready` | boolean | Scripts directory accessible |
| `pipeline.scripts_count` | int | Number of Python processor scripts |
| `pipeline.processors` | object | Per-processor availability |
| `pipeline.models_ready` | boolean | Models directory accessible |
| `pipeline.scripts_integrity` | object | SHA256 checksum verification results |
| `schema.ok` | boolean | All required migrations applied |
| `identities.synced` | boolean | Identity file count matches DB count |
| `integrations.tmdb` | object | TMDB API key config and reachability |
#### Health status rules
| Condition | status |
|-----------|--------|
| All services ok | `ok` |
| Any service error | `degraded` |
| Postgres or Redis error | `degraded` (server still responds) |
---
### Stats Endpoints
| Method | Endpoint | Auth | Description |
|--------|----------|------|-------------|
| GET | `/api/v1/stats/sftpgo` | No | SFTPGo service status |

184
docs_v1.0/doc_wasm/modules/03_register.md Normal file
View File

@@ -0,0 +1,184 @@
<!-- module: register -->
<!-- description: File registration — register, scan -->
<!-- depends: 01_auth -->
## File Registration
### `POST /api/v1/files/register`
**Auth**: Required
**Scope**: file-level
Register a video file for processing. Returns the file's metadata and UUID.
**New in v0.1.2**: Registration now **automatically triggers the processing pipeline** — no need to call `POST /api/v1/file/:file_uuid/process` separately. The system will:
1. Register the file and run ffprobe
2. Auto-run offline TMDb probe (reads local identity files, no API calls)
3. Create a monitor job for the worker
4. Worker starts all 10 processors (Cut → ASR → ASRX → YOLO → OCR → Face → Pose → VisualChunk → Story → 5W1H)
If the file already exists (same content hash), returns the existing record with `already_exists: true`.
#### Request Parameters
| Field | Type | Required | Default | Description |
|-------|------|----------|---------|-------------|
| `file_path` | string | Yes | — | Path to video file on disk |
| `pattern` | string | No | — | Regex pattern for batch register (requires `file_path` to be a directory) |
| `user_id` | integer | No | — | User ID to associate with registration |
| `content_hash` | string | No | — | Pre-computed SHA-256 hash (skips computation) |
#### Example
```bash
# Register a single file
curl -s -X POST "$API/api/v1/files/register" \
-H "Content-Type: application/json" \
-H "X-API-Key: $KEY" \
-d '{"file_path": "/path/to/video.mp4"}'
# Batch register files matching a pattern in a directory
curl -s -X POST "$API/api/v1/files/register" \
-H "Content-Type: application/json" \
-H "X-API-Key: $KEY" \
-d '{"file_path": "/path/to/dir", "pattern": ".*\\.mp4$"}'
```
#### Response (200)
```json
{
"success": true,
"file_uuid": "3a6c1865...",
"file_name": "video.mp4",
"file_path": "/path/to/video.mp4",
"file_type": "video",
"duration": 120.5,
"width": 1920,
"height": 1080,
"fps": 24.0,
"total_frames": 2892,
"already_exists": false,
"message": "File registered successfully"
}
```
| Field | Type | Description |
|-------|------|-------------|
| `success` | boolean | Always true on 200 |
| `file_uuid` | string | 32-char hex UUID of the registered file |
| `file_name` | string | File name (auto-renamed if name conflict) |
| `file_path` | string | Canonical path on disk |
| `file_type` | string | `"video"`, `"audio"`, or `"unknown"` |
| `duration` | float | Duration in seconds |
| `width` | integer | Video width in pixels |
| `height` | integer | Video height in pixels |
| `fps` | float | Frames per second |
| `total_frames` | integer | Total frame count |
| `already_exists` | boolean | True if same content was already registered |
| `message` | string | Human-readable status |
#### Error Responses
| HTTP | When |
|------|------|
| `401` | Missing or invalid API key |
| `400` | Invalid request body |
| `404` | File path does not exist |
---
### `GET /api/v1/files/scan`
**Auth**: Required
**Scope**: file-level
Scan the filesystem directory and list all media files, showing which are registered, processing, or unregistered.
#### Query Parameters
| Field | Type | Required | Default | Description |
|-------|------|----------|---------|-------------|
| `page` | integer | No | 1 | Page number (1-based) |
| `page_size` | integer | No | all | Items per page (alias: `limit`) |
| `limit` | integer | No | all | Max items (alias for `page_size`) |
| `pattern` | string | No | — | Regex filter on file name (e.g., `.*\\.mp4$`) |
| `sort_by` | string | No | `name` | Sort field: `name`, `size`, `modified`, `status` |
| `sort_order` | string | No | `asc` | Sort direction: `asc` or `desc` |
#### Example
```bash
# Full scan
curl -s "$API/api/v1/files/scan" -H "X-API-Key: $KEY" | jq '{total, registered_count, unregistered_count}'
# Paginated (page 1, 5 per page)
curl -s "$API/api/v1/files/scan?page=1&page_size=5" -H "X-API-Key: $KEY" | jq '{page, total_pages, files: [.files[].file_name]}'
# Regex filter: only mp4 files
curl -s "$API/api/v1/files/scan?pattern=.*\\.mp4$" -H "X-API-Key: $KEY" | jq '{filtered_total, files: [.files[].file_name]}'
# Sort by file size (largest first)
curl -s "$API/api/v1/files/scan?sort_by=size&sort_order=desc&page_size=5" -H "X-API-Key: $KEY" | jq '[.files[] | {file_name, file_size}]'
# Sort by modified time (most recent first)
curl -s "$API/api/v1/files/scan?sort_by=modified&sort_order=desc&page_size=5" -H "X-API-Key: $KEY" | jq '[.files[] | {file_name, modified_time}]'
# Sort by status
curl -s "$API/api/v1/files/scan?sort_by=status&page_size=5" -H "X-API-Key: $KEY" | jq '[.files[] | {file_name, status}]'
```
#### Response (200)
```json
{
"files": [
{
"file_name": "video.mp4",
"file_size": 12345678,
"is_registered": true,
"file_uuid": "3a6c1865...",
"status": "completed",
"registration_time": "2026-05-16T12:00:00Z",
"job_id": 42
}
],
"total": 107,
"filtered_total": 80,
"page": 1,
"page_size": 20,
"total_pages": 4,
"registered_count": 26,
"unregistered_count": 81
}
```
| Field | Type | Description |
|-------|------|-------------|
| `files` | array | Array of file info objects (paginated) |
| `files[].file_name` | string | File name |
| `files[].relative_path` | string | Path relative to scan root |
| `files[].file_path` | string | Absolute path on disk |
| `files[].file_size` | integer | File size in bytes |
| `files[].modified_time` | string | Last modified timestamp (ISO8601) |
| `files[].is_registered` | boolean | Whether file is registered in DB |
| `files[].file_uuid` | string | 32-char hex UUID (only if registered) |
| `files[].status` | string | `"completed"`, `"processing"`, `"registered"`, `"unregistered"`, or `null` |
| `files[].registration_time` | string | DB registration timestamp (only if registered) |
| `files[].job_id` | integer | Processing job ID (only if a job exists) |
| `total` | integer | Total files found on disk (unfiltered) |
| `filtered_total` | integer | Files matching regex filter |
| `page` | integer | Current page number |
| `page_size` | integer | Items per page |
| `total_pages` | integer | Total pages |
| `registered_count` | integer | Files registered in DB |
| `unregistered_count` | integer | Files not yet registered |
#### Notes
| Feature | Behavior |
|---------|----------|
| **Regex** | Case-insensitive (`(?i)` prefix auto-applied). Applied to `file_name`. |
| **Sort order** | Default (`sort_by=name`): registered files first, then alphabetically. `sort_by=status`: alphabetical by status string. |
| **Pagination** | `page_size` and `limit` are aliases. Default: show all results. |
| **Processing order** | `pattern` regex filter → `sort_by`/`sort_order` → `page`/`page_size` slice. |

138
docs_v1.0/doc_wasm/modules/04_lookup.md Normal file
View File

@@ -0,0 +1,138 @@
<!-- module: lookup -->
<!-- description: File lookup by name and unregistration -->
<!-- depends: 01_auth, 03_register -->
## File Lookup
### `GET /api/v1/files/lookup`
**Auth**: Required
**Scope**: file-level
Search registered files by file name. Performs a case-insensitive LIKE search on the file name column. Returns basic info about matching files.
#### Query Parameters
| Field | Type | Required | Description |
|-------|------|----------|-------------|
| `file_name` | string | Yes | File name to search for (partial matches supported) |
#### Example
```bash
# Look up a specific file
curl -s "$API/api/v1/files/lookup?file_name=video.mp4" \
-H "X-API-Key: $KEY"
# Partial name search
curl -s "$API/api/v1/files/lookup?file_name=charade" \
-H "X-API-Key: $KEY" | jq '.matches[].file_name'
```
#### Response (200)
```json
{
"file_name": "video.mp4",
"exists": true,
"matches": [
{
"file_uuid": "a03485a40b2df2d3",
"file_name": "video.mp4",
"file_type": "video",
"status": "completed"
}
],
"next_name": "video (2).mp4"
}
```
| Field | Type | Description |
|-------|------|-------------|
| `file_name` | string | Searched name |
| `exists` | boolean | Exact name match exists |
| `matches` | array | Array of matching registered files |
| `matches[].file_uuid` | string | 32-char hex UUID |
| `matches[].file_name` | string | Registered file name |
| `matches[].file_type` | string | `"video"`, `"audio"`, or `null` |
| `matches[].status` | string | Registration/processing status |
| `next_name` | string | Suggested name for avoiding conflicts |
---
## Unregister
### `POST /api/v1/unregister`
**Auth**: Required
**Scope**: file-level
Delete a registered file from the system. Supports single file by UUID, or batch by directory + regex pattern.
#### What gets deleted
| Removed (default) | Not removed |
|---------|-------------|
| Database records (videos, chunks, embeddings, processor_results, pre_chunks) | The original source video file on disk |
| Processor output JSON files (`{uuid}.*.json`) — unless `delete_output_files: false` | Temp/working directories |
| In-memory cache entries | |
| MongoDB cached lists | |
> ⚠️ Database deletion is **irreversible**. To keep output files, set `"delete_output_files": false`.
#### Request Parameters
At least one mode must be specified: either `file_uuid` alone, or `file_path` + `pattern` together.
| Field | Type | Required | Default | Description |
|-------|------|----------|---------|-------------|
| `file_uuid` | string | * | — | Single file UUID to delete |
| `file_path` | string | * | — | Directory path (for batch delete) |
| `pattern` | string | * | — | Regex pattern (requires `file_path`) |
| `delete_output_files` | boolean | No | `true` | If `true`, also delete processor output JSON files (`{uuid}.*.json`). Set to `false` to keep them. |
#### Example
```bash
# Delete a single file by UUID (default: also deletes output JSON files)
curl -s -X POST "$API/api/v1/unregister" \
-H "Content-Type: application/json" \
-H "X-API-Key: $KEY" \
-d '{"file_uuid": "'"$FILE_UUID"'"}'
# Keep output JSON files, only delete DB records
curl -s -X POST "$API/api/v1/unregister" \
-H "Content-Type: application/json" \
-H "X-API-Key: $KEY" \
-d '{"file_uuid": "'"$FILE_UUID"'", "delete_output_files": false}'
# Batch delete all mp4 files in a directory
curl -s -X POST "$API/api/v1/unregister" \
-H "Content-Type: application/json" \
-H "X-API-Key: $KEY" \
-d '{"file_path": "/path/to/dir", "pattern": ".*\\.mp4$"}'
```
#### Response (200)
```json
{
"success": true,
"file_uuid": "a03485a40b2df2d3",
"message": "Video unregistered successfully"
}
```
| Field | Type | Description |
|-------|------|-------------|
| `success` | boolean | True if deletion succeeded |
| `file_uuid` | string | UUID of the deleted file (single mode) |
| `message` | string | Human-readable status |
#### Error Responses
| HTTP | When |
|------|------|
| `400` | Neither `file_uuid` nor `file_path`+`pattern` provided |
| `404` | File UUID not found |
| `401` | Missing or invalid API key |

236
docs_v1.0/doc_wasm/modules/05_process.md Normal file
View File

@@ -0,0 +1,236 @@
<!-- module: process -->
<!-- description: Processing pipeline — trigger, probe, progress, jobs -->
<!-- depends: 01_auth, 03_register -->
## Processing Pipeline
### `POST /api/v1/file/:file_uuid/process`
**Auth**: Required
**Scope**: file-level
Trigger the processing pipeline for a registered file. Creates a monitor job that the worker picks up and processes sequentially. Returns immediately with the job info—processing runs asynchronously in the background.
#### Request Parameters
| Field | Type | Required | Default | Description |
|-------|------|----------|---------|-------------|
| `processors` | string[] | No | all | Specific processors to run: `["cut","asr","asrx","yolo","ocr","face","pose","visual_chunk","story","5w1h"]` |
| `rules` | string[] | No | all | Rule names to apply (currently unused) |
#### Example
```bash
# Run all processors
curl -s -X POST "$API/api/v1/file/$FILE_UUID/process" \
-H "Content-Type: application/json" \
-H "X-API-Key: $KEY" -d '{}'
# Run specific processors only
curl -s -X POST "$API/api/v1/file/$FILE_UUID/process" \
-H "Content-Type: application/json" \
-H "X-API-Key: $KEY" \
-d '{"processors": ["asr", "face", "yolo"]}'
```
#### Response (200)
```json
{
"success": true,
"job_id": 42,
"file_uuid": "3a6c1865...",
"status": "processing",
"pids": [12345, 12346],
"message": "Processing triggered for video.mp4"
}
```
| Field | Type | Description |
|-------|------|-------------|
| `success` | boolean | Always true on 200 |
| `job_id` | integer | Monitor job ID (for job tracking) |
| `file_uuid` | string | 32-char hex UUID of the file |
| `status` | string | `"processing"` |
| `pids` | integer[] | Process IDs of started processors |
| `message` | string | Human-readable status |
#### Error Responses
| HTTP | When |
|------|------|
| `404` | File UUID not found |
| `401` | Missing or invalid API key |
---
### `GET /api/v1/file/:file_uuid/probe`
**Auth**: Required
**Scope**: file-level
Get ffprobe metadata for a registered file. Returns video/audio stream info, codec details, duration, resolution, and frame rate.
#### Example
```bash
curl -s "$API/api/v1/file/$FILE_UUID/probe" -H "X-API-Key: $KEY"
```
#### Response (200)
```json
{
"file_uuid": "3a6c1865...",
"file_name": "video.mp4",
"file_size": 794863677,
"duration": 120.5,
"width": 1920,
"height": 1080,
"fps": 24.0,
"total_frames": 2892,
"cached": true,
"format": {
"filename": "/path/to/video.mp4",
"format_name": "mov,mp4,m4a,3gp",
"duration": "120.5",
"size": "12345678",
"bit_rate": "819200"
},
"streams": [
{
"index": 0,
"codec_name": "h264",
"codec_type": "video",
"width": 1920,
"height": 1080,
"r_frame_rate": "24/1",
"duration": "120.5"
}
]
}
```
| Field | Type | Description |
|-------|------|-------------|
| `file_uuid` | string | 32-char hex UUID |
| `file_name` | string | File name |
| `file_size` | integer | File size in bytes (from filesystem) |
| `duration` | float | Duration in seconds |
| `width` | integer | Video width in pixels |
| `height` | integer | Video height in pixels |
| `fps` | float | Frames per second |
| `total_frames` | integer | Estimated total frames |
| `cached` | boolean | True if result was from cached probe JSON |
| `format` | object | Container format info (ffprobe format section) |
| `streams` | array | Array of stream info objects |
---
### `GET /api/v1/progress/:file_uuid`
**Auth**: Required
**Scope**: file-level
Get real-time processing progress for a file via Redis pub/sub. Includes per-processor status, current/total frames, ETA, and system resource stats.
#### Pipeline Order
| Order | Processor | Dependencies | Description |
|-------|-----------|-------------|-------------|
| 1 | `cut` | — | Scene detection |
| 2 | `asr` | cut | Speech-to-text (per scene) |
| 3 | `asrx` | asr | Speaker diarization |
| 4 | `yolo` | — | Object detection |
| 5 | `ocr` | — | Text recognition |
| 6 | `face` | — | Face detection & embedding |
| 7 | `pose` | — | Pose estimation |
| 8 | `visual_chunk` | yolo | Visual scene chunks |
| 9 | `story` | asr, asrx, cut, yolo, face | Scene summaries (template) |
| 10 | `5w1h` | story | 5W1H analysis (Gemma4 LLM) |
All processors except `story` and `5w1h` run concurrently when their dependencies are met. Story and 5W1H run sequentially after their prerequisites.
#### Example
```bash
curl -s "$API/api/v1/progress/$FILE_UUID" -H "X-API-Key: $KEY" | jq '{overall_progress, processors: [.processors[] | {processor_type, status}]}'
```
#### Response (200)
```json
{
"file_uuid": "3a6c1865...",
"overall_progress": 71,
"cpu_percent": 45.2,
"gpu_percent": 30.1,
"memory_percent": 62.4,
"processors": [
{"processor_type": "asr", "status": "complete", "progress": 100},
{"processor_type": "yolo", "status": "running", "progress": 65},
{"processor_type": "face", "status": "pending", "progress": 0}
]
}
```
| Field | Type | Description |
|-------|------|-------------|
| `file_uuid` | string | 32-char hex UUID |
| `overall_progress` | integer | Overall progress percentage (0–100) |
| `processors` | array | Per-processor status list |
| `processors[].processor_type` | string | Processor name (`asr`, `cut`, `yolo`, etc.) |
| `processors[].status` | string | `"pending"`, `"running"`, `"complete"`, or `"failed"` |
| `processors[].progress` | integer | Per-processor progress (0–100) |
| `processors[].eta_seconds` | integer | Estimated seconds remaining (running processors) |
| `processors[].current` | integer | Current frame count |
| `processors[].total` | integer | Total frame count |
| `cpu_percent` | float | Current CPU usage |
| `gpu_percent` | float | Current GPU utilization |
| `memory_percent` | float | Current memory usage |
---
### `GET /api/v1/jobs`
**Auth**: Required
**Scope**: system-level
List all processing jobs (monitor jobs) in the system. Shows job status, which file each job is processing, and current processor info.
#### Example
```bash
curl -s "$API/api/v1/jobs" -H "X-API-Key: $KEY" | jq '{count, jobs: [.jobs[] | {uuid, status}]}'
```
#### Response (200)
```json
{
"jobs": [
{
"id": 42,
"uuid": "3a6c1865...",
"status": "running",
"current_processor": "yolo",
"created_at": "2026-05-16T12:00:00Z",
"started_at": "2026-05-16T12:01:00Z"
}
],
"count": 15,
"page": 1,
"page_size": 20
}
```
| Field | Type | Description |
|-------|------|-------------|
| `jobs` | array | Array of job info objects |
| `jobs[].id` | integer | Job ID |
| `jobs[].uuid` | string | File UUID being processed |
| `jobs[].status` | string | `"pending"`, `"running"`, `"completed"`, `"failed"` |
| `jobs[].current_processor` | string | Currently active processor, or null |
| `count` | integer | Total job count |
| `page` | integer | Current page number |
| `page_size` | integer | Jobs per page |

145
docs_v1.0/doc_wasm/modules/06_search.md Normal file
View File

@@ -0,0 +1,145 @@
<!-- module: search -->
<!-- description: Vector search, BM25, smart search, universal search, visual search -->
<!-- depends: 01_auth -->
## Search APIs
### `POST /api/v1/search/smart`
**Auth**: Required
**Scope**: file-level
Semantic vector search using EmbeddingGemma-300m. Generates a query embedding via EmbeddingGemma (port 11436), then searches pgvector `story_parent` and `llm_parent` chunks by cosine similarity.
#### Request Parameters
| Field | Type | Required | Default | Description |
|-------|------|----------|---------|-------------|
| `file_uuid` | string | Yes | — | File UUID to search within |
| `query` | string | Yes | — | Search text |
| `limit` | integer | No | 5 | Max results to return |
| `page` | integer | No | 1 | Page number |
| `page_size` | integer | No | 5 | Items per page |
#### Example
```bash
curl -s -X POST "$API/api/v1/search/smart" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $JWT" \
-d '{"file_uuid": "'"$FILE_UUID"'", "query": "Audrey Hepburn"}'
```
#### Response (200)
```json
{
"query": "Audrey Hepburn",
"results": [
{
"parent_id": 1087822,
"scene_order": 1087822,
"start_frame": 104438,
"end_frame": 104538,
"fps": 24.0,
"start_time": 4351.6,
"end_time": 4355.76,
"summary": "[4352s-4356s, 4s] Cast: Audrey Hepburn. Total: 2 lines, 10 words. Speakers: Audrey Hepburn (2 lines)",
"similarity": 0.67
}
],
"page": 1,
"page_size": 5,
"strategy": "semantic_vector_search"
}
```
---
### `POST /api/v1/search/universal`
**Auth**: Required
**Scope**: file-level
Multi-type BM25 full-text search across chunks, frames, and persons. Uses PostgreSQL `tsvector`.
#### Request Parameters
| Field | Type | Required | Default | Description |
|-------|------|----------|---------|-------------|
| `query` | string | Yes | — | Search text |
| `file_uuid` | string | No | — | Restrict to specific file |
| `types` | string[] | No | `["chunk","frame","person"]` | Search types |
| `limit` | integer | No | 10 | Max results per type |
| `page` | integer | No | 1 | Page number |
| `page_size` | integer | No | 20 | Items per page |
#### Example
```bash
curl -s -X POST "$API/api/v1/search/universal" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $JWT" \
-d '{"file_uuid": "'"$FILE_UUID"'", "query": "Cary Grant"}'
```
#### Response (200)
```json
{
"results": [
{
"type": "chunk",
"chunk_id": "bd80fec92b0b6963d177a2c55bf713e2_2",
"chunk_type": "story_child",
"start_frame": 5103,
"end_frame": 5127,
"start_time": 212.64,
"end_time": 213.64,
"text": "[213s-214s] Cary Grant: \"Olá!\"",
"score": 0.9
}
],
"total": 20,
"took_ms": 18
}
```
---
### `POST /api/v1/search/frames`
**Auth**: Required
**Scope**: file-level
Search face detection frames by identity name or trace ID.
---
### `POST /api/v1/search/identity_text`
**Auth**: Required
**Scope**: file-level
Search text chunks spoken by a specific identity.
---
### Visual Search
| Method | Endpoint | Description |
|--------|----------|-------------|
| POST | `/api/v1/search/visual` | Search visual chunks |
| POST | `/api/v1/search/visual/class` | Search by object class |
| POST | `/api/v1/search/visual/density` | Search by object density |
| POST | `/api/v1/search/visual/combination` | Search by object combination |
| POST | `/api/v1/search/visual/stats` | Visual chunk statistics |
#### Embedding Model
| Detail | Value |
|--------|-------|
| **Model** | EmbeddingGemma-300m |
| **Endpoint** | `POST /api/v1/embeddings` on port 11436 |
| **Dimension** | 768 |
| **Storage** | pgvector (`chunk.embedding` column) |

333
docs_v1.0/doc_wasm/modules/07_identity.md Normal file
View File

@@ -0,0 +1,333 @@
<!-- module: identity -->
<!-- description: Global identities — CRUD, detail, files, faces, bind, unbind, search -->
<!-- depends: 01_auth -->
## Global Identities
### `GET /api/v1/identities`
**Auth**: Required
**Scope**: identity-level
List all registered identities with pagination.
#### Example
```bash
curl -s "$API/api/v1/identities?page=1&page_size=20" -H "X-API-Key: $KEY" | jq '{count, identities: [.identities[] | {name}]}'
```
---
### `GET /api/v1/identity/:identity_uuid`
**Auth**: Required
**Scope**: identity-level
Get detailed information for a specific identity, including metadata and TMDb references.
#### Example
```bash
curl -s "$API/api/v1/identity/$IDENTITY_UUID" -H "X-API-Key: $KEY"
```
#### Response (200)
```json
{
"success": true,
"identity_uuid": "a9a901056d6b46ff92da0c3c1a57dff4",
"name": "Cary Grant",
"identity_type": "people",
"source": "tmdb",
"status": "confirmed",
"tmdb_id": 112,
"tmdb_profile": "{output}/identities/{identity_uuid}/profile.jpg",
"metadata": {},
"reference_data": {},
"created_at": "2026-05-16T12:00:00Z",
"updated_at": null
}
```
| Field | Type | Description |
|-------|------|-------------|
| `identity_uuid` | string | Identity identifier |
| `name` | string | Identity name |
| `identity_type` | string | `"people"` or null |
| `source` | string | `.json`, `auto`, `tmdb`, `user_defined`, or `merged` |
| `status` | string | `"confirmed"`, `"pending"`, or `"inactive"` |
| `tmdb_id` | integer | TMDb person ID (only if source = tmdb) |
| `tmdb_profile` | string | Local profile image path (`{output}/identities/{uuid}/profile.jpg`) |
| `metadata` | object | Metadata JSON (tmdb_character, cast_order, etc.) |
| `created_at` | string | Creation timestamp |
---
### `DELETE /api/v1/identity/:identity_uuid`
**Auth**: Required
**Scope**: identity-level
Delete an identity permanently.
---
### `GET /api/v1/identity/:identity_uuid/files`
**Auth**: Required
**Scope**: identity-level
Get all files where this identity appears. Returns per-file summary including face count, confidence, and appearance time range.
#### Example
```bash
curl -s "$API/api/v1/identity/$IDENTITY_UUID/files" -H "X-API-Key: $KEY"
```
---
### `GET /api/v1/identity/:identity_uuid/faces`
**Auth**: Required
**Scope**: identity-level
Get all face detection records associated with this identity.
#### Example
```bash
curl -s "$API/api/v1/identity/$IDENTITY_UUID/faces" -H "X-API-Key: $KEY"
```
| Field | Type | Description |
|-------|------|-------------|
| `file_uuid` | string | File where face was detected |
| `frame_number` | integer | Frame number of detection |
| `face_id` | string | Face ID (format: `face_{frame_number}`) |
| `confidence` | float | Detection confidence |
---
### `GET /api/v1/identity/:identity_uuid/chunks`
**Auth**: Required
**Scope**: identity-level
Get all text chunks (sentences) spoken while this identity's face was on screen. Useful for finding what a person said.
#### Example
```bash
curl -s "$API/api/v1/identity/$IDENTITY_UUID/chunks" -H "X-API-Key: $KEY"
```
#### Response (200)
```json
{
"success": true,
"identity_uuid": "a9a901056d6b46ff92da0c3c1a57dff4",
"data": [
{
"id": 0,
"file_uuid": "bd80fec92b0b6963d177a2c55bf713e2",
"chunk_id": "bd80fec92b0b6963d177a2c55bf713e2_2",
"chunk_type": "sentence",
"start_frame": 5103,
"end_frame": 5127,
"fps": 24.0,
"start_time": 212.64,
"end_time": 213.64,
"text_content": "[213s-214s] Cary Grant: \"Olá!\""
}
]
}
```
| Field | Type | Description |
|-------|------|-------------|
| `file_uuid` | string | File identifier |
| `chunk_id` | string | Sentence chunk identifier |
| `start_frame` | integer | Frame-accurate start position |
| `end_frame` | integer | Frame-accurate end position |
| `fps` | float | Frames per second |
| `start_time` | float | Start time in seconds |
| `end_time` | float | End time in seconds |
| `text_content` | string | Spoken text content |
---
### `POST /api/v1/identity/:identity_uuid/bind`
**Auth**: Required
**Scope**: identity-level
Bind a face detection to an identity. Associates the face trace with the identity for future search and recognition.
#### Request Parameters
| Field | Type | Required | Description |
|-------|------|----------|-------------|
| `file_uuid` | string | Yes | File where face is detected |
| `face_id` | string | Yes | Face ID (format: `{frame}_{idx}`) |
#### Example
```bash
curl -s -X POST "$API/api/v1/identity/$IDENTITY_UUID/bind" \
-H "X-API-Key: $KEY" \
-H "Content-Type: application/json" \
-d '{"file_uuid": "'"$FILE_UUID"'", "face_id": "1_5"}'
```
---
### `POST /api/v1/identity/:identity_uuid/unbind`
**Auth**: Required
**Scope**: identity-level
Unbind a face detection from an identity. Removes the identity association from the face record.
---
### `GET /api/v1/identities/search`
**Auth**: Required
**Scope**: identity-level
Search identities by name (ILIKE search). Returns matching identity records.
#### Example
```bash
curl -s "$API/api/v1/identities/search?q=Cary" -H "X-API-Key: $KEY"
```
| Field | Type | Description |
|-------|------|-------------|
| `name` | string | Identity name |
| `source` | string | Identity source |
| `tmdb_id` | integer | TMDb ID (if source = tmdb) |
| `file_uuid` | string | Associated file |
---
---
### `POST /api/v1/identity/upload`
**Auth**: Required
**Scope**: identity-level
Upload an identity.json file to create or update an identity. Accepts the same format as the identity.json files stored on disk.
If an identity with the same `name` already exists, it will be updated with the new values.
#### Request
The request body is an `IdentityFile` object:
| Field | Type | Required | Description |
|-------|------|----------|-------------|
| `identity_uuid` | string | Yes | Identity identifier |
| `name` | string | Yes | Identity display name |
| `identity_type` | string | No | `"people"` or null |
| `source` | string | No | `.json`, `auto`, `tmdb`, `user_defined`, or `merged` |
| `status` | string | No | `"confirmed"`, `"pending"`, or `"inactive"` |
| `tmdb_id` | integer | No | TMDb person ID |
| `tmdb_profile` | string | No | TMDb profile image URL |
| `metadata` | object | No | Arbitrary metadata JSON |
| `file_bindings` | array | No | Array of `{ file_uuid, trace_ids, face_count }` (informational) |
#### Example
```bash
curl -s -X POST "$API/api/v1/identity/upload" \
-H "X-API-Key: $KEY" \
-H "Content-Type: application/json" \
-d '{
"version": 1,
"identity_uuid": "a9a901056d6b46ff92da0c3c1a57dff4",
"name": "Cary Grant",
"identity_type": "people",
"source": ".json",
"status": "confirmed",
"metadata": {},
"file_bindings": []
}'
```
#### Response (200)
```json
{
"success": true,
"identity_uuid": "a9a901056d6b46ff92da0c3c1a57dff4",
"name": "Cary Grant",
"message": "Identity uploaded successfully"
}
```
---
---
### `POST /api/v1/identity/:identity_uuid/profile-image`
**Auth**: Required
**Scope**: identity-level
Upload a profile image (JPEG or PNG) for an identity. The image is saved to `{output}/identities/{uuid}/profile.{ext}`.
Uses `multipart/form-data` with field name `image`.
#### Example
```bash
curl -s -X POST "$API/api/v1/identity/$IDENTITY_UUID/profile-image" \
-H "X-API-Key: $KEY" \
-F "image=@/path/to/photo.jpg"
```
#### Response (200)
```json
{
"success": true,
"identity_uuid": "a9a901056d6b46ff92da0c3c1a57dff4",
"path": "/path/to/output/identities/.../profile.jpg",
"message": "Profile image saved: profile.jpg"
}
```
#### Error Responses
| HTTP | When |
|------|------|
| `400` | Missing image field or unsupported format |
| `404` | Identity not found |
| `415` | Unsupported image type (use JPEG or PNG) |
---
### `GET /api/v1/identity/:identity_uuid/profile-image`
**Auth**: Required
**Scope**: identity-level
Retrieve the profile image for an identity. Returns the raw image data with appropriate Content-Type header.
```bash
curl -s "$API/api/v1/identity/$IDENTITY_UUID/profile-image" \
-H "X-API-Key: $KEY" -o profile.jpg
```
| Response Header | Value |
|----------------|-------|
| `content-type` | `image/jpeg` or `image/png` |

65
docs_v1.0/doc_wasm/modules/08_identity_agent.md Normal file
View File

@@ -0,0 +1,65 @@
<!-- module: identity_agent -->
<!-- description: Identity agent — match from photo, match from trace -->
<!-- depends: 01_auth, 07_identity -->
## Identity Agent
### `POST /api/v1/agents/identity/match-from-photo`
**Auth**: Required
**Scope**: file-level
Upload a face photo to match against known identities. Detects face via InsightFace, extracts 512D embedding via CoreML FaceNet, then searches pgvector for the closest identity.
#### Request
`multipart/form-data` with field `image` (JPEG/PNG) and optional `file_uuid`.
#### Example
```bash
curl -s -X POST "$API/api/v1/agents/identity/match-from-photo" \
-H "Authorization: Bearer $JWT" \
-F "image=@/path/to/face.jpg" \
-F "file_uuid=$FILE_UUID"
```
#### Response (200)
```json
{
"success": true,
"matches": [
{
"identity_uuid": "a9a90105...",
"name": "Cary Grant",
"similarity": 0.87
}
]
}
```
---
### `POST /api/v1/agents/identity/match-from-trace`
**Auth**: Required
**Scope**: file-level
Match a face trace (tracked face across frames) against known identities. Samples 3 angles from the trace, generates embeddings, and searches pgvector.
#### Request Parameters
| Field | Type | Required | Description |
|-------|------|----------|-------------|
| `file_uuid` | string | Yes | File containing the trace |
| `trace_id` | integer | Yes | Face trace ID to match |
#### Example
```bash
curl -s -X POST "$API/api/v1/agents/identity/match-from-trace" \
-H "Authorization: Bearer $JWT" \
-H "Content-Type: application/json" \
-d '{"file_uuid": "'"$FILE_UUID"'", "trace_id": 10}'
```

146
docs_v1.0/doc_wasm/modules/08_media.md Normal file
View File

@@ -0,0 +1,146 @@
<!-- module: media -->
<!-- description: Video streaming & frame extraction -->
<!-- depends: 01_auth -->
## Video Streaming & Frame Extraction
All video streaming endpoints support the following common query parameters:
| Field | Type | Required | Default | Description |
|-------|------|----------|---------|-------------|
| `mode` | string | No | `normal` | `normal` or `debug` (draws detection overlays) |
| `audio` | string | No | `on` | `on` or `off` |
---
### `GET /api/v1/file/:file_uuid/video`
Stream the full video file with range support for seeking.
**Auth**: Required
**Scope**: file-level
#### Response
- **200**: Video stream (`Content-Type` based on file extension)
- **206**: Partial content (range request)
- Supports `Range` header for seeking
---
### `GET /api/v1/file/:file_uuid/trace/:trace_id/video`
Stream video with highlights for a specific face trace (follows a single person across frames with bounding box overlay).
**Auth**: Required
**Scope**: file-level
---
### `GET /api/v1/file/:file_uuid/video/bbox`
Stream video with bounding box overlay for all detected objects/faces.
**Auth**: Required
**Scope**: file-level
Uses a built-in 5×7 bitmap font renderer to draw labels directly on video frames via FFmpeg `drawtext` filter.
---
### `GET /api/v1/file/:file_uuid/thumbnail`
Extract a single frame from a video as JPEG image. Uses FFmpeg `select` filter.
**Auth**: Required
**Scope**: file-level
#### Query Parameters
| Field | Type | Required | Default | Description |
|-------|------|----------|---------|-------------|
| `frame` | integer | Yes | — | Zero-based frame number to extract |
| `x` | integer | No | — | Crop start X (left edge). Requires `y`, `w`, `h`. |
| `y` | integer | No | — | Crop start Y (top edge). Requires `x`, `w`, `h`. |
| `w` | integer | No | — | Crop width in pixels. Requires `x`, `y`, `h`. |
| `h` | integer | No | — | Crop height in pixels. Requires `x`, `y`, `w`. |
All four crop params (`x`, `y`, `w`, `h`) must be provided together or omitted.
#### Example
```bash
# Extract frame 1000 (full frame)
curl -s "$API/api/v1/file/bd80fec92b0b6963d177a2c55bf713e2/thumbnail?frame=1000" \
-H "Authorization: Bearer $JWT" -o frame_1000.jpg
# Extract and crop face region (x=320, y=240, w=160, h=160)
curl -s "$API/api/v1/file/bd80fec92b0b6963d177a2c55bf713e2/thumbnail?frame=1000&x=320&y=240&w=160&h=160" \
-H "Authorization: Bearer $JWT" -o face_crop.jpg
```
#### Response
- **200**: `image/jpeg` binary data
- **404**: File not found
- **500**: FFmpeg error (e.g., frame number exceeds video duration)
### `GET /api/v1/file/:file_uuid/clip`
Extract a video clip (time range) as MPEG-TS stream. Uses FFmpeg `-ss` fast seek.
**Auth**: Required
**Scope**: file-level
#### Query Parameters
| Field | Type | Required | Default | Description |
|-------|------|----------|---------|-------------|
| `start_frame` | integer | No* | — | Start frame (zero-based). **Frame-accurate** — use this for precision. |
| `end_frame` | integer | No* | — | End frame (zero-based, inclusive). Requires `start_frame`. |
| `start_time` | float | No* | — | Start time in seconds. Approximate (FPS-dependent). Fallback if frames not given. |
| `end_time` | float | No* | — | End time in seconds. Approximate (FPS-dependent). Fallback if frames not given. |
| `fps` | float | No | video FPS | Override frames-per-second for frame↔time calculation. Defaults to video's detected FPS. |
| `mode` | string | No | `normal` | `normal` or `debug` (draws "CLIP" overlay) |
| `audio` | string | No | `on` | `on` or `off` |
Either (`start_frame`+`end_frame`) OR (`start_time`+`end_time`) must be provided.
#### Example
```bash
# Clip by frame range (primary)
curl -s "$API/api/v1/file/bd80fec92b0b6963d177a2c55bf713e2/clip?start_frame=0&end_frame=47" \
-H "Authorization: Bearer $JWT" -o clip.ts
# Clip by time range (fallback)
curl -s "$API/api/v1/file/bd80fec92b0b6963d177a2c55bf713e2/clip?start_time=30&end_time=45" \
-H "Authorization: Bearer $JWT" -o clip.ts
```
#### Response
- **200**: `video/mp2t` MPEG-TS stream
- **400**: Missing/invalid range parameters
- **404**: File not found
- **500**: FFmpeg error
#### Technical Notes
| Detail | Value |
|--------|-------|
| **Backend** | FFmpeg (`ffmpeg-full`) |
| **Seek** | `-ss` before `-i` (fast keyframe seek) |
| **Format** | MPEG-TS (`mpegts` muxer, pipe-safe) |
| **Codec** | H.264 + AAC |
| **Cache** | `Cache-Control: public, max-age=86400` (24h) |
---
| Detail | Value |
|--------|-------|
| **Backend** | FFmpeg (`ffmpeg-full`) |
| **Filter** | `select=eq(n\,FRAME)` to select frame, optional `crop=W:H:X:Y` |
| **Output** | Single JPEG via pipe (`image2pipe`, `mjpeg` codec) |
| **Cache** | `Cache-Control: public, max-age=86400` (24h) |
| **Frame number** | Zero-based (`frame=0` = first frame of video) |

109
docs_v1.0/doc_wasm/modules/09_tmdb.md Normal file
View File

@@ -0,0 +1,109 @@
<!-- module: tmdb -->
<!-- description: TMDb enrichment endpoints — prefetch, probe, resource, check -->
<!-- depends: 01_auth, 03_register -->
## TMDb Enrichment
> **Offline operation**: TMDb prefetch now checks local identity files first (`identities/_index.json` + `*.tmdb.json`).
> If local files exist, no external API call is made. Internet is only needed for initial data seeding.
### Overview
TMDb enrichment is an optional identity enrichment step that can be run after Pipeline face detection completes. The workflow is:
1. **Prefetch** (requires internet): Download movie cast data from TMDb API → cache to `{file_uuid}.tmdb.json`
2. **Probe**: Read local cache → create identities for **all** cast members (`source='tmdb'`) + save `identity.json` + download profile image to `{OUTPUT}/identities/{uuid}/profile.jpg`
3. **Match**: The worker automatically matches video faces against TMDb identities when `MOMENTRY_TMDB_PROBE_ENABLED=true`
### `POST /api/v1/agents/tmdb/prefetch`
**Auth**: Required
**Scope**: file-level
Fetch TMDb cast data for a registered file and cache it locally. This is the only step requiring internet access.
#### Request Parameters
| Field | Type | Required | Description |
|-------|------|----------|-------------|
| `file_uuid` | string | Yes | File UUID to enrich |
#### Example
```bash
curl -s -X POST "$API/api/v1/agents/tmdb/prefetch" \
-H "Content-Type: application/json" \
-H "X-API-Key: $KEY" \
-d '{"file_uuid": "'"$FILE_UUID"'"}'
```
#### Response (200)
```json
{"success": true, "file_uuid": "...", "cache_path": "/output/...tmdb.json"}
```
### `POST /api/v1/file/:file_uuid/tmdb-probe`
**Auth**: Required
**Scope**: file-level
Read local TMDb cache and create/update identities. Requires prefetch to have been run first.
#### Example
```bash
curl -s -X POST "$API/api/v1/file/$FILE_UUID/tmdb-probe" \
-H "X-API-Key: $KEY" | jq '{identities_created, movie_title}'
```
#### Response (200 — identities created)
```json
{"success": true, "identities_created": 15, "movie_title": "Charade"}
```
#### Response (200 — no cache)
```json
{"success": false, "message": "No TMDb cache found. Run tmdb-prefetch first."}
```
### `GET /api/v1/resource/tmdb`
**Auth**: Required
**Scope**: system-level
View TMDb resource status including configuration, identity counts, and cache file count.
#### Example
```bash
curl -s "$API/api/v1/resource/tmdb" -H "X-API-Key: $KEY" \
| jq '{identities_seeded, cache_files}'
```
### `POST /api/v1/resource/tmdb/check`
**Auth**: Required
**Scope**: system-level
Ping the TMDb API to verify connectivity and measure latency.
#### Example
```bash
curl -s -X POST "$API/api/v1/resource/tmdb/check" \
-H "X-API-Key: $KEY" | jq '.status'
```
#### Response
```json
{
"api_key_configured": true,
"enabled": false,
"api_reachable": true,
"api_latency_ms": 120
}
```

178
docs_v1.0/doc_wasm/modules/10_pipeline.md Normal file
View File

@@ -0,0 +1,178 @@
<!-- module: pipeline -->
<!-- description: Pipeline processors, ingestion status, stats endpoints -->
<!-- depends: 01_auth -->
## Pipeline
### Dependency Graph
```mermaid
flowchart TB
subgraph Processors["10 Processors"]
Cut[Cut] --> ASR[ASR]
ASR --> ASRX[ASRX]
ASRX --> Story[Story]
Cut --> Story
YOLO[YOLO] --> VisualChunk[VisualChunk]
VisualChunk --> Story
Face[Face] --> Story
Story --> FiveW1H[5W1H]
OCR[OCR]
Pose[Pose]
end
subgraph Ingestion["入庫 (Post-Processing)"]
ASR --> Rule1[Rule 1 Sentence]
ASRX --> Rule1
Rule1 --> Vectorize[Auto-Vectorize]
Rule1 --> Phase1[Phase 1 Pack]
Cut --> Rule3[Rule 3 Scene]
ASR --> Rule3
Face --> Trace[Face Trace]
Trace --> Qdrant[Qdrant Sync]
Trace --> TraceChunks[Trace Chunks]
Trace --> TKG[TKG Builder]
Face --> TMDbMatch[TMDb Match]
Face --> SceneMeta[Scene Metadata]
YOLO --> SceneMeta
Face --> IdentityAgent[Identity Agent]
ASRX --> IdentityAgent
Cut --> Agent5W1H[5W1H Agent]
ASR --> Agent5W1H
Agent5W1H --> Phase2[Phase 2 Pack]
end
style Processors fill:#1a1a2e,stroke:#e94560
style Ingestion fill:#16213e,stroke:#0f3460
```
### Pipeline Completion Flow
The pipeline is **not complete** until both the 10 processors AND the 入庫 (ingestion) steps have finished. The worker polls every 3 seconds and only marks the job as `completed` when all ingestion steps verify OK.
```
10 processors done
↓ (job status stays "running")
Algorithm 1 Trigger: Rule 1 + Vectorize + Phase 1 Pack
↓ (job runs in parallel)
Algorithm 2 Trigger: Face Trace → TKG, Scene Metadata, Identity Agent, 5W1H Agent
↓ (poll checks every 3s)
Ingestion verification: rule1 ✓ vectorize ✓ rule3 ✓ face_trace ✓ tkg ✓ scene_meta ✓ 5w1h ✓
↓
job status = "completed"
```
### 10 Processor Stages
| # | Processor | Depends On | Description |
|---|-----------|------------|-------------|
| 1 | `Cut` | — | Scene boundary detection (PySceneDetect) |
| 2 | `ASR` | Cut | Automatic speech recognition (faster-whisper) |
| 3 | `ASRX` | ASR | Speaker diarization + ASR refinement |
| 4 | `YOLO` | — | Object detection (YOLOv8) |
| 5 | `OCR` | — | Optical character recognition |
| 6 | `Face` | — | Face detection + recognition (InsightFace + CoreML) |
| 7 | `Pose` | — | Pose estimation |
| 8 | `VisualChunk` | YOLO | Visual object chunking |
| 9 | `Story` | ASRX + Cut + YOLO + Face | Narrative scene summarization (LLM, with embedding) |
| 10 | `5W1H` | Story | Who/What/When/Where/Why extraction (LLM, with embedding) |
### 入庫 (Post-Processing / Ingestion)
These steps run after the 10 processors and are **required for pipeline completion**. The worker checks all of them before marking the job as done.
| # | Step | Triggers When | Verification |
|---|------|--------------|-------------|
| 1 | **Rule 1 Sentence Chunking** | ASR + ASRX done | `chunk` table has rows with `chunk_type = 'sentence'` |
| 2 | **Auto-Vectorize** | Rule 1 done | `chunk.embedding` IS NOT NULL for sentence chunks |
| 3 | **Phase 1 Pack** | Rule 1 done | `release_pack.py --phase 1` executed |
| 4 | **Rule 3 Scene Chunking** | All 10 processors done + Cut + ASR | `chunk` table has rows with `chunk_type = 'cut'` |
| 5 | **Face Trace** | All 10 processors done + Face | `face_detections.trace_id` IS NOT NULL |
| 6 | **Qdrant Face Sync** | Face Trace done | Qdrant face_embedding collection populated |
| 7 | **Trace Chunks** | Face Trace done | `chunk` table has rows with `chunk_type = 'trace'` |
| 8 | **TKG Builder** | Face Trace done | `tkg_nodes` + `tkg_edges` tables have rows |
| 9 | **TMDb Face Matching** | TMDb enabled + Face done | `face_detections.identity_id` IS NOT NULL |
| 10 | **Heuristic Scene Metadata** | Face + YOLO done | `{file_uuid}.scene_meta.json` exists on disk |
| 11 | **Identity Agent** | Face + ASRX done | `identities` with `source = 'identity_agent'` |
| 12 | **5W1H Agent** | Cut + ASR done | `chunk.summary_text` IS NOT NULL for cut chunks |
| 13 | **Release Pack** | 5W1H Agent done | `release_pack.py --phase 2` executed |
### Ingestion Status
Check real-time ingestion status for a file:
```bash
curl "$API/api/v1/stats/ingestion-status/{file_uuid}"
```
Returns per-step `done` / `pending` status with detail counts.
#### Example
```bash
curl "http://localhost:3003/api/v1/stats/ingestion-status/bd80fec9c42afb0307eb28f22c64c76a" | jq '.steps[] | {name, status, detail}'
```
#### Response
```json
{
"file_uuid": "bd80fec9c42afb0307eb28f22c64c76a",
"steps": [
{ "name": "rule1_sentence", "status": "pending", "detail": "0 sentence chunks" },
{ "name": "auto_vectorize", "status": "pending", "detail": "0 embedded" },
{ "name": "rule3_scene", "status": "pending", "detail": "0 scene chunks" },
{ "name": "face_trace", "status": "pending", "detail": "0 traces" },
{ "name": "trace_chunks", "status": "pending", "detail": "0 trace chunks" },
{ "name": "tkg", "status": "pending", "detail": "0 nodes, 0 edges" },
{ "name": "identity_match", "status": "pending", "detail": "0 identities" },
{ "name": "scene_metadata", "status": "pending", "detail": null },
{ "name": "5w1h", "status": "pending", "detail": "0 scenes with 5W1H" }
]
}
```
### Stats Endpoints
| Method | Endpoint | Auth | Description |
|--------|----------|------|-------------|
| GET | `/api/v1/stats/sftpgo` | No | SFTPGo service status |
| GET | `/api/v1/stats/ingestion-status/:file_uuid` | No | Per-file ingestion checklist |
### Configuration
### `POST /api/v1/config/cache`
**Auth**: Required
**Scope**: system-level
Toggle the Redis cache on or off.
#### Request Parameters
| Field | Type | Required | Description |
|-------|------|----------|-------------|
| `enabled` | boolean | Yes | `true` to enable, `false` to disable |
#### Example
```bash
curl -s -X POST "$API/api/v1/config/cache" \
-H "Content-Type: application/json" \
-H "X-API-Key: $KEY" \
-d '{"enabled": false}'
```
### Unmounted Routes
The following routes are defined in source code but are **NOT** currently mounted in the router:
| Endpoint | Source file |
|----------|-------------|
| `/api/v1/search/persons` | `universal_search.rs` (not mounted) |
| `/api/v1/who` | `who.rs` |
| `/api/v1/who/candidates` | `who.rs` |

57
docs_v1.0/doc_wasm/modules/11_error_codes.md Normal file
View File

@@ -0,0 +1,57 @@
<!-- module: error_codes -->
<!-- description: Standard API error codes -->
<!-- depends: -->
## Error Response Format
All API errors follow this JSON structure:
```json
{
"success": false,
"error": {
"code": "E001_NOT_FOUND",
"message": "Resource not found",
"details": {"resource": "file_uuid", "value": "abc"}
}
}
```
## Error Code List
### Generic Errors (E0xx)
| Code | HTTP | Description |
|------|------|-------------|
| `E001_NOT_FOUND` | 404 | Resource not found (file, identity, chunk) |
| `E002_DUPLICATE` | 409 | Resource already exists |
| `E003_VALIDATION` | 400 | Request parameter validation failed |
| `E004_UNAUTHORIZED` | 401 | Invalid API key or token |
| `E005_INTERNAL` | 500 | Internal server error |
### Processor Errors (E1xx)
| Code | HTTP | Description |
|------|------|-------------|
| `E101_PROCESSOR_FAIL` | 500 | Python script execution failed |
| `E102_TIMEOUT` | 504 | Processing timeout |
| `E103_RESUME_FAIL` | 500 | Resume failed (checkpoint not found) |
| `E104_NO_VIDEO` | 400 | Video file path not found |
### Identity Errors (E2xx)
| Code | HTTP | Description |
|------|------|-------------|
| `E201_FACE_NOT_FOUND` | 404 | Face detection not found |
| `E202_MERGE_CONFLICT` | 409 | Identity merge conflict |
| `E203_CANDIDATE_EMPTY` | 404 | No candidates available for confirmation |
### TMDb Errors (E3xx)
| Code | HTTP | Description |
|------|------|-------------|
| `E301_TMDB_NO_KEY` | 400 | `TMDB_API_KEY` environment variable not set |
| `E302_TMDB_UNREACHABLE` | 502 | TMDb API unreachable or timed out |
| `E303_TMDB_CACHE_NOT_FOUND` | 200 | No local TMDb cache; run prefetch first |
| `E304_TMDB_PROBE_FAILED` | 500 | TMDb probe execution failed |
| `E305_TMDB_MOVIE_NOT_FOUND` | 404 | No matching TMDb movie found from filename |

118
docs_v1.0/doc_wasm/modules/12_agent.md Normal file
View File

@@ -0,0 +1,118 @@
# Agent Endpoints
Agent endpoints provide AI-powered capabilities including translation, identity analysis, and 5W1H extraction.
## POST /api/v1/agents/translate
Translate text between languages using Gemma4 (llama.cpp, port 8082).
### Request
```json
{
"text": "Hello, welcome to Momentry Core.",
"target_language": "Traditional Chinese",
"source_language": "English"
}
```
| Field | Type | Required | Description |
|-------|------|----------|-------------|
| `text` | string | ✅ | Text to translate |
| `target_language` | string | ✅ | Target language name (e.g. "Traditional Chinese", "Japanese") |
| `source_language` | string | ❌ | Source language (default: "auto") |
### Response
```json
{
"success": true,
"translated_text": "您好，歡迎使用 Momentry Core。",
"source_language_detected": "English",
"model_used": "google_gemma-4-26B-A4B-it-Q5_K_M.gguf"
}
```
### Supported Language Pairs (tested)
| Source | Target | Quality |
|--------|--------|---------|
| English | Traditional Chinese | ✅ |
| English | Japanese | ✅ |
| Chinese | English | ✅ |
| English | French | ✅ |
| Chinese | Japanese | ✅ |
### Model
- **Model**: Gemma4 26B (Q5_K_M)
- **Engine**: llama.cpp at `localhost:8082`
- **Endpoint**: `/v1/chat/completions` (OpenAI-compatible)
- **Temperature**: 0.1
- **Max tokens**: 1024
### Errors
| Status | Condition |
|--------|-----------|
| 500 | LLM unreachable or response parse failure |
| 401 | Missing/invalid auth |
---
## POST /api/v1/agents/5w1h/analyze
Extract 5W1H (Who, What, When, Where, Why, How) from a scene. Uses Gemma4 LLM on port 8082.
### Request
```json
{
"file_uuid": "3abeee81d94597629ed8cb943f182e94",
"scene_id": 42
}
```
### Response
```json
{
"success": true,
"5w1h": {
"who": ["Cary Grant"],
"what": ["discussing plans"],
"when": ["1963"],
"where": ["Paris"],
"why": ["vacation"],
"how": ["in person"]
}
}
```
## POST /api/v1/agents/5w1h/batch
Batch analyze all scenes in a file for 5W1H extraction. Uses the pipeline's `parent_chunk_5w1h.py --mode llm`.
### Request
```json
{
"file_uuid": "3abeee81d94597629ed8cb943f182e94"
}
```
## GET /api/v1/agents/5w1h/status
Get status of the 5W1H agent pipeline for a file.
---
## Embedding Model
| Detail | Value |
|--------|-------|
| **Model** | EmbeddingGemma-300m |
| **Endpoint** | `POST /v1/embeddings` on port 11436 |
| **Dimension** | 768 |
| **Used by** | `parent_chunk_5w1h.py --embed`, story, 5W1H, search |