Merge branch 'main' of http://192.168.110.200:3000/admin/momentry_core

docs_v1.0/DESIGN/LaunchDaemon_Config_M5Max128.md

@@ -0,0 +1,421 @@
+---
+title: LaunchDaemon Architecture (M5Max128 Reference)
+version: 1.0
+date: 2026-05-27
+author: M5Max128
+status: reference
+---
+
+# LaunchDaemon Architecture Reference
+
+> **Scope**: M5Max128 local configuration (resource-managed binaries)
+> **Note**: M5Max48 uses build-from-source approach via start_momentry.sh. Both approaches are valid and independent.
+
+## Overview
+
+| Machine | Approach | Status |
+|---------|----------|--------|
+| M5Max128 | LaunchDaemon + resource binaries | Reference document |
+| M5Max48 | start_momentry.sh + build from source | Main branch |
+
+## Architecture Principles
+
+```
+/Library/LaunchDaemons/ (system-level, boot before login)
+  ├── com.momentry.postgresql.plist  (P1, no dependency)
+  ├── com.momentry.redis.plist       (P1, no dependency)
+  ├── com.momentry.qdrant.plist      (P2, no dependency)
+  ├── com.momentry.mongodb.plist     (P2, no dependency)
+  └── com.momentry.gitea.plist       (P3, depends on PostgreSQL)
+
+Experimental services:
+  └── com.momentry.startup.plist     (LLM, Embedding, Playground, etc.)
+```
+
+## Key Design Points
+
+### 1. Binary Location
+
+All binaries are resource-managed under `/Users/accusys/momentry_resources/bin/`:
+
+| Service | Binary Path |
+|---------|-------------|
+| PostgreSQL | `/Users/accusys/pgsql/18.3/bin/postgres` |
+| Redis | `/Users/accusys/momentry_resources/bin/redis-server` |
+| Qdrant | `/Users/accusys/momentry_resources/bin/qdrant` |
+| MongoDB | `/Users/accusys/momentry_resources/bin/mongod` |
+| Gitea | `/Users/accusys/momentry_resources/bin/gitea` |
+
+### 2. Root Boot → User Execution
+
+LaunchDaemons run at boot (root), but use `UserName` key to switch to user:
+
+```xml
+<key>UserName</key>
+<string>accusys</string>
+```
+
+### 3. Unified Log Path
+
+All logs go to `/Users/accusys/momentry/logs/`:
+
+```xml
+<key>StandardOutPath</key>
+<string>/Users/accusys/momentry/logs/<service>.log</string>
+
+<key>StandardErrorPath</key>
+<string>/Users/accusys/momentry/logs/<service>.error.log</string>
+```
+
+## Plist Templates
+
+### PostgreSQL
+
+```xml
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
+<plist version="1.0">
+<dict>
+    <key>Label</key>
+    <string>com.momentry.postgresql</string>
+    
+    <key>UserName</key>
+    <string>accusys</string>
+    
+    <key>WorkingDirectory</key>
+    <string>/Users/accusys/momentry/var/postgresql</string>
+    
+    <key>ProgramArguments</key>
+    <array>
+        <string>/Users/accusys/pgsql/18.3/bin/postgres</string>
+        <string>-D</string>
+        <string>/Users/accusys/momentry/var/postgresql</string>
+    </array>
+    
+    <key>RunAtLoad</key>
+    <true/>
+    
+    <key>KeepAlive</key>
+    <true/>
+    
+    <key>StandardOutPath</key>
+    <string>/Users/accusys/momentry/logs/postgresql.log</string>
+    
+    <key>StandardErrorPath</key>
+    <string>/Users/accusys/momentry/logs/postgresql.error.log</string>
+</dict>
+</plist>
+```
+
+### Redis (ACL Authentication)
+
+```xml
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
+<plist version="1.0">
+<dict>
+    <key>Label</key>
+    <string>com.momentry.redis</string>
+    
+    <key>UserName</key>
+    <string>accusys</string>
+    
+    <key>WorkingDirectory</key>
+    <string>/Users/accusys/momentry/var/redis</string>
+    
+    <key>ProgramArguments</key>
+    <array>
+        <string>/Users/accusys/momentry_resources/bin/redis-server</string>
+        <string>--port</string>
+        <string>6379</string>
+        <string>--bind</string>
+        <string>0.0.0.0</string>
+        <string>--aclfile</string>
+        <string>/Users/accusys/momentry/etc/redis/users.acl</string>
+        <string>--dir</string>
+        <string>/Users/accusys/momentry/var/redis</string>
+        <string>--logfile</string>
+        <string>/Users/accusys/momentry/logs/redis.log</string>
+    </array>
+    
+    <key>RunAtLoad</key>
+    <true/>
+    
+    <key>KeepAlive</key>
+    <true/>
+    
+    <key>StandardOutPath</key>
+    <string>/Users/accusys/momentry/logs/redis.log</string>
+    
+    <key>StandardErrorPath</key>
+    <string>/Users/accusys/momentry/logs/redis.error.log</string>
+</dict>
+</plist>
+```
+
+### Redis ACL File
+
+Location: `/Users/accusys/momentry/etc/redis/users.acl`
+
+```
+user default on sanitize-payload ~* &* +@all >accusys
+user accusys on sanitize-payload ~* &* +@all >accusys
+```
+
+**Redis 8.x Authentication**:
+```bash
+# Old (deprecated): redis-cli -a accusys ping
+# New (recommended): redis-cli --user default --pass accusys ping
+```
+
+### Qdrant
+
+```xml
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
+<plist version="1.0">
+<dict>
+    <key>Label</key>
+    <string>com.momentry.qdrant</string>
+    
+    <key>UserName</key>
+    <string>accusys</string>
+    
+    <key>WorkingDirectory</key>
+    <string>/Users/accusys/momentry/var/qdrant/</string>
+    
+    <key>ProgramArguments</key>
+    <array>
+        <string>/Users/accusys/momentry_resources/bin/qdrant</string>
+    </array>
+    
+    <key>EnvironmentVariables</key>
+    <dict>
+        <key>QDRANT__STORAGE__STORAGE_PATH</key>
+        <string>/Users/accusys/momentry/var/qdrant/</string>
+        <key>QDRANT__SERVICE__HOST</key>
+        <string>0.0.0.0</string>
+        <key>QDRANT__SERVICE__HTTP_PORT</key>
+        <string>6333</string>
+        <key>HOME</key>
+        <string>/Users/accusys</string>
+    </dict>
+    
+    <key>RunAtLoad</key>
+    <true/>
+    
+    <key>KeepAlive</key>
+    <true/>
+    
+    <key>StandardOutPath</key>
+    <string>/Users/accusys/momentry/logs/qdrant.log</string>
+    
+    <key>StandardErrorPath</key>
+    <string>/Users/accusys/momentry/logs/qdrant.error.log</string>
+</dict>
+</plist>
+```
+
+### MongoDB
+
+```xml
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
+<plist version="1.0">
+<dict>
+    <key>Label</key>
+    <string>com.momentry.mongodb</string>
+    
+    <key>UserName</key>
+    <string>accusys</string>
+    
+    <key>ProgramArguments</key>
+    <array>
+        <string>/Users/accusys/momentry_resources/bin/mongod</string>
+        <string>--dbpath</string>
+        <string>/Users/accusys/momentry/var/mongodb</string>
+        <string>--logpath</string>
+        <string>/Users/accusys/momentry/logs/mongodb.log</string>
+        <string>--port</string>
+        <string>27017</string>
+        <string>--bind_ip</string>
+        <string>0.0.0.0</string>
+    </array>
+    
+    <key>RunAtLoad</key>
+    <true/>
+    
+    <key>KeepAlive</key>
+    <true/>
+    
+    <key>StandardOutPath</key>
+    <string>/Users/accusys/momentry/logs/mongodb.log</string>
+    
+    <key>StandardErrorPath</key>
+    <string>/Users/accusys/momentry/logs/mongodb.error.log</string>
+    
+    <key>WorkingDirectory</key>
+    <string>/Users/accusys/momentry/var/mongodb</string>
+</dict>
+</plist>
+```
+
+### Gitea (with Wrapper Script)
+
+```xml
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
+<plist version="1.0">
+<dict>
+    <key>Label</key>
+    <string>com.momentry.gitea</string>
+    
+    <key>UserName</key>
+    <string>accusys</string>
+    
+    <key>WorkingDirectory</key>
+    <string>/Users/accusys/momentry/var/gitea</string>
+    
+    <key>ProgramArguments</key>
+    <array>
+        <string>/Users/accusys/momentry_core/scripts/start_gitea.sh</string>
+    </array>
+    
+    <key>EnvironmentVariables</key>
+    <dict>
+        <key>HOME</key>
+        <string>/Users/accusys</string>
+        <key>GITEA_WORK_DIR</key>
+        <string>/Users/accusys/momentry/var/gitea</string>
+    </dict>
+    
+    <key>RunAtLoad</key>
+    <true/>
+    
+    <key>KeepAlive</key>
+    <true/>
+    
+    <key>StandardOutPath</key>
+    <string>/Users/accusys/momentry/logs/gitea.log</string>
+    
+    <key>StandardErrorPath</key>
+    <string>/Users/accusys/momentry/logs/gitea.error.log</string>
+</dict>
+</plist>
+```
+
+## Wrapper Script: start_gitea.sh
+
+Gitea depends on PostgreSQL. Wrapper script ensures PostgreSQL is ready:
+
+```bash
+#!/bin/bash
+
+PG_BIN="/Users/accusys/pgsql/18.3/bin"
+GITEA_BIN="/Users/accusys/momentry_resources/bin/gitea"
+GITEA_CONFIG="/Users/accusys/momentry/etc/gitea/app.ini"
+
+MAX_WAIT=60
+WAITED=0
+
+# Wait for PostgreSQL
+while ! "$PG_BIN/pg_isready" -q 2>/dev/null; do
+    if [ $WAITED -ge $MAX_WAIT ]; then
+        echo "ERROR: PostgreSQL not ready after $MAX_WAIT seconds"
+        exit 1
+    fi
+    sleep 2
+    WAITED=$((WAITED + 2))
+done
+
+# Start Gitea
+"$GITEA_BIN" web --config "$GITEA_CONFIG"
+```
+
+## Install Script: install_launchdaemons.sh
+
+```bash
+#!/bin/bash
+
+PLIST_DIR="/Users/accusys/momentry_core/momentry_runtime/plist"
+DAEMON_DIR="/Library/LaunchDaemons"
+LOG_DIR="/Users/accusys/momentry/logs"
+
+mkdir -p "$LOG_DIR"
+
+DAEMONS=(
+    "com.momentry.postgresql"
+    "com.momentry.redis"
+    "com.momentry.qdrant"
+    "com.momentry.mongodb"
+    "com.momentry.gitea"
+)
+
+for daemon in "${DAEMONS[@]}"; do
+    plist_name="${daemon}.plist"
+    src="${PLIST_DIR}/${plist_name}"
+    dest="${DAEMON_DIR}/${plist_name}"
+    
+    if launchctl list "$daemon" >/dev/null 2>&1; then
+        sudo launchctl unload -w "$dest" 2>/dev/null
+    fi
+    
+    sudo cp "$src" "$dest"
+    sudo chown root:wheel "$dest"
+    sudo chmod 644 "$dest"
+    sudo launchctl load -w "$dest"
+done
+```
+
+## Comparison: M5Max128 vs M5Max48
+
+| Aspect | M5Max128 | M5Max48 |
+|--------|----------|---------|
+| **Approach** | LaunchDaemon (system-level) | start_momentry.sh (user script) |
+| **Binaries** | Resource-managed (`momentry_resources/bin/`) | Build from source (`services/*/target/`) |
+| **PostgreSQL data** | `/Users/accusys/momentry/var/postgresql` | `/Users/accusys/pgsql/data` |
+| **Redis auth** | ACL file (`users.acl`) | `--requirepass` (deprecated) |
+| **LLM path** | Resource binary | `/Users/accusys/llama/bin/` |
+| **Gitea** | Independent LaunchDaemon | Not in startup script |
+| **MongoDB** | Independent LaunchDaemon | Not in startup script |
+
+## Installation Steps (M5Max128)
+
+```bash
+# 1. Ensure directories exist
+mkdir -p /Users/accusys/momentry/logs
+mkdir -p /Users/accusys/momentry/var/{postgresql,redis,qdrant,mongodb,gitea}
+
+# 2. Install LaunchDaemons (requires sudo)
+sudo /Users/accusys/momentry_core/scripts/install_launchdaemons.sh
+
+# 3. Verify services
+/Users/accusys/pgsql/18.3/bin/pg_isready
+/Users/accusys/momentry_resources/bin/redis-cli --user default --pass accusys ping
+curl http://localhost:6333/healthz
+curl http://localhost:3000/
+
+# 4. Reboot test
+sudo reboot
+
+# 5. Post-reboot verification
+launchctl list | grep com.momentry
+```
+
+## Notes
+
+1. **Independence**: M5Max128's LaunchDaemons do not conflict with M5Max48's startup script. Each machine has its own approach.
+
+2. **Resource Management**: M5Max128 uses pre-built binaries from `momentry_resources/bin/`, avoiding build dependencies.
+
+3. **Redis ACL**: Redis 8.x uses ACL authentication, not `--requirepass`. This is the modern approach.
+
+4. **Gitea Wrapper**: Essential because Gitea depends on PostgreSQL. The wrapper ensures PostgreSQL is ready before starting Gitea.
+
+---
+
+## Version History
+
+| Version | Date | Author | Changes |
+|---------|------|--------|---------|
+| 1.0 | 2026-05-27 | M5Max128 | Initial reference document |

docs_v1.0/DESIGN/Processor_Refactoring_Assessment.md

@@ -0,0 +1,352 @@
+---
+title: Processor Refactoring Assessment (M5Max128 Research)
+version: 1.0
+date: 2026-05-27
+author: M5Max128
+status: reference
+---
+
+# Processor Refactoring Assessment
+
+> **Scope**: M5Max128 research documentation for M5Max48 implementation reference
+> **Workspace**: ~/workspace/ (22 modules)
+
+## Executive Summary
+
+22 processor modules evaluated for Rust/Swift/Python refactoring feasibility.
+
+### Priority Matrix
+
+| Phase | Language | Modules | Effort | Benefit |
+|-------|----------|---------|--------|---------|
+| 1 | Swift | OCR, Pose, Face | Low | Remove Python wrappers |
+| 2 | Rust | TKG, Resume, Redis | Low | Remove infrastructure deps |
+| 3 | Rust | Cut | Medium | Pure CPU logic |
+| 4 | Swift | YOLO | Medium | ANE acceleration |
+| 5 | Python | Others (keep) | - | ML/LLM dependencies |
+
+---
+
+## Phase 1: Swift Modules (Immediate Gain)
+
+### workspace_ocr
+
+| Metric | Value |
+|--------|-------|
+| Swift Suitability | 10/10 |
+| Current State | Thin Python wrapper around swift_ocr |
+| Refactoring | Delete Python wrapper, Rust calls swift_ocr directly |
+| LOC Change | Python: -122, Rust: ~50 |
+| Risk | Low |
+| Effort | 1 day |
+
+**Current Architecture**:
+```
+Rust (ocr.rs) → PythonExecutor → ocr_processor.py → subprocess → swift_ocr
+```
+
+**Target Architecture**:
+```
+Rust (ocr.rs) → subprocess → swift_ocr
+```
+
+### workspace_pose
+
+| Metric | Value |
+|--------|-------|
+| Swift Suitability | 10/10 |
+| Current State | Thin Python wrapper around swift_pose |
+| Refactoring | Delete Python wrapper, Rust calls swift_pose directly |
+| LOC Change | Python: -150, Rust: ~50 |
+| Risk | Low |
+| Effort | 1 day |
+
+**Current Architecture**:
+```
+Rust (pose.rs) → PythonExecutor → pose_processor.py → subprocess → swift_pose
+```
+
+**Target Architecture**:
+```
+Rust (pose.rs) → subprocess → swift_pose
+```
+
+### workspace_face
+
+| Metric | Value |
+|--------|-------|
+| Swift Suitability | 9/10 |
+| Current State | Swift detect + Python embedding (FaceNet CoreML) |
+| Refactoring | Merge detection + embedding into single Swift binary |
+| LOC Change | Python: -337, Swift: +100 (embedding) |
+| Risk | Medium |
+| Effort | 2-3 days |
+
+**Current Architecture**:
+```
+Stage 1: Python → swift_face (Vision detect) → bbox + landmarks
+Stage 2: Python → OpenCV crop → CoreML FaceNet → 512D embedding
+```
+
+**Target Architecture**:
+```
+Swift: Vision detect → crop → VNCoreMLModel (FaceNet) → embedding → face.json
+```
+
+### workspace_face_recognition
+
+| Metric | Value |
+|--------|-------|
+| Status | **Superseded** |
+| Recommendation | Do not refactor. Archive/remove. |
+| Note | Replaced by face_processor.py (Apple Vision + CoreML) |
+
+---
+
+## Phase 2: Rust Modules (Infrastructure)
+
+### workspace_tkg
+
+| Metric | Value |
+|--------|-------|
+| Rust Suitability | **10/10** |
+| Current State | Python psycopg2 + SQL queries |
+| Dependencies | PostgreSQL, JSON I/O (no ML) |
+| Refactoring | Pure Rust with sqlx/tokio-postgres |
+| LOC Change | Python: -469, Rust: ~350 |
+| Risk | Low |
+| Effort | 1-2 days |
+
+**Graph Structure**:
+```
+NODES:
+  (face_trace)  - one per trace_id
+  (object)      - one per YOLO class
+  (speaker)     - one per speaker_id
+
+EDGES:
+  (face) -[:CO_OCCURS_WITH]-> (object)   same frame
+  (face) -[:SPEAKS_AS]-> (speaker)       temporal overlap
+  (face) -[:CO_OCCURS_WITH]-> (face)     same frame
+```
+
+### workspace_resume_framework
+
+| Metric | Value |
+|--------|-------|
+| Rust Suitability | **10/10** |
+| Current State | Python file I/O + signal handling |
+| Dependencies | File I/O, timers (no ML) |
+| Refactoring | Pure Rust struct with auto-save |
+| LOC Change | Python: -484, Rust: ~150 |
+| Risk | Low |
+| Effort | 1 day |
+
+**Rust Design**:
+```rust
+struct ResumeFramework {
+    path: PathBuf,
+    save_interval: Duration,
+    last_save: Instant,
+    position: Option<u64>,
+}
+
+impl ResumeFramework {
+    fn load_checkpoint(&mut self) -> Result<Option<u64>>
+    fn save_checkpoint(&self, position: u64) -> Result<()>
+    fn auto_save_tick(&mut self, position: u64) -> Result<bool>
+    fn finalize(&mut self, total: u64) -> Result<()>
+}
+```
+
+### workspace_redis_publisher
+
+| Metric | Value |
+|--------|-------|
+| Rust Suitability | **10/10** |
+| Current State | Python redis-py pub/sub |
+| Dependencies | Redis TCP (no ML) |
+| Refactoring | Pure Rust with redis-rs |
+| LOC Change | Python: -195, Rust: ~100 |
+| Risk | Low |
+| Effort | 1 day |
+
+**Rust Design**:
+```rust
+use redis::AsyncCommands;
+
+struct ProgressPublisher {
+    client: redis::Client,
+    channel: String,
+}
+
+impl ProgressPublisher {
+    async fn info(&self, processor: &str, msg: &str) -> Result<()>
+    async fn progress(&self, processor: &str, current: u32, total: u32, msg: &str) -> Result<()>
+    async fn complete(&self, processor: &str, msg: &str) -> Result<()>
+    async fn error(&self, processor: &str, msg: &str) -> Result<()>
+}
+```
+
+---
+
+## Phase 3: Rust CPU Logic
+
+### workspace_cut
+
+| Metric | Value |
+|--------|-------|
+| Rust Suitability | 8/10 |
+| Current State | Python PySceneDetect |
+| Dependencies | Pure CPU (histogram diff) |
+| Refactoring | Port ContentDetector algorithm to Rust |
+| LOC Change | Python: -106, Rust: ~300 |
+| Risk | Medium |
+| Effort | 2-3 days |
+| Challenge | HSV histogram + adaptive threshold |
+
+**Algorithm to Port**:
+- Frame-to-frame HSV/Luma histogram difference
+- Rolling average threshold
+- min_scene_len enforcement
+
+---
+
+## Phase 4: Swift ANE Acceleration
+
+### workspace_yolo
+
+| Metric | Value |
+|--------|-------|
+| Swift Suitability | 8/10 |
+| Current State | Python ultralytics (YOLOv8) |
+| Dependencies | CoreML model conversion needed |
+| Refactoring | Create swift_yolo with VNCoreMLModel |
+| LOC Change | Python: -496, Swift: ~300 |
+| Risk | Medium |
+| Effort | 2-3 days |
+| Challenge | CoreML model conversion, async handling |
+
+**Swift Approach**:
+1. Convert YOLOv8 → CoreML: `yolo export model=yolov8s.pt format=coreml`
+2. Create swift_yolo.swift with VNCoreMLModel
+3. AVAssetReader for frame extraction
+4. ANE-accelerated inference
+
+---
+
+## Phase 5: Python Keep (ML/LLM Dependencies)
+
+### Modules to Keep in Python
+
+| Module | Reason |
+|--------|--------|
+| asr | whisper/faster-whisper (no Rust/Swift equivalent) |
+| asrx | speaker diarization (pyannote) |
+| audio_taxonomy | librosa/tensorflow |
+| lip | MediaPipe lip tracking |
+| caption | LLM generation |
+| scene | ML scene classification |
+| story | LLM generation |
+| story_pipeline | LLM pipeline |
+| tmdb_agent | API agent |
+| identity_agent | LLM agent |
+| voice_embedding | ML embedding |
+| mediapipe_holistic | MediaPipe (no Rust/Swift binding) |
+| visual_chunk | Visual processing |
+
+---
+
+## Implementation Roadmap
+
+### Week 1: Swift Wrapper Removal
+
+1. OCR: Modify `ocr.rs` to call swift_ocr directly
+2. Pose: Modify `pose.rs` to call swift_pose directly
+3. Test both with sample videos
+
+### Week 2: Rust Infrastructure
+
+4. redis_publisher: Create `src/core/redis_publisher.rs`
+5. resume_framework: Create `src/core/resume.rs`
+6. TKG: Create `src/core/processor/tkg.rs`
+
+### Week 3: Swift Enhancement
+
+7. Face: Extend swift_face.swift with CoreML embedding
+8. Test face embedding pipeline
+
+### Week 4: Rust Algorithm Port
+
+9. Cut: Port ContentDetector to Rust
+10. Test scene detection
+
+### Week 5: Swift ANE
+
+11. YOLO: Convert yolov8s → CoreML
+12. Create swift_yolo.swift
+13. Test object detection
+
+---
+
+## Total Effort Estimate
+
+| Phase | LOC (Rust/Swift) | Effort |
+|-------|------------------|--------|
+| 1 | ~100 | 1-2 days |
+| 2 | ~600 | 3-4 days |
+| 3 | ~100 | 2-3 days |
+| 4 | ~300 | 2-3 days |
+| 5 | ~300 | 2-3 days |
+| **Total** | ~1400 | **10-15 days** |
+
+---
+
+## Dependency Removal Summary
+
+| Dependency | Removed By |
+|------------|------------|
+| Python runtime | All Swift/Rust refactors |
+| redis-py | redis_publisher (Rust) |
+| psycopg2 | TKG (Rust) |
+| PySceneDetect | Cut (Rust) |
+| ultralytics (YOLO) | swift_yolo |
+| OpenCV (face crop) | Face Swift embedding |
+| InsightFace | Already superseded |
+
+---
+
+## Appendix: Module Summary Table
+
+| Module | Language | Suitability | Status | Action |
+|--------|----------|-------------|--------|--------|
+| ocr | Swift | 10/10 | Active | Delete wrapper |
+| pose | Swift | 10/10 | Active | Delete wrapper |
+| face | Swift | 9/10 | Active | Extend Swift |
+| face_recognition | - | - | Superseded | Archive |
+| yolo | Swift | 8/10 | Active | Create Swift |
+| cut | Rust | 8/10 | Active | Port algorithm |
+| tkg | Rust | 10/10 | Active | Pure Rust |
+| resume_framework | Rust | 10/10 | Active | Pure Rust |
+| redis_publisher | Rust | 10/10 | Active | Pure Rust |
+| asr | Python | 2/10 | Keep | ML dependency |
+| asrx | Python | 2/10 | Keep | ML dependency |
+| audio_taxonomy | Python | 2/10 | Keep | ML dependency |
+| lip | Python | 2/10 | Keep | ML dependency |
+| caption | Python | 2/10 | Keep | LLM |
+| scene | Python | 2/10 | Keep | ML |
+| story | Python | 2/10 | Keep | LLM |
+| story_pipeline | Python | 2/10 | Keep | LLM |
+| tmdb_agent | Python | 4/10 | Keep | API |
+| identity_agent | Python | 4/10 | Keep | LLM |
+| voice_embedding | Python | 2/10 | Keep | ML |
+| mediapipe_holistic | Python | 2/10 | Keep | ML |
+| visual_chunk | Python | 3/10 | Keep | Visual |
+
+---
+
+## Version History
+
+| Version | Date | Author | Changes |
+|---------|------|--------|---------|
+| 1.0 | 2026-05-27 | M5Max128 | Initial assessment from workspace research |

docs_v1.0/DESIGN/Thumbnail_JPEG_Validation_Impl.md

@@ -0,0 +1,279 @@
+---
+title: Thumbnail JPEG Validation Implementation
+version: 1.0.0
+date: 2026-05-27
+author: M5Max128
+status: ready_for_implementation
+---
+
+# Thumbnail JPEG Validation Implementation
+
+## Overview
+
+Add JPEG quality validation to all ffmpeg image extraction endpoints to prevent:
+- Empty images (0 bytes)
+- Corrupted JPEG (missing header/footer)
+- Incomplete JPEG (truncated output)
+
+## Files to Create/Modify
+
+### 1. Create: `src/core/thumbnail/validator.rs`
+
+```rust
+use anyhow::{bail, Result};
+
+pub const JPEG_MIN_SIZE: usize = 100;
+pub const JPEG_SOI_MARKER: [u8; 3] = [0xFF, 0xD8, 0xFF];
+pub const JPEG_EOI_MARKER: [u8; 2] = [0xFF, 0xD9];
+
+pub fn validate_jpeg(data: &[u8]) -> Result<()> {
+    if data.len() < JPEG_MIN_SIZE {
+        bail!("JPEG too small: {} bytes (minimum {})", data.len(), JPEG_MIN_SIZE);
+    }
+
+    if data[0..3] != JPEG_SOI_MARKER {
+        bail!("Invalid JPEG header: expected {:02X?}, got {:02X?}", JPEG_SOI_MARKER, &data[0..3]);
+    }
+
+    if data[data.len() - 2..] != JPEG_EOI_MARKER {
+        bail!("Incomplete JPEG: missing EOI marker, got {:02X?}", &data[data.len() - 2..]);
+    }
+
+    Ok(())
+}
+
+pub fn is_valid_jpeg(data: &[u8]) -> bool {
+    validate_jpeg(data).is_ok()
+}
+
+pub fn jpeg_size_ok(data: &[u8]) -> bool {
+    data.len() >= JPEG_MIN_SIZE
+}
+
+pub fn jpeg_header_ok(data: &[u8]) -> bool {
+    data.len() >= 3 && data[0..3] == JPEG_SOI_MARKER
+}
+
+pub fn jpeg_footer_ok(data: &[u8]) -> bool {
+    data.len() >= 2 && data[data.len() - 2..] == JPEG_EOI_MARKER
+}
+```
+
+### 2. Modify: `src/core/thumbnail/mod.rs`
+
+Add module declaration at line 1:
+
+```rust
+pub mod validator;
+
+use anyhow::{Context, Result};
+// ... rest of file
+```
+
+### 3. Modify: `src/api/media_api.rs`
+
+Location: `face_thumbnail()` function, after ffmpeg output check (around line 754)
+
+Add validation:
+
+```rust
+if !output.status.success() {
+    return Err(StatusCode::INTERNAL_SERVER_ERROR);
+}
+
+// ADD THIS LINE:
+crate::core::thumbnail::validator::validate_jpeg(&output.stdout)
+    .map_err(|_| StatusCode::INTERNAL_SERVER_ERROR)?;
+
+Ok(Response::builder()
+    // ... rest of response
+```
+
+### 4. Modify: `src/api/trace_agent_api.rs`
+
+Location: `get_trace_thumbnail()` function, after reading bytes (around line 544)
+
+Add validation:
+
+```rust
+let bytes = tokio::fs::read(&tmp).await.map_err(|e| {
+    (StatusCode::INTERNAL_SERVER_ERROR, Json(serde_json::json!({"error": e.to_string()})))
+})?;
+
+let _ = tokio::fs::remove_file(&tmp).await;
+
+// ADD THIS LINE:
+crate::core::thumbnail::validator::validate_jpeg(&bytes)
+    .map_err(|e| {
+        (StatusCode::INTERNAL_SERVER_ERROR, Json(serde_json::json!({"error": e.to_string()})))
+    })?;
+
+Ok(Response::builder()
+    // ... rest of response
+```
+
+### 5. Modify: `src/core/frame_cache.rs`
+
+Location: `FrameManager::extract()`, when iterating extracted frames (around line 73)
+
+Replace the frame collection logic:
+
+```rust
+for entry in &entries {
+    let fname = entry.file_name();
+    let fname_str = fname.to_string_lossy();
+    if let Some(num_str) = fname_str
+        .strip_prefix("frame_")
+        .and_then(|s| s.strip_suffix(".jpg"))
+    {
+        if let Ok(frame_num) = num_str.parse::<u64>() {
+            let frame_path = entry.path();
+            // ADD VALIDATION:
+            if let Ok(data) = std::fs::read(&frame_path) {
+                if crate::core::thumbnail::validator::is_valid_jpeg(&data) {
+                    let timestamp = frame_num as f64 / fps;
+                    frames.push(CachedFrame {
+                        path: frame_path,
+                        frame_number: frame_num,
+                        timestamp_secs: timestamp,
+                    });
+                } else {
+                    info!("[FrameCache] Skipping invalid JPEG: {:?}", frame_path);
+                }
+            }
+        }
+    }
+}
+```
+
+## Python Scripts (Optional Enhancement)
+
+### 6. Create: `scripts/utils/jpeg_validator.py`
+
+```python
+#!/usr/bin/env python3
+"""JPEG validation utilities for ffmpeg-extracted frames."""
+
+JPEG_MIN_SIZE = 100
+JPEG_SOI_MARKER = bytes([0xFF, 0xD8, 0xFF])
+JPEG_EOI_MARKER = bytes([0xFF, 0xD9])
+
+
+def validate_jpeg(data: bytes) -> bool:
+    """Validate JPEG by checking header, footer, and minimum size."""
+    if len(data) < JPEG_MIN_SIZE:
+        return False
+    if data[:3] != JPEG_SOI_MARKER:
+        return False
+    if data[-2:] != JPEG_EOI_MARKER:
+        return False
+    return True
+
+
+def validate_jpeg_file(path: str) -> bool:
+    """Validate JPEG file on disk."""
+    try:
+        with open(path, "rb") as f:
+            data = f.read()
+        return validate_jpeg(data)
+    except Exception:
+        return False
+
+
+def filter_valid_jpegs(paths: list[str]) -> list[str]:
+    """Filter list of paths to only valid JPEGs."""
+    return [p for p in paths if validate_jpeg_file(p)]
+```
+
+### 7. Modify: `scripts/thumbnail_extractor.py`
+
+Location: After extracting each thumbnail (around line 65)
+
+Add validation:
+
+```python
+if result.returncode == 0 and os.path.exists(output_file):
+    # ADD VALIDATION:
+    if validate_jpeg_file(output_file):
+        extracted.append(output_file)
+        print(f"  Extracted: {output_file} at {ts:.1f}s", file=sys.stderr)
+    else:
+        print(f"  Invalid JPEG at {ts:.1f}s", file=sys.stderr)
+        os.remove(output_file)  # Clean up invalid file
+else:
+    print(f"  Failed to extract frame at {ts:.1f}s", file=sys.stderr)
+```
+
+### 8. Modify: `scripts/caption_processor.py`
+
+Location: `extract_frames()` function, after ffmpeg extraction (around line 70)
+
+Add validation:
+
+```python
+try:
+    subprocess.run(cmd, capture_output=True, check=False)
+    if os.path.exists(output_file):
+        # ADD VALIDATION:
+        if validate_jpeg_file(output_file):
+            frames.append({"index": i, "timestamp": timestamp, "path": output_file})
+        else:
+            os.remove(output_file)  # Clean up invalid file
+except Exception:
+    pass
+```
+
+### Python Scripts Affected
+
+| Script | Function | Line | Priority |
+|--------|----------|------|----------|
+| `thumbnail_extractor.py` | `extract_thumbnails()` | 65 | High (user-facing) |
+| `caption_processor.py` | `extract_frames()` | 70 | Medium |
+| `caption_processor_contract_v1.py` | `extract_frames()` | 310 | Medium |
+| `ocr_processor_contract_v1.py` | `extract_frames()` | 367 | Medium |
+| `qa/executor.py` | `extract_frames()` | 93 | Low (QA only) |
+| `face_cross_validate.py` | `extract_frames()` | 16 | Low (testing) |
+| `face_mediapipe_test.py` | `extract_frames()` | 25 | Low (testing) |
+| `analyze_video_faces.py` | `extract_video_frames()` | 61 | Low (analysis) |
+
+## Validation Logic
+
+| Check | Condition | Error if failed |
+|-------|-----------|-----------------|
+| Minimum size | `len() >= 100` | "JPEG too small" |
+| SOI marker | `[0..3] == [0xFF,0xD8,0xFF]` | "Invalid JPEG header" |
+| EOI marker | `[-2..] == [0xFF,0xD9]` | "Incomplete JPEG" |
+
+## Testing
+
+After implementation, run:
+
+```bash
+source ~/.cargo/env
+export MOMENTRY_PYTHON_PATH="/Users/accusys/momentry_core/venv/bin/python"
+cargo clippy --lib
+cargo test --lib
+```
+
+Expected: 220 passed, 0 failed
+
+## Commit Message
+
+```
+feat: add JPEG validation to thumbnail endpoints
+
+- Create validator module with JPEG header/footer/size checks
+- Add validation to face_thumbnail endpoint
+- Add validation to get_trace_thumbnail endpoint
+- Filter invalid JPEGs in FrameManager::extract
+- (Optional) Add Python jpeg_validator utility for script validation
+
+Prevents serving corrupted/incomplete JPEG images to frontend.
+```
+
+## Version History
+
+| Version | Date | Author | Changes |
+|---------|------|--------|---------|
+| 1.0.0 | 2026-05-27 | M5Max128 | Implementation plan ready |
+| 1.1.0 | 2026-05-27 | M5Max128 | Added Python scripts section |

docs_v1.0/DESIGN/Thumbnail_QA_Analysis.md

@@ -0,0 +1,340 @@
+---
+title: Thumbnail Endpoint Quality Assurance Analysis
+version: 1.0.0
+date: 2026-05-27
+author: M5Max128
+status: research_complete
+---
+
+# Thumbnail Endpoint Quality Assurance Analysis
+
+## Scope
+
+| Item | Status |
+|------|--------|
+| Research | Complete |
+| Implementation | Pending (M5Max48) |
+| Affected Endpoints | 2 |
+
+## Overview
+
+Thumbnail endpoints currently lack quality validation, resulting in potential anomalies:
+- **Empty images** - ffmpeg produces 0 bytes output
+- **Black frames** - extracted frame is all black
+- **Corrupted JPEG** - incomplete ffmpeg output
+
+## Affected Endpoints
+
+| Endpoint | File | Line |
+|----------|------|------|
+| `/api/v1/file/:file_uuid/thumbnail` | `src/api/media_api.rs` | 700-764 |
+| `/api/v1/file/:file_uuid/trace/:trace_id/thumbnail` | `src/api/trace_agent_api.rs` | 514-556 |
+
+---
+
+## Anomaly Classification
+
+### Type 1: Empty Image (No Frame)
+
+**Symptom**: Returns 0 bytes or very small JPEG
+
+**Root Causes**:
+1. `frame_number > total_frames` - requested frame exceeds video length
+2. Video file missing or corrupted
+3. Codec does not support frame-level seek
+4. ffmpeg `-vf select` filter finds no matching frame
+
+**Code Locations**:
+- `media_api.rs:710-716` - `query_auto_representative_frame()` may return invalid frame
+- `media_api.rs:720-728` - `file_path` query may return non-existent file
+- `media_api.rs:754-756` - only checks `output.status.success()`, not output content
+
+### Type 2: Black Frame
+
+**Symptom**: Returns valid JPEG but all black or very dark
+
+**Root Causes**:
+1. `crop` parameters exceed video dimensions (`x+w > width` or `y+h > height`)
+2. Extracted frame is from fade-in/fade-out transition
+3. Video has black opening/closing credits
+4. Low-light scene
+
+**Code Locations**:
+- `media_api.rs:731-735` - crop validation missing
+- `trace_agent_api.rs:530` - crop may exceed dimensions
+
+### Type 3: Corrupted JPEG
+
+**Symptom**: Returns incomplete JPEG (browser shows broken image)
+
+**Root Causes**:
+1. ffmpeg stdout pipe interrupted before completion
+2. ffmpeg process killed mid-output
+3. JPEG encoder failure
+4. Incomplete write to stdout buffer
+
+**Code Locations**:
+- `media_api.rs:751` - pipe output may be truncated
+- `media_api.rs:758-763` - no JPEG validation before serving
+
+---
+
+## Current Quality Mechanisms
+
+### Endpoint 1: `face_thumbnail`
+
+| Mechanism | Status | Location |
+|-----------|--------|----------|
+| Representative frame selection | Present | `tkg::query_auto_representative_frame()` |
+| ffmpeg success check | Present | `output.status.success()` |
+| JPEG validation | Missing | - |
+| Size validation | Missing | - |
+| Black frame detection | Missing | - |
+| Retry mechanism | Missing | - |
+
+### Endpoint 2: `get_trace_thumbnail`
+
+| Mechanism | Status | Location |
+|-----------|--------|----------|
+| Blur detection (candidate selection) | Present | `select_rep_face()` lines 463-480 |
+| Confidence filter (>0.7) | Present | `select_rep_face()` line 429 |
+| QC metadata filter | Present | `select_rep_face()` line 430 |
+| ffmpeg success check | Present | `status.status.success()` |
+| JPEG validation | Missing | - |
+| Black frame detection (extraction) | Missing | - |
+| Retry mechanism | Missing | - |
+
+**Note**: `select_rep_face()` has sophisticated quality control for SELECTING the representative face, but the actual EXTRACTION step lacks validation.
+
+---
+
+## Root Cause Analysis
+
+### A. Input Data Problems
+
+| Problem | Impact | Condition |
+|---------|--------|-----------|
+| `frame_number > total_frames` | Empty image | TKG returns wrong frame, user passes invalid value |
+| `crop exceeds dimensions` | Black frame / error | face bbox incorrect, video resolution changed |
+| Video file missing | 500 error | File deleted/moved |
+| Codec不支持seek | Empty/corrupted | Some codecs only support sequential read |
+
+### B. ffmpeg Execution Problems
+
+| Problem | Impact | Cause |
+|---------|--------|-------|
+| `select` no output | Empty JPEG | frame超出範圍 → ffmpeg skips all frames |
+| Pipe interrupted | Corrupted JPEG | stdout buffer full, ffmpeg terminated early |
+| `-ss` imprecise | Wrong frame | input seeking approximate, error ±5 frames |
+| crop failure | Black frame / 500 | `x+w > width` or `y+h > height` |
+
+### C. Quality Control Gaps
+
+| Gap | Impact | Current |
+|-----|--------|---------|
+| No JPEG validation | Corrupted image served | Only checks exit code |
+| No size check | 0 bytes returned | No output length check |
+| No black detection | Black frame served | blurdetect only in candidate selection |
+| No retry | Single failure = error | No retry mechanism |
+
+---
+
+## Concrete Failure Cases
+
+### Case 1: Frame Exceeds Range
+
+```
+Video: total_frames=1000 (DB record)
+Actual: video has only 950 frames (file truncated)
+Request: frame=980
+ffmpeg: select=eq(n\,980) → no match
+Output: 0 bytes JPEG
+Frontend: blank image
+```
+
+### Case 2: Crop Exceeds Dimensions
+
+```
+Video: 1920x1080
+face_bbox: x=1850, y=1050, w=100, h=100
+ffmpeg: crop=100:100:1850:1050
+Result: x+100=1950 > 1920 → ffmpeg error or black border
+```
+
+### Case 3: Seek Imprecise
+
+```
+Video: 25fps
+Request: frame=1000 (40 seconds)
+ffmpeg -ss 40.0 -i video
+Actual: seeks to frame 995~1005 range
+Result: extracts different face than select_rep_face chose
+```
+
+### Case 4: Pipe Interrupted
+
+```
+ffmpeg -i large_video -vf select=eq(n\,50000) -f image2pipe -
+Video large, select needs scan to frame 50000
+Pipe buffer full → ffmpeg may be killed or terminate early
+Output: incomplete JPEG (missing FFD9 footer)
+```
+
+---
+
+## Recommended Fixes
+
+### Phase P0: Critical (Must Implement)
+
+| Fix | Description | LOC | Location |
+|-----|-------------|-----|----------|
+| **Frame validation** | `frame <= total_frames` | ~20 | `media_api.rs:707-718` |
+| **Crop validation** | `x+w <= width, y+h <= height` | ~15 | `media_api.rs:731-735` |
+| **JPEG header check** | `data[0..3] == [0xFF,0xD8,0xFF]` | ~10 | Helper function |
+| **JPEG footer check** | `data[-2..] == [0xFF,0xD9]` | ~10 | Helper function |
+| **Minimum size check** | `data.len() > 100` | ~5 | Helper function |
+
+### Phase P1: Important (Should Implement)
+
+| Fix | Description | LOC | Location |
+|-----|-------------|-----|----------|
+| **Black frame detection** | ffmpeg `-vf blackdetect` filter | ~30 | After extraction |
+| **Output seeking** | Move `-ss` after `-i` for precision | ~5 | `trace_agent_api.rs:527` |
+
+### Phase P2: Enhancement (Nice to Have)
+
+| Fix | Description | LOC | Location |
+|-----|-------------|-----|----------|
+| **Retry mechanism** | Max 3 attempts, offset +30 frames each | ~50 | Both endpoints |
+| **Fallback frame** | Extract middle frame if all fail | ~30 | Both endpoints |
+
+---
+
+## Implementation Plan
+
+### Step 1: Create Validation Module
+
+Create `src/core/thumbnail/validator.rs`:
+
+```rust
+pub fn validate_jpeg(data: &[u8]) -> Result<()> {
+    // P0-1: Minimum size
+    if data.len() < 100 {
+        bail!("JPEG too small: {} bytes", data.len());
+    }
+    
+    // P0-2: JPEG header (SOI marker)
+    if data[0..3] != [0xFF, 0xD8, 0xFF] {
+        bail!("Invalid JPEG header");
+    }
+    
+    // P0-3: JPEG footer (EOI marker)
+    if data[data.len()-2..] != [0xFF, 0xD9] {
+        bail!("Incomplete JPEG");
+    }
+    
+    Ok(())
+}
+```
+
+### Step 2: Add Frame/Crop Validation
+
+In `media_api.rs`:
+
+```rust
+// P0-4: Validate frame number
+let total_frames: i64 = sqlx::query_scalar(...)
+    .bind(&file_uuid)
+    .fetch_one(pool)
+    .await?;
+    
+if frame > total_frames {
+    return Err(StatusCode::BAD_REQUEST);
+}
+
+// P0-5: Validate crop dimensions
+if let (Some(x), Some(y), Some(w), Some(h)) = (q.x, q.y, q.w, q.h) {
+    let (width, height): (i32, i32) = sqlx::query_as(...)
+        .bind(&file_uuid)
+        .fetch_one(pool)
+        .await?;
+    
+    if x + w > width || y + h > height {
+        return Err(StatusCode::BAD_REQUEST);
+    }
+}
+```
+
+### Step 3: Integrate Validation
+
+In both endpoints, after ffmpeg extraction:
+
+```rust
+// Apply validation
+validate_jpeg(&output.stdout)
+    .map_err(|_| StatusCode::INTERNAL_SERVER_ERROR)?;
+```
+
+---
+
+## Testing Strategy
+
+### Test Cases
+
+| Test | Input | Expected |
+|------|-------|----------|
+| Valid frame | `frame=500` (valid) | JPEG returned |
+| Frame exceeds | `frame=999999` | 400 BAD_REQUEST |
+| Valid crop | `x=100,y=100,w=200,h=200` | JPEG returned |
+| Crop exceeds | `x=1800,y=1000,w=200,h=200` | 400 BAD_REQUEST |
+| Empty video | corrupted video file | 500 INTERNAL_ERROR |
+| Black frame | fade-out frame | Retry or fallback |
+
+---
+
+## Files to Modify
+
+| File | Changes |
+|------|---------|
+| `src/core/thumbnail/mod.rs` | Add validator module |
+| `src/core/thumbnail/validator.rs` | New file (validation helpers) |
+| `src/api/media_api.rs` | Add validation in `face_thumbnail()` |
+| `src/api/trace_agent_api.rs` | Add validation in `get_trace_thumbnail()` |
+
+---
+
+## Estimated Effort
+
+| Phase | LOC | Time |
+|-------|-----|------|
+| P0 (Critical) | ~60 | 1-2 days |
+| P1 (Important) | ~35 | 1 day |
+| P2 (Enhancement) | ~80 | 2-3 days |
+| **Total** | ~175 | 4-6 days |
+
+---
+
+## Version History
+
+| Version | Date | Author | Changes |
+|---------|------|--------|---------|
+| 1.0.0 | 2026-05-27 | M5Max128 | Initial analysis complete |
+
+---
+
+## Next Steps for M5Max48
+
+1. Read this document
+2. Implement P0 fixes first
+3. Test with edge cases
+4. Add P1/P2 as needed
+5. Update `AGENTS.md` if adding new validation commands
+
+---
+
+## References
+
+- `docs_v1.0/DESIGN/Processor_Refactoring_Assessment.md` - Processor refactoring priorities
+- `src/api/media_api.rs:700-764` - face_thumbnail implementation
+- `src/api/trace_agent_api.rs:394-556` - select_rep_face and get_trace_thumbnail
+- `ffmpeg -vf blackdetect` documentation

docs_v1.0/OPERATIONS/VERIFICATION_RUSTDESK_2026-05-22.md

@@ -0,0 +1,26 @@
+# RustDesk Verification Report
+
+**Date**: 2026-05-22
+**Status**: Verified
+
+## Source
+
+| Item | Value |
+|------|-------|
+| Version | 1.4.6 |
+| Source | GitHub release |
+| Source repo | `admin/rustdesk` (Gitea) |
+| Binary | `~/Applications/RustDesk.app` |
+| Install | DMG → copy to ~/Applications |
+
+## Verification
+
+| Check | Result |
+|-------|--------|
+| DMG downloaded | ✅ 24MB, aarch64 |
+| .app bundle | ✅ 59MB |
+| Architecture | Apple Silicon (arm64) |
+
+## Linked Documents
+
+- `docs_v1.0/OPERATIONS/Services_Inventory.md`

release/system/v1.0/services/sources_manifest.json

@@ -263,6 +263,14 @@
       "file_count": 1279,
       "gitea_repo": "http://192.168.110.200:3000/admin/yt.git",
       "verification_doc": "docs_v1.0/OPERATIONS/Services_Inventory.md"
+    },
+    {
+      "name": "rustdesk-1.4.6-aarch64.dmg",
+      "type": "file",
+      "size_mb": 24,
+      "description": "RustDesk remote desktop",
+      "gitea_repo": "http://192.168.110.200:3000/admin/rustdesk.git",
+      "verification_doc": "docs_v1.0/OPERATIONS/VERIFICATION_RUSTDESK_2026-05-22.md"
     }
   ]
 }