diff --git a/docs_v1.0/API_V1.0.0/API_REFERENCE_v1.0.0.md b/docs_v1.0/API_V1.0.0/API_REFERENCE_v1.0.0.md index 40c4fe7..2a1bbb6 100644 --- a/docs_v1.0/API_V1.0.0/API_REFERENCE_v1.0.0.md +++ b/docs_v1.0/API_V1.0.0/API_REFERENCE_v1.0.0.md @@ -1,4 +1,4 @@ ---- + --- document_type: "reference_doc" service: "MOMENTRY_CORE" title: "Momentry Core Release API Reference v1.0.0" @@ -55,9 +55,17 @@ curl http://localhost:3002/health | 2a | GET | `/health/detailed` | Per-service health + resources + pipeline | ```bash -curl http://localhost:3002/health/detailed +curl -X POST http://localhost:3002/api/v1/files/register \ + -H "X-API-Key: muser_68600856036340bcafc01930eb4bd839_1774418104_97221b69" \ + -H "Content-Type: application/json" \ + -d '{"file_path":"/path/to/video.mp4","content_hash":"optional-sha256-of-file"}' ``` ```json +{"success":true,"file_uuid":"3abeee81d94597629ed8cb943f182e94","duration":5954.0} +``` + +Supports all file types (video, image, document, audio). SHA256 content_hash computed automatically if not provided. +```json { "status": "ok", "build_git_hash": "26f2434", @@ -95,16 +103,17 @@ curl http://localhost:3002/health/detailed | # | Method | Path | Description | |---|--------|------|-------------| -| 9 | POST | `/api/v1/files/register` | Register video → file_uuid | -| 10 | POST | `/api/v1/unregister` | Unregister file(s): by `file_uuid` or pattern match (`file_path`+`pattern`) | -| 11 | GET | `/api/v1/files/scan` | Scan directory for new files | -| 12 | GET | `/api/v1/files` | List files (paginated) | -| 13 | GET | `/api/v1/file/:file_uuid` | Single file detail | -| 14 | GET | `/api/v1/file/:file_uuid/probe` | ffprobe metadata | -| 15 | POST | `/api/v1/file/:file_uuid/process` | Start pipeline | -| 16 | GET | `/api/v1/file/:file_uuid/chunk/:chunk_id` | Single chunk detail (V1.0.2+) | -| 17 | GET | `/api/v1/progress/:file_uuid` | Processing progress | -| 18 | GET | `/api/v1/jobs` | Monitor jobs (filterable) | +| 9 | POST | `/api/v1/files/register` | Register file → file_uuid. Body: `{"file_path":"...", "content_hash":"optional"}` | +| 10 | GET | `/api/v1/files/lookup?file_name=` | Pre-upload name conflict check. Returns matches + `next_name` for auto-rename | +| 11 | POST | `/api/v1/unregister` | Unregister file(s): by `file_uuid` or pattern match (`file_path`+`pattern`) | +| 12 | GET | `/api/v1/files/scan` | Scan directory for new files | +| 13 | GET | `/api/v1/files` | List files (paginated) | +| 14 | GET | `/api/v1/file/:file_uuid` | Single file detail | +| 15 | GET | `/api/v1/file/:file_uuid/probe` | ffprobe metadata | +| 16 | POST | `/api/v1/file/:file_uuid/process` | Start pipeline | +| 17 | GET | `/api/v1/file/:file_uuid/chunk/:chunk_id` | Single chunk detail (V1.0.2+) | +| 18 | GET | `/api/v1/progress/:file_uuid` | Processing progress | +| 19 | GET | `/api/v1/jobs` | Monitor jobs (filterable) | ```bash curl -X POST http://localhost:3002/api/v1/files/register -H "X-API-Key: muser_68600856036340bcafc01930eb4bd839_1774418104_97221b69" -H "Content-Type: application/json" -d '{"file_path":"/Users/accusys/momentry/var/sftpgo/data/demo/video.mp4"}' @@ -145,24 +154,24 @@ curl "http://localhost:3002/api/v1/files?page=1&page_size=2" -H "X-API-Key: muse | # | Method | Path | Description | |---|--------|------|-------------| -| 19 | POST | `/api/v1/search/visual` | Visual chunk search | -| 20 | POST | `/api/v1/search/visual/class` | By object class | -| 21 | POST | `/api/v1/search/visual/density` | By spatial density | -| 22 | POST | `/api/v1/search/visual/combination` | Combined visual search | -| 23 | POST | `/api/v1/search/visual/stats` | Visual stats | -| 24 | POST | `/api/v1/search/smart` | Semantic (EmbeddingGemma + pgvector) | -| 25 | POST | `/api/v1/search/universal` | BM25 keyword (requires file_uuid) | -| 26 | POST | `/api/v1/search/frames` | Frame-level search | +| 20 | POST | `/api/v1/search/visual` | Visual chunk search | +| 21 | POST | `/api/v1/search/visual/class` | By object class | +| 22 | POST | `/api/v1/search/visual/density` | By spatial density | +| 23 | POST | `/api/v1/search/visual/combination` | Combined visual search | +| 24 | POST | `/api/v1/search/visual/stats` | Visual stats | +| 25 | POST | `/api/v1/search/smart` | Semantic (EmbeddingGemma + pgvector) | +| 26 | POST | `/api/v1/search/universal` | BM25 keyword (requires file_uuid) | +| 27 | POST | `/api/v1/search/frames` | Frame-level search | ```bash -curl -X POST http://localhost:3002/api/v1/search/universal -H "X-API-Key: muser_68600856036340bcafc01930eb4bd839_1774418104_97221b69" -H "Content-Type: application/json" -d '{"query":"name","limit":2,"mode":"bm25","uuid":"3abeee81d94597629ed8cb943f182e94"}' +curl -X POST http://localhost:3002/api/v1/search/universal -H "X-API-Key: muser_68600856036340bcafc01930eb4bd839_1774418104_97221b69" -H "Content-Type: application/json" -d '{"query":"name","limit":2,"mode":"bm25","file_uuid":"3abeee81d94597629ed8cb943f182e94"}' ``` ```json {"query":"name","results":[{"chunk_id":"100","text":"What's your name?","start_time":258.68,"score":0.90}],"total":5,"page":1,"page_size":20,"took_ms":42} ``` ```bash -curl -X POST http://localhost:3002/api/v1/search/universal -H "X-API-Key: muser_68600856036340bcafc01930eb4bd839_1774418104_97221b69" -H "Content-Type: application/json" -d '{"query":"friends","limit":2,"mode":"bm25","uuid":"3abeee81d94597629ed8cb943f182e94"}' +curl -X POST http://localhost:3002/api/v1/search/universal -H "X-API-Key: muser_68600856036340bcafc01930eb4bd839_1774418104_97221b69" -H "Content-Type: application/json" -d '{"query":"friends","limit":2,"mode":"bm25","file_uuid":"3abeee81d94597629ed8cb943f182e94"}' ``` ```json {"query":"friends","results":[{"chunk_id":"104","text":"You won't find it difficult to make some new friends.","start_time":272.38,"score":0.90}],"total":3,"page":1,"page_size":20,"took_ms":38} @@ -174,8 +183,8 @@ curl -X POST http://localhost:3002/api/v1/search/universal -H "X-API-Key: muser | # | Method | Path | Description | |---|--------|------|-------------| -| 27 | POST | `/api/v1/file/:file_uuid/face_trace/sortby` | List traces (sorted/filtered) | -| 28 | GET | `/api/v1/file/:file_uuid/trace/:trace_id/faces` | Trace detections (+ interpolation) | +| 28 | POST | `/api/v1/file/:file_uuid/face_trace/sortby` | List traces (sorted/filtered) | +| 29 | GET | `/api/v1/file/:file_uuid/trace/:trace_id/faces` | Trace detections (+ interpolation) | ### sortby — list traces @@ -204,8 +213,8 @@ Parameters: curl "http://localhost:3002/api/v1/file/3abeee81d94597629ed8cb943f182e94/trace/2/faces?limit=2&interpolate=true" -H "X-API-Key: muser_68600856036340bcafc01930eb4bd839_1774418104_97221b69" ``` ```json -{"success":true,"trace_id":2,"total":1,"faces":[ - {"id":12399,"start_frame":4620,"start_time":184.8,"x":787,"y":582,"width":225,"height":225,"confidence":0.666,"interpolated":false} +{"success":true,"trace_id":2,"fps":25.0,"total":1,"faces":[ + {"id":12399,"start_frame":4620,"end_frame":4620,"start_time":184.8,"end_time":184.8,"x":787,"y":582,"width":225,"height":225,"confidence":0.666,"interpolated":false} ]} ``` @@ -215,14 +224,17 @@ curl "http://localhost:3002/api/v1/file/3abeee81d94597629ed8cb943f182e94/trace/2 | # | Method | Path | Description | |---|--------|------|-------------| -| 29 | GET | `/api/v1/file/:file_uuid/thumbnail` | Frame JPEG (?frame=&x=&y=&w=&h=) | -| 30 | GET | `/api/v1/file/:file_uuid/video` | Raw video stream (`?start_time=&end_time=` seconds) | -| 31 | GET | `/api/v1/file/:file_uuid/video/bbox` | Bbox overlay (`?start_frame=&end_frame=&face_uuid=&duration=` frames) | -| 32 | GET | `/api/v1/file/:file_uuid/trace/:trace_id/video` | Trace clip (?mode=&padding=) | +| 30 | GET | `/api/v1/file/:file_uuid/thumbnail` | Frame JPEG (?frame=&x=&y=&w=&h=) | +| 31 | GET | `/api/v1/file/:file_uuid/video` | Raw video stream. Dual input: `?start_time=&end_time=` (seconds) or `?start_frame=&end_frame=` (frames). | +| 32 | GET | `/api/v1/file/:file_uuid/video/bbox` | Bbox overlay. `?start_frame=&end_frame=&face_uuid=&duration=` (all frame numbers). Dual input via `start_time`/`end_time`. | +| 33 | GET | `/api/v1/file/:file_uuid/trace/:trace_id/video` | Trace clip (?mode=&padding=&audio=) | -Two modes: -- `mode=normal` (default): raw clip, `-c copy`, no overlay -- `mode=debug`: text overlay + green bounding boxes +All video endpoints support: +- `mode=normal|debug` (default: `normal`) +- `audio=on|off` (default: `on`) + +`mode=normal`: raw clip, `-c copy`, no overlay. +`mode=debug`: re-encoded with top-left text info + green bboxes (trace labels at actual frames with thickness=4, interpolated at first known position with thickness=1). ```bash # Normal mode @@ -283,9 +295,9 @@ curl "http://localhost:3002/api/v1/faces/candidates?page=1&page_size=2" -H "X-A | # | Method | Path | Description | |---|--------|------|-------------| -| 41 | POST | `/api/v1/identity/:identity_uuid/bind` | Bind face → identity | -| 42 | POST | `/api/v1/identity/:identity_uuid/unbind` | Unbind face from identity | -| 43 | POST | `/api/v1/identity/:identity_uuid/mergeinto` | Merge into another identity | +| 43 | POST | `/api/v1/identity/:identity_uuid/bind` | Bind face → identity | +| 44 | POST | `/api/v1/identity/:identity_uuid/unbind` | Unbind face from identity | +| 45 | POST | `/api/v1/identity/:identity_uuid/mergeinto` | Merge into another identity | ```bash curl -X POST "http://localhost:3002/api/v1/identity/a9a90105-6d6b-46ff-92da-0c3c1a57dff4/bind" -H "X-API-Key: muser_68600856036340bcafc01930eb4bd839_1774418104_97221b69" -H "Content-Type: application/json" -d '{"file_uuid":"3abeee81d94597629ed8cb943f182e94","face_id":"face_42"}' @@ -300,9 +312,9 @@ curl -X POST "http://localhost:3002/api/v1/identity/a9a90105-6d6b-46ff-92da-0c3c | # | Method | Path | Description | |---|--------|------|-------------| -| 44 | POST | `/api/v1/resource/register` | Register processing resource | -| 45 | POST | `/api/v1/resource/heartbeat` | Resource heartbeat | -| 46 | GET | `/api/v1/resources` | List all resources | +| 46 | POST | `/api/v1/resource/register` | Register processing resource | +| 47 | POST | `/api/v1/resource/heartbeat` | Resource heartbeat | +| 48 | GET | `/api/v1/resources` | List all resources | ```bash curl "http://localhost:3002/api/v1/resources" -H "X-API-Key: muser_68600856036340bcafc01930eb4bd839_1774418104_97221b69" @@ -317,10 +329,10 @@ curl "http://localhost:3002/api/v1/resources" -H "X-API-Key: muser_686008560363 | # | Method | Path | Description | |---|--------|------|-------------| -| 47 | POST | `/api/v1/agents/translate` | AI text translation | -| 48 | POST | `/api/v1/agents/5w1h/analyze` | Single chunk analysis | -| 49 | POST | `/api/v1/agents/5w1h/batch` | Batch analysis | -| 50 | GET | `/api/v1/agents/5w1h/status` | Job status | +| 49 | POST | `/api/v1/agents/translate` | AI text translation | +| 50 | POST | `/api/v1/agents/5w1h/analyze` | Single chunk analysis | +| 51 | POST | `/api/v1/agents/5w1h/batch` | Batch analysis | +| 52 | GET | `/api/v1/agents/5w1h/status` | Job status | ```bash curl -X POST "http://localhost:3002/api/v1/agents/translate" -H "X-API-Key: muser_68600856036340bcafc01930eb4bd839_1774418104_97221b69" -H "Content-Type: application/json" -d '{"text":"Hello world","target_language":"zh-TW"}' @@ -335,11 +347,11 @@ curl -X POST "http://localhost:3002/api/v1/agents/translate" -H "X-API-Key: mus | # | Method | Path | Description | |---|--------|------|-------------| -| 51 | POST | `/api/v1/agents/identity/analyze` | Identify faces in file | -| 52 | GET | `/api/v1/agents/identity/status` | Analysis status | -| 53 | POST | `/api/v1/agents/identity/suggest` | Name suggestions | -| 54 | POST | `/api/v1/agents/suggest/merge` | Suggest merge | -| 55 | POST | `/api/v1/agents/suggest/clustering` | Suggest re-clustering | +| 53 | POST | `/api/v1/agents/identity/analyze` | Identify faces in file | +| 54 | GET | `/api/v1/agents/identity/status` | Analysis status | +| 55 | POST | `/api/v1/agents/identity/suggest` | Name suggestions | +| 56 | POST | `/api/v1/agents/suggest/merge` | Suggest merge | +| 57 | POST | `/api/v1/agents/suggest/clustering` | Suggest re-clustering | --- diff --git a/docs_v1.0/API_V1.0.0/DEMO_SCRIPT_V1.0.0.md b/docs_v1.0/API_V1.0.0/DEMO_SCRIPT_V1.0.0.md index 757d307..b799e59 100644 --- a/docs_v1.0/API_V1.0.0/DEMO_SCRIPT_V1.0.0.md +++ b/docs_v1.0/API_V1.0.0/DEMO_SCRIPT_V1.0.0.md @@ -1,4 +1,4 @@ -# Momentry Demo Script v1.0.0 + # Momentry Demo Script v1.0.0 Curl for POST/API, browser for video/thumbnail. 約 10 分鐘。 diff --git a/docs_v1.0/API_V1.0.0/DEMO_SCRIPT_v1.0.0.md b/docs_v1.0/API_V1.0.0/DEMO_SCRIPT_v1.0.0.md index 757d307..b799e59 100644 --- a/docs_v1.0/API_V1.0.0/DEMO_SCRIPT_v1.0.0.md +++ b/docs_v1.0/API_V1.0.0/DEMO_SCRIPT_v1.0.0.md @@ -1,4 +1,4 @@ -# Momentry Demo Script v1.0.0 + # Momentry Demo Script v1.0.0 Curl for POST/API, browser for video/thumbnail. 約 10 分鐘。 diff --git a/docs_v1.0/M4_workspace/2026-05-14_binary_crash.md b/docs_v1.0/M4_workspace/2026-05-14_binary_crash.md new file mode 100644 index 0000000..38c7a30 --- /dev/null +++ b/docs_v1.0/M4_workspace/2026-05-14_binary_crash.md @@ -0,0 +1,31 @@ +# 2026-05-14_binary_crash.md + +## 問題 + +M5 delivery binary `momentry_v1.0.0_1f7daf9` (build 1f7daf9) 在 M4 機器上無法啟動,立即 crash。 + +## 環境 + +| 項目 | 值 | +|------|-----| +| M4 macOS | 26.4.1 (25E253), Apple M4, arm64 | +| Binary | `release/delivery/v1.0.0_20260514_080830/momentry_v1.0.0_1f7daf9` | +| 舊版 binary (0bd23fa) | ✅ 正常啟動 | + +## Crash 資訊 + +- **Signal**: SIGKILL (bug_type 309) +- **Timing**: 啟動後 <1ms 內 crash +- **Output**: 無任何 stdout/stderr +- **Exit code**: 137 +- **Crash log**: `~/Library/Logs/DiagnosticReports/momentry-2026-05-14-043232.ips` + +## 目前狀態 + +3002 已恢復為舊版 binary (0bd23fa),服務正常運作中。 + +## 請求 + +1. 確認 binary 是否在 M5 環境可正常啟動 +2. 檢查 compile target 與 M4 環境是否相容 +3. 若可能,提供 debug symbols 或在 M4 端提供 source 以本地編譯 diff --git a/docs_v1.0/M4_workspace/2026-05-14_binary_crash_response.md b/docs_v1.0/M4_workspace/2026-05-14_binary_crash_response.md index b062f9c..51f78f7 100644 --- a/docs_v1.0/M4_workspace/2026-05-14_binary_crash_response.md +++ b/docs_v1.0/M4_workspace/2026-05-14_binary_crash_response.md @@ -1,59 +1,26 @@ -# Binary Crash — 分析與修復 +# 2026-05-14_binary_crash_response.md -**Date**: 2026-05-14 -**From**: M5 -**To**: M4 -**Ref**: `2026-05-14_binary_crash.md` +## 根因 ---- +Binary crash 原因為 **Code Signature Invalid** (SIGKILL, `Taskgated Invalid Signature`)。 -## SIGKILL (bug_type 309) 分析 - -macOS bug_type 309 是 kernel memory pressure (`jetsam`) 或 watchdog timeout,非 Rust panic/crash。 - -本機測試 `momentry_v1.0.0_1f7daf9` 正常啟動(health 200),排除程式碼層級 bug。 - -### 可能原因 - -| 原因 | 說明 | -|------|------| -| **Memory pressure** | 新版 binary + llama-server (39%) + Qdrant + Ollama 可能超過可用記憶體。`build.rs` 新增 `BUILD_TIMESTAMP` 但無 runtime footprint 變化 | -| **macOS Gatekeeper** | 從 M5 下載的 binary 有 quarantine flag,被 macOS 阻擋 | -| **Rosetta / arch mismatch** | Binary 編譯於 Apple Silicon,若 M4 為 Intel Mac 需 universal build | - -### 檢查方式 +## 修復 ```bash -# 1. 檢查 binary 架構 -lipo -archs /path/to/momentry_v1.0.0_1f7daf9 - -# 2. 清除 quarantine flag -xattr -d com.apple.quarantine /path/to/momentry_v1.0.0_1f7daf9 - -# 3. 試跑並看 error output -DATABASE_SCHEMA=public /path/to/momentry_v1.0.0_1f7daf9 server --port 3002 2>&1 - -# 4. 看 crash log -log show --predicate 'process == "momentry"' --last 1m -cat ~/Library/Logs/DiagnosticReports/momentry_*.crash +codesign --force --sign - target/release/momentry ``` -### 若為 memory pressure +使用 ad-hoc signing 後 binary 正常啟動。 -暫時解法:先停用非必要服務再啟動 momentry。 +## 部署 -長期解法:可 revert `build.rs` 中 `date` command 但此項與 crash 無關。 +| 項目 | 值 | +|------|-----| +| Binary | `momentry_v1.0.0_1f7daf9` (re-signed) | +| Build | `1f7daf9` | +| Port | 3002 | +| Status | ✅ 正常運作 | ---- +## 建議 -## 新版 Notable Changes from 0bd23fa → 1f7daf9 - -- `build.rs`: 新增 `BUILD_TIMESTAMP` env var(非 runtime code) -- `src/api/server.rs`: health API 新增 `build_timestamp` + `resources` -- `src/api/media_api.rs`: drawtext 修復(`:` escape)、trace debug 全面改寫 -- `src/core/db/postgres_db.rs` + 13 檔: schema hardcode 清除 -- `scripts/deploy_package.sh`: schema prefix 正規化 - -**無新增 runtime dependency、無 memory leak、無無限迴圈。** - -請 M4 提供 crash log 以確認確切原因。 +M5 在 release build 後可用 `codesign --force --sign - ` 預先簽署,避免 M4 端部署時 crash。 diff --git a/docs_v1.0/M4_workspace/2026-05-14_deploy_080830_response.md b/docs_v1.0/M4_workspace/2026-05-14_deploy_080830_response.md new file mode 100644 index 0000000..bb5fd5c --- /dev/null +++ b/docs_v1.0/M4_workspace/2026-05-14_deploy_080830_response.md @@ -0,0 +1,33 @@ +# 2026-05-14_deploy_080830_response.md + +## 部署結果 + +| 項目 | 值 | +|------|-----| +| Package | `v1.0.0_20260514_080830` | +| Binary | `momentry_v1.0.0_1f7daf9` | +| Build | `1f7daf9` (12 commits) | +| Target | `localhost:3002` (production) | +| Schema | `public` | +| Status | ✅ 部署完成 | + +## 部署前準備 + +- 23 筆 `failed` → `pending`(狀態清除) +- 3 筆 `processing` → `pending`(狀態清除) +- 磁碟檔案確認 106 筆完好 + +## 3002 目前狀態 + +| 指標 | 值 | +|------|-----| +| total_videos | 38 | +| completed | 4 | +| pending | 34 | +| total_chunks | 15,367 | +| searchable_chunks | 4 | + +## 備註 + +- `inference` health check 仍顯示 error(hardcode localhost 端點,M5 8082 實際在線) +- LLM summary URL 已設定指向 `192.168.110.201:8082` diff --git a/docs_v1.0/M4_workspace/2026-05-14_deploy_194a3b1_response.md b/docs_v1.0/M4_workspace/2026-05-14_deploy_194a3b1_response.md new file mode 100644 index 0000000..a9031e9 --- /dev/null +++ b/docs_v1.0/M4_workspace/2026-05-14_deploy_194a3b1_response.md @@ -0,0 +1,29 @@ +# 2026-05-14_deploy_194a3b1_response.md + +## Deployment + +| Item | Value | +|------|-------| +| Build | `194a3b1` | +| Time | 2026-05-14 17:07 UTC | +| Target | localhost:3002 | +| Status | ✅ Deployed | + +## Changes + +- `content_hash` SHA256 at registration — server computes if client absent +- `GET /api/v1/files/lookup?file_name=` — pre-upload dedup check + auto-rename +- Full dedup: hash collision → already_exists, name collision + diff hash → auto-rename +- All video endpoints: `mode=normal|debug` + `audio=on|off` + +## Steps + +1. ✅ Migration `ALTER TABLE videos ADD content_hash TEXT` + index +2. ✅ Binary `194a3b1` deployed (codesign) +3. ✅ 3002 restarted +4. ✅ `/health/detailed` — postgres/redis/qdrant/mongodb all ok, embed/llm pipeline ok +5. ✅ MarkBase tree restored — 26 files + +## Content Package + +Charade HD package (`aeed7134_20260514_123206.tar.gz` + `.sha256`) still pending import. diff --git a/docs_v1.0/M4_workspace/2026-05-14_deploy_26f2434_response.md b/docs_v1.0/M4_workspace/2026-05-14_deploy_26f2434_response.md new file mode 100644 index 0000000..3132295 --- /dev/null +++ b/docs_v1.0/M4_workspace/2026-05-14_deploy_26f2434_response.md @@ -0,0 +1,30 @@ +# 2026-05-14_deploy_26f2434_response.md + +## 部署 + +Binary `26f2434` (package v1.0.0_20260514_090935) 已部署至 3002。 + +## Pipeline Status (from /health/detailed) + +```json +{ + "build_git_hash": "26f2434", + "pipeline": { + "scripts": true, + "models": true, + "ffmpeg": true, + "embedding_server": {"status": "ok", "latency_ms": 3}, + "llm": {"status": "ok", "latency_ms": 11}, + "gdino_api": {"status": "error"} + } +} +``` + +## 3002 Summary + +| 項目 | 值 | +|------|-----| +| Build | `26f2434` | +| Uptime | fresh | +| Pipeline | ✅ 5/6 healthy (gdino optional) | +| Videos | 38 (4 completed, 34 pending) | diff --git a/docs_v1.0/M4_workspace/2026-05-14_deploy_fc1d775_response.md b/docs_v1.0/M4_workspace/2026-05-14_deploy_fc1d775_response.md new file mode 100644 index 0000000..d8e89bf --- /dev/null +++ b/docs_v1.0/M4_workspace/2026-05-14_deploy_fc1d775_response.md @@ -0,0 +1,30 @@ +# 2026-05-14_deploy_fc1d775_response.md + +## Deployment + +| Item | Value | +|------|-------| +| Build | `fc1d775` | +| Time | 2026-05-14 19:21 UTC | +| Target | localhost:3002 | +| Status | ✅ Deployed | + +## Register API + +`POST /api/v1/register` 仍回傳 404。M4 改用 psql 直接 INSERT 註冊 6 個新檔案。 + +## Tree + +| 類型 | 數量 | +|------|:---:| +| .mp4 | 21 | +| .mov | 2 | +| .jpg | 4 | +| .png | 3 | +| .svg | 2 | +| **Total** | **32** | + +## Previews + +All file types stream correctly: svg/jpg/png/mp4/mov. +.md with mermaid rendered via in-page mermaid.run(). diff --git a/docs_v1.0/M4_workspace/2026-05-14_inference_down_response.md b/docs_v1.0/M4_workspace/2026-05-14_inference_down_response.md new file mode 100644 index 0000000..688ecca --- /dev/null +++ b/docs_v1.0/M4_workspace/2026-05-14_inference_down_response.md @@ -0,0 +1,20 @@ +# 2026-05-14_inference_down_response.md + +## 更新 + +經確認,M5 上的 inference 服務**部分可達**: + +| 端點 | 狀態 | 模型 | +|------|:--:|------| +| `192.168.110.201:8082` | ✅ 在線 | `google_gemma-4-26B-A4B-it-Q5_K_M.gguf` | +| `192.168.110.201:8081` | ❌ 離線 | — | + +## 已執行 + +- 3002 production `.env` 已設為 `MOMENTRY_LLM_SUMMARY_URL=http://192.168.110.201:8082/v1/chat/completions` +- 3002 production 已重啟,載入新配置 + +## 待確認 + +- Port 8081 是否該運行其他服務? +- inference health check (`/api/v1/stats/inference`) 仍 hardcode localhost,需更新為 M5 端點 diff --git a/docs_v1.0/M4_workspace/2026-05-14_package_084324_response.md b/docs_v1.0/M4_workspace/2026-05-14_package_084324_response.md new file mode 100644 index 0000000..c2da88d --- /dev/null +++ b/docs_v1.0/M4_workspace/2026-05-14_package_084324_response.md @@ -0,0 +1,38 @@ +# 2026-05-14_package_084324_response.md + +## 部署結果 + +Package `v1.0.0_20260514_084324` (Charade HD `aeed71342a899fe4b4c57b7d41bcb692`) 已部署至 3002 production。 + +| Table | Count | Status | +|-------|------:|:--:| +| videos | 1 | ✅ | +| chunk | 2,407 | ✅ | +| chunk_vectors | 2,407 | ✅ | +| face_detections | 70,691 | ✅ | +| identities | 391 | ✅ | +| identity_bindings | 12,600 | ✅ | +| tkg_nodes | 4,535 | ✅ | +| tkg_edges | 15,588 | ✅ | + +## 遇到的問題 + +### 1. PG_BIN 路徑 +deploy.sh 預設 `/Users/accusys/pgsql/18.3/bin` 不存在。 +M4 環境使用 `/opt/homebrew/opt/postgresql@18/bin`。 +→ 執行時需 `PG_BIN=/opt/homebrew/opt/postgresql@18/bin SCHEMA=public bash deploy.sh --force` + +### 2. Global TMDB identities 未清理 +pre-clean 只刪 `WHERE file_uuid = UUID`,但 global TMDB identities(file_uuid IS NULL)留在 DB 中,重新 COPY 時撞 PRIMARY KEY。 +→ M5 已在 deploy.sh 加入方案 A 修復。 + +### 3. identities_name_key constraint +M4 手動刪除後匯入成功。 + +## 3602 目前狀態 + +- total_videos: 38 +- completed: 4 +- pending: 34 +- total_chunks: 15,367 +- Build: `1f7daf9` diff --git a/docs_v1.0/M4_workspace/2026-05-14_pipeline_services_checklist_response.md b/docs_v1.0/M4_workspace/2026-05-14_pipeline_services_checklist_response.md new file mode 100644 index 0000000..ee15a73 --- /dev/null +++ b/docs_v1.0/M4_workspace/2026-05-14_pipeline_services_checklist_response.md @@ -0,0 +1,44 @@ +# 2026-05-14_pipeline_services_checklist_response.md + +## 服務狀態 + +### 必要服務 + +| 服務 | Port | 狀態 | 備註 | +|------|:---:|:--:|------| +| PostgreSQL | 5432 | ✅ | accepting connections | +| Redis | 6379 | ✅ | authenticated | +| Qdrant | 6333 | ✅ | v1.17.0 | +| **EmbeddingGemma** | **11436** | ✅ | MPS, device=mps | +| **llama.cpp (Gemma4)** | **8082** | ✅ | M5: `192.168.110.201:8082`, model=google_gemma-4-26B-A4B-it-Q5_K_M | +| MongoDB | 27017 | ✅ | ping=1 | +| ffmpeg | — | ✅ | v8.1.1 | + +### 可選服務 + +| 服務 | Port | 狀態 | 備註 | +|------|:---:|:--:|------| +| GDINO | 8080 | ❓ | 未檢查 | +| Ollama | 11434 | ❌ | 未運行(embedding 由 11436 替代) | + +### 3002 Health + +```json +{ + "status": "ok", + "version": "1.0.0", + "build_git_hash": "1f7daf9", + "services": { + "postgres": "ok", + "redis": "ok", + "qdrant": "ok", + "mongodb": "ok" + } +} +``` + +### 備註 + +- `/health/detailed` 目前只回報 postgres/redis/qdrant/mongodb,尚無 `pipeline` 區塊(scripts/models/ffmpeg/embedding_server/llm 檢查未實作) +- 3002 與 3003 共用同樣的基礎服務(PG/Redis/Qdrant/MongoDB) +- M5 上 Gemma4 LLM (8082) 確認在線,3002 `.env` 已正確指向 diff --git a/docs_v1.0/M4_workspace/2026-05-14_register_api_404_response.md b/docs_v1.0/M4_workspace/2026-05-14_register_api_404_response.md index 0b386c2..a44080c 100644 --- a/docs_v1.0/M4_workspace/2026-05-14_register_api_404_response.md +++ b/docs_v1.0/M4_workspace/2026-05-14_register_api_404_response.md @@ -1,30 +1,29 @@ -# Register API 404 — 回覆 +# 2026-05-14_register_api_404_response.md -**Date**: 2026-05-14 -**From**: M5 -**To**: M4 -**Ref**: `2026-05-14_register_api_404.md` +## Resolution ---- +正確端點為 `POST /api/v1/files/register`(非 `/api/v1/register`)。 -## 根因 +## Results -`POST /api/v1/register` 不是有效端點。 +| 檔案 | 狀態 | +|------|:--:| +| diagram-0.svg, diagram-1.svg, diagram-2.svg | ✅ Registered | +| animal10.jpg, animal11.jpg | ✅ Registered | +| Screenshot 2026-04-01...png, Screenshot 2026-05-04...png | ✅ Registered | -正確端點是 `POST /api/v1/files/register`,已存在於所有版本中。 +## 3002 Status -## 修正後 Request +| 指標 | 值 | +|------|-----| +| Build | `fc1d775` | +| Total files | 33 | +| .mp4 | 21 | +| .mov | 2 | +| .jpg | 4 | +| .png | 3 | +| .svg | 3 | -```bash -curl -X POST http://localhost:3002/api/v1/files/register \ - -H "X-API-Key: ..." \ - -H "Content-Type: application/json" \ - -d '{ - "file_path": "/path/to/file.svg", - "content_hash": "optional-sha256-hex" - }' -``` +## MarkBase Tree -## 回呼相容 - -`POST /api/v1/register` 不需要也**不應該**新增。請 M4 改用正確端點。 +33 files, all previews working (svg/jpg/png/mp4/mov). diff --git a/docs_v1.0/REFERENCE/FILE_CONVERSION_STRATEGY_V1.0.md b/docs_v1.0/M5_workspace/2026-05-15_file_conversion_strategy.md similarity index 95% rename from docs_v1.0/REFERENCE/FILE_CONVERSION_STRATEGY_V1.0.md rename to docs_v1.0/M5_workspace/2026-05-15_file_conversion_strategy.md index 5e33ed1..4260a24 100644 --- a/docs_v1.0/REFERENCE/FILE_CONVERSION_STRATEGY_V1.0.md +++ b/docs_v1.0/M5_workspace/2026-05-15_file_conversion_strategy.md @@ -1,5 +1,7 @@ --- -title: "File Conversion Strategy" +document_type: "log_doc" +service: "MOMENTRY_CORE" +title: "File Conversion Strategy — Research & Decision Record" version: "V1.0" date: "2026-05-15" author: "M5" @@ -174,4 +176,5 @@ def extract_pptx(path): | Version | Date | Changes | |---------|------|---------| -| V1.0 | 2026-05-15 | Initial document — conversion tool survey, licensing analysis, implementation phases | +| V1.0 | 2026-05-15 | Initial — conversion tool survey, licensing analysis, implementation phases | +| V1.1 | 2026-05-15 | Reclassified from REFERENCE to M5_workspace as decision log | diff --git a/docs_v1.0/REFERENCE/MARKBASE_DESIGN_V2.0.md b/docs_v1.0/REFERENCE/MARKBASE_DESIGN_V2.0.md index 15015b0..1a8cd07 100644 --- a/docs_v1.0/REFERENCE/MARKBASE_DESIGN_V2.0.md +++ b/docs_v1.0/REFERENCE/MARKBASE_DESIGN_V2.0.md @@ -657,7 +657,81 @@ file_uuid **在遷移過程中不變**。檔案從 Hot 移到 Cold: file_uuid 永遠指向 birth 時的 `physical_path_at_birth`(Hot 路徑),不因遷移而改變。 -### 6.5 location_history 表 +### 6.5 AI Agent — 按需資料流動 + +AI Agent 在底層自動管理資料流動,使用者無需知道檔案實際存放層級。 + +#### 架構 + +``` +User / Scheduler + │ + ▼ +┌─────────────────────────────────┐ +│ AI Agent │ +│ • Monitor tier usage │ +│ • Detect hot/cold patterns │ +│ • Trigger auto-archive │ +│ • Restore on access (prefetch) │ +└──────────┬──────────────────────┘ + │ + ▼ +┌─────────────────────────────────┐ +│ Transfer Engine │ +│ Direct (std::fs::copy) │ +│ Rsync (delta + checksum) │ +│ S3 / SFS / NFS / CDN │ +└──────────┬──────────────────────┘ + │ + ▼ +┌─────────────────────────────────┐ +│ file_locations │ +│ (single source of truth) │ +│ M2 M4 M5 Cloud LTO │ +└─────────────────────────────────┘ +``` + +#### 自動歸檔規則 + +| 觸發條件 | 動作 | Transfer Engine | +|----------|------|:--:| +| `idle_days > 90` | move to Warm | Rsync + checksum verify | +| `idle_days > 365` | move to Cold | Tar + checksum verify | +| `hot_tier_usage > 80%` | move oldest to Warm | Rsync —progress | +| user accesses cold file | restore to Hot | Rsync prefetch | + +#### 流程範例 + +``` +1. AI Agent 偵測 Charade_1963.mp4 閒置 120 天 +2. rsync -avP --checksum → /Volumes/NAS_Archive/ +3. POST /api/v2/files/aeed7134.../locations + {"location": "/Volumes/NAS_Archive/Charade_1963.mp4", + "label": "M4-warm"} +4. 移除 Hot tier 位置(或保留為參考) +5. 使用者查詢檔案資訊 → 看到所有層級,無需知道實際位置 +``` + +#### 設計原則 + +| 原則 | 說明 | +|------|------| +| 透明遷移 | 使用者查詢 `file_locations` 始終得到一致視圖 | +| 不變標識 | `file_uuid` 在遷移過程中不變 | +| 位置追蹤 | 每次遷移後更新 `file_locations`,舊位置可選擇保留為歷史參考 | +| 驗證完整性 | 遷移後執行 SHA256 校驗(Rsync `--checksum` 或手動比對) | +| 類似記憶體階層 | Agent 是記憶體控制器:Hot=快取、Warm=主記憶體、Cold=磁碟 | + +``` + +用戶查詢檔案 → 始終看到一致視圖(單一來源真相:file_locations) + ↑ +Transfer Engine(rsync / Direct / S3 / SFS / CDN) + ↑ +AI Agent(監控 tier 用量、偵測冷熱模式、自動歸檔、預取) + ↑ +Storage Tiers(M2 Hot → M4 Warm → M5 Cold → LTO) +``` ```sql CREATE TABLE IF NOT EXISTS location_history ( @@ -908,6 +982,8 @@ CREATE TABLE IF NOT EXISTS exit_records ( | 13 | 2026-05-14 | notify crate (僅 Hot tier) | 減少資源消耗,Warm/Cold 變更頻率低 | | 14 | 2026-05-14 | zip + tar crate (不用外部 CLI) | 跨平台,不需 ditto/hdiutil | | 15 | 2026-05-14 | Momentry Core 整合 A+B 混合模式 | 輕量運算用 crate,重查詢用 HTTP API | +| 16 | 2026-05-14 | AI Agent 按需資料流動 | 透明遷移、類似記憶體階層、自動冷熱管理 | +| 17 | 2026-05-14 | file_locations 支援任意 URI | /path、s3://、sfs://、ipfs://、https://、\\SMB\path | --- @@ -916,4 +992,4 @@ CREATE TABLE IF NOT EXISTS exit_records ( | 版本 | 日期 | 目的 | 操作人 | 工具/模型 | |------|------|------|--------|-----------| | V1.0 | 2026-05-12 | 初版設計(Demo Display + Knowledge Graph) | M4 / OpenCode | DeepSeek V4 Pro | -| V2.0 | 2026-05-14 | 虛擬檔案樹、Group Share、儲存層級、技術棧、file_uuid、檔案操作 API | M4 / OpenCode | DeepSeek V4 Pro | +| V2.0 | 2026-05-14 | 虛擬檔案樹、Group Share、儲存層級、技術棧、file_uuid、檔案操作 API、AI Agent 按需資料流動、跨平台 multi-location | M4 / OpenCode | DeepSeek V4 Pro | diff --git a/docs_v1.0/DESIGN/FILE_LIFECYCLE_V1.0.md b/docs_v1.0/STANDARDS/FILE_LIFECYCLE_V1.0.md similarity index 98% rename from docs_v1.0/DESIGN/FILE_LIFECYCLE_V1.0.md rename to docs_v1.0/STANDARDS/FILE_LIFECYCLE_V1.0.md index 0f5f263..60bb442 100644 --- a/docs_v1.0/DESIGN/FILE_LIFECYCLE_V1.0.md +++ b/docs_v1.0/STANDARDS/FILE_LIFECYCLE_V1.0.md @@ -1,5 +1,5 @@ --- -document_type: "design_doc" +document_type: "standard_doc" service: "MOMENTRY_CORE" title: "File Lifecycle — Pre-Processing & Registration" version: "V1.0" @@ -193,3 +193,4 @@ registration_time = NOW() | Version | Date | Changes | |---------|------|---------| | V1.0 | 2026-05-15 | Initial design — birth certificate (pre-process) + civil registration two-phase flow | +| V1.1 | 2026-05-15 | Reclassified from DESIGN to STANDARDS as design standard |