feat: wire TKG builder into worker pipeline + face-face edges
- Auto-run tkg_builder.py after face trace store + Qdrant sync + trace chunks - Add face-face CO_OCCURS_WITH edges (two traces in same frame) - docs: TKG integration report for M4
This commit is contained in:
79
docs_v1.0/M4_workspace/2026-05-09_TKG_integration.md
Normal file
79
docs_v1.0/M4_workspace/2026-05-09_TKG_integration.md
Normal file
@@ -0,0 +1,79 @@
|
||||
# TKG Integration — Pipeline Auto-Build + Trace Chunks
|
||||
|
||||
## 變更摘要
|
||||
|
||||
### 1. TKG Builder 現在是 pipeline 的一環
|
||||
|
||||
`tkg_builder.py` 不再需要手動執行。worker 在 face trace store + Qdrant sync 完成後自動呼叫:
|
||||
|
||||
```
|
||||
Face Processor ✅
|
||||
→ store_traced_faces.py (trace_id 寫入 face_detections)
|
||||
→ sync_face_embeddings() (Qdrant)
|
||||
→ ingest_traces() (產生 trace chunks)
|
||||
→ tkg_builder.py ← 自動!
|
||||
```
|
||||
|
||||
### 2. TKG Builder 新增 face↔face edges
|
||||
|
||||
原本只有 face↔object (`CO_OCCURS_WITH`) 和 face↔speaker (`SPEAKS_AS`)。
|
||||
現在加上 **face↔face** — 兩個不同 trace 出現在同一 frame 就建立 edge。
|
||||
|
||||
```
|
||||
(face_trace:2) -[:CO_OCCURS_WITH]-> (face_trace:5)
|
||||
表示 trace 2 與 trace 5 曾同時出現在畫面中
|
||||
```
|
||||
|
||||
### 3. Trace Chunks(新 chunk_type)
|
||||
|
||||
每個 face trace 對應一個 `chunk_type = 'trace'` 的 chunk:
|
||||
|
||||
| 欄位 | 內容 |
|
||||
|------|------|
|
||||
| `chunk_type` | `'trace'` |
|
||||
| `start_frame` / `end_frame` | trace 出現的時間區間 |
|
||||
| `text_content` | 該區間內的 ASR 文字 |
|
||||
| `metadata.trace_id` | face_detections 的 trace_id |
|
||||
| `metadata.bbox` | 平均位置 `{x, y, width, height}` |
|
||||
| `metadata.co_appearances` | 同框的其他 trace `[{trace_id, overlap_frames, overlap_secs}]` |
|
||||
|
||||
### 4. Search API 擴充
|
||||
|
||||
`SearchFilters` 新增:
|
||||
|
||||
| Filter | 用法 |
|
||||
|--------|------|
|
||||
| `chunk_type` | `"trace"` / `"sentence"` / `"cut"` |
|
||||
| `co_appears_with_trace_id` | 找出與指定 trace 同框的所有 chunk |
|
||||
|
||||
範例:
|
||||
```json
|
||||
POST /api/v1/search/universal
|
||||
{
|
||||
"query": "",
|
||||
"types": ["chunk"],
|
||||
"filters": {"chunk_type": "trace", "co_appears_with_trace_id": 5}
|
||||
}
|
||||
```
|
||||
|
||||
## TKG Graph 結構(完整)
|
||||
|
||||
```
|
||||
NODES:
|
||||
face_trace — 每個 trace_id
|
||||
object — 每個 YOLO class (person, car, ...)
|
||||
speaker — 每個 speaker_id (SPEAKER_0, ...)
|
||||
|
||||
EDGES:
|
||||
face_trace -[:CO_OCCURS_WITH]-> object 同 frame
|
||||
face_trace -[:CO_OCCURS_WITH]-> face_trace 同 frame(新增)
|
||||
face_trace -[:SPEAKS_AS]-> speaker 時間重疊 > 30%
|
||||
```
|
||||
|
||||
## git pull 後需執行
|
||||
|
||||
```bash
|
||||
cd /Users/accusys/momentry_core_0.1 && git pull
|
||||
```
|
||||
|
||||
已包含在 `9f5afd1` 和 `b902763`。
|
||||
Reference in New Issue
Block a user