feat: backup architecture docs, source code, and scripts
This commit is contained in:
Warren
231
docs_v1.0/AI_AGENTS/SUMMARIZATION/CHUNK_RULE_4_SUMMARY.md
Normal file
231
docs_v1.0/AI_AGENTS/SUMMARIZATION/CHUNK_RULE_4_SUMMARY.md
Normal file
@@ -0,0 +1,231 @@
|
||||
---
|
||||
document_type: "architecture_design"
|
||||
service: "MOMENTRY_CORE"
|
||||
title: "Momentry Core Chunk Rule 4: 摘要分析級檢索 (Summary 5W1H Chunk) (v1.0)"
|
||||
date: "2026-04-21"
|
||||
version: "V1.0"
|
||||
status: "active"
|
||||
owner: "Warren"
|
||||
created_by: "OpenCode"
|
||||
tags:
|
||||
- "momentry"
|
||||
- "core"
|
||||
- "摘要分析級檢索"
|
||||
- "rule"
|
||||
ai_query_hints:
|
||||
- "查詢 Momentry Core Chunk Rule 4: 摘要分析級檢索 (Summary 5W1H Chunk) (v1.0) 的內容"
|
||||
- "Momentry Core Chunk Rule 4: 摘要分析級檢索 (Summary 5W1H Chunk) (v1.0) 的主要目的是什麼？"
|
||||
- "如何操作或實施 Momentry Core Chunk Rule 4: 摘要分析級檢索 (Summary 5W1H Chunk) (v1.0)？"
|
||||
---
|
||||
|
||||
# Momentry Core Chunk Rule 4: 摘要分析級檢索 (Summary 5W1H Chunk) (v1.0)
|
||||
|
||||
| 項目 | 內容 |
|
||||
|------|------|
|
||||
| 建立者 | OpenCode |
|
||||
| 建立時間 | 2026-04-21 |
|
||||
| 文件版本 | V1.0 |
|
||||
|
||||
---
|
||||
|
||||
## 版本歷史
|
||||
|
||||
| 版本 | 日期 | 目的 | 操作人 | 工具/模型 |
|
||||
|------|------|------|--------|-----------|
|
||||
| V1.0 | 2026-04-21 | 定義 Rule 4: 基於 LLM 5W1H 分析的最高層級摘要結構 | OpenCode | OpenCode / Qwen3.6-Plus |
|
||||
|
||||
---
|
||||
|
||||
## 0. 設計目標
|
||||
|
||||
**Rule 4** 的核心概念是**「情節理解」(Storyline Understanding)**。透過將多個場景 (Rule 3) 聚合，並利用大型語言模型 (Gemma4) 進行深度分析，提取 5W1H 結構化資訊，使系統能夠回答複雜的「情節相關問題」。
|
||||
|
||||
- **核心原則**: 5-10 個場景 (Rule 3) = 1 個摘要區塊 (Summary Chunk)。
|
||||
- **結構**: 頂層 Parent Chunk。
|
||||
- **特徵**: 包含 LLM 生成的完整摘要與 **5W1H** (Who, What, When, Where, Why, How) 分析結果。
|
||||
- **優勢**: 支援宏觀劇情檢索、人物動線追蹤與複雜問答 (RAG)。
|
||||
|
||||
---
|
||||
|
||||
## 1. 數據源與聚合邏輯
|
||||
|
||||
Rule 4 是處理管線的終點，依賴 **Rule 3** 的產出以及 **LLM 服務**。
|
||||
|
||||
1. **Rule 3 Chunks (Primary)**: 提供場景級的文本摘要與元數據。
|
||||
- *聚合策略*: 將連續的 5-10 個 Rule 3 Chunks 視為一個「敘事區塊」。
|
||||
2. **LLM Processor (Gemma4)**:
|
||||
- *任務*: 讀取該區塊內所有 Rule 3 的摘要與 ASR 文本。
|
||||
- *輸出*:
|
||||
- **Summary**: 流暢的劇情描述。
|
||||
- **5W1H**: 結構化的關鍵要素提取。
|
||||
3. **Visual/Audio Retention**:
|
||||
- 保留區塊內所有出現過的 `face_ids` (Who) 和 `objects` (What/Where)。
|
||||
|
||||
---
|
||||
|
||||
## 2. Chunk 結構定義
|
||||
|
||||
### 2.1 資料庫結構 (PostgreSQL)
|
||||
|
||||
```sql
|
||||
CREATE TABLE chunks_rule4 (
|
||||
id UUID PRIMARY KEY,
|
||||
asset_uuid UUID NOT NULL,
|
||||
chunk_type VARCHAR(20) DEFAULT 'summary',
|
||||
|
||||
-- 時間軸 (繼承自第一個與最後一個 Rule 3 子區塊)
|
||||
start_frame INT NOT NULL,
|
||||
end_frame INT NOT NULL,
|
||||
start_time_sec DOUBLE PRECISION,
|
||||
end_time_sec DOUBLE PRECISION,
|
||||
|
||||
-- LLM 生成內容
|
||||
summary TEXT NOT NULL, -- 劇情摘要
|
||||
analysis_5w1h JSONB, -- 結構化分析結果
|
||||
|
||||
-- 聚合元數據
|
||||
faces JSONB, -- 區塊內所有人物
|
||||
objects JSONB, -- 區塊內重要物件
|
||||
|
||||
-- 向量索引
|
||||
embedding vector(768), -- 摘要與 5W1H 的混合向量
|
||||
created_at TIMESTAMPTZ DEFAULT NOW()
|
||||
);
|
||||
|
||||
-- 關聯子區塊
|
||||
ALTER TABLE parent_chunks ADD COLUMN rule4_parent_id UUID REFERENCES chunks_rule4(id);
|
||||
```
|
||||
|
||||
### 2.2 5W1H 結構 (JSONB)
|
||||
|
||||
```json
|
||||
{
|
||||
"who": ["Cary Grant", "Audrey Hepburn"], // 主要人物 (對應 Face ID)
|
||||
"what": ["Searching for the stamps", "Car chase"], // 核心事件
|
||||
"where": ["Paris", "Bank", "Car"], // 地點/場景 (對應 Visual Objects)
|
||||
"when": "Night", // 時間背景 (對應 Time of day)
|
||||
"why": "To pay off a debt", // 動機
|
||||
"how": "By sneaking into the vault" // 手段/過程
|
||||
}
|
||||
```
|
||||
|
||||
### 2.3 JSON 產出範例
|
||||
|
||||
```json
|
||||
{
|
||||
"chunk_id": "550e...0004",
|
||||
"type": "summary",
|
||||
"summary": "Peter 和 Regina 計劃潛入銀行金庫尋找郵票。他們在夜間開車前往，途中遭遇巡邏隊盤查，但最終利用機智脫身。",
|
||||
"start_frame": 5000,
|
||||
"end_frame": 8000,
|
||||
"analysis_5w1h": {
|
||||
"who": ["peter_joshua", "regina_lampert"],
|
||||
"what": ["heist_planning", "evasion"],
|
||||
"where": ["car", "street", "bank_exterior"],
|
||||
"when": "night",
|
||||
"why": "retrieve_stamps",
|
||||
"how": "stealth_deception"
|
||||
},
|
||||
"metadata": {
|
||||
"rule3_count": 7
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 3. 搜尋能力定義
|
||||
|
||||
Rule 4 是 **RAG (Retrieval-Augmented Generation)** 的核心數據源。
|
||||
|
||||
### 3.1 劇情摘要搜尋 (Plot Search)
|
||||
* **場景**: "這部片在講什麼？"、"他們找到郵票了嗎？"
|
||||
* **邏輯**:
|
||||
1. 搜尋 `summary` 向量。
|
||||
2. 返回包含該情節的完整摘要區塊。
|
||||
|
||||
### 3.2 5W1H 結構化查詢 (Structured Query)
|
||||
* **場景**: "找出所有 **Cary Grant (Who)** 在 **車上 (Where)** 的片段"。
|
||||
* **邏輯**:
|
||||
1. 過濾 `analysis_5w1h` JSONB 欄位。
|
||||
2. `who` 包含 "Cary Grant" **AND** `where` 包含 "car"。
|
||||
3. 這種查詢比傳統關鍵字搜索更精準，因為它是經過 LLM 理解後的結構化數據。
|
||||
|
||||
### 3.3 動機與原因搜尋 (Why/How)
|
||||
* **場景**: "他為什麼要偷東西？"
|
||||
* **邏輯**:
|
||||
1. 針對 `analysis_5w1h.why` 進行語意比對。
|
||||
|
||||
---
|
||||
|
||||
## 4. 處理流程 (LLM Pipeline)
|
||||
|
||||
Rule 4 的生成需要呼叫 `llm_engine` (Gemma4) 服務。
|
||||
|
||||
### 4.1 演算法邏輯 (Pseudocode)
|
||||
|
||||
```python
|
||||
# 輸入: rule3_chunks (List of Scene Chunks)
|
||||
|
||||
# 1. 分組 (每 5-10 個場景一組)
|
||||
for group in chunks(rule3_chunks, size=7):
|
||||
|
||||
# 2. 準備 LLM 上下文
|
||||
context_text = "\n".join([chunk.summary for chunk in group])
|
||||
context_objects = aggregate_objects(group)
|
||||
|
||||
prompt = f"""
|
||||
Analyze the following video scenes and extract the 5W1H information.
|
||||
Scenes:
|
||||
{context_text}
|
||||
|
||||
Return JSON format:
|
||||
{{
|
||||
"summary": "A brief summary of these scenes.",
|
||||
"5w1h": {{
|
||||
"who": ["List of characters"],
|
||||
"what": ["Main events"],
|
||||
...
|
||||
}}
|
||||
}}
|
||||
"""
|
||||
|
||||
# 3. 呼叫 LLM (Gemma4 via Service Registry)
|
||||
response = llm_service.chat(prompt)
|
||||
result = parse_json(response)
|
||||
|
||||
# 4. 建立 Rule 4 Chunk
|
||||
rule4_chunk = {
|
||||
"summary": result["summary"],
|
||||
"analysis_5w1h": result["5w1h"],
|
||||
"start_frame": group[0].start_frame,
|
||||
"end_frame": group[-1].end_frame,
|
||||
"faces": aggregate_faces(group),
|
||||
"objects": aggregate_objects(group)
|
||||
}
|
||||
|
||||
# 5. 儲存並關聯
|
||||
rule4_id = store_rule4_chunk(rule4_chunk)
|
||||
for chunk in group:
|
||||
link_rule3_to_rule4(chunk.id, rule4_id)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 5. 總結
|
||||
|
||||
Rule 4 將 Momentry 從「影片搜尋引擎」提升為**「影片知識圖譜」**。
|
||||
|
||||
| 特性 | 實作方式 |
|
||||
|------|----------|
|
||||
| **粒度** | 情節/敘事區塊 (5-10 場景) |
|
||||
| **核心技術** | LLM 5W1H 提取 (Gemma4) |
|
||||
| **數據結構** | 摘要文本 + JSONB 5W1H 結構 |
|
||||
| **向量內容** | 混合向量 (Summary + 5W1H) |
|
||||
| **適用場景** | 問答系統 (RAG)、劇情回顧、複雜條件過濾 |
|
||||
|
||||
**四層架構總覽：**
|
||||
1. **Rule 1 (Sentence)**: 精確台詞檢索。
|
||||
2. **Rule 2 (Visual)**: 畫面物件檢索。
|
||||
3. **Rule 3 (Scene)**: 場景上下文檢索。
|
||||
4. **Rule 4 (Summary)**: 劇情理解與知識問答。
|
||||
Reference in New Issue
Block a user