docs: add configurable Redis key prefix to Redis Keys and Monitoring docs
- Add new section 2 in MOMENTRY_CORE_REDIS_KEYS.md explaining configurable prefix - Update environment variables to include MOMENTRY_REDIS_PREFIX - Update Rust implementation examples to use REDIS_KEY_PREFIX static - Add warning note in MOMENTRY_CORE_MONITORING.md about configurable prefix - Update shell scripts to use REDIS_PREFIX environment variable - Document default prefixes: momentry: (production), momentry_dev: (development)
This commit is contained in:
accusys
283
docs/MOMENTRY_CORE_REDIS_KEYS.md
Normal file
283
docs/MOMENTRY_CORE_REDIS_KEYS.md
Normal file
@@ -0,0 +1,283 @@
|
||||
# Momentry Core Redis Key 設計規範
|
||||
|
||||
| 項目 | 內容 |
|
||||
|------|------|
|
||||
| 建立者 | Warren |
|
||||
| 建立時間 | 2026-03-17 |
|
||||
| 文件版本 | V1.0 |
|
||||
|
||||
---
|
||||
|
||||
## 版本歷史
|
||||
|
||||
| 版本 | 日期 | 目的 | 操作人 | 工具/模型 |
|
||||
|------|------|------|--------|-----------|
|
||||
| V1.0 | 2026-03-17 | 創建文件 | Warren | OpenCode / MiniMax M2.5 |
|
||||
| V1.1 | 2026-03-25 | 新增可配置 Redis Key Prefix | Warren | OpenCode / GLM-5 |
|
||||
|
||||
---
|
||||
|
||||
## 1. 概述
|
||||
|
||||
本文檔說明 momentry_core 如何使用 Redis 作為監控和狀態管理系統。
|
||||
|
||||
## 2. 可配置 Redis Key Prefix
|
||||
|
||||
### 2.1 環境變數
|
||||
|
||||
從 V1.1 開始，所有 Redis Keys 都支援自定義前綴：
|
||||
|
||||
```bash
|
||||
MOMENTRY_REDIS_PREFIX=momentry:
|
||||
```
|
||||
|
||||
此設定允許多個 momentry 實例共用同一個 Redis 伺服器，例如：
|
||||
- **生產環境**: `MOMENTRY_REDIS_PREFIX=momentry:`
|
||||
- **開發環境**: `MOMENTRY_REDIS_PREFIX=momentry_dev:`
|
||||
|
||||
### 2.2 Key 格式
|
||||
|
||||
所有 Key 都遵循以下格式：
|
||||
|
||||
```
|
||||
{prefix}{key_type}:{uuid}
|
||||
```
|
||||
|
||||
範例 (生產環境):
|
||||
```
|
||||
momentry:job:5dea6618a606e7c7
|
||||
momentry:jobs:active
|
||||
momentry:health:current
|
||||
```
|
||||
|
||||
範例 (開發環境):
|
||||
```
|
||||
momentry_dev:job:5dea6618a606e7c7
|
||||
momentry_dev:jobs:active
|
||||
momentry_dev:health:current
|
||||
```
|
||||
|
||||
### 2.3 預設值
|
||||
|
||||
| Binary | 預設 Port | 預設 Redis Prefix |
|
||||
|--------|-----------|-------------------|
|
||||
| `momentry` (生產) | 3002 | `momentry:` |
|
||||
| `momentry_playground` (開發) | 3003 | `momentry_dev:` |
|
||||
|
||||
## 3. UUID 使用時機
|
||||
|
||||
### 3.1 全局 Keys（無 UUID）
|
||||
|
||||
- 單一實例全局狀態
|
||||
- 聚合統計數據
|
||||
|
||||
### 3.2 Per-Video Keys（UUID 必要）
|
||||
|
||||
- 每個視頻獨立處理狀態
|
||||
- 即時進度追蹤
|
||||
|
||||
### 3.3 Per-Processor Keys（UUID + Processor 必要）
|
||||
|
||||
- 每個 processor 獨立狀態
|
||||
|
||||
## 4. Key 命名空間
|
||||
|
||||
```
|
||||
momentry
|
||||
├── health: # 健康檢查
|
||||
│ ├── current # 當前狀態 (TTL: 60s)
|
||||
│ └── services # 依賴服務狀態
|
||||
├── config: # 配置
|
||||
├── stats: # 聚合統計
|
||||
│ ├── total_jobs # 總 Jobs 數
|
||||
│ ├── processed_today # 今日處理數
|
||||
│ ├── cpu:current # 當前 CPU 使用
|
||||
│ └── memory:current # 當前 Memory 使用
|
||||
├── jobs: # Jobs 管理
|
||||
│ ├── active # Set: 運行中 UUIDs
|
||||
│ ├── completed # Set: 完成 UUIDs
|
||||
│ └── failed # Set: 失敗 UUIDs
|
||||
├── job:{uuid} # Per-Video Job 狀態 (TTL: 24h)
|
||||
│ ├── status # 狀態 String
|
||||
│ ├── video_path # 視頻路徑
|
||||
│ ├── current_processor # 當前 processor
|
||||
│ ├── progress_total # 總進度
|
||||
│ ├── progress_current # 當前進度
|
||||
│ ├── started_at # 開始時間
|
||||
│ ├── updated_at # 最後更新
|
||||
│ └── processor:{name} # Per-Processor 狀態
|
||||
│ ├── status
|
||||
│ ├── progress
|
||||
│ ├── current
|
||||
│ ├── total
|
||||
│ └── started_at
|
||||
├── progress:{uuid} # Pub/Sub 頻道 (即時進度)
|
||||
├── result:{uuid} # 處理結果 Hash
|
||||
├── output:{uuid} # 輸出路徑
|
||||
├── metrics:{uuid} # Per-Video 指標
|
||||
│ ├── cpu # CPU 歷史 List (100條, TTL: 1h)
|
||||
│ ├── memory # Memory 歷史 List (100條, TTL: 1h)
|
||||
│ └── duration # 處理時長
|
||||
└── log:{uuid} # 處理日誌 String
|
||||
```
|
||||
|
||||
## 5. Key 詳細說明
|
||||
|
||||
### 全局 Keys
|
||||
|
||||
| Key | Type | TTL | 說明 |
|
||||
|-----|------|-----|------|
|
||||
| `momentry:health:current` | String | 60s | 當前健康狀態 |
|
||||
| `momentry:health:services` | Hash | 60s | 依賴服務健康狀態 |
|
||||
| `momentry:stats:total_jobs` | String | - | 總 Jobs 數 |
|
||||
| `momentry:stats:processed_today` | String | 86400s | 今日處理數 |
|
||||
| `momentry:stats:cpu:current` | String | 10s | 當前 CPU 使用 |
|
||||
| `momentry:stats:memory:current` | String | 10s | 當前 Memory 使用 |
|
||||
| `momentry:jobs:active` | Set | - | 運行中 Job UUIDs |
|
||||
| `momentry:jobs:completed` | Set | - | 完成 Job UUIDs |
|
||||
| `momentry:jobs:failed` | Set | - | 失敗 Job UUIDs |
|
||||
|
||||
### Per-Video Keys
|
||||
|
||||
| Key | Type | TTL | 說明 |
|
||||
|-----|------|-----|------|
|
||||
| `momentry:job:{uuid}` | Hash | 24h | Job 完整狀態 |
|
||||
| `momentry:job:{uuid}:status` | String | 24h | Job 狀態 |
|
||||
| `momentry:progress:{uuid}` | Pub/Sub | - | 即時進度頻道 |
|
||||
| `momentry:result:{uuid}` | Hash | 24h | 處理結果 |
|
||||
| `momentry:output:{uuid}` | String | 24h | 輸出路徑 |
|
||||
| `momentry:metrics:{uuid}:cpu` | List | 1h | CPU 歷史 (100條) |
|
||||
| `momentry:metrics:{uuid}:memory` | List | 1h | Memory 歷史 (100條) |
|
||||
| `momentry:metrics:{uuid}:duration` | String | 24h | 處理時長 |
|
||||
| `momentry:log:{uuid}` | String | 24h | 處理日誌 |
|
||||
|
||||
### Per-Processor Keys
|
||||
|
||||
| Key | Type | TTL | 說明 |
|
||||
|-----|------|-----|------|
|
||||
| `momentry:job:{uuid}:processor:{name}` | Hash | 24h | Processor 狀態 |
|
||||
| `momentry:job:{uuid}:processor:{name}:status` | String | 24h | 狀態 |
|
||||
| `momentry:job:{uuid}:processor:{name}:progress` | String | 24h | 進度百分比 |
|
||||
| `momentry:job:{uuid}:processor:{name}:current` | String | 24h | 當前項目 |
|
||||
| `momentry:job:{uuid}:processor:{name}:total` | String | 24h | 總項目數 |
|
||||
| `momentry:job:{uuid}:processor:{name}:started_at` | String | 24h | 開始時間 |
|
||||
|
||||
## 6. TTL 策略
|
||||
|
||||
| Key 類型 | TTL | 原因 |
|
||||
|----------|-----|------|
|
||||
| Health | 60s | 需要定期更新 |
|
||||
| Job | 24h | 處理完成後保留一天 |
|
||||
| Processor | 24h | 處理完成後保留一天 |
|
||||
| Metrics | 1h | 只保留近期歷史 |
|
||||
| Progress Pub/Sub | - | 不持久，僅即時訊息 |
|
||||
| Stats | 無 | 持久統計 |
|
||||
|
||||
## 7. 訊息格式
|
||||
|
||||
### Pub/Sub 訊息 (progress:{uuid})
|
||||
|
||||
```json
|
||||
{
|
||||
"type": "info | progress | complete | error",
|
||||
"processor": "yolo | ocr | face | pose | cut | asr | asrx",
|
||||
"timestamp": 1700000000,
|
||||
"data": {
|
||||
"message": "Processing frame 5000",
|
||||
"current": 5000,
|
||||
"total": 14315
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Job 狀態 Hash
|
||||
|
||||
```json
|
||||
{
|
||||
"uuid": "5dea6618a606e7c7",
|
||||
"video_path": "/path/to/video.mp4",
|
||||
"status": "running",
|
||||
"current_processor": "yolo",
|
||||
"progress_total": 70,
|
||||
"progress_current": 50,
|
||||
"started_at": 1700000000,
|
||||
"updated_at": 1700000100,
|
||||
"error_count": 0,
|
||||
"last_error": ""
|
||||
}
|
||||
```
|
||||
|
||||
### Processor 狀態 Hash
|
||||
|
||||
```json
|
||||
{
|
||||
"name": "yolo",
|
||||
"status": "running",
|
||||
"progress": 70,
|
||||
"current_frame": 10000,
|
||||
"total_frames": 14315,
|
||||
"started_at": 1700000000,
|
||||
"updated_at": 1700000100
|
||||
}
|
||||
```
|
||||
|
||||
## 8. 實作函數 (Rust)
|
||||
|
||||
所有 Redis Key 生成函數使用 `REDIS_KEY_PREFIX` 靜態變數：
|
||||
|
||||
```rust
|
||||
use crate::core::config::REDIS_KEY_PREFIX;
|
||||
|
||||
fn global_key(key: &str) -> String {
|
||||
format!("{}{}", REDIS_KEY_PREFIX, key)
|
||||
}
|
||||
|
||||
fn job_key(uuid: &str) -> String {
|
||||
format!("{}job:{}", REDIS_KEY_PREFIX, uuid)
|
||||
}
|
||||
|
||||
fn processor_key(uuid: &str, processor: &str) -> String {
|
||||
format!("{}job:{}:processor:{}", REDIS_KEY_PREFIX, uuid, processor)
|
||||
}
|
||||
|
||||
fn progress_channel(uuid: &str) -> String {
|
||||
format!("{}progress:{}", REDIS_KEY_PREFIX, uuid)
|
||||
}
|
||||
|
||||
fn metrics_key(uuid: &str, metric: &str) -> String {
|
||||
format!("{}metrics:{}:{}", REDIS_KEY_PREFIX, uuid, metric)
|
||||
}
|
||||
|
||||
fn jobs_set_key(status: &str) -> String {
|
||||
format!("{}jobs:{}", REDIS_KEY_PREFIX, status)
|
||||
}
|
||||
```
|
||||
|
||||
**注意**: `REDIS_KEY_PREFIX` 定義於 `src/core/config.rs`，由環境變數 `MOMENTRY_REDIS_PREFIX` 控制。
|
||||
|
||||
## 9. 環境變數
|
||||
|
||||
```bash
|
||||
# Redis 連接
|
||||
REDIS_URL=redis://localhost:6379
|
||||
REDIS_PASSWORD=accusys
|
||||
REDIS_DB=0
|
||||
|
||||
# Redis Key Prefix (可選，預設: momentry:)
|
||||
MOMENTRY_REDIS_PREFIX=momentry:
|
||||
|
||||
# 生產環境範例 (.env)
|
||||
MOMENTRY_SERVER_PORT=3002
|
||||
MOMENTRY_REDIS_PREFIX=momentry:
|
||||
|
||||
# 開發環境範例 (.env.development)
|
||||
MOMENTRY_SERVER_PORT=3003
|
||||
MOMENTRY_REDIS_PREFIX=momentry_dev:
|
||||
```
|
||||
|
||||
## 11. 監控腳本
|
||||
|
||||
使用 Redis 進行監控的腳本應參考：
|
||||
|
||||
- `monitor/service/momentry_redis_monitor.sh` - Redis 健康檢查
|
||||
- `monitor/service/momentry_job_monitor.sh` - Job 狀態監控
|
||||
Reference in New Issue
Block a user