admin/momentry_core
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: implement skin_tone_trace node builder and standardize TKG node naming

Browse Source 
- Add build_skin_tone_trace_nodes() to tkg.rs (Fitzpatrick I-VI classification)
- Add skin_tone_trace_nodes field to TkgResult
- Standardize node naming: _trace -> _track (text uses _region)
- Add external_id format column to Node Types table
- Add storage names to Edge Types table
- Create TKG_FORMATION_V1.0.md with Phase 0-4 definition, flow diagram, queries
- Add cross-reference from identity_agent_v4.0.md to TKG Formation
- Update Python scripts to executable mode
This commit is contained in:
Accusys
2026-06-25 03:09:16 +08:00
parent 406b2d5524
commit 4273576612
8 changed files with 680 additions and 72 deletions
Download Patch File Download Diff File Expand all files Collapse all files

83
docs_v1.0/API_WORKSPACE/modules/15_tkg.md
View File

@@ -6,19 +6,23 @@
TKG is a time-aligned knowledge graph built from multi-processor outputs (face, yolo, ocr, pose, asrx, gaze, lip, appearance). It produces 9 node types and 14 edge types stored in `dev.tkg_nodes` and `dev.tkg_edges`.
**Node naming convention:** All trace types use `_track` suffix. Text uses `_region` (non-temporal).
**See also:** `docs_v1.0/DESIGN/TKG_FORMATION_V1.0.md` for formation phases, flow diagrams, and query examples.
### Node Types
| Node Type | Description | Key Properties |
|-----------|-------------|----------------|
| `face_track` | A tracked face identity over time | `trace_id`, `frame_count`, `status`, `pending_identity_name`, `confidence`, `identity_uuid` (see Identity Agent section below) |
| `gaze_track` | Gaze direction over time | `direction` (frontal/left/right/up/down + diagonals) |
| `lip_track` | Lip movement synced with speech | `speaker_id`, `lip_area_range` |
| `text_region` | Spoken text aligned to time | `speaker_id`, `text`, `start_time`, `end_time` |
| `appearance_trace` | Human appearance (clothing) over time | `clothing_color`, `upper_cloth`, `lower_cloth` |
| `skin_tone_trace` | Fitzpatrick skin tone classification | `fitzpatrick_type` (I–VI) |
| `accessory` | Detected accessories | `type` (glasses/hat/etc.), `confidence` |
| `object` | YOLO-detected object | `class`, `confidence`, `frame_count` |
| `speaker` | ASRX speaker segment | `speaker_id`, `segment_count`, `total_duration` |
| Node Type | External ID Format | Description | Key Properties |
|-----------|-------------------|-------------|----------------|
| `face_track` | `face_track_{trace_id}` | A tracked face identity over time | `trace_id`, `frame_count`, `status`, `pending_identity_name`, `confidence`, `identity_uuid` |
| `gaze_track` | `gaze_track_{id}` | Gaze direction over time | `direction` (frontal/left/right/up/down + diagonals) |
| `lip_track` | `lip_track_{id}` | Lip movement synced with speech | `speaker_id`, `lip_area_range` |
| `text_region` | `text_region_{id}` | Spoken text aligned to time | `speaker_id`, `text`, `start_time`, `end_time` |
| `appearance_trace` | `appearance_{trace_id}` | Human appearance (clothing) over time | `clothing_color`, `upper_cloth`, `lower_cloth` |
| `skin_tone_trace` | `skin_tone_{trace_id}` | Fitzpatrick skin tone classification | `fitzpatrick_type` (I–VI) |
| `accessory` | `accessory_{id}` | Detected accessories | `type` (glasses/hat/etc.), `confidence` |
| `object` | `object_{class}_{id}` | YOLO-detected object | `class`, `confidence`, `frame_count` |
| `speaker` | `speaker_{speaker_id}` | ASRX speaker segment | `speaker_id`, `segment_count`, `total_duration` |
---
@@ -69,15 +73,16 @@ Identity Agent marks face_track nodes with identity binding status.
### Edge Types
| Edge Type | Source → Target | Description |
|-----------|-----------------|-------------|
| `co_occurs` | object ↔ object | Two objects appear together in same frame |
| `speaker_face` | speaker ↔ face_trace | Speaker matched to face trace via lip sync |
| `face_face` | face_trace ↔ face_trace | Two face traces interact (mutual gaze) |
| `mutual_gaze` | gaze_trace ↔ gaze_trace | Two people looking at each other |
| `lip_sync` | lip_trace ↔ text_trace | Lip movement aligned with spoken text |
| `has_appearance` | face_trace ↔ appearance_trace | Face has specific appearance |
| `wears` | face_trace ↔ accessory | Face wears an accessory |
| Edge Type | Storage Name | Source → Target | Description |
|-----------|--------------|-----------------|-------------|
| `co_occurs` | `CO_OCCURS_WITH` | object ↔ object | Two objects appear together in same frame |
| `speaker_face` | `SPEAKS_AS` | speaker → face_track | Speaker matched to face track via lip sync |
| `face_face` | `INTERACTS_WITH` | face_track ↔ face_track | Two face tracks interact (mutual gaze) |
| `mutual_gaze` | `MUTUAL_GAZE` | gaze_track ↔ gaze_track | Two people looking at each other |
| `lip_sync` | `LIP_SYNC` | lip_track → text_region | Lip movement aligned with spoken text |
| `has_appearance` | `HAS_APPEARANCE` | face_track → appearance_trace | Face has specific appearance |
| `wears` | `WEARS` | face_track → accessory | Face wears an accessory |
| `hand_object` | `HOLDS` | hand → object | Hand holding object |
---
@@ -102,10 +107,10 @@ curl -s -X POST "$API/api/v1/file/$FILE_UUID/tkg/rebuild" \
"success": true,
"file_uuid": "d3f9ae8e471a1fc4d47022c66091b920",
"result": {
"face_trace_nodes": 16,
"gaze_trace_nodes": 16,
"lip_trace_nodes": 12,
"text_trace_nodes": 24,
"face_track_nodes": 16,
"gaze_track_nodes": 16,
"lip_track_nodes": 12,
"text_region_nodes": 24,
"appearance_trace_nodes": 8,
"skin_tone_trace_nodes": 5,
"accessory_nodes": 3,
@@ -143,18 +148,18 @@ Query TKG nodes with pagination and optional type filter.
| Field | Type | Required | Default | Description |
|-------|------|----------|---------|-------------|
| `node_type` | string | No | all | Filter by node type: `face_trace`, `gaze_trace`, `lip_trace`, `text_trace`, `appearance_trace`, `skin_tone_trace`, `accessory`, `object`, `speaker` |
| `node_type` | string | No | all | Filter by node type: `face_track`, `gaze_track`, `lip_track`, `text_region`, `appearance_trace`, `skin_tone_trace`, `accessory`, `object`, `speaker` |
| `page` | integer | No | 1 | Page number |
| `page_size` | integer | No | 100 | Items per page (max 500) |
#### Example
```bash
# Get all face_trace nodes
# Get all face_track nodes
curl -s -X POST "$API/api/v1/file/$FILE_UUID/tkg/nodes" \
-H "X-API-Key: $KEY" \
-H "Content-Type: application/json" \
-d '{"node_type": "face_trace", "page": 1, "page_size": 50}'
-d '{"node_type": "face_track", "page": 1, "page_size": 50}'
# Get all nodes
curl -s -X POST "$API/api/v1/file/$FILE_UUID/tkg/nodes" \
@@ -175,12 +180,12 @@ curl -s -X POST "$API/api/v1/file/$FILE_UUID/tkg/nodes" \
"nodes": [
{
"id": 1,
"node_type": "face_trace",
"external_id": "trace_0",
"label": "Face Trace 0",
"node_type": "face_track",
"external_id": "face_track_0",
"label": "Face Track 0",
"properties": {
"trace_id": 0,
"face_count": 142,
"frame_count": 142,
"avg_confidence": 0.87
}
}
@@ -224,17 +229,17 @@ Query TKG edges with pagination and optional filters.
#### Example
```bash
# Get all co_occurrence edges
# Get all co_occurs edges
curl -s -X POST "$API/api/v1/file/$FILE_UUID/tkg/edges" \
-H "X-API-Key: $KEY" \
-H "Content-Type: application/json" \
-d '{"edge_type": "co_occurs"}'
# Get edges between face_trace and speaker nodes
# Get edges between face_track and speaker nodes
curl -s -X POST "$API/api/v1/file/$FILE_UUID/tkg/edges" \
-H "X-API-Key: $KEY" \
-H "Content-Type: application/json" \
-d '{"source_type": "speaker", "target_type": "face_trace"}'
-d '{"source_type": "speaker", "target_type": "face_track"}'
```
#### Response (200)
@@ -298,12 +303,12 @@ curl -s "$API/api/v1/file/$FILE_UUID/tkg/node/1" \
"success": true,
"node": {
"id": 1,
"node_type": "face_trace",
"external_id": "trace_0",
"label": "Face Trace 0",
"node_type": "face_track",
"external_id": "face_track_0",
"label": "Face Track 0",
"properties": {
"trace_id": 0,
"face_count": 142,
"frame_count": 142,
"avg_confidence": 0.87
}
},
@@ -422,4 +427,4 @@ curl -s "$API/api/v1/file/$FILE_UUID/processor-counts" \
---
*Updated: 2026-06-20 12:00:00*
*Updated: 2026-06-25 17:00:00*