admin/momentry_core
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
166 Commits 1 Branch 1 Tag
fcaaeadf06b4e5f218deeb89fc5aeeefa8e2fe25
Commit Graph

166 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Accusys
fcaaeadf06 fix: deploy.sh missing REQUIRED_FILES variable 2026-05-13 20:20:26 +08:00
Accusys
1d69a88741 fix: deploy.sh build check lenient + per-file import order (M4 feedback) 
- Accept SRV_BUILD=unknown (skip build check, only compare version)
- Per-table import with explicit FK order (nodes before edges)
 2026-05-13 20:15:44 +08:00
Accusys
3dc09cf802 docs: add M4 notification protocol to AGENTS.md 2026-05-13 20:04:44 +08:00
Accusys
78b7a10ace docs: add M4 notification protocol — standardize response format 2026-05-13 20:01:45 +08:00
Accusys
ffc30d7377 M4 handover: coordinate fixes, detector registry, deploy v2, YOLOv8s, identity lifecycle 
- Fix swift_pose/swift_ocr Y-flip bugs (BUG-003~006)
- Add heuristic_scene module + post-processing trigger (replaces Places365)
- YOLOv5nu → YOLOv8s CoreML (+33% detections, +390% scene indicators)
- Per-table SQL export (split 4.7GB single file → 478MB max per table)
- Version/build check in deploy.sh (compare /health vs file_info.json)
- Add file_uuid column to identities table + backfill
- Identity pre-clean step in deploy (avoids UNIQUE conflicts on re-deploy)
- Stranger_xxx naming fix with UUID context
- Add DETECTOR_REGISTRY.md (25 detectors), DETECTOR_SELECTION_SOP.md
- Update SPATIAL_COORDINATE_REGISTRY.md (P layer, 6-layer architecture)
- New IDENTITY_LIFECYCLE.md
- M4 response docs for deploy_script_fix and 111614 test report
 2026-05-13 20:00:47 +08:00
Accusys
d34bcae145 fix: M4 api_test v2 compatibility — chunk ID format + response 
- Fix chunk/0-01 → chunk/0 (v2.0 sequential chunk IDs)
- Identity UUID 2b0ddefe (Cary Grant) confirmed working in v2.0
- api_test.sh: 39/39 passed
- Response doc to M4_HANDOVER/ + M4_workspace/
 2026-05-13 05:00:59 +08:00
Accusys
5c1d8a67b2 docs: M4 handover V2.0 — complete package with TMDB, sqlite-vec, deploy scripts 
- Package v20260512_203344.tar.gz: 1.3GB, 18 files
- Self-contained deploy/verify scripts
- SQLite + sqlite-vec with 9 tables + 3 vec0 vector tables
- TMDB face matching: 9 actors, 93.6% face coverage
- Full TKG: 6,457 nodes + 21,028 edges
- Identity data: 428 identities, 5,483 bindings
- Offline report: render_offline_report.py
- All reports: ERP, SFTPGo, Service Inventory
 2026-05-13 04:40:30 +08:00
Accusys
c0c0e6e8ea feat: self-contained deploy/verify scripts in release package 
- Add deploy.sh: imports data.sql, copies video, copies output files, verifies
- Add verify.sh: checks file integrity + DB/offline status
- Both scripts included in tar.gz via release package command
- Package now deployable standalone without release CLI
 2026-05-13 04:35:43 +08:00
Accusys
48c3b13c37 fix: restore identity_id after face_dedup, rebuild package v20260512 
- Re-ran identity_bind.py to restore identity_id on face_detections
- Dedup cleanup had removed rows with identity_id, kept NULL rows
- 70691 face_detections now have identity_id, 428 identities
- Full package rebuild: 169MB sqlite, 1358MB tar.gz
- identities.json: 428 identities + 5483 bindings + 5483 trace maps
- TMDB matching complete: Audrey Hepburn 843 traces, Cary Grant 482
 2026-05-13 04:30:18 +08:00
Accusys
fff2af8ad1 fix: identity names now show in all trace tooltips (online + offline) 
- Online: remove IDENTITY filter gating on identity_note — always show
- Offline: fix id_names scope bug — was overwritten by top10-only dict
- Both reports now show 'identity: PERSON_xxx' for all 2000 timeline traces
- All 5483 traces have identity mapping (verified in SQLite)
 2026-05-13 03:19:26 +08:00
Accusys
8d4d29ce6e fix: offline report shows identity names in trace tooltips 
- Load trace_to_identity mapping from SQLite face_detections
- Query identity names from identities table
- Show 'identity: PERSON_xxx' in each trace bar tooltip
- Works in both full view and --identity filtered view
 2026-05-13 03:14:49 +08:00
Accusys
bbf8e64752 feat: offline report from SQLite, no PostgreSQL needed 
- Add render_offline_report.py — reads .sqlite directly
- Reports include: DB contents, TKG breakdown, density histogram,
  trace timeline, top identities, identity details card
- Supports --identity filter (like online mode)
- Add release visualize-offline <sqlite> [-i identity] [-o output]
- Works with exported .sqlite from export_sqlite.py
- Uses sqlite-vec vec0 tables for vector metadata
 2026-05-13 03:10:59 +08:00
Accusys
007fe10c2e feat: TKG completion, PG audit, SQLite backup with Qdrant voice vectors 
- Add voice_embeddings vec0 table (192D) from Qdrant to SQLite export
- Add tkg_nodes + tkg_edges tables to SQLite export
- Clean orphan TKG data (2414 nodes, 64 chunks)
- Rebuild TKG for both Charade files with speaker nodes
- Create asrx.json from chunk speaker metadata for TKG builder
- PG audit: pre_chunks 1.8GB (largest), 3 empty tables found
- Update release package to include all output files (not just JSON)
- Full backup: 9 SQLite tables + 3 vec0 vector tables
 2026-05-13 03:03:38 +08:00
Accusys
2992a0e650 feat: service inventory, ERP reports, sqlite-vec integration, visualize tool 
- Add SERVICE_INVENTORY_V1.0.0.md (25 source-verified tools, 3.7GB)
- Add ERP_SELECTION_REPORT.md (Odoo CE vs ERPNext comparison)
- Add SFTPGO_ODOO_REPLACEMENT.md (SFTPGo migration plan)
- Add SERVICE_GO_GITEA_BUILD.md (Go compiler + Gitea build report)
- Add release visualize command (face trace heatmap + identity filter)
- Add sqlite-vec integration (160MB SQLite with vec0 vector tables)
- Add export_identities.py, export_sqlite.py, render_face_heatmap.py
- Add Go, Gitea, Rust/Cargo, Swift, yt-dlp, SQLite, sqlite-vec to service CLI
- Fix package to include identities and identity_bindings in data.sql
- Update release list to show all deployed video stats
- Add V1.0.0 YAML frontmatter to all docs (DOCS_STANDARD compliant)
 2026-05-13 02:37:45 +08:00
Accusys
cac60c6093 fix: M4 Phase 1 bugs - dev.chunks refs, search_path, uuid column 
Bug fixes from M4 report:
- 4 remaining dev.chunks → dev.chunk in SQL queries
- search_path includes public for pgvector extension
- get_chunk_by_chunk_id_and_uuid: uuid → file_uuid
- New endpoint: GET /api/v1/file/:uuid/chunk/:chunk_id
 2026-05-11 10:21:06 +08:00
Accusys
39ba5ddf76 feat: Phase 1 handover - schema migration, correction mechanism, API fixes 
Schema changes: dev.chunks->dev.chunk, remove old_chunk_id/chunk_index
Correction: asr-1.json format, generate/apply scripts
API: 37/37 endpoints fixed and tested
Docs: HANDOVER_V2.0.md for M4
 2026-05-11 07:03:22 +08:00
Accusys
ef894a44ad docs: update Phase 1 report with all Qdrant collections + voice embeddings fix 
- Fixed asrx_processor_custom.py: embeddings now passed to asrx.json
- Voice embeddings (192D ECAPA-TDNN) extracted for all 1815 ASRX segments
- momentry_dev_voice Qdrant collection created (1815 vectors)
- Updated Phase 1 report with 6 collections, key decisions
 2026-05-10 01:11:42 +08:00
Accusys
d043b6adae docs: Phase 1 completion report + LLM reasoning off fix 2026-05-09 22:03:34 +08:00
Accusys
e7f311e7b8 docs: Phase 1 checklist 6 sections (含 face trace + TKG + identities) 
release_pack.py: identities 併入 phase 1（不限 phase >= 2）
 2026-05-09 18:14:22 +08:00
Accusys
6fc1d2b54d dashboard: copy button, dedup files, /api/all single call 2026-05-09 18:00:46 +08:00
Accusys
4f1e546104 feat: Momentry Dashboard web app (Flask) 
- Realtime dashboard on port 5050
- Pipeline checklist (8 stages ✅/❌)
- System health: CPU, memory, disk, GPU, 4 services
- Redis metrics: memory, clients, hit rate, keys
- DB table counts: videos, chunks, face_detections, identities, TKG
- Processor timing chart
- Auto-refresh every 15 seconds + manual refresh button

Usage: python3 scripts/dashboard.py
Open: http://localhost:5050
 2026-05-09 17:43:26 +08:00
Accusys
06caea51e7 feat: pipeline status dashboard (checklist + health + timing) 
Usage:
  python3 scripts/pipeline_status.py              # formatted table
  python3 scripts/pipeline_status.py --json        # machine-readable JSON

Shows:
- 8-stage checklist with pass/fail per stage
- System health: CPU, memory, disk, GPU, 4 services
- Processor timing from DB
- All in under 1 second
 2026-05-09 17:29:38 +08:00
Accusys
fc16e7b1c3 fix: Phase 1 pipeline fully operational 
- store_traced_faces.py: add --uuid arg for PythonExecutor compat
- tkg_builder.py: add --uuid arg + timestamp_secs column fix
- release_pack.py: fix pg_dump/psql paths, proper JSON escaping
- pipeline_checklist.py: new independent verification tool

Phase 1 checklist 8/8 PASS:
ASR ✅ ASRX ✅ sentence chunks ✅ vector embeddings ✅
face trace ✅ TKG graph ✅ trace chunks ✅ Phase 1 release ✅
 2026-05-09 17:21:17 +08:00
Accusys
3a4fd4136d docs: wiki vs RAG distinction in model lifecycle 
wiki is not traditional RAG:
- RAG: ephemeral query-time augmentation
- Wiki: permanent model corrections, versioned, packaged with model
- Edits accumulate across versions as ground truth
 2026-05-09 14:28:47 +08:00
Accusys
e2509a650c docs: wiki mechanism for model adjustability 
Each momentry model includes a wiki/ directory for user-contributed knowledge:
- identity labels, object labels, ASR corrections
- Edits feed back into next model version
- TKG edges enriched by wiki data
 2026-05-09 14:24:40 +08:00
Accusys
076af4cba1 docs: Phase 3 object identity design (v3 model) 
- Object instance tracking (similar to face trace)
- Custom detector for stamps, guns, etc.
- TKG integration for object-face-speaker graph
- Upgrade path: yolov5nu → yolov8m, fine-tune, zero-shot
 2026-05-09 14:22:37 +08:00
Accusys
19669a1f91 refactor: model naming v1(base)/v2, Qdrant collection naming 
- Phase 1 = v1 (base model, sentence chunk embedding)
- Phase 2 = v2 (full pipeline + 5W1H)
- Naming leaves room for v3, v4, etc.
- Qdrant collection: momentry_dev_v1 (active model under dev)
- Release packaging exports Qdrant points snapshot
 2026-05-09 14:14:04 +08:00
Accusys
227c647a43 docs: momentry model vs core architecture 
Pipeline = training → produces momentry model per video
Core = inference engine → serves APIs from model
Phase 1 = tiny model (sentence chunks)
Phase 2 = full model (complete + 5W1H)
 2026-05-09 14:03:00 +08:00
Accusys
28652f5b76 feat: phased release packaging (Phase 1 + Phase 2) 
- scripts/release_pack.py: packages output_json + schema + chunks + vectors
- Phase 1: triggered after ASR+ASRX+Rule 1+vectorization (sentence chunk delivery)
- Phase 2: triggered after full pipeline + 5W1H Agent (full delivery)
- Both phases include all available {uuid}.*.json files
- Non-overlapping directories: release/phase1/ and release/phase2/
 2026-05-09 13:58:55 +08:00
Accusys
7237a1811e feat: verification agent for processor output validation 
- New src/verification/ module: verify_output() checks JSON structure/completeness per processor type
- Worker: after processor succeeds, verification agent gates the result
- Passed -> mark completed + cleanup_temp_files (remove .tmp/.partial/.err/timestamp backups)
- Failed -> mark failed with verification details, preserve files for inspection
- cleanup_temp_files() keeps only the canonical {uuid}.{proc}.json
 2026-05-09 13:30:00 +08:00
Accusys
e068b70777 docs: git bundle instructions for M4 (SSH disabled) 2026-05-09 06:25:45 +08:00
Accusys
a0774cb9ab feat: wire TKG builder into worker pipeline + face-face edges 
- Auto-run tkg_builder.py after face trace store + Qdrant sync + trace chunks
- Add face-face CO_OCCURS_WITH edges (two traces in same frame)
- docs: TKG integration report for M4
 2026-05-09 06:22:27 +08:00
Accusys
b902763d45 feat: trace chunks with co-appearance relationships 
- New trace_ingest module: creates chunks for each face trace (time + bbox + ASR text)
- Computes pairwise time overlaps between traces -> co_appearances in metadata
- Worker auto-triggers after face trace store + Qdrant sync
- SearchFilters: chunk_type filter (sentence/cut/trace/visual)
- SearchFilters: co_appears_with_trace_id filter
 2026-05-09 06:18:32 +08:00
Accusys
9f5afd1b86 fix: file-based source of truth for worker + backup protocol 
- Worker: check {uuid}.{processor}.json existence before starting processor
- Worker: timestamp-copy backup existing output files before re-run (no delete, no overwrite)
- Executor: partial output saved as .json.partial (not .json) to avoid false completion
- Start script: removed set-e, log dir changed to momentry/logs, Qdrant collection status fix
- docs: M4 release incident report + M4/M5 collaboration protocol
 2026-05-09 05:32:13 +08:00
Accusys
b220920e64 Visual scene classification: Phase 1+2 complete 
- Extracted visual_stats per scene (face count, size, objects, duration, density)
- Classified 1130 scenes into 18 types (establishing/close_up/medium/long/two/group × dialogue/sparse/silent)
- All from existing data, no LLM needed
- Scene type stored in cut chunk metadata
 2026-05-08 14:38:00 +08:00
Accusys
283da8e767 Fix trace/3128: drawtext + filter_complex_script 
- Replaced bitmap font (~195K drawbox commands) with drawtext (~2.2K)
- Write filter to temp file, use -/filter_complex to bypass ARG_MAX
- Added ffmpeg stderr logging for debugging
 2026-05-08 14:03:30 +08:00
Accusys
1f103e796b Video endpoints: use ffmpeg-full for drawtext, fix ARG_MAX via filter_complex_script 
- Added FFMPEG Lazy static + ffmpeg_cmd() with DYLD_LIBRARY_PATH
- Replaced bitmap font rendering with drawtext (1 filter vs 35 per letter)
- Large traces (>1000 detections) may still fail (ARG_MAX with -vf)
 2026-05-08 13:55:08 +08:00
Warren
7d89ff77d0 docs: add purpose/use case for each visualization type 2026-05-08 13:50:10 +08:00
Warren
6234972f37 docs: add trace visualization catalog V1-V5 with priority 2026-05-08 13:49:06 +08:00
Warren
77598a4713 docs: add XY trajectory visual encoding guide with color/size/fade 2026-05-08 13:45:35 +08:00
Warren
bb8e79cbc2 docs: add X-axis + xy trajectory modes, API request format 2026-05-08 13:43:12 +08:00
Warren
a0f3382d13 docs: expanded trajectory section with interpretation guide and SQL 2026-05-08 13:40:10 +08:00
Warren
e5f252a3ec docs: heatmap as full-pipeline visual search visualization layer 2026-05-08 13:37:53 +08:00
Warren
2058599e63 docs: trace heatmap spec v1.0.0 — spatial + temporal + combined 2026-05-08 13:33:57 +08:00
Warren
f469197ce6 M4: add ffmpeg-full licensing evaluation for M5's doc 2026-05-08 13:28:16 +08:00
Warren
3ff783e4aa docs: full demo script with narration (10 steps) 2026-05-08 13:24:32 +08:00
Warren
606405b941 M4: suggest replace bitmap font with drawtext filter 2026-05-08 13:21:40 +08:00
Warren
ac59789f6e Merge branch 'main' of 192.168.110.201:/Users/accusys/momentry_core_0.1/ 2026-05-08 13:16:28 +08:00
Accusys
14d95cab8e Notify M4: video endpoints root cause 2026-05-08 13:16:05 +08:00
Accusys
485dc4010c Fix video endpoints: DB file_path did not match actual file 
Root cause: video file was renamed on disk but DB still had old path.
Old: ...Charade ... │ Comedy ...mp4
New: ...Old_Time_Movie_Show_-_Charade_1963.HD.mov

All 3 endpoints (trace/video, video/bbox, thumbnail) now return 200.
 2026-05-08 13:14:00 +08:00