Documents the journey from Rust pipeline snowball bug through
5 iterations of pgvector-based matching to the final 11-identity
centroid approach with dual-gate and ambiguity cleanup.
wiki is not traditional RAG:
- RAG: ephemeral query-time augmentation
- Wiki: permanent model corrections, versioned, packaged with model
- Edits accumulate across versions as ground truth
Each momentry model includes a wiki/ directory for user-contributed knowledge:
- identity labels, object labels, ASR corrections
- Edits feed back into next model version
- TKG edges enriched by wiki data
Pipeline = training → produces momentry model per video
Core = inference engine → serves APIs from model
Phase 1 = tiny model (sentence chunks)
Phase 2 = full model (complete + 5W1H)
- New src/verification/ module: verify_output() checks JSON structure/completeness per processor type
- Worker: after processor succeeds, verification agent gates the result
- Passed -> mark completed + cleanup_temp_files (remove .tmp/.partial/.err/timestamp backups)
- Failed -> mark failed with verification details, preserve files for inspection
- cleanup_temp_files() keeps only the canonical {uuid}.{proc}.json
- New trace_ingest module: creates chunks for each face trace (time + bbox + ASR text)
- Computes pairwise time overlaps between traces -> co_appearances in metadata
- Worker auto-triggers after face trace store + Qdrant sync
- SearchFilters: chunk_type filter (sentence/cut/trace/visual)
- SearchFilters: co_appears_with_trace_id filter
- Add helper functions to extract text from nested content structure
- Update SearchResult to include uuid field
- Add PostgreSQL function get_chunk_by_chunk_id_and_uuid to handle duplicate chunk_ids
- Update Qdrant search functions to extract uuid from payload
- Change embedding model to nomic-embed-text-v2-moe:latest
- Update Qdrant collection name to momentry_rule1
- Fix MongoDB authentication and disable cache for development
- Improve error handling in processor.rs
- Update documentation with new embedding model
- Add POST /api/v1/config/cache for cache toggle
- Add POST /api/v1/unregister for video deletion
- Add CHUNK_DATA_STRUCTURE.md for marcom training
- Fix processor_results query in delete_video
- Add BACKUP_VERSIONING.md with comprehensive backup management
- Update backup_all.sh with v2 version marking
- v2 naming format: {service}_{type}_v2_{date}_{time}.{ext}
- Fixed MongoDB backup path to /opt/homebrew/var/mongodb
- Added momentry_output backup for probe.json files
- Added restore functions for v2 format
- Update backup_monitor.sh for v2 filename parsing
- Updated VIDEO_REGISTRATION.md with Probe API documentation
- Updated JOB_WORKER_IMPLEMENTATION_PLAN.md status to implemented
- Updated MOMENTRY_CORE_MONITORING.md with Job Worker monitoring
- Updated SERVICES.md with Momentry Playground and Job Worker info
- Add ProbeRequest/ProbeResponse structures
- Support relative and absolute paths
- Cache probe.json for repeated requests
- Return video metadata (uuid, duration, width, height, fps)
- Include cached flag to indicate cache hit
- Export FormatInfo and StreamInfo from probe module
- Update API_ENDPOINTS.md documentation
- Add separate momentry_playground binary with distinct configuration
- Production (momentry): Port 3002, Redis prefix 'momentry:'
- Development (momentry_playground): Port 3003, Redis prefix 'momentry_dev:'
- Add SERVER_PORT and REDIS_KEY_PREFIX config via environment variables
- Replace all hardcoded Redis key prefixes with configurable values
- Create .env.development for playground environment settings
- Update .env with production defaults
- Add dotenv dependency for environment file loading
Configuration isolation allows running both binaries simultaneously
without port conflicts or Redis key collisions.
- Add version summary table to SERVICES.md
- Add version quick reference to PYTHON.md
- Add version quick reference to NODEJS.md
- Update related documents references
- Architecture overview for mixed runtime environments
- Python version management with venv
- Node.js version locking for n8n
- Isolation principles for coexisting environments
- Workflow integration patterns
- Monitoring configuration
- Troubleshooting guide
- New service decision tree
