Commit Graph

55 Commits

Author SHA1 Message Date
Accusys
e1572907ae feat: ASRX hybrid pipeline, identity history, worker fixes, checkpoint system 2026-06-02 07:13:23 +08:00
Accusys
874d688987 feat: deploy hybrid search (semantic+keyword+identity) with RRF fusion
- Replace smart_search with hybrid RRF implementation
- Add speaker_detections table for identity-agent binding
- Fix identity queries: direct SQL to avoid type mismatches
- Add debug logs to job_worker for processor debugging
- Deployed to production (3002) successfully

Key changes:
- search.rs: Complete rewrite with 3 strategies + RRF
- postgres_db.rs: speaker_detections table + identity query fixes
- job_worker.rs: Debug logs for output file checks

Tested:
- Hybrid search works with semantic + keyword + identity
- Identity search: 'identity:Charade' returns correct results
- Chinese keyword search: '調光' matches Charade summaries

Bugs found:
- Case mismatch: 'ASRX' vs 'asrx' in processors field
- Missing CUT dependency for ASRX processor
2026-06-01 15:15:17 +08:00
Accusys
0d58a738a1 feat: add processor state machine and alert mechanism
- Add ProcessorJobStatus enum (8 states: Idle/Waiting/Ready/Pending/Running/Completed/Failed/Skipped)
- Add processor_alerts table (migrations/034)
- Add emit_processor_alert() to redis_client.rs
- Add ConditionResult enum + check_dependencies() to job_worker.rs
2026-05-30 10:03:49 +08:00
Accusys
de88fd4e44 fix: restore accidentally deleted type definitions
Add back PipelineType enum, ProcessorType::pipeline() method, and
OLLAMA_URL/EMBED_URL/LLM_HEALTH_URL config constants — all of
which were deleted in commits 78923a89 and 0856b92e while the
referencing code was left intact, causing 5 compilation errors.
2026-05-25 08:50:53 +08:00
Accusys
d7f89a962b fix: frame_number is BIGINT in DB, use i64 not i32
frame_number column in face_detections table is defined as BIGINT (INT8).
Using i32 caused sqlx type mismatch at runtime. Fixed in:
- identity_agent_api.rs: query_as tuples and HashMap key
- qdrant_db.rs: upsert_face_embedding signature and row extraction
2026-05-25 04:07:30 +08:00
Accusys
a2b71fef0d fix: i64→i32 for INT4 cols (identity_binding, identity_agent, qdrant_db) 2026-05-25 03:18:50 +08:00
M5Max128
78923a8973 fix: system consistency - store_vector, search, worker trigger
- store_vector: stub -> actual PG embedding storage
- search_parent_chunks_semantic: include sentence chunks
- Remove early return in check_and_complete_job
2026-05-24 23:20:02 +08:00
M5Max128
f8bcc0356c feat: frame/time pipeline split + output validation
- Add PipelineType enum + pipeline() to ProcessorType
- Split ProcessorPool into frame_slots (max 2) and time_slots (max 1)
- Add can_start_for() for pipeline-aware scheduling
- Add validate_output_file() — checks JSON validity before marking complete
- Add 3 unit tests for validate_output_file()
- Create DESIGN/FRAME_TIME_PIPELINE_V1.0.md (492 lines)
2026-05-23 21:14:28 +08:00
M5Max128
dddb5d4cbd refactor: centralize port config + fix 8082 conflict
- Add EMBED_URL, OLLAMA_URL, LLM_HEALTH_URL to config.rs
- Fix health.rs hardcoded ports → config references
- Fix sync_db.rs Ollama URL → config::OLLAMA_URL
- Create config/port_registry.tsv (single source of truth for ports)
- Remove Caddy 8082 proxy block (port belongs to LLM)
- Fix .env LLM_URL: localhost → 127.0.0.1 (avoid IPv6 Caddy conflict)
2026-05-23 02:54:34 +08:00
M5Max128
701e71463d feat: identity PATCH update, alias system, name UNIQUE removal
- Add PATCH /api/v1/identity/:identity_uuid endpoint
- Migration 030: remove name UNIQUE, add tmdb_id index
- TMDb upsert: ON CONFLICT (name) -> ON CONFLICT (tmdb_id)
- get_or_create_identity: pre-check by name
- upload_identity: ON CONFLICT (name) -> ON CONFLICT (uuid)
- Search: include aliases in identity text search
- Add scripts/llm_metadata_enhancer.py
- Add DESIGN/IdentityUpdateAndAliasSystem.md
2026-05-22 08:35:32 +08:00
Accusys
bebaa743ed feat: trace-level matching, health watcher/worker status, timezone config 2026-05-21 01:08:30 +08:00
Accusys
1ea23a6d51 fix: identity detail 502 - IdentityDetailRecord.id i32->i64 type mismatch panic
- identities.id is BIGINT (8 bytes), Rust struct was i32 (4 bytes)
- sqlx type mismatch caused panic, crashing backend process
- Proxy returned 502 due to empty reply from crashed backend
- Phase 5: 17/23 passed (was 16/23)
2026-05-19 18:33:21 +08:00
Accusys
77098b88ba feat: Phase 2-5 API test scripts + create_monitor_job fix
Phase 2: 10/10 passed 
Phase 3: 7/7 passed 
Phase 4: 9/11 passed (2 known bugs - i.name column)
Phase 5: 13/23 passed (10 failures - pre-existing bugs)

Fixes:
- create_monitor_job: ON CONFLICT (uuid) DO UPDATE to prevent duplicate key errors
- test scripts: Correct request bodies for all visual search endpoints
2026-05-19 16:05:46 +08:00
Accusys
ea6ea02925 fix: delete_video - add file existence check + fix pre_chunks UUID cast
- unregister: check file exists before delete, return 200 with success:false if not found
- delete_video: cast pre_chunks.file_uuid parameter as UUID (::uuid)
- Added Phase 2 test script (10/10 endpoints passed)
2026-05-19 15:51:25 +08:00
Accusys
611441662f fix: register_resource - use ON CONFLICT (resource_id) DO UPDATE instead of RETURNING id
- resources table uses resource_id as PK (no auto-increment id column)
- Make register idempotent: duplicate registration updates status + heartbeat
- Added Phase 1 API test script (15 endpoints, 100% pass)
2026-05-19 14:22:40 +08:00
Accusys
67ca846ccd feat: ASR output frame numbers + rename start/end to start_time/end_time
- Python: asr_processor.py detects FPS from CUT/ffprobe (no fallback), outputs start_frame/end_frame
- Rust: All AsrSegment structs use start_time/end_time with #[serde(alias)] for backward compat
- store_asr_chunks: prefers ASR output frames, falls back to time-based conversion
- Added backward compatibility test for old JSON format (start/end)

Breaking change: ffprobe/CUT FPS failure now aborts instead of using default 24fps
2026-05-19 13:22:38 +08:00
Accusys
7b6da4f0d8 fix: identities API - use real_name instead of name for cross-schema compatibility 2026-05-19 10:21:49 +08:00
Accusys
362c63007c feat: smart search response includes start_frame/end_frame/fps, add limit param 2026-05-18 01:21:43 +08:00
Accusys
088aefdac7 fix: pipeline timeline log, chunk lookup, face processor no fallback, Qdrant UUID script, delete safety rules 2026-05-18 00:36:14 +08:00
Accusys
a880c80556 fix: face_detections INSERT in pipeline, add dependency graph doc 2026-05-17 22:16:20 +08:00
Accusys
3164a65554 update: pipeline, search, clip, embedding fixes 2026-05-17 19:46:35 +08:00
Accusys
5317cb4bec feat: schema tracking, SHA256 integrity, identity UUID fix, 3-angle face match, cuts table, trace stranger_id 2026-05-16 03:10:50 +08:00
Accusys
c41f7e0c6e feat: schema version tracking, SHA256 integrity, setup scripts, bug fixes 2026-05-15 18:06:36 +08:00
Accusys
0e73d2a2ce test: add unified probe unit tests (8 Rust + 6 Python), fix pre-existing test compilation errors 2026-05-15 14:58:44 +08:00
Accusys
43cf702d05 feat: add 'unregistered' status — all incomplete files migrated to unregistered 2026-05-15 13:17:31 +08:00
Accusys
e86aebccee feat: register INSERT now uses status='registered' + registration_time=NOW() 2026-05-15 12:46:42 +08:00
Accusys
37799fff4e fix: add identity_uuid to /identities list + /file/:uuid/identities responses 2026-05-15 10:14:22 +08:00
Accusys
fdcec82274 fix: file/identities — replace NULL first/last_appearance with actual start_frame/end_frame + start_time/end_time + fps 2026-05-15 10:07:35 +08:00
Accusys
301a95e2bc refactor: remove all dev.* and public.* schema hardcodes from runtime code
14 files updated to use schema::table_name() instead of hardcoded schema
prefixes. Only src/bin/release.rs intentionally retains dev.* references.
2026-05-14 14:40:14 +08:00
Accusys
e6aa45d7ea fix: /files total count from DB (was hardcoded 0) 2026-05-13 20:45:23 +08:00
Accusys
2e7dd44552 fix: scan extensions add jpg/png, /files status from DB (2/4 M4 items) 2026-05-13 20:43:37 +08:00
Accusys
ffc30d7377 M4 handover: coordinate fixes, detector registry, deploy v2, YOLOv8s, identity lifecycle
- Fix swift_pose/swift_ocr Y-flip bugs (BUG-003~006)
- Add heuristic_scene module + post-processing trigger (replaces Places365)
- YOLOv5nu → YOLOv8s CoreML (+33% detections, +390% scene indicators)
- Per-table SQL export (split 4.7GB single file → 478MB max per table)
- Version/build check in deploy.sh (compare /health vs file_info.json)
- Add file_uuid column to identities table + backfill
- Identity pre-clean step in deploy (avoids UNIQUE conflicts on re-deploy)
- Stranger_xxx naming fix with UUID context
- Add DETECTOR_REGISTRY.md (25 detectors), DETECTOR_SELECTION_SOP.md
- Update SPATIAL_COORDINATE_REGISTRY.md (P layer, 6-layer architecture)
- New IDENTITY_LIFECYCLE.md
- M4 response docs for deploy_script_fix and 111614 test report
2026-05-13 20:00:47 +08:00
Accusys
cac60c6093 fix: M4 Phase 1 bugs - dev.chunks refs, search_path, uuid column
Bug fixes from M4 report:
- 4 remaining dev.chunks → dev.chunk in SQL queries
- search_path includes public for pgvector extension
- get_chunk_by_chunk_id_and_uuid: uuid → file_uuid
- New endpoint: GET /api/v1/file/:uuid/chunk/:chunk_id
2026-05-11 10:21:06 +08:00
Accusys
39ba5ddf76 feat: Phase 1 handover - schema migration, correction mechanism, API fixes
Schema changes: dev.chunks->dev.chunk, remove old_chunk_id/chunk_index
Correction: asr-1.json format, generate/apply scripts
API: 37/37 endpoints fixed and tested
Docs: HANDOVER_V2.0.md for M4
2026-05-11 07:03:22 +08:00
Accusys
d8714aa46e Fix semantic search: query chunks instead of empty parent_chunks table 2026-05-08 01:29:10 +08:00
Accusys
573714788f Release v1.0.0 candidate 2026-05-08 00:48:15 +08:00
Warren
74b6182eba feat: media API (video/bbox/thumbnail), UUID unification, dot matrix text, portal fixes, API dictionary V1.3 2026-05-06 13:34:49 +08:00
Warren
e75c4d6f07 cleanup: remove dead code and duplicate docs
- Remove session-ses_2f27.md (161KB raw session log)
- Remove 49 ROOT_* duplicate files across REFERENCE/
- Remove 14 duplicate files between REFERENCE/ root and history/
- Remove asr_legacy.rs (dead code, replaced by asr.rs)
- Remove src/core/worker/ (duplicate JobWorker)
- Remove src/core/layers/ (empty directory)
- Remove 4 .bak files in src/
- Remove 7 dead private methods in worker/processor.rs
- Remove backup directory from git tracking
2026-05-04 01:31:21 +08:00
Warren
ee81e343ce chore: remove obsolete APIs (register, probe, n8n, videos, people)
- Remove /api/v1/register (replaced by /api/v1/files/register)
- Remove /api/v1/probe (replaced by /api/v1/files/:uuid)
- Remove /api/v1/n8n/... (n8n workflow only)
- Remove /api/v1/unregister (high risk)
- Remove /api/v1/videos list (replaced by /api/v1/files)
- Remove /api/v1/people (merged into /api/v1/identities)
- Clean up dead code and unused structs
2026-04-30 22:16:24 +08:00
Warren
2b23d1cfbd feat: update core API, database layer, and worker modules
- Remove unused imports (n8n_search, universal_search, Client, Arc, etc.)
- Update API endpoints for identity, face recognition, search
- Fix postgres_db.rs search_videos parent_uuid column
- Add snapshot API and identity agent API
- Clean up backup files (.bak, .bak2)
2026-04-30 15:07:02 +08:00
Warren
c15f7cd4af feat: implement Phase 5 Resource Registry & Heartbeat 2026-04-25 23:12:15 +08:00
Warren
4686c5abc4 feat: complete Phase 4 Candidate Workflow (Confirm/Reject API) 2026-04-25 22:27:31 +08:00
Warren
e84982e7d9 feat: Phase 3 API (Identity, Files, Candidates) and pre_chunks migration 2026-04-25 22:19:12 +08:00
Warren
1f84e5469f feat: backup architecture docs, source code, and scripts 2026-04-25 17:15:45 +08:00
Warren
59809dae1f chore: backup before migration to new repo 2026-04-23 16:46:02 +08:00
Warren
37d2b66c56 feat: add PostgreSQL schema isolation for playground environment
- Create schema.rs utility module with table_name() function
- Add schema prefix to all SQL queries in postgres_db.rs
- Support dev schema for playground, public for production
- Add DATABASE_SCHEMA, MONGODB_DATABASE, QDRANT_COLLECTION config
- Fix 40+ functions including videos, chunks, frames, vectors, etc.
- Update Cargo dependencies
2026-03-31 10:30:33 +08:00
Warren
2393d81a3f feat: fix Chinese text search and duplicate chunk_id bug
- Add helper functions to extract text from nested content structure
- Update SearchResult to include uuid field
- Add PostgreSQL function get_chunk_by_chunk_id_and_uuid to handle duplicate chunk_ids
- Update Qdrant search functions to extract uuid from payload
- Change embedding model to nomic-embed-text-v2-moe:latest
- Update Qdrant collection name to momentry_rule1
- Fix MongoDB authentication and disable cache for development
- Improve error handling in processor.rs
- Update documentation with new embedding model
2026-03-29 04:44:28 +08:00
Warren
82955504f3 feat: 新增 Job Worker 系統與 API 文檔全面更新 2026-03-26 16:16:34 +08:00
Warren
e4d3365ada feat: add cache toggle, unregister API and chunk training docs
- Add POST /api/v1/config/cache for cache toggle
- Add POST /api/v1/unregister for video deletion
- Add CHUNK_DATA_STRUCTURE.md for marcom training
- Fix processor_results query in delete_video
2026-03-25 15:44:47 +08:00
accusys
383201cacd feat: Initial v0.9 release with API Key authentication
## v0.9.20260325_144654

### Features
- API Key Authentication System
- Job Worker System
- V2 Backup Versioning

### Bug Fixes
- get_processor_results_by_job column mapping

Co-authored-by: OpenCode
2026-03-25 14:53:41 +08:00