feat: Phase 1 handover - schema migration, correction mechanism, API fixes
Schema changes: dev.chunks->dev.chunk, remove old_chunk_id/chunk_index Correction: asr-1.json format, generate/apply scripts API: 37/37 endpoints fixed and tested Docs: HANDOVER_V2.0.md for M4
This commit is contained in:
73
docs/GUN_DETECTOR_SCAN_REPORT.md
Normal file
73
docs/GUN_DETECTOR_SCAN_REPORT.md
Normal file
@@ -0,0 +1,73 @@
|
||||
# Gun Detector Scan Report — YOLOv8n on Charade (1963)
|
||||
|
||||
**Date:** 2026-05-10
|
||||
**Model:** `models/gun/gun_detector/weights/best.pt`
|
||||
**Base:** YOLOv8n fine-tuned on Roboflow gun dataset (905 images)
|
||||
**Classes:** grenade, knife, pistol, rifle
|
||||
**Scan script:** `scripts/gun_detector_scan.py`
|
||||
|
||||
## Scan Method
|
||||
|
||||
- **121 scan points**: 2 ASR "gun" mentions + 114 fixed intervals (60s) + 5 original hit timestamps
|
||||
- **Per point**: scan ±30 frames at every 3rd frame = ~20 frames per point
|
||||
- **Total frames processed**: ~2,420
|
||||
- **Runtime**: ~2 min
|
||||
|
||||
## Results
|
||||
|
||||
| Class | Detections | Top Confidence |
|
||||
|-------|-----------|---------------|
|
||||
| pistol | **82** | 0.887 |
|
||||
| rifle | 55 | 0.822 |
|
||||
| grenade | 35 | 0.797 |
|
||||
| knife | 38 | 0.810 |
|
||||
| **Total** | **210** (after dedup) | — |
|
||||
|
||||
## Original 5 Pistol Timestamps
|
||||
|
||||
| Timestamp | Original | This Scan | Delta |
|
||||
|-----------|----------|-----------|-------|
|
||||
| 3188s (53:08) | pistol 0.387 | ✅ **0.474** | +22% |
|
||||
| 5461s (91:01) | pistol 0.355 | ✅ **0.346** | −3% |
|
||||
| 6309s (1:45:09) | pistol 0.374 | ❌ Not found | — |
|
||||
| 6377s (1:46:17) | gun 0.316 | ✅ **0.757** | +140% |
|
||||
| 6479s (1:47:59) | pistol 0.300 | ✅ **0.815** | +172% |
|
||||
|
||||
## Top Pistol Detections
|
||||
|
||||
| Time | Confidence | Image |
|
||||
|------|-----------|-------|
|
||||
| 84:00 (5040s) | **0.887** | `5040s_pistol_0.887.jpg` |
|
||||
| 90:00 (5400s) | **0.816** | `5400s_pistol_0.816.jpg` |
|
||||
| 108:00 (6480s) | **0.815** | `6480s_pistol_0.815.jpg` |
|
||||
| 48:59 (2939s) | **0.805** | `2939s_pistol_0.805.jpg` |
|
||||
| 53:07 (3187s) | **0.474** | `3187s_pistol_0.474.jpg` |
|
||||
| 91:00 (5459s) | **0.346** | `5459s_pistol_0.346.jpg` |
|
||||
|
||||
## Analysis
|
||||
|
||||
### Model Performance
|
||||
|
||||
Compared to the original evaluation (May 7, 24 sample points, all FP):
|
||||
|
||||
- This scan found **significantly more detections** (210 vs 7)
|
||||
- Confidence values are **much higher** (0.887 vs 0.45 max)
|
||||
- 4/5 original pistol timestamps recovered
|
||||
|
||||
### Cautions
|
||||
|
||||
1. **Training data mismatch**: Model was trained on 905 close-up gun photos, NOT movie frames. High confidence ≠ real gun.
|
||||
2. **Stamp false positive confirmed**: t=5400s (identified in original eval as stamp → pistol) continues to fire at 0.816
|
||||
3. **Pattern suggests overconfidence**: Many detections at regular intervals (every 60s, same objects) suggest the model is detecting non-gun objects with high confidence
|
||||
|
||||
### Verified Findings
|
||||
|
||||
The original 5 pistol images from the gun_detections/ directory (3188s, 5461s, 6309s, 6377s, 6479s) were all produced by the same YOLOv8n model. The user previously stated that none of these have been confirmed as real guns.
|
||||
|
||||
## Files
|
||||
|
||||
| File | Description |
|
||||
|------|-------------|
|
||||
| `output_dev/gun_detections/gun_detections.json` | All 210 deduped detections |
|
||||
| `output_dev/gun_detections/*.jpg` | Annotated screenshots (one per detection) |
|
||||
| `scripts/gun_detector_scan.py` | Scan script (reproducible) |
|
||||
Reference in New Issue
Block a user