14e886cc08
- Identity agent: per-face max matching, multi-round with derived seeds from high-confidence faces, angle diversity filter (cosine sim < 0.90) - Pending person API: POST /file/:file_uuid/pending-person + GET /file/:file_uuid/pending-persons with status=pending, source=manual - Update API docs (07_identity.md)
711 lines
38 KiB
HTML
711 lines
38 KiB
HTML
<!DOCTYPE html>
|
||
<html lang="en">
|
||
<head>
|
||
<meta charset="UTF-8">
|
||
<title>06 Search - Momentry API Docs</title>
|
||
<style>
|
||
* { margin: 0; padding: 0; box-sizing: border-box; }
|
||
body { font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, sans-serif; background: #f5f5f5; color: #333; padding: 40px; }
|
||
.container { max-width: 960px; margin: 0 auto; background: white; border-radius: 12px; box-shadow: 0 2px 12px rgba(0,0,0,0.08); padding: 40px; }
|
||
h1 { font-size: 24px; margin: 24px 0 12px; }
|
||
h2 { font-size: 20px; margin: 20px 0 10px; color: #222; }
|
||
h3 { font-size: 16px; margin: 16px 0 8px; color: #444; }
|
||
p { line-height: 1.6; margin: 8px 0; }
|
||
table { border-collapse: collapse; width: 100%; margin: 12px 0; font-size: 14px; }
|
||
th, td { border: 1px solid #ddd; padding: 8px 12px; text-align: left; }
|
||
th { background: #f0f0f0; font-weight: 600; }
|
||
code { background: #f0f0f0; padding: 2px 6px; border-radius: 3px; font-size: 13px; }
|
||
pre { background: #f8f8f8; border: 1px solid #ddd; border-radius: 6px; padding: 12px; overflow-x: auto; margin: 12px 0; }
|
||
pre code { background: none; padding: 0; }
|
||
a { color: #0066cc; }
|
||
.back { display: inline-block; margin-bottom: 20px; color: #666; }
|
||
.back:hover { color: #333; }
|
||
.topbar { display: flex; justify-content: space-between; align-items: center; margin-bottom: 20px; }
|
||
.logout-btn { font-size: 13px; color: #999; text-decoration: none; }
|
||
.logout-btn:hover { color: #cc0000; }
|
||
</style>
|
||
</head>
|
||
<body>
|
||
<div class="container">
|
||
<div class="topbar">
|
||
<a class="back" href="index.html">← Back to index</a>
|
||
<a class="logout-btn" href="#" onclick="fetch('/api/v1/auth/logout',{method:'POST'}).then(()=>window.location.reload());return false">Logout</a>
|
||
</div>
|
||
<!-- module: search -->
|
||
<!-- description: Vector search, BM25, smart search, universal search, LLM reranked search, frame search -->
|
||
<!-- depends: 01_auth -->
|
||
|
||
<h2>Search APIs</h2>
|
||
<h3><code>POST /api/v1/search/smart</code></h3>
|
||
<p><strong>Auth</strong>: Required
|
||
<strong>Scope</strong>: global / file-level</p>
|
||
<p>Semantic vector search using EmbeddingGemma-300m. Generates a query embedding via EmbeddingGemma (port 11436), then searches pgvector <code>story_parent</code> and <code>llm_parent</code> chunks by cosine similarity.</p>
|
||
<h4>Request Parameters</h4>
|
||
<table class="table">
|
||
<thead>
|
||
<tr>
|
||
<th>Field</th>
|
||
<th>Type</th>
|
||
<th>Required</th>
|
||
<th>Default</th>
|
||
<th>Description</th>
|
||
</tr>
|
||
</thead>
|
||
<tbody>
|
||
<tr>
|
||
<td><code>query</code></td>
|
||
<td>string</td>
|
||
<td>Yes</td>
|
||
<td>—</td>
|
||
<td>Search text</td>
|
||
</tr>
|
||
<tr>
|
||
<td><code>file_uuid</code></td>
|
||
<td>string</td>
|
||
<td>No</td>
|
||
<td>—</td>
|
||
<td>File UUID to search within. If omitted, searches all files (global search)</td>
|
||
</tr>
|
||
<tr>
|
||
<td><code>limit</code></td>
|
||
<td>integer</td>
|
||
<td>No</td>
|
||
<td>5</td>
|
||
<td>Max results to return</td>
|
||
</tr>
|
||
<tr>
|
||
<td><code>page</code></td>
|
||
<td>integer</td>
|
||
<td>No</td>
|
||
<td>1</td>
|
||
<td>Page number</td>
|
||
</tr>
|
||
<tr>
|
||
<td><code>page_size</code></td>
|
||
<td>integer</td>
|
||
<td>No</td>
|
||
<td>5</td>
|
||
<td>Items per page</td>
|
||
</tr>
|
||
</tbody>
|
||
</table>
|
||
<h4>Example (Global Search)</h4>
|
||
<div class="codehilite"><pre><span></span><code>curl<span class="w"> </span>-s<span class="w"> </span>-X<span class="w"> </span>POST<span class="w"> </span><span class="s2">"</span><span class="nv">$API</span><span class="s2">/api/v1/search/smart"</span><span class="w"> </span><span class="se">\</span>
|
||
<span class="w"> </span>-H<span class="w"> </span><span class="s2">"Content-Type: application/json"</span><span class="w"> </span><span class="se">\</span>
|
||
<span class="w"> </span>-H<span class="w"> </span><span class="s2">"Authorization: Bearer </span><span class="nv">$JWT</span><span class="s2">"</span><span class="w"> </span><span class="se">\</span>
|
||
<span class="w"> </span>-d<span class="w"> </span><span class="s1">'{"query": "Audrey Hepburn"}'</span>
|
||
</code></pre></div>
|
||
|
||
<h4>Example (File-specific Search)</h4>
|
||
<div class="codehilite"><pre><span></span><code>curl<span class="w"> </span>-s<span class="w"> </span>-X<span class="w"> </span>POST<span class="w"> </span><span class="s2">"</span><span class="nv">$API</span><span class="s2">/api/v1/search/smart"</span><span class="w"> </span><span class="se">\</span>
|
||
<span class="w"> </span>-H<span class="w"> </span><span class="s2">"Content-Type: application/json"</span><span class="w"> </span><span class="se">\</span>
|
||
<span class="w"> </span>-H<span class="w"> </span><span class="s2">"Authorization: Bearer </span><span class="nv">$JWT</span><span class="s2">"</span><span class="w"> </span><span class="se">\</span>
|
||
<span class="w"> </span>-d<span class="w"> </span><span class="s1">'{"file_uuid": "'</span><span class="s2">"</span><span class="nv">$FILE_UUID</span><span class="s2">"</span><span class="s1">'", "query": "Audrey Hepburn"}'</span>
|
||
</code></pre></div>
|
||
|
||
<h4>Response (200)</h4>
|
||
<div class="codehilite"><pre><span></span><code><span class="p">{</span>
|
||
<span class="w"> </span><span class="nt">"query"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Audrey Hepburn"</span><span class="p">,</span>
|
||
<span class="w"> </span><span class="nt">"results"</span><span class="p">:</span><span class="w"> </span><span class="p">[</span>
|
||
<span class="w"> </span><span class="p">{</span>
|
||
<span class="w"> </span><span class="nt">"file_uuid"</span><span class="p">:</span><span class="w"> </span><span class="s2">"a6fb22eebefaef17e62af874997c5944"</span><span class="p">,</span>
|
||
<span class="w"> </span><span class="nt">"parent_id"</span><span class="p">:</span><span class="w"> </span><span class="mi">1087822</span><span class="p">,</span>
|
||
<span class="w"> </span><span class="nt">"scene_order"</span><span class="p">:</span><span class="w"> </span><span class="mi">1087822</span><span class="p">,</span>
|
||
<span class="w"> </span><span class="nt">"start_frame"</span><span class="p">:</span><span class="w"> </span><span class="mi">104438</span><span class="p">,</span>
|
||
<span class="w"> </span><span class="nt">"end_frame"</span><span class="p">:</span><span class="w"> </span><span class="mi">104538</span><span class="p">,</span>
|
||
<span class="w"> </span><span class="nt">"fps"</span><span class="p">:</span><span class="w"> </span><span class="mf">24.0</span><span class="p">,</span>
|
||
<span class="w"> </span><span class="nt">"start_time"</span><span class="p">:</span><span class="w"> </span><span class="mf">4351.6</span><span class="p">,</span>
|
||
<span class="w"> </span><span class="nt">"end_time"</span><span class="p">:</span><span class="w"> </span><span class="mf">4355.76</span><span class="p">,</span>
|
||
<span class="w"> </span><span class="nt">"summary"</span><span class="p">:</span><span class="w"> </span><span class="s2">"[4352s-4356s, 4s] Cast: Audrey Hepburn. Total: 2 lines, 10 words. Speakers: Audrey Hepburn (2 lines)"</span><span class="p">,</span>
|
||
<span class="w"> </span><span class="nt">"similarity"</span><span class="p">:</span><span class="w"> </span><span class="mf">0.67</span>
|
||
<span class="w"> </span><span class="p">}</span>
|
||
<span class="w"> </span><span class="p">],</span>
|
||
<span class="w"> </span><span class="nt">"page"</span><span class="p">:</span><span class="w"> </span><span class="mi">1</span><span class="p">,</span>
|
||
<span class="w"> </span><span class="nt">"page_size"</span><span class="p">:</span><span class="w"> </span><span class="mi">5</span><span class="p">,</span>
|
||
<span class="w"> </span><span class="nt">"strategy"</span><span class="p">:</span><span class="w"> </span><span class="s2">"semantic_vector_search"</span>
|
||
<span class="p">}</span>
|
||
</code></pre></div>
|
||
|
||
<table class="table">
|
||
<thead>
|
||
<tr>
|
||
<th>Field</th>
|
||
<th>Type</th>
|
||
<th>Description</th>
|
||
</tr>
|
||
</thead>
|
||
<tbody>
|
||
<tr>
|
||
<td><code>results[].file_uuid</code></td>
|
||
<td>string</td>
|
||
<td>File UUID where result was found</td>
|
||
</tr>
|
||
</tbody>
|
||
</table>
|
||
<hr />
|
||
<h3><code>POST /api/v1/search/universal</code></h3>
|
||
<p><strong>Auth</strong>: Required
|
||
<strong>Scope</strong>: global / file-level</p>
|
||
<p>Multi-type BM25 full-text search across chunks, frames, and persons. Uses PostgreSQL <code>tsvector</code>.</p>
|
||
<h4>Request Parameters</h4>
|
||
<table class="table">
|
||
<thead>
|
||
<tr>
|
||
<th>Field</th>
|
||
<th>Type</th>
|
||
<th>Required</th>
|
||
<th>Default</th>
|
||
<th>Description</th>
|
||
</tr>
|
||
</thead>
|
||
<tbody>
|
||
<tr>
|
||
<td><code>query</code></td>
|
||
<td>string</td>
|
||
<td>Yes</td>
|
||
<td>—</td>
|
||
<td>Search text</td>
|
||
</tr>
|
||
<tr>
|
||
<td><code>file_uuid</code></td>
|
||
<td>string</td>
|
||
<td>No</td>
|
||
<td>—</td>
|
||
<td>Restrict to specific file. If omitted, searches all files (global search)</td>
|
||
</tr>
|
||
<tr>
|
||
<td><code>types</code></td>
|
||
<td>string[]</td>
|
||
<td>No</td>
|
||
<td><code>["chunk","frame","person"]</code></td>
|
||
<td>Search types</td>
|
||
</tr>
|
||
<tr>
|
||
<td><code>limit</code></td>
|
||
<td>integer</td>
|
||
<td>No</td>
|
||
<td>10</td>
|
||
<td>Max results per type</td>
|
||
</tr>
|
||
<tr>
|
||
<td><code>page</code></td>
|
||
<td>integer</td>
|
||
<td>No</td>
|
||
<td>1</td>
|
||
<td>Page number</td>
|
||
</tr>
|
||
<tr>
|
||
<td><code>page_size</code></td>
|
||
<td>integer</td>
|
||
<td>No</td>
|
||
<td>20</td>
|
||
<td>Items per page</td>
|
||
</tr>
|
||
</tbody>
|
||
</table>
|
||
<h4>Example (Global Search)</h4>
|
||
<div class="codehilite"><pre><span></span><code>curl<span class="w"> </span>-s<span class="w"> </span>-X<span class="w"> </span>POST<span class="w"> </span><span class="s2">"</span><span class="nv">$API</span><span class="s2">/api/v1/search/universal"</span><span class="w"> </span><span class="se">\</span>
|
||
<span class="w"> </span>-H<span class="w"> </span><span class="s2">"Content-Type: application/json"</span><span class="w"> </span><span class="se">\</span>
|
||
<span class="w"> </span>-H<span class="w"> </span><span class="s2">"Authorization: Bearer </span><span class="nv">$JWT</span><span class="s2">"</span><span class="w"> </span><span class="se">\</span>
|
||
<span class="w"> </span>-d<span class="w"> </span><span class="s1">'{"query": "Cary Grant"}'</span>
|
||
</code></pre></div>
|
||
|
||
<h4>Example (File-specific Search)</h4>
|
||
<div class="codehilite"><pre><span></span><code>curl<span class="w"> </span>-s<span class="w"> </span>-X<span class="w"> </span>POST<span class="w"> </span><span class="s2">"</span><span class="nv">$API</span><span class="s2">/api/v1/search/universal"</span><span class="w"> </span><span class="se">\</span>
|
||
<span class="w"> </span>-H<span class="w"> </span><span class="s2">"Content-Type: application/json"</span><span class="w"> </span><span class="se">\</span>
|
||
<span class="w"> </span>-H<span class="w"> </span><span class="s2">"Authorization: Bearer </span><span class="nv">$JWT</span><span class="s2">"</span><span class="w"> </span><span class="se">\</span>
|
||
<span class="w"> </span>-d<span class="w"> </span><span class="s1">'{"file_uuid": "'</span><span class="s2">"</span><span class="nv">$FILE_UUID</span><span class="s2">"</span><span class="s1">'", "query": "Cary Grant"}'</span>
|
||
</code></pre></div>
|
||
|
||
<h4>Response (200)</h4>
|
||
<div class="codehilite"><pre><span></span><code><span class="p">{</span>
|
||
<span class="w"> </span><span class="nt">"results"</span><span class="p">:</span><span class="w"> </span><span class="p">[</span>
|
||
<span class="w"> </span><span class="p">{</span>
|
||
<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"chunk"</span><span class="p">,</span>
|
||
<span class="w"> </span><span class="nt">"file_uuid"</span><span class="p">:</span><span class="w"> </span><span class="s2">"a6fb22eebefaef17e62af874997c5944"</span><span class="p">,</span>
|
||
<span class="w"> </span><span class="nt">"chunk_id"</span><span class="p">:</span><span class="w"> </span><span class="s2">"bd80fec92b0b6963d177a2c55bf713e2_2"</span><span class="p">,</span>
|
||
<span class="w"> </span><span class="nt">"chunk_type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"story_child"</span><span class="p">,</span>
|
||
<span class="w"> </span><span class="nt">"start_frame"</span><span class="p">:</span><span class="w"> </span><span class="mi">5103</span><span class="p">,</span>
|
||
<span class="w"> </span><span class="nt">"end_frame"</span><span class="p">:</span><span class="w"> </span><span class="mi">5127</span><span class="p">,</span>
|
||
<span class="w"> </span><span class="nt">"start_time"</span><span class="p">:</span><span class="w"> </span><span class="mf">212.64</span><span class="p">,</span>
|
||
<span class="w"> </span><span class="nt">"end_time"</span><span class="p">:</span><span class="w"> </span><span class="mf">213.64</span><span class="p">,</span>
|
||
<span class="w"> </span><span class="nt">"text"</span><span class="p">:</span><span class="w"> </span><span class="s2">"[213s-214s] Cary Grant: \"Olá!\""</span><span class="p">,</span>
|
||
<span class="w"> </span><span class="nt">"score"</span><span class="p">:</span><span class="w"> </span><span class="mf">0.9</span>
|
||
<span class="w"> </span><span class="p">},</span>
|
||
<span class="w"> </span><span class="p">{</span>
|
||
<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"frame"</span><span class="p">,</span>
|
||
<span class="w"> </span><span class="nt">"file_uuid"</span><span class="p">:</span><span class="w"> </span><span class="s2">"a6fb22eebefaef17e62af874997c5944"</span><span class="p">,</span>
|
||
<span class="w"> </span><span class="nt">"frame_number"</span><span class="p">:</span><span class="w"> </span><span class="mi">5105</span><span class="p">,</span>
|
||
<span class="w"> </span><span class="nt">"timestamp"</span><span class="p">:</span><span class="w"> </span><span class="mf">212.72</span><span class="p">,</span>
|
||
<span class="w"> </span><span class="nt">"score"</span><span class="p">:</span><span class="w"> </span><span class="mf">0.7</span><span class="p">,</span>
|
||
<span class="w"> </span><span class="nt">"objects"</span><span class="p">:</span><span class="w"> </span><span class="kc">null</span><span class="p">,</span>
|
||
<span class="w"> </span><span class="nt">"ocr_texts"</span><span class="p">:</span><span class="w"> </span><span class="kc">null</span><span class="p">,</span>
|
||
<span class="w"> </span><span class="nt">"faces"</span><span class="p">:</span><span class="w"> </span><span class="kc">null</span>
|
||
<span class="w"> </span><span class="p">},</span>
|
||
<span class="w"> </span><span class="p">{</span>
|
||
<span class="w"> </span><span class="nt">"type"</span><span class="p">:</span><span class="w"> </span><span class="s2">"person"</span><span class="p">,</span>
|
||
<span class="w"> </span><span class="nt">"file_uuid"</span><span class="p">:</span><span class="w"> </span><span class="s2">"a6fb22eebefaef17e62af874997c5944"</span><span class="p">,</span>
|
||
<span class="w"> </span><span class="nt">"identity_id"</span><span class="p">:</span><span class="w"> </span><span class="mi">12</span><span class="p">,</span>
|
||
<span class="w"> </span><span class="nt">"identity_uuid"</span><span class="p">:</span><span class="w"> </span><span class="s2">"a9a901056d6b46ff92da0c3c1a57dff4"</span><span class="p">,</span>
|
||
<span class="w"> </span><span class="nt">"name"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Cary Grant"</span><span class="p">,</span>
|
||
<span class="w"> </span><span class="nt">"appearance_count"</span><span class="p">:</span><span class="w"> </span><span class="mi">542</span><span class="p">,</span>
|
||
<span class="w"> </span><span class="nt">"score"</span><span class="p">:</span><span class="w"> </span><span class="mf">0.95</span>
|
||
<span class="w"> </span><span class="p">}</span>
|
||
<span class="w"> </span><span class="p">],</span>
|
||
<span class="w"> </span><span class="nt">"total"</span><span class="p">:</span><span class="w"> </span><span class="mi">20</span><span class="p">,</span>
|
||
<span class="w"> </span><span class="nt">"took_ms"</span><span class="p">:</span><span class="w"> </span><span class="mi">18</span>
|
||
<span class="p">}</span>
|
||
</code></pre></div>
|
||
|
||
<table class="table">
|
||
<thead>
|
||
<tr>
|
||
<th>Field</th>
|
||
<th>Type</th>
|
||
<th>Description</th>
|
||
</tr>
|
||
</thead>
|
||
<tbody>
|
||
<tr>
|
||
<td><code>results[].type</code></td>
|
||
<td>string</td>
|
||
<td>Result type: <code>chunk</code>, <code>frame</code>, or <code>person</code></td>
|
||
</tr>
|
||
<tr>
|
||
<td><code>results[].file_uuid</code></td>
|
||
<td>string</td>
|
||
<td>File UUID where result was found (all types)</td>
|
||
</tr>
|
||
</tbody>
|
||
</table>
|
||
<hr />
|
||
<h3><code>POST /api/v1/search/frames</code></h3>
|
||
<p><strong>Auth</strong>: Required
|
||
<strong>Scope</strong>: global / file-level</p>
|
||
<p>Search frames by YOLO objects, OCR text, face IDs, or pose detections. Filters frames based on visual content detected during processing.</p>
|
||
<h4>Request Parameters</h4>
|
||
<table class="table">
|
||
<thead>
|
||
<tr>
|
||
<th>Field</th>
|
||
<th>Type</th>
|
||
<th>Required</th>
|
||
<th>Default</th>
|
||
<th>Description</th>
|
||
</tr>
|
||
</thead>
|
||
<tbody>
|
||
<tr>
|
||
<td><code>file_uuid</code></td>
|
||
<td>string</td>
|
||
<td>No</td>
|
||
<td>—</td>
|
||
<td>Restrict to specific file</td>
|
||
</tr>
|
||
<tr>
|
||
<td><code>object_class</code></td>
|
||
<td>string</td>
|
||
<td>No</td>
|
||
<td>—</td>
|
||
<td>Filter by YOLO object class (e.g., <code>person</code>, <code>car</code>, <code>dog</code>)</td>
|
||
</tr>
|
||
<tr>
|
||
<td><code>ocr_text</code></td>
|
||
<td>string</td>
|
||
<td>No</td>
|
||
<td>—</td>
|
||
<td>Filter by OCR text content (ILIKE match)</td>
|
||
</tr>
|
||
<tr>
|
||
<td><code>face_id</code></td>
|
||
<td>string</td>
|
||
<td>No</td>
|
||
<td>—</td>
|
||
<td>Filter by face detection ID</td>
|
||
</tr>
|
||
<tr>
|
||
<td><code>time_range</code></td>
|
||
<td>[float, float]</td>
|
||
<td>No</td>
|
||
<td>—</td>
|
||
<td>Filter by time range <code>[start_secs, end_secs]</code></td>
|
||
</tr>
|
||
<tr>
|
||
<td><code>limit</code></td>
|
||
<td>integer</td>
|
||
<td>No</td>
|
||
<td>100</td>
|
||
<td>Max results</td>
|
||
</tr>
|
||
</tbody>
|
||
</table>
|
||
<h4>Example</h4>
|
||
<div class="codehilite"><pre><span></span><code><span class="c1"># Search for frames containing "person" objects</span>
|
||
curl<span class="w"> </span>-s<span class="w"> </span>-X<span class="w"> </span>POST<span class="w"> </span><span class="s2">"</span><span class="nv">$API</span><span class="s2">/api/v1/search/frames"</span><span class="w"> </span><span class="se">\</span>
|
||
<span class="w"> </span>-H<span class="w"> </span><span class="s2">"Content-Type: application/json"</span><span class="w"> </span><span class="se">\</span>
|
||
<span class="w"> </span>-H<span class="w"> </span><span class="s2">"X-API-Key: </span><span class="nv">$KEY</span><span class="s2">"</span><span class="w"> </span><span class="se">\</span>
|
||
<span class="w"> </span>-d<span class="w"> </span><span class="s1">'{"file_uuid": "'</span><span class="s2">"</span><span class="nv">$FILE_UUID</span><span class="s2">"</span><span class="s1">'", "object_class": "person", "limit": 20}'</span>
|
||
|
||
<span class="c1"># Search for frames with specific OCR text</span>
|
||
curl<span class="w"> </span>-s<span class="w"> </span>-X<span class="w"> </span>POST<span class="w"> </span><span class="s2">"</span><span class="nv">$API</span><span class="s2">/api/v1/search/frames"</span><span class="w"> </span><span class="se">\</span>
|
||
<span class="w"> </span>-H<span class="w"> </span><span class="s2">"Content-Type: application/json"</span><span class="w"> </span><span class="se">\</span>
|
||
<span class="w"> </span>-H<span class="w"> </span><span class="s2">"X-API-Key: </span><span class="nv">$KEY</span><span class="s2">"</span><span class="w"> </span><span class="se">\</span>
|
||
<span class="w"> </span>-d<span class="w"> </span><span class="s1">'{"file_uuid": "'</span><span class="s2">"</span><span class="nv">$FILE_UUID</span><span class="s2">"</span><span class="s1">'", "ocr_text": "hello", "time_range": [10.0, 30.0]}'</span>
|
||
</code></pre></div>
|
||
|
||
<h4>Response (200)</h4>
|
||
<div class="codehilite"><pre><span></span><code><span class="p">{</span>
|
||
<span class="w"> </span><span class="nt">"frames"</span><span class="p">:</span><span class="w"> </span><span class="p">[</span>
|
||
<span class="w"> </span><span class="p">{</span>
|
||
<span class="w"> </span><span class="nt">"frame_number"</span><span class="p">:</span><span class="w"> </span><span class="mi">1200</span><span class="p">,</span>
|
||
<span class="w"> </span><span class="nt">"timestamp"</span><span class="p">:</span><span class="w"> </span><span class="mf">50.0</span><span class="p">,</span>
|
||
<span class="w"> </span><span class="nt">"file_uuid"</span><span class="p">:</span><span class="w"> </span><span class="s2">"d3f9ae8e471a1fc4d47022c66091b920"</span><span class="p">,</span>
|
||
<span class="w"> </span><span class="nt">"objects"</span><span class="p">:</span><span class="w"> </span><span class="p">[{</span><span class="nt">"class"</span><span class="p">:</span><span class="w"> </span><span class="s2">"person"</span><span class="p">,</span><span class="w"> </span><span class="nt">"confidence"</span><span class="p">:</span><span class="w"> </span><span class="mf">0.95</span><span class="p">,</span><span class="w"> </span><span class="nt">"bbox"</span><span class="p">:</span><span class="w"> </span><span class="p">[</span><span class="mi">100</span><span class="p">,</span><span class="w"> </span><span class="mi">50</span><span class="p">,</span><span class="w"> </span><span class="mi">300</span><span class="p">,</span><span class="w"> </span><span class="mi">400</span><span class="p">]}],</span>
|
||
<span class="w"> </span><span class="nt">"ocr_texts"</span><span class="p">:</span><span class="w"> </span><span class="p">[</span><span class="s2">"Hello World"</span><span class="p">],</span>
|
||
<span class="w"> </span><span class="nt">"faces"</span><span class="p">:</span><span class="w"> </span><span class="p">[{</span><span class="nt">"face_id"</span><span class="p">:</span><span class="w"> </span><span class="s2">"face_42"</span><span class="p">,</span><span class="w"> </span><span class="nt">"confidence"</span><span class="p">:</span><span class="w"> </span><span class="mf">0.88</span><span class="p">}],</span>
|
||
<span class="w"> </span><span class="nt">"pose_persons"</span><span class="p">:</span><span class="w"> </span><span class="p">[{</span><span class="nt">"trace_id"</span><span class="p">:</span><span class="w"> </span><span class="mi">2</span><span class="p">,</span><span class="w"> </span><span class="nt">"bbox"</span><span class="p">:</span><span class="w"> </span><span class="p">[</span><span class="mi">120</span><span class="p">,</span><span class="w"> </span><span class="mi">60</span><span class="p">,</span><span class="w"> </span><span class="mi">280</span><span class="p">,</span><span class="w"> </span><span class="mi">380</span><span class="p">]}]</span>
|
||
<span class="w"> </span><span class="p">}</span>
|
||
<span class="w"> </span><span class="p">],</span>
|
||
<span class="w"> </span><span class="nt">"total"</span><span class="p">:</span><span class="w"> </span><span class="mi">15</span>
|
||
<span class="p">}</span>
|
||
</code></pre></div>
|
||
|
||
<table class="table">
|
||
<thead>
|
||
<tr>
|
||
<th>Field</th>
|
||
<th>Type</th>
|
||
<th>Description</th>
|
||
</tr>
|
||
</thead>
|
||
<tbody>
|
||
<tr>
|
||
<td><code>frames</code></td>
|
||
<td>array</td>
|
||
<td>Array of matching frame objects</td>
|
||
</tr>
|
||
<tr>
|
||
<td><code>frames[].frame_number</code></td>
|
||
<td>integer</td>
|
||
<td>Frame number in video</td>
|
||
</tr>
|
||
<tr>
|
||
<td><code>frames[].timestamp</code></td>
|
||
<td>float</td>
|
||
<td>Timestamp in seconds</td>
|
||
</tr>
|
||
<tr>
|
||
<td><code>frames[].file_uuid</code></td>
|
||
<td>string</td>
|
||
<td>File UUID</td>
|
||
</tr>
|
||
<tr>
|
||
<td><code>frames[].objects</code></td>
|
||
<td>array/null</td>
|
||
<td>YOLO detections in this frame</td>
|
||
</tr>
|
||
<tr>
|
||
<td><code>frames[].ocr_texts</code></td>
|
||
<td>array/null</td>
|
||
<td>OCR text strings in this frame</td>
|
||
</tr>
|
||
<tr>
|
||
<td><code>frames[].faces</code></td>
|
||
<td>array/null</td>
|
||
<td>Face detections in this frame</td>
|
||
</tr>
|
||
<tr>
|
||
<td><code>frames[].pose_persons</code></td>
|
||
<td>array/null</td>
|
||
<td>Pose-detected persons in this frame</td>
|
||
</tr>
|
||
<tr>
|
||
<td><code>total</code></td>
|
||
<td>integer</td>
|
||
<td>Total matching frame count</td>
|
||
</tr>
|
||
</tbody>
|
||
</table>
|
||
<hr />
|
||
<h3><code>POST /api/v1/search/llm-smart</code></h3>
|
||
<p><strong>Auth</strong>: Required
|
||
<strong>Scope</strong>: global / file-level</p>
|
||
<p>Smart search with LLM re-ranking. First fetches candidate results via RRF (Reciprocal Rank Fusion) using the existing smart search, then uses an LLM (Gemma4 on port 8000) to re-rank candidates by relevance to the query.</p>
|
||
<h4>Request Parameters</h4>
|
||
<table class="table">
|
||
<thead>
|
||
<tr>
|
||
<th>Field</th>
|
||
<th>Type</th>
|
||
<th>Required</th>
|
||
<th>Default</th>
|
||
<th>Description</th>
|
||
</tr>
|
||
</thead>
|
||
<tbody>
|
||
<tr>
|
||
<td><code>query</code></td>
|
||
<td>string</td>
|
||
<td>Yes</td>
|
||
<td>—</td>
|
||
<td>Search text</td>
|
||
</tr>
|
||
<tr>
|
||
<td><code>file_uuid</code></td>
|
||
<td>string</td>
|
||
<td>No</td>
|
||
<td>—</td>
|
||
<td>File UUID to search within</td>
|
||
</tr>
|
||
<tr>
|
||
<td><code>limit</code></td>
|
||
<td>integer</td>
|
||
<td>No</td>
|
||
<td>10</td>
|
||
<td>Max results to return</td>
|
||
</tr>
|
||
</tbody>
|
||
</table>
|
||
<h4>Pipeline</h4>
|
||
<div class="codehilite"><pre><span></span><code><span class="w"> </span><span class="mf">1.</span><span class="w"> </span><span class="n">smart_search</span><span class="w"> </span><span class="n">→</span><span class="w"> </span><span class="k">fetch</span><span class="w"> </span><span class="n">N</span><span class="w"> </span><span class="n">candidates</span><span class="w"> </span><span class="p">(</span><span class="k">limit</span><span class="w"> </span><span class="n">×</span><span class="w"> </span><span class="mi">3</span><span class="p">,</span><span class="w"> </span><span class="n">clamped</span><span class="w"> </span><span class="mi">10</span><span class="o">-</span><span class="mi">20</span><span class="p">)</span>
|
||
<span class="w"> </span><span class="mf">2.</span><span class="w"> </span><span class="n">LLM</span><span class="w"> </span><span class="n">rerank</span><span class="w"> </span><span class="n">→</span><span class="w"> </span><span class="n">re</span><span class="o">-</span><span class="k">order</span><span class="w"> </span><span class="k">by</span><span class="w"> </span><span class="n">relevance</span><span class="w"> </span><span class="k">using</span><span class="w"> </span><span class="n">Gemma4</span>
|
||
<span class="w"> </span><span class="mf">3.</span><span class="w"> </span><span class="n">trim</span><span class="w"> </span><span class="n">→</span><span class="w"> </span><span class="k">return</span><span class="w"> </span><span class="n">top</span><span class="w"> </span><span class="n n-Quoted">`limit`</span><span class="w"> </span><span class="n">results</span>
|
||
</code></pre></div>
|
||
|
||
<h4>Example</h4>
|
||
<div class="codehilite"><pre><span></span><code>curl<span class="w"> </span>-s<span class="w"> </span>-X<span class="w"> </span>POST<span class="w"> </span><span class="s2">"</span><span class="nv">$API</span><span class="s2">/api/v1/search/llm-smart"</span><span class="w"> </span><span class="se">\</span>
|
||
<span class="w"> </span>-H<span class="w"> </span><span class="s2">"Content-Type: application/json"</span><span class="w"> </span><span class="se">\</span>
|
||
<span class="w"> </span>-H<span class="w"> </span><span class="s2">"X-API-Key: </span><span class="nv">$KEY</span><span class="s2">"</span><span class="w"> </span><span class="se">\</span>
|
||
<span class="w"> </span>-d<span class="w"> </span><span class="s1">'{"query": "two people having a conversation about business", "limit": 5}'</span>
|
||
</code></pre></div>
|
||
|
||
<h4>Response (200)</h4>
|
||
<div class="codehilite"><pre><span></span><code><span class="p">{</span>
|
||
<span class="w"> </span><span class="nt">"query"</span><span class="p">:</span><span class="w"> </span><span class="s2">"two people having a conversation about business"</span><span class="p">,</span>
|
||
<span class="w"> </span><span class="nt">"results"</span><span class="p">:</span><span class="w"> </span><span class="p">[</span>
|
||
<span class="w"> </span><span class="p">{</span>
|
||
<span class="w"> </span><span class="nt">"file_uuid"</span><span class="p">:</span><span class="w"> </span><span class="s2">"d3f9ae8e471a1fc4d47022c66091b920"</span><span class="p">,</span>
|
||
<span class="w"> </span><span class="nt">"parent_id"</span><span class="p">:</span><span class="w"> </span><span class="mi">1234</span><span class="p">,</span>
|
||
<span class="w"> </span><span class="nt">"scene_order"</span><span class="p">:</span><span class="w"> </span><span class="mi">1234</span><span class="p">,</span>
|
||
<span class="w"> </span><span class="nt">"start_frame"</span><span class="p">:</span><span class="w"> </span><span class="mi">5000</span><span class="p">,</span>
|
||
<span class="w"> </span><span class="nt">"end_frame"</span><span class="p">:</span><span class="w"> </span><span class="mi">5200</span><span class="p">,</span>
|
||
<span class="w"> </span><span class="nt">"fps"</span><span class="p">:</span><span class="w"> </span><span class="mf">24.0</span><span class="p">,</span>
|
||
<span class="w"> </span><span class="nt">"start_time"</span><span class="p">:</span><span class="w"> </span><span class="mf">208.3</span><span class="p">,</span>
|
||
<span class="w"> </span><span class="nt">"end_time"</span><span class="p">:</span><span class="w"> </span><span class="mf">216.7</span><span class="p">,</span>
|
||
<span class="w"> </span><span class="nt">"summary"</span><span class="p">:</span><span class="w"> </span><span class="s2">"[208s-217s, 9s] Two people discussing project timeline..."</span><span class="p">,</span>
|
||
<span class="w"> </span><span class="nt">"similarity"</span><span class="p">:</span><span class="w"> </span><span class="mf">0.72</span>
|
||
<span class="w"> </span><span class="p">}</span>
|
||
<span class="w"> </span><span class="p">],</span>
|
||
<span class="w"> </span><span class="nt">"page"</span><span class="p">:</span><span class="w"> </span><span class="mi">1</span><span class="p">,</span>
|
||
<span class="w"> </span><span class="nt">"page_size"</span><span class="p">:</span><span class="w"> </span><span class="mi">5</span><span class="p">,</span>
|
||
<span class="w"> </span><span class="nt">"strategy"</span><span class="p">:</span><span class="w"> </span><span class="s2">"llm_reranked"</span>
|
||
<span class="p">}</span>
|
||
</code></pre></div>
|
||
|
||
<table class="table">
|
||
<thead>
|
||
<tr>
|
||
<th>Field</th>
|
||
<th>Type</th>
|
||
<th>Description</th>
|
||
</tr>
|
||
</thead>
|
||
<tbody>
|
||
<tr>
|
||
<td><code>strategy</code></td>
|
||
<td>string</td>
|
||
<td>Always <code>"llm_reranked"</code> for this endpoint</td>
|
||
</tr>
|
||
<tr>
|
||
<td><code>results</code></td>
|
||
<td>array</td>
|
||
<td>Re-ranked search results (same format as smart search)</td>
|
||
</tr>
|
||
</tbody>
|
||
</table>
|
||
<h4>Fallback</h4>
|
||
<p>If LLM reranking fails (model unavailable, timeout), falls back to RRF order without error.</p>
|
||
<hr />
|
||
<h3>Visual Search</h3>
|
||
<p><strong>Auth</strong>: Required
|
||
<strong>Scope</strong>: global / file-level</p>
|
||
<p>Search text chunks → find associated identities. Returns chunks where face detections overlap with text content.</p>
|
||
<h4>Query Parameters</h4>
|
||
<table class="table">
|
||
<thead>
|
||
<tr>
|
||
<th>Field</th>
|
||
<th>Type</th>
|
||
<th>Required</th>
|
||
<th>Default</th>
|
||
<th>Description</th>
|
||
</tr>
|
||
</thead>
|
||
<tbody>
|
||
<tr>
|
||
<td><code>q</code></td>
|
||
<td>string</td>
|
||
<td>Yes</td>
|
||
<td>—</td>
|
||
<td>Search text (ILIKE match)</td>
|
||
</tr>
|
||
<tr>
|
||
<td><code>file_uuid</code></td>
|
||
<td>string</td>
|
||
<td>No</td>
|
||
<td>—</td>
|
||
<td>Restrict to specific file. If omitted, searches all files (global search)</td>
|
||
</tr>
|
||
<tr>
|
||
<td><code>limit</code></td>
|
||
<td>integer</td>
|
||
<td>No</td>
|
||
<td>50</td>
|
||
<td>Max results</td>
|
||
</tr>
|
||
<tr>
|
||
<td><code>page</code></td>
|
||
<td>integer</td>
|
||
<td>No</td>
|
||
<td>1</td>
|
||
<td>Page number</td>
|
||
</tr>
|
||
<tr>
|
||
<td><code>page_size</code></td>
|
||
<td>integer</td>
|
||
<td>No</td>
|
||
<td>50</td>
|
||
<td>Items per page</td>
|
||
</tr>
|
||
</tbody>
|
||
</table>
|
||
<h4>Example (Global Search)</h4>
|
||
<div class="codehilite"><pre><span></span><code>curl<span class="w"> </span>-s<span class="w"> </span><span class="s2">"</span><span class="nv">$API</span><span class="s2">/api/v1/search/identity_text?q=love"</span><span class="w"> </span>-H<span class="w"> </span><span class="s2">"X-API-Key: </span><span class="nv">$KEY</span><span class="s2">"</span>
|
||
</code></pre></div>
|
||
|
||
<h4>Example (File-specific Search)</h4>
|
||
<div class="codehilite"><pre><span></span><code>curl<span class="w"> </span>-s<span class="w"> </span><span class="s2">"</span><span class="nv">$API</span><span class="s2">/api/v1/search/identity_text?file_uuid=</span><span class="nv">$FILE_UUID</span><span class="s2">&q=love"</span><span class="w"> </span>-H<span class="w"> </span><span class="s2">"X-API-Key: </span><span class="nv">$KEY</span><span class="s2">"</span>
|
||
</code></pre></div>
|
||
|
||
<h4>Response (200)</h4>
|
||
<div class="codehilite"><pre><span></span><code><span class="p">{</span>
|
||
<span class="w"> </span><span class="nt">"success"</span><span class="p">:</span><span class="w"> </span><span class="kc">true</span><span class="p">,</span>
|
||
<span class="w"> </span><span class="nt">"total"</span><span class="p">:</span><span class="w"> </span><span class="mi">5</span><span class="p">,</span>
|
||
<span class="w"> </span><span class="nt">"results"</span><span class="p">:</span><span class="w"> </span><span class="p">[</span>
|
||
<span class="w"> </span><span class="p">{</span>
|
||
<span class="w"> </span><span class="nt">"file_uuid"</span><span class="p">:</span><span class="w"> </span><span class="s2">"a6fb22eebefaef17e62af874997c5944"</span><span class="p">,</span>
|
||
<span class="w"> </span><span class="nt">"chunk_id"</span><span class="p">:</span><span class="w"> </span><span class="s2">"llm_parent_..._256_270"</span><span class="p">,</span>
|
||
<span class="w"> </span><span class="nt">"start_time"</span><span class="p">:</span><span class="w"> </span><span class="mf">256.256</span><span class="p">,</span>
|
||
<span class="w"> </span><span class="nt">"end_time"</span><span class="p">:</span><span class="w"> </span><span class="mf">270.228</span><span class="p">,</span>
|
||
<span class="w"> </span><span class="nt">"text_content"</span><span class="p">:</span><span class="w"> </span><span class="s2">"...lack of affection..."</span><span class="p">,</span>
|
||
<span class="w"> </span><span class="nt">"identity_id"</span><span class="p">:</span><span class="w"> </span><span class="mi">9</span><span class="p">,</span>
|
||
<span class="w"> </span><span class="nt">"identity_name"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Audrey Hepburn"</span><span class="p">,</span>
|
||
<span class="w"> </span><span class="nt">"identity_source"</span><span class="p">:</span><span class="w"> </span><span class="s2">"tmdb"</span><span class="p">,</span>
|
||
<span class="w"> </span><span class="nt">"trace_id"</span><span class="p">:</span><span class="w"> </span><span class="mi">94</span>
|
||
<span class="w"> </span><span class="p">}</span>
|
||
<span class="w"> </span><span class="p">]</span>
|
||
<span class="p">}</span>
|
||
</code></pre></div>
|
||
|
||
<table class="table">
|
||
<thead>
|
||
<tr>
|
||
<th>Field</th>
|
||
<th>Type</th>
|
||
<th>Description</th>
|
||
</tr>
|
||
</thead>
|
||
<tbody>
|
||
<tr>
|
||
<td><code>results[].file_uuid</code></td>
|
||
<td>string</td>
|
||
<td>File UUID where chunk was found</td>
|
||
</tr>
|
||
<tr>
|
||
<td><code>results[].identity_id</code></td>
|
||
<td>integer</td>
|
||
<td>Identity ID if face was detected</td>
|
||
</tr>
|
||
<tr>
|
||
<td><code>results[].trace_id</code></td>
|
||
<td>integer</td>
|
||
<td>Face trace ID</td>
|
||
</tr>
|
||
</tbody>
|
||
</table>
|
||
<hr />
|
||
<h3>Visual Search (Planned)</h3>
|
||
<table class="table">
|
||
<thead>
|
||
<tr>
|
||
<th>Method</th>
|
||
<th>Endpoint</th>
|
||
<th>Status</th>
|
||
<th>Description</th>
|
||
</tr>
|
||
</thead>
|
||
<tbody>
|
||
<tr>
|
||
<td>POST</td>
|
||
<td><code>/api/v1/search/visual</code></td>
|
||
<td>Not implemented</td>
|
||
<td>Search visual chunks</td>
|
||
</tr>
|
||
<tr>
|
||
<td>POST</td>
|
||
<td><code>/api/v1/search/visual/class</code></td>
|
||
<td>Not implemented</td>
|
||
<td>Search by object class</td>
|
||
</tr>
|
||
<tr>
|
||
<td>POST</td>
|
||
<td><code>/api/v1/search/visual/density</code></td>
|
||
<td>Not implemented</td>
|
||
<td>Search by object density</td>
|
||
</tr>
|
||
<tr>
|
||
<td>POST</td>
|
||
<td><code>/api/v1/search/visual/combination</code></td>
|
||
<td>Not implemented</td>
|
||
<td>Search by object combination</td>
|
||
</tr>
|
||
<tr>
|
||
<td>POST</td>
|
||
<td><code>/api/v1/search/visual/stats</code></td>
|
||
<td>Not implemented</td>
|
||
<td>Visual chunk statistics</td>
|
||
</tr>
|
||
</tbody>
|
||
</table>
|
||
<h4>Embedding Model</h4>
|
||
<table class="table">
|
||
<thead>
|
||
<tr>
|
||
<th>Detail</th>
|
||
<th>Value</th>
|
||
</tr>
|
||
</thead>
|
||
<tbody>
|
||
<tr>
|
||
<td><strong>Model</strong></td>
|
||
<td>EmbeddingGemma-300m</td>
|
||
</tr>
|
||
<tr>
|
||
<td><strong>Endpoint</strong></td>
|
||
<td><code>POST /api/v1/embeddings</code> on port 11436</td>
|
||
</tr>
|
||
<tr>
|
||
<td><strong>Dimension</strong></td>
|
||
<td>768</td>
|
||
</tr>
|
||
<tr>
|
||
<td><strong>Storage</strong></td>
|
||
<td>pgvector (<code>chunk.embedding</code> column)</td>
|
||
</tr>
|
||
</tbody>
|
||
</table>
|
||
<hr />
|
||
<p><em>Updated: 2026-06-20 — Added llm-smart search, completed frames search documentation, marked visual search as planned</em></p>
|
||
</div>
|
||
</body>
|
||
</html> |