Sign In

video-cli

Package Overview
Dependencies
Maintainers
1
Versions
7
Alerts
File Explorer

Advanced tools

License
Socket logo

Install Socket

Detect and block malicious and high-risk dependencies

Install

video-cli - npm Package Compare versions

Comparing version
0.2.0
 to
0.2.2
+249
skills/video-cli/SKILL.md
---
name: video-cli
description: Makes videos searchable and inspectable for AI agents. Use when the user needs to answer questions about a video, search for moments, extract frames or clips, read on-screen text, or navigate video content by topic.
allowed-tools: Bash(video-cli:*)
---
# Video Inspection with video-cli
video-cli makes videos searchable and inspectable for AI agents. It ingests a video file, transcribes audio, OCRs on-screen text, and builds embeddings -- all locally. Once set up, you can ask natural-language questions with grounded citations, search for specific moments, navigate by chapter, and extract frames or clips. All commands return JSON.
## Quick Start
```bash
# one-time setup: ingest + transcribe + analyze + embed
video-cli setup recording.mp4
# ask a question with grounded citations
video-cli ask <id> "what is the main argument?"
```
## Commands
### Quick Start
```bash
video-cli setup <file>
# Runs ingest + transcribe + analyze + embed in one step.
# Returns: { id, sourceName, durationSec, watchpoints, utterances, ocrItems, embeddings, ready }
video-cli ask <video-id> <question>
# Answer a question with grounded citations.
# Internally: search -> gather context -> synthesize via LLM
# Returns: { id, query, answer, citations[], suggestedFollowUps[] }
```
### Navigation
```bash
video-cli search <video-id> <query> [--top N]
# Semantic + lexical + description search. Returns ranked matches.
video-cli context <video-id> --at <seconds> [--window N]
# Everything around a timestamp: transcript, OCR, frame descriptions, scene changes.
# JIT enrichment: describes frames on demand if not cached.
video-cli chapters <video-id>
# Segment video into chapters from scene changes + topic shifts.
video-cli next <video-id> --from <seconds>
# Next significant moment (scene change, utterance, OCR change).
video-cli grep <video-id> <exact-text>
# Exact substring search over transcript + OCR. No embeddings needed.
```
### Extraction
```bash
video-cli frame <video-id> --at <seconds> [--output <path>]
# Extract a single frame as JPG.
video-cli clip <video-id> --at <seconds> [--pre N] [--post N] [--duration N] [--output <path>]
# Extract a short video clip around a timestamp. Use --duration for a symmetric clip around --at.
```
### Pipeline
These run individually if you need fine-grained control. `setup` runs them all.
```bash
video-cli ingest <file> [--watchpoints N] [--scene-threshold N]
# Probe video, detect scene changes, pick watchpoints. Local only, no API calls.
video-cli transcribe <video-id> [--chunk-seconds N] [--limit N] [--provider <name>] [--model <name>] [--trim-silence]
# Transcribe audio. Default: Gemini (gemini-transcribe). Use --provider elevenlabs for word-level timestamps + audio events, or --provider deepgram.
video-cli ocr <video-id> [--limit N] [--provider <name>] [--model <name>]
# OCR representative frames. Default: Gemini flash-lite.
video-cli embed <video-id> [--dimensions N] [--no-frames] [--no-transcript] [--no-ocr]
# Build embeddings from transcript + OCR + frame descriptions.
video-cli describe <video-id> [--interval N] [--model <name>]
# Dense frame descriptions at N-second intervals. Optional, enriches search.
```
### Inspection
```bash
video-cli list
# List all ingested videos.
video-cli inspect <video-id>
# Full manifest JSON.
video-cli timeline <video-id>
# All watchpoints + scene change timestamps.
video-cli watchpoints <video-id> [--limit N] [--materialize]
# Raw watchpoint data with optional frame extraction.
video-cli bundle <video-id> [--limit N]
# Evidence bundle: watchpoints + coverage windows + frame paths.
video-cli brief <video-id> [--limit N] [--output <path>]
# Render evidence bundle as Markdown.
video-cli config
# Show current runtime configuration.
```
## Output Shapes
### `ask`
```json
{
"id": "abc123-def456",
"query": "tell me about Sal Stewart",
"answer": "Sal Stewart is predicted to break out in 2026...",
"citations": [
{ "atSec": 176.3, "source": "transcript", "text": "Sal Stewart is going to absolutely mash..." },
{ "atSec": 177, "source": "frame", "text": "Cincinnati Reds player wearing red jersey..." }
],
"suggestedFollowUps": [
"What are his specific stats?",
"Who else is predicted to break out?"
]
}
```
### `search`
```json
{
"id": "abc123-def456",
"query": "cost function",
"matchCount": 3,
"matches": [
{
"score": 1.82,
"source": "transcript",
"atSec": 198.5,
"startSec": 195.0,
"endSec": 210.0,
"text": "the cost function measures how far off..."
}
]
}
```
### `context`
```json
{
"id": "abc123-def456",
"atSec": 176,
"windowSec": 10,
"startSec": 166,
"endSec": 186,
"utterances": ["..."],
"ocrItems": ["..."],
"frames": ["..."],
"sceneChanges": [165.2, 177.0, 181.3],
"suggestedCommands": [
"video-cli frame abc123-def456 --at 177",
"video-cli clip abc123-def456 --at 176 --pre 5 --post 15",
"video-cli next abc123-def456 --from 186"
]
}
```
### `chapters`
```json
{
"id": "abc123-def456",
"durationSec": 600,
"chapterCount": 5,
"chapters": [
{
"index": 0,
"startSec": 0,
"endSec": 120.5,
"durationSec": 120.5,
"utteranceCount": 14,
"text": "Welcome to the presentation...",
"summary": "Title slide with speaker introduction"
}
]
}
```
## Session Examples
### Quick answer (1-2 calls)
```bash
video-cli setup lecture.mp4
# { "id": "lec-abc123", "ready": true, "durationSec": 3600, ... }
video-cli ask lec-abc123 "what is the main argument?"
# { "answer": "The main argument is...", "citations": [...] }
```
### Exploration (3-5 calls)
```bash
video-cli setup lecture.mp4
# { "id": "lec-abc123", ... }
video-cli ask lec-abc123 "what topics are covered?"
# sees "cost function" in citations at 198s
video-cli context lec-abc123 --at 198 --window 15
# reads transcript + OCR + frame descriptions around that moment
video-cli frame lec-abc123 --at 198
# extracts the diagram frame as JPG
# { "output": "data/videos/lec-abc123/frames/frame-198_000.jpg" }
```
### Deep dive with chapters (5+ calls)
```bash
video-cli setup broadcast.mp4
# { "id": "bcast-def456", "durationSec": 1200, ... }
video-cli chapters bcast-def456
# { "chapterCount": 8, "chapters": [{ "index": 0, "startSec": 0, ... }, ...] }
video-cli context bcast-def456 --at 154 --window 20
# reads everything around chapter 5
video-cli search bcast-def456 "playoff implications"
# finds exact moment at 238s
video-cli clip bcast-def456 --at 238 --pre 5 --post 10
# extracts 15-second evidence clip
# { "output": "data/videos/bcast-def456/clips/clip-238_000.mp4" }
```
## Notes
- All commands return JSON to stdout. Progress messages go to stderr.
- `setup` is the recommended entry point. It runs `ingest`, `transcribe`, `analyze`, and `embed` in sequence.
- `ask` performs JIT enrichment: if frame descriptions are missing for the relevant region, it generates them on demand and caches them.
- `context` output includes `suggestedCommands` -- the agent always knows what to try next.
- Video IDs are deterministic hashes of file identity (path + size + mtime). Re-ingesting the same file returns the same ID.
- Requires `ffmpeg`, `ffprobe`, Node >= 22, and `GEMINI_API_KEY` in the environment. One API key powers everything. ElevenLabs and Deepgram are available as optional overrides via `--provider elevenlabs` / `--provider deepgram`.
+55
src/lib/transcript.js
function collectTranscriptEntries(transcript) {
const entries = [];
if (!transcript || !Array.isArray(transcript.items)) {
return entries;
}
for (const chunk of transcript.items) {
if (Array.isArray(chunk.utterances) && chunk.utterances.length > 0) {
for (const utterance of chunk.utterances) {
const text = String(utterance.transcript || '').trim();
if (!text) continue;
entries.push({
startSec: Number(utterance.startSec || 0),
endSec: Number(utterance.endSec || utterance.startSec || 0),
speaker: utterance.speaker ?? null,
text,
});
}
continue;
}
if (Array.isArray(chunk.segments) && chunk.segments.length > 0) {
for (const segment of chunk.segments) {
const text = String(segment.text || '').trim();
if (!text) continue;
entries.push({
startSec: Number(segment.startSec || 0),
endSec: Number(segment.endSec || segment.startSec || 0),
speaker: segment.speaker ?? null,
text,
});
}
continue;
}
const chunkText = String(chunk.text || '').trim();
if (!chunkText) {
continue;
}
entries.push({
startSec: Number(chunk.startSec || 0),
endSec: Number(chunk.endSec || chunk.startSec || 0),
speaker: chunk.speaker ?? null,
text: chunkText,
});
}
return entries;
}
module.exports = {
collectTranscriptEntries,
};
+5
-2
package.json
{
"name": "video-cli",
"version": "0.2.0",
"version": "0.2.2",
"description": "Local-first video REPL for AI agents — search, ask, navigate, extract",

@@ -16,2 +16,3 @@ "license": "MIT",

"src/",
"skills/",
"SKILL.md",

@@ -39,3 +40,5 @@ "README.md",

"scripts": {
"test": "node --test",
"test": "node --test --experimental-test-isolation=none",
"skills:sync": "node scripts/sync-skill-docs.js",
"skills:check": "node scripts/sync-skill-docs.js --check",
"eval": "node evals/run.js",

@@ -42,0 +45,0 @@ "eval:json": "node evals/run.js --json",

+73
-42
README.md

@@ -8,7 +8,7 @@ <p align="center">

<p align="center">
<strong>Make a video feel like a codebase — searchable, inspectable, citeable.</strong>
<strong>Make a video behave like a codebase: searchable, inspectable, citeable.</strong>
</p>
<p align="center">
One API key. Two cents per hour of video. Every answer grounded in timestamps.
Video is opaque to most tools. `video-cli` turns it into local artifacts an agent can query with evidence.
</p>

@@ -23,65 +23,96 @@

## Why It Exists
Video is hard to work with programmatically. You can watch it, but you cannot `grep` it, cite it, diff it, or hand it to an agent and expect repeatable answers.
`video-cli` exists to make a video usable as a working surface: it extracts transcript spans, OCR text, frame descriptions, embeddings, timestamps, frames, and clips so an agent can search and answer with grounded evidence.
## Quick Start
```bash
node video-cli.js init # set up API key (secure, interactive)
node video-cli.js setup video.mp4 # ingest + transcribe + analyze + embed
node video-cli.js ask <id> "what is the main argument?"
video-cli init
video-cli setup video.mp4
video-cli ask <id> "what is the main argument?"
```
That's it. `setup` ingests, transcribes, OCRs, and embeds. `ask` returns a grounded answer with timestamps and citations. Most sessions are 1-3 commands.
That is the normal path: initialize credentials once, ingest a video once, then ask questions against the local artifacts.
## What It Does
## Install
Published package:
```bash
npm install -g video-cli
video-cli init
```
You have a video. → setup → Now it's searchable.
ask "what happened at the end?"
search "pricing discussion"
context --at 3:45
chapters
frame --at 2:30
clip --at 2:30 --pre 5 --post 10
Repo checkout:
```bash
git clone https://github.com/Dexin-Huang/video-cli
cd video-cli
cp .env.example .env
node video-cli.js init
```
Every command returns JSON. An AI agent can `setup` a meeting recording then `ask` questions about it — with cited timestamps, frame paths, and suggested follow-ups.
The published package is the normal user path. The repo-local `node video-cli.js ...` form is mainly for development and contributor workflows.
## Commands
## Onboarding
| Tier | Commands | What they do |
|------|----------|-------------|
| **Intent** | `setup`, `ask` | One-shot: ingest a video or answer a question |
| **Navigate** | `search`, `context`, `chapters`, `next`, `grep` | Drill into specific moments |
| **Extract** | `frame`, `clip` | Pull out frames or video clips |
| **Pipeline** | `ingest`, `transcribe`, `analyze`, `embed` | Run pipeline steps individually |
| **Inspect** | `list`, `status`, `inspect`, `brief`, `config` | Check what's available |
1. Add your API key with `video-cli init`.
2. Run `video-cli setup <video-file>` to build the artifacts.
3. Ask a grounded question with `video-cli ask <video-id> "<question>"`.
4. Use `video-cli search`, `context`, `chapters`, `frame`, or `clip` when you need to inspect a specific moment.
## Example
```bash
video-cli setup lecture.mp4
# { "id": "lec-abc123", "ready": true, ... }
video-cli ask lec-abc123 "what is the main argument?"
# { "answer": "...", "citations": [...], "suggestedFollowUps": [...] }
video-cli context lec-abc123 --at 198 --window 15
# { "utterances": [...], "ocrItems": [...], "frames": [...], "sceneChanges": [...] }
video-cli clip lec-abc123 --at 198 --pre 5 --post 10
# { "output": "data/videos/lec-abc123/clips/clip-198_000.mp4" }
```
## Command Surface
| Area | Commands | Purpose |
|------|----------|---------|
| **Intent** | `setup`, `ask` | Ingest a video or answer a question with evidence |
| **Navigate** | `search`, `context`, `chapters`, `next`, `grep` | Find and inspect specific moments |
| **Extract** | `frame`, `clip` | Pull out a still image or short clip |
| **Pipeline** | `ingest`, `transcribe`, `ocr`, `analyze`, `embed`, `describe` | Run individual stages when you need control |
| **Inspect** | `list`, `status`, `inspect`, `timeline`, `watchpoints`, `bundle`, `brief`, `config` | Check readiness and inspect artifacts |
| **Automation** | `eval:generate`, `eval:run` | Measure retrieval quality |
## How It Works
```
```text
video.mp4
→ ffmpeg scene detection (free, local)
→ Gemini transcription ($0.0002/min)
→ Gemini OCR + frame descriptions ($0.003)
→ Gemini embeddings ($0.0002)
→ searchable JSON artifacts on disk
-> ffmpeg scene detection
-> transcription
-> OCR + frame descriptions
-> embeddings
-> searchable JSON artifacts on disk
query
→ semantic + lexical + description search (local, instant)
→ Gemini synthesizes answer ($0.0001)
→ { answer, citations[], suggestedFollowUps[], framePaths[] }
-> semantic + lexical + description search
-> grounded answer with citations and follow-ups
```
## Cost
## Configuration
| Video length | Setup cost | Per query |
|---|---|---|
| 5 min | $0.002 | $0.0001 |
| 1 hour | $0.018 | $0.0001 |
| 10 hours | $0.18 | $0.0001 |
`video-cli` reads `video-cli.config.json` in the repo root and merges it with `video-cli.config.example.json`, plus environment overrides. The main knobs are provider selection, transcription chunking, OCR model choice, and embedding dimensions.
One API key (`GEMINI_API_KEY`) powers everything. Optionally swap in ElevenLabs or Deepgram for transcription via `--provider`.
Use `video-cli config` to see the resolved runtime config. Use `GEMINI_API_KEY` for the default path, and override providers with `--provider` or env vars when needed.
## Requirements
- Node >= 22
- Node 22 or newer
- `ffmpeg` and `ffprobe`

@@ -92,3 +123,3 @@ - `GEMINI_API_KEY` in `.env`

See **[SKILL.md](SKILL.md)** — the complete agent-facing reference with output shapes, flags, and session examples.
See [SKILL.md](SKILL.md) for the agent-facing command reference and output shapes.

@@ -98,3 +129,3 @@ ## Development

```bash
npm test # 16 tests, no API calls
npm test # 20 tests, no API calls
npm run eval # retrieval quality eval

@@ -101,0 +132,0 @@ ```

+3
-3
SKILL.md

@@ -38,3 +38,3 @@ ---

```bash
video-cli search <video-id> <query> [--top N] [--threshold N] [--hybrid]
video-cli search <video-id> <query> [--top N]
# Semantic + lexical + description search. Returns ranked matches.

@@ -62,4 +62,4 @@

video-cli clip <video-id> --at <seconds> [--pre N] [--post N] [--output <path>]
# Extract a short video clip around a timestamp.
video-cli clip <video-id> --at <seconds> [--pre N] [--post N] [--duration N] [--output <path>]
# Extract a short video clip around a timestamp. Use --duration for a symmetric clip around --at.
```

@@ -66,0 +66,0 @@ 

+72
-9
src/cli.js

@@ -14,3 +14,3 @@ const { ensureDataRoot, getRepoRoot } = require('./lib/store');

setup: 'video-cli setup <file>\n\nRun the full pipeline: ingest + transcribe + analyze + embed.\nCreates all artifacts needed for search and ask.\n\nFlags:\n --adaptive Adaptive watchpoint selection (default: true)\n --watchpoints N Max watchpoints (default: auto)\n --scene-threshold Scene detection threshold (default: 0.35)',
ask: 'video-cli ask <video-id> <question>\n\nAnswer a question with grounded citations.\nInternally: search → context → synthesize via Gemini Flash-Lite.\n\nReturns: answer, citations with timestamps, suggested follow-ups, frame paths.',
ask: 'video-cli ask <video-id> <question>\n\nAnswer a question with grounded citations.\nInternally: search -> context -> synthesize via Gemini Flash-Lite.\n\nReturns: answer, citations with timestamps, suggested follow-ups, frame paths.',
search: 'video-cli search <video-id> <query> [--top N]\n\nSemantic + lexical + description search.\nReturns ranked matches with scores and timestamps.',

@@ -21,10 +21,21 @@ context: 'video-cli context <video-id> --at <seconds> [--window N] [--no-enrich]\n\nEverything around a timestamp: transcript, OCR, frame descriptions, scene changes.\nJIT enrichment: describes frames on demand if not cached.',

grep: 'video-cli grep <video-id> <text>\n\nExact substring search across transcript and OCR text.\nReturns matching segments with timestamps.',
frame: 'video-cli frame <video-id> --at <seconds> [--out <path>]\n\nExtract a single frame as JPG at the given timestamp.\nDefaults to writing in the current directory.',
clip: 'video-cli clip <video-id> --at <seconds> [--duration N] [--out <path>]\n\nExtract a video clip starting at the given timestamp.\nDefault duration: 10 seconds.',
ingest: 'video-cli ingest <file>\n\nProbe video metadata and extract adaptive watchpoint frames.\nFirst step of the pipeline — run before transcribe/analyze.\n\nFlags:\n --adaptive Adaptive watchpoint selection (default: true)\n --watchpoints N Max watchpoints (default: auto)\n --scene-threshold Scene detection threshold (default: 0.35)',
frame: 'video-cli frame <video-id> --at <seconds> [--output <path>]\n\nExtract a single frame as JPG at the given timestamp.\nAlias: --out <path>.',
clip: 'video-cli clip <video-id> --at <seconds> [--pre N] [--post N] [--duration N] [--output <path>]\n\nExtract a video clip centered on the given timestamp.\nUse --duration as shorthand for a symmetric clip around --at.\nAlias: --out <path>.',
ingest: 'video-cli ingest <file>\n\nProbe video metadata and extract adaptive watchpoint frames.\nFirst step of the pipeline - run before transcribe/analyze.\n\nFlags:\n --adaptive Adaptive watchpoint selection (default: true)\n --watchpoints N Max watchpoints (default: auto)\n --scene-threshold Scene detection threshold (default: 0.35)',
transcribe: 'video-cli transcribe <video-id>\n\nTranscribe audio track using ElevenLabs or Gemini.\nProduces word-level timestamps.',
ocr: 'video-cli ocr <video-id>\n\nExtract on-screen text from representative watchpoint frames.',
analyze: 'video-cli analyze <video-id>\n\nRun OCR + frame description in one pass via Gemini.\nProduces per-frame OCR text and visual descriptions.',
embed: 'video-cli embed <video-id>\n\nBuild text embeddings for transcript and OCR segments.\nRequired for semantic search.',
describe: 'video-cli describe <video-id> [--interval N]\n\nGenerate dense visual descriptions across the video timeline.',
status: 'video-cli status <video-id>\n\nShow artifact readiness and pipeline completion status.\nLists which steps have been run and what is missing.',
inspect: 'video-cli inspect <video-id> [--timeline] [--watchpoints]\n\nFull manifest dump for a video.\nOptionally include timeline events or watchpoint details.',
timeline: 'video-cli timeline <video-id>\n\nShow watchpoints plus scene-change timestamps.',
watchpoints: 'video-cli watchpoints <video-id> [--limit N] [--materialize]\n\nShow raw watchpoints. Use --materialize to also extract frame images.',
bundle: 'video-cli bundle <video-id> [--limit N]\n\nBuild an evidence bundle with watchpoints, frame paths, and coverage windows.',
brief: 'video-cli brief <video-id> [--limit N] [--output <path>]\n\nWrite a Markdown brief for a video evidence bundle.\nAlias: --out <path>.',
list: 'video-cli list\n\nList all ingested videos.',
config: 'video-cli config\n\nShow the merged runtime configuration.',
install: 'video-cli install --skills\n\nInstall the bundled Claude Code skill for agent integration.',
'eval:generate': 'video-cli eval:generate <video-id>\n\nGenerate evaluation queries from descriptions and transcript artifacts.',
'eval:run': 'video-cli eval:run <video-id> [--top N]\n\nRun retrieval evaluation against generated query cases.',
};

@@ -66,2 +77,3 @@

if (command === 'init') return runInit();
if (command === 'install') return runInstall(flags);
if (command === 'cleanup') return runCleanup(positionals, flags);

@@ -86,6 +98,7 @@ if (command === 'ingest') return runIngest(positionals, flags, helpers);

const lines = [
'video-cli \u2014 video REPL for AI agents',
'video-cli - video REPL for AI agents',
'',
'Quick Start:',
' init Set up API key (interactive, secure)',
' install --skills Install Claude Code skill for agent use',
' cleanup [video-id] [--all] Remove artifacts, data, or everything',

@@ -108,5 +121,7 @@ ' setup <file> Full pipeline: ingest + transcribe + analyze + embed',

' ingest <file> Probe video + adaptive watchpoints',
' transcribe <video-id> Audio \u2192 transcript',
' transcribe <video-id> Audio -> transcript',
' ocr <video-id> OCR representative watchpoint frames',
' analyze <video-id> OCR + describe in one pass (Gemini)',
' embed <video-id> Build embeddings (Gemini)',
' describe <video-id> Dense visual descriptions across the timeline',
'',

@@ -117,5 +132,12 @@ 'Inspection:',

' inspect <video-id> Full manifest (--timeline, --watchpoints)',
' timeline <video-id> Watchpoints + scene-change timestamps',
' watchpoints <video-id> Watchpoint details (--materialize to extract frames)',
' bundle <video-id> Evidence bundle with coverage windows',
' brief <video-id> Markdown summary',
' config Current config',
'',
'Evaluation:',
' eval:generate <video-id> Generate retrieval eval queries',
' eval:run <video-id> Run retrieval eval metrics',
'',
"Use 'video-cli <command> --help' for details on a specific command.",

@@ -152,2 +174,5 @@ ];

flags[withoutPrefix] = true;
if (withoutPrefix.startsWith('no-') && withoutPrefix.length > 3) {
flags[withoutPrefix.slice(3)] = false;
}
}

@@ -187,2 +212,6 @@

if (value === false) {
return false;
}
const normalized = String(value).trim().toLowerCase();

@@ -199,2 +228,30 @@ if (['1', 'true', 'yes', 'on'].includes(normalized)) {

async function runInstall(flags) {
const fs = require('node:fs');
const path = require('node:path');
const os = require('node:os');
if (!flags.skills) {
console.error('Usage: video-cli install --skills');
console.error(' Installs the Claude Code skill for agent integration.');
return;
}
const skillSource = path.join(__dirname, '..', 'skills', 'video-cli');
const skillDest = path.join(os.homedir(), '.claude', 'skills', 'video-cli');
if (!fs.existsSync(skillSource)) {
console.error('Skill files not found. Reinstall video-cli: npm install -g video-cli');
process.exit(1);
}
fs.mkdirSync(skillDest, { recursive: true });
for (const file of fs.readdirSync(skillSource)) {
fs.copyFileSync(path.join(skillSource, file), path.join(skillDest, file));
}
console.error('Installed video-cli skill to ' + skillDest);
console.error('Claude Code will discover it automatically on next launch.');
}
async function runCleanup(positionals, flags) {

@@ -234,2 +291,8 @@ const fs = require('node:fs');

const configPath = path.join(getRepoRoot(), 'video-cli.config.json');
if (fs.existsSync(configPath)) {
fs.unlinkSync(configPath);
console.error(`Removed: ${configPath}`);
}
console.error('');

@@ -243,3 +306,3 @@ console.error('All data and credentials removed.');

console.error(' video-cli cleanup <video-id> Remove one video\'s artifacts');
console.error(' video-cli cleanup --all Remove all data + API key');
console.error(' video-cli cleanup --all Remove all data + API key + config');
}

@@ -265,3 +328,3 @@

console.error('video-cli init — set up your Gemini API key');
console.error('video-cli init - set up your Gemini API key');
console.error('');

@@ -322,3 +385,3 @@ console.error('Get your key at: https://aistudio.google.com/apikey');

// Write to .env
const lines = existing ? existing.split(/\r?\n/).filter(l => !l.startsWith('GEMINI_API_KEY=')) : [];
const lines = existing ? existing.split(/\r?\n/).filter(line => !line.startsWith('GEMINI_API_KEY=')) : [];
lines.push(`GEMINI_API_KEY=${key}`);

@@ -325,0 +388,0 @@ fs.writeFileSync(envPath, lines.filter(Boolean).join('\n') + '\n');

+3
-2
src/commands/inspect.js

@@ -145,4 +145,5 @@ const path = require('node:path');

const bundle = buildEvidenceBundle(manifest, Math.max(1, Math.floor(limit)));
const output = flags.output
? path.resolve(String(flags.output))
const outputFlag = flags.output ?? flags.out;
const output = outputFlag
? path.resolve(String(outputFlag))
: createArtifactPath(id, '', 'brief.md');

@@ -149,0 +150,0 @@ 

+20
-6
src/commands/media.js

@@ -18,4 +18,5 @@ const path = require('node:path');

const manifest = loadManifest(id);
const output = flags.output
? path.resolve(String(flags.output))
const outputFlag = flags.output ?? flags.out;
const output = outputFlag
? path.resolve(String(outputFlag))
: createArtifactPath(id, 'frames', `frame-${formatSecondsForFile(atSec)}.jpg`);

@@ -34,5 +35,17 @@

const atSec = parseNumberFlag(flags, 'at', Number.NaN);
const preSec = parseNumberFlag(flags, 'pre', 5);
const postSec = parseNumberFlag(flags, 'post', 5);
const hasPre = Object.prototype.hasOwnProperty.call(flags, 'pre');
const hasPost = Object.prototype.hasOwnProperty.call(flags, 'post');
const hasDuration = Object.prototype.hasOwnProperty.call(flags, 'duration');
let preSec = parseNumberFlag(flags, 'pre', 5);
let postSec = parseNumberFlag(flags, 'post', 5);
if (hasDuration && !hasPre && !hasPost) {
const durationSec = parseNumberFlag(flags, 'duration', Number.NaN);
if (durationSec <= 0) {
throw new Error('Invalid numeric value for --duration');
}
preSec = durationSec / 2;
postSec = durationSec / 2;
}
if (!Number.isFinite(atSec)) {

@@ -43,4 +56,5 @@ throw new Error('Missing required numeric flag: --at');

const manifest = loadManifest(id);
const output = flags.output
? path.resolve(String(flags.output))
const outputFlag = flags.output ?? flags.out;
const output = outputFlag
? path.resolve(String(outputFlag))
: createArtifactPath(id, 'clips', `clip-${formatSecondsForFile(atSec)}.mp4`);

@@ -47,0 +61,0 @@ 

+14
-4
src/commands/pipeline.js

@@ -110,3 +110,5 @@ const path = require('node:path');

const sceneThreshold = parseNumberFlag(flags, 'scene-threshold', 0.35);
const requestedWatchpoints = parseNumberFlag(flags, 'watchpoints', 12);
const requestedWatchpoints = Object.prototype.hasOwnProperty.call(flags, 'watchpoints')
? parseNumberFlag(flags, 'watchpoints', Number.NaN)
: null;

@@ -117,2 +119,6 @@ const identity = getFileIdentity(resolvedInput);

const durationSec = Number(probe.format.duration || 0);
const autoWatchpointTarget = Math.max(6, Math.min(24, Math.ceil(Math.max(durationSec, 1) / 30)));
const watchpointCap = Number.isFinite(requestedWatchpoints)
? Math.max(1, Math.floor(requestedWatchpoints))
: null;

@@ -126,4 +132,8 @@ let changePointsSec;

watchpoints = pickAdaptiveWatchpoints(durationSec, sceneScores, {
minCount: Math.max(6, Math.ceil(durationSec / 60)),
maxCount: Math.max(requestedWatchpoints, Math.ceil(durationSec / 15)),
minCount: watchpointCap === null
? Math.max(6, Math.ceil(durationSec / 60))
: Math.min(watchpointCap, Math.max(3, Math.ceil(durationSec / 60))),
maxCount: watchpointCap === null
? Math.max(autoWatchpointTarget, Math.ceil(durationSec / 15))
: watchpointCap,
sigmaMultiplier: 1.0,

@@ -134,3 +144,3 @@ minGapSec: 3,

changePointsSec = detectSceneChanges(resolvedInput, sceneThreshold);
watchpoints = pickWatchpoints(durationSec, changePointsSec, requestedWatchpoints);
watchpoints = pickWatchpoints(durationSec, changePointsSec, watchpointCap ?? autoWatchpointTarget);
}

@@ -137,0 +147,0 @@ 

+0
-4
src/lib/describe.js

@@ -177,6 +177,2 @@ const fs = require('node:fs');

const hasCoverage = descriptions && Array.isArray(descriptions.items) &&
descriptions.items.some(d => d.atSec >= startSec && d.atSec <= endSec);
if (hasCoverage) return descriptions;
const apiKey = process.env.GEMINI_API_KEY || null;

@@ -183,0 +179,0 @@ const descModel = model || 'gemini-3.1-flash-lite-preview';

+12
-7
src/lib/embed.js
const crypto = require('node:crypto');
const fs = require('node:fs');
const { batchAsync, fetchWithRetry, extractGeminiError, guessMimeType } = require('./net');
const { collectTranscriptEntries } = require('./transcript');

@@ -117,8 +118,5 @@ const DEFAULT_MODEL = 'gemini-embedding-2-preview';

if (sources.transcript && transcript && Array.isArray(transcript.items)) {
let index = 0;
for (const chunk of transcript.items) {
for (const utt of (chunk.utterances || [])) {
const text = String(utt.transcript || '').trim();
if (text) transcriptPending.push({ index: index++, utt, text });
}
const entries = collectTranscriptEntries(transcript);
for (let index = 0; index < entries.length; index += 1) {
transcriptPending.push({ index, entry: entries[index], text: entries[index].text });
}

@@ -128,3 +126,10 @@ }

item => embedText({ apiKey, text: item.text, model, taskType, dimensions }),
item => ({ source: 'transcript', index: item.index, startSec: item.utt.startSec, endSec: item.utt.endSec, speaker: item.utt.speaker ?? null, text: item.text }));
item => ({
source: 'transcript',
index: item.index,
startSec: item.entry.startSec,
endSec: item.entry.endSec,
speaker: item.entry.speaker ?? null,
text: item.text,
}));

@@ -131,0 +136,0 @@ const ocrPending = [];

+41
-10
src/lib/media.js

@@ -49,16 +49,40 @@ const crypto = require('node:crypto');

'-hide_banner', '-i', filePath,
'-filter:v', `select='gt(scene,${t})',showinfo`,
'-vsync', 'vfr', '-f', 'null', nullSink,
'-filter:v', `select='gt(scene,${t})',metadata=print`,
'-an', '-f', 'null', nullSink,
], { allowFailure: true });
return parseSceneScoreEvents(result.stderr);
}
function parseSceneScoreEvents(stderr) {
const seen = new Set();
const entries = [];
for (const m of (result.stderr || '').matchAll(/pts_time:([0-9]+(?:\.[0-9]+)?)/g)) {
const atSec = Number(Number(m[1]).toFixed(3));
if (!Number.isFinite(atSec)) continue;
const key = atSec.toFixed(3);
if (seen.has(key)) continue;
const lines = String(stderr || '').split(/\r?\n/);
let pendingAtSec = null;
for (const line of lines) {
const timeMatch = line.match(/pts_time:([0-9]+(?:\.[0-9]+)?)/);
if (timeMatch) {
pendingAtSec = Number(Number(timeMatch[1]).toFixed(3));
}
const scoreMatch = line.match(/lavfi\.scene_score=([0-9]+(?:\.[0-9]+)?)/);
if (!scoreMatch || !Number.isFinite(pendingAtSec)) {
continue;
}
const key = pendingAtSec.toFixed(3);
if (seen.has(key)) {
pendingAtSec = null;
continue;
}
seen.add(key);
entries.push({ atSec, score: t });
entries.push({
atSec: pendingAtSec,
score: Number(Number(scoreMatch[1]).toFixed(6)),
});
pendingAtSec = null;
}
return entries.sort((a, b) => a.atSec - b.atSec);

@@ -68,4 +92,10 @@ }

function pickAdaptiveWatchpoints(durationSec, sceneScores, options = {}) {
const minCount = options.minCount || Math.max(6, Math.ceil(durationSec / 30));
const maxCount = options.maxCount || Math.max(minCount, Math.ceil(durationSec / 15));
const requestedMinCount = Number.isFinite(options.minCount)
? Math.max(1, Math.floor(options.minCount))
: Math.max(6, Math.ceil(durationSec / 30));
const requestedMaxCount = Number.isFinite(options.maxCount)
? Math.max(1, Math.floor(options.maxCount))
: Math.max(requestedMinCount, Math.ceil(durationSec / 15));
const maxCount = Math.max(1, requestedMaxCount);
const minCount = Math.min(maxCount, Math.max(1, requestedMinCount));
const sigmaMultiplier = options.sigmaMultiplier || 1.0;

@@ -442,3 +472,4 @@ const minGapSec = options.minGapSec || 3;

pickWatchpoints,
parseSceneScoreEvents,
probeVideo,
};
+18
-22
src/lib/search.js
const { cosineSimilarity } = require('./embed');
const { collectTranscriptEntries } = require('./transcript');

@@ -158,9 +159,10 @@ const STOP_WORDS = new Set('the and that this with from have are was were for not they what been will when your like just also about into more some very then there here these those would could other than show where which does scene moment visual first after before during being make made'.split(' '));

const audioEvents = [];
for (const entry of collectTranscriptEntries(transcript)) {
if (entry.endSec > startSec && entry.startSec < endSec) {
utterances.push(entry);
}
}
if (transcript && Array.isArray(transcript.items)) {
for (const chunk of transcript.items) {
for (const utt of (chunk.utterances || [])) {
if (utt.endSec > startSec && utt.startSec < endSec) {
utterances.push({ startSec: utt.startSec, endSec: utt.endSec, speaker: utt.speaker ?? null, text: utt.transcript });
}
}
for (const event of (chunk.audioEvents || [])) {

@@ -209,10 +211,3 @@ if (event.startSec >= startSec && event.startSec <= endSec) audioEvents.push(event);

// Collect all utterances with timestamps
const utterances = [];
if (transcript && Array.isArray(transcript.items)) {
for (const chunk of transcript.items) {
for (const utt of (chunk.utterances || [])) {
utterances.push({ startSec: utt.startSec, endSec: utt.endSec, text: utt.transcript || '' });
}
}
}
const utterances = collectTranscriptEntries(transcript);

@@ -278,11 +273,12 @@ // Build chapter boundaries from scene change clusters

// Next utterance start
if (transcript && Array.isArray(transcript.items)) {
for (const chunk of transcript.items) {
for (const utt of (chunk.utterances || [])) {
if (utt.startSec > fromSec + 1) {
candidates.push({ atSec: utt.startSec, type: 'utterance', text: utt.transcript, endSec: utt.endSec, speaker: utt.speaker ?? null });
break;
}
}
if (candidates.some(c => c.type === 'utterance')) break;
for (const entry of collectTranscriptEntries(transcript)) {
if (entry.startSec > fromSec + 1) {
candidates.push({
atSec: entry.startSec,
type: 'utterance',
text: entry.text,
endSec: entry.endSec,
speaker: entry.speaker ?? null,
});
break;
}

@@ -289,0 +285,0 @@ }