Local-first visual second brain for AI-readable video, screen, snapshot, robotics, and source-spine memory.
ARC-StreamMemory turns video files, screen recordings, screenshots, DAW/plugin sessions, game footage, browser work, robotics camera feeds, and app UI states into deterministic, cryptographically hashed, AI-readable visual memory modules.
It is designed as the visual second brain / AI sight spine for the GareBear99 ARC ecosystem: frames become indexed evidence, summaries become module attachments, hashes become source-spine proof, and capture sessions become replayable memory bundles.
ARC-StreamMemory is for anyone searching for:
- AI visual memory
- visual second brain
- video memory for LLMs
- screen recording for AI
- local-first multimodal memory
- visual RAG / frame retrieval
- deterministic video archive
- cryptographic frame hashing
- robotics camera memory
- FFmpeg frame sampling for AI
- reproducible visual evidence bundles
- AI-readable screenshots and session replay
visual source
→ regular FFmpeg video/snapshot ingest
→ chosen AI frame-speed schedule
→ frame hashes
→ seeded source spine
→ OCR-ready/event-ready timeline
→ AI digest
→ ARC-Core-style receipts
→ OmniBinary-style chunk map
→ Arc-RAR-style bundle manifest
→ local source-spine viewer
→ AI module attachment JSON
The output is not just an MP4 or screenshots. It is a structured memory object that another AI can inspect, replay, cite, and attach to a module.
Normal screen recording answers:
What happened? Maybe watch the whole video again.
ARC-StreamMemory answers:
What happened?
→ Read the AI digest.
→ Jump to event 000042.
→ Open frame_000042.
→ Verify the frame hash.
→ Follow the ARC receipt.
→ Follow the OmniBinary-style pointer.
→ Package/restore through the Arc-RAR-style bundle manifest.
That makes the memory AI-readable, deterministic, replayable, and source-verifiable.
| Capability | Status | Notes |
|---|---|---|
| Demo generated visual-memory session | ✅ complete | deterministic proof session |
| Snapshot folder ingest | ✅ complete | screenshots/images into AI memory |
| Regular FFmpeg video ingest | ✅ complete if FFmpeg is installed | sample MP4/MOV/MKV/WebM at chosen FPS |
| AI frame-speed policy | ✅ complete | 0.2, 0.5, 1, 2, 5, 10 FPS-style workflows |
| Frame hashing | ✅ complete | SHA-256 per frame |
| Memory spine hashing | ✅ complete | deterministic session root lineage |
| Seeded source spine | ✅ complete | replayable seed/hash lineage |
| AI digest | ✅ complete | Markdown + JSON digest |
| AI module attachment | ✅ complete | structured object for AI consumption |
| ARC-Core-style receipts | ✅ complete | export format ready |
| OmniBinary-style chunk map | ✅ complete | pointer format ready |
| Arc-RAR-style bundle manifest | ✅ complete | ZIP export today, Arc-RAR integration later |
| Local HTML viewer | ✅ complete | inspect frames, events, digest, JSON |
| ARC-FusionCapture adapter layer | ✅ complete as adapter/spec | robotics-ready capture policy + sensor sync scaffolding |
| Native live screen capture | 🚧 integration gate | not falsely claimed yet |
| Real OCR engine | 🚧 integration gate | placeholder/index format ready |
| Native OmniBinary persistence | 🚧 integration gate | export map ready |
| Native Arc-RAR packaging | 🚧 integration gate | manifest ready |
python scripts/create_demo_session.py
python scripts/build_stream_memory.py examples/demo_session --title "ARC demo visual memory"
python scripts/hash_memory_spine.py examples/demo_session
python scripts/build_seed_spine.py examples/demo_session
python scripts/build_ai_digest.py examples/demo_session
python scripts/validate_memory_bundle.py examples/demo_session
python scripts/make_bundle.py examples/demo_session --out release_evidence/demo_streammemory_bundle.zipOpen viewer:
python -m http.server 8080Then open:
http://localhost:8080/viewer/index.html?session=../examples/demo_session/memory/module_attachment.json
ARC-StreamMemory works with normal FFmpeg today.
python scripts/ffmpeg_probe.py
python scripts/ingest_video.py input.mp4 --fps 1 --out sessions/video_memory
python scripts/build_stream_memory.py sessions/video_memory --title "Video memory"
python scripts/hash_memory_spine.py sessions/video_memory
python scripts/build_seed_spine.py sessions/video_memory
python scripts/build_ai_digest.py sessions/video_memory
python scripts/validate_memory_bundle.py sessions/video_memoryRecommended AI frame rates:
| FPS | Use case |
|---|---|
| 0.2 | long passive session memory |
| 0.5 | lightweight visual diary |
| 1 | general AI inspection default |
| 2 | UI debugging / GitHub / DAW flows |
| 5 | detailed interaction review |
| 10 | motion-sensitive review |
ARC-StreamMemory now includes a compatibility layer for the planned ARC-FusionCapture runtime.
ARC-FusionCapture is the future robotics/media capture layer that should wrap regular FFmpeg with:
- camera/feed profiles
- robotics capture modes
- hardware-acceleration selection
- sensor timestamp sync
- rolling buffer policy
- event-triggered clips
- AI-friendly frame-speed output
- ARC-Core receipts
- OmniBinary pointers
- Arc-RAR bundle manifests
Current files:
integrations/arc_fusion_capture/README.md
integrations/arc_fusion_capture/profile_presets.json
scripts/build_capture_policy.py
scripts/fusion_capture_adapter.py
scripts/sync_sensor_log.py
schemas/capture_policy.schema.json
schemas/sensor_frame.schema.json
schemas/robotics_session.schema.json
Example robotics-style adapter flow:
python scripts/build_capture_policy.py --mode robot_navigation --fps 2 --out sessions/nav_test/memory/capture_policy.json
python scripts/fusion_capture_adapter.py --input robot_run.mp4 --policy sessions/nav_test/memory/capture_policy.json --out sessions/nav_test
python scripts/sync_sensor_log.py sessions/nav_test examples/sensor_log.jsonl
python scripts/build_stream_memory.py sessions/nav_test --title "Robot navigation memory"
python scripts/build_seed_spine.py sessions/nav_test
python scripts/validate_memory_bundle.py sessions/nav_testsession/
├─ frames/
├─ memory/
│ ├─ capture_policy.json
│ ├─ frame_index.json
│ ├─ event_timeline.jsonl
│ ├─ ocr_index.jsonl
│ ├─ ai_digest.md
│ ├─ ai_digest.json
│ ├─ module_attachment.json
│ ├─ memory_spine.json
│ ├─ seed_spine.json
│ └─ session_summary.md
├─ receipts/arc_receipts.jsonl
├─ omnibinary/chunk_map.json
├─ arcrar/bundle_manifest.json
├─ reports/validation_report.json
└─ reports/bundle_export_report.json
ARC-StreamMemory stores visual memory with a deterministic seed chain:
capture_policy_hash
+ source_fingerprint
+ frame_schedule_hash
+ ordered_frame_hashes
+ chunk_hash
= session_root_seed
This gives the session a reproducible source spine:
root_seed
→ chunk_000001
→ frame_000001
→ frame_hash
→ event_receipt
→ module_attachment_pointer
This pattern was strengthened using the seeded replay/source-lineage direction from Seeded-Universe-Recreation-Engine.
ARC-StreamMemory is part of a linked ARC / TizWildin / GareBear99 ecosystem. Every repo below is either a source reference, benchmark target, integration target, discovery surface, or public-router anchor.
| Repo | Role in ARC-StreamMemory |
|---|---|
| ARC-Core | authority layer, receipts, event truth, source governance |
| omnibinary-runtime | binary-addressable memory spine and chunk-ledger inspiration |
| Arc-RAR | portable archive/restore bundle direction |
| ARC-Neuron-LLMBuilder | local AI memory, governed build loop, module attachment use case |
| ARC-AudioBench | benchmark/evidence methodology companion |
| Seeded-Universe-Recreation-Engine | seeded source-spine / deterministic replay storage model |
| FreeEQ8 | DAW/plugin testing screen-memory target |
| FreeVox8 | spectral plugin / visual-memory benchmark target |
| Voxel Audio | audio-visual router/session target |
| Instrudio | Web Audio / MIDI / virtual instrument memory target |
| awesome-audio-plugins-dev | Start Here technical discovery list |
| awesome-audio-lists | root hub for audio/list/discovery resources |
| awesome-python-audio-science | research/citation bridge for Python audio science, MIR, DSP, ML audio |
| awesome-music-platforms | artist/music platform discovery map |
| TizWildinEntertainmentHUB | public router for plugins, lists, packs, deconstructed loops, visualizers |
| TizWildin-Release-Vault | release surface for music assets, deconstructed loops, and packs |
Use ARC-StreamMemory to turn debugging videos and UI sessions into reproducible visual memory modules.
Use it to archive DAW/plugin tests, FreeEQ8/FreeVox8 validation sessions, pluginval videos, and UI regressions.
Use regular FFmpeg now, then connect ARC-FusionCapture later for sensor-synced camera memory and robot black-box replay.
Use CITATION.cff, seeded spines, hashes, and module attachments to make visual sessions citable and reproducible.
Use it to capture game states, UI flows, visual bugs, and build history as replayable evidence bundles.
ai-memory
visual-memory
video-memory
screen-recording
multimodal-ai
visual-rag
local-first
ffmpeg
robotics
computer-vision
cryptographic-hashing
reproducible-research
second-brain
arc-core
omnibinary
arc-rar
This package is complete as a public source release foundation for deterministic visual memory:
- ingest
- indexing
- hashing
- seeded source-spine storage
- AI digest
- module attachment
- viewer
- validation
- regular FFmpeg workflow
- ARC-FusionCapture adapter layer
- bundle export
Remaining integration gates are intentionally not overclaimed:
- native live screen capture
- real OCR engine hookup
- native OmniBinary persistence
- native Arc-RAR packaging
- live ARC-Core API sync
- production robotics sensor bus integration
MIT License.