5 Archive Tools · 228 Databases · 1TB NVMe

Nothing gets lost

Five tools for document archival, IPFS content addressing, web snapshots, and encrypted backups. 228 SQLite databases with 156,675 FTS5 entries across 7 nodes.

View Tools Storage Map

Tools

Five archive and backup tools

Full-text search, IPFS pinning, web archival, and encrypted backups. All data stays local.

Document Archive

FTS5 full-text search across all documents. Metadata tagging, version history, instant retrieval from 228 SQLite databases.

document_archive.py

IPFS Content Tracker

IPFS CID tracking and pin management. Content-addressed storage ensures nothing changes or disappears.

ipfs_content_tracker.py

IPFS Pinner

Automated IPFS pinning with garbage collection and replication. Keeps important content available.

ipfs_pinner.py

Web Archiver

Website snapshots with Wayback Machine integration. Diff tracking shows what changed between captures.

web_archiver.py

Backup Manager

Incremental backups with encryption. Multi-destination: local disk, S3-compatible (MinIO on Cecilia), rclone to Google Drive.

backup_manager.py

Memory System

156,675 entries in FTS5 index. Every conversation, code snippet, and decision — searchable in milliseconds from ~/.blackroad/.

~/.blackroad/*.db

Database Inventory

Storage across the fleet

Alice

PostgreSQL, Qdrant vector DB, Pi-hole DNS, 48+ domain configs

77% disk (3.2GB free)

Cecilia

MinIO object storage, 4 rclone instances to Google Drive, 16 Ollama models

MinIO S3

Octavia

1TB NVMe primary store, Gitea (207 repos, 61MB blackroad-os), NATS messaging

931GB NVMe

Lucidia

334 web apps in /var/www, 21 GitHub runner dirs (19GB), 14 Docker images

42% disk (238GB SD)

Mac

228 SQLite databases in ~/.blackroad/, 156,675 FTS5 entries (~184MB index)

~184MB index

gematria

DO droplet NYC3, 4cpu/8GB — SSH down but WireGuard alive, rsync target

droplet

anastasia

DO droplet NYC1, 1cpu/1GB — WG hub, Headscale, Nginx, 94% disk full

94% full

Memory System

228 databases, 156,675 memories

The BlackRoad memory system stores everything in SQLite with FTS5 full-text search indexes. Every conversation, every code snippet, every decision — searchable in milliseconds. Located at ~/.blackroad/ on all nodes.

228

SQLite DBs

156,675

FTS5 Entries

~184MB

Total Index

Backup Topology

Where data flows

Mac → Google Drive

rclone sync of all project files and databases

every 6h

Mac → gematria

rsync to DigitalOcean droplet (NYC3) via WireGuard

every 12h

Cecilia → Google Drive

4 concurrent rclone instances (needs consolidation)

continuous

Cecilia → GitHub

github-relay.sh mirrors Gitea repos to GitHub

every 30m

Octavia NVMe

1TB primary data store — Gitea, Docker volumes, model weights

primary

Daily backup-sync

Comprehensive fleet backup + sovereign-mesh sync

daily 3am

Memory sync

Nightly memory database consolidation across nodes

daily 6am

Storage Health

Known issues

Lucidia SD card degrading

"mmc0: Card stuck being busy!" in dmesg. Swap growing to 1.3GB/8.5GB. SD card showing wear.

critical

Alice SD 77% full

Only 3.2GB free. Kernel 6.1.21 (2023) — needs full OS migration to Bookworm.

critical

Lucidia 21 runner dirs = 19GB

Old GitHub Actions runner directories under /home/blackroad/runners/. Could reclaim 19GB.

degraded

Lucidia swap growing

1.3GB/8.5GB used. Swap on degrading SD card accelerates wear.

degraded

anastasia 94% disk

DigitalOcean droplet NYC1 nearly full. 1cpu/1GB — limited capacity.

advisory

Cecilia 4 rclone instances

All syncing same Google Drive. Should consolidate to 1 instance with proper scheduling.

advisory