Fleet changelogs · dev.ecs0.net
rdmbair15m5-changelog-20260823-0108-obsidian-vault-consolidation

Obsidian MasterVault consolidation — rdmbair15m5

Consolidated a 6.1 GB / 74,497-file Obsidian vault down to 3.3 GB / 36,465 files by SHA-256 content dedup, removed all zero-byte notes, merged a duplicated Apple Notes tree, and disabled a competing sync engine. Nothing was deleted — everything removed is in a reversible quarantine with a manifest.

Host: rdmbair15m5 · Window: 2026-08-22 22:45 → 2026-08-23 01:08 EDT Operator: claude@rdmbair15m5 · Authorized by: Rich ("you take all actions with obsidian")

Why

Rich enabled Obsidian Sync (Plus) at 22:38 EDT and reported sync had been getting stuck historically. Vault needed consolidating before it replicated off-machine.

Root cause of stuck sync

Three sync engines writing one directory: the iCloud container hosting the vault, Obsidian Sync, and the obsidian-git plugin. Plus 24 enabled plugins, several of which full-index every note at startup. Ruled out: iCloud eviction — container reads caught-up, 0 .icloud stubs, 0 dataless files.

Corrections to earlier statements in-session

  1. Reported the vault "doubled in two hours." Wrong. Only 31 files were created after 22:40. The 22:45 measurement (35,084 .md / 3.1 GB) caught iCloud still materializing; ~30k files were .name.md.icloud stubs that do not match *.md. True pre-cleanup vault: 65,746 .md / 74,497 files / 6.1 GB.
  2. Reported google-drive-sync as ENABLED. Wrong — installed but not enabled. Only obsidian-git was actively competing.
  3. First dataless check used awk '$1>0 && $2==0', which excludes zero-byte files by construction — it could not have detected zero-byte placeholders. Rich caught this. Re-verified via ubiquity xattrs + .icloud stub count + brctl status.

Actions taken

# Action Result
1 APFS cp -Rc clone backup ~/Backups/obsidian-20260823-0053 — 74,654 files verified identical, 34s
2 Quit Obsidian stopped all writers; filesystem frozen for analysis
3 Full-vault SHA-256 dedup (74,497 files hashed, 0 unreadable) 14,353 duplicate groups, 32,748 redundant byte-identical copies
4 Quarantine dupes + 5,440 zero-byte notes 38,188 files moved, re-verified at move time
5 Disabled obsidian-git competing sync engine removed
6 Deferred dataview, make-md, obsidian-excalidraw-plugin 24 → 20 enabled; re-enable after first clean sync
7 Merged root Apple Notes/ into 30-Ingestion/Apple-Notes/ 10,733 moved; 6,196 name collisions kept BOTH, suffixed (from-vault-root)
8 Removed 5 empty top-level dirs, filed 25 root notes to 00-Inbox/_from-vault-root/, removed empty 30-Ingestion/Credentials/ root loose files 9,217 → 0
9 Restarted Obsidian PID 41565, vault intact at 34,249 .md

Verification

V=~/Library/Mobile\ Documents/iCloud~md~obsidian/Documents/MasterVault
find "$V" -type f -name '*.md' | wc -l          # 34249
find "$V" -type f | wc -l                       # 36465
find "$V" -type f -name '*.md' -size 0 | wc -l  # 0
find "$V" -maxdepth 1 -type f | wc -l           # 0
du -sh "$V"                                     # 3.3G
brctl status iCloud~md~obsidian                 # caught-up

Reversibility

Outstanding — needs Rich

  1. Attachments: 90-Attachments/ — sync or exclude from Obsidian Sync?
  2. Obsidian Sync Plus exact storage + per-file caps — must be read from the app.
  3. Legacy vaults (~27 GB, 8 vaults) still in the iCloud container — disposition?
  4. Move MasterVault OUT of the iCloud container — recommended, not yet done. Deferred deliberately so Sync reconciles the cleanup first rather than two large changes at once.
  5. Re-enable dataview / make-md / excalidraw after first clean full sync.