Fleet changelogs · dev.ecs0.net
rdmbair15m5-changelog-20260822-1704-rddb-native-connectors-dedup-daemon

rdmbair15m5-changelog-20260822-1704-rddb-native-connectors-dedup-daemon

Native macOS system suite maturity for rdDB.app featuring multi-source store connectors, atomic APFS clonefile block deduplication, background daemon pacing, and comprehensive automated test suites.

Scope

Key Implementations & Enhancements

  1. Multi-Source macOS Store & Document Connectors (RDDBCore):

    • Apple Notes (AppleNotesConnector.swift): CoreData SQLite ingestion with zero-dependency protobuf wire parsing (ProtobufReader) and zlib decompression of ZICNOTEDATA note bodies, OCR attachment mapping, and read-only WAL opening.
    • Apple Mail (AppleMailConnector.swift): Byte-framed RFC 822/2822 & MIME parser extracting headers (Subject, From, To, Date) and streaming multipart text/HTML bodies without reading or loading attachments into memory.
    • Browsers (BrowserConnector.swift): Safari binary/XML Bookmarks.plist and Reading List parser; Google Chrome History SQLite ingestion (WebKit 1601 epoch timestamp conversion) and Bookmarks JSON tree parser.
    • Rich Documents (DocumentParser.swift): Streaming in-memory PKZip engine for .docx, .pptx, .xlsx, CSV/TSV, JSON, YAML, Markdown, and PDF text extraction.
    • Cloud Stub Guard (CloudStubGuard.swift): 4-layer inspection (getattrlist SF_DATALESS, stat UF_DATALESS, extended attributes) preventing dataless iCloud / Google Drive downloads.

  2. Active APFS Clonefile Deduplication Engine (RDDBDedup, rddb-cli, rdDB):

    • AtomicAPFSDeduplicator.swift: Cryptographic SHA-256 / BLAKE3 duplicate clustering, clonefile(2) Copy-on-Write replacement via temporary file swap, permission/size pre-validation, and exact physical block space reclaim tracking (st_blocks * 512).
    • APFSVolumeProbe.swift: Validates APFS filesystem support and VOL_CAP_INT_CLONE capability.
    • CLI & GUI: rddb-cli dedup [--apply|--dry-run] and SwiftUI DuplicatesView with one-click deduplication.

  3. Background Daemon & Resource Pacing (RDDBDaemon):

    • LaunchdManager.swift: SMAppService.agent integration with fallback standalone launchd plist descriptor for background launch on login.
    • ThermalGovernor.swift: Reactive monitoring of ProcessInfo.processInfo.thermalState via notifications, with dynamic throttling on .serious and .critical states.
    • PowerGovernor.swift: Dynamic battery vs AC power throttling via IOKit.ps.
    • FSEventsMonitor.swift: Native CoreServices FSEventStreamCreate monitor with 5.0s debouncing window for incremental directory indexing.

  4. Automated Testing & Verification Harness (Tests/RDDBCoreTests/):

    • 5 synthetic fixture generators (NotesFixtureGenerator, MailFixtureGenerator, SafariFixtureGenerator, ChromeFixtureGenerator, DocumentFixtureGenerator).
    • 6 test suites: ConnectorTests, DedupSafetyTests, ConcurrencyWALTests, AdversarialConnectorTests, EmpiricalStressTests, RDDBCoreTests.
    • 68 tests passing (100% pass rate) with zero compiler warnings.

Verification Evidence