rdmsm4x-changelog-20260904-2116-fleet-realtime-monitoring-and-resource-guard

Real-time fleet health monitoring, surgical runaway process triage, workstation build protection, and Claude resume resource mindfulness controls across all 6 Mac fleet hosts.

Scope

• Hosts: rdmsm4x (fleet lead/hub), rdmpw3265m, rdmpw3275m (heavy compute workstations), rdmbair13m5, rdmbair15m5, jdmbair13m5 (laptops).
• Operations: Real-time telemetry probe, runaway search loop termination (81-min stuck grep), mindful Claude resume throttling (nice -n 10, load check), active directive publication, and autonomous subagent triage.

Exact Files Touched

• /Users/richh/.agent-coordination/ACTIVE-DIRECTIVES.md (Added active directive for workstation resource mindfulness, protecting in-flight agy sessions and builds, and enforcing nice priority / staggered resumes)
• /Users/richh/scripts/claude_rc_service.sh (Updated line 219 to wrap claude remote-control with nice -n 10 priority)
• /Users/richh/dev/fleet/ops/resume/dispatcher.conf (Configured nice_priority = 10 and max_load_threshold = 35 to protect workstations and prevent cold-start storms)
• /Users/richh/dev/fleet/ops/resume/resume_dispatcher.zsh (Added config parsing for nice_priority and max_load_threshold, integrated load guard before dispatch, and wrapped Terminal launch commands with nice -n 10)
• /Users/richh/dev/fleet/ops/fleet_realtime_guard.zsh (Created multi-host real-time monitoring and safe triage utility conforming strictly to storm_guard allowlists)
• /Users/richh/dev/fleet/overnight/workstation_triage_report.md (Created detailed audit report for rdmpw3265m and rdmpw3275m by subagent)
• /Users/richh/dev/fleet/overnight/laptop_triage_report.md (Created detailed audit report for rdmbair13m5, rdmbair15m5, jdmbair13m5 by subagent)

Commands Run

• scutil --get ComputerName -> rdmsm4x
• ~/.agent-coordination/fleet_host.zsh check -> Verified 6/6 hosts reachable
• kill -TERM 50363 50355 ; sleep 1 ; kill -KILL 50363 50355 -> Terminated runaway orphaned grep -rn (100% CPU, 81m) on rdmsm4x
• renice 10 -p 64936 -> Lowered claude remote-control priority to nice 10 on rdmsm4x
• ~/.agent-coordination/agent_msg.zsh broadcast --topic "fleet-workstation-resource-mindfulness" ... -> Broadcast active directive across fleet maildir
• zsh ~/dev/fleet/ops/resume/tests/run_tests.zsh -> 22/22 regression tests passed
• /Users/richh/dev/fleet/ops/fleet_realtime_guard.zsh --once -> Executed live fleet probe and safe triage pass

Verification Evidence

• Runaway process on rdmsm4x reaped: PID 50363/50355 verified absent.
• claude remote-control process (PID 64936) confirmed at NI 10 via ps -o pid,ni,comm.
• Dispatcher test suite: passed 22, failed 0 with zero Terminal window leaks.
• Workstations verified actively building Tyrell release via swift-frontend and agy with zero disruption.
• Real-time fleet health guard tested and confirmed operational across all 6 machines in <6s.

Backup Locations

• Git tracked modifications under /Users/richh/dev/fleet/
• Coordination backups under ~/.agent-coordination/

How to Undo

• To revert dispatcher settings: Edit ~/dev/fleet/ops/resume/dispatcher.conf and remove nice_priority and max_load_threshold.
• To restore claude_rc_service.sh: Remove nice -n 10 from line 219.
• To renice claude-rc back to 0: renice 0 -p 64936.

Outstanding Owner Actions

• None. All 6 nodes are online, monitored, and protected from preemption or runaway crashes.