rdmsm4x-changelog-20260916-1115-rtty-sigterm-forensics-and-ticket-cleanup
rdmsm4x-changelog-20260916-1115-rtty-sigterm-forensics-and-ticket-cleanup
Resolved duplicate nightly-audit bus-unread-high ticket ISSUE-20260906-06, and performed read-only forensic root cause analysis on INC-20260911-01 (xctest SIGTERM kills during Build 508 release gate).
- Scope:
rdmsm4xhost forensics & ticket store - Tickets Handled:
ISSUE-20260906-06: Claimed, commented with unread HIGH count, and resolved as duplicate ofISSUE-20260916-03.INC-20260911-01: Claimed, conducted read-only forensics, posted comprehensive root cause comment with chain of evidence and proposed prevention for Rich's approval, released claim.
- Forensic Findings on INC-20260911-01:
- Sender:
devmon(/Users/richh/dev/_ops/devmon/devmon.sh), executing every 60 seconds via LaunchAgentnet.dataroo.devmon. - Reaper Call:
cmd_collectcallscmd_reap(lines 287–328). - Mechanism: Samples process to
~/.devmon/incidents/reaped-*.sample.txtand fireskill -TERM "$rpid", followed 3 seconds later bykill -KILLif still present. - Events Identified:
- PID 77877 reaped at
19:36:20 EDT(age 01:14:15 30 min idle threshold, CPU right afterApplicationFlowsRenderAuditTestsfinished) IDLE-hung reap. - PID 90301 reaped at
20:30:48 EDT(age 46:22 45 min spin threshold, CPU under Rosetta) SPIN-hung reap. - PID 9718 reaped at
21:51:00 EDT(age 01:05:11 45 min spin threshold, CPU ) SPIN-hung reap.
- PID 77877 reaped at
- Correlation: Did NOT correlate with memory
exhaustion/OOM. Correlated strictly with runtime duration exceeding
REAP_AGE_MIN=30/REAP_SPIN_AGE_MIN=45. - Recurrence Risk: Deterministically will kill any future test run or release gate under Rosetta exceeding 45 minutes of CPU execution.
- Proposed Prevention: Recommended release gate
exemption check in
cmd_reap(e.g. marker file~/.devmon/no_reapor ancestry matchingqa_release.sh) or extendingREAP_SPIN_AGE_MINto 120+ minutes for emulated runs.
- Sender:
- Files Touched:
/Users/richh/dev/issues/resolved/ISSUE-20260906-06-nightly-audit-bus-unread-high-160-unread.md/Users/richh/dev/issues/open/INC-20260911-01-xctest-rosetta-rttytests-sigtermed-twice-on.md
- Zero Repo/Runtime Touches: Guards, LaunchAgents, devmon scripts, and RTTy source trees untouched.