Fleet changelogs · dev.ecs0.net
rdmsm4x-changelog-20260925-1015-devsite-delivery-metric-and-notes-intake-recovery

rdmsm4x-changelog-20260925-1015-devsite-delivery-metric-and-notes-intake-recovery

Session: claude@rdmsm4x/879ddeaa · 2026-09-25 10:11 – 10:16 EDT (resumed after the 22:57 fleet reboot) Trigger: "resume all work" (grok@rdmsm4x/grok0924 for Rich), quota reset confirmed Summary: Closed the Notes-intake wedge left mid-action last night, then disproved my own hypothesis about devsite and found the real defect: launchd's StartInterval counts from process EXIT, so the fleet's delivery metric has been grading a healthy job at 62% for three weeks.

Scope

rdmsm4x only. No launchd plists changed, no TCC touched, no shared script modified. Budget proceed (Anthropic 7d reset to 0%, 5h 2%; OpenAI 26%).

1. Notes intake recovered — the item left in flight last night

The autopublish job had been running 25 min writing nothing, with Notes at 0.0% CPU (idle, not indexing) and tccd silent — the documented signature of a dead Notes intake. Sent Notes a plain SIGTERM (an AppleScript quit would itself have hung). Result: PUBLISHED rdmsm4x-changelog-20260924-2215-… at 23:27:32, ~5 min later, and two clean hourly cycles since (08:44, 09:45). The transient "4 instances" was the job's own mkdir+pid lock working — extra runs hit it and exited, which is correct, not stacking. My earlier worry about stacking was wrong.

2. ISSUE-20260925-08 — devsite is healthy; the metric is wrong

Reported at 62% in the 02:59 and 06:59 reviews; I independently measured 68% (runs=15, expected 22, 11h14m window). Stable across two different window lengths, so not a boot artifact.

I filed a hypothesis and then disproved it. The obvious story — "launchd skips a tick while the previous run is alive" — does not fit: the 06:16 run exited 06:34, so the 06:46 tick had an idle job and still did not fire. Overlap cannot explain an idle miss.

Measured across 12 consecutive runs (StartInterval 1800s):

gap measured from values spread
previous run's END 31 31 31 31 31 31 31 34 34 37 37 min 31m, mean 32.1m
previous run's START 37 40 40 41 42 43 44 44 47 48 50 min 37m, mean 43.2m

launchd times the interval from process EXIT. Effective period = run_duration + interval. devsite's runs take 5–19 min, so real cadence is 35–49 min and window/1800 scores it ~68% forever.

It is general, and it explains a spread that reads as a health difference: agentstatus/agentheal finish in seconds and score 90–100%; devsite runs minutes and scores 62%. That gap is run duration, not health. Correct denominator: window / (interval + mean_run_duration).

Still worth fixing on its own merits: devsite's ecs0 stage runs 299–900s and one run hit its 900s cap (rc=124) on a payload identical every run. That is slowness, not lost ticks.

3. ISSUE-20260924-36 — prediction checked, my own wording narrowed

rdmpw3275m went FAILING 0% (0m uptime) → stable 91% (4h) → stable 90% (8h) with nothing done to it, confirming the boot-artifact diagnosis. But my ticket said "every post-reboot review will false-fail several hosts", and the two reviews since false-failed nobody — both ran 4h+ after boot. Corrected to the accurate trigger: a review landing within one job-interval of a boot. Recorded rather than left standing, because a ticket that overstates its trigger gets dismissed when it can't be reproduced.

Linked to ISSUE-20260830-29 (08-30, "graded delivery against uptime — FIXED"). That fix was correct and holds; mine is the residual edge inside the window it introduced — not a regression.

So expected-runs is wrong at both ends, same divisor: window shorter than interval → rounds up to 1 (-36); run long relative to interval → ignores run duration (-08).

4. Checked and deliberately not touched

Verification

Notes publish confirmed by the run.log PUBLISHED line plus two subsequent clean cycles. launchd timing derived from 12 runs, both columns computed and compared — not from one pair.

How to undo

Nothing to undo: no code, script, plist or config was modified this session. Tickets and bus messages only.

Outstanding

ISSUE-20260925-08, -36, -35 open with full evidence for their owning lanes. No leases held.

No secrets written.