rdmbair15m5-changelog-20260923-2025-dead-sessions-audit-and-lanes-remediation

Summary: Resolved root causes for nightly audit failures by stopping claim resurrection on resolved/closed tickets in ticket CLI, updating nightly_audit.zsh CHECK 11, synchronizing the git mirror, acknowledging high-priority agent mail, reconciling active lanes in LANES.md, and resolving tickets TASK-20260911-01 and ISSUE-20260906-09.

1. Scope & Hosts Touched

• Hosts: rdmbair15m5 (local coordinator & Tyrell Build 32 soak canary), rdmsm4x (canonical dev root & ticket authority).
• Repositories:
  • dev/issues (canonical: rdmsm4x:~/dev/issues, remote: git.ecs0.net:git/dev/issues.git, backup: github.com/richhdoty/rdmsm4x-dev-issues)
  • root/fleet (canonical: rdmsm4x:~/dev/fleet, remote: git.ecs0.net:git/root/fleet.git, backup: github.com/richhdoty/rdmsm4x-dev-fleet)

2. Changes Made

1. Resolved TASK-20260911-01 (Tyrell Build 18 Canary):

   • Verified Build 18 evaluation was complete, failed initial RSS gate, diagnosed allocation churn in SyncEngine.sync, fixed in commit e14041a, and was succeeded by Build 19 and subsequent production builds through Build 32.
   • Resolved ticket in canonical store (issues/resolved/TASK-20260911-01...).

2. Fixed Claim Resurrection Defect (ISSUE-20260906-09):

   • Root Cause: When ticket reindex scanned tickets, index_ticket_file() inspected claim_history without verifying ticket status. Since cmd_resolve() did not append an event: resolve entry to frontmatter claim_history, every resolved ticket with prior claim history was re-inserted into the claims table upon reindexing, causing 446 resolved tickets to appear as "dead session" leases.
   • Fix in bin/ticket:
     • Updated index_ticket_file(): explicit guard if meta.get("status") in ("resolved", "closed"): latest_claim_entry = None ensures completed tickets are never inserted into claims.
     • Updated cmd_resolve(): appends event: resolve to claim_history in ticket frontmatter.
     • Updated append_claim_history(): clears assigned_to on resolve events (matching release).
     • Added regression test to tests/test_claim_leases_and_plans.py: asserts that reindex does not resurrect claims on resolved tickets.
   • Fix in fleet/ops/audit/nightly_audit.zsh (v1.4):
     • CHECK 11 updated: dead = [c for c in d.get("claims", []) if not c.get("live") and c.get("status") not in ("resolved", "closed")].
   • Store Reindex & Cleanup:
     • Executed ticket reindex, dropping stale claims in index.sqlite from 452 to active/open claims only.
     • Released expired leases on 6 inactive/blocked tickets (TASK-20260903-31, REV-20260831-04, TASK-20260905-01, TASK-20260831-17, TASK-20260903-36, TASK-20260914-01).
     • Resolved ISSUE-20260906-09 with test evidence.

3. Reconciled LANES.md (CHECK 12):

   • Added active rows for TASK-20260923-02 (Tyrell Build 32 canary on rdmpw3275m / rdmbair15m5) and TASK-20260923-03 (RTTy Build 518 on rdmsm4x).
   • [lanes-reconcile] audit check now reports 0 missing sessions.

4. Synchronized Fleet Git Mirror:

   • Ran fleet_git_mirror.zsh on rdmsm4x, replicating all 210 repos to rdmbair13m5.local (rsync_rc=0).
   • Cleared [git-mirror] finding.

5. Agent Mailbox Drainage:

   • Read and acknowledged unread high-priority messages (20260923-121651-035F2C76 and 20260923-084812-0DB8F052).
   • Cleared [bus-unread-high] finding.

3. Verification Evidence

• Claim Leases Test Suite: 64 / 64 checks passed in python3 /Users/richh/dev/issues/tests/test_claim_leases_and_plans.py (exit 0).
• Fleet Nightly Audit (nightly_audit.zsh report): Findings reduced from 8 to 3:
  • [bus-unread-high] -> 0 unread (PASS)
  • [git-mirror] -> 0 diverged (PASS)
  • [dead-sessions] -> 0 dead claims (PASS)
  • [lanes-reconcile] -> 0 missing rows (PASS)
  • [boot-contract] -> clean (PASS)
  • Only remaining 3 findings are active operations: apps/RTTy unpushed (Build 518 in-progress), Intel Homebrew source bottles (managed by rdmpw3275m), and version cadence (Build 32 soak in flight).
• Tyrell Build 32 Canary Soak (rdmbair15m5, PID 32126):
  • Elapsed: >18,062s (>5h 01m)
  • Physical footprint: 75.59 MiB (stable, below 100 MiB limit)
  • Open file descriptors: 25
  • Latency: sub-millisecond on ports 43117 and 43118 (0.0003s - 0.001s)
  • State: short_gate_passed, long gate verdict expected at ~21:31 EDT.

4. Commits Applied

• dev/issues: c6e040f (fix(ticket): prevent reindex claim resurrection on resolved/closed tickets (ISSUE-20260906-09))
• dev/issues: 91fb181 (chore(lanes): add Build 32 and RTTy 518 active rows to LANES.md)
• root/fleet: 85043bb (fix(ops/audit): ignore resolved and closed tickets in dead-sessions CHECK 11 (ISSUE-20260906-09))

5. How to Undo

• To revert ticket CLI changes: git -C /Users/richh/dev/issues revert c6e040f 91fb181
• To revert audit changes: git -C /Users/richh/dev/fleet revert 85043bb