rdmbair15m5-changelog-20260922-0245-keychain-search-list-dev-update-v2.02-fleet-normalization

Guaranteed preservation of login.keychain-db in user search list, upgraded dev_update to v2.02, hardened fleet_signing_rollout to v1.1, normalized search list order across all 5 spokes, and resolved incident ISSUE-20260921-24.

Scope

• Hosts: rdmsm4x, rdmbair15m5, rdmbair13m5, jdmbair13m5, rdmpw3265m, rdmpw3275m
• Repositories:
  • scripts (git.ecs0.net:git/root/scripts.git / github.com/richhdoty/rdmsm4x-dev-scripts.git)
  • fleet (git.ecs0.net:git/root/fleet.git / github.com/richhdoty/rdmsm4x-dev-fleet.git)
  • issues (git.ecs0.net:git/dev/issues.git / github.com/richhdoty/rdmsm4x-dev-issues.git)
  • dev-root-rdmbair15m5 (git.ecs0.net:git/fleet/dev-root-rdmbair15m5.git / github.com/richhdoty/rdmbair15m5-dev-root-docs.git)

Exact Files Touched

• scripts/dev_update_v2.02.zsh (new version, mode 755)
• scripts/dev_update.zsh (symlink updated -> dev_update_v2.02.zsh)
• fleet/ops/keysync/fleet_signing_rollout.zsh (upgraded to v1.1 with safe array ordering and normalize command)
• issues/resolved/ISSUE-20260921-24-rdmsm4x-keychain-search-list-lost-login-.md (resolved ticket)
• SYNC-STATUS.md (authoritative decision and parity log)
• LLM/Claude/changelogs/rdmbair15m5-changelog-20260922-0245-keychain-search-list-dev-update-v2.02-fleet-normalization.md (file archive)

Root Cause & Mechanism

• During an automated maintenance or doctor pass running over SSH, capturing security list-keychains -d user 2>/dev/null in dev_update_v2.01.zsh line 5627 failed or returned empty in a subshell/non-interactive context.
• When cur was empty, security list-keychains -d user -s ${(f)cur} "$kc" executed security list-keychains -d user -s "$kc", wiping out all existing keychains and leaving only fleet-signing.keychain-db.
• Consequently, login.keychain-db (which exclusively stores Developer ID Application private keys on rdmsm4x) was dropped from user keychain lookup, breaking production Developer ID signing and notarization preflights.
• A secondary defect in fleet_signing_rollout.zsh line 45 positioned "$KC" first in the search list command, placing fleet-signing.keychain-db ahead of login.keychain-db across all 5 spokes.

Remediation Applied

1. dev_update_v2.02.zsh:
   • Replaced fragile read-set logic with strict rc checking and whitespace validation.
   • Enforced that login.keychain-db is guaranteed present and FIRST in the search list.
   • Appended fleet-signing.keychain-db after login.keychain-db only if absent.
   • Incremented version to 2.02 with updated header changelog; verified embedded payload integrity (--verify-embedded rc=0).
   • Executed full --dry-run on rdmsm4x: exit 10 (informational mas upgrade advisory only), zero errors, zero false mutations.
2. fleet_signing_rollout.zsh v1.1:
   • Rewrote install_host to construct search list safely with login.keychain-db first and fleet-signing.keychain-db last.
   • Added normalize [host ...] command to repair search list ordering without rebuilding keychains.
   • Executed normalize across all 5 spokes (rdmbair15m5, rdmbair13m5, jdmbair13m5, rdmpw3265m, rdmpw3275m), ensuring [login.keychain-db, fleet-signing.keychain-db] order fleet-wide.
3. Fleet Deployment:
   • Deployed dev_update_v2.02.zsh and updated ~/dev/scripts/dev_update.zsh symlink on all 6 hosts (rdmsm4x, rdmbair15m5, rdmbair13m5, jdmbair13m5, rdmpw3265m, rdmpw3275m). Verified --version output on each host.
4. Ticket Resolution:
   • Claimed and resolved ISSUE-20260921-24 in issues store via ticket resolve on rdmsm4x.
   • Broadcasted resolution announcement via agent_msg.zsh broadcast and synced across all 6 hosts.

Verification Evidence

• dev_update_v2.02.zsh --verify-embedded: rc=0 (payload sha256 f87802488dca verified).
• dev_update_v2.02.zsh --version on all 6 hosts: dev_update v2.02 (self-contained: footlights 1.1.0 + dev_update 2.15, payload sha f87802488dca).
• fleet_signing_rollout.zsh normalize: confirmed login.keychain-db first, fleet-signing.keychain-db second across all 5 spokes.
• fleet_signing_rollout.zsh verify: verify: hosts=5 failed=0 (all 5 spokes report unlock=0 sign=0 team=ZU2882L4HT).
• rdmsm4x signing verification: codesign probe with fleet_sign_unlock passed; Developer ID Application identity 1C07FCD80C36EAD5D9E3B73F7ACCCA5368DC73FB valid and active in user search list.
• Git commits pushed to canonical fleet and private GitHub remotes:
  • scripts: 817bda4
  • fleet: 4a719a7
  • issues: 156f416
  • dev-root-rdmbair15m5: 50cf0c7

Outstanding Owner Actions

• None. All changes verified, committed, pushed, and active.