rdmbair15m5-changelog-20260829-1825-terminal-profiles-syslog-podman

Normalized Apple Terminal.app profiles fleet-wide to ttyrell.dev 3 (JetBrainsMonoNFM 15pt, 165x45 geometry, cd ~/dev startup, EastAsianAmbiguousWide=false), validated Nerd Fonts, deployed unified terminal init and session telemetry logger with RFC5424 streaming, provisioned redundant Podman/OCI logging stacks on rdmpw3265m (log0.ecs0.net) and rdmpw3275m (log1.ecs0.net), created Cloudflare DNS records for log0/log1/db under ecs0.net, and provisioned PostgreSQL 16 on db.ecs0.net with live syslog archival.

Scope

Hosts: rdmbair15m5 (origin), rdmsm4x, rdmbair13m5, rdmpw3265m, rdmpw3275m (fleet-wide rollout; jdmbair13m5 queued on wakeup)
Endpoints: log0.ecs0.net (100.113.170.117), log1.ecs0.net (100.95.173.70), db.ecs0.net (100.113.170.117)

Changes Applied

Apple Terminal Profile Normalization:
- Deployed ~/scripts/sync_terminal_profiles.zsh and canonical profile asset ttyrell_dev_3_profile.b64.
- Updated Default Window Settings and Startup Window Settings to ttyrell.dev 3.
- Normalized all profiles across all reachable fleet hosts with:
  - Font: JetBrainsMonoNFM-Regular (15pt)
  - Startup Command: cd ~/dev
  - Window Size: 165 cols x 45 rows
  - EastAsianAmbiguousWide: False (eliminates 2x cell drift in Ink/TUI runners)
  - UnlimitedScrollback: True (100,000 lines)
  - BackgroundBlur: 0.1698
  - Profile counts updated: rdmsm4x (13), rdmbair13m5 (14), rdmbair15m5 (76), rdmpw3265m (78), rdmpw3275m (144).
Nerd Font Validation:
- Created and executed ~/scripts/validate_nerd_fonts.zsh across all fleet hosts.
- Verified that JetBrainsMonoNFM-Regular is present, valid, and indexed in font caches across all machines.
Terminal Init & Session Logger:
- Created ~/scripts/terminal_init.zsh (standardizes cd ~/dev on start in $HOME, truecolor, Homebrew shellenv, agent helpers).
- Created ~/scripts/agent_session_logger.zsh (hybrid logger creating local structured logs at ~/.local/state/terminal-sessions/<hostname>_tty_session_<agent>_<sessionID>_<start>-<end>.log and streaming RFC5424 syslog messages to log0.ecs0.net:1514).
- Sourced in ~/.zshrc on all fleet hosts.
Syslog Infrastructure & Podman/OCI Ingestion:
- Primary Syslog (log0.ecs0.net on rdmpw3265m): Grafana Alloy (port 514 UDP/TCP, 1514 UDP RFC5424) -> Grafana Loki (port 3100) -> Grafana UI (port 3000).
- Backup Syslog (log1.ecs0.net on rdmpw3275m): Redundant Alloy + Loki + Grafana stack.
- DNS Records in Cloudflare ecs0.net:
  - log0.ecs0.net -> 100.113.170.117
  - log1.ecs0.net -> 100.95.173.70
  - db.ecs0.net -> 100.113.170.117
- Fleet Syslog Routing: Updated /etc/syslog.conf on all hosts to route *.* @100.113.170.117 over Tailscale.
PostgreSQL Database (db.ecs0.net):
- Deployed PostgreSQL 16 on rdmpw3265m (100.113.170.117:5432) with persistent volume at /Users/richh/data/postgres.
- Databases initialized: general, syslog_archive, agent_metrics.
- Table created: syslog_archive.session_logs.
- Archival worker: ~/scripts/syslog_db_archiver.py successfully tested and verified archiving Loki syslog entries into PostgreSQL.

Verification Evidence

sync_terminal_profiles.zsh --verify: Returned All profiles fully normalized and compliant on all 5 reachable hosts.
validate_nerd_fonts.zsh: Returned [OK] JetBrainsMonoNFM-Regular found and PASSED on all 5 hosts.
Port verification:
- nc -u -z -v 100.113.170.117 514 -> Succeeded (log0.ecs0.net)
- nc -u -z -v 100.95.173.70 514 -> Succeeded (log1.ecs0.net)
- nc -z -v 100.113.170.117 5432 -> Succeeded (db.ecs0.net)
- nc -z -v 100.113.170.117 3000 -> Succeeded (Grafana UI)
Ingestion verification: Sent test messages from rdmbair15m5, rdmsm4x, rdmbair13m5, rdmpw3265m, rdmpw3275m; Loki indexed all streams; syslog_db_archiver.py archived 30 rows into PostgreSQL syslog_archive.session_logs.

Rollback & Undo

Plist backups saved at: ~/scripts/terminal_profile_backups/com.apple.Terminal.plist.* on each host.
Syslog conf backups saved at: /etc/syslog.conf.bak-* on each host.