rdmpw3275m-changelog-20260901-2328-add-parallel-prefetch-to-cascadelake-script

Summary: Added upfront parallel source archive pre-fetching (brew fetch --build-from-source) with 8 concurrent download workers to brew_recompile_cascadelake.zsh. Pre-fetched 49 AI/compute packages in 14 seconds. Deployed to rdmpw3275m, rdmpw3265m, and rdmsm4x:~/dev/scripts/.

1. Scope

• Hosts: rdmpw3275m, rdmpw3265m, rdmsm4x.
• Files Modified:
  • ~/dev/scripts/brew_recompile_cascadelake.zsh
  • rdmpw3265m:~/dev/scripts/brew_recompile_cascadelake.zsh
  • rdmsm4x:~/dev/scripts/brew_recompile_cascadelake.zsh

2. Changes Made

1. Parallel Pre-Fetch Phase:
   • Pre-downloads all required source archives using xargs -P 8 before compilation begins.
   • Eliminates network wait times during sequential build loops.
2. New Pre-Fetch Flags:
   • --fetch-only: Pre-fetches all tarballs in parallel and exits without compiling.
   • --no-prefetch: Skips upfront pre-fetching.
   • --fetch-workers <N>: Configures parallel download concurrency (default: 8).

3. Verification Evidence

• Pre-fetch Benchmark on rdmpw3275m:

  ==> Pre-fetching source tarballs for 49 packages in parallel (8 workers)...
  ✔ Pre-fetch complete in 14s (all source archives cached locally)
  ✔ Fetch-only mode complete. Exiting.