Fix k3d-dev race conditions, add timeout for k3d hang, improve /etc/hosts output

General MR

Summary

Fixes race conditions, k3d hangs, and unclear post-deployment instructions in k3d-dev.sh.

Race condition fixes:

• Wait for MetalLB CRDs (ipaddresspools, l2advertisements) to be established before applying config
• Decouple proxy container startup into a separate batch so MetalLB config failures don't skip proxies
• Make CoreDNS custom configmap idempotent (--dry-run=client -o yaml | kubectl apply -f -)

k3d hang workaround:

• Remove ssh -t from run_batch_execute. Pseudo-TTY allocation caused hangs on non-interactive batch scripts.
• Wrap k3d cluster create in timeout (default 300s, configurable via --k3d-timeout) to detect the known agent readiness hang (https://github.com/k3d-io/k3d/issues/1420)
• Post-timeout verification (k3d cluster list) confirms the cluster is healthy

UX improvement:

• Label /etc/hosts instructions with gateway names (Public gateway, Passthrough gateway) when using dual public IPs

Post-setup health check:

• Verify at least one IPAddressPool exists after MetalLB config is applied

Tests:

• 7 new BATS tests covering --k3d-timeout arg parsing, batch file creation, and dual-gateway print_instructions output (25 total, all passing)

Relevant logs/screenshots

Clean -a run completing in ~4 minutes with all fixes applied:

MetalLB is installed
Waiting for MetalLB CRDs to be established...
customresourcedefinition.apiextensions.k8s.io/ipaddresspools.metallb.io condition met
customresourcedefinition.apiextensions.k8s.io/l2advertisements.metallb.io condition met
ipaddresspool.metallb.io/primary created
ipaddresspool.metallb.io/secondary created
l2advertisement.metallb.io/primary created
l2advertisement.metallb.io/secondary created
Starting proxy containers...
Verifying MetalLB setup...
================================================================================
====================== DEPLOYMENT FINISHED =====================================
================================================================================

New /etc/hosts output with labeled gateways:

This cluster uses two public IPs routed to different Istio gateways.
Add BOTH lines to your /etc/hosts:

  # Public gateway (40.38.17.186)
  40.38.17.186 grafana.dev.bigbang.mil kibana.dev.bigbang.mil ...

  # Passthrough gateway (56.136.3.110)
  56.136.3.110 keycloak.dev.bigbang.mil vault.dev.bigbang.mil

Linked Issue

Closes #3125 (closed): issue

Upgrade Notices

N/A. k3d-dev.sh is a local developer tool, not part of the deployed chart. Existing usage is unchanged; the new --k3d-timeout flag is optional and defaults to previous behavior.