General MR

Summary

This MR adds network policy template for allow-egress-storage-mimir-mimir. This was identified during testing NetworkPolicies on our staging cluster.

Relevant logs/screenshots

Errors before NetworkPolicy

ts=2025-09-02T20:30:31.56871211Z caller=main.go:225 level=info msg="Starting application" version="(version=2.16.0, branch=HEAD, revision=b4f36da)"
ts=2025-09-02T20:30:31.569538273Z caller=server.go:368 level=info msg="server listening on addresses" http=[::]:8080 grpc=[::]:9095
ts=2025-09-02T20:30:31.579339898Z caller=module_service.go:82 level=info msg=starting module=sanity-check
ts=2025-09-02T20:30:31.579364101Z caller=module_service.go:82 level=info msg=starting module=activity-tracker
ts=2025-09-02T20:30:31.57940494Z caller=sanity_check.go:32 level=info msg="Checking directories read/write access"
ts=2025-09-02T20:30:31.580146469Z caller=sanity_check.go:37 level=info msg="Directories read/write access successfully checked"
ts=2025-09-02T20:30:31.580152353Z caller=sanity_check.go:39 level=info msg="Checking object storage config"
ts=2025-09-02T20:30:41.580907117Z caller=sanity_check.go:115 level=warn msg="Unable to successfully connect to configured object storage (will retry)" err="blocks storage: unable to successfully send a request to object storage: Get \"https://staging-objectstore-mimir.s3.dualstack.us-gov-west-1.amazonaws.com/blocks/sanity-check-at-startup\": context deadline exceeded"
ts=2025-09-02T20:30:52.679985691Z caller=sanity_check.go:115 level=warn msg="Unable to successfully connect to configured object storage (will retry)" err="blocks storage: unable to successfully send a request to object storage: Get \"https://staging-objectstore-mimir.s3.dualstack.us-gov-west-1.amazonaws.com/blocks/sanity-check-at-startup\": context deadline exceeded"
ts=2025-09-02T20:31:06.344582059Z caller=sanity_check.go:115 level=warn msg="Unable to successfully connect to configured object storage (will retry)" err="blocks storage: unable to successfully send a request to object storage: Get \"https://staging-objectstore-mimir.s3.dualstack.us-gov-west-1.amazonaws.com/blocks/sanity-check-at-startup\": context deadline exceeded"

ts=2025-09-02T20:35:13.744136575Z caller=memberlist_client.go:594 level=info msg="memberlist fast-join starting" nodes_found=15 to_join=8
ts=2025-09-02T20:35:13.758931255Z caller=memberlist_client.go:600 level=info msg="fast-joining node failed" node=192.168.153.216:7946 err="1 error occurred:\n\t* Failed to join 192.168.153.216:7946: EOF\n\n"
ts=2025-09-02T20:35:13.786260027Z caller=memberlist_client.go:614 level=info msg="memberlist fast-join finished" joined_nodes=8 elapsed_time=43.685512ms
ts=2025-09-02T20:35:13.786282076Z caller=memberlist_client.go:626 level=info phase=startup msg="joining memberlist cluster" join_members=dns+mimir-mimir-gossip-ring.mimir.svc.cluster.local.:7946
ts=2025-09-02T20:35:13.85397484Z caller=memberlist_client.go:633 level=info phase=startup msg="joining memberlist cluster succeeded" reached_nodes=14 elapsed_time=67.685975ms
ts=2025-09-02T20:35:23.750740993Z caller=sanity_check.go:115 level=warn msg="Unable to successfully connect to configured object storage (will retry)" err="blocks storage: unable to successfully send a request to object storage: Get \"https://staging-objectstore-mimir.s3.dualstack.us-gov-west-1.amazonaws.com/blocks/sanity-check-at-startup\": context deadline exceeded"
ts=2025-09-02T20:35:33.743564663Z caller=log.go:245 level=warn msg="Got ping for unexpected node 'mimir-mimir-ingester-zone-a-0-cbaf0b32' from=192.168.154.151:7946"
ts=2025-09-02T20:35:33.74584719Z caller=log.go:245 level=warn msg="Got ping for unexpected node 'mimir-mimir-ingester-zone-a-0-cbaf0b32' from=192.168.155.4:7946"
ts=2025-09-02T20:35:35.35758174Z caller=sanity_check.go:115 level=warn msg="Unable to successfully connect to configured object storage (will retry)" err="blocks storage: unable to successfully send a request to object storage: Get \"https://staging-objectstore-mimir.s3.dualstack.us-gov-west-1.amazonaws.com/blocks/sanity-check-at-startup\": context deadline exceeded"
ts=2025-09-02T20:35:35.749897471Z caller=log.go:245 level=warn msg="Got ping for unexpected node 'mimir-mimir-ingester-zone-a-0-cbaf0b32' from=192.168.154.111:7946"
ts=2025-09-02T20:35:35.750599371Z caller=log.go:245 level=warn msg="Got ping for unexpected node 'mimir-mimir-ingester-zone-a-0-cbaf0b32' from=192.168.155.4:7946"
ts=2025-09-02T20:35:35.75182754Z caller=log.go:245 level=warn msg="Got ping for unexpected node 'mimir-mimir-ingester-zone-a-0-cbaf0b32' from=192.168.150.184:7946"
ts=2025-09-02T20:35:35.75380119Z caller=log.go:245 level=warn msg="Got ping for unexpected node 'mimir-mimir-ingester-zone-a-0-cbaf0b32' from=192.168.153.70:7946"
ts=2025-09-02T20:35:35.754461409Z caller=log.go:245 level=warn msg="Got ping for unexpected node 'mimir-mimir-ingester-zone-a-0-cbaf0b32' from=192.168.152.196:7946"
ts=2025-09-02T20:35:35.754710246Z caller=log.go:245 level=warn msg="Got ping for unexpected node 'mimir-mimir-ingester-zone-a-0-cbaf0b32' from=192.168.152.153:7946"
ts=2025-09-02T20:35:37.548069664Z caller=log.go:245 level=warn msg="Got ping for unexpected node 'mimir-mimir-ingester-zone-a-0-cbaf0b32' from=192.168.150.184:7946"
ts=2025-09-02T20:35:38.744200687Z caller=log.go:245 level=info msg="Suspect mimir-mimir-ingester-zone-a-0-cbaf0b32 has failed, no acks received"
ts=2025-09-02T20:35:39.550449175Z caller=log.go:245 level=warn msg="Got ping for unexpected node 'mimir-mimir-ingester-zone-a-0-cbaf0b32' from=192.168.154.151:7946"
ts=2025-09-02T20:35:39.552438798Z caller=log.go:245 level=warn msg="Got ping for unexpected node 'mimir-mimir-ingester-zone-a-0-cbaf0b32' from=192.168.154.111:7946"
ts=2025-09-02T20:35:39.554156839Z caller=log.go:245 level=warn msg="Got ping for unexpected node 'mimir-mimir-ingester-zone-a-0-cbaf0b32' from=192.168.153.116:7946"
ts=2025-09-02T20:35:41.924612371Z caller=log.go:245 level=info msg="Marking mimir-mimir-alertmanager-0-90f5ad2f as failed, suspect timeout reached (2 peer confirmations)"
ts=2025-09-02T20:35:43.74458014Z caller=log.go:245 level=info msg="Suspect mimir-mimir-alertmanager-0-90f5ad2f has failed, no acks received"
ts=2025-09-02T20:35:43.854616686Z caller=memberlist_client.go:552 level=info msg="initiating cleanup of obsolete entries"
ts=2025-09-02T20:35:45.46933321Z caller=log.go:245 level=info msg="Marking mimir-mimir-store-gateway-zone-a-0-993b5da3 as failed, suspect timeout reached (2 peer confirmations)"
ts=2025-09-02T20:35:48.78153204Z caller=sanity_check.go:115 level=warn msg="Unable to successfully connect to configured object storage (will retry)" err="blocks storage: unable to successfully send a request to object storage: Get \"https://staging-objectstore-mimir.s3.dualstack.us-gov-west-1.amazonaws.com/blocks/sanity-check-at-startup\": context deadline exceeded"
ts=2025-09-02T20:35:49.236706351Z caller=log.go:245 level=warn msg="Got ping for unexpected node 'mimir-mimir-ingester-zone-a-0-cbaf0b32' from=192.168.152.196:7946"
ts=2025-09-02T20:35:51.238040801Z caller=log.go:245 level=warn msg="Got ping for unexpected node 'mimir-mimir-ingester-zone-a-0-cbaf0b32' from=192.168.154.151:7946"
ts=2025-09-02T20:35:51.239883987Z caller=log.go:245 level=warn msg="Got ping for unexpected node 'mimir-mimir-ingester-zone-a-0-cbaf0b32' from=192.168.154.184:7946"
ts=2025-09-02T20:35:51.240515844Z caller=log.go:245 level=warn msg="Got ping for unexpected node 'mimir-mimir-ingester-zone-a-0-cbaf0b32' from=192.168.152.244:7946"