1.19.3 image crashes due to permissions issue

Summary

The 1.19.3 image will not start. The logs indicate it is probably a permissions issue.

We hit this issue working through big-bang/product/packages/istio-controlplane#114 (closed) and big-bang/product/packages/istio-operator#55 (closed), and it is blocking us.

Steps to reproduce

This can be reproduced by running this version of the image under Big Bang.

What is the current bug behavior?

Container crashes.

What is the expected correct behavior?

Container does not crash.

Relevant logs and/or screenshots

2023-10-16T20:38:27.540162Z	info	FLAG: --caCertFile=""
2023-10-16T20:38:27.540205Z	info	FLAG: --clusterAliases="[]"
2023-10-16T20:38:27.540210Z	info	FLAG: --clusterID="Kubernetes"
2023-10-16T20:38:27.540213Z	info	FLAG: --clusterRegistriesNamespace="istio-system"
2023-10-16T20:38:27.540216Z	info	FLAG: --configDir=""
2023-10-16T20:38:27.540219Z	info	FLAG: --ctrlz_address="localhost"
2023-10-16T20:38:27.540230Z	info	FLAG: --ctrlz_port="9876"
2023-10-16T20:38:27.540233Z	info	FLAG: --domain="cluster.local"
2023-10-16T20:38:27.540236Z	info	FLAG: --grpcAddr=":15010"
2023-10-16T20:38:27.540241Z	info	FLAG: --help="false"
2023-10-16T20:38:27.540244Z	info	FLAG: --httpAddr=":8080"
2023-10-16T20:38:27.540246Z	info	FLAG: --httpsAddr=":15017"
2023-10-16T20:38:27.540252Z	info	FLAG: --keepaliveInterval="30s"
2023-10-16T20:38:27.540255Z	info	FLAG: --keepaliveMaxServerConnectionAge="30m0s"
2023-10-16T20:38:27.540259Z	info	FLAG: --keepaliveTimeout="10s"
2023-10-16T20:38:27.540261Z	info	FLAG: --kubeconfig=""
2023-10-16T20:38:27.540266Z	info	FLAG: --kubernetesApiBurst="160"
2023-10-16T20:38:27.540272Z	info	FLAG: --kubernetesApiQPS="80"
2023-10-16T20:38:27.540276Z	info	FLAG: --log_as_json="false"
2023-10-16T20:38:27.540280Z	info	FLAG: --log_caller=""
2023-10-16T20:38:27.540284Z	info	FLAG: --log_output_level="default:info"
2023-10-16T20:38:27.540288Z	info	FLAG: --log_rotate=""
2023-10-16T20:38:27.540292Z	info	FLAG: --log_rotate_max_age="30"
2023-10-16T20:38:27.540296Z	info	FLAG: --log_rotate_max_backups="1000"
2023-10-16T20:38:27.540299Z	info	FLAG: --log_rotate_max_size="104857600"
2023-10-16T20:38:27.540304Z	info	FLAG: --log_stacktrace_level="default:none"
2023-10-16T20:38:27.540311Z	info	FLAG: --log_target="[stdout]"
2023-10-16T20:38:27.540324Z	info	FLAG: --meshConfig="./etc/istio/config/mesh"
2023-10-16T20:38:27.540327Z	info	FLAG: --monitoringAddr=":15014"
2023-10-16T20:38:27.540331Z	info	FLAG: --namespace="istio-system"
2023-10-16T20:38:27.540337Z	info	FLAG: --networksConfig="./etc/istio/config/meshNetworks"
2023-10-16T20:38:27.540352Z	info	FLAG: --profile="true"
2023-10-16T20:38:27.540360Z	info	FLAG: --registries="[Kubernetes]"
2023-10-16T20:38:27.540363Z	info	FLAG: --secureGRPCAddr=":15012"
2023-10-16T20:38:27.540366Z	info	FLAG: --shutdownDuration="10s"
2023-10-16T20:38:27.540371Z	info	FLAG: --tls-cipher-suites="[]"
2023-10-16T20:38:27.540374Z	info	FLAG: --tlsCertFile=""
2023-10-16T20:38:27.540376Z	info	FLAG: --tlsKeyFile=""
2023-10-16T20:38:27.540380Z	info	FLAG: --vklog="0"
2023-10-16T20:38:27.548096Z	info	initializing Istiod admin server
Error: failed to create discovery service: error initializing kube client: failed reading mesh config: cannot read mesh config file open ./etc/istio/config/mesh: permission denied
2023-10-16T20:38:27.557405Z	error	failed to create discovery service: error initializing kube client: failed reading mesh config: cannot read mesh config file open ./etc/istio/config/mesh: permission denied

Possible fixes

(If you can, link to the line of code that might be responsible for the problem)

Tasks

• Bug has been identified and corrected within the container

Please read the Iron Bank Documentation for more info

mentioned in issue big-bang/product/packages/istio-operator#55 (closed)

mentioned in issue big-bang/product/packages/istio-controlplane#114 (closed)

mentioned in merge request big-bang/product/packages/istio-controlplane!176 (merged)

mentioned in merge request big-bang/product/packages/istio-operator!89 (merged)

assigned to @colton.freeman

added StatusDoing label

testing this with Jonathan, waiting for him to get access to our staging registry for better tests.

Update.

we did get a working ubi9-minimal version in but working a distroless build for this currently.

The labels for this issue have been adjusted because one or more labels were identified as being incorrectly set.

added OwnerIronbank PRIORITY1 labels

Any update on this issue?

Any update here? We verified a fix: we just needed to add WORKDIR / to the Dockerfile. A test build was pushed to the BB package pipeline, and the pipeline tests passed. Any idea when we can get this re-released to unblock the downstream BB issues?

I am working that now. Should be in master shortly. I will message you on MM when it is available.

added StatusVerification label and removed StatusDoing label

assigned to @alfontaine

closed