Cannot use hardened rook/cassandra:v1.5.11
Summary
My goal is to deploy Rook Cassandra nodes using images in Ironbank in AWS.
BONUS: Enable istio injection on each namespace from the latest BigBang release.
registry1.dso.mil/ironbank/rook/cassandra:v1.5.11
Approach
- Deploy rook-cass-operator using
operator.yaml
and therook/cassandra:v1.5.11
image from Ironbank - Deploy rook-cass-cluster using
cluster.yaml
and thecassandra:3.11.6
image
Note: rook-cass-cluster also uses rook/cassandra:v1.5.11
in an initContainer called rook-install
Steps to reproduce
- Create a Namespace and Secret with Ironbank registry creds 2 Add imagePullSecrets to operator.yaml
- Apply https://repo1.dso.mil/dsop/rook/cassandra/-/blob/development/deployment/operator.yaml
- Create another Namespace and Secret with Ironbank registry creds
- Create storage class and specify name in cluster.yaml
- Apply https://repo1.dso.mil/dsop/rook/cassandra/-/blob/development/deployment/cluster.yaml
What is the current bug behavior?
Note: rook-cass-operator and rook-cass-cluster deploy successfully with unhardened Internet images.
TEST CASE 1:
- rook-cass-operator:
registry1.dso.mil/ironbank/rook/cassandra:v1.5.11
(Istio injection disabled) - rook-cass-cluster (rook-install container):
registry1.dso.mil/ironbank/rook/cassandra:v1.5.11
(Istio injection enabled) - rook-cass-cluster (cassandra container):
cassandra:3.11.6
(unavailable in Ironbank)
Note: By enabling istio injection on the rook-cass-cluster namespace, the private-registry imagePullSecret is appended to the manifest so I can pull from Ironbank in the rook-install initContainer (otherwise there is no imagePullSecret).
Using the Ironbank imagePullSecret, the rook-cass-operator successfully deploys using the hardened image.
Problem: User 1001 does not have permission to execute a 'cp' command in rook-install initContainer.
cp: cannot open '/sidecar/plugins/jmx_prometheus.jar' for reading: Permission denied
cp: cannot open '/sidecar/plugins/jolokia.jar' for reading: Permission denied
cp: cannot open '/sidecar/tini' for reading: Permission denied
Copy command found in rook/pkg/operator/cassandra/controller/util/resource.go
TEST CASE 2:
- rook-cass-operator:
registry1.dso.mil/ironbank/rook/cassandra:v1.5.11
(Istio injection enabled) - rook-cass-cluster (rook-install container):
registry1.dso.mil/ironbank/rook/cassandra:v1.5.11
(Istio injection enabled) - rook-cass-cluster (cassandra container):
cassandra:3.11.6
(unavailable in Ironbank)
Problem: Istio injection on rook-cass-operator namespace causes error below.
2021-05-31 15:01:28.587945 I | rookcmd: starting Rook v1.5.11 with arguments '/usr/local/bin/rook cassandra operator'
2021-05-31 15:01:28.588074 I | rookcmd: flag values: --help=false, --log-flush-frequency=5s, --log-level=INFO, --operator-image=, --service-account=
2021-05-31 15:01:28.588078 I | cassandracmd: starting cassandra operator
failed to get container image: failed to find image for container
Problem: How can cassandra:3.11.6
be pulled from anywhere except the Internet and is there an effort for it be hardened and placed in Ironbank?
I have been unsuccessful adjusting the following parameters to create an imagePullSecret and change where the image is being pulled. How can cassandra:3.11.6
be pulled from a custom registry?
cluster.yaml
mode: cassandra
version: "3.11.6"
What is the expected correct behavior?
Deploy single replica of rook-cass-operator and any number of rook-cass-cluster nodes using Ironbank images and Istio injection on both namespaces.
Relevant logs and/or screenshots
Relevant portions of logs are displayed above, but full logs can be reproduced at your request.
Possible fixes
Suggestion for test case 1: Make /sidecar
recursively readable by user 1001. Afterward, testing may produce new errors to be resolved.
Defintion of Done
-
Bug has been identified and corrected within the container