Attention Iron Bank Customers: On March 27, 2025, we are moving SBOM artifacts from the Anchore Scan job to the Build job to streamline the container hardening pipeline. If you currently download SBOMs from the Anchore Scan job, you can still get them from the Build job and from other sources, including IBFE and image attestations.
Big Bang has several policies for Kubernetes resources to ensure best practices and security. For example, images must be pulled from Iron Bank, or containers must be run as non-root. These policies are currently enforced by [OPA Gatekeeper](https://repo1.dso.mil/big-bang/product/packages/policy), which gets deployed as the first package in Big Bang.
Big Bang has several policies for Kubernetes resources to ensure best practices and security. For example, images must be pulled from Iron Bank, and containers must be run as non-root. These policies are currently enforced by [Kyverno](https://repo1.dso.mil/big-bang/product/packages/kyverno), which replaces OPA Gatekeeper as the primary policy management tool in Big Bang.
When integrating your package, you must adhere to the policies that are enforced or your resources will be denied by the Kubernetes admission controller. The following is how to identify and fix policy violations.
When integrating your package, you must adhere to the policies enforced by Kyverno, or your resources will be denied by the Kubernetes admission controller. This document will guide you on how to identify and resolve policy violations using Kyverno.
## Prerequisites
-a K8s cluster with Big Bang installed.
-A K8s cluster with Big Bang installed.
- cluster admin access to the cluster with [kubectl](https://kubernetes.io/docs/tasks/tools/).
## Integration
### 1. Deploying a Policy Enforcement Tool (OPA Gatekeeper)
### 1. Deploying a Policy Enforcement Tool (Kyverno)
The policy enforcement tool is deployed as the first package in the default Big Bang configuration. This is so that the enforcement tool can effectively protect the cluster from the start. Your package will be deployed on top of the Big Bang enforcement tool. The policy enforcement tool will control your package's access to the cluster.
Kyverno is deployed as the first package in the default Big Bang configuration. This setup allows Kyverno to protect the cluster from the start by enforcing policies on all resources. Your package will interact with the cluster under the governance of Kyverno's policy engine.
### 2. Identifying Violations Found on Your Application
In the following section, you will be shown how to identify violations found in your package. The app [PodInfo](https://repo1.dso.mil/big-bang/apps/sandbox/podinfo) will be used for all of the examples. Gatekeeper has three enforcement actions `deny`, `dryrun`, and `warn`. Only `deny` will prohibit access to the cluster, but the `warn` and `dryrun` constraints should be fixed as well as they are generally best practice.
In the following section, you will learn how to identify violations found in your package. The app [PodInfo](https://repo1.dso.mil/big-bang/apps/sandbox/podinfo) will be used for all examples. Kyverno provides detailed reports on policy violations. To check for any issues with your package, use the following commands:
In this example we will be attempting to install PodInfo onto our cluster:
To deploy an application like PodInfo and check for violations:
```bash
➜ helm install flux-podinfo chart
NAME: flux-podinfo
LAST DEPLOYED: Mon Feb 14 11:24:26 2022
NAMESPACE: default
STATUS: deployed
REVISION: 1
NOTES:
1. Get the application URL by running these commands:
echo"Visit http://127.0.0.1:8080 to use your application"
resource Deployment/default/flux-podinfo was blocked due to the following policies
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/flux-podinfo 0/1 0 0 19s
disallow-root:
autogen-ensure-non-root: 'validation error: Running as root is not allowed. rule
autogen-ensure-non-root failed at path /spec/template/spec/containers/0/securityContext/'
NAME DESIRED CURRENT READY AGE
replicaset.apps/flux-podinfo-84d5bccfd6 1 0 0 19s
```
There was a policy violation, and Kyverno blocked the deployment while providing feedback.
In order to get more information on why our deployment isn't available, we can check the events of the K8s cluster. This will show us if there are policy violations, but will also reveal any other issues in our cluster.
Kyverno’s output is structured to give clear information about which policies were violated and why, making it easy for users to understand the necessary changes to comply with the policy requirements.
```bash
➜ kubectl get events -n default
NAMESPACE LAST SEEN TYPE REASON OBJECT MESSAGE
default 31s Warning FailedCreate replicaset/flux-podinfo-84d5bccfd6 Error creating: admission webhook "validation.gatekeeper.sh" denied the request: [no-privileged-containers] Privileged container is not allowed: podinfo, securityContext: {"privileged": true}
```
We can see that the issue in our example is that our PodInfo application is running containers as privileged.
### 3. Fixing Policy Violations
To get more information as to how to fix this issue we can get the logs of the gatekeeper control plane
Upon identifying the policy violation(s), modify your Kubernetes manifests according to the Kyverno policies. For instance, if a policy requires that no containers run as privileged, you should ensure your deployment manifests respect this rule.
This is going to output a lot of logs to sift through so we can do a simple `grep` command looking for the specific policy denial, in this case no-privileged-containers.
While fixing the violation in the application is preferred, sometimes an exception to the policy is necessary, allowing the violation to remain.
### 3. Fixing Policy Violations
If you require an exception to a policy, please refer to our [exception doc](https://repo1.dso.mil/big-bang/product/packages/policy/-/blob/main/docs/exceptions.md) for more information.
We can see the `constraint_action: deny` indicates that our resource was denied access to the cluster. The `constraint_name` and `constraint_kind` can provide us a way to get more information as to why our resource was denied. Running the following command will help you do so.
```bash
kubectl get <constraint_kind>.constraints.gatekeeper.sh/<constraint_name> -o json | jq '.metadata.annotations'
```
## Validation
Replacing the command with our information give us the following:
After fixing the violation, we can run `helm upgrade flux-podinfo chart`. We can then check all the events in our cluster. This will show us if we've fixed our policy violation, but will also reveal non-policy related issues.
```bash
kubectl get K8sPSPPrivilegedContainer2.constraints.gatekeeper.sh/no-privileged-containers -o json | jq '.metadata.annotations'
```
```json
{
"constraints.gatekeeper/description":"Containers must not run as privileged.",
Lastly, run `kubectl get all -n default` to verify that your deployment was successful.
The annotations provide us documentation information for specific policies as well as the source code to view that policy.
To fix this issue, navigate to your package's `chart/values.yaml` or `deployment.yaml` and remove `privileged: true` or explicitly set it to `false`.
### 4. Exemptions to Policy Exceptions
Fixing the violation in the application is preferred, but sometimes we need to make an exception to the policy and leave the violation in place.
If you require an exception to a policy, please reference our [exception doc](https://repo1.dso.mil/big-bang/product/packages/policy/-/blob/main/docs/exceptions.md) for more information.
After we fixed the violation, we can run `helm upgrade flux-podinfo chart`. We can now check all the events in our cluster. This will show us if we've fixed our policy violation, but will also reveal non-policy related issues.
