@@ -9,105 +9,92 @@ The pluggable constructs in Renovate include:
Renovate handles opening pull requests for out-of-date dependencies and can be customized for individual projects.
## Background
---
In addition to the `Dockerfile`, there are custom Ironbank files that need to be updated when a new version is discovered.
## How we use Renovate in Ironbank
***download.yaml**
***greylist**
***Jenkinsfile**
Renovate is an opensource project which we have implemented in Repo1's Gitlab. Our renovate-bot runs daily so any vendor that wants create their own configuration of Renovate is free to do so. The follow subsequent sections outline how we utilize Renovate on opensource projects that are mangaged by the Container Hardening Team to watch for updates to to upstream resouces in those projects. Ultimately, the vendor is responsible for the updating of their container in Ironbank, the use of Renovate is not required but is an option we provide for their use.
### download.yaml
Renonvate has a capability set of updating dependancies from a wide range of sources, however, due to the nature of our container hardening process we utilize a very narrow subset of Renovate's capabilites. Gitlab projects in Repo1 do not contain any of the application source code, only the process of how that application is hardened into a container image. Therefore, managers and datasource that are targeted at updating source code depenancies are not used.
Ironbank requires contributors to define external dependencies in a `download.yaml` file. The build pipeline executes within a disconnected environment, so dependencies are fetched and added to the build context beforehand. Versions, tags, and digests must be updated for a new release.
The Ironbank pipeline requires all upstream resouces to be declared in the hardening_manifest. Due to this we use Renovate, more specifically a Custom Manager and the Regex Manager, to parse the hardening_manifest for declared versions and check for updates using the outlined datasource in the renovate.json file.
There is a limitation for this usage. Your upstream resource must be stored in a location for which renovate has a datasoure (a full list of supported datasources can be found [here](https://docs.renovatebot.com/modules/datasource/) ). Unfortunatly, this is a limiting factor for some projects.
The pipeline uses the greylist to inject the `BASE_IMAGE` and `BASE_TAG` build arguments. These image tags must also be updated.
It is important to remove any references to a specific application version in the filename of the hardening_manifest.yaml file when downloading artifacts that are not images. This way, if the artifact version is bumped, the COPY statement included in the Dockerfile will be able to handle this updated version, without failing because it attempts to copy an older version which is no longer being downloaded.
### Jenkinsfile
For example, use the filename kubernetes.tar.gz instead of the filename kubernetes-1.18.3 because the Dockerfile COPY kubernetes-1.18.3.tar.gz statement will throw an error that a resource cannot be found if the Kubernetes artifact version is bumped to 1.18.4. COPY kubernetes.tar.gz will handle any version bumps that occur and result in successful copying of the hardening_manifest.yaml artifact for the build.
The Jenkinsfile defines the output image tag. Renovate can be used to track an upstream dependency to mirror version tags.
---
```
@Library('DCCSCR@master') _
dccscrPipeline(version: "10.0.0")
```
### Hardening_manifest
Specifically, there are three pieces of data in the `hardening_manifest` that Renovate watches.
## Additional Changes
- tags
- labels
- resources
It is important to remove any references to a specific application version in the filename of the `download.yaml` file when downloading artifacts that are not images. This way, if the artifact version is bumped, the COPY statement included in the Dockerfile will be able to handle this updated version, without failing because it attempts to copy an older version which is no longer being downloaded.
*Thefollowing are example snippits from the ArgoCD project hardening_manifest that identify these:*
For example, use the filename `kubernetes.tar.gz` instead of the filename `kubernetes-1.18.3` because the Dockerfile `COPY kubernetes-1.18.3.tar.gz` statement will throw an error that a resource cannot be found if the Kubernetes artifact version is bumped to `1.18.4`. `COPY kubernetes.tar.gz` will handle any version bumps that occur and result in successful copying of the `download.yaml` artifact for the build.
```yaml
tags:
-"v1.8.7"
Renovate is also configured to run on the `development` branch of each project so that the versions on `master` are not updated without going through the Iron Bank approval process. It is necessary to change the default branch (Project Settings > Repository > Default branch) to the `development` branch in order for Renovate to function correctly.
A custom **manager** is developed for Ironbank to handle extracting dependencies and datasources from the `download.yaml` file.
A custom **manager** is developed for Ironbank to handle extracting dependencies and datasources from the `hardening_manifest.yaml` file resource section.
`docker`, `github-releases`, and `ruby-gems` **datasources** are supported, but other native datasources can be added as well. The manager parses the `url` key to determine the type of dependency.
`docker` and `github-releases`**datasources** are supported, but other native datasources can be added as well. The manager parses the `url` key to determine the type of dependency.
Renovate includes a regex **manager** that can extract dependencies with a regular expression. This is useful for file formats that do not have an associated manager. Within Ironbank, the regex manager is used to handle updating the `Jenkinsfile` and `.greylist`.
The example `renovate.json` below will update the **version:** string if there is an update to the jboss/keycloak docker image found on docker.io.
```json
{
"fileMatch":["^Jenkinsfile$"],
"matchStrings":[
"version:\\s+\"(?<currentValue>.*?)\""
],
"depNameTemplate":"jboss/keycloak",
"datasourceTemplate":"docker"
}
```
Notice the `depNameTemplate` and `datasourceTemplate` that defines the dependency metadata. The `Jenkinsfile` does not include the dependency name or source, so it must be defined in the manager configuration.
Renovate includes a regex **manager** that can extract dependencies with a regular expression. This is useful for file formats that do not have an associated manager. Within Ironbank, the regex manager is used to handle updating the `hardening_manifest` tags and labels section.
## Usage
The example `renovate.json` below will update the **tags:** and **labels:** strings if there is an update to the argocd docker image found on docker.io.
A `renovate-bot` account is created in GitLab and added as a developer for Ironbank.
A Jenkins job for each repository is created under the `renovate` folder. Each job is configured to run on a schedule such as `H H/4 * * *` (runs once every 4 hours).
Renovate updates the **Dockerfile**`LABEL version=1.5.3` and **Jenkinsfile**`dccscrPipeline(version: 1.5.3)` lines.
```json
{
"regexManagers":[{
"fileMatch":["^Dockerfile$"],
"matchStrings":[
"version=\"(?<currentValue>.*?)\""
],
"depNameTemplate":"argoproj/argocd",
"datasourceTemplate":"docker"
},{
"fileMatch":["^Jenkinsfile$"],
"matchStrings":[
"version:\\s+\"(?<currentValue>.*?)\""
],
"depNameTemplate":"argoproj/argocd",
"datasourceTemplate":"docker"
}]
"reviewers":[
"jeason"
]
}
```
#### dccscr-whitelist/renovate.json
Renovate updates the **argocd.greylist**`"image_tag": "v1.5.3"` line.
```json
"regexManagers":[
{
"fileMatch":["^argocd.greylist$"],
"matchStrings":[
"\"image_tag\": \"(?<currentValue>.*?)\""
],
"depNameTemplate":"argoproj/argocd",
"datasourceTemplate":"docker"
}
]
```
### <a name="github"></a>GitHub
---
#### download.yaml
**Note**
For more indepth documentation and configuration options you can visit the official documentation [here](https://docs.renovatebot.com/configuration-options/).
Renovate updates the **download.yaml**`url` and `validation.value`.
In order to begin using Renovate in your project you must create a renovate.json file in the base of your project. Any project with a renovate.json file our renovate-bot will detect and run against.
Renovate updates the **Dockerfile**`LABEL version=3.4.6` and **Jenkinsfile**`dccscrPipeline(version: 3.4.6)` lines.
If an update detected a merge request will be created with those changes. An Issue will be created and associated with that merge request. It is then required that the normal container hardening process if followed with that merge request. The pipeline must pass, merged into development, and any new CVEs must be justified before requesting it to be merged to master.
```json
{
"regexManagers":[{
"fileMatch":["^Dockerfile$"],
"matchStrings":[
"version=\"(?<currentValue>.*?)\""
],
"depNameTemplate":"etcd-io/etcd",
"datasourceTemplate":"github-releases"
},{
"fileMatch":["^Jenkinsfile$"],
"matchStrings":[
"version:\\s+\"(?<currentValue>.*?)\""
],
"depNameTemplate":"etcd-io/etcd",
"datasourceTemplate":"github-releases"
}]
}
```
## Limitations
#### dccscr-whitelist/renovate.json
### Renovate Datasources
Renovate updates the **etcd.greylist**`"image_tag": "3.4.6"` line.
The base image in this repository does not support many of the native Renovate datasources. The image provides support for the minimum feature set required by Ironbank (docker, github-releases).
```json
"regexManagers":[
{
"fileMatch":["^etcd.greylist$"],
"matchStrings":[
"\"image_tag\": \"(?<currentValue>.*?)\""
],
"depNameTemplate":"etcd-io/etcd",
"datasourceTemplate":"github-releases"
}
]
```
Also metioned before your upstream resource must be stored in a location that for which renovate has a datasoure. Renovate ships with several datasources such as docker, github-releases, and pypi. A vendor may not publish to one of these supported platforms and as such could be a limiting factor. A custom datasource could be implemented to support fetching release information.
### <a name="yum"></a>YUM
### YUM
Renovate does not currently support package managers. An open feature request can be found [here](https://github.com/renovatebot/renovate/issues/6188).
One challenge with package manager support is dealing with **transitive** dependencies. In the Dockerfile below, installing `java-11-openjdk-headless` with `dnf` will include any of its dependencies as well.
```shell
RUN dnf install-y java-11-openjdk-headless &&\
dnf clean all &&\
rm-rf /var/cache/dnf
```
```shell
RUN dnf install -y java-11-openjdk-headless && \
dnf clean all && \
rm -rf /var/cache/dnf
```
In this example, Renovate needs to gather dependency metadata to handle upgrades. But where should it get this data?
There are a few approaches that can be applied to solve this issue:
1. Do no use Renovate! Trigger a new build periodically with `dnf update -y` regardless of whether or not a dependency has been updated. This mechanism could be extended to first pull the existing image and check for updates before triggering a build:
```shell
docker run --rm--entrypoint=/bin/bash -it[repository]/[image]:[tag] dnf list updates
docker run --rm--entrypoint=/bin/bash -it[repository]/[image]:[tag] dnf list updates
```
2. Define a manifest for the image and include it within the repository. Renovate could trigger updates for each dependency in the manifest. For example:
Automated dependency updates present some challenges.
1. Software upgrades are not always as simple as bumping a version. A `Dockerfile` may require additional changes that accompany a version bump.
- How are we made aware of these changes?
- How can we keep these changes in sync?
Software upgrades are not always as simple as bumping a version. A `Dockerfile` may require additional changes that accompany a version bump.
2. No datasource for vendor supply chain. Renovate ships with several datasources such as docker, github-releases, and pypi. A vendor may not publish to one of these supported platforms. A custom datasource must be implemented to support fetching release information.
- How are we made aware of these changes?
- How can we keep these changes in sync?
## Limitations
The base image in this repository does not support many of the native Renovate datasources. The image provides support for the minimum feature set required by Ironbank (docker, github-releases).
Renovate is one tool that can be used to solve some of these issues but additional work needs to be done to move those updats through the hardening process.
