diff --git a/Dockerfile b/Dockerfile index ee6e4762f5e1aa77d83531f18fc2785f9afee320..91cad7a90fbee38972798dce59f05b04b38e8bf2 100644 --- a/Dockerfile +++ b/Dockerfile @@ -2,7 +2,7 @@ ARG BASE_REGISTRY=registry1.dso.mil ARG BASE_IMAGE=ironbank/opensource/nodejs/nodejs14 ARG BASE_TAG=14.15.5 -FROM renovate/renovate:24.119.5-slim as builder +FROM renovate/renovate:25.21.11-slim as builder FROM ${BASE_REGISTRY}/${BASE_IMAGE}:${BASE_TAG} diff --git a/README.md b/README.md index 2e594feb56008ea73b6622c4e3f5504a2ec2758d..7890de15fef56a57da19623bf237bc8823b664a8 100644 --- a/README.md +++ b/README.md @@ -9,8 +9,6 @@ The pluggable constructs in Renovate include: Renovate handles opening Gitlab Merge Requests for out-of-date dependencies and can be customized for individual projects. ---- - ## How we use Renovate in Ironbank Renovate is an opensource project which we have implemented in Repo1's Gitlab. Our renovate-bot runs daily so any vendor that wants to create their own configuration of Renovate is free to do so. The follow subsequent sections outline how we utilize Renovate on opensource projects that are mangaged by the Container Hardening Team to watch for updates to upstream resouces in those projects. Ultimately, the vendor is responsible for the updating of their container in Ironbank, the use of Renovate is not required but is an option we provide for their use. @@ -20,19 +18,17 @@ Renonvate has a capability set of updating dependancies from a wide range of sou The Ironbank pipeline requires all upstream resouces to be declared in the hardening_manifest. Due to this we use Renovate, more specifically a custom manager call `Ironbank` and the `Regex Manager`, to parse the hardening_manifest for declared versions and check for updates using the supported datasources in our `Ironbank Manager` or what is outlined in the renovate.json file for the `Regex Manager`. --- -**Note** - There is a limitation for this usage. Your upstream resource must be stored in a location for which renovate has a datasoure (a full list of supported datasources can be found [here](https://docs.renovatebot.com/modules/datasource/) ). Unfortunatly, this is a limiting factor for some projects. - - -- +**Note** - It is important to remove any references to a specific application version in the filename of the hardening_manifest.yaml file when downloading artifacts that are not images. This way, if the artifact version is bumped, the COPY statement included in the Dockerfile will be able to handle this updated version, without failing because it attempts to copy an older version which is no longer being downloaded. +There is a limitation for this usage. Your upstream resource must be stored in a location for which renovate has a datasoure (a full list of supported datasources can be found [here](https://docs.renovatebot.com/modules/datasource/) ). Unfortunatly, this is a limiting factor for some projects. For these cases we use an alternative approach, while not as elegant as Renovate it works. - For example, use the filename kubernetes.tar.gz instead of the filename kubernetes-1.18.3 because the Dockerfile COPY kubernetes-1.18.3.tar.gz statement will throw an error that a resource cannot be found if the Kubernetes artifact version is bumped to 1.18.4. COPY kubernetes.tar.gz will handle any version bumps that occur and result in successful copying of the hardening_manifest.yaml artifact for the build. +> **_Important:_** It is important to remove any references to a specific application version in the filename of the hardening_manifest.yaml file when downloading artifacts that are not images. This way, if the artifact version is bumped, the COPY statement included in the Dockerfile will be able to handle this updated version, without failing because it attempts to copy an older version which is no longer being downloaded. +> **example:** For example, use the filename kubernetes.tar.gz instead of the filename kubernetes-1.18.3 because the Dockerfile COPY kubernetes-1.18.3.tar.gz statement will throw an error that a resource cannot be found if the Kubernetes artifact version is bumped to 1.18.4. COPY kubernetes.tar.gz will handle any version bumps that occur and result in successful copying of the hardening_manifest.yaml artifact for the build. --- -### Hardening_manifest +## Hardening_manifest Specifically, there are three pieces of data in the `hardening_manifest` that Renovate needs to watch and update. @@ -55,34 +51,38 @@ resources: ``` -### Ironbank Manager +We use two Renovate managers to fully update the `hardening_manifest`: `Ironbank` and `Regex` + +## IronbankManager -The `Ironbank Manager` is a custom **manager** that was developed for use in Repo1 to handle extracting dependencies from the `hardening_manifest.yaml` file resource section. +The `Ironbank Manager` is a custom **manager** that was developed for use in Repo1 to handle extracting dependencies from the `hardening_manifest.yaml` file resource section. If you have a hardening_manifest.yaml file and a renovate.json file the `Ironbank Manager` will automatically parse the resource section and look for updates for each resource that comes from a supported datasource. -`docker`, `github-releases`, `github-tags`, `ruby-gems`, and `pypi`**datasources** are currently supported, but other native datasources can be added as well. The manager parses the `url` key to determine the type of dependency. +`docker`, `github-releases`, `github-tags`, `ruby-gems`, and `pypi` **datasources** are currently the only supported datasource, but other native datasources can be added as well upon request. The manager parses the `url` key to determine the type of dependency. ```yaml - url: docker://{registry}/{repo}@{digest} -- url: https://github.com/${repo}/releases/download/${version}/${archive} +- url: https://github.com/${repo}/.../${version}/${archive} - url: https://rubygems.org/downloads/{import}-{version}.gem - url: https://files.pythonhosted.org/../.../{import}-{version}...whl ``` ---- -**Note** - - In order for you to take advantage of the custom Ironbank Manager you only need to have an empty renovate.json file in the base of your Gitlab project. The Ironbank Manager will parse the hardening_manifest (if it exists) and check for updates for supported sources by default without any additional configuration. +Any other datasource in your hardening_manifest.yaml file will not be automatically renovated. When selecting where you are pulling your resources from please take this into consideration, all things being equal we prefer you pull from one of the supported datasources. --- -### Regex Manager +**Note** +In order for you to take advantage of the custom Ironbank Manager you only need to have an empty renovate.json file in the base of your Gitlab project. The existance of the renovate.json file is what we use to identify projects to run the Renovate-bot against. The Ironbank Manager will parse the hardening_manifest (if it exists) and check for updates for supported sources by default without any additional configuration. + +## RegexManager -Renovate includes a regex **manager** that can extract dependencies with a regular expression. This is useful for file formats that do not have an associated manager. Within Ironbank, the regex manager is used to handle updating the `hardening_manifest` tags and labels section. +Renovate includes a **regex manager** that can extract dependencies with a regular expression. This is useful for file formats that do not have an associated manager. Within Ironbank, the regex manager is used to handle updating the `hardening_manifest` **tags** and **labels** section. The example `renovate.json` below will update the **tags:** and **labels:** strings if there is an update to the argocd docker image found on docker.io. ```json - { +{ + "baseBranches": ["development"], + "regexManagers": [ "fileMatch": [ "^hardening_manifest.yaml$" ], @@ -102,13 +102,56 @@ The example `renovate.json` below will update the **tags:** and **labels:** stri "depNameTemplate": "argoproj/argocd", "datasourceTemplate": "docker" } + ] +} ``` -Notice the `depNameTemplate` and `datasourceTemplate` that defines the dependency metadata. +> **_Important:_** You only need to add the `Regex Manager` to your renovate.json file if your main resource (that is the resource that dictates the tag and label) comes from a supported datasource. + +There are cases where the main resource does not come from a supported datasource but other supporting resource in the hardening_manifest do have supporting datasources. In those cases you will still need a renovate.json file but you won't use a `Regex Manager`. See [Workflow](##RenovateWorkflow) on minimum renovate.json requirements + +When using the `Regex Manager` you will need to provide the what datasource it is and the dependancy name. Notice the `depNameTemplate` and `datasourceTemplate` that defines this information in the above example. + +--- +**Note** +The following examples can help you determine what you will put for the `depNameTemplate` and `datasourceTemplate` + + Docker: + + if from docker hub the depNameTemplate will be the docker hub path + + - "depNameTemplate": "argoproj/argocd", + "datasourceTemplate": "docker" + + any other docker registry the depNameTemplate will be the full path + + - "depNameTemplate": "gcr.io/tekton-releases/github.com/tektoncd/triggers/cmd/controller", + "datasourceTemplate": "docker" + + Github: + + if the source is github and the url contains 'archive' or 'tags' you will use github-tags + as the datasourceTemplate and github path as the depNameTemplate + + - "depNameTemplate": "postgres/postgres", + "datasourceTemplate": "github-tags" + + else it will be github-releases + + - "depNameTemplate": "gohugoio/hugo", + "datasourceTemplate": "github-releases" + + Pypi and Rubygems: + + These will just the name of the package depNameTemplate + with the corrisponding datasourceTemplate + + - "depNameTemplate": "fluentd", + "datasourceTemplate": "rubygems" ### Major Versions -Repositories that a pinned to a major version (i.e. `dsop/opensource/redis/redis6`) should disable major revision bumps. +Repositories that a pinned to a major version (i.e. `postgresql10`) should disable major revision bumps. ```json { @@ -120,7 +163,7 @@ Repositories that a pinned to a major version (i.e. `dsop/opensource/redis/redis ### Reviewers -Add reviewers for each repository so `MAINTAINERS` will be notified of a new pull request. Renovate can also be configured to automerge pull requests. +Add reviewers for each repository so `MAINTAINERS` will be notified of a new pull request. ```json { @@ -130,16 +173,41 @@ Add reviewers for each repository so `MAINTAINERS` will be notified of a new pul } ``` +### Automerge + +Renovate can also be configured to automerge pull requests (currently under testing). + +```json +{ + "automerge": true, + "gitLabAutomerge": true, +} +``` + --- **Note** -For more indepth documentation and configuration options you can visit the official documentation [here](https://docs.renovatebot.com/configuration-options/). +Renovate is very configurable and provides many configration options that can be put in the renovate.json file. For more indepth documentation and configuration options you can visit the official documentation [here](https://docs.renovatebot.com/configuration-options/). + +## RenovateWorkflow + +In order to begin using Renovate in your project you must have a renovate.json file in the base of your project. Any project with a renovate.json file the renovate-bot will detect and run against it. --- -## Renovate Workflow +**Note** This is an example of the most basic renovate.json you can have: + +```json +{ + "baseBranches": ["development"] +} +``` + +This will ensure the `Ironbank Manager` will run against your resources list in your `hardening_manifest` without updating either the tag or labels. + +--- -In order to begin using Renovate in your project you must create a renovate.json file in the base of your project. Any project with a renovate.json file the renovate-bot will detect and run against it. +You only need to add a renovate.json file if there are resources in your `hardening_manifest` file that have supporting [datasources](##IronbankManager). You must also add your [regex managers](##RegexManager) to update the tag and label to the correct version if your main resource has a supported datasource. If your main resouce has a supported datasource and you have proper regex managers in the renovate.json file any updates to that resource will produce a completely updated hardening_manifest in the generated merge request in Gitlab. If an update is detected a merge request will be created with those changes. An Issue will be created and associated with that merge request. It is then required that the normal container hardening process if followed with that merge request. The pipeline must pass, merged into development, and any new CVEs must be justified before requesting it to be merged to master. @@ -147,7 +215,7 @@ If an update is detected a merge request will be created with those changes. An ### Renovate Datasources -The base image in this repository does not support many of the native Renovate datasources. The image provides support for the minimum feature set required by Ironbank (docker, github-releases). +The base image in this repository does not support many of the native Renovate datasources. The image provides support for the minimum feature set required by Ironbank (eg, docker, github-releases, github-tags, rubygems, pypi). Also metioned before your upstream resource must be stored in a location for which renovate has a datasoure. Renovate ships with several datasources such as docker, github-releases, rubygems, and pypi. A vendor's upstream resource may not publish to one of these supported platforms and as such could be a limiting factor for the use of Renovate. A custom datasource would have to be implemented to support fetching release information for these resources. diff --git a/hardening_manifest.yaml b/hardening_manifest.yaml index 6c8165d0da64c2c13d128db8f7e37c732a19d3a3..b2918f55a9e4a28e5e58a512f531ddbdb4a9895f 100644 --- a/hardening_manifest.yaml +++ b/hardening_manifest.yaml @@ -8,13 +8,13 @@ name: "container-hardening-tools/renovate/renovate" # The most specific version should be the first tag and will be shown # on ironbank.dsop.io tags: -- "24.119.5" +- "25.21.11" - "latest" # Build args passed to Dockerfile ARGs args: BASE_IMAGE: "opensource/nodejs/nodejs14" - BASE_TAG: "14.16.1" + BASE_TAG: "14.17.0" # Docker image labels labels: @@ -27,7 +27,7 @@ labels: org.opencontainers.image.url: "https://github.com/renovatebot/renovate" ## Name of the distributing entity, organization or individual org.opencontainers.image.vendor: "WhiteSource" - org.opencontainers.image.version: "24.119.5" + org.opencontainers.image.version: "25.21.11" ## Keywords to help with search (ex. "cicd,gitops,golang") mil.dso.ironbank.image.keywords: "automation,dependency,updates" ## This value can be "opensource" or "commercial" @@ -37,8 +37,8 @@ labels: # List of resources to make available to the offline build context resources: -- tag: renovate/renovate:24.119.5-slim - url: docker://docker.io/renovate/renovate@sha256:18e6630668bbc6cdf7f953a30b7ecbe101b07cb012347d7afc0bbec12eceeeb0 +- tag: renovate/renovate:25.21.11-slim + url: docker://docker.io/renovate/renovate@sha256:64a443248fb82639515f2475d0d6d2d9bc350089a86929a31dbe371c1648198d - filename: helm-docs.tar.gz url: https://github.com/norwoodj/helm-docs/releases/download/v1.5.0/helm-docs_1.5.0_Linux_x86_64.tar.gz validation: