UNCLASSIFIED - NO CUI

`nginx-alpine;1.26.0` image is dysfunctional

Summary

We use nginx-alpine provided by IronBank as base image for our proxy.

Since the upgrade to the version 1.26.0, the system is not working anymore. The web server refuses to start, where the same configuration was fine in the previous image.

For our image built outside of IronBank, we use nginx:1.26.0-alpine3.19 without any issue.

We investigated, and found the server { } block was not working at all and the cause of the error. We tested that with the base image nginx-alpine, and confirmed it was located only in it and not in our code.

Steps to reproduce

For this reproducer, we will use a very simple configuration:

# default.conf
server {
    listen 9080;

    location / {
        root /;
    }
}

Let's start with the working version, using version 1.25.3:

$ ls
default.conf
$ docker run -it --rm -v "$(pwd):/etc/nginx/conf.d/" registry1.dso.mil/ironbank/opensource/nginx/nginx-alpine:1.25.3
/docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration
/docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/
/docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh
10-listen-on-ipv6-by-default.sh: info: Getting the checksum of /etc/nginx/conf.d/default.conf
10-listen-on-ipv6-by-default.sh: info: /etc/nginx/conf.d/default.conf differs from the packaged version
/docker-entrypoint.sh: Launching /docker-entrypoint.d/20-envsubst-on-templates.sh
20-envsubst-on-templates.sh: ERROR: /etc/nginx/templates exists, but /etc/nginx/conf.d is not writable
/docker-entrypoint.sh: Launching /docker-entrypoint.d/30-tune-worker-processes.sh
/docker-entrypoint.sh: Configuration complete; ready for start up
2024/06/28 13:51:05 [notice] 1#1: using the "epoll" event method
2024/06/28 13:51:05 [notice] 1#1: nginx/1.25.3
2024/06/28 13:51:05 [notice] 1#1: built by gcc 12.2.1 20220924 (Alpine 12.2.1_git20220924-r10)
2024/06/28 13:51:05 [notice] 1#1: OS: Linux 6.6.22-linuxkit
2024/06/28 13:51:05 [notice] 1#1: getrlimit(RLIMIT_NOFILE): 1048576:1048576
2024/06/28 13:51:05 [notice] 1#1: start worker processes
2024/06/28 13:51:05 [notice] 1#1: start worker process 31
2024/06/28 13:51:05 [notice] 1#1: start worker process 32
2024/06/28 13:51:05 [notice] 1#1: start worker process 33

Everything works well. Now we will do the same for the version 1.26.0

$ ls
default.conf
$ docker run -it --rm -v "$(pwd):/etc/nginx/conf.d/" registry1.dso.mil/ironbank/opensource/nginx/nginx-alpine:1.26.0
/docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration
/docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/
/docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh
10-listen-on-ipv6-by-default.sh: info: Getting the checksum of /etc/nginx/conf.d/default.conf
10-listen-on-ipv6-by-default.sh: info: /etc/nginx/conf.d/default.conf differs from the packaged version
/docker-entrypoint.sh: Launching /docker-entrypoint.d/20-envsubst-on-templates.sh
20-envsubst-on-templates.sh: ERROR: /etc/nginx/templates exists, but /etc/nginx/conf.d is not writable
/docker-entrypoint.sh: Launching /docker-entrypoint.d/30-tune-worker-processes.sh
/docker-entrypoint.sh: Configuration complete; ready for start up
$ # the process has stoped and did an `exit 1`

if we do a check, using the nginx -t command, we can clearly see there is an issue:

$ docker run -it --rm -v "$(pwd):/etc/nginx/conf.d/" --entrypoint=sh registry1.dso.mil/ironbank/opensource/nginx/nginx-alpine:1.26.0 -c 'nginx -t'
2024/06/28 13:58:53 [info] 1#1: ngx_hkdf_expand(av_token_key) failed
nginx: configuration file /etc/nginx/nginx.conf test failed

If we remote the very simple default.conf, it works again… but it's a bit difficult to configure nginx without the server { } directive. Of course, the same command with 1.25.3 is successful.

What is the current bug behavior?

Impossible to start the nginx web server if the configuration contains the server {} directive

What is the expected correct behavior?

Works as the official nginx:1.26.0-alpine3.19 or the previous 1.25.3.

Relevant logs and/or screenshots

Everything is available in the reproducer. The only error we see is related to this, but we don't know if it's a red herring or not.

Possible fixes

Don't know. In the meantime, we will have to downgrade to the last compatible base image, eg 1.25.3.

Tasks

  • Bug has been identified and corrected within the container

Please read the Iron Bank Documentation for more info

UNCLASSIFIED - NO CUI