Image missing multiple resources required to function
Summary
Image does not function as an EFS storage provider.
Steps to reproduce
Basic testing of functionality can be performed using docker
on an AWS EC2 instance.
Along with the EC2 instance an EFS filesystem with resolvable DNS name is required (but won't actually be used).
What is the current bug behavior?
A number of underlying issues have been identified, which are listed here in sequential order:
- The container terminates if certain configuration files are not made available via mounts
- The
amazon-efs-mount-watchdog
process doesn't start because the executable doesn't exist - The
amazon-efs-mount-watchdog
process fails because a directory it writes a log to doesn't exist (unless manually mounted) - EFS filesystems can't be mounted because the
mount.efs
script doesn't exist - The
mount.efs
script hashbang doesn't execute because the hashbang can't be parsed as-is on the RHEL base image - EFS filesystems can't be mounted because it depends on
nfs-utils
, which is not installed
Refer to the "Relevant logs and/or screenshots" section below for more details.
What is the expected correct behavior?
The following is a list of expected behavior that is witnessed when using the upstream amazon image:
- The container uses default configuration if none is mounted as a volume
- The
amazon-efs-mount-watchdog
process exists and runs even if a log directory isn't mounted - The
mount.efs
script exists and executes, allowing mounting of EFS filesystems
Refer to the "Relevant logs and/or screenshots" section below for more details.
Relevant logs and/or screenshots
The following tests were run on a CentOS 7 EC2 instance, where:
-
<DOCKER_MIRROR>
is a token for the docker mirror used to pull Ironbank containers -
<EFS_DNS>
is a token for the dns name of the EFS filesystem created for testing -
<CENTOS_MIRROR>
is a token for the CentOS package repository mirror (I usedhttps://mirror.aarnet.edu.au/pub/centos/8-stream/BaseOS/x86_64/os/
)
Expected Behavior
Demonstration of expected behavior using the upstream amazon image. Note:
- No volumes were mounted and the container did not terminate
- Two processes are running
- There are executables for mounting efs and nfs
- The container is kept running during these test examples so that files can be copied from it
$ aws_id=$(docker run -d <DOCKER_MIRROR>/amazon/aws-efs-csi-driver:release-1.3)
$ docker logs $aws_id
I0816 04:05:28.339981 1 config_dir.go:62] Mounted directories do not exist, creating directory at '/etc/amazon/efs'
I0816 04:05:31.458846 1 driver.go:140] Did not find any input tags.
I0816 04:05:31.459941 1 driver.go:113] Registering Node Server
I0816 04:05:31.459960 1 driver.go:115] Registering Controller Server
I0816 04:05:31.459987 1 driver.go:118] Starting efs-utils watchdog
I0816 04:05:31.460136 1 efs_watch_dog.go:215] Copying /etc/amazon/efs/efs-utils.conf since it doesn't exist
I0816 04:05:31.460307 1 efs_watch_dog.go:215] Copying /etc/amazon/efs/efs-utils.crt since it doesn't exist
I0816 04:05:31.461115 1 driver.go:124] Starting reaper
I0816 04:05:31.461143 1 driver.go:127] Listening for connections on address: &net.UnixAddr{Name:"/tmp/csi.sock", Net:"unix"}
$ docker top $aws_id
UID PID PPID C STIME TTY TIME CMD
root 61891 61872 0 04:10 ? 00:00:00 /bin/aws-efs-csi-driver
root 61926 61891 0 04:10 ? 00:00:00 python3 /usr/bin/amazon-efs-mount-watchdog
$ docker exec -it $aws_id /bin/bash
bash-4.2# cat /var/log/amazon/efs/mount-watchdog.log
2022-08-16 04:05:31 UTC - INFO - amazon-efs-mount-watchdog, version 1.32.1, is enabled and started
bash-4.2# mkdir /tmp/mp
bash-4.2# mount -t efs <EFS_DNS> /tmp/mp
b'mount.nfs4: Operation not permitted'
bash-4.2# ls -l /sbin/mount*
-rwxr-xr-x. 1 root root 118171 Apr 27 23:42 /sbin/mount.efs
-rwsr-xr-x. 1 root root 113272 Aug 17 2018 /sbin/mount.nfs
lrwxrwxrwx. 1 root root 9 Apr 27 23:42 /sbin/mount.nfs4 -> mount.nfs
-rwxr-xr-x. 1 root root 35809 Aug 17 2018 /sbin/mountstats
bash-4.2# exit
exit
Issues
- Attempting to run the ironbank container results in it terminating as a result of missing configuration
$ id=$(docker run -d <DOCKER_MIRROR>/ironbank/opensource/kubernetes-sigs/aws-efs-csi-driver:release-1.3)
$ docker logs $id
I0816 04:12:01.282940 1 config_dir.go:62] Mounted directories do not exist, creating directory at '/etc/amazon/efs'
I0816 04:12:04.449909 1 driver.go:140] Did not find any input tags.
I0816 04:12:04.450281 1 driver.go:113] Registering Node Server
I0816 04:12:04.450301 1 driver.go:115] Registering Controller Server
I0816 04:12:04.450316 1 driver.go:118] Starting efs-utils watchdog
F0816 04:12:04.450359 1 main.go:65] open /etc/amazon/efs-static-files/: no such file or directory
$ docker top $id
Error response from daemon: Container 58ec282d48a5c07e98f95562116dd1561b5841adbf0bdd8f3573e8788f3f6a09 is not running
$ docker rm $id > /dev/null
- This can be avoided by making the default configurations from the amazon container available. While the container doesn't terminate when this is done, it can be noted that the
amazon-efs-mount-watchdog
process fails to start
$ mkdir files
$ docker cp $aws_id:/etc/amazon/efs-static-files/efs-utils.conf files/
$ docker cp $aws_id:/etc/amazon/efs-static-files/efs-utils.crt files/
$ id=$(docker run -d -v $(pwd)/files:/etc/amazon/efs-static-files <DOCKER_MIRROR>/ironbank/opensource/kubernetes-sigs/aws-efs-csi-driver:release-1.3)
$ docker logs $id
I0816 04:17:18.078792 1 config_dir.go:62] Mounted directories do not exist, creating directory at '/etc/amazon/efs'
I0816 04:17:21.251746 1 driver.go:140] Did not find any input tags.
I0816 04:17:21.252094 1 driver.go:113] Registering Node Server
I0816 04:17:21.252113 1 driver.go:115] Registering Controller Server
I0816 04:17:21.252131 1 driver.go:118] Starting efs-utils watchdog
I0816 04:17:21.252701 1 efs_watch_dog.go:215] Copying /etc/amazon/efs/efs-utils.conf since it doesn't exist
I0816 04:17:21.252887 1 efs_watch_dog.go:215] Copying /etc/amazon/efs/efs-utils.crt since it doesn't exist
I0816 04:17:21.253505 1 driver.go:124] Starting reaper
I0816 04:17:21.253531 1 driver.go:127] Listening for connections on address: &net.UnixAddr{Name:"/tmp/csi.sock", Net:"unix"}
E0816 04:17:21.253542 1 efs_watch_dog.go:284] Process amazon-efs-mount-watchdog exits exec: "amazon-efs-mount-watchdog": executable file not found in $PATH
$ docker top $id
UID PID PPID C STIME TTY TIME CMD
root 62358 62339 0 04:17 ? 00:00:00 /bin/aws-efs-csi-driver
$ docker stop $id > /dev/null
$ docker rm $id > /dev/null
- Making the
amazon-efs-mount-watchdog
executable from the amazon container available allows the process to be executed, however it constantly fails because the log directory doesn't exist
$ docker cp $aws_id:/usr/bin/amazon-efs-mount-watchdog .
$ id=$(docker run -d -v $(pwd)/files:/etc/amazon/efs-static-files -v $(pwd)/amazon-efs-mount-watchdog:/usr/bin/amazon-efs-mount-watchdog <DOCKER_MIRROR>/ironbank/opensource/kubernetes-sigs/aws-efs-csi-driver:release-1.3)
$ docker logs $id
I0816 04:20:43.797643 1 config_dir.go:62] Mounted directories do not exist, creating directory at '/etc/amazon/efs'
I0816 04:20:46.977449 1 driver.go:140] Did not find any input tags.
I0816 04:20:46.977787 1 driver.go:113] Registering Node Server
I0816 04:20:46.977815 1 driver.go:115] Registering Controller Server
I0816 04:20:46.977849 1 driver.go:118] Starting efs-utils watchdog
I0816 04:20:46.977965 1 efs_watch_dog.go:215] Copying /etc/amazon/efs/efs-utils.conf since it doesn't exist
I0816 04:20:46.978117 1 efs_watch_dog.go:215] Copying /etc/amazon/efs/efs-utils.crt since it doesn't exist
I0816 04:20:46.978860 1 driver.go:124] Starting reaper
I0816 04:20:46.978886 1 driver.go:127] Listening for connections on address: &net.UnixAddr{Name:"/tmp/csi.sock", Net:"unix"}
E0816 04:20:47.198299 1 efs_watch_dog.go:330] amazon-efs-mount-watchdog[Error]: Traceback (most recent call last):
File "/usr/bin/amazon-efs-mount-watchdog", line 1918, in <module>
E0816 04:20:47.218056 1 efs_watch_dog.go:284] Process amazon-efs-mount-watchdog exits exit status 120
E0816 04:20:47.318985 1 efs_watch_dog.go:330] amazon-efs-mount-watchdog[Error]: Traceback (most recent call last):
File "/usr/bin/amazon-efs-mount-watchdog", line 1918, in <module>
E0816 04:20:47.339956 1 efs_watch_dog.go:284] Process amazon-efs-mount-watchdog exits exit status 120
E0816 04:20:47.467262 1 efs_watch_dog.go:330] amazon-efs-mount-watchdog[Error]: Traceback (most recent call last):
<removed for brevity>
$ docker top $id
UID PID PPID C STIME TTY TIME CMD
root 62646 62627 2 04:20 ? 00:00:00 /bin/aws-efs-csi-driver
root 62854 62646 0 04:21 ? 00:00:00 python3 /usr/bin/amazon-efs-mount-watchdog
$ docker exec -it $id /bin/bash
[root@36f22d5ad3e0 /]# amazon-efs-mount-watchdog
Traceback (most recent call last):
File "/usr/bin/amazon-efs-mount-watchdog", line 1918, in <module>
main()
File "/usr/bin/amazon-efs-mount-watchdog", line 1875, in main
bootstrap_logging(config)
File "/usr/bin/amazon-efs-mount-watchdog", line 586, in bootstrap_logging
handler = RotatingFileHandler(
File "/usr/local/lib/python3.10/logging/handlers.py", line 155, in __init__
BaseRotatingHandler.__init__(self, filename, mode, encoding=encoding,
File "/usr/local/lib/python3.10/logging/handlers.py", line 58, in __init__
logging.FileHandler.__init__(self, filename, mode=mode,
File "/usr/local/lib/python3.10/logging/__init__.py", line 1169, in __init__
StreamHandler.__init__(self, self._open())
File "/usr/local/lib/python3.10/logging/__init__.py", line 1201, in _open
return open_func(self.baseFilename, self.mode,
FileNotFoundError: [Errno 2] No such file or directory: '/var/log/amazon/efs/mount-watchdog.log'
[root@36f22d5ad3e0 /]# exit
exit
$ docker stop $id > /dev/null
$ docker rm $id > /dev/null
- Making the log directory available allows the process to run, however if we attempt to mount something with a type of
efs
, we get an unexpected failure because themount.efs
executable doesn't exist
$ mkdir -p log/efs
$ id=$(docker run -d -v $(pwd)/files:/etc/amazon/efs-static-files -v $(pwd)/amazon-efs-mount-watchdog:/usr/bin/amazon-efs-mount-watchdog -v $(pwd)/log:/var/log/amazon <DOCKER_MIRROR>/ironbank/opensource/kubernetes-sigs/aws-efs-csi-driver:release-1.3)
$ docker logs $id
I0816 04:25:49.554575 1 config_dir.go:62] Mounted directories do not exist, creating directory at '/etc/amazon/efs'
I0816 04:25:52.683765 1 driver.go:140] Did not find any input tags.
I0816 04:25:52.685186 1 driver.go:113] Registering Node Server
I0816 04:25:52.685205 1 driver.go:115] Registering Controller Server
I0816 04:25:52.685218 1 driver.go:118] Starting efs-utils watchdog
I0816 04:25:52.685673 1 efs_watch_dog.go:215] Copying /etc/amazon/efs/efs-utils.conf since it doesn't exist
I0816 04:25:52.685897 1 efs_watch_dog.go:215] Copying /etc/amazon/efs/efs-utils.crt since it doesn't exist
I0816 04:25:52.686419 1 driver.go:124] Starting reaper
I0816 04:25:52.686439 1 driver.go:127] Listening for connections on address: &net.UnixAddr{Name:"/tmp/csi.sock", Net:"unix"}
$ docker top $id
UID PID PPID C STIME TTY TIME CMD
root 63672 63653 0 04:25 ? 00:00:00 /bin/aws-efs-csi-driver
root 63706 63672 0 04:25 ? 00:00:00 python3 /usr/bin/amazon-efs-mount-watchdog
$ docker exec -it $aws_id /bin/bash
bash-4.2# mount -t efs <EFS_DNS>
mount: <EFS_DNS>: can't find in /etc/fstab.
bash-4.2# mount -t efs <EFS_DNS> /te
b'mount.nfs4: mount point /te does not exist'
bash-4.2# exit
exit
$ docker exec -it $id /bin/bash
[root@7066c7a39360 /]# cat /var/log/amazon/efs/mount-watchdog.log
2022-08-16 04:25:52 UTC - INFO - amazon-efs-mount-watchdog, version 1.32.1, is enabled and started
[root@7066c7a39360 /]# mkdir /tmp/mp
[root@7066c7a39360 /]# mount -t efs <EFS_DNS> /tmp/mp
mount: /tmp/mp: permission denied.
[root@7066c7a39360 /]# ls -l /sbin/mount*
ls: cannot access '/sbin/mount*': No such file or directory
[root@7066c7a39360 /]# exit
exit
$ docker stop $id > /dev/null
$ docker rm $id > /dev/null
- Making the
mount.efs
file from the amazon container available doesn't work without the following additional steps:- Create a symlink for the python executable so the hashbang in
mount.efs
works (there might be a better way to do this) - Install the
nfs-utils
package because it efs depends on it.nfs-utils
is not available in the RHEL repository, so we need to get it from CentOS (I used the vortex/nfs source as reference)
- Create a symlink for the python executable so the hashbang in
$ docker cp $aws_id:/sbin/mount.efs .
$ id=$(docker run -d -v $(pwd)/files:/etc/amazon/efs-static-files -v $(pwd)/amazon-efs-mount-watchdog:/usr/bin/amazon-efs-mount-watchdog -v $(pwd)/log:/var/log/amazon -v $(pwd)/mount.efs:/sbin/mount.efs <DOCKER_MIRROR>/ironbank/opensource/kubernetes-sigs/aws-efs-csi-driver:release-1.3)
$ docker exec -it $id /bin/bash
[root@28b5410c3f7d /]# mkdir /tmp/mp
[root@28b5410c3f7d /]# mount -t efs <EFS_DNS> /tmp/mp
env: ‘python3’: No such file or directory
[root@28b5410c3f7d /]# ln -s /usr/local/bin/python3 /usr/bin/python3
[root@28b5410c3f7d /]# mount -t efs <EFS_DNS> /tmp/mp
Traceback (most recent call last):
File "/sbin/mount.efs", line 3458, in <module>
main()
File "/sbin/mount.efs", line 3447, in main
mount_nfs(
File "/sbin/mount.efs", line 1635, in mount_nfs
proc = subprocess.Popen(
File "/usr/local/lib/python3.10/subprocess.py", line 969, in __init__
self._execute_child(args, executable, preexec_fn, close_fds,
File "/usr/local/lib/python3.10/subprocess.py", line 1845, in _execute_child
raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: '/sbin/mount.nfs4'
[root@28b5410c3f7d /]# dnf config-manager --add-repo <CENTOS_MIRROR>
Adding repo from: https://mirror.aarnet.edu.au/pub/centos/8-stream/BaseOS/x86_64/os/
[root@28b5410c3f7d /]# dnf install -y --nodocs --nogpgcheck --repo mirror.aarnet.edu.au_pub_centos_8-stream_BaseOS_x86_64_os_ nfs-utils
created by dnf config-manager from https://mirror.aarnet.edu.au/pub/centos/8-stream/BaseOS/x86_64/os/ 5.0 MB/s | 25 MB 00:04
Last metadata expiration check: 0:00:05 ago on Tue Aug 16 04:38:49 2022.
Dependencies resolved.
===========================================================================================================================================================================================================
Package Architecture Version Repository Size
===========================================================================================================================================================================================================
Installing:
nfs-utils x86_64 1:2.3.3-56.el8 mirror.aarnet.edu.au_pub_centos_8-stream_BaseOS_x86_64_os_ 514 k
Installing dependencies:
diffutils x86_64 3.6-6.el8 mirror.aarnet.edu.au_pub_centos_8-stream_BaseOS_x86_64_os_ 358 k
e2fsprogs-libs x86_64 1.45.6-4.el8 mirror.aarnet.edu.au_pub_centos_8-stream_BaseOS_x86_64_os_ 233 k
gssproxy x86_64 0.8.0-21.el8 mirror.aarnet.edu.au_pub_centos_8-stream_BaseOS_x86_64_os_ 119 k
keyutils x86_64 1.5.10-9.el8 mirror.aarnet.edu.au_pub_centos_8-stream_BaseOS_x86_64_os_ 66 k
kmod x86_64 25-19.el8 mirror.aarnet.edu.au_pub_centos_8-stream_BaseOS_x86_64_os_ 126 k
libbasicobjects x86_64 0.1.1-40.el8 mirror.aarnet.edu.au_pub_centos_8-stream_BaseOS_x86_64_os_ 31 k
libcollection x86_64 0.7.0-40.el8 mirror.aarnet.edu.au_pub_centos_8-stream_BaseOS_x86_64_os_ 48 k
libevent x86_64 2.1.8-5.el8 mirror.aarnet.edu.au_pub_centos_8-stream_BaseOS_x86_64_os_ 253 k
libini_config x86_64 1.3.1-40.el8 mirror.aarnet.edu.au_pub_centos_8-stream_BaseOS_x86_64_os_ 70 k
libnfsidmap x86_64 1:2.3.3-56.el8 mirror.aarnet.edu.au_pub_centos_8-stream_BaseOS_x86_64_os_ 122 k
libpath_utils x86_64 0.2.1-40.el8 mirror.aarnet.edu.au_pub_centos_8-stream_BaseOS_x86_64_os_ 34 k
libref_array x86_64 0.1.5-40.el8 mirror.aarnet.edu.au_pub_centos_8-stream_BaseOS_x86_64_os_ 33 k
libselinux-utils x86_64 2.9-5.el8 mirror.aarnet.edu.au_pub_centos_8-stream_BaseOS_x86_64_os_ 243 k
libverto-libevent x86_64 0.3.0-5.el8 mirror.aarnet.edu.au_pub_centos_8-stream_BaseOS_x86_64_os_ 16 k
policycoreutils x86_64 2.9-19.el8 mirror.aarnet.edu.au_pub_centos_8-stream_BaseOS_x86_64_os_ 374 k
python3-pyyaml x86_64 3.12-12.el8 mirror.aarnet.edu.au_pub_centos_8-stream_BaseOS_x86_64_os_ 193 k
quota x86_64 1:4.04-14.el8 mirror.aarnet.edu.au_pub_centos_8-stream_BaseOS_x86_64_os_ 214 k
quota-nls noarch 1:4.04-14.el8 mirror.aarnet.edu.au_pub_centos_8-stream_BaseOS_x86_64_os_ 95 k
rpcbind x86_64 1.2.5-9.el8 mirror.aarnet.edu.au_pub_centos_8-stream_BaseOS_x86_64_os_ 70 k
Transaction Summary
===========================================================================================================================================================================================================
Install 20 Packages
Total download size: 3.1 M
Installed size: 8.2 M
Downloading Packages:
(1/20): gssproxy-0.8.0-21.el8.x86_64.rpm 90 kB/s | 119 kB 00:01
(2/20): e2fsprogs-libs-1.45.6-4.el8.x86_64.rpm 156 kB/s | 233 kB 00:01
(3/20): keyutils-1.5.10-9.el8.x86_64.rpm 215 kB/s | 66 kB 00:00
(4/20): diffutils-3.6-6.el8.x86_64.rpm 215 kB/s | 358 kB 00:01
(5/20): kmod-25-19.el8.x86_64.rpm 422 kB/s | 126 kB 00:00
(6/20): libbasicobjects-0.1.1-40.el8.x86_64.rpm 121 kB/s | 31 kB 00:00
(7/20): libcollection-0.7.0-40.el8.x86_64.rpm 201 kB/s | 48 kB 00:00
(8/20): libnfsidmap-2.3.3-56.el8.x86_64.rpm 449 kB/s | 122 kB 00:00
(9/20): libevent-2.1.8-5.el8.x86_64.rpm 661 kB/s | 253 kB 00:00
(10/20): libini_config-1.3.1-40.el8.x86_64.rpm 226 kB/s | 70 kB 00:00
(11/20): libpath_utils-0.2.1-40.el8.x86_64.rpm 147 kB/s | 34 kB 00:00
(12/20): libref_array-0.1.5-40.el8.x86_64.rpm 140 kB/s | 33 kB 00:00
(13/20): libverto-libevent-0.3.0-5.el8.x86_64.rpm 70 kB/s | 16 kB 00:00
(14/20): libselinux-utils-2.9-5.el8.x86_64.rpm 440 kB/s | 243 kB 00:00
(15/20): nfs-utils-2.3.3-56.el8.x86_64.rpm 1.0 MB/s | 514 kB 00:00
(16/20): policycoreutils-2.9-19.el8.x86_64.rpm 980 kB/s | 374 kB 00:00
(17/20): python3-pyyaml-3.12-12.el8.x86_64.rpm 617 kB/s | 193 kB 00:00
(18/20): quota-4.04-14.el8.x86_64.rpm 813 kB/s | 214 kB 00:00
(19/20): quota-nls-4.04-14.el8.noarch.rpm 373 kB/s | 95 kB 00:00
(20/20): rpcbind-1.2.5-9.el8.x86_64.rpm 277 kB/s | 70 kB 00:00
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total 965 kB/s | 3.1 MB 00:03
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
Preparing : 1/1
Installing : libref_array-0.1.5-40.el8.x86_64 1/20
Installing : libevent-2.1.8-5.el8.x86_64 2/20
Installing : libcollection-0.7.0-40.el8.x86_64 3/20
Installing : libbasicobjects-0.1.1-40.el8.x86_64 4/20
Installing : libverto-libevent-0.3.0-5.el8.x86_64 5/20
Installing : quota-nls-1:4.04-14.el8.noarch 6/20
Installing : python3-pyyaml-3.12-12.el8.x86_64 7/20
Installing : libselinux-utils-2.9-5.el8.x86_64 8/20
Installing : libpath_utils-0.2.1-40.el8.x86_64 9/20
Installing : libini_config-1.3.1-40.el8.x86_64 10/20
Installing : gssproxy-0.8.0-21.el8.x86_64 11/20
Running scriptlet: gssproxy-0.8.0-21.el8.x86_64 11/20
Installing : libnfsidmap-1:2.3.3-56.el8.x86_64 12/20
Installing : kmod-25-19.el8.x86_64 13/20
Installing : keyutils-1.5.10-9.el8.x86_64 14/20
Installing : e2fsprogs-libs-1.45.6-4.el8.x86_64 15/20
Running scriptlet: e2fsprogs-libs-1.45.6-4.el8.x86_64 15/20
Installing : quota-1:4.04-14.el8.x86_64 16/20
Installing : diffutils-3.6-6.el8.x86_64 17/20
Running scriptlet: diffutils-3.6-6.el8.x86_64 17/20
install-info: No such file or directory for /usr/share/info/diffutils.info
Installing : policycoreutils-2.9-19.el8.x86_64 18/20
Running scriptlet: policycoreutils-2.9-19.el8.x86_64 18/20
Running scriptlet: rpcbind-1.2.5-9.el8.x86_64 19/20
Installing : rpcbind-1.2.5-9.el8.x86_64 19/20
Running scriptlet: rpcbind-1.2.5-9.el8.x86_64 19/20
Running scriptlet: nfs-utils-1:2.3.3-56.el8.x86_64 20/20
Installing : nfs-utils-1:2.3.3-56.el8.x86_64 20/20
Running scriptlet: nfs-utils-1:2.3.3-56.el8.x86_64 20/20
System has not been booted with systemd as init system (PID 1). Can't operate.
Failed to connect to bus: Host is down
System has not been booted with systemd as init system (PID 1). Can't operate.
Failed to connect to bus: Host is down
Verifying : diffutils-3.6-6.el8.x86_64 1/20
Verifying : e2fsprogs-libs-1.45.6-4.el8.x86_64 2/20
Verifying : gssproxy-0.8.0-21.el8.x86_64 3/20
Verifying : keyutils-1.5.10-9.el8.x86_64 4/20
Verifying : kmod-25-19.el8.x86_64 5/20
Verifying : libbasicobjects-0.1.1-40.el8.x86_64 6/20
Verifying : libcollection-0.7.0-40.el8.x86_64 7/20
Verifying : libevent-2.1.8-5.el8.x86_64 8/20
Verifying : libini_config-1.3.1-40.el8.x86_64 9/20
Verifying : libnfsidmap-1:2.3.3-56.el8.x86_64 10/20
Verifying : libpath_utils-0.2.1-40.el8.x86_64 11/20
Verifying : libref_array-0.1.5-40.el8.x86_64 12/20
Verifying : libselinux-utils-2.9-5.el8.x86_64 13/20
Verifying : libverto-libevent-0.3.0-5.el8.x86_64 14/20
Verifying : nfs-utils-1:2.3.3-56.el8.x86_64 15/20
Verifying : policycoreutils-2.9-19.el8.x86_64 16/20
Verifying : python3-pyyaml-3.12-12.el8.x86_64 17/20
Verifying : quota-1:4.04-14.el8.x86_64 18/20
Verifying : quota-nls-1:4.04-14.el8.noarch 19/20
Verifying : rpcbind-1.2.5-9.el8.x86_64 20/20
Installed products updated.
Installed:
diffutils-3.6-6.el8.x86_64 e2fsprogs-libs-1.45.6-4.el8.x86_64 gssproxy-0.8.0-21.el8.x86_64 keyutils-1.5.10-9.el8.x86_64 kmod-25-19.el8.x86_64
libbasicobjects-0.1.1-40.el8.x86_64 libcollection-0.7.0-40.el8.x86_64 libevent-2.1.8-5.el8.x86_64 libini_config-1.3.1-40.el8.x86_64 libnfsidmap-1:2.3.3-56.el8.x86_64
libpath_utils-0.2.1-40.el8.x86_64 libref_array-0.1.5-40.el8.x86_64 libselinux-utils-2.9-5.el8.x86_64 libverto-libevent-0.3.0-5.el8.x86_64 nfs-utils-1:2.3.3-56.el8.x86_64
policycoreutils-2.9-19.el8.x86_64 python3-pyyaml-3.12-12.el8.x86_64 quota-1:4.04-14.el8.x86_64 quota-nls-1:4.04-14.el8.noarch rpcbind-1.2.5-9.el8.x86_64
Complete!
[root@28b5410c3f7d /]# mount -t efs <EFS_DNS> /tmp/mp
b'mount.nfs4: Operation not permitted'
[root@28b5410c3f7d /]# ls -l /sbin/mount*
-rwxr-xr-x. 1 1000 994 118171 Apr 27 23:42 /sbin/mount.efs
-rwsr-xr-x. 1 root root 201528 Jul 2 01:09 /sbin/mount.nfs
lrwxrwxrwx. 1 root root 9 Jul 2 01:09 /sbin/mount.nfs4 -> mount.nfs
-rwxr-xr-x. 1 root root 42844 Jul 2 01:09 /sbin/mountstats
[root@28b5410c3f7d /]# exit
exit
$ docker stop $id $aws_id > /dev/null
$ docker rm $id $aws_id > /dev/null
Possible fixes
- Install the
nfs-utils
package and it's dependencies - Create symlink for python
- One of the following:
- Copy the
amazon-efs-mount-watchdog
file,mount.efs
file and/etc/amazon
directory from the amazon container into this container - Install the
amazon-efs-utils
package (this option is likely to have a better result, however it has the downside of needing to manually manage the versions of another dependency)
- Copy the
Using the vortex/nfs source as reference, I believe either of the following patches would resolve this bug:
- Copy files from docker image: efs-patch-copy.txt
- Install
amazon-efs-utils
: efs-patch-rpm.txt (upon further testing, this solution might still have issues - specifically that the/etc/amazon/efs-static-files
are still missing)
Note that in the above patches I have used an Australian CentOS mirror and the hashes reported by it. This should likely be changed and the hashes verified.
Tasks
-
Bug has been identified and corrected within the container