Twistlock Defender install fails
Bug
Description
Expecting to deploy Twistlock Defenders.
After deploying Twistlock, adding a license, none of the kubernetes defenders come up on K3d:
This error is seen:
❯ k describe pods -n twistlock twistlock-defender-ds-xqb74
Name: twistlock-defender-ds-xqb74
Namespace: twistlock
Priority: 0
Node: k3d-dev-env-agent-1/172.18.0.4
Start Time: Tue, 09 Feb 2021 06:35:04 -0500
Labels: app=twistlock-defender
controller-revision-hash=f844969bd
pod-template-generation=5
Annotations: container.apparmor.security.beta.kubernetes.io/twistlock-defender: unconfined
Status: Running
IP: 172.18.0.4
IPs:
IP: 172.18.0.4
Controlled By: DaemonSet/twistlock-defender-ds
Containers:
twistlock-defender:
Container ID: containerd://83846c6fecd3247c02e6802203976fc71c0df4ebe123d2040c6ba807f78d03c8
Image: registry-auth.twistlock.com/tw_bbzc81abegfiqtnruvspkazws2ze0dby/twistlock/defender:defender_20_04_163
Image ID: registry-auth.twistlock.com/tw_bbzc81abegfiqtnruvspkazws2ze0dby/twistlock/defender@sha256:09109b321cbf3a638bd18792666e8d6fedc03cd27e63de9b67011aace8166898
Port: <none>
Host Port: <none>
State: Waiting
Reason: RunContainerError
Last State: Terminated
Reason: StartError
Message: failed to create containerd task: OCI runtime create failed: container_linux.go:370: starting container process caused: process_linux.go:459: container init caused: rootfs_linux.go:59: mounting "/etc/passwd" to rootfs at "/run/k3s/containerd/io.containerd.runtime.v2.task/k8s.io/83846c6fecd3247c02e6802203976fc71c0df4ebe123d2040c6ba807f78d03c8/rootfs/etc/passwd" caused: not a directory: unknown
Exit Code: 128
Started: Wed, 31 Dec 1969 19:00:00 -0500
Finished: Tue, 09 Feb 2021 06:36:39 -0500
Ready: False
Restart Count: 4
Limits:
cpu: 900m
memory: 512Mi
Requests:
cpu: 256m
memory: 512Mi
Environment:
WS_ADDRESS: wss://twistlock-console:8081
DEFENDER_TYPE: cri
DEFENDER_LISTENER_TYPE: none
LOG_PROD: true
SYSTEMD_ENABLED: false
DOCKER_CLIENT_ADDRESS: /var/run/docker.sock
DEFENDER_CLUSTER_ID: 0d07fb32-a85a-3617-5ec1-441c8a07fb3c
MONITOR_SERVICE_ACCOUNTS: false
MONITOR_ISTIO: false
COLLECT_POD_LABELS: true
INSTALL_BUNDLE: eyJzZWNyZXRzIjp7fSwiZ2xvYmFsUHJveHlPcHQiOnsiaHR0cFByb3h5IjoiIiwibm9Qcm94eSI6IiIsImNhIjoiIiwidXNlciI6IiIsInBhc3N3b3JkIjp7ImVuY3J5cHRlZCI6IiJ9fX0=
CONTAINERIZED_HOST: true
Mounts:
/dev/log from syslog-socket (rw)
/etc/passwd from passwd (ro)
/host from host-root (rw)
/prisma-static-data from defender-data (rw)
/run from iptables-lock (rw)
/var/lib/twistlock from data-folder (rw)
/var/lib/twistlock/certificates from certificates (rw)
/var/log/audit from auditd-log (rw)
/var/run from docker-sock-folder (rw)
/var/run/docker/netns from docker-netns (ro)
/var/run/secrets/kubernetes.io/serviceaccount from twistlock-service-token-q9b7x (ro)
Conditions:
Type Status
Initialized True
Ready False
ContainersReady False
PodScheduled True
Volumes:
certificates:
Type: Secret (a volume populated by a Secret)
SecretName: twistlock-secrets
Optional: false
syslog-socket:
Type: HostPath (bare host directory volume)
Path: /dev/log
HostPathType:
host-root:
Type: HostPath (bare host directory volume)
Path: /
HostPathType:
data-folder:
Type: HostPath (bare host directory volume)
Path: /var/lib/twistlock
HostPathType:
docker-netns:
Type: HostPath (bare host directory volume)
Path: /var/run/docker/netns
HostPathType:
passwd:
Type: HostPath (bare host directory volume)
Path: /etc/passwd
HostPathType:
docker-sock-folder:
Type: HostPath (bare host directory volume)
Path: /var/run
HostPathType:
auditd-log:
Type: HostPath (bare host directory volume)
Path: /var/log/audit
HostPathType:
defender-data:
Type: EmptyDir (a temporary directory that shares a pod's lifetime)
Medium:
SizeLimit: <unset>
iptables-lock:
Type: HostPath (bare host directory volume)
Path: /run
HostPathType:
twistlock-service-token-q9b7x:
Type: Secret (a volume populated by a Secret)
SecretName: twistlock-service-token-q9b7x
Optional: false
QoS Class: Burstable
Node-Selectors: <none>
Tolerations: node.kubernetes.io/disk-pressure:NoSchedule
node.kubernetes.io/memory-pressure:NoSchedule
node.kubernetes.io/network-unavailable:NoSchedule
node.kubernetes.io/not-ready:NoExecute
node.kubernetes.io/pid-pressure:NoSchedule
node.kubernetes.io/unreachable:NoExecute
node.kubernetes.io/unschedulable:NoSchedule
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled <unknown> Successfully assigned twistlock/twistlock-defender-ds-xqb74 to k3d-dev-env-agent-1
Warning FailedMount 97s kubelet, k3d-dev-env-agent-1 MountVolume.SetUp failed for volume "twistlock-service-token-q9b7x" : failed to sync secret cache: timed out waiting for the condition
Normal Pulled 3s (x5 over 95s) kubelet, k3d-dev-env-agent-1 Container image "registry-auth.twistlock.com/tw_bbzc81abegfiqtnruvspkazws2ze0dby/twistlock/defender:defender_20_04_163" already present on machine
Normal Created 3s (x5 over 95s) kubelet, k3d-dev-env-agent-1 Created container twistlock-defender
Warning Failed 3s (x5 over 95s) kubelet, k3d-dev-env-agent-1 Error: failed to create containerd task: OCI runtime create failed: container_linux.go:370: starting container process caused: process_linux.go:459: container init caused: rootfs_linux.go:59: mounting "/etc/passwd" to rootfs at "/run/k3s/containerd/io.containerd.runtime.v2.task/k8s.io/twistlock-defender/rootfs/etc/passwd" caused: not a directory: unknown
Warning BackOff 3s (x9 over 93s) kubelet, k3d-dev-env-agent-1 Back-off restarting failed containerBigBang Version
1.0.7