Pipeline Jobs Can't Clone Code
Describe the bug
Pipeline jobs are unable to clone code due to a missing network policy.
To Reproduce
Steps to reproduce the behavior:
- Deploy Gitlab and Gitlab-runner in separate namespaces (like in BB 2.x)
- Run a pipeline in Gitlab
- See error
Expected behavior
Pipeline job is able to clone the repo's code
Actual behavior
Running with gitlab-runner 15.10.0 (456e3482)
on gitlab-runner-bd7db4bff-sslp8 YSRNdq-a, system ID: r_NkX8WsMADIFJ
Preparing the "kubernetes" executor
00:00
Using Kubernetes namespace: gitlab-runner
Using Kubernetes executor with image registry1.dso.mil/ironbank/redhat/ubi/ubi8:8.7 ...
Using attach strategy to execute scripts...
Preparing environment
Waiting for pod gitlab-runner/runner-ysrndq-a-project-2-concurrent-08hmh5 to be running, status is Pending
Waiting for pod gitlab-runner/runner-ysrndq-a-project-2-concurrent-08hmh5 to be running, status is Pending
ContainersNotInitialized: "containers with incomplete status: [init-permissions istio-init]"
ContainersNotReady: "containers with unready status: [istio-proxy build helper]"
ContainersNotReady: "containers with unready status: [istio-proxy build helper]"
Waiting for pod gitlab-runner/runner-ysrndq-a-project-2-concurrent-08hmh5 to be running, status is Pending
ContainersNotInitialized: "containers with incomplete status: [init-permissions istio-init]"
ContainersNotReady: "containers with unready status: [istio-proxy build helper]"
ContainersNotReady: "containers with unready status: [istio-proxy build helper]"
Waiting for pod gitlab-runner/runner-ysrndq-a-project-2-concurrent-08hmh5 to be running, status is Pending
ContainersNotInitialized: "containers with incomplete status: [init-permissions istio-init]"
ContainersNotReady: "containers with unready status: [istio-proxy build helper]"
ContainersNotReady: "containers with unready status: [istio-proxy build helper]"
Waiting for pod gitlab-runner/runner-ysrndq-a-project-2-concurrent-08hmh5 to be running, status is Pending
ContainersNotInitialized: "containers with incomplete status: [init-permissions istio-init]"
ContainersNotReady: "containers with unready status: [istio-proxy build helper]"
ContainersNotReady: "containers with unready status: [istio-proxy build helper]"
Waiting for pod gitlab-runner/runner-ysrndq-a-project-2-concurrent-08hmh5 to be running, status is Pending
ContainersNotInitialized: "containers with incomplete status: [istio-init]"
ContainersNotReady: "containers with unready status: [istio-proxy build helper]"
ContainersNotReady: "containers with unready status: [istio-proxy build helper]"
Waiting for pod gitlab-runner/runner-ysrndq-a-project-2-concurrent-08hmh5 to be running, status is Pending
ContainersNotReady: "containers with unready status: [istio-proxy build helper]"
ContainersNotReady: "containers with unready status: [istio-proxy build helper]"
Waiting for pod gitlab-runner/runner-ysrndq-a-project-2-concurrent-08hmh5 to be running, status is Pending
ContainersNotReady: "containers with unready status: [istio-proxy build helper]"
ContainersNotReady: "containers with unready status: [istio-proxy build helper]"
Waiting for pod gitlab-runner/runner-ysrndq-a-project-2-concurrent-08hmh5 to be running, status is Pending
ContainersNotReady: "containers with unready status: [istio-proxy build helper]"
ContainersNotReady: "containers with unready status: [istio-proxy build helper]"
Waiting for pod gitlab-runner/runner-ysrndq-a-project-2-concurrent-08hmh5 to be running, status is Pending
ContainersNotReady: "containers with unready status: [istio-proxy build helper]"
ContainersNotReady: "containers with unready status: [istio-proxy build helper]"
Running on runner-ysrndq-a-project-2-concurrent-08hmh5 via gitlab-runner-bd7db4bff-sslp8...
Getting source from Git repository
Fetching changes with git depth set to 20...
Initialized empty Git repository in /builds/gitlab-instance-efc8be6b/test/.git/
Created fresh repository.
remote: upstream connect error or disconnect/reset before headers. retried and the latest reset reason: connection failure
fatal: unable to access 'http://gitlab-webservice-default.gitlab.svc.cluster.local:8181/gitlab-instance-efc8be6b/test.git/': The requested URL returned error: 503
ERROR: Job failed: command terminated with exit code 1
Additional context
Adding a network policy that would allow the job pods to talk to the gitlab webservice fixes the issue. Here is an example policy:
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: egress-runner-jobs-gitlab-runner
namespace: gitlab-runner
spec:
egress:
- to:
- namespaceSelector:
matchLabels:
app.kubernetes.io/name: gitlab
podSelector:
matchLabels:
app: webservice
podSelector:
matchLabels: {}
policyTypes:
- Egress
Looks similar or the same as big-bang/bigbang#1499 (closed)
Edited by Daniel Palmer