UNCLASSIFIED - NO CUI

Pipeline Jobs Can't Clone Code

Describe the bug

Pipeline jobs are unable to clone code due to a missing network policy.

To Reproduce

Steps to reproduce the behavior:

  1. Deploy Gitlab and Gitlab-runner in separate namespaces (like in BB 2.x)
  2. Run a pipeline in Gitlab
  3. See error

Expected behavior

Pipeline job is able to clone the repo's code

Actual behavior

Running with gitlab-runner 15.10.0 (456e3482)
  on gitlab-runner-bd7db4bff-sslp8 YSRNdq-a, system ID: r_NkX8WsMADIFJ
Preparing the "kubernetes" executor
00:00
Using Kubernetes namespace: gitlab-runner
Using Kubernetes executor with image registry1.dso.mil/ironbank/redhat/ubi/ubi8:8.7 ...
Using attach strategy to execute scripts...
Preparing environment
Waiting for pod gitlab-runner/runner-ysrndq-a-project-2-concurrent-08hmh5 to be running, status is Pending
Waiting for pod gitlab-runner/runner-ysrndq-a-project-2-concurrent-08hmh5 to be running, status is Pending
	ContainersNotInitialized: "containers with incomplete status: [init-permissions istio-init]"
	ContainersNotReady: "containers with unready status: [istio-proxy build helper]"
	ContainersNotReady: "containers with unready status: [istio-proxy build helper]"
Waiting for pod gitlab-runner/runner-ysrndq-a-project-2-concurrent-08hmh5 to be running, status is Pending
	ContainersNotInitialized: "containers with incomplete status: [init-permissions istio-init]"
	ContainersNotReady: "containers with unready status: [istio-proxy build helper]"
	ContainersNotReady: "containers with unready status: [istio-proxy build helper]"
Waiting for pod gitlab-runner/runner-ysrndq-a-project-2-concurrent-08hmh5 to be running, status is Pending
	ContainersNotInitialized: "containers with incomplete status: [init-permissions istio-init]"
	ContainersNotReady: "containers with unready status: [istio-proxy build helper]"
	ContainersNotReady: "containers with unready status: [istio-proxy build helper]"
Waiting for pod gitlab-runner/runner-ysrndq-a-project-2-concurrent-08hmh5 to be running, status is Pending
	ContainersNotInitialized: "containers with incomplete status: [init-permissions istio-init]"
	ContainersNotReady: "containers with unready status: [istio-proxy build helper]"
	ContainersNotReady: "containers with unready status: [istio-proxy build helper]"
Waiting for pod gitlab-runner/runner-ysrndq-a-project-2-concurrent-08hmh5 to be running, status is Pending
	ContainersNotInitialized: "containers with incomplete status: [istio-init]"
	ContainersNotReady: "containers with unready status: [istio-proxy build helper]"
	ContainersNotReady: "containers with unready status: [istio-proxy build helper]"
Waiting for pod gitlab-runner/runner-ysrndq-a-project-2-concurrent-08hmh5 to be running, status is Pending
	ContainersNotReady: "containers with unready status: [istio-proxy build helper]"
	ContainersNotReady: "containers with unready status: [istio-proxy build helper]"
Waiting for pod gitlab-runner/runner-ysrndq-a-project-2-concurrent-08hmh5 to be running, status is Pending
	ContainersNotReady: "containers with unready status: [istio-proxy build helper]"
	ContainersNotReady: "containers with unready status: [istio-proxy build helper]"
Waiting for pod gitlab-runner/runner-ysrndq-a-project-2-concurrent-08hmh5 to be running, status is Pending
	ContainersNotReady: "containers with unready status: [istio-proxy build helper]"
	ContainersNotReady: "containers with unready status: [istio-proxy build helper]"
Waiting for pod gitlab-runner/runner-ysrndq-a-project-2-concurrent-08hmh5 to be running, status is Pending
	ContainersNotReady: "containers with unready status: [istio-proxy build helper]"
	ContainersNotReady: "containers with unready status: [istio-proxy build helper]"
Running on runner-ysrndq-a-project-2-concurrent-08hmh5 via gitlab-runner-bd7db4bff-sslp8...
Getting source from Git repository
Fetching changes with git depth set to 20...
Initialized empty Git repository in /builds/gitlab-instance-efc8be6b/test/.git/
Created fresh repository.
remote: upstream connect error or disconnect/reset before headers. retried and the latest reset reason: connection failure
fatal: unable to access 'http://gitlab-webservice-default.gitlab.svc.cluster.local:8181/gitlab-instance-efc8be6b/test.git/': The requested URL returned error: 503
ERROR: Job failed: command terminated with exit code 1

Additional context

Adding a network policy that would allow the job pods to talk to the gitlab webservice fixes the issue. Here is an example policy:

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: egress-runner-jobs-gitlab-runner
  namespace: gitlab-runner
spec:
  egress:
  - to:
    - namespaceSelector:
        matchLabels:
          app.kubernetes.io/name: gitlab
      podSelector:
        matchLabels:
          app: webservice
  podSelector:
    matchLabels: {}
  policyTypes:
  - Egress

Looks similar or the same as big-bang/bigbang#1499 (closed)

Edited by Daniel Palmer