UNCLASSIFIED - NO CUI

Skip to content
Snippets Groups Projects
Commit 23d0f3a1 authored by Tunde Oladipupo's avatar Tunde Oladipupo
Browse files

updated airgap doc

parent 48a9800c
No related branches found
No related tags found
1 merge request!363Moved over resources from /big-bang/terraform-modules/air-gap-deployment
Hide whitespace changes
Inline Side-by-side
docs/airgap/README.md
+ 48
− 24
View file @ 23d0f3a1
......@@ -88,7 +88,7 @@ You can follow the process below to setup git with `repositories.tar.gz` on the
```bash
$ sudo useradd --create-home --shell /bin/bash git
$ ssh-keygen -f ~/.ssh/identity
$ ssh-keygen -b 4096 -t rsa -f ~/.ssh/identity -q -N ""
```
- Create .SSH folder for `git` user
......@@ -114,18 +114,29 @@ $ ssh-keygen -f ~/.ssh/identity
$ sudo tar -xvf repositories.tar.gz --directory /home/git/
```
- Add Hostname alias
```bash
PRIVATEIP=$( curl http://169.254.169.254/latest/meta-data/local-ipv4 )
sudo sed -i -e '1i'$PRIVATEIP' 'myhostname.com'\' /etc/hosts
sudo sed -i -e '1i'$PRIVATEIP' 'host.k3d.internal'\' /etc/hosts #only for k3d
```
- To test the client key;
```bash
GIT_SSH_COMMAND='ssh -i /[client-private-key-path] -o IdentitiesOnly=yes' git clone git@[hostname/IP]:/home/git/repos/[sample-repo]
#For example;
PRIVATEIP=$( curl http://169.254.169.254/latest/meta-data/local-ipv4 )
GIT_SSH_COMMAND='ssh -i ~/.ssh/identity -o IdentitiesOnly=yes' git clone git@$PRIVATEIP:/home/git/repos/istio-controlplane
GIT_SSH_COMMAND='ssh -i ~/.ssh/identity -o IdentitiesOnly=yes' git clone git@host.k3d.internal:/home/git/repos/bigbang
#checkout release branch
git checkout 1.3.0
```
## Private Registry
Images needed to run BB in your cluster is packaged as part of the release in `images.tar.gz`. You can see the list of required images in `images.txt`. In our airgap environment, we need to setup a registry that our cluster can pull required images from or an existing cluster where we can copy images from `images.tar.gz` into.
......@@ -134,16 +145,28 @@ Images needed to run BB in your cluster is packaged as part of the release in `i
To setup the registry, we will be using `registry:2` to run a private registry with self-signed certificate.
First, untar `images.tar.gz`;
- First, untar `images.tar.gz`;
```bash
tar -xf images.tar.gz -C .
tar -xvf images.tar.gz -C .
```
Use the script [registry.sh](./scripts/registry.sh) to create registry;
- SCP `registry:2` tar file
```bash
docker save -o registry2.tar registry:2
docker save -o k3s.tar rancher/k3s:v1.20.5-rc1-k3s1 #check release matching version
scp registry2.tar k3s.tar ubuntu@hostname:~ #modify according to your environment
docker load -i registry2.tar #on your registry server
docker load -i k3s.tar
```
- Use the script [registry.sh](./scripts/registry.sh) to create registry;
```bash
$ sudo ./registry.sh
$ chmod +x registry.sh && sudo ./registry.sh
Required information:
Enter bit size for certs (Ex. 4096): 4096
......@@ -181,7 +204,7 @@ Notes
To see images in the registry;
=========================
curl https://myregistry.com:5443/v2/_catalog -k
curl https://myhostname.com:5443/v2/_catalog -k
=========================
```
......@@ -191,7 +214,7 @@ A folder is created with TLS certs that we are going to supply to our k8s cluste
You can ensure the images are now loaded in the registry;
```bash
curl -k https://10.0.52.144:5443/v2/_catalog
curl -k https://myhostname.com:5443/v2/_catalog
{"repositories":["ironbank/anchore/engine/engine","ironbank/anchore/enterprise/enterprise","ironbank/anchore/enterpriseui/enterpriseui","ironbank/big-bang/argocd","ironbank/bitnami/analytics/redis-exporter","ironbank/elastic/eck-operator/eck-operator","ironbank/elastic/elasticsearch/elasticsearch","ironbank/elastic/kibana/kibana","ironbank/fluxcd/helm-controller","ironbank/fluxcd/kustomize-controller","ironbank/fluxcd/notification-controller","ironbank/fluxcd/source-controller","ironbank/gitlab/gitlab/alpine-certificates","ironbank/gitlab/gitlab/cfssl-self-sign","ironbank/gitlab/gitlab/gitaly",...]
```
......@@ -246,15 +269,19 @@ spec:
#### RKE2 cluster
```yaml
#registries.yaml
mirrors:
registry.dso.mil:
endpoint:
- https://myregistry.com:5443
- https://myhostname.com:5443
registry1.dso.mil:
endpoint:
- https://myhostname.com:5443
registry1.dso.mil:
endpoint:
- https://myregistry.com:5443
- https://myhostname.com:5443
configs:
myregistry.com:5443:
myhostname.com:5443:
tls:
ca_file: "/etc/ssl/certs/registry1.pem"
```
......@@ -263,10 +290,7 @@ configs:
## Installing Big Bang
Untar bigbag
```bash
$ tar -xf bigbang-[version].tar.gz -C .
$ cd bigbang
```
......@@ -276,10 +300,13 @@ Install Flux 2 into the cluster using the provided artifacts. These are located
kubectl apply -f ./scripts/deploy/flux.yaml
After Flux is up and running you are ready to deploy Big Bang. We will do this using Helm. To first check to see if Flux is ready you can do.
kubectl get all -n flux-system
You can watch to see if Flux is reconciling the projects by watching the progress.
```bash
watch kubectl get all -n flux-system
```
We need a namespace for our preparations and eventually for Big Bang to deploy into.
......@@ -308,8 +335,8 @@ For your Git repository you have two options for setting up the credentials.
Option 1: Use an existing secret.
cd ~/.ssh
ssh-keygen -q -N "" -f ./identity
ssh-keyscan <YOUR GIT URL HERE> ./known_hosts
ssh-keygen -b 4096 -t rsa -f ~/.ssh/identity -q -N ""
ssh-keyscan <YOUR GIT URL HERE> ./known_hosts
kubectl create secret generic -n bigbang ssh-credentials \
--from-file=./identity \
......@@ -388,12 +415,9 @@ Take the values from each of these files and place in the correct fields in the
Then install Big Bang using Helm.
helm upgrade -i bigbang chart -n bigbang --create-namespace -f values.yaml
watch kubectl get gitrepositories,kustomizations,hr,po -A
** Note that the --create-namespace isn't needed if you created it earlier, but it doesn't hurt anything.
You can watch to see if Flux is reconciling the projects by watching the progress.
watch kubectl get gitrepositories,kustomizations,hr,po -A
You should see the diffent projects configure working through their reconciliation starting with "gatekeeper".
docs/airgap/developer/developer.md
+ 11
− 20
View file @ 23d0f3a1
......@@ -6,16 +6,14 @@ To test Airgap BigBang on k3d
- Launch ec2 instance of size `c5.2xlarge` and ssh into the instance with at least 50GB storage.
- Install [Docker](https://docs.docker.com/engine/install/ubuntu/)
- Install [K3D](https://k3d.io/#installation)
- Install `k3d` and `docker` cli tools
- Download `images.tar.gz`, `repositories.tar.gz` and `bigbang-version.tar.gz` from BigBang release.
```bash
$ curl -O https://umbrella-bigbang-releases.s3-us-gov-west-1.amazonaws.com/umbrella/1.2.0/repositories.tar.gz
$ curl -O https://umbrella-bigbang-releases.s3-us-gov-west-1.amazonaws.com/umbrella/1.2.0/images.tar.gz
$ curl -O https://repo1.dso.mil/platform-one/big-bang/bigbang/-/archive/1.2.0/bigbang-1.2.0.tar.gz
$ curl -O https://umbrella-bigbang-releases.s3-us-gov-west-1.amazonaws.com/umbrella/1.3.0/repositories.tar.gz
$ curl -O https://umbrella-bigbang-releases.s3-us-gov-west-1.amazonaws.com/umbrella/1.3.0/images.tar.gz
$ sudo apt install -y net-tools
```
......@@ -28,15 +26,15 @@ To test Airgap BigBang on k3d
mirrors:
registry.dso.mil:
endpoint:
- https://[$PRIVATEIP]:5443
- https://host.k3d.internal:5443
registry1.dso.mil:
endpoint:
- https://[$PRIVATEIP]:5443
- https://host.k3d.internal:5443
docker.io:
endpoint:
- https://[$PRIVATEIP]:5443
- https://host.k3d.internal:5443
configs:
myregistry.com:5443:
host.k3d.internal:5443:
tls:
ca_file: "/etc/ssl/certs/registry1.pem"
```
......@@ -47,22 +45,17 @@ To test Airgap BigBang on k3d
```bash
$ PRIVATEIP=$( curl http://169.254.169.254/latest/meta-data/local-ipv4 )
$ k3d cluster create --api-port "${PRIVATEIP}:33989" -s 1 -a 2 -v "${HOME}/registries.yaml:/etc/rancher/k3s/registries.yaml" -v /etc/machine-id:/etc/machine-id -v "${HOME}/certs/myregistry.com.public.pem:/etc/ssl/certs/registry1.pem" --k3s-server-arg "--disable=traefik" --k3s-server-arg "--disable=metrics-server" --k3s-server-arg "--tls-san=$PRIVATEIP" -p 80:80@loadbalancer -p 443:443@loadbalancer
$ k3d cluster create --image "rancher/k3s:v1.20.5-rc1-k3s1" --api-port "33989" -s 1 -a 2 -v "${HOME}/registries.yaml:/etc/rancher/k3s/registries.yaml" -v /etc/machine-id:/etc/machine-id -v "${HOME}/certs/host.k3d.internal.public.pem:/etc/ssl/certs/registry1.pem" --k3s-server-arg "--disable=traefik" --k3s-server-arg "--disable=metrics-server" --k3s-server-arg "--tls-san=$PRIVATEIP" -p 80:80@loadbalancer -p 443:443@loadbalancer
```
- Bock all egress with `iptables` except those going to instance IP before deploying bigbang by running [k3d_airgap.sh](./scripts/k3d_airgap.sh)
```bash
```
```bash
$ sudo ./k3d_airgap.sh
$ curl https://index.docker.io/ #shouldnt work
$ curl https://$PRIVATEIP:5443/v2/_catalog -k #show return list of images
curl https://$PRIVATEIP:5443/v2/repositories/rancher/library-busybox/tags
```
......@@ -72,7 +65,8 @@ To test Airgap BigBang on k3d
- Test that mirroring is working
```bash
$ kubectl run -i --tty test --image=registry1.dso.mil/rancher/library-busybox:1.31.1 --image-pull-policy='Always' -- sh
$ curl -k -X GET https://$PRIVATEIP:5443/v2/rancher/local-path-provisioner/tags/list
$ kubectl run -i --tty test --image=registry1.dso.mil/rancher/local-path-provisioner:v0.0.19 --image-pull-policy='Always' --command sleep infinity -- sh
$ kubectl run test --image=registry1.dso.mil/rancher/library-busybox:1.31.1 --image-pull-policy='Always' --restart=Never --command sleep infinity
$ telnet default.kube-system.svc.cluster.local 443
$ kubectl describe po test
......@@ -87,7 +81,4 @@ $ kubectl describe po test #should fail
$ kubectl delete po test
```
-
- Proceed to [bigbang deployment process](../README.md#installing-big-bang)
\ No newline at end of file
docs/airgap/scripts/airgap.sh
+ 1
− 0
View file @ 23d0f3a1
......@@ -4,3 +4,4 @@ PUBLICINTERFACE=$( route | grep '^default' | grep -o '[^ ]*$' )
iptables -I DOCKER-USER -i ${PUBLICINTERFACE} -j DROP
iptables -I DOCKER-USER -d 10.42.0.0/16 -j RETURN
iptables -I DOCKER-USER -d 10.43.0.0/16 -j RETURN
iptables -A DOCKER-USER -j RETURN
\ No newline at end of file
docs/airgap/scripts/registry.sh
+ 8
− 7
View file @ 23d0f3a1
#!/usr/bin/env bash
set -e
# Installs/Configures:
# - Docker Registy Container with self-signed cert
......@@ -37,10 +38,8 @@ prompt STATE "Enter the 'State' for the cert (Ex. CO)"
prompt LOCATION "Enter the 'Location' for the cert (Ex. ColoradoSprings)"
prompt ORGANIZATION "Enter the 'Organization' for the cert (Ex. PlatformOne)"
prompt OUNIT "Enter the 'Organizational Unit' for the cert (Ex. Bigbang)"
prompt COMMON "Enter the 'Common Name' for the cert (Must be a FQDN (at least one period character) E.g. myregistry.com)"
PRIVATEIP=$( curl http://169.254.169.254/latest/meta-data/local-ipv4 )
prompt COMMON "Enter the 'Common Name' for the cert (Must be a FQDN (at least one period character) E.g. host.k3d.internal"
prompt ALTNAMES "Enter the 'Subject Alternative Names' for the cert E.g. DNS:host.k3d.internal,IP:PRIVATEIP)"
# ... Certs ...
# ~~~~~~~~~~~~~
......@@ -48,7 +47,7 @@ PRIVATEIP=$( curl http://169.254.169.254/latest/meta-data/local-ipv4 )
# ... prep certs ...
echo -e "\nGenerating certs ..."
mkdir certs
mkdir -p certs
cd certs
# Generate a root key
openssl genrsa -out rootCA.key ${BITS}
......@@ -67,10 +66,11 @@ openssl req -new -key ${COMMON}.key \
-out ${COMMON}.csr
# Sign certificate request
echo subjectAltName = IP:${PRIVATEIP} > extfile.cnf
echo subjectAltName = DNS:${COMMON},${ALTNAMES} > extfile.cnf
openssl x509 -req -in ${COMMON}.csr -CA rootCA.crt -CAkey rootCA.key -CAcreateserial -days ${DAYS} \
-out ${COMMON}.crt -extfile extfile.cnf
openssl rsa -in ${COMMON}.key -text > ${COMMON}.private.pem
openssl x509 -inform PEM -in ${COMMON}.crt > ${COMMON}.public.pem
......@@ -100,7 +100,8 @@ Notes
To see images in the registry;
=========================
curl https://${PRIVATEIP}:5443/v2/_catalog -k
For example,
curl https://host.k3d.internal:5443/v2/_catalog -k
=========================
"
\ No newline at end of file
docs/airgap/scripts/values.yaml
+ 30
− 30
View file @ 23d0f3a1
# -- Domain used for BigBang created exposed services, can be overridden by individual packages.
hostname: bigbang.dev
registryCredentials:
registry: 10.25.21.9
registry: host.k3d.internal
username: ""
password: ""
email: ""
......@@ -16,33 +16,33 @@ flux:
eckoperator:
enabled: false
enabled: true
git:
repo: ssh://git@10.25.21.9/home/git/repos/eck-operator
repo: ssh://git@host.k3d.internal/home/git/repos/eck-operator
fluentbit:
enabled: false
enabled: true
git:
repo: ssh://git@10.25.21.9/home/git/repos/fluentbit
repo: ssh://git@host.k3d.internal/home/git/repos/fluentbit
logging:
enabled: false
enabled: true
git:
repo: ssh://git@10.25.21.9/home/git/repos/elasticsearch-kibana
repo: ssh://git@host.k3d.internal/home/git/repos/elasticsearch-kibana
istio:
enabled: false
enabled: true
git:
repo: ssh://git@10.25.21.9/home/git/repos/istio-controlplane
repo: ssh://git@host.k3d.internal/home/git/repos/istio-controlplane
istiooperator:
enabled: false
enabled: true
git:
repo: ssh://git@10.25.21.9/home/git/repos/istio-operator
repo: ssh://git@host.k3d.internal/home/git/repos/istio-operator
clusterAuditor:
enabled: false
enabled: true
git:
repo: ssh://git@10.25.21.9/home/git/repos/cluster-auditor
repo: ssh://git@host.k3d.internal/home/git/repos/cluster-auditor
values:
resources:
requests:
......@@ -50,68 +50,68 @@ clusterAuditor:
memory: 0.5Gi
monitoring:
enabled: false
enabled: true
git:
repo: ssh://git@10.25.21.9/home/git/repos/monitoring
repo: ssh://git@host.k3d.internal/home/git/repos/monitoring
gatekeeper:
enabled: true
git:
repo: ssh://git@10.25.21.9/home/git/repos/policy
repo: ssh://git@host.k3d.internal/home/git/repos/policy
twistlock:
enabled: false
enabled: true
git:
repo: ssh://git@10.25.21.9/home/git/repos/twistlock
repo: ssh://git@host.k3d.internal/home/git/repos/twistlock
# Explicitly enable all addons for CI
addons:
argocd:
enabled: false
git:
repo: ssh://git@10.25.21.9/home/git/repos/argocd
repo: ssh://git@host.k3d.internal/home/git/repos/argocd
authservice:
enabled: false
git:
repo: ssh://git@10.25.21.9/home/git/repos/authservice
repo: ssh://git@host.k3d.internal/home/git/repos/authservice
chains:
minimal:
callback_uri: "https://minimal.bigbang.dev"
gitlab:
enabled: false
git:
repo: ssh://git@10.25.21.9/home/git/repos/gitlab
repo: ssh://git@host.k3d.internal/home/git/repos/gitlab
sso:
enabled: false
gitlabRunner:
enabled: false
git:
repo: ssh://git@10.25.21.9/home/git/repos/gitlab-runner
repo: ssh://git@host.k3d.internal/home/git/repos/gitlab-runner
anchore:
enabled: false
git:
repo: ssh://git@10.25.21.9/home/git/repos/anchore-enterprise
repo: ssh://git@host.k3d.internal/home/git/repos/anchore-enterprise
sonarqube:
enabled: false
enabled: true
git:
repo: ssh://git@10.25.21.9/home/git/repos/sonarqube
repo: ssh://git@host.k3d.internal/home/git/repos/sonarqube
minioOperator:
enabled: false
git:
repo: ssh://git@10.25.21.9/home/git/repos/minio-operator
repo: ssh://git@host.k3d.internal/home/git/repos/minio-operator
minio:
enabled: false
git:
repo: ssh://git@10.25.21.9/home/git/repos/minio
repo: ssh://git@host.k3d.internal/home/git/repos/minio
haproxy:
enabled: false
git:
repo: ssh://git@10.25.21.9/home/git/repos/haproxy
repo: ssh://git@host.k3d.internal/home/git/repos/haproxy
mattermostoperator:
enabled: false
git:
repo: ssh://git@10.25.21.9/home/git/repos/mattermost-operator
repo: ssh://git@host.k3d.internal/home/git/repos/mattermost-operator
mattermost:
enabled: false
git:
repo: ssh://git@10.25.21.9/home/git/repos/mattermost
\ No newline at end of file
repo: ssh://git@host.k3d.internal/home/git/repos/mattermost
\ No newline at end of file
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment

UNCLASSIFIED - NO CUI