Spike: Twistlock/Prisma Cloud Compute Operator

Research checklist items #1 and #2.

If issues are found, submit issues per #3 and label Epic as blocked.

If no issues are found, no additional work required. Issues will be created for #0,#4,#5,etc.

added to epic &115 (closed)

See Epic checklist: https://repo1.dso.mil/groups/platform-one/big-bang/-/epics/115

changed milestone to %1.16.0

changed iteration to Big Bang Iterations Aug 24, 2021 - Sep 6, 2021

changed the description

set weight to 3

assigned to @micah.nagel

added statusdoing label

added ~5347 twistlock labels

changed the description

To get operator manifests:

kustomize build https://github.com/PaloAltoNetworks/prisma-cloud-compute-operator.git//config/deploy > pcc-operator.yaml

Then apply. I'm working through the proper CR for deploying console + defenders. I've been able to get the console to come up but as of yet been unsuccessful getting it to create the initial user or add the license.

Opened a number of upstream issues...

• Better failure handling - currently its extremely hard to debug failures/current status, this is somewhere between a bug and feature request
• Spec credentials not used - this is a bug where you cannot provide the credentials via the spec.
• Secret credentials handling - feature request, the current way to provide creds via a secret requires the secret to exist before the operator, and the creds get mounted to the operator
• Docs issue - some broken links in their documentation which include the deployment scenario example, made it difficult to figure out how to get this working.
• monitorIstio doesn't work - bug due to RBAC
• expose http for console - no way to expose http currently so the way we use istio would not work...potentially could work with passthrough but that gets messy and we supported http in the past with twistlock so its likely just an oversight that it wasn't exposed in the operator
• allow config of rules - operator provides no way to supply a set of rules to pre-configure the instance with

Steps to get the operator to deploy properly:

• kubectl create ns twistlock
• create a secret like the below, the secret name must be pcc-credentials, all data you can change. To get the accessToken either "look at your license bundle" (what upstream says) or deploy twistlock separate from the operator, add your license and grab the token from the console (Manage -> System -> (1) and copy)

apiVersion: v1
kind: Secret
metadata:
  name: pcc-credentials
  namespace: twistlock
type: Opaque
stringData:
  accessToken: "put your access token here"
  license: "put your license here"
  password: "Bigbang#123"
  username: "bigbang"

• kubectl apply -f ^^^^that secret^^^^ (yeah you can do fancy oneline EOF stuff but I'm not smart)
• kustomize build https://github.com/PaloAltoNetworks/prisma-cloud-compute-operator.git//config/deploy > pcc-operator.yaml
• kubectl apply -f pcc-operator.yaml (yes this could be a oneliner with the above if you don't care to see the yaml output)

With that the operator should be deployed and be ready to run an instance. Now to create the instance:

• create a private-registry secret in the twistlock ns with your IB creds as dockerconfigjson (it's pretty easy to copy from another BB ns)
• kubectl apply -f ^^^that private reg file^^^
• create a yaml file for the instance:

apiVersion: pcc.paloaltonetworks.com/v1alpha1
kind: ConsoleDefender
metadata:
  name: pcc-consoleplusdefender
  namespace: twistlock
spec:
  orchestrator: "kubernetes"
  consoleConfig:
    imageName: "registry1.dso.mil/ironbank/twistlock/console/console:21.04.439"
    imagePullSecret: "private-registry"
  defenderConfig:
    docker: false
    imageName: "registry1.dso.mil/ironbank/twistlock/defender/defender:21.04.439"
    imagePullSecret: "private-registry"
  namespace: twistlock
  version: '21_04_439'

• kubectl apply -f ^^^that instance file^^^

If you did everything exactly as that says you should have a default admin account with the creds you specified, license setup, and defender pods created. If you're on k3d you may have some errors on the defenders due to the file paths it tries to mount by default. If you stray from the steps you're asking for trouble and I won't help you...

But actually, there's a lot of trickiness with this between actual bugs and undocumented quirks/annoyances with how the operator works. I submitted the above linked issues upstream which hopefully cover a lot of my "complaints" and the issues with the operator. In it's current state I don't believe its ready for BB consumption.

I plan to spend a bit more time on this tomorrow morning to verify that the defenders are able to go to running. I will need to modify my k3d setup so that certain volumes are mapped into the nodes, but otherwise expect no issues.

Was able to get defenders to go to a running state but they don't seem to connect to the console. Running into the same issue even with the generated yaml from the console though so it may not be an issue with the operators. I plan to try the hub/spoke setup as well to see if that yields any different results/verify that a console can be deployed independently.

Spun up the console separately from defenders and all worked the same way. Still unable to get the defenders to output any logs/"register" with the console... this seems mostly related to doing this on k3d and not having the proper volume mounting of the docker sock and other things the defenders are looking for.

I'm going to close this spike out and add some comments on the epic referencing the issues.

closed

added statusreview label and removed statusdoing label

mentioned in epic &115 (closed)

removed statusreview label