UNCLASSIFIED - NO CUI

Add multi-node capablity to k3d cluster provider

I'm working on a bigbang package integration in cooperation with Harness. As of now their application has a hardcoded requirement of at least 3 nodes. I'm working with them to see if this requirement can be changed, but in the meantime wanted to pursue adding multiple nodes to the BB pipeline so I can at least get a tag out for the package. This is as simple as adding agents: 3 to the k3d config yaml.

As far as I can tell adding more nodes in k3d only adds a little bit of overhead and doesn't increase resource consumption by any notable amount. I have a few ideas for implementation around how it's enabled, getting the line into the yaml is a very simple yq command yq -i e '.agents |= 3' k3d-config.yaml. Options for enabling this functionality are as follows:

  • Check for existence of label
    • This allows anyone to enable/disable multi-node capability which can be a good or a bad thing depending on your perspective
    • This does present a problem when it comes to the tag pipeline since I don't think they can be easily labeled so I think this is kind of a non-starter
  • Check value of some bash variable ($MULTI_NODE = true)
    • This restricts enabling the feature to people that can change CI vars
    • Tag pipelines also use CI vars (I think) so they would have multiple nodes as well
  • Check for a value or file in the tests dir
    • Would allow anyone to enable the feature
    • Would run in tag ci
    • Use test-values.yaml
      • Kind of weird since that file is meant to be a list of overrides for the chart
      • Uses existing file/pattern
    • Use some other file
      • Won't interfere with other things