Make rds_mapvalues support multiple destinations for db_values settings

General MR

Summary

This MR modifies the CI RDS mapping functionality to allow database configuration values to be injected in more than one location within the values hierarchy.

In the current state of the utility, a given package can say "Give me a database and inject the resulting configuration into these locations", like so:

bigbang_ci:
  rds:
    enabled: true
    host: .package.config.host
    database: .package.config.database
    port: .package.config.port
    username: .package.config.username
    password: .package.config.password

... and the resulting database user, host, port, dbname, and password will be injected into the values at the paths indicated. This works fine. However, there are times where this information needs to be injected in multiple places.

Consider the case of the mattermost pipeline merging into the bigbang umbrella. The mattermost package itself requires a CI RDS database when being tested in the package pipeline, and it is configured thusly at the package level:

bigbang_ci:
  rds:
    enabled: true
    database: ".postgresql.auth.database"
    host: ".postgresql.auth.host"
    username: ".postgresql.auth.username"
    password: ".postgresql.auth.password"

However at the bigbang umbrella level, not only is the configuration in a different location in the values hierarchy, it needs to be duplicated in at least one location for an Istio Hardening Egress Gateway rule.

Moving the values to the correct place in the hierarchy for the first case is trivial using the package-mapping.yaml facility:

  bigbang_ci:
    rds:
      enabled: true
      database: ".addons.mattermost.database.database"
      host: ".addons.mattermost.database.host"
      port: ".addons.mattermost.database.port"
      username: ".addons.mattermost.database.username"
      password: ".addons.mattermost.database.password"

... But getting it into the second location in the test-values for 0the istio hardening egress gateway rule is non-trivial.

      istio:
        hardened:
          customServiceEntries:
            - name: "postgresql-service-entries"
              enabled: true 
              spec:
                hosts:
                  - "cirds.cqkqilzbp4x2.us-gov-west-1.rds.amazonaws.com"
                location: MESH_EXTERNAL
                ports:
                  - number: 5432
                    protocol: TCP
                    name: postgresql
                resolution: DNS

In the current case being presented in the MR, the hostname for the CI RDS database is being hardcoded as a convenience. One could try to use a helm value reference, like so:

      istio:
        hardened:
          customServiceEntries:
            - name: "postgresql-service-entries"
              enabled: true 
              spec:
                hosts:
                  - "{{ .database.host }}"

... However, given the way this particular package's values are managed and the way its templates are rendered in the umbrella, I'm not clear if the location in question would be available to this chart when Helm finally got around to rendering it. I wasn't able to make it work in my initial quick testing, perhaps I didn't look hard enough.

Either way, the package maintainer is left with two options:

1. Modify the package mapping configuration to inject the value into one central location that is guaranteed to be available by all helm operations which might need it, and modify all templates to point to that single reference. This is cleaner in some aspects, but can be a little bit mind-bending for the template author given the way the umbrella pipeline moves values around.
2. Modify the package mapping configuration to explicitly inject the value into whatever locations need it. This leads to a more verbose configuration, and a slightly more complicated pipeline logic, but IMO is more easily grokked form a single location.

This MR presumes that #2 (closed) is the better approach, and implements changes to the rds_mapvalues method to allow for the package maintainer to specify each element in the map as either a string or an array of strings, specifying either a single or multiple destinations for a given configuration element.

bigbang_ci:
  rds:
    host: 
      - .stub.config.host
      - .stub.config.host_two
    database: 
      - .stub.config.database
    port: .stub.config.port
    username: .stub.config.username
    password: .stub.config.password

In this example, the generated database hostname is injected into two locations. The database name is injected into a single location, even though it is specified as a list. The other items are specified as a single string specifying a single destination.

Relevant logs/screenshots

One new test added to exercise this functionality, all tests passing

$ BATS_RDS_USER_DOCKER=true bats ci_rds_test.sh 
ci_rds_test.sh
 ✓ rds_requested enabled
 ✓ rds_new_databasename format
 ✓ rds_newdb internal psql
 ✓ rds_requested disabled
 ✓ rds_requested malformed
 ✓ rds_mapvalues complete
 ✓ rds_mapvalues complete arrays
 ✓ rds_mapvalues incomplete map definition
 ✓ rds_mapvalues missing files
 ✓ rds_create_multiple 
 ✓ rds_create notenabled
 ✓ rds_create creator fails
 ✓ rds_create creator succeeds
 ✓ rds_delete internal
 ✓ rds_purge internal
 ✓ rds_values_merge fails with missing files
 ✓ rds_values_merge fails with invalid yaml
 ✓ rds_values_merge succeeds with valid yaml files
 ✓ rds_get_accessor_role sets valid credentials
 ✓ rds_get_accessor_role only assumes the role once within the lifetime

20 tests, 0 failures

Linked Issue

Discovered while working bigbang MR #5482 for mattermost issue #168

Upgrade Notices

N/A