Velero - Unable to restore PersistentVolumes due to x509 Errors

The Classified Deployments team has been path finding with Velero ahead of the Big Bang velero-operator release as we have an urgent need to take and restore backups of several dynamically provisioned PVCs that we have running with our clusters. What we're seeing in the classified spaces is that Backups are completing successfully but restores are not.

Restores fail with the following approximate error:

error executing PVAction for persistent volumes
Caused by POST https://ec2.region.gov    x509 certificate signed by unknown authority

For whatever reason, velero is not able to successfully create new PersistentVolumes from Snapshots. On the other hand it doesn't seem to have any issues creating Snapshots of running volumes and placing all associated metadata into an S3 bucket of our choosing.

We're using the following install command, using the velero CLI, to create a YAML manifest which we edit slightly to add in additional proxy environment variables to velero.

velero install --dry-run \
               --output yaml \
               --provider aws \
               --bucket velero-backups \
               --backup-location-config region=us-gov-west-1 \
               --snapshot-location-config region=us-gov-west-1 \
               --image localhost:5000/ironbank/opensource/velero/velero:v1.5.3 \
               --plugins localhost:5000/ironbank/opensource/velero/velero-plugin-for-aws:v1.2.0 \
               --no-secret \
               --prefix blue-k8s-cluster \
               --cacert /path/to/highside/bundle.crt > velero.yaml

You'll note that we actually pass in a CA Bundle with the --cacert but from what we can tell this only gets added to the BackupStorageLocation CRD and isn't used anywhere else.

Note: Backups and Restores both work in our low-side UC2S environment in us-gov-west-1 which does not require any additional certificates, just proxy settings.

This issue serves more as a flagpole to call this out to attention and to identify that the velero-operator will probably need to be able to take in a certificate file as well in addition to whatever the fix is for the x509 issue on the Restore action.