Authservice cannot use JWKS URI when using the SSO IdP uses a custom CA

Bug

Description

Thanks to @kevin.wilder for the help in discovering / triaging this.

When you are using SSO with an IdP that has a custom CA, Authservice uses the value of sso.certificateAuthority.cert to communicate with the IdP. This works as expected in most cases. However, specifically when using jwksUri, Authservice does not load this CA cert and fails to grab the JWKS data. This occurs even if you specify jwks.

BigBang Version

This applies to every version of Big Bang if using jwksUri, but only after 1.52.0 if using hardcoded jwks.

added authservice label

@kevin.wilder and I have the following solution:

# Global SSO parameters
sso:
  name: SSO
  url: https://keycloak.<DOMAIN>/auth/realms/baby-yoda
  saml:
    # Retrieve from 
    # curl https://keycloak.<DOMAIN>/auth/realms/baby-yoda/protocol/saml/descriptor; echo
    metadata: <md:EntityDescriptor xmlns="urn:oasis:names:tc:SAML:2.0:metadata" xmlns:md="urn:oasis:names:tc:SAML:2.0:metadata" xmlns:saml="urn:oasis:names:tc:SAML:2.0:assertion" xmlns:ds="http://www.w3.org/2000/09/xmldsig#" entityID="https://keycloak.<DOMAIN>/auth/realms/baby-yoda">...</md:EntityDescriptor>

  oidc:
    # The JSON Web Key Set (JWKS) containing the public keys used to verify any JSON Web Token (JWT) issued by the IDP
    # The jwks is used by Istio authservice
    # Must be updated for every new deployment of keycloak.  Example of where to get the jwks
    # https://keycloak.<DOMAIN>/auth/realms/baby-yoda/protocol/openid-connect/certs
    # double quotes must be escaped  \"xxxx\"
    # curl https://keycloak.<DOMAIN>/auth/realms/baby-yoda/protocol/openid-connect/certs | sed 's@"@\\"@g'); echo
    jwks: "{\"keys\":[...]}"
    # Recent versions of authservice allow filling in of a URI for jwks which will be fetched on your behalf
    jwksUri: null


  # private CA for IdP
  certificateAuthority:
    cert: |
      -----BEGIN CERTIFICATE-----
      MII...
      -----END CERTIFICATE-----

Ultimately, it would be preferred if Authservice could mount the CA certificate in the pod so that we can use JWKS URI.

Here is the error when using a private CA. This makes sense because authservice tries to use jwksUri and does not trust it. So then authservice SSO fails because authservice does not have the jwks info that is needed.

[2023-06-14 17:26:02.256] [console] [error] Get: unexpected exception: CERTIFICATE_VERIFY_FAILED (SSL routines, OPENSSL_internal) [asio.ssl:268435581]
[2023-06-14 17:26:02.256] [console] [warning] operator(): HTTP connection error

The workaround solution of setting jwks and nulling jwksUri is undocumented as the jwks key was removed from all relevant documentation and example configs around 4 months ago.
@ryan.j.garcia if you would grant me developer role on the bigbang repository I will open a documentation MR.

The ideal solution is for the authservice package chart to volume mount the private certificate authority cert in the appropriate place so that the authservice container will trust the private CA and be able to use the jwksUri.

Perhaps bigbang would prefer to go for the ideal package fix solution rather than update documentation with a temporary workaround.

added community-contribution label

added 1 deleted label

mentioned in issue #1564 (closed)

mentioned in merge request big-bang/product/packages/authservice!110 (merged)

mentioned in merge request !2896 (merged)

closed with merge request !2896 (merged)

mentioned in commit f929f2e0

Can you please re-open this @ryan.j.garcia ? We are still seeing this error with authservice chart 0.5.3-bb.11.

While /etc/pki/tls/certs/oidc-ca.crt is present in the pod, it needs to be added to the default CA bundle for most http clients.

Just to clarify, this change results in exactly the same issue? Like you cannot use JWKS URI? Or are you facing a different but similar issue? It might be worth opening a new issue, as both Kevin and I solved our JWKS URI CA cert issue via this method.

I was assuming this issue was not closed due to the workaround pointed out in #1555 (comment 1350768), but rather due to the change in !2896 (merged).

That is correct.

changed milestone to %2.6.0