UNCLASSIFIED - NO CUI

Skip to content

Keycloak Cluster

We have deployed Keycloak on Big Bang 1.47.0 using default configuration for KeyCloak on AWS. It seems the clustering is not working. When we create a new realms, sometimes we do not see the realm and sometimes it shows up. Here is our log, which indicates both instances of keycloak are running in standalone mode. We get the similar logs in keycloak-0 instance. When we enable the DEBUG logs, we see that the DNS_PING returns two IPs.

We need help in setting up keycloak in a clustered mode in our Big Bang environment. The keycloak app itself works fine. We do have passthrough Istio gateway set up for keycloak.

Setting JGroups discovery to dns.DNS_PING with properties {dns_query=>keycloak-headless.keycloak.svc.cluster.local}
19:40:14,181 INFO  [org.jboss.as.clustering.jgroups] (ServerService Thread Pool -- 42) WFLYCLJG0001: Activating JGroups subsystem. JGroups version 4.2.20
19:40:14,207 INFO  [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 38) WFLYCLINF0001: Activating Infinispan subsystem.
19:40:14,783 WARN  [org.wildfly.clustering.web.undertow] (ServerService Thread Pool -- 54) WFLYCLWEBUT0007: No routing provider found for default-server; using legacy provider based on static configuration
19:40:18,103 INFO  [org.jboss.modcluster] (ServerService Thread Pool -- 56) MODCLUSTER000001: Initializing mod_cluster version 1.4.4.Final
19:40:18,199 INFO  [org.jboss.modcluster] (ServerService Thread Pool -- 56) MODCLUSTER000032: Listening to proxy advertisements on /224.0.1.105:23364
19:40:23,097 INFO  [org.jgroups.protocols.pbcast.GMS] (ServerService Thread Pool -- 56) keycloak-1: no members discovered after 3016 ms: creating cluster as coordinator
19:40:25,482 INFO  [org.infinispan.CONTAINER] (ServerService Thread Pool -- 60) ISPN000556: Starting user marshaller 'org.wildfly.clustering.infinispan.marshalling.jboss.JBossMarshaller'
19:40:25,482 INFO  [org.infinispan.CONTAINER] (ServerService Thread Pool -- 58) ISPN000556: Starting user marshaller 'org.wildfly.clustering.infinispan.marshalling.jboss.JBossMarshaller'
19:40:25,485 INFO  [org.infinispan.CONTAINER] (ServerService Thread Pool -- 61) ISPN000556: Starting user marshaller 'org.wildfly.clustering.infinispan.spi.marshalling.InfinispanProtoStreamMarshaller'
19:40:25,486 INFO  [org.infinispan.CONTAINER] (ServerService Thread Pool -- 59) ISPN000556: Starting user marshaller 'org.wildfly.clustering.infinispan.spi.marshalling.InfinispanProtoStreamMarshaller'
19:40:25,489 INFO  [org.infinispan.CONTAINER] (ServerService Thread Pool -- 56) ISPN000556: Starting user marshaller 'org.wildfly.clustering.infinispan.spi.marshalling.InfinispanProtoStreamMarshaller'
19:40:25,997 INFO  [org.infinispan.CLUSTER] (ServerService Thread Pool -- 56) ISPN000094: Received new cluster view for channel ejb: [keycloak-1|0] (1) [keycloak-1]
19:40:26,097 INFO  [org.infinispan.CLUSTER] (ServerService Thread Pool -- 58) ISPN000094: Received new cluster view for channel ejb: [keycloak-1|0] (1) [keycloak-1]
19:40:26,098 INFO  [org.infinispan.CLUSTER] (ServerService Thread Pool -- 59) ISPN000094: Received new cluster view for channel ejb: [keycloak-1|0] (1) [keycloak-1]
19:40:26,098 INFO  [org.infinispan.CLUSTER] (ServerService Thread Pool -- 61) ISPN000094: Received new cluster view for channel ejb: [keycloak-1|0] (1) [keycloak-1]
19:40:26,098 INFO  [org.infinispan.CLUSTER] (ServerService Thread Pool -- 60) ISPN000094: Received new cluster view for channel ejb: [keycloak-1|0] (1) [keycloak-1]
19:40:27,701 INFO  [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 60) WFLYCLINF0002: Started sessions cache from keycloak container
19:40:27,705 INFO  [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 61) WFLYCLINF0002: Started offlineSessions cache from keycloak container
19:40:27,779 INFO  [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 63) WFLYCLINF0002: Started actionTokens cache from keycloak container
19:40:27,783 INFO  [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 66) WFLYCLINF0002: Started clientSessions cache from keycloak container
19:40:27,783 INFO  [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 67) WFLYCLINF0002: Started authenticationSessions cache from keycloak container
19:40:27,787 INFO  [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 64) WFLYCLINF0002: Started work cache from keycloak container
19:40:27,883 INFO  [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 65) WFLYCLINF0002: Started offlineClientSessions cache from keycloak container
19:40:27,890 INFO  [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 68) WFLYCLINF0002: Started loginFailures cache from keycloak container
19:40:27,891 INFO  [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 59) WFLYCLINF0002: Started keys cache from keycloak container
19:40:27,892 INFO  [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 56) WFLYCLINF0002: Started realms cache from keycloak container
19:40:27,892 INFO  [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 62) WFLYCLINF0002: Started users cache from keycloak container
19:40:27,894 INFO  [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 58) WFLYCLINF0002: Started authorization cache from keycloak container
19:40:27,984 INFO  [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 64) WFLYCLINF0002: Started http-remoting-connector cache from ejb container
19:40:30,318 INFO  [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 64) WFLYCLINF0002: Started realmRevisions cache from keycloak container
19:40:30,383 INFO  [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 64) WFLYCLINF0002: Started userRevisions cache from keycloak container
19:40:30,389 INFO  [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 64) WFLYCLINF0002: Started authorizationRevisions cache from keycloak container
[root@ip-172-31-10-226 ec2-user]# kubectl logs -n keycloak keycloak-0 | grep cluster
Setting JGroups discovery to dns.DNS_PING with properties {dns_query=>keycloak-headless.keycloak.svc.cluster.local}
19:42:39,812 INFO  [org.jboss.as.clustering.jgroups] (ServerService Thread Pool -- 42) WFLYCLJG0001: Activating JGroups subsystem. JGroups version 4.2.20
19:42:40,200 INFO  [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 38) WFLYCLINF0001: Activating Infinispan subsystem.
19:42:40,311 WARN  [org.wildfly.clustering.web.undertow] (ServerService Thread Pool -- 54) WFLYCLWEBUT0007: No routing provider found for default-server; using legacy provider based on static configuration
19:42:42,622 INFO  [org.jboss.modcluster] (ServerService Thread Pool -- 56) MODCLUSTER000001: Initializing mod_cluster version 1.4.4.Final
19:42:42,703 INFO  [org.jboss.modcluster] (ServerService Thread Pool -- 56) MODCLUSTER000032: Listening to proxy advertisements on /224.0.1.105:23364
19:42:48,020 INFO  [org.jgroups.protocols.pbcast.GMS] (ServerService Thread Pool -- 56) keycloak-0: no members discovered after 3012 ms: creating cluster as coordinator
19:42:50,513 INFO  [org.infinispan.CONTAINER] (ServerService Thread Pool -- 58) ISPN000556: Starting user marshaller 'org.wildfly.clustering.infinispan.marshalling.jboss.JBossMarshaller'
19:42:50,514 INFO  [org.infinispan.CONTAINER] (ServerService Thread Pool -- 59) ISPN000556: Starting user marshaller 'org.wildfly.clustering.infinispan.marshalling.jboss.JBossMarshaller'
19:42:50,517 INFO  [org.infinispan.CONTAINER] (ServerService Thread Pool -- 56) ISPN000556: Starting user marshaller 'org.wildfly.clustering.infinispan.spi.marshalling.InfinispanProtoStreamMarshaller'
19:42:50,519 INFO  [org.infinispan.CONTAINER] (ServerService Thread Pool -- 57) ISPN000556: Starting user marshaller 'org.wildfly.clustering.infinispan.spi.marshalling.InfinispanProtoStreamMarshaller'
19:42:50,520 INFO  [org.infinispan.CONTAINER] (ServerService Thread Pool -- 60) ISPN000556: Starting user marshaller 'org.wildfly.clustering.infinispan.spi.marshalling.InfinispanProtoStreamMarshaller'
19:42:51,107 INFO  [org.infinispan.CLUSTER] (ServerService Thread Pool -- 58) ISPN000094: Received new cluster view for channel ejb: [keycloak-0|0] (1) [keycloak-0]
19:42:51,107 INFO  [org.infinispan.CLUSTER] (ServerService Thread Pool -- 59) ISPN000094: Received new cluster view for channel ejb: [keycloak-0|0] (1) [keycloak-0]
19:42:51,109 INFO  [org.infinispan.CLUSTER] (ServerService Thread Pool -- 57) ISPN000094: Received new cluster view for channel ejb: [keycloak-0|0] (1) [keycloak-0]
19:42:51,109 INFO  [org.infinispan.CLUSTER] (ServerService Thread Pool -- 56) ISPN000094: Received new cluster view for channel ejb: [keycloak-0|0] (1) [keycloak-0]
19:42:51,110 INFO  [org.infinispan.CLUSTER] (ServerService Thread Pool -- 60) ISPN000094: Received new cluster view for channel ejb: [keycloak-0|0] (1) [keycloak-0]
19:42:52,718 INFO  [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 56) WFLYCLINF0002: Started actionTokens cache from keycloak container
19:42:52,718 INFO  [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 57) WFLYCLINF0002: Started sessions cache from keycloak container
19:42:52,722 INFO  [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 66) WFLYCLINF0002: Started offlineSessions cache from keycloak container
19:42:52,802 INFO  [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 60) WFLYCLINF0002: Started authorization cache from keycloak container
19:42:52,721 INFO  [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 59) WFLYCLINF0002: Started loginFailures cache from keycloak container
19:42:52,723 INFO  [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 64) WFLYCLINF0002: Started work cache from keycloak container
19:42:52,727 INFO  [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 67) WFLYCLINF0002: Started clientSessions cache from keycloak container
19:42:52,729 INFO  [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 58) WFLYCLINF0002: Started users cache from keycloak container
19:42:52,731 INFO  [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 65) WFLYCLINF0002: Started realms cache from keycloak container
19:42:52,731 INFO  [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 68) WFLYCLINF0002: Started offlineClientSessions cache from keycloak container
19:42:52,802 INFO  [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 62) WFLYCLINF0002: Started keys cache from keycloak container
19:42:52,807 INFO  [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 63) WFLYCLINF0002: Started authenticationSessions cache from keycloak container
19:42:52,907 INFO  [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 69) WFLYCLINF0002: Started http-remoting-connector cache from ejb container
19:42:54,811 INFO  [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 63) WFLYCLINF0002: Started realmRevisions cache from keycloak container
19:42:54,815 INFO  [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 63) WFLYCLINF0002: Started userRevisions cache from keycloak container
19:42:54,819 INFO  [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 63) WFLYCLINF0002: Started authorizationRevisions cache from keycloak container

UNCLASSIFIED - NO CUI