Keep being redirected to login page with Keycloak OIDC provider and pinniped

[rbuffi]

My goal is to authenticate to kubeapps with keycloak and pinniped. I have configured everything but i keep bein redirected to the login page...

Here is my values.yaml:

authProxy:

  enabled: true

  skipKubeappsLoginPage: false

  provider: oidc

  clientID: kubeapps

  clientSecret: xxxx

  cookieSecret: xxx

  emailDomain: "*"

  extraFlags:

    - --cookie-refresh=0

    - --ssl-insecure-skip-verify

    - --cookie-secure=false

    - --scope=openid groups email

    - --oidc-issuer-url=https://kc.testlab.xxxx.local/realms/kubeapps

    - --pass-authorization-header=true
 

pinnipedProxy:

  enabled: true

  clusters:

     - name: default

       apiServiceURL: https://x.x.x.x/

       certificateAuthorityData: xxxx

       isKubeappsCluster: true

       pinnipedConfig:

         enabled: true

I now have set up the impersonation proxy:

apiVersion: v1

items:

- apiVersion: [config.concierge.pinniped.dev/v1alpha1](http://config.concierge.pinniped.dev/v1alpha1)

  kind: CredentialIssuer

  metadata:

    creationTimestamp: "2024-06-25T14:36:04Z"

    generation: 2

    labels:

      app: pinniped-concierge

    name: pinniped-concierge-config

    resourceVersion: "16012020"

    uid: a6b6b570-311b-4b00-9706-71f44671cfa7

  spec:

    impersonationProxy:

      mode: enabled

      service:

        annotations:

          [service.beta.kubernetes.io/aws-load-balancer-connection-idle-timeout](http://service.beta.kubernetes.io/aws-load-balancer-connection-idle-timeout): "4000"

        type: LoadBalancer

  status:

    strategies:

    - lastUpdateTime: "2024-06-25T14:36:13Z"

      message: could not find a healthy kube-controller-manager pod (0 candidates)

      reason: CouldNotFetchKey

      status: Error

      type: KubeClusterSigningCertificate

    - frontend:

        impersonationProxyInfo:

          certificateAuthorityData: xxxx

          endpoint: https://x.x.x.x/

        type: ImpersonationProxy

      lastUpdateTime: "2024-06-25T22:41:48Z"

      message: impersonation proxy is ready to accept client connections

      reason: Listening`

And jwtauthenticator:

apiVersion: v1

items:

- apiVersion: [authentication.concierge.pinniped.dev/v1alpha1](http://authentication.concierge.pinniped.dev/v1alpha1)

  kind: JWTAuthenticator

  metadata:

    creationTimestamp: "2024-06-26T00:20:50Z"

    generation: 1

    name: jwt-authenticator

    resourceVersion: "16033939"

    uid: ac12cf5c-228d-494c-9f1f-80044a75f01c

  spec:

    audience: kubeapps

    claims:

      groups: groups

      username: email

    issuer: https://kc.testlab.x.x/realms/kubeapps

    tls:

      certificateAuthorityData: xxxx
kind: List

metadata:

  resourceVersion: ""

With this config i'm able to authenticate to kubeapps with keycloak but after authentication i'm being redirected to the login page. In kubeapps auth-proxy pod logging I see nothing strange and nothing being logged in pinniped-proxy pod!

[10.244.1.1:45115](http://10.244.1.1:45115/) - 372269b1-2a3d-4de1-88b6-31843b95e5e5 - [user@anonymous.nl](mailto:user@anonymous.nl) [2024/06/26 08:31:24] [AuthSuccess] Authenticated via OAuth2: Session{email:user@anonymous user:93424824-a080-4690-ae1d-8346c40efc0e [PreferredUsername:user@anonymous.nl](mailto:PreferredUsername%3Auser@anonymous.nl) token:true id_token:true created:2024-06-26 08:31:24.585448393 +0000 UTC m=+2920.873606365 expires:2024-06-26 08:36:24.500799825 +0000 UTC m=+3220.788957799 refresh_token:true groups:[kubeapps-admin]}

[10.244.1.1:45115](http://10.244.1.1:45115/) - 372269b1-2a3d-4de1-88b6-31843b95e5e5 - - [2024/06/26 08:31:24] 192.168.210.116 GET - "/oauth2/callback?state=7mtfXVKtt4-AbTYHzCvZIlvAizmJ1CdwH-LIu2rPo_s%3A%2F&session_state=96c2dfdb-3722-4d3d-bb52-e54c3d501829&iss=https%3A%2F%2Fkc.testlab.xxx.local%2Frealms%2Fkubeapps&code=4fa74f08-ca24-4193-8bb1-d0db9b293f4f.96c2dfdb-3722-4d3d-bb52-e54c3d501829.cb382bec-bc96-4750-a889-7e34456c8a8d" HTTP/1.1 "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/[126.0.0.0](http://126.0.0.0/) Safari/53

But in the apiserver logging I see the following:

I0626 08:56:41.131411       1 handler.go:232] Adding GroupVersion [identity.concierge.pinniped.dev](http://identity.concierge.pinniped.dev/) v1alpha1 to ResourceManager

I0626 08:56:41.144661       1 handler.go:232] Adding GroupVersion [login.concierge.pinniped.dev](http://login.concierge.pinniped.dev/) v1alpha1 to ResourceManager

E0626 08:57:06.728431       1 controller.go:102] loading OpenAPI spec for "[v1alpha1.identity.concierge.pinniped.dev](http://v1alpha1.identity.concierge.pinniped.dev/)" failed with: failed to download [v1alpha1.identity.concierge.pinniped.dev](http://v1alpha1.identity.concierge.pinniped.dev/): resource not found

I0626 08:57:06.728494       1 controller.go:109] OpenAPI AggregationController: action for item [v1alpha1.identity.concierge.pinniped.dev](http://v1alpha1.identity.concierge.pinniped.dev/): Rate Limited Requeue.

E0626 08:57:06.828889       1 controller.go:102] loading OpenAPI spec for "[v1alpha1.login.concierge.pinniped.dev](http://v1alpha1.login.concierge.pinniped.dev/)" failed with: failed to download [v1alpha1.login.concierge.pinniped.dev](http://v1alpha1.login.concierge.pinniped.dev/): resource not found

 1 authentication.go:73] "Unable to authenticate the request" err="invalid bearer token"
When I try to decode the token as described (https://kubeapps.dev/docs/latest/howto/oidc/oauth2oidc-debugging/)
I get the following error:

{"alg":"RS256","typ" : "JWT","kid" : "wkF65vug7ZdfpsKzc5Fpt_qCUHNZo_37uwxhDzoU5v8"}base64: invalid input

In the concierge pod logging I do not see any token requests.

I able to get token with pinniped-cli and keycloak/pinniped impersonating proxy:

pinniped-cli-windows-amd64.exe login oidc --issuer https://kc.testlab.xxx.local/realms/kubeapps --ca-bundle-data XXXX  --client-id kubeapps --enable-concierge --concierge-endpoint https://192.168.x.x --concierge-authenticator-name jwt-authenticator --concierge-authenticator-type jwt --scopes openid,groups,email --concierge-ca-bundle-data xxxx

`Wed, 26 Jun 2024 14:30:07 CEST rest/warnings.go:70 Use tokens from the TokenRequest API or manually created secret-based tokens instead of auto-generated secret-based tokens.

Result:
 {"kind":"ExecCredential","apiVersion":"[client.authentication.k8s.io/v1beta1](http://client.authentication.k8s.io/v1beta1)","spec":{"interactive":false},"status":{"expirationTimestamp":"2024-06-26T12:35:07Z","clientCertificateData":"-----BEGIN CERTIFICATE-----\nCERTIFICATE\n-----END CERTIFICATE-----\n","clientKeyData":"-----BEGIN PRIVATE KEY-----\nKEY\n-----END PRIVATE KEY-----\n"}}`

• Version: latest
• Talos version 1.7
• K8s version 1.28