SWITCH Cloud Blog


Leave a comment

Enable Keystone federated users to use CLI tools with Application Credentials

On this blog I talked in the past about Keystone authentication for your Kubernetes cluster. The solution described works great if you have Openstack users stored in the keystone mysql database. However, in real production systems, it is common to access Openstack with a Federated login. The web login works with a redirect to an Identity provider that will confirm the user identity and will redirect again to the Openstack dashboard.

It is a well known problem that the federated login process needs to go through web pages redirects to enter the necessary information, and this does not work for users that need to authenticate with CLI tools. In our case the CLI tool is kubectl.

A small team of people at SWITCH and GARR worked jointly to find a solution for this use case.

The good news is that the Keystone developers already implemented a solution for this problem, the Keystone Application Credentials. This is a feature available since the Queens release of Openstack. The key idea is that a user can login on the web interface with the federated login process, and then from the dashboard identity panel he can create new credentials to be consumed directly from CLI tools.

The following three screenshoots show the user journey to create an Application Credential in the Openstack Horizon dashboard:

Select Application Credential in the Identity Panel

Select Application Credential in the Identity Panel

 

 

Enter the data to create an application credential

Enter the data to create an application credential

 

 

Download the openrc file to store your application credential

Download the openrc file to store your application credential

 

So what is missing to authenticate with kubectl using a keystone application credential ?

Starting at kubectl v1.11 the specific cloud providers API implementations were moved out of the kubernetes source tree. You have to modify our client configuration as follows:

contexts:
- context:
    cluster: kubernetes
    namespace: keystone-daadb4bcc9704054b108de8ed263dfc2
    user: openstackgarr
  name: garr

users:
- name: openstackgarr
  user:
    exec:
      apiVersion: client.authentication.k8s.io/v1alpha1
      command: /bin/client-keystone-auth
      args:
      - --domain-name=myDomain
      - --keystone-url=https://keystone.doman.tld:5000/v3
      env:
      - name: OS_USERNAME
        value: username
      - name: OS_PASSWORD
        value: secret
      - name: OS_PROJECT_NAME
        value: test
 

Note that in this configuration snippet we are still using username and password. If you want to test this setup make sure the client-keystone-auth version is newer than v0.2.0 or that is patched including commit 66961abd. The version v0.2.0 is not able to request keystone project scoped token, so your setup will not work.

The software client-keystone-auth uses the golang library gophercloud to talk to the Kubernetes API.

To reach our goal, the first step was to patch gophercloud to implement the application_credential authentication method described in the Queens spec.

This patch enables any golang application to easily access the application credentials authentication method, so it could be useful to other golang software tools, like for example Terraform.

Please note that the patch implements just the possibility to issue a token authenticating with the application credential. I did not propose a gophercloud patch that implements the full create/update/delete workflow for application credentials, because this went beyond the scope of this work.

Once the gophercloud PR was merged I proposed the a PR for client-keystone-auth to use the new gophercloud feature.

At the time of this writing the latest PR is still not merged, so you will need to compile the code yourself to test it.

Now that all the code is there, we can change the client configuration to use the application credential instead of username and password as we show in the example below:

contexts:
- context:
    cluster: kubernetes
    namespace: keystone-daadb4bcc9704054b108de8ed263dfc2
    user: openstackgarr
  name: garr

users:
- name: openstackgarr
  user:
    exec:
      apiVersion: client.authentication.k8s.io/v1alpha1
      command: /bin/client-keystone-auth
      args:
      - --domain-name=myDomain
      - --keystone-url=https://keystone.doman.tld:5000/v3
      - --user-name=username
      - --application-credential-name=kuberneteslogin
      - --application-credential-secret=thisismysecret 

You can also use environment variables instead of command line arguments. The client-keystone-auth supports the same variable names as the official openstack client.

Conclusion: you can today use kubectl with your keystone application credentials, this is really useful if you have a federated login to the Openstack cloud.

I would like to thank the people that participated to this development work with me, especially Giuseppe Attardi , Roberto Di Lallo  and Joe Topjian that helped in implementation, discussion and code review.