Discovering The Truth Behind Kubernetes Secrets

We have been talking recently about ConfigMap being one of the objects to store a different configuration for Kubernetes based-workloads. But what happens with sensitive data?

This is an interesting question, and the initial answer from the Kubernetes platform was to provide a Secrets object. Based on its definition from the Kubernetes official website, they define secrets like this:

A Secret is an object that contains a small amount of sensitive data such as a password, a token, or a key. Such information might otherwise be put in a Pod specification or in a container image. Using a Secret means that you don’t need to include confidential data in your application code

So, by default, secrets are what you should use to store your sensitive data. From the technical perspective, to use them, they behave very similar to ConfigMap, so you can link it to the different environment variables, mount it inside a pod, or even have specific usages for managing credentials for different kinds of accounts such as Service Accounts. This classifies the different types of secrets that you can create:

• Opaque: This defines a generic secret that you can use for any purpose (mainly configuration data or configuration files)
• Service-Account-Token: This defines the credentials for service accounts, but this is deprecated and no longer in use since Kubernetes 1.22.
• Docker-Registry Credentials: This defines credentials to connect to the Docker registry to download images as part of your deployment process.
• Basic or SSH Auth: This defines specific secrets to handle authentication.
• TLS Secret:
• Bootstrap Secrets:
  

But is it safe to use Kubernetes Secrets to store sensitive data? The main answer for any question in any tech-related topic is: It depends. But some controversy has arisen that this topic is also covered in the official Kubernetes page, highlighting the following aspects:

Kubernetes Secrets are, by default, stored unencrypted in the API server’s underlying data store (etcd). Anyone with API access can retrieve or modify a Secret, and so can anyone with access to etcd. Additionally, anyone who is authorized to create a Pod in a namespace can use that access to read any Secret in that namespace; this includes indirect access such as the ability to create a Deployment.

So, the main thing is, by default, this is a very, very insecure way. It seems more like a categorization of the data than a proper secure handle. Also, Kubernetes provide some tips to try to make this alternative more secure:

• Enable Encryption at Rest for Secrets.
• Enable or configure RBAC rules that restrict reading data in Secrets (including indirect means).
• Where appropriate, also use mechanisms such as RBAC to limit which principals are allowed to create new Secrets or replace existing ones.
  

But that can be not enough, and that has created room for third-party and cloud providers to provide their solution that covers these needs and at the same time also offer additional features. Some of these options are the ones shown below:

• Cloud Key Management Systems: Pretty much all the big cloud providers provide some way of Secret Management to go beyond these features and mitigate those risks. If we talk about AWS, there is AWS Secrets Manager , if we are talking about Azure, we have Azure Key Vault , and in the case of Google, we also have Google Secret Manager.
  
• Sealed Secrets is a project that tries to extend Secrets to provide more security, especially on the Configuration as a Code approach, offers a safe way to store those objects in the same kind of repositories as you expose any other Kubernetes resource file. In its own words, “ The SealedSecret can be decrypted only by the controller running in the target cluster, and nobody else (not even the original author) can obtain the original Secret from the SealedSecret.”
  
• Third-party Secrets Managers that are similar to the ones from the Cloud Providers that allows a more independent approach, and there are several players here such as Hashicorp Vault or CyberArk Secret Manager
  
• Finally also, Spring Cloud Config can provide security to store data that are related to sensitive configuration concepts such as passwords and at the same time covers the same need as the ConfigMap provides from a unified perspective.
  

I hope this article has helped to understand the purpose of the Secrets in Kubernetes and, at the same time, the risks regarding its security and how we can mitigate them or even rely on other solutions that provide a more secure way to handle this critical piece of information.