In 2022, KSOC’s own Jimmy Mesta initiated the OWASP Top 10 for Kubernetes open source project. Given the size and complexity of Kubernetes deployments, Kubernetes security blindspots present a significant challenge for teams. Like other OWASP Top-10 lists, this one provides an excellent starting point for tackling those challenges.
This blog will detail the top 10 prioritized risks any team handling Kubernetes should be familiar with.
1. Insecure Workload Configurations
The applications you are running on your cluster are the most visible thing to outsiders, present the largest attack surface, and are the most likely to contain security vulnerabilities or misconfigurations. The details or those possible vulnerabilities or misconfigurations are as varied as the possible workloads you could be running, so it is up to you and your application teams to determine the precise hardening steps for your specific environment. That said, as a general rule, there are a few things you can do by default that will provide a layer of defense against many vulnerabilities or misconfigurations in those workloads:
- Prevent containers from running as root or pods from running in “privileged mode.” This will protect your clusters from container escape attacks in which vulnerabilities in a workload are used as an attack vector against the nodes in your cluster themselves, potentially granting bad actors the ability to do all kinds of devious things to your other workloads, data sources, or infrastructure.
- Limit container access to sensitive volumes or directories. Your workloads may need access to some of these data sources in order to do their work. Think carefully about what access they need and whether it needs to be read-write or just read access.
2. Supply Chain Vulnerabilities
Concerns about the security of your workloads also extend to the security of the supply chains and sources from which those workloads are drawn. A supply chain compromise such as a bad mirror, a typo-squat attack, or a solar-winds-esque back-dooring of a package could result in you unknowingly running intentionally vulnerable packages. Following best practices for hardening those packages may not be enough to mitigate intentional inserted vulnerabilities or malware. Your best defense is to take action at the cluster level by implementing limits on the sources from which you can install container images and performing validation of those images during admissions control.
3. Overly Permissive RBAC Configurations
One of the most tried and true security principles across all technology stacks is the Principle of Least Privilege: only give users/actors the permissions they explicitly need to perform the actions they are tasked with. This principle substantially narrows the number of things an attacker can do with a compromised account, and when couple with strong authentication (see number 6 on the list) makes it very difficult for attackers to elevate their privileges to be able to take more destructive actions. In the Kubernetes landscape, the consensus method for implementing the Principle of Least Privilege is to use Role Based Access Control, or RBAC, to assign permissions to users. RBAC is not easy to implement, and a common cause of security vulnerabilities within Kubernetes clusters is roles provisioned with too many permissions. KSOC can help audit assigned permissions and actual usage of those permissions across your cluster, helping you scope your roles appropriately.
4. Lack of Centralized Policy Enforcement
The architecture of your cluster may introduce multiple ways of establishing and enforcing policies. A service mesh may supply its own mechanism for network routing policies, for example. Similarly, admissions control may provide a mechanism for defining policies around what images are allowed in the cluster. If these different components are being managed by separate teams or are drawing their policies from separate policy stores, you may find that this complexity results in gaps and conflicts. This, in turn, may cause vulnerabilities that you thought had been addressed to persist. As a result, its worth having a centralized policy enforcement tool, using something like OPA to store and distribute policies across all the various enforcement tools available in your cluster.
5. Inadequate Logging and Monitoring
You can’t protect your cluster if you don’t know what’s happening in your cluster. If you have visibility blindspots resulting from inadequate logging or monitoring, you will have security blindspots, too. As a result, you need to be collecting appropriate logs so that you can have the visibility necessary for cluster defense. A commonly overlooked source of these logs is the host nodes in your cluster themselves.
6. Broken Authentication Mechanisms
Access to your cluster needs to be brokered by a sufficiently strong authentication mechanism. This includes both administrative access to manage cluster resources themselves and user access to the workloads running on your cluster. These will probably use different authentication mechanisms, but both should be adequately secured so as to provide a strong assurance about user identity during authentication. Ideally, this means using multi-factor authentication—a must for administrative access to your cluster, and depending on the nature of your workloads either a requirement or strong recommendation to users.
7. Missing Network Segmentation Controls
At its core, Kubernetes is an abstraction over infrastructure resources that allows hosting applications across hosts, data centers, or even cloud providers. To do that, Kubernetes has to handle a fair amount of networking. That is meant to be transparent to your workloads, but at the cluster level you need to architect that networking securely. This is especially true for cluster ingress and egress: you should know what load balancers and ingress gateways exist on your network, what external resources are allowed to access those ingresses, and similarly what egresses are available to users and services in your cluster and what external resources are accessible through those egress points. Internally, you should use a service mesh to monitor and enforce expected East-West traffic so that services are only able to communicate with allowed services.
8. Secrets Management Failures
Kubernetes offers support for protecting and managing secrets used by your workloads through secrets objects, which get mounted separately to your pods, allowing more granular access controls. You should review how those are stored and retrieved. You may also want to have policies in place for secret rotation and run books ready for when secrets need to be revoked and rotated in response to an incident or breach.
9. Misconfigured Cluster Components
Moving deeper into the Kubernetes stack, you should also be monitoring the configuration of your cluster components themselves. This is particularly true for third party products, tools, and helpers you may be using and for the host machines you are using as your cluster nodes: have you hardened and secured these hosts/components? Part of confirming that these components are adequately configured and secured is performing a full threat model on your Kubernetes environments.
10. Outdated Vulnerable Kubernetes Components
Finally, your Kubernetes cluster is not a static entity: your workloads and cluster components are all constantly being updated. Not keeping up with these updates exposes you to possible security risks fixed by available patches.
KSOC, the Kubernetes Security Operations Center, helps customers quickly and easily identify and remediate Kubernetes misconfigurations, vulnerabilities, and permissions issues. Get in touch with the team to learn more.