Shared Surface · Network
Networking
By default a Kubernetes cluster is one flat network. Every pod can reach every other pod, and the control plane itself becomes a bridge between tenants.
One flat LAN, by default
Out of the box, Kubernetes networking is completely open: every pod can talk to every other pod, across every namespace. Namespaces are an organizational boundary, not a network one. Unless you add NetworkPolicies (and run a CNI that enforces them), Tenant A’s workload can connect straight to Tenant B’s.
DNS
Another point to consider for multi-tenant clusters when thinking about network security, is that DNS is used for service discovery and by default all workloads in a cluster use the same DNS database to store that information.
As a result, any tenant who can make DNS requests in the cluster will be able to enumerate all the services running in the cluster. CoreDNS-enum is a good example of a brute-force tool which can enumerate every service running in a cluster.
SSRF by design
Another concern with relation to networking in multi-tenant clusters is that the Kubernetes API server provides a fully-fledged proxy, which is used for managing and troubleshooting applications deployed to the cluster.
As such the API server will generally be able to reach any workload at the network level, so if a malicious tenant can misuse that feature to connect to other tenant’s workloads, they may be able to bypass network policies.
A good example of this comes from a 2019 blog by kinvolk. In that blog they noted that a user who can edit a pod’s status, can change the IP address listed there, and then that modified IP address will be used by the API server for proxying. This effectively allows for network policy bypass in a multi-tenant cluster.
Additionally where nodes are shared, there are other SSRF vectors in Kubernetes which go via the Kubelet and could be used by attackers in this type of cluster.