Container Registries

Whilst this isn’t directly part of the Kubernetes cluster, pretty much every cluster relies on one or more container registries, and they’ll often share a registry across tenants. An oblique example of this was the OpenAI Hugging Face incident, where a large set of containerized agents used a shared Artifactory registry as an ad-hoc message board.

This makes registries another shared surface where a malicious tenant might look to expand their access. Compromising a base image used across the cluster can easily lead to privilege escalation, and there can be multiple routes to this access, either directly via the container registry API or indirectly via something like a Continuous Integration system.

Shared build and registry infrastructure

A more direct example comes from Wiz’s research into Hugging Face Spaces. Spaces lets users run an application on Hugging Face’s infrastructure by supplying a Dockerfile. When Wiz looked at the network connections from their build, they found the built image was pushed to an internal registry that also served other customers, and due to insufficient scoping they could pull and push (so overwrite) every image in it.

That’s the multi-tenant registry problem in a nutshell. The registry was “internal”, so it was treated as trusted, but the tenants were running their own code right next to it.

Shared image caches

There’s also a node-level version of this problem. Once a private image has been pulled onto a node, it’s in that node’s image cache, and another tenant’s pod scheduled on the same node can potentially use it without having the registry credentials. The multi-tenancy benchmarks recommend an imagePullPolicy of Always for tenant workloads, so the registry gets asked to authorise every pull.

What can you do about it?

• Give each tenant their own registry, or at least their own repositories with separate credentials, and make sure “internal” registries still require authentication.
• Treat CI systems that can push to the registry as part of the tenant boundary, as they’re often the easiest way in.
• Use fully qualified image names, ideally pinned by digest rather than a mutable tag, and use admission control to restrict which registries tenants can pull from.