Last updated at Mon, 22 Jan 2024 19:18:09 GMT
Recently at Rapid7 Labs, we’ve noticed an increase in activity on ports related to the management of a Kubernetes cluster. In this post, we provide background context to Kubernetes and how it relates to the issues we see, as well as offer some guidance for securing a Kubernetes cluster.
These days, more and more people are deploying their software using container services such as Docker. Containers make it easy for developers to replicate programming environments in development and production. The more you use containers, the greater the need will be to manage the deployment of these containers for all your software services. This is where Kubernetes comes in. Kubernetes is an open source system for helping you automate deployment of containers.
One of the key components of the Kubernetes ecosystem is a kubelet. A kubelet runs on every node of the Kubernetes cluster. It monitors and manages the containers created by Kubernetes’ YAML and JSON specifications. The kubelet also runs an HTTP server which exposes a REST API to watch and control the node.
By default, the kubelet will run the webserver on port 10250. However, security for the kubelet is not configured out of the box; it is the user’s responsibility to set up the authentication for the HTTP server.
From the Kubernetes documentation:
By default, requests to the kubelet’s HTTPS endpoint that are not rejected by other configured authentication methods are treated as anonymous requests, and given a username of system:anonymous and a group of system:unauthenticated.
Any request that is successfully authenticated (including an anonymous request) is then authorized. The default authorization mode is AlwaysAllow, which allows all requests.
This means an unconfigured Kubernetes cluster is vulnerable to attacks from the outside world by default. The API contains PUT, POST and DELETE requests for many resources, including a resource which can allow remote execution on containers! A senior security engineer from Handy discovered and demonstrated a proof of concept of this kind of attack earlier this year:
Recently at Rapid7 Labs, we’ve noticed surge in traffic on port 10250 from May 2018 onwards. It appears more and more bots are scanning for the HTTP traffic server. Analysis for other Kubernetes ports don’t show as big of a spike in traffic use, but show that people do periodically target these ports. As a result, Rapid7 Labs is adding Kubernetes ports 10250, 10255, and 6443 to its Sonar studies in order to understand the exposure of this vulnerability to the internet.
Graph i shows the honeypot access patterns starting from January. Here we see the spike in number of connections a day from unique IPv4s starting around May.
Graphs ii, iii, iv show access patterns on some other default Kubernetes ports. They’re not as active as 10250 but show that they’re still scanned for from time to time. It’s still important to secure these ports as well.
In conclusion it is best to check your Kubernetes cluster to see if the authorizations are configured correctly. To avoid an insecure cluster, set up your kublets for authentication either with bearer tokens or x509 certificates. Also, you can start the kubelet with the --anonymous-auth=false flag to prevent kubelet from accepting every request by default. Once you have enabled authentication, configure authorization as well. If you do decide to go without authentication, authorization will let you make sure that anonymous users can only access limited APIs. Also be sure to take a look at the issues in the Kubernetes GitHub repository for additional information.
Interested in how Rapid7 can help you secure your container environment? Learn more about Docker and container security from Rapid7.
