Understanding Ubiquiti Discovery Service Exposures

Last updated at Thu, 01 Aug 2019 14:21:28 GMT

Introduction

On Jan. 29, the Rapid7 Labs team was informed of an interesting tweet by Jim Troutman indicating that Ubiquiti devices were being exploited and used to conduct denial-of-service (DoS) attacks using a service on 10001/UDP. Quick sleuthing by the security community showed that this issue has been brewing since the summer of 2018. Ubiquiti recently acknowledged that this was an issue, has released a workaround, and is in the process of putting together an official fix.

Research has learned that this service is used for a variety of things, including device discovery to facilitate easily locating of Ubiquiti devices in a managed environment. At least this portion of the protocol is quite simple, requiring a simple 4-byte message that elicits a large response including the name, model, firmware version, IPs, MACs, and sometimes the ESSID if it is a wireless device of some manner. A simple POC of this functionality can be seen below when run against a mostly default Ubiquiti mFi device:

$  echo -ne "\x01\x00\x00\x00" | socat -t 1 udp:10.6.67.25:10001 - | hexdump -C
00000000  01 00 00 5b 02 00 0a f0  9f c2 4c 50 51 0a 06 43  |...[......LPQ..C|
00000010  19 01 00 06 f0 9f c2 4c  50 51 0a 00 04 00 7e da  |.......LPQ....~.|
00000020  da 0b 00 09 6d 46 69 34  63 35 30 35 31 0c 00 03  |....mFi4c5051...|
00000030  50 38 55 0d 00 00 0e 00  01 02 03 00 22 4d 46 2e  |P8U........."MF.|
00000040  61 72 39 33 33 78 2e 76  32 2e 30 2e 32 35 2e 31  |ar933x.v2.0.25.1|
00000050  32 33 38 2e 31 34 30 36  32 34 2e 31 33 35 36     |238.140624.1356|

Because this is UDP, this protocol unfortunately suffers from UDP amplification vulnerabilities, as warned previously by Rapid7 and US-CERT, among others. The amplification factor is 30-35x but does not appear to suffer from multi-packet responses, at least with what is known today. With such a large quantity of potentially vulnerable devices exposed, a DoS harnessing the available bandwidth and power of these systems could be used to conduct an attack in excess of 1Tbps, which is a crippling amount of traffic to all but the most fortified infrastructure.

It is unclear what other capabilities exist in this service, but it would not be surprising if there were other management capabilities baked in or nearby.

Exposure

In order to understand more about this issue and inform fellow defenders in the information security community, we performed a Sonar study of port 10001/UDP, where we collected and analyzed the responses by parsing out the distinct fields returned in UDP payload. This showed 498,624 unique IPv4s with port 10001/UDP open, 487,021 unique IPv4s confirmed to be speaking this discovery protocol, and 486,388 unique physical devices based on MAC address tuples found in the responses. That is a lot of devices.

Examining where these devices live shows that Brazil is home to more than half of these, with large chunks in the U.S., Spain, and elsewhere:

Similar analysis by registered owning organization shows that many of these are ISPs, many of which are probably wireless ISPs (WISPs).

By decoding the responses, we are able to learn about the nature of these devices and clues as to how or why they are exposed publicly. For example, by grouping by the model names returned by these responses, we see big clusters around all sorts of Ubiquiti models/devices:

In some cases, the discovery response includes the ESSID of the device if it is wireless-enabled. The patterns observed seem to reinforce the smaller ISP/WISP theory, as many of them seem to map back to defaults correlating popular ISPs around the world.

By inspecting the name of the device, which will generally default to something based on the model name or whatever is configured at initial install, we see even more troubling patterns:

It seems that attackers have already identified additional problems with these devices and have exploited over 17,000 of them, as evinced by the defaced hostnames.

We can also see from the product version table above that along with being exposed to attackers, most of the devices in each product family are running outdated versions. This may explain the exploitation we described previously given the dangerous combination of service exposure, unpatched vulnerabilities, and default credentials (some of the hacker-changed device names noted the presence of default credentials).

The port 10001 exposure also reveals the internal network IP addressing scheme of behind the target devices. 192.168.0.0/16 addresses are the clear favorite, though some owners have chosen to mix it up a bit:

Examining the global honeypots that we monitor as part of Project Heisenberg, we have been seeing traffic destined to this UDP port for over a year, the vast majority of which appears to be traffic similar in nature to the discovery mechanism we described above.

Recommendations

Given the known possibility for abuse of this protocol in DoS attacks and the evidence and reports of active exploitation, Rapid7 suggests that all affected entities audit their external exposure for these devices and restrict or control access to this service as appropriate, which could include firewall or ACL rules, or disabling the affected service using recommendations from Ubiquiti.

Rapid7 communicated these findings with US-CERT (VU#993645) and has reached out to CERT Brazil, as well as Ubiquiti.

The raw data from this study is available on Rapid7 Opendata, along with all of our UDP and other studies. Metasploit modules and Nexpose/InsightVM coverage are in the works for discovering Ubiquiti devices using this method.

Rapid7 will continue to monitor this situation and will update here/elsewhere as appropriate. We welcome any feedback on this topic and encourage collaboration. Reach out to us via the comments, on Twitter (@rapid7), or via research@rapid7.com.

Updates

We have several updates to share on this topic since our original post nearly two weeks ago:

Community response

There have been several updates from the Ubiquiti community and a Twitter response from Ubiquiti that suggest that these devices are exposed due more to poor security hygiene than anything else—failure to update, use secure passwords, or follow vendor recommendations with regards to security best practices. A ZDNet article has another report indicating that disabling this service is problematic and may have been the cause of some of this exposure, which may explain why Ubiquiti’s original response to the security implications of this issue seem to have gone silent. Shortly after our original publication earlier this month, Ubiquiti posted an article that describes this discovery functionality and how to disable it, if desired.

This research also received mentions in the 700th episode of GRC’s Security Now! podcast, SANS Daily Network Security Podcast (Stormcast) for Monday, Feb. 4, 2019, among others.

Exposure changes

In our original post on this two weeks ago, we reported 487,021 unique IPv4s advertising this service. A follow-up scan on Feb. 2, 2019, saw a small 1% drop, which could easily be natural internet fluctuation and oddities—however, our most recent survey in preparation for this update discovered a significant drop of 25%! Digging into both of these decreases in exposed systems, we immediately discovered this was caused entirely by updates to systems or infrastructure in Brazil. As you may recall from our original post on this, we identified Brazil as being responsible for more than half of the exposed Ubiquiti systems worldwide and noted that we had been in communication with the Brazil CERT in an effort to remedy this exposure. It is an encouraging sign that these numbers improved so dramatically.

Improved coverage

The Metasploit team landed a PR that added the
auxiliary/scanner/ubiquiti/ubiquiti_discover module, which can be used to locate or discover affected Ubiquiti devices using this service. Seen here running against a test lab, we were able to discover the name, MAC, model, and firmware version of the remote devices:

Customers of Rapid7’s Nexpose or InsightVM products can obtain fingerprint and other coverage for this by updating to product version 6.5.51 or later.

We use Nmap and zmap heavily in our research and wanted to ensure that support was trued up there as well. Tom Sellers has added a discovery probe to Nmap as well as an Nmap script that can be used to extract the name, model, firmware, and other metadata from discovered Ubiquiti devices. Additionally zmap landed a PR that adds similar coverage.

Discovery version 2

As part of adding coverage as mentioned above, Tom Sellers discovered a slightly newer and different version of this discovery protocol. This protocol, like the earlier version, operates on 10001/UDP but uses a different discovery probe: \x02\x08\x00\x00. From the responses, we can extract information such as model, firmware, UniFi SDN version, MAC address, internal IP address, and the management state (adopted vs. unmanaged).

This newer protocol appears to primarily be in use by Ubiquiti’s Cloud Key management devices. Ubiquiti’s UniFI SDN Controller software is used to manage UniFi family devices on a network. The software can run on Linux, macOS, and Windows. The Cloud Key product simplifies running the Controller software by providing a cost-effective appliance to run the software as opposed to provisioning a discrete computer or virtual machine.

Our study of this newer protocol detected 347 devices on the internet that responded correctly. Interestingly, six of the devices that responded were switches manufactured by Netonix. We saw their gear show up in our original surveys, which is reflected in the table above.

The tables below provide some insight into the population of devices that responded:

The oldest firmware that we observed was one instance of UCK.mtk7623.v0.3.4.9c350a9.151019.1212. The release date is included in the firmware name, so that version looks to have been released in October 2015.

** Field not supported by Netonix

** Field not supported by Netonix