Last updated at Wed, 07 Mar 2018 14:39:33 GMT

(Many thanks to Jon Hart and Tom Sellers for their research and content for this blog post.)

We started performing weekly monitoring of open/amplification-vulnerable memcached servers after the recent memcrashed amplification distributed denial-of-service (DDoS) attack and today we have some truly awesome news to report, along with some evidence that the recent spate of DDoS attacks may not be the last.

The Good News: Substantial Decline in Exposed memcached Instances

Project Sonar has recorded a drop from nearly 140,000 exposed unique TCP memcached endpoints through 2017 down to just under 58,000 in our March 1, 2018 scan, and an additional drop to just under 54,000 on March 6, 2018. While we don’t have Sonar-specific stats on UDP prior to March of 2018, even with just two days of monitoring we have seen a significant drop there, too: from almost 18,000 unique UDP memcached endpoints on March 1, 2018, to under 12,000 on March 5, 2018.

Apart from a >50% remediation rate after the 2017 Barix exposure, we’ve never really seen a drop this large in a publicly exposed service. Even the WannaCry disaster did not prompt a serious decrease in SMB exposure, though it did dip slightly before leveling out in 2017 (SMB exposure grew a bit later in the year).

Several things could have contributed to this massive drop in exposure, including:

  • CVE-2018-1000115 was assigned to this issue. Assigning a CVE improves visibility of the vulnerability and enables organizations and technology/security vendors to develop tools and processes to identify vulnerable systems and begin remediation procedures faster.
  • Some Linux distributions, including Red Hat, have now disabled the UDP listener and configured memcached to only listen on the loopback. Several other distributions will likely follow, or already have; Ubuntu, for example, has had the reconfigured the default package install to expose memcached only on the loopback interface since 2007 and as such is unaffected by this out of the box (they just recently disabled UDP, however, to further lock things down).

A >1Tb/s DDoS is nothing to sneeze at, particularly when it hits such a central and critical service such as Github.

What does memcached exposure look like now?

Even though the exposure has been significantly reduced, it’s not all good news.

While the memcached service should never be exposed directly to the internet without source IP restrictions, for either TCP or UDP, one would hope that folks stay current with versions even if they’re making exposure mistakes (yes, we are ever the eternal optimists). Sadly, this is absolutely not the case; and, unlike a fine wine, vintage 2012 and 2013 memcached services do not age well:

That mix of versions is riddled with remote execution, buffer overflow, service bypass and denial of service vulnerabilities. Even with the reduced number of exposed nodes, 10,000 recruits joining in an attack with a worst-case scenario amplification factor of 10,000 to 50,000 can still do a great deal of damage. The outlook is even less pleasant when you consider the above mentioned RCEs, which add potential for these nodes to be turned into command and control nodes.

If you are interested in any of the raw data for the above mentioned Sonar findings, see the below studies:

As mentioned in our last post about memcached, a Metasploit module for detecting the amplification issues has been in the works and was released on March 5, 2018. A module for detecting the version of memcached over UDP is in the works too.

Project Sonar will continue to monitor memcached exposure.

The Bad News: More DDoS in the Works?

There’s also a bit more disconcerting news to share.

Rapid7’s Heisenberg Cloud passive sensor network has detected a series of probes on other ports used for amplification attacks (we’re calling these “ampli-ports”). Below is a breakdown of the unusual activity for six of them: the DNS probes are likely related to another recent DDoS, and the most recent spike was on the Simple Service Discovery Protocol (SSDP).

Attackers may be taking inventory of available amplification DDoS targets for their bot armies in preparation for future attacks this year.

Rapid7 is monitoring these probes and will continue to report on any unusual (good or bad) activity related to this very curious sequential probing of DDoS services. As noted above, you can find our Project Sonar data at scans.io, monitor live memcached stats at Shadowserver, and continue to reach out to research@rapid7.com with questions about these studies, our Heisenberg Cloud findings, or other studies.