Slamming SQL Servers for Fun and [Crypto]Profit

Some opportunistic attackers appear to have taken a few sick days while we settled into our new pandemic routines: We recorded an order-of-magnitude drop in what had been a daily barrage of SQL Server brute-force attacks that had been going on in our Project Lorelei honeypot fleet since October 2019:

Guardicore reported their observations of the “Vollgar Campaign” earlier this year, and 360 Total Security provided updated insights on KingMiner (which Check Point researchers first noted back in 2018). 

You’re taking a huge risk putting any database query interface directly on the internet and doubly so if they can be accessed with simple credentials, but—as noted in the Rapid7 2020 NICER Report—there are over 98,000 SQL Server instances accessible via the internet (a small fraction of which are honeypots like ours, but the vast majority are instances folks accidentally —or deliberately?—placed in harm’s way). 

What impressed the Rapid7 threat researchers is just how efficient this campaign was:

An average of 7,500 unique source systems were capable of generating over 100 million unique credential sessions, changing multiple parameters associated with SQL Server login sessions. Imagine if all of your DevOps-fueled processes were that streamlined.

These attacks installed cryptominers, but many also attempted to gain access to other, internal systems via EternalBlue and other exploits. While our telemetry is purely external, it’s possible these same techniques could be used to exploit internal SQL Server instances as well. If you’re not looking for Windows event ID 18456 (with a source of MSSQLSERVER) now might be a good time to set that up and then do a quick check of the processes running on your SQL Server fleet.

A Quarter of Cowrie

For those not steeped in honeypot lore, a popular honeypot service to run is Cowrie, which is “a medium to high interaction SSH and Telnet honeypot designed to log brute-force attacks and the shell interaction performed by the attacker. In medium interaction mode (shell) it emulates a UNIX system in Python, in high interaction mode (proxy) it functions as an SSH and telnet proxy to observe attacker behavior to another system.” We have a number of custom honeypots in the Heisenberg fleet, but Cowrie is excellent at what it does and we make great use of it in the Labs team.

Now, everyone and their cousin runs SSH honeypots, so it generally takes something unique for us to talk about the activity we see in them. This quarter, our anomaly detection processes indicated an “outside the norm” uptick in unique sessions per day:

So, we dug in to see what was up. Our analysis flagged nearly 8,000 source botnet IPs attempting to grab Mirai-like malware from remote IPs—5.206.227.228 and 5.206.227.18—in the Dotsi hosting provider network.

These sources come from all across the internet and are composed of devices ranging from web servers to routers, cameras, DVRs, and other “Internet of Things (IoT)” items:

These are rather “noisy” campaigns that are attacking any SSH endpoint they can find on the internet, despite the fact that it is highly unlikely physical home routers (and other types of IoT devices) will be sitting in enterprise or cloud networks, but they continue to try (and succeed) since there are nearly 18 million SSH servers on the internet. 

Rapid7 highly recommends decreasing the reputation of pretty much anything coming from Dotsi’s network (AS49349) and the list of above autonomous systems. We also strongly suggest not exposing SSH to the internet unless absolutely necessary and configured extremely well (see Mozilla SSH Guidelines).

As we did in Q1 2020, we continued to expand on our ability to gather and share more context on the threats that Rapid7’s Managed Detection and Response (MDR) team reports to our customers. In Q2 we added the ability to capture multiple MITRE ATT&CK tactics and techniques per report, we deepened our integration with the data from the Cyber Threat Alliance, and we refined the assignment of category and impact. We feel that these changes, especially supporting multiple tactics and techniques, has given us better visibility into the different steps that transpire during breaches.

As seen in Figure 5, we also revisited our industry breakdown graphic to further analyze the count of incidents. This new graphic shows the number of threat reports, which can be interpreted as incidents, per customer in that industry and gives us an indication of customer size (blue represents small organizations, and slate represents large).

The very first observation here is the number of threats that small companies in the manufacturing industry have endured. Over time we’ll be able to tell if this amount is systematic, or just a temporary trend. It is also worth noting that smaller companies in both manufacturing and finance have a significantly higher number of threat reports as compared to peers in other industries, as well as a significantly higher number of reports than the larger organizations within their respective industries.

As in previous quarters, manufacturing, finance, retail, and healthcare were the most targeted industries, with manufacturing taking a big leap between Q1 and Q2. Much of this could be due to five individual organizations with 15 or more threat reports. Nonetheless, if you are in the manufacturing industry, especially if you are a smaller business, attackers are targeting you.

Lastly, from finance to retail and on down to the end of the list, we see a shift from small to large businesses being targeted. We also see that the number of incidents per organization is smaller, compared to the top three.

While we are just reading high-level data here, we hypothesize that the significant number of organizations with 15 incidents or more in the manufacturing industry is likely related to the heightened targeting of that industry, whereas only seeing one or two businesses with 10 or more breaches in the other industries is likely related to the lower maturity levels of their security programs.

As mentioned at the beginning of this section, we’ve enhanced our ability to derive threat classification(s) for the incidents we report on. We believe this helps us better understand the nature of the threat at a high level.

As we discussed in the 2020 Q1 Threat Report, attackers have been targeting credentials as a growing trend. Q2 is no different, with 35% of the incidents reported to MDR customers involving the compromise of user credentials. This makes sense when you consider that many organizations have shifted to using Software-as-a-Service (SaaS) providers where traditional IT infrastructure might have been used in the past. With organizations using cloud-based email and document management to communicate sensitive business information (in addition to storing PII- and PHI-related information on employees), CRM tools to store sensitive customer information, and SSO solutions to connect them all, it is no surprise that attackers target credentials.

Not to be outdone, malware, trojans, and all other endpoint-based compromises continue to disrupt the productivity of employees and business operations.

Looking at the threat classification count per report in Figure 6, we can see that the overwhelming number of incidents only involve one threat type (often due to catching the threat early in the attack lifecycle). However, in about one-third of cases, two threat types were involved (such as phishing leading to credential compromise), and a few incidents included three types or more. 

The understanding that attacks and breaches don’t happen in a vacuum is exceedingly important. Attackers and common threats alike perform a number of different operations during a breach. Each one of those operations is an opportunity for the detection and response teams to respond and contain the threat.

Let’s dive a bit deeper on this concept by looking at the number of alerts involved in each incident in Figure 7.

While in many incidents we see one alert, most have two or more alerts that indicate the threat. The concept of alert overload is a very real concern; threat detection teams don’t need less alerts, they need more meaningful grouping and prioritization of alerts.

Each incident does have a tip of the iceberg: the alert that kicks off the entire investigation. Here are the top 10 alert types the Rapid7 MDR SOC team reported in Q2 2020.

We continue to see WScript as a favorite way for attackers to execute code on victim machines. Typically these represent malware or a lure delivered via spear phishing as an attachment.

The main reason we present this information is to inspire detection and response teams to augment their threat indicator-based detection with behavior-based detections. These alerts all look for suspicious behaviors on endpoints including parent/child process relationships, abnormal use of built-in command line tools, and unusual processes initiating network traffic.

Next we’ll bring back the MITRE ATT&CK tactics and techniques observed during the incidents. Please note that we have excluded the ‘Valid Account’ technique under the ‘Initial Access’ tactic (there were 141 instances of this technique/tactic).

With our increased ability to assign multiple tactics and techniques to incidents reported to Rapid7’s MDR customers, we gain greater visibility into how far successful attacks are progressing. As you can see by the color coding, detecting the threats in the initial phases of the attack lifecycle not only reduces the impact of the breach, but also reduces the cost in hours for detection and response teams. This is extremely important considering most, if not all, security programs have a backlog of work that needs to be done.

Further, if you are in the process of building your detection and response capabilities to align with the MITRE ATT&CK framework, prioritizing the most-used techniques that are also early in the attack lifecycle will net greater reduction in financial and reputational impact when a breach does occur.

Lastly, we draw your attention to the overwhelming number of incidents that involve masquerading, aka ATT&CK Technique T1036. If you are not familiar with this family of techniques, these activities enable attackers to subvert or evade most controls and threat prevention tools. As such, it is imperative that your threat mitigation strategy include tools, people, and processes to prevent known threats, detect unknown threats, and speed mitigation and remediation.

The following graph offers a different visualization of the number of times the masquerading technique was used by attackers.

Dwell time is another important component of the detection and response strategy. The longer attackers are present in the enterprise infrastructure, the more likely they are to accomplish their mission objectives.

If you have been keeping up with our quarterly Threat Reports, you will immediately note the shift to the right as it relates to how quickly the Rapid7 MDR SOC team identifies and reports threats. There are a few reasons for this. First, as you’ll note in Figure 12, the 2–3 month category is heavily influenced by the finance, healthcare, and wholesale industries. Further, between our enhanced capabilities to extract more data from the incidents that comprise this report and additional capabilities created by the MDR team in terms of threat hunts and network sensor visibility, we are detecting indicators of compromise that we weren’t able to detect before.

Just as we say that all threats cannot be prevented and security programs must compliment prevention strategies with people and technology to enable threat detection, we also advise that increased maturity in threat detection comes with different methodologies to detect threats.

Threat hunting is a common technique used by Rapid7’s most seasoned threat analysts where they apply statistical and differential views to data types that are not commonly used by alerting engines. For example, on a monthly basis, MDR analysts review Windows Registry Run Keys for suspicious entries. This would be a grueling process if they weren’t able to eliminate entries that have not changed in the last 30 days. They’ll review Windows Scheduled Tasks with the same intent and process. Using these hunting techniques, they are able to uncover breaches even when the attacker is stealthy enough to evade detection through traditional alerting.

Threat hunting programs are expensive. They require specialized technology and specialized knowledge and experience to create and carry out. As such, they’re not for every security program. However, if you find yourself having implemented advanced alerting techniques such as process/endpoint behavior analytics, User Behavior Analytics, and Attacker Behavior Analytics, threat hunting is the next evolution. 

Before we move on to our Focus on Response, let’s take a look at the most common processes identified by Rapid7’s MDR in reporting incidents to our customers.

Understanding which programs are used by attackers during breaches gives security programs the opportunity to remove the opportunity by limiting which programs can run on the endpoint. From elaborate block/allow list technology to Windows Group Policy, many solutions exist, so if the overwhelming majority of the users in your organization would have no need for these tools, they shouldn’t be allowed to run.

In a new feature this quarter, we’ve asked Robert Knapp from Rapid7’s Incident Response fly away team and Bryce Abdo from Rapid7’s Threat Intelligence & Detections Engineering team to shed light on some memorable breaches from this quarter. These are their stories. 

Managed Detection & Response: Organic Intrusion Data

One of the best forms of threat intelligence is that which is derived from raw intrusion data. The Rapid7 Managed Detection & Response (MDR) team protects our clients around the clock against threat actor intrusions, by continuously monitoring hundreds of environments for suspicious or malicious activity. When a high-severity intrusion is identified, MDR will spin up a Remote Incident Response (RIR) to work with the client to provide deeper forensics, thwart the attacker(s), and remediate the intrusion.

One of the core duties of the Rapid7 Threat Intelligence & Detection Engineering (TIDE) team is to support MDR Incident Response engagements with tactical and operational intelligence from our vast set of agent telemetry data and intrusion reports. The TIDE team ingests indicators, provides deep-dive analysis, reverse engineers malware, and performs pivots necessary to ascertain adversary methods and tradecraft. This IR lifecycle results in new detections engineered and deployed for all Rapid7 detection and response customers.

From the Frontlines: Repeat Targeting

During Q2 of 2020, Rapid7 MDR identified and responded to numerous intrusions detected across our client base. In two of these cases, we identified repeated targeting of a client’s environment.

During these two cases, MDR identified suspicious PowerShell execution. The execution of PowerShell included commands to export email database (.PST) files on an email server asset. As analysis of the first intrusion progressed, Rapid7 MDR and IR analysts identified vast historical evidence of exported and exfiltrated email inbox files corresponding to 20+ targeted, executive leadership user accounts on the mail server.

Additional analysis revealed that the initial infection vector was variants of Chopper web shells that had been implanted on a server. This tried-and-true approach to gaining initial footholds into networks is still used quite often today. The attackers used the webshells to execute commands and enumerate the network using built-in native Windows commands. Through the use of the webshells, the attackers also uploaded and executed publicly available tools such as: Dumpert, ProcDump, ProcDump64, WinRAR, AdFind to perform various tasks such as additional enumeration and credential gathering. Further forensic analysis of multiple systems also revealed infections of PlugX backdoor variants, a trojan that is used by many different threat actors.

Although some of the tools used by the adversaries in both of these intrusions were variations of open-source tooling, the tradecraft and methodology employed by the adversary overlapped in multiple ways. It is often difficult to ascertain the end goal of the adversary in most intrusions; however, the persistent targeting of executive email correspondence indicates that the goal of the adversary was espionage related. These indicator and methodology overlaps indicate that the organization had been and is repeatedly a target for an Advanced Persistent Threat (APT).

Rapid7 recommends the following action items to help mitigate intrusions like this from occurring:

• Review and audit privileged service and domain admin account activity.
• Disable interactive and local login rights from service accounts.
• Disable credential caching on servers.
• Use unique passwords for all local admin accounts and use a password complexity policy.
• Implement file integrity monitoring for internet-facing web servers.
• Implement Multi Factor Authentication (MFA).
• Segment the network based on data sensitivity and type.
• Implement application whitelisting for critical systems.

From the Frontlines: Ransomware Operators

During the early stages of intrusions, TIDE performs adversary infrastructure analysis and pivoting of the observed indicators. The indicators assessed often lead to the discovery of additional attacker infrastructure with overlapping tradecraft in some cases. These pivots help analysts by providing context as to what type of adversary they are dealing with during an intrusion. 

Targeted, “big-game-hunting” ransomware operators are quite active this year. Their goal is to mass encrypt client environments for ransom of cryptocurrency, and in some cases, to additionally pilfer through and steal proprietary data. The TTP’s employed by these adversaries vary widely across clusters of activity; however, Rapid7 focuses on a methodology-based approach to catching these types of attacks on our client networks. From a detection engineering perspective, it is important for the TIDE team to strike a balance between indicators built for specific tools, and indicators developed to alert on the underlying methodology used by the adversaries. The following intrusion use case highlights this activity as of late.

In May of Q2, Rapid7 MDR observed suspicious PowerShell commands in real-time process data from a client asset. MDR Analysts identified similar commands executing on a total of 25 different assets. Our team had determined that the purpose of the PowerShell invocation was to deploy Cobalt Strike, a very common post-exploitation framework that is used by both penetration testers and threat actors alike.

Analysis of the activity before the intrusion on patient-zero revealed that the intrusion was the result of a phishing attack. A primary tactic of targeted ransomware operators is to leverage initial infections of a commodity Trojan to further compromise an environment with a post-exploitation framework. The initial infection vector of the Valak trojan, coupled with the usage of Cobalt Strike framework, were consistent with this tradecraft.

The TIDE team collected available network indicators and performed infrastructure pivoting which revealed a vast collection of live attacker infrastructure. TIDE used sources such as Passive DNS records and network scan data to identify and confirm the additional infrastructure. The Cobalt Strike infrastructure primarily relied on two distinct factors: an infrastructure using dynamic DNS domains, and an additional infrastructure which used fallback server techniques and multiple C2 channels. 

The domains used by the adversary overlapped across multiple data points: domain registration profile (Namecheap), TLS metadata (LetsEncrypt), and a hosting provider known for its lax egress security controls. However, the strongest overlap of these indicators were the Cobalt Strike “Malleable C2” profiles the servers were using, which were custom to the attacker and not a publicly or commonly available C2 profile. These profiles across this cluster of servers indicated that the adversary was skilled at infrastructure setup and operational security,

Although ransomware and extortion were prevented in this case due to our MDR teams’ intervention, TIDE assessed with high confidence that the tradecraft employed by the adversary was indicative of an attempted ransomware deployment by an uncategorized threat actor.

Rapid7 recommends the following action items to help mitigate intrusions like this from occurring:

• User Awareness Training for phishing campaigns.
• Implement centralized logging for all applications.
• Implement Two-Factor Authentication for critical systems.
• Prevent Execution of Office Macros via Group Policy.
• Integrate an Anti-Phishing Service or Email security gateway.
• Separate privileged domain access from a standard user account.
• Block outgoing connections for ‘regsvr32’.
• Remove Interactive and Local Login Rights from Service Accounts.

Incident Response Services

When they’re not conducting proactive service engagements, such as Tabletop and Purple Team exercises, the Rapid7 Incident Response Services (IR) team is busy responding to major incidents for our MDR, retainer, and off-the-street customers.

From the Frontlines: Ransomware Disruption

During Q2 of 2020, Rapid7’s Incident Response Services team was contacted by an organization that believed they were observing signs of malware worming its way through their internal network. Upon initial investigation, Rapid7’s Incident Response Services (IR) and Threat Intelligence & Detection Engineering (TIDE) teams identified they were, in fact, in the midst of an active compromise by a threat actor known as EvilCorp. Multiple TTPs aligned with the preparation for deployment of a ransomware variant known as WastedLocker, leaving little time to waste.

The initial infection vector was consistent with other public reports of popular websites serving fake Chrome updates as users browsed them. Native Windows binary Wscript.exe was used to execute a .js payload within a compressed .zip file that had been downloaded to the workstation of what was determined to be the initial point of entry. The attackers had thirty (30) days of access to the target’s network, a length of access atypical for most ransomware operators. Over the course of the incident, the attackers utilized a combination of eight (8) open-source offensive security tools, the popular penetration testing framework Cobalt Strike, and multiple native Windows binaries to conduct various phases in their operation. 

The open-source offensive security tools served multiple purposes for the attackers. Rapid7 observed multiple approaches at credential harvesting. Invoke-MassMimikatz was used in an attempt to acquire locally stored credentials from multiple endpoints, while KeeThief was used in an attempt to acquire credentials stored within the popular password management solution KeePass. The Invoke-TheHash suite also was used to enumerate targets and execute binaries on remote hosts via WMI and SMB. Finally, the Get-TimedScreenshot module of open-source offensive security tool PowerSploit was used to capture screenshots in 30 second intervals on a system of particular interest.

Cobalt Strike payloads were observed being executed throughout the environment. Across multiple hosts, the attackers passed .CSPROJ files containing the XML of a custom C# project to msbuild.exe, a native Windows binary, that was used to compile and execute the inline C# on the fly. The attackers utilized additional native Windows binaries through the course of their operation, including qwinsta.exe, nltest.exe, net.exe, wmic.exe, and rundll32.exe.

In total, thirteen (13) sets of user credentials were leveraged by the attackers to escalate privilege, move laterally throughout the environment, and attempt to execute on their objectives. The types of accounts varied between attacker-created local accounts, numerous privileged domain accounts, and a local Administrator account.

A combination of defense in depth, the deployment of EDR tools, and a customer Operations team that was quick to implement containment, eradication, and recovery recommendations allowed Rapid7 to help disrupt the attackers before the WastedLocker payloads could be deployed within the environment.

Rapid7 recommends the following action items to help mitigate intrusions like this from occurring:

• Review and audit privileged service and domain admin account activity.
• Generate, aggregate, and review PowerShell Script Block logs on a regular basis
• Disable credential caching on servers.
• Use unique passwords for all local admin accounts and use a password complexity policy.
• Segment the network based on data sensitivity and type.
• Monitor the usage of LOLbins (Living Off the Land Binaries) commonly abused by attackers (such as powershell.exe, cmd.exe, at.exe, etc)

Rapid7 (Nasdaq: RPD) is advancing security with visibility, analytics, and automation delivered through our Insight cloud. Our solutions simplify the complex, allowing security teams to work more effectively with IT and development to reduce vulnerabilities, monitor for malicious behavior, investigate and shut down attacks, and automate routine tasks. Customers around the globe rely on Rapid7 technology, services, and research to improve security outcomes and securely advance their organizations. For more information, or to get involved in our threat research, visit www.rapid7.com.