The Insider - Case Study

 

An MSSP running on LimaCharlie.io recently had an incident which they shared with us, and in turn we are sharing it with you.

It is a great example of the importance of having endpoint data and how a good EDR can shorten the Incident Response process by orders of magnitude compared to unidirectional logging (like syslog or Windows Event Logs).

Infrastructure

Security products were deployed on the customer's network: an Anti-Virus, a SIEM and LimaCharlie. LimaCharlie was configured with all events going to an AWS S3 bucket for long term retention and to a Splunk host via SFTP for daily operations.

The customer's network consisted of an Intranet which included workstations, a DNS server and a Wordpress server. The Intranet / Internet boundary had a firewall and was NATing internal IPs.

Workstations were Windows (7+), MacOS and Linux Debian, all (including the Wordpress server) running LC.

An external server maintained by a 3rd party could by accessed by SFTP and hosted assets were used by the internal Wordpress server.

All workstations on the Intranet had a page from the Wordpress server as their default web page (it was an internal company portal).

normal.png

Initial Event

This MSSP initially began an investigation because their customer received complaints from workers that the performance of their workstations changes abruptly and started running slowly. No new deployments had occured and no alerts were raised by the Anti-Virus product or the SIEM.

Only hosts on the Intranet seemed affected, the fleet of laptops roaming in other networks (coffee shops etc) had no performance issues.

Investigation

MSSP analysts began looking at events from a few of the affected machines before and after the performance issue was reported:

Splunk Query: spath "routing.event_type" | search "routing.event_type"=DNS_REQUEST

Results:

event:    {    [-]    
         DNS_TYPE:     1    
         DOMAIN_NAME:     <hidden>    
         IP_ADDRESS:     <hidden>        
         MESSAGE_ID:     4488    
         PROCESS_ID:     0    
      }
routing:    {    [-]    
         arch:     2    
         event_id:     8f66b3e3-30ea-4d41-a950-32ad27f31afe    
         event_time:     1537776210013    
         event_type:     DNS_REQUEST    
         ext_ip:     <hidden>    
         hostname:     WORKSTATION-6
         iid:     88dd2804-6c20-4adb-990c-10474ffb3e02    
         int_ip:     <hidden>    
         moduleid:     2    
         oid:     <hidden>    
         parent:     3fd02810c64aaae2dba8db675a53185a    
         plat:     268435456    
         sid:     <hidden>    
         tags:    [    [+]    
        ]    
         this:     392d199861b8ddb0a9ffe892866fa821    
      }    
}

What they found was that the affected hosts began a pattern of DNS requests to the external SFTP server as well as several domains triggering an alert in LimaCharlie on the coinblocker feed.

The DNS activity was followed by connections from the browser.

Looking in detail at events around the time of the connection using LimaCharlie's visualization tool, Digger, they found the browser was creating a .js file in the browser cache.

The following Splunk command confirmed that all affected workstations also had the same .js.

Splunk Query: *| spath "routing.event_type" | search "routing.event_type"=NEW_DOCUMENT AND malicious.js.

Thankfully, the files were cached in memory by LimaCharlie. Using the doc_cache_get command in LimaCharlie, the analysts were able to retrieve a cached copy of the javascript file in memory.

As expected (because of the concurrent hits from coinblocker), the javascript turned out to be a Crypto-currency miner.

Interestingly, the LimaCharlie activity did not show any other connectivity to the external SFTP server prior to the fetching of the malicious .js file.

What caused the .js to be fetched? Why was the entire internal network affected at once?

Further analysis of the events showed that the malicious .js was always fetched right after a connection to the internal Wordpress server. This clearly indicated that the Wordpress server may have served as a type of Watering-hole attack.

Suspecting parts of the Wordpress site may have been modified, the MSSP analysts begin looking for the malicious .js file name in the PHP code by retrieving the files using LimaCharlie's file_get commmand. They quickly found that a <script> tag pointing to a file hosted on the external SFTP server has been added to the header of the main portal page. The remote URL event even containsed a malicious.js?id=<?php echo time(); ?> URL parameter, seemingly to make the URL accessed appear more random and legitimate-looking.

This completed the picture of what happenned. The attacker hosted a malicious Cryoto-curency miner on the external SFTP site, then gained access to the Intranet Wordpress server and injected the remote inclusion of the malicious javascript, which looks like this:

altered.png

Who?

The next logical question any Incident Responder has is "how did they get in?".

This is where an interesting twist gets introduced into the plot. The Wordpress server has LimaCharlie installed which means a wide variety of events was collected from around the time the malicious injection occured. Initially expecting to see indications of a Wordpress exploit, the MSSP analysts instead saw a simple command vim ./header.php. The originator of this process was an authenticated user via ssh.

There was no indication of exploitation or leaked credentials of any kind. Since LimaCharlie was installed on the workstations, the access could be corroborated within seconds from the LimaCharlie logs of that user's workstation at the time of infection.

As it turns out an employee had decided to make some money on the side from the company's assets.

From there, the customer's internal security and HR department take the lead...

Lessons

The LimaCharlie EDR helped get a successful outcome for this incident at a few critical points in the investigation.

Manual inspection of the impacted hosts could certainly have been possible without LimaCharlie. But inspecting manually a few hosts would have required administrators/security personel to physically go on location, and that is expensive and incredibly inefficient.

Retrieving the malicious javascript file would have been a potentially complex endeavor. Determining which file is the correct one without seeing the historical timing of the DNS anc connection events would have been difficult. Even if the file was identified, its retrieval is usually complicated by the constant turnover of files on disc. The LimaCharlie file caching dramatically streamlined this process by allowing the analysts to get a copy of the file after deletion, and remotely.

Finally being able to correlate the activity log of multiple hosts, their processes, command lines and network connections to paint a full picture is extremely valuable in interpreting data in order to form a complete understanding of the attack.

Of course, from our point of view, it doesn't make sense to forego all these capabilities when an affordable package like LimaCharlie exists and it literally takes minutes to onboard.

crypto.png