We've seen time and again that dividing your security attention between the inside and the outside threat (and unfortunately the blend of both - when outsider leverages or becomes an insider) is an ongoing challenge. If you check out our last 1-2 years of Federal IT Security Surveys, you'll see the insider is still a pretty big concern that's far less understood and harder to solve (more on that -Internal Federal Cybersecurity Threats Nearly as Prevalent as External, SolarWinds Survey Reveals), spreading from training to actual technical controls to the challenges of monitoring. In the interest of giving you a bit of a head start, here's some insight into some ways you can monitor for malicious insiders with Log & Event Manager (LEM).
(Note: Anywhere you see a screenshot below, be sure to click to see a full version - they might look fuzzy otherwise.)
Endpoint Monitoring with File Integrity Monitoring (FIM) and USB-Defender
Out of the box, LEM includes both built-in File Integrity Monitoring (FIM) - which can audit for file and registry access/changes - and USB-Defender - which monitors USB device access. On systems where you may have potential exposure - think kiosks, systems with access to confidential data, servers, and shared workstations - deploying FIM and USB-Defender will allow you to:
- Monitor for unexpected copying of files and data to USB devices that can indicate data is being exfiltrated
- Attempts to bypass application installation and access policies by running applications directly from USB devices that can put systems at risk
- Changes to system settings and files that can indicate potential unexpected modifications, either due to malware, policy bypassing, or intentional abuse
Out of the box, you'll want to look at the following LEM content:
- Default FIM Monitors - the Windows Server template can also be applied to workstations as a place to start
- Filters of interest:
- Endpoint Monitoring > USB-Defender
- Change Management > USB File Auditing, All File Audit Activity
- Rules of interest can be found in the categories:
- Activity Types > USB Device Monitoring, File Auditing
System and Endpoint Monitoring for Authentication and Change Events
Beyond tracking files and USB Devices, on servers and workstations alike authentication and changes can offer unique insights into what's happening on the network, and provide critical clues when it comes time to investigate. Windows does not audit the mechanism a user used to log on, or changes made to local system accounts, at a domain controller, so without insight into the actual workstations and member servers directly you'll be missing pieces of the puzzle. Deploy agents to all your critical member servers and that same pool of workstations you need insight into and get to tracking the local Event Logs. With this data, you can see:
- Users logging on unexpectedly - unused accounts suddenly being used, service accounts being used to access the wrong systems, admin accounts being used incorrectly
- Remote access - usage of remote desktop vs. interactive logins, access from VPN accounts/addresses, contractors authenticating to unexpected systems
- Additional users & privileges - users being added to local or domain admins, local users being created
Out of the box, you'll want to look for the following LEM content:
- Filters of interest in these categories:
- Change Management
- Authentication
- Endpoint Monitoring
- Rules of interest in the following categories:
- Change Management
- Authentication
- Activity Types > Inappropriate Usage
Network Device Traffic Monitoring
If we move off of the systems themselves, we should also be able to detect behavior patterns that look abnormal using network traffic events, too. Sometimes putting agents on all workstations is infeasible, not to mention accounting for transient or new devices, and BYOD if you've got that in the mix as well. Log activity from all the devices you can that can monitor traffic patterns and connectivity - IDS/IPS, firewalls, wireless APs/WLAN controllers, routers, switches, VPNs, etc. With network traffic data, we can look for:
- If you've got a proxy or similar policy in place, users attempting to bypass proxy policies with direct communication on port 80 (i.e. network traffic that's not outbound from your proxy server)
- Network traffic to/from unexpected hosts or ports - your servers/workstations will generally communicate to a smaller subset of known hosts, traffic outside of this pattern would be unexpected
- Excessive network traffic - sometimes traffic patterns can become clear without utilizing netflow or deep packet inspection based on sheer event numbers, types, or behavior patterns alone
Out of the box, you'll want to look for the following LEM content:
- Filters of interest:
- Start from the out of the box filters in IT Operations and Security and build from them, especially the traffic filters
- Rules of interest in the following categories:
- Activity Types > Network
- Devices > Firewalls
Check out our thwackCamp session on using firewall log data, too - thwackCamp 2015 - Digging for Security Gold: Using Firewall Logs to Find Security Issues.
Traditional Malware and Security Event Detection
You can definitely put your existing investments in pure security technology to work for you here, too. The name of the game is defense in depth, and while traditional malware detection, IDS and IPS, and other tools might not be enough alone, each one of them can play an important part in helping detect potential abuse or piecing together fingerprints during an investigation. Infected endpoints are a gateway to the interior of the network and not all of us are victims of zero-days but rather some kind of combination of existing malware and other techniques that gives us a good chance of detecting it somewhere along the way. With these feeds, you'll see things like:
- Antivirus/anti-malware technology cleaning or having trouble cleaning potential infections
- IDS and IPS systems detecting potentially unwanted payloads or symptoms of infections or even exfiltration
- Triggers from any other security systems you've got to put to work for you that generate event streams - wireless security, data leak prevention, etc
- System errors and crash reports - potential malware causing leaks to affect the system in unexpected ways
Out of the box, you'll want to look for the following LEM content:
- Filters of interest include:
- Security > Virus Attacks, IDS
- IT Operations > Windows Error Events
- Rules of interest in the following categories:
- Security > Malware
- Devices > IDS and IPS (and related device types for your systems)
Threat Intelligence and Dynamic Feeds to Detect Malicious Traffic
Thinking forward, if you've seen our LEM What We're Working on page, you'll note we're talking a little bit about Threat Intelligence Feeds. We're working on adding the capability for LEM to dynamically download a list of known bad actors - potentially infected hosts, botnets, command and control networks, spammers, and general IPs up to no good - and automatically use that to detect communication on your network. This will be a really good way to see:
- When someone internal is communicating with a potentially malicious host, which can indicate they've already been infected
- When you're being probed, attacked, or otherwise communicated with externally by a potentially malicious host, which can indicate an incoming attempt
- Communication to/from spam, denial of service, or similar hosts that can indicate phishing attempts, zombies on your network, or other security issues
Watch for more on that here - when we've got more to discuss we'll update this post with how to use it to detect malicious insiders more specifically.
Manually, you can create and import lists of potentially unwanted IPs and ports and compare those to traffic as well. If you've got a list of known good ports that should be used to communicate on your network (especially inside>outside), or known applications if you're using Next-Gen firewalls, or known IP addresses when we're talking servers and controlled communication, build User-Defined Groups and rules/filters that compare to them.
What About Other SolarWinds Products? How Can They Help, Too?
Sure! Here are some ideas on using other products to help you detect potential malicious behavior internally:
- Network Performance Monitor: monitor for unexpected firewall/network performance issues and high bandwidth utilization that can indicate an outbreak or single host is infected
- Netflow Traffic Analyzer: building on the above unknown traffic patterns, look for possible unexpected hosts, ports, or communication patterns that might give you an idea something is wrong
- User Device Tracker: useful when tracking and potentially detecting issues at endpoints - the "who" to go with the "where"
- Server & Application Monitor and even Virtualization Manager: look for systems & applications performing unexpectedly or becoming unstable, these can be early warnings for security issues, too
- Database Performance Analyzer: building on that, look for batch transactions, long-running queries, and sudden performance issues, identify their sources
- Network Configuration Manager and Firewall Security Manager: as always, cover your bases with configuration first!
- Patch Manager: track systems out of compliance with patching policies, out of date systems are MUCH more likely to be victims of malware and other security issues
Feel free to let us know if you've got any content you're interested in seeing around detecting malicious insiders, any ideas or successful stories yourselves, or any other questions we can help with in the comments!