Is your MSSP providing true MDR capabilities?
When IT industry analyst Gartner defined Managed Detection and Response services (MDR) in 2016, it represented a paradigm shift around how most legacy Managed Security Service Providers (MSSPs) delivered services. Before 2016, most MSSPs focused heavily on Security Information Event Management (SIEM) as-a-Service (aaS) to provide continuous monitoring, review device alerts, and potentially provide Incident Response (IR) services in the form of forensics support. These services were largely reactive and the majority of response burden often fell on the client.
Clients began to adopt Endpoint Detection and Response (EDR) and Security Orchestration Automation Response (SOAR) tools, as well as other technologies that enable the proactive detection, threat hunting, and response. As they largely struggled with managing these tools, traditional MSSPs seized the opportunity to bundle support for them into their services.
This move made them look like MDR providers, but does technology support actually equate to delivering the outcomes of an integrated managed service? Or is it simply an attempt to try to mimic MDR capabilities? The issue continues to characterize the debate around what true MDR is and how it is best delivered.
This is the third paper in a four part series, "What's Missing from Your MDR Solution" that will help readers:
- Review the role of MITRE ATT&CK in MDR service offerings
- Identify traits to look for in four key teams focused on threat intelligence
- Have questions to ask vendors to assess the capabilities and activities of each team
The Role of the MITRE ATT&CK
The collection of tactics, techniques and procedures (TTPs) of threat actors, also known as Threat Intelligence (TI), is vital to helping MSSPs understand how to detect the latest types of attacks and potential ways to respond to them. The broader and richer the sources of TI, the better the defenses an MSSP can stand up for a wider range of clients. Just about every MSSP will claim they leverage TI in delivering their services, invariably as procured feeds.
Guidance:
An MSSP should articulate the vital role that TI plays and how it is widely applied throughout their services. If this is not the case, it should be a primary concern for any buyer.
One source that some MSSPs reference is the MITRE Advanced Tactics, Techniques, and Common Knowledge (MITRE ATT&CK) framework. Starting in 2013, the MITRE Foundation took the approach of cataloging observed methods threat actors used for attack purposes. Today, MITRE organizes the data into 14 categories that describe the short-term intention or tactic (e.g., Initial Access, Exfiltration) and outlines techniques and sub-techniques related to the various tactics, of which there are ~525 across all tactics as of this publication.
Each technique has a technical description, along with ways to both detect and mitigate the threat, making it helpful for communicating alert context and appropriate next steps. This is the one reason that since 2018, SIEM, EDR, and other security tool vendors have been rushing to integrate MITRE details into their alerting and detections.
MITRE ATT&CK is valuable but has some inherent weaknesses. The first is that the framework relies on partnerships with government agencies, open source intelligence (OSINT), cyber tool vendors, forensics investigators, and white hat network defenders to share information for the common good so that more organizations can understand the enemy better to bolster defenses. This can create a potential lag in attack visibility for months rather than days. Similar to signature-based detections, MITRE ATT&CK also suffers from requiring pre-knowledge of the TTP, so the framework cannot possibly cover all possible attacks. Despite its shortcomings, MITRE ATT&CK provides a strong reference baseline for TI today.
Guidance:
When talking to an MSSP providing MDR services, ask how the organization applies the ATT&CK framework
and its capability to collect more current, advanced threat intelligence.
Who can take you beyond MITRE ATT&CK?
There are four teams that can help you go above and beyond MITRE ATT&CK. First, the amount of TI an MSSP is able to collect is a function of the number of sources they leverage and the resources dedicated to that activity, sometimes referred to as a “Research” function.
The next two teams, White Hats and Incident Response specialists, typically split time between their practice area and the Research Group. Their daily work provides the Research team with ready sources of software vulnerabilities, current threat actor behaviors, and/or fresh indicators of compromise (IoCs) to go beyond the shared OSINT information from MITRE.
The fourth key team, made up of Detection Engineers and Data Scientists, is perhaps the most important. The former’s skill set includes the ability to translate understanding of specific attack behavior into a “detection envelope.” Using TI provided by MITRE ATT&CK and the other three groups to build the model of what bad looks like, detection engineers work with SIEM, EDR, and other technology teams to build the advanced detection logic
1. Researchers
Research Group Team Members:
• OSINT Analysts/Focused Threat Group Specialists
• Impersonation Specialists (to infiltrate groups and forums)
• Government Agency/Vendor Liaisons
• Forensics Investigators
• Vulnerability Researchers (to identify new Zero-Day threats or review new patches)
• Malware Reverse Engineers
The first half of this list often focuses on collecting specific indicators of compromise (IoCs). This may also include evidence of previous data breaches that could trigger deeper forensics investigations for clients. They also may see threat actor communication related to pending attacks, allowing the MSSP to warn affected clients to review appropriate defenses.
Forensic Investigators, Vulnerability Researchers, and Malware Reverse Engineers play the important role of breaking down the key attack elements that can really enhance the service delivery; their work is often on the bleeding edge of attacks. Put simply, there are fewer people with this type of training, and they are harder to staff and less commonly seen at MSSPs.
- Vulnerability Researchers look for undisclosed vulnerabilities in existing software that threat actors might use in an attack, which threat hunters can use. They also take additional time to review the mechanisms of known exploits used against critical vulnerabilities and validate patch efficacy as vendors roll out patches to help clients avoid a false sense of security.
- Malware Reverse Engineers break down malware into separate components of the attack sequence to identify techniques and IoCs to use for detection logic or proactive hunting. As malware often leverages some kind of vulnerability, this work also sheds light on key vulnerabilities for the Vulnerability Researchers to focus on.
- Forensics Investigators provide context on logging sources to consider for building new detections based on where they identified forensics artifacts in investigations.
Guidance:
This presence of the above bulleted group indicates a more mature TI collection approach and stronger capabilities
to leverage and apply knowledge as effective detection and response. Buyers should be looking for the different
established skill-sets that support this in their service provider.
2. White Hat Attackers
One group that shares common features with the Vulnerability Researchers are White Hat Attacker teams delivering penetration testing and attack simulation services. Their daily work centers on identifying and exploiting vulnerabilities, generating information they can share with the Vulnerability Researchers. Their TTPs also mimic those of genuine threat actors to avoid detection during the testing, and perform different forms of exploit that may provide more insights that detection and response services can incorporate.
While penetration testing can offer some basic insights, service providers offering advanced, goal-based (“Red Team”), or threat intelligence led penetration tests (e.g., CREST, Simulated Targeted Attack and Response methodology) often face and evade a range of EDR tools in their engagements. As EDR tools are a critical component of an MDR suite, organizations that understand how to identify tool bypass or failure can offer a stronger defense for their MDR clients. Testing organizations closely guard these techniques, so it is unlikely that an MSSP without this type of security practice would be able to find similar detections through general open-source research like MITRE or directly from the EDR vendor.
3. Incident Response Specialists
Some service providers claim they have an internal Incident Response (IR) team that offers digital forensics support and helps clients respond to an incident, but there are three characteristics to look for that would put this team in a better position to bolster threat intelligence collection.
1. The first aspect to consider is how recognized the team is as a standalone practice providing services for non-MDR clients. Non-MDR client investigations provide a chance to see a broader set of attack data and evidence of TTPs than the MDR practice may have visibility into.
2. Next, does the team have any certifications that might show advanced skills? One certification to look for is the Payment Card Industry (PCI) Council “Preferred Forensic Investigator” (PFI) designation, which currently only includes only 23 companies worldwide. PFI organizations perform forensic audits when the PCI council suspects a retailer as the source of a cardholder data breach which may involve complex network environments, and normally require advanced forensics skills to validate the allegations against the retailer.
3. Finally, the regularity of complex investigations performed by the team is a subtlety not often discussed. Complex investigations are typically required for sophisticated threat groups that have been in an environment for months or years before defenders discover them. In these situations, IR teams would perform forensic analysis on multiple devices to look for persistence mechanisms or other indicators of compromise, and apply more advanced techniques to ensure eradication.
The artifacts collected during these engagements often provide deeper insights into more sophisticated attacks to use for MDR client detections. If a service provider’s IR team only performs investigations on commodity attacks affecting only one system in an environment where the client does not see value in a full root-cause analysis investigation, it is less likely their IR team will have the skills or exposure to the variety of attack data that standalone practices see.
Guidance:
Buyers should ask about the most persistent engagement the team has performed recently, evidence of
large complex investigations, and confirm how actively information is shared between the response and forensics
professional and Research.
4. Detection Engineers and Data Scientists
For attacks that are not easily identified with detection logic, Detection Engineers help guide the four Ws—when, where, why, and what to hunt for. The skills of this group influence the detection ability and quality the service provides, especially for more sophisticated attacks. The size of the team will determine how quickly they can create and implement new detections, and develop the MITRE ATT&CK mapping and linking for recently identified TTPs and/or technologies.
Data Scientists are responsible for the advanced analytics tools used to deliver the service, and they explore new detection possibilities to address advanced attack methods. They often are responsible for constructing anonymized data lakes to mine for new IoCs or behavioral patterns through advanced search queries and applying machine learning models. Similar to some of the members of the Research Group, their presence is a sign of maturity and significant investment by an MSSP to enhance the services.
Guidance:
The following questions can identify the strength and focus of this team compared to other service providers:
• How many new detection analytics did the team developed in the last year?
• How did that improve on the MITRE ATT&CK coverage?
• What is the typical release cycle for new analytics?
• How do they test the efficacy of those analytics?
Conclusion: How does your MSSP measure up?
To wrap up this paper, we thought to share the definition of an MDR provider as provided by leading security analyst Gartner,
“Managed detection and response (MDR) providers deliver 24/7 threat monitoring, detection and lightweight response services to customers leveraging a combination of technologies deployed at the host and network layers, advanced analytics, threat intelligence, and human expertise in incident investigation and response. MDR providers undertake incident validation, and can offer remote response services, such as threat containment, and support in bringing a customer’s environment back to some form of 'known good'.”
This definition suggests that technology support alone is not enough. To differentiate between service providers delivering true MDR capabilities and those just offering “MDR services” around technology platforms, focus on the composition, experience level, and the on-going work of the four key teams that contribute to the detection engineering, threat hunting, and IR activities, which is directly related to how the organization collects and applies threat intelligence (TI).
Main takeaways from this paper:
- As more and more MSSPs and other vendors move toward the MDR suite of EDR tools for proactive hunting and response, SIEM for log monitoring and other devices that give them the appearance of delivering MDR services, remember that MDR is not technology-based alone.
- There should be a clear focus by the MSSP on integrating the MITRE ATT&CK framework as a minimum baseline for threat intelligence, while using a Research Group, Penetration Testing Practice and a Standalone IR Practice to capture additional TI and IoCs that may not be immediately available through TI vendors or OSINT sources like MITRE ATT&CK.
- Validate how the MSSP applies the collected threat intelligence and how that improves detection quality, ticket context and threat hunting methodologies.
- Make sure to assess the size and roles of team members these teams have, putting priority on the Vulnerability Researchers, Malware Reverse Engineers, Forensic Specialist, Red Team Penetration Testers and Data Scientists.
The final paper in this series, “What’s Missing From Your Managed Detection and Response Solution?” focuses on the challenges of building similar teams and skills in-house and the value of outsourcing these activities to MDR Providers to help organizations build the business case and/or evaluate the total cost of ownership of a full functioning Security Operation Center.
About the author
MSS Sales Specialist, Security Consulting, NCC Group, NA
John Hollner has been selling managed security services for over a decade with a heavy focus on SIEM and EDR technologies, the core of MDR solutions. He has worked extensively with financial services, healthcare, legal and manufacturing organizations during that time. He is currently the RFP Manager in North America, where he draws on his industry knowledge to differentiate the NCC Group services.
Read part 4 of this series or learn more about our MDR solutions.
This guide and series will seek to provide clarity around a fast-moving, cluttered managed security marketplace. The installments of this guide will cover:
Part 1: Common buzzwords associated with MDR and their true meaning
Part 2: Why Endpoint Detection and Response (EDR) can only take you so far
Part 3: Is your MSSP providing true MDR capability?
Part 4: In-house MDR creates more problems than it solves, and the case for outsourcing MDR