Industry Trends

Q&A: Today’s Cyber Threat Landscape – 3rd Party Testing and Sandboxes

By Bill McGee | August 02, 2016

Sandbox technology and the ongoing effort to defend and protect against cyber threats continues. Independent testing offers organizations valuable insight into what solutions are best for their use case. Below we discuss this maturing technology and the role of testing with Fortinet’s David Finger.

I saw the announcement from NSS Labs about their recent Breach Detection Group Test results. What is the significance?

Cyberattacks and breaches continue to be top of mind across organizations, and because of that, sandbox technology (or Breach Detection in NSS terms), which was covered in the recent test, is increasingly important. In fact, I view it as a “must-have” technology going forward. However, there are literally dozens of vendors in this market, each claiming to have the best solution. This can be bewildering for companies looking to spend their valuable security budget wisely. Which is why independent testing is so important for security in general (our founder has blogged about this previously), and for this particular solution area in particular. It helps take the guesswork out of the shortlist and selection process, and separates real functionality from marketing hype.

What I like about NSS Labs is that they have an open process to set each test methodology - encouraging feedback from vendors and collecting requirements from end user clients, in addition to leveraging the experience of their own test staff. They also test on a recurring basis, and invite public participation from any vendor that feels they have a product that meets the test requirements. Finally, their rating method is simple and data driven.

You mentioned sandboxing as a “must-have” technology. Why that one as opposed to others?

If you think about it, over the years we as a security industry have developed and deployed a lot of great technologies - signatures, heuristics, reputations, behavior analysis, and more - at many places in the organization to detect and block cyberattacks. However, they are almost all designed for in-line deployment, requiring them to be fast and accurate and to not impede business. Knowing this, cybercriminals are crafting attacks to pass quick inspection, or introduce a degree of doubt during analysis, to allow them to pass through traditional blocking technologies. By contrast, sandboxing is designed to inspect deeper and longer in order to detect those very threats. So it’s quite different than most of the security technology we have deployed to date. And based on its ability to detect sophisticated and previously unknown attacks, it has become absolutely necessary.

Given that sandboxes are often a detection tool, they can create significant work. How can you recommend them as a “must-have” for everyone?

Personally, I am not a fan of sandboxes that work independently from the other security components already deployed by organizations and create that heavy investigation and response effort you mention. I encourage all folks that I speak with to not only make sure they have a sandbox, but to make sure their solution is well integrated into their overall security infrastructure. Ensuring that sandbox technology is a central service that shares objects and intelligence across multiple security components is something strategic for us, and I think it should be for most organizations. In fact, it’s a critical part of our Security Fabric vision.

Any last take-aways from this report today?

A few items I found interesting, and a call-out of how Fortinet did in the testing:

  • Of the dozens of vendors talking about how great their sandbox is, only 7 participated in the test. I found it disappointing that so few participated, especially now that this is the 3rd NSS BDS test.
  • At the same time, it was heartening to me that the average effectiveness of all products tested was much higher this year, at almost 95%! Also, that the average TCO was much improved to ~$80/Mbps. So, for the most part, products are getting better and more affordable.
  • As for Fortinet, we submitted both our appliance, and cloud service sandbox offerings integrated with our enterprise firewall and endpoint protection product, demonstrating that regardless of form factor or integration point you can feel confident in the Fortinet sandbox because it is integrated into our Fortinet Security Fabric.


You can read the detailed reports for our products here.

And our sandbox detection is now a part of our free Cyber Threat Assessment Program, making it easy for organizations to get a sense of its value.