Security practitioners have long hoped for a solution that would provide real-time behavioral threat detection at the scale of modern corporate networks. However, historically enterprises have had to rely on network intrusion detection which limits by known vulnerability patterns and endpoint protection to identify threats with limited behavior capabilities with agents. While both categories can do part of the job, they are challenged by advanced and zero-day threats. This has left a massive gap in network visibility for some organizations.
Luckily, the research and development of artificial intelligence (AI) technology have been accelerating rapidly in recent years. Scientists and engineers have been making significant progress in many AI-related fields, such as machine learning, natural language processing, and computer vision. AI is already being used in a variety of industries, and it shows great promise in the world of cybersecurity. AI-powered network detection systems are able to identify and respond to threats much faster than traditional systems, and they're only getting better as the technology evolves. In this blog, I will share some of the advancements in AI development to overcome challenges in network detection and response.
Most of the Network Detection and Response (NDR) tools available today use unsupervised machine learning to detect and respond to network threats. NDR systems are trained to establish the normal patterns in network traffic in order to later identify activity outside those norms that may indicate malicious activity. Once a threat is detected, the NDR system can take initiate action to contain the attack and notify the security team.
However, nowadays the majority of network traffic is encrypted. This means NDR can only rely on limited information in the traffic header for pattern recognition. Recently, with the introduction of the latest encryption standard (TLS1.3), it has become more difficult for detection tools to obtain visibility of the network traffic because more of the domain information is being encrypted.
To solve this problem, a more experienced automation and pattern labelling approach is needed for traffic fingerprints pattern. Moreover, the introduction of encrypted traffic fingerprinting with network detection combined with other metrics such as indicators of compromise (IOC) enhances the confidence level of machine learning to detect a possible threat.
It is important to understand that files traversed across the network cannot be easily decoded with passive monitoring by network traffic alone, especially in an encrypted environment. The intercepting of all files across encrypted networks can be done by firewall deep inspection and endpoint systems. However, in order to process a massive number of files across a network in near real-time require a different approach instead of sandboxing or simulation.
The introduction of AI detection of file object recognition capability enhances network detection to understand the threat impact of an attack. Object recognition is the process of identifying objects in images or videos. However, such ideas can be repurposed for the identification of relationships between files as objects relationship in split seconds. The process involves the recognition of code blocks pattern and their relationship to malicious code using an Artificial Neural Network (ANN). In order to have accurate model training, it requires a large amount and continuous malicious file sampling. Since every organization’s file patterns are different, NDR should have on-premises AI modelling learning capability along with supervision by cloud base model. Therefore, a global threat intelligence with millions of samples plays a vital role in providing the baseline for model training.
In order to effectively understand the threat detected by network and file detection for hunting purposes, the best approach is by leveraging the framework of tactics, techniques, and procedures (TTP). When an abnormality is detected, both network and file detection can map an attack into a scenario to understand the tactics of the threat actor, the technique or method of being used in an attempt to compromise the system, and the procedure involved in the cyber kill chain.
Artificial intelligence can play a significant role in advancing cybersecurity of network detection. The advancement of AI in NDR increases the ability to detect advanced threats, however, in order to provide rapid response and remediation, the attack scenarios should be able to be incorporated into the various cyber kill chain tools for protection. The use of a security fabric security solution can effectively respond to and remediate the threats detected in NDR according to different stages in the kill chain providing a simple integration and orchestration to automate proper protection tools to block the attack.