This code is an implementation for paper: EXTRACTOR: Extracting Attack Behavior from Threat Reports, a nice work on Threat Report Extracting in Cyber Threat Intelligence (CTI) .
- Environment
- NLP submodules
- NLP pretrained models
- Dependent libraries
- Usage
- Example
- Demo
- Reference
The knowledge on attacks contained in Cyber Threat Intelligence (CTI) reports is very important to effectively identify and quickly respond to cyber threats. However, this knowledge is often embedded in large amounts of text, and therefore difficult to use effectively. To address this challenge, we propose a novel approach and tool called EXTRACTOR that allows precise automatic extraction of concise attack behaviors from CTI reports. EXTRACTOR makes no strong assumptions about the text and is capable of extracting attack behaviors as provenance graphs from unstructured text. We evaluate EXTRACTOR using real-world incident reports from various sources as well as reports of DARPA adversarial engagements that involve several attack campaigns on various OS platforms of Windows, Linux, and FreeBSD. Our evaluation results show that EXTRACTOR can extract concise provenance graphs from CTI reports and show that these graphs can successfully be used by cyber-analytics tools in threat-hunting.
this code supports python3; not support python2
download model for spacy
python -m spacy download en_core_web_lg
download nltk when setting param crf is false
import nltk
nltk.download('averaged_perceptron_tagger')cd $PROJECT_HOME
git submodule init
git submodule updatedownload pretrain model for allennlp
wget -c -t 0 https://s3-us-west-2.amazonaws.com/allennlp/models/srl-model-2018.05.25.tar.gz
mv srl-model-2018.05.25.tar.gz srl-model.tar.gz # in current dirLinux:
Ubuntu: sudo apt install graphviz
Fedora: sudo yum install graphviz
Debian: sudo apt install graphviz
Redhat/Centos: sudo yum install graphviz # Stable and development rpms for Redhat Enterprise, or CentOS systems* available but are out of date.
Mac:
sudo port install graphviz
brew install graphviz
dot xxx.dot -T png -o xxx.pngRun EXTRACTOR with
python3 main.py [-h] [--asterisk ASTERISK] [--crf CRF] [--rmdup RMDUP] [--elip ELIP] [--gname GNAME] [--input_file INPUT_FILE]
Depending on the usage, each argument helps to provide a different representation of the attack behavior.
[--asterisk true] creates abstraction and can be used to replace anything that is not perceived as IOC/system entity into a wild-card. This representation can be used to be searched within the audit-logs.
[--crf true/false] allows activating or deactivating of the co-referencing module.
[--rmdup true/false] enables removal of duplicate nodes-edge.
[--elip true/false] is to choose whether to replace ellipsis subjects using the surrounding subject or not.
[--input_file path/filename.txt] is to pass the text file to the application.
[--gname graph_name] is to specify the name output graph (two files will be created, e.g., graph.pdf and graph.dot).
python3 main.py --asterisk true --crf true --rmdup true --elip true --input_file input.txt --gname mygraph`
python main.py --asterisk false --crf false --rmdup false --input_file input.txt
python main.py --asterisk false --crf true --rmdup false --input_file input.txt
python main.py --asterisk true --crf true --rmdup true --elip true --input_file input.txt --gname mygraph
python main.py --asterisk true --crf false --rmdup true --elip true --input_file input.txt --gname mygraph
- EXTRACTOR: Extracting Attack Behavior from Threat Reports. Paper
- EXTRACTOR. Code
- Passive/Active sentence Transformer. Code
If this project help you reduce time to develop, you can give me a cup of coffee :)
AliPay(支付宝)
WechatPay(微信)
GPL-3.0 © Kun