Filter Content Inspection

Content Inspection Filter

Description

This filter's goal is to inspect data provided to it, and find patterns using the Yara engine. As of today (29^th of July, 2019), this filter only works in pair with Rsyslog and its impcap and mmdarwin modules.

Filter Code

0x79617261

Dependencies

• yara (libyara-dev) 3.1 or above, and 4.x

Parameters

Content_inspection supports a set of parameters, listed in a Json formatted file and given to the filter through the general Darwin configuration file. The different parameters are:

• yaraScanType (string, default=none): defines the scan type, can be either stream or packet.
  • when set to stream, the filter will follow TCP streams and assemble packets to scan this reassembled stream.
  • when set to packet, the filter will only scan each packet's data for matches
• yaraRuleFile (string, default=none): full path to the yara rule file to be used by the Yara library (additional rules can be added by including the corresponding files to this "main" rule file).
• yaraScanMaxSize (int, default=4096): set the maximum buffer length to use for the scan (will only scan the first n bytes in packet scan type, and will create a buffer of n bytes for stream, that will be shifted for every new packet).
• maxConnections (int, default=8192): when following streams, limits the number of concurrents network flows.
• streamStoreFolder (string, default=none): this parameter should not be used most of the time, and could mostly be useful for developers. It gives the filter a folder path to store every recomposed stream it receives, this can be useful to recover full stream conversations for analysis, but will take a lot of disk space and IO.
• maxMemoryUsage (int, default=1024): this parameter is used by the memory management system to cap the amount of memory used by the filter: to improve efficienty, the content inspection filter creates pools of object to use and/or reuse during workflows. As such, one can set the size allocated to pools to limit the memory taken by them. This does in no way represent the full memory usage taken by the filter, and will only permit coarse memory upper limit. the parameter sets the limit in MegaBytes (MB).
• redis_socket_path (string) (optional) : the redis in which the filter will stock the data
• alert_redis_list_name (string) (optional) : the key under which the filter will stock the raised alerts in the redis
• alert_redis_channel_name (string) (optional) : the redis channel in which the raised alerts will be publish
• log_file_path (string) (optional) : the path in which the filter will write raised alerts

For the filter to start, at least yaraScanType or streamStoreFolder should be given. If yaraScanType is given, yaraRuleFile should also be given.

Example

Here is an example of a configuration file specific to content_inspection:

{
    "maxConnections": 12000,
    "yaraRuleFile": "/etc/darwin/rules",
    "yaraScanType": "stream",
    "yaraScanMaxSize": 20000,
    "maxMemoryUsage": 560,
    "redis_socket_path": "/var/sockets/redis/redis.sock",
    "alert_redis_list_name": "darwin_alerts",
    "alert_redis_channel_name": "darwin.alerts",
    "log_file_path": "/var/log/darwin/alerts.log"
}

And the parameters for the general Darwin configuration file:

{
    "content_inspection_1": {
        "exec_path": "/root/darwin/build/darwin_content_inspection",
        "config_file": "/root/darwin/conf/content_inspection.conf",
        "decision_filter": "",
        "nb_thread": 1,
        "log_level": "DEBUG",
        "cache_size": 0,
        "output": "LOG",
        "next_filter": ""
    }
}

Rsyslog input

WARNING: this filter cannot work with the python client, and can't receive bulk call. Please use it only with Rsyslog and provided configuration below !

Modules necessary

To work alongside this filter, Rsyslog must be used with those compiled modules:

• impcap
• mmdarwin

Impcap

The Impcap module is a network sniffing module, it will recover information and data from packets on the network. Please go to its corresponding wiki to get full documentation on how to configure it.

Mmdarwin

The Mmdarwin module comes after Impcap, and handles the transmission of data to darwin, the configuration of Mmdarwin in the rsyslog configuration file should be:

action(type="mmdarwin" socketpath="/path/to/darwin/sockets/content_inspection_1.sock" fields=["!impcap", "!data"] key="content_inspection" filtercode="0x79617261")