Filter Anomaly

Anomaly filter

Description

It's a filter that apply algorithms on connection data, and find anomalies in this data.

Filter Code

0x414D4C59

Dependencies

• armadillo 9.400 or above, with:
  • lapack (preferably lapacke)
  • blas (preferably openblas)
• mlpack 3.0.1 or above

Darwin configuration

Example of darwin configuration for this filter :

{
    "anomaly_1": {
        "exec_path": "/root/darwin/build/darwin_anomaly",
        "config_file":"",
        "output": "LOG",
        "next_filter": "",
        "nb_thread": 1,
        "log_level": "DEBUG",
        "cache_size": 0
    }
}

Config file

• redis_socket_path (optional) : the redis in which the filter will store the data
• alert_redis_list_name (optional) : the key under which the filter will store the raised alerts in the redis
• alert_redis_channel_name (optional) : the redis channel in which the raised alerts will be publish
• log_file_path (optional) : the path in which the filter will write it's results

Example :

{
    "redis_socket_path": "/var/sockets/redis/redis.sock",
    "alert_redis_list_name": "darwin_alerts",
    "alert_redis_channel_name": "darwin.alerts",
    "log_file_path": "/var/log/darwin/alerts.log"
}

Body

[ 
   [ 
      ["<net_src_ip>",<nb_dst_port_udp>,<nb_host_udp>,<nb_dst_port_tcp>,<nb_host_tcp>,<nb_host_icmp>], 
      [..] 
   ], 
      [..] 
]

Each array provided need more than 10 lines of data.

Example

Here is an example of a body:

[ 
   [     
      ["127.0.0.1",160,290,1000,990,980],
      ["127.0.0.2",90,27,40,96,82],
      ["127.0.0.3",84,11,20,87,59],
      ["127.0.0.4",44,67,84,78,35],
      ["127.0.0.5",57,57,51,8,94],
      ["127.0.0.6",15,24,7,24,74],
      ["127.0.0.7",77,51,48,12,31],
      ["127.0.0.8",101,93,55,94,53],
      ["127.0.0.9",20,34,44,95,74],
      ["127.0.0.10",6,35,62,41,89],
      ["127.0.0.11",29,31,96,18,87],
   ], 
   [
      ["127.0.0.12",74,61,26,2,79],
      ["127.0.0.13",93,34,47,86,97],
      ["127.0.0.14",36,36,4,45,60],
      ["127.0.0.15",22,75,62,26,75],
      ["127.0.0.16",34,36,29,60,46],
      ["127.0.0.17",23,88,15,99,95],
      ["127.0.0.18",86,22,30,39,30],
      ["127.0.0.19",7,16,38,37,73],
      ["127.0.0.20",70,69,57,50,81],
      ["127.0.0.21",61,47,76,51,31],
      ["127.0.0.22",64,70,37,84,7],
      ["127.0.0.23",28,75,14,88,81],
      ["127.0.0.24",75,52,22,39,87],
   ],
]

Results

Send back certitude 0 or 100. 100 for anomaly found in the data cluster, 0 else. For example we can have [100, 0] for the body precised upside.

If the LOG output is precised in the darwin configuration, the body send by the filter will be for example :

{
  "alert_type": "darwin",
  "alert_subtype": "anomaly",
  "alert_time": "2020-08-11T18:49:58Z",
  "level": "high",
  "rule_name": "Abnormal Number of Unique Port Connexion",
  "tags": ["attack.discovery", "attack.t1046", "attack.command_and_control", "attack.defense_evasion", "attack.t1205"],
  "entry": "127.0.0.1",
  "score": 100,
  "evt_id": "f49ffb06-288d-48a6-92b7-09d2e709208e",
  "details": {
    "ip": "127.0.0.1",
    "udp_nb_host": 160.000000,
    "udp_nb_port": 290.000000,
    "tcp_nb_host": 1000.000000,
    "tcp_nb_port": 990.000000,
    "icmp_nb_host": 980.000000,
    "distance": 1611.169966
  }
}

Alert Details

"details": {
    "ip": <string, suspected IP>, 
    "udp_nb_host": <float, number of unique host connected via udp>,
    "udp_nb_port": <float, number of unique port connected via udp>,
    "tcp_nb_host": <float, number of unique host connected via tcp>,
    "tcp_nb_port": <float, number of unique port connected via tcp>,
    "distance": <float, distance to the closest normal asset>
}