Skip to content

Latest commit

 

History

History
11 lines (6 loc) · 2.36 KB

README.FILTERS.md

File metadata and controls

11 lines (6 loc) · 2.36 KB

A message filtering capability was added to the libnmsg I/O loop in nmsg 0.11.0.

Message filters are callbacks which can decide to accept a message into the output stream, silently drop it, or pass it to the next filter in the chain. Filters hook into the nmsg_io processing loop and can alter or replace messages. See nmsg/filter.h for the documentation on filter semantics.

A message filter callback can either be provided as a function pointer + data pointer via nmsg_io_add_filter(), or it can be contained in an external shared object (a "module" or "plugin") which can be loaded and initialized via nmsg_io_add_filter_module(). See nmsg/fltmod_plugin.h for the interface that filter plugin providers must implement, and see nmsg/fltmod.h for the interface used by filter plugin consumers, like nmsg_io.

nmsg_io_add_filter() and nmsg_io_add_filter_module() can be called repeatedly to build up a linear filter chain. The order of filters in the chain is the same as the order of calls to nmsg_io_add_filter() or nmsg_io_add_filter_module(). Each input message processed by the nmsg_io loop is passed to each filter in sequence, until a filter returns a terminal verdict (ACCEPT or DROP). If a filter returns a non-terminal verdict (DECLINED), the message is passed to the next filter in the chain. If the end of the filter chain is reached, the nmsg_io loop's filter policy is applied. (The default is ACCEPT, but it can be changed to DROP with nmsg_io_set_filter_policy()).

Two new command-line options have been added to nmsgtool. -F or --filter loads a filter module, and --policy sets the default filter chain policy. These are relatively thin wrappers around nmsg_io_add_filter_module() and nmsg_io_set_filter_policy().

A sample filter module is provided, which functions as both a code sample of how to provide a filter plugin using the nmsg/fltmod_plugin.h interface, as well as providing message sampling functionality. It can perform either systematic count-based sampling or uniform probabilistic sampling. E.g., try nmsgtool --filter sample,count=10 [...] to systematically sample every 1-in-10 messages in the input stream, or nmsgtool --filter sample,random=0.10 [...] to probabilistically sample every message in the input stream with probability 10%.