Merge remote-tracking branch 'upstream/dev' into alerts

ingesters/ingesters.md

@@ -318,6 +318,29 @@ Log-Source-Override=DEAD:BEEF::FEED:FEBE
 Log-Source-Override=::1
 ```
 
+### Attach
+
+All ingesters support the `Attach` global configuration stanza, which allows [intrinsic enumerated values](intrinsic_enumerated_values) to be attached to entries during ingest. Intrinsic enumerated values can later be accessed with the [intrinsic](/search/intrinsic/intrinsic) search module.
+
+The `Attach` stanza takes any key/value pair, and will attach it to every entry as an enumerated value at the time of ingest. For example:
+
+```
+[Attach]
+	foo = "bar"
+	ingester = "my ingester"
+```
+
+Will attach an EV "foo" with the contents "bar" to every entry, as well as "ingester" with the value "my ingester".
+
+Additionally, the below variables can be used to populate values:
+
+```
+[Attach]
+	time = $NOW 		# add the current timestamp
+	host = $HOSTNAME	# add the hostname the ingester is running on
+	uuid = $UUID		# add the ingester's UUID
+```
+
 ## Data Consumer Configuration
 
 Besides the global configuration options, each ingester which uses a config file will need to define at least one *data consumer*. A data consumer is a config definition which tells the ingester:

search/extractionmodules.md

@@ -76,6 +76,7 @@ csv <csv/csv>
 dump <dump/dump>
 fields <fields/fields>
 grok <grok/grok>
+intrinsic <intrinsic/intrinsic>
 ip <ip/ip>
 ipfix <ipfix/ipfix>
 j1939 <j1939/j1939>
@@ -101,6 +102,7 @@ xml <xml/xml>
 * [dump](dump/dump) - dump entries from a resource into the pipeline.
 * [fields](fields/fields) - extract data from entries using arbitrary field separators.
 * [grok](grok/grok) - extract data from complicated text structures using pre-defined regular expressions.
+* [intrinsic](intrinsic/intrinsic) - populate entries with enumerated values that were created at ingest time.
 * [ip](ip/ip) - convert & filter IP addresses.
 * [ipfix](ipfix/ipfix) - extract data from IPFIX records.
 * [j1939](j1939/j1939) - parse J1939 data.

search/intrinsic/intrinsic.md

@@ -0,0 +1,23 @@
+# Intrinsic
+
+The `intrinsic` module extracts enumerated values that were created at ingest time to a given search. By default, when using the text or raw renderers, all intrinsic enumerated values are added to the search. When using any other render module, such as table, the intrinsic module must be used. 
+
+## Supported Options
+
+The intrinsic module has no flags.
+
+## Arguments and syntax
+
+The intrinsic module simply takes a list of enumerated values to extract, and optionally a filter for each enumerated value. 
+
+For example, to extract the enumerated values "foo" and "bar":
+
+```gravwell
+tag=data intrinsic foo bar | table
+```
+
+Additionally, to filter "foo" to just entries where foo is equal to "potato":
+
+```gravwell
+tag=data intrinsic foo == "potato" bar | table
+```

search/search.md

@@ -127,6 +127,13 @@ tag=netflow netflow Src Dst
 
 Further examples throughout the documentation should help clarify the use of enumerated values.
 
+(intrinsic_enumerated_values)=
+## Intrinsic Enumerated Values
+
+Intrinsic Enumerated Values are Enumerated Values which are created at the time of ingest. Intrinsic enumerated values are optionally created by ingesters and often contain metadata or pre-processed extractions. Use the [intrinsic](intrinsic/intrinsic) module to work with intrinsic enumerated values.
+
+After extracting an intrinsic enumerated value, they are treated the same as regular enumerated values.
+
 ## Quoting and tokenizing
 
 When specifying arguments to Gravwell modules, be mindful of special characters. Most modules treat spaces, tabs, newlines, and the following characters as separators: !#$%&'()*+,-./:;<=>?@