add backticks to the sphinx gravwell query lexer, update eval/regex for

formatting and raw string support

gravy_lexer.py

@@ -48,7 +48,7 @@ class GravwellLexer(RegexLexer):
             # strings
             #
             (r'"([^"])*$', Error),  # non-teminated string
-            (r'"', String.Delimiter, "string"),
+            (r'["`]', String.Delimiter, "string"),
             #
             # open comment
             #

search/eval/eval.md

@@ -1,14 +1,14 @@
-## Eval
+# Eval
 
 Eval is most commonly used for performing AND and OR logic on searches and enumerated values. However, the eval module is a bit of a Swiss Army knife, providing access to a limited subset of the Anko programming language (a dynamically-typed Go-like language, see [https://github.com/mattn/anko/](https://github.com/mattn/anko/) and the [Gravwell documentation for the Anko language](/scripting/scripting)) to allow flexible operations on data within Gravwell. The eval module will execute exactly one expression or statement. In order to keep this page relatively simple, this section provides only a brief overview of some example eval invocations; more details are available [in this article](/scripting/eval)
 
-### Syntax
+## Syntax
 
 `eval <expression>`
 
 The <expression> must be a single Anko expression, as described in [the eval documentation](/scripting/eval).
 
-### Examples
+## Examples
 
 A simple application of the eval module might be to separate out Reddit comments which are less than 20 characters long. We do this by using the json module to extract the `Body` field, then passing it to eval with an expression which evaluates to true whenever the length of the comment’s `Body` field is less than 20. Only entries for which the expression evaluate to true are allowed to continue down the pipeline. Finally, we simply send the result of the eval to the table module to display the comment bodies which are less than 20 characters long.
 
@@ -40,7 +40,7 @@ if len(Body) <= 10 { setEnum("postlen", "short"); setEnum(“anotherEnum”, “
 switch DstPort { case 80: setEnum(“protocol”, “http”); case 22: setEnum(“protocol”, “ssh”); default: setEnum(“protocol”, “unknown”) }
 ```
 
-### Further reference
+## Further reference
 
 * [The Gravwell documentation for the Anko language](/scripting/scripting) is a generic description of the Anko scripting language
 * [The eval module article](/scripting/eval) describes the eval module in more detail.

search/regex/regex.md

@@ -1,4 +1,4 @@
-## Regex
+# Regex
 
 Regex is a pipeline module that uses regular expressions to match text data. It is an extremely powerful way of matching complex patterns and extracting enumerable fields from text. For those unfamiliar with regular expressions, a decent starting point is [the Wikipedia article](https://en.wikipedia.org/wiki/Regular_expression).
 
@@ -12,7 +12,16 @@ For example, the following search will enumerate the method, user, and ip from a
 
 Because regular expressions can get very long, the regex module takes the `-r` flag, which specifies a resource containing a regular expression. When populating the resource, do not include "wrapping quotes" around the whole expression as you would when typing directly into a search: e.g. `".*ssh.*Accepted"` becomes `.*ssh.*Accepted`. This is because the quotes are normally stripped out by the search parser prior to being handed to the regex module.
 
-### Supported Options
+## Raw strings
+
+To facilitate using escape sequences in regular expressions, you can use backticks to prevent Gravwell from unescaping your input. For example:
+
+```gravwell
+tag=syslog grep sshd | regex `shd.*Accepted (?P<method>\S*) for (?P<user>\S*) from (?P<ip>[0-9]+.[0-9]+.[0-9]+.[0-9]+)`
+```
+
+
+## Supported Options
 
 * `-e <arg>`: The “-e” option operates on an enumerated value instead of on the entire record. For example, a pipeline that showed packets not headed for port 80 but that have HTTP text would be `tag=pcap packet ipv4.DstPort!=80 tcp.Payload | regex -e Payload ".*GET \/ HTTP\/1.1.*"`
 * `-r <arg>`: The “-r” option specifies that the regular expression statement is located in a resource file. 
@@ -23,7 +32,7 @@ Because regular expressions can get very long, the regex module takes the `-r` f
 Storing especially large regular expressions in resource files can clean up queries, and allows for easy reuse.  If `-r` is specified, do not specify a regular expression in the query -- instead the contents of the resource will be used. Handy!
 ```
 
-### Inline Filtering
+## Inline Filtering
 
 The regex module supports inline filtering to allow for down-selecting data directly within the regex module.  The inline filtering also enables regex to engage accelerators to dramatically reduce the amount of data that needs to be processed.  Inline filtering is achieved in the same manner as other modules by using comparison operators.  If a filter is enabled that specifies equality ("equal", "not equal", "contains", "not contains") any entry that fails the filter specification will be dropped entirely.  If a field is specified as not equal "!=" and the field does not exist, the field is not extracted but the entry won't be dropped entirely.
 
@@ -35,17 +44,19 @@ The regex module supports inline filtering to allow for down-selecting data dire
 | ~ | Subset | Field contains the value
 | !~ | Not Subset | Field does NOT contain the value
 
-#### Filtering Examples
+### Filtering Examples
 
 ```gravwell
-tag=syslog regex "shd.*Accepted (?P<method>\S*) for (?P<user>\S*) from (?P<ip>[0-9]+.[0-9]+.[0-9]+.[0-9]+)" user==root ip ~ "192.168"
+tag=syslog regex `shd.*Accepted (?P<method>\S*) for (?P<user>\S*) from (?P<ip>[0-9]+.[0-9]+.[0-9]+.[0-9]+)` user==root ip ~ "192.168"
 ```
 
-### Parameter Structure
+## Parameter Structure
+
 ```
 regex <argument list> <regular expression> <filter arguments>
 ```
-### Example Search
+
+## Example Search
 ```gravwell
-tag=syslog grep sshd | regex *shd.*Accepted (?P<method>\S*) for (?P<user>\S*) from (?P<ip>[0-9]+.[0-9]+.[0-9]+.[0-9]+)"
+tag=syslog grep sshd | regex `shd.*Accepted (?P<method>\S*) for (?P<user>\S*) from (?P<ip>[0-9]+.[0-9]+.[0-9]+.[0-9]+)`
 ```