Skip to content

Commit

Permalink
Doc update
Browse files Browse the repository at this point in the history
  • Loading branch information
@mcorbin
mcorbin committed Oct 31, 2021
1 parent b7056c4 commit 24b494b
Show file tree
Hide file tree
Showing 24 changed files with 613 additions and 502 deletions.
9 changes: 8 additions & 1 deletion site/mirabelle/content/api/_index.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,12 @@ The body is a json string like `{"config": "<stream-config>"}`.

<stream-config> should be the stream EDN configuration as base64. For example `{:actions {:action :info}}` as base64 would be `ezphY3Rpb25zIHs6YWN0aW9uIDppbmZvfX0=`.

You can also pass the `:default` option, like in the configuration, in order to specify if you want your stream to be used by default.
You can pass the `:default` option, like in the configuration, in order to specify if you want your stream to be used by default.

You can also set the `:persist` field to `true` in your stream configuration in order to persist it on disk.
In that case, Mirabelle will write the stream configuration into the **first** directory referenced into the `:stream {:directory ..;}` configuration. The file name will be `<stream-name>.edn`.

Deleting a stream through the API will also delete its file on the filesystem.

---

Expand Down Expand Up @@ -73,6 +78,8 @@ curl localhost:5558/api/v1/stream/trololo

Delete a stream by name.

If the stream was created using the API (see `Add a stream`) and persisted on disk, this call will also deleted its configuration file.

---

```
Expand Down
20 changes: 20 additions & 0 deletions site/mirabelle/content/changelog/_index.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,26 @@ weight: 60
chapter: false
---

This is the list of user-facing changes for Mirabelle.

## v0.7.0

- Add `/health` endpoint. Both `/health` and `/healthz` return the same thing.
- Improve websocket error messages on bad queries.
- Improve error messages in test mode.
- Add the ability to persist on the file system streams created through the API. It can be enabled by setting `persist` to `true` in the stream configuration.
- Add `coll-where` action. This action can be used to filter a list of events based on a condition.
- Add `coll-sort` action, which can be used to sort a collection of events based on a field.
- Add `moving-time-window` action. This window always return the events which arrived the last `duration` seconds.
- Add `ssort` action. This action buffers events for some time and send them one by one sorted to downstream actions. It can be a nice way to tolerate some late events in streams.
- Allow nesting `:or` add `and` values in conditions. For example `[:and [:or [:= :service "foo"] [:= :service "bar"]] [:> :metric 1]]` is now a value condition.

## v0.6.0

- Fix `default` action.
- Use maps instead of plain values for all actions dealing with internal states.
- Add `aggr-sum` action, which can be used to sum over time events.

## v0.5.0

- Elasticsearch I/O
Expand Down
36 changes: 35 additions & 1 deletion site/mirabelle/content/howto/stream/_index.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ Not all availables actions and I/O are listed here. You can see the full list in
- [Modifying events: update fields, values, set default values, select fields...](/howto/stream/#modifying-events)
- [Detect state transitions](/howto/stream/#detect-state-transitions)
- [Work with windows (time windows, events windows...)](/howto/stream/#events-windows)
- [Sort events](/howto/stream/#sort-events)
- [Action on lists of events: max, min, count, percentiles, rate...](/howto/stream/#actions-on-list-of-events)
- [Combine events from multiple hosts, correlate events between each others](/howto/stream/#coalesce-and-project)
- [The throttle action](/howto/stream/#throttle)
Expand Down Expand Up @@ -452,7 +453,32 @@ The first one, `fixed-time-window`, will buffer all events during a defined dura

The `fixed-event-window` action will created windows not based on time, but based on the number of events the action receives. For example, `(fixed-event-window {:size 60})` will buffer events until 10 are buffered, and then pass the window downstream.

The `moving-event-window` action works like `fixed-event-window` but will pass events downstream for every event received. For example, `(moving-event-window {size 10})` will in that case always send downstream the last 10 events.
The `moving-event-window` action works like `fixed-event-window` but will pass events downstream for every event received. For example, `(moving-event-window {:size 10})` will in that case always send downstream the last 10 events.

The `moving-time-window` action will return for each event all events from the last `duration` seconds, for example: `(moving-time-window {:duration 60})`.

#### Sort events

The `ssort` action can be used to

- bufferize events for a given duration
- Send then downstream one by one sorted based on a field.

Let's take for example `(ssort {:duration 10 :field :time})`. For this input:

```clojure
{:time 1} {:time 10} {:time 4} {:time 9} {:time 13} {:time 31}
```

The output would be:

```clojure
{:time 1} {:time 4} {:time 9} {:time 10} {:time 13}
```

Events are emitted downstream after twice the duration period. In this example, events received between times `0` and `10` will for example be emitted at time `20`.

This action can be very useful to tolerate late events in streams. You could for example use `ssort` in front of a `fixed-time-window` stream in order to wait for late events before creating the time window.

#### Actions on list of events

Expand All @@ -479,6 +505,10 @@ Let's take this example which creates windows of 10 events and forwards them to
(coll-top 10
(info))
(coll-bottom 10
(info))
(coll-where [:= :service "foo"]
(info))
(coll-sort :time
(info)))
```

Expand All @@ -502,6 +532,10 @@ In this example, we generate 60-seconds time windows and pass them to the `coll-

The `coll-percentiles` action will produce for each quantile an event with the `:quantile` key set to the quantile value, and the `;metric` field set to the value computed from the list of events for this quantile. The quantiles `0` and `1` can also be used to get the smallest of biggest event.

`coll-where` will keep only events in the list matching the provided conditio (see the [filtering events](/howto/stream/#filtering-events) section.

`coll-sort` will sort events in the list based on the field passed as parameter.

If needed, you can also flatten a list of events, to get back a single event using `flatten`:

```clojure
Expand Down
14 changes: 7 additions & 7 deletions site/mirabelle/public/404.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -9,13 +9,13 @@
<title>404 Page not found</title>


<link href="/css/nucleus.css?1633872343" rel="stylesheet">
<link href="/css/fontawesome-all.min.css?1633872343" rel="stylesheet">
<link href="/css/hybrid.css?1633872343" rel="stylesheet">
<link href="/css/featherlight.min.css?1633872343" rel="stylesheet">
<link href="/css/perfect-scrollbar.min.css?1633872343" rel="stylesheet">
<link href="/css/theme.css?1633872343" rel="stylesheet">
<link href="/css/hugo-theme.css?1633872343" rel="stylesheet">
<link href="/css/nucleus.css?1635692565" rel="stylesheet">
<link href="/css/fontawesome-all.min.css?1635692565" rel="stylesheet">
<link href="/css/hybrid.css?1635692565" rel="stylesheet">
<link href="/css/featherlight.min.css?1635692565" rel="stylesheet">
<link href="/css/perfect-scrollbar.min.css?1635692565" rel="stylesheet">
<link href="/css/theme.css?1635692565" rel="stylesheet">
<link href="/css/hugo-theme.css?1635692565" rel="stylesheet">

<style>
:root #header + #content > #left > #rlblock_left {
Expand Down
54 changes: 29 additions & 25 deletions site/mirabelle/public/api/index.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -12,20 +12,20 @@
<title>HTTP API :: Mirabelle</title>


<link href="/css/nucleus.css?1633872343" rel="stylesheet">
<link href="/css/fontawesome-all.min.css?1633872343" rel="stylesheet">
<link href="/css/hybrid.css?1633872343" rel="stylesheet">
<link href="/css/featherlight.min.css?1633872343" rel="stylesheet">
<link href="/css/perfect-scrollbar.min.css?1633872343" rel="stylesheet">
<link href="/css/auto-complete.css?1633872343" rel="stylesheet">
<link href="/css/atom-one-dark-reasonable.css?1633872343" rel="stylesheet">
<link href="/css/theme.css?1633872343" rel="stylesheet">
<link href="/css/tabs.css?1633872343" rel="stylesheet">
<link href="/css/hugo-theme.css?1633872343" rel="stylesheet">
<link href="/css/nucleus.css?1635692565" rel="stylesheet">
<link href="/css/fontawesome-all.min.css?1635692565" rel="stylesheet">
<link href="/css/hybrid.css?1635692565" rel="stylesheet">
<link href="/css/featherlight.min.css?1635692565" rel="stylesheet">
<link href="/css/perfect-scrollbar.min.css?1635692565" rel="stylesheet">
<link href="/css/auto-complete.css?1635692565" rel="stylesheet">
<link href="/css/atom-one-dark-reasonable.css?1635692565" rel="stylesheet">
<link href="/css/theme.css?1635692565" rel="stylesheet">
<link href="/css/tabs.css?1635692565" rel="stylesheet">
<link href="/css/hugo-theme.css?1635692565" rel="stylesheet">



<script src="/js/jquery-3.3.1.min.js?1633872343"></script>
<script src="/js/jquery-3.3.1.min.js?1635692565"></script>

<style>
:root #header + #content > #left > #rlblock_left{
Expand Down Expand Up @@ -58,14 +58,14 @@ <h2>Mirabelle</p>
<span data-search-clear=""><i class="fas fa-times"></i></span>
</div>

<script type="text/javascript" src="/js/lunr.min.js?1633872343"></script>
<script type="text/javascript" src="/js/auto-complete.js?1633872343"></script>
<script type="text/javascript" src="/js/lunr.min.js?1635692565"></script>
<script type="text/javascript" src="/js/auto-complete.js?1635692565"></script>
<script type="text/javascript">

var baseurl = "https:\/\/www.mirabelle.mcorbin.fr";

</script>
<script type="text/javascript" src="/js/search.js?1633872343"></script>
<script type="text/javascript" src="/js/search.js?1635692565"></script>


</div>
Expand Down Expand Up @@ -611,7 +611,10 @@ <h3 id="add-a-stream">Add a stream</h3>
</ul>
<p>The body is a json string like <code>{&quot;config&quot;: &quot;&lt;stream-config&gt;&quot;}</code>.</p>
<p><!-- raw HTML omitted --> should be the stream EDN configuration as base64. For example <code>{:actions {:action :info}}</code> as base64 would be <code>ezphY3Rpb25zIHs6YWN0aW9uIDppbmZvfX0=</code>.</p>
<p>You can also pass the <code>:default</code> option, like in the configuration, in order to specify if you want your stream to be used by default.</p>
<p>You can pass the <code>:default</code> option, like in the configuration, in order to specify if you want your stream to be used by default.</p>
<p>You can also set the <code>:persist</code> field to <code>true</code> in your stream configuration in order to persist it on disk.
In that case, Mirabelle will write the stream configuration into the <strong>first</strong> directory referenced into the <code>:stream {:directory ..;}</code> configuration. The file name will be <code>&lt;stream-name&gt;.edn</code>.</p>
<p>Deleting a stream through the API will also delete its file on the filesystem.</p>
<hr>
<pre><code>curl -H &quot;Content-Type: application/json&quot; -X POST --data '{&quot;config&quot;: &quot;ezphY3Rpb25zIHs6YWN0aW9uIDppbmZvfX0=&quot;}' 127.0.0.1:5558/api/v1/stream/foobar

Expand All @@ -637,6 +640,7 @@ <h3 id="add-a-stream">Add a stream</h3>
<li><strong>DELETE</strong> <code>/api/v1/stream/&lt;stream-name&gt;</code></li>
</ul>
<p>Delete a stream by name.</p>
<p>If the stream was created using the API (see <code>Add a stream</code>) and persisted on disk, this call will also deleted its configuration file.</p>
<hr>
<pre><code>curl -X DELETE localhost:5558/api/v1/stream/trololo

Expand Down Expand Up @@ -989,19 +993,19 @@ <h3 id="add-a-stream">Add a stream</h3>
<div style="left: -1000px; overflow: scroll; position: absolute; top: -1000px; border: none; box-sizing: content-box; height: 200px; margin: 0px; padding: 0px; width: 200px;">
<div style="border: none; box-sizing: content-box; height: 200px; margin: 0px; padding: 0px; width: 200px;"></div>
</div>
<script src="/js/clipboard.min.js?1633872343"></script>
<script src="/js/perfect-scrollbar.min.js?1633872343"></script>
<script src="/js/perfect-scrollbar.jquery.min.js?1633872343"></script>
<script src="/js/jquery.sticky.js?1633872343"></script>
<script src="/js/featherlight.min.js?1633872343"></script>
<script src="/js/highlight.pack.js?1633872343"></script>
<script src="/js/clipboard.min.js?1635692565"></script>
<script src="/js/perfect-scrollbar.min.js?1635692565"></script>
<script src="/js/perfect-scrollbar.jquery.min.js?1635692565"></script>
<script src="/js/jquery.sticky.js?1635692565"></script>
<script src="/js/featherlight.min.js?1635692565"></script>
<script src="/js/highlight.pack.js?1635692565"></script>
<script>hljs.initHighlightingOnLoad();</script>
<script src="/js/modernizr.custom-3.6.0.js?1633872343"></script>
<script src="/js/learn.js?1633872343"></script>
<script src="/js/hugo-learn.js?1633872343"></script>
<script src="/js/modernizr.custom-3.6.0.js?1635692565"></script>
<script src="/js/learn.js?1635692565"></script>
<script src="/js/hugo-learn.js?1635692565"></script>


<script src="/mermaid/mermaid.js?1633872343"></script>
<script src="/mermaid/mermaid.js?1635692565"></script>

<script>
mermaid.initialize({ startOnLoad: true });
Expand Down
48 changes: 24 additions & 24 deletions site/mirabelle/public/categories/index.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -12,20 +12,20 @@
<title>Categories :: Mirabelle</title>


<link href="/css/nucleus.css?1633872343" rel="stylesheet">
<link href="/css/fontawesome-all.min.css?1633872343" rel="stylesheet">
<link href="/css/hybrid.css?1633872343" rel="stylesheet">
<link href="/css/featherlight.min.css?1633872343" rel="stylesheet">
<link href="/css/perfect-scrollbar.min.css?1633872343" rel="stylesheet">
<link href="/css/auto-complete.css?1633872343" rel="stylesheet">
<link href="/css/atom-one-dark-reasonable.css?1633872343" rel="stylesheet">
<link href="/css/theme.css?1633872343" rel="stylesheet">
<link href="/css/tabs.css?1633872343" rel="stylesheet">
<link href="/css/hugo-theme.css?1633872343" rel="stylesheet">
<link href="/css/nucleus.css?1635692565" rel="stylesheet">
<link href="/css/fontawesome-all.min.css?1635692565" rel="stylesheet">
<link href="/css/hybrid.css?1635692565" rel="stylesheet">
<link href="/css/featherlight.min.css?1635692565" rel="stylesheet">
<link href="/css/perfect-scrollbar.min.css?1635692565" rel="stylesheet">
<link href="/css/auto-complete.css?1635692565" rel="stylesheet">
<link href="/css/atom-one-dark-reasonable.css?1635692565" rel="stylesheet">
<link href="/css/theme.css?1635692565" rel="stylesheet">
<link href="/css/tabs.css?1635692565" rel="stylesheet">
<link href="/css/hugo-theme.css?1635692565" rel="stylesheet">



<script src="/js/jquery-3.3.1.min.js?1633872343"></script>
<script src="/js/jquery-3.3.1.min.js?1635692565"></script>

<style>
:root #header + #content > #left > #rlblock_left{
Expand Down Expand Up @@ -58,14 +58,14 @@ <h2>Mirabelle</p>
<span data-search-clear=""><i class="fas fa-times"></i></span>
</div>

<script type="text/javascript" src="/js/lunr.min.js?1633872343"></script>
<script type="text/javascript" src="/js/auto-complete.js?1633872343"></script>
<script type="text/javascript" src="/js/lunr.min.js?1635692565"></script>
<script type="text/javascript" src="/js/auto-complete.js?1635692565"></script>
<script type="text/javascript">

var baseurl = "https:\/\/www.mirabelle.mcorbin.fr";

</script>
<script type="text/javascript" src="/js/search.js?1633872343"></script>
<script type="text/javascript" src="/js/search.js?1635692565"></script>


</div>
Expand Down Expand Up @@ -895,19 +895,19 @@ <h1>
<div style="left: -1000px; overflow: scroll; position: absolute; top: -1000px; border: none; box-sizing: content-box; height: 200px; margin: 0px; padding: 0px; width: 200px;">
<div style="border: none; box-sizing: content-box; height: 200px; margin: 0px; padding: 0px; width: 200px;"></div>
</div>
<script src="/js/clipboard.min.js?1633872343"></script>
<script src="/js/perfect-scrollbar.min.js?1633872343"></script>
<script src="/js/perfect-scrollbar.jquery.min.js?1633872343"></script>
<script src="/js/jquery.sticky.js?1633872343"></script>
<script src="/js/featherlight.min.js?1633872343"></script>
<script src="/js/highlight.pack.js?1633872343"></script>
<script src="/js/clipboard.min.js?1635692565"></script>
<script src="/js/perfect-scrollbar.min.js?1635692565"></script>
<script src="/js/perfect-scrollbar.jquery.min.js?1635692565"></script>
<script src="/js/jquery.sticky.js?1635692565"></script>
<script src="/js/featherlight.min.js?1635692565"></script>
<script src="/js/highlight.pack.js?1635692565"></script>
<script>hljs.initHighlightingOnLoad();</script>
<script src="/js/modernizr.custom-3.6.0.js?1633872343"></script>
<script src="/js/learn.js?1633872343"></script>
<script src="/js/hugo-learn.js?1633872343"></script>
<script src="/js/modernizr.custom-3.6.0.js?1635692565"></script>
<script src="/js/learn.js?1635692565"></script>
<script src="/js/hugo-learn.js?1635692565"></script>


<script src="/mermaid/mermaid.js?1633872343"></script>
<script src="/mermaid/mermaid.js?1635692565"></script>

<script>
mermaid.initialize({ startOnLoad: true });
Expand Down
Loading

0 comments on commit 24b494b

Please sign in to comment.