Skip to content

Commit

Permalink
add trigger docs + updates
Browse files Browse the repository at this point in the history
  • Loading branch information
ctrl-alt-lulz committed Jan 30, 2024
1 parent f245edc commit 4255995
Show file tree
Hide file tree
Showing 14 changed files with 397 additions and 9 deletions.
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
{
"label": "Actions",
"position": 4,
"link": {
"type": "generated-index",
"description": "How to build a manage mockingbird triggered actions."
}
}
78 changes: 78 additions & 0 deletions apps/docusaurus/zeusfyi/docs/mockingbird/actions/actions.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
---
sidebar_position: 2
displayed_sidebar: mockingbird
---

# API Approvals

## Overview

The Actions tab is an essential feature for overseeing and managing API trigger calls within your workflow.
It serves as a control panel where users can approve or reject pending actions, as well as view the historical
results and responses of past actions. This ensures that all external communications via API calls are vetted by human
oversight.

## Approving and Rejecting API Triggers

### Action Approval Process

#### Review Request Summary:

Examine the details of the API call request, including the Workflow Result ID, Approval ID, and the Request Summary.

#### Approve or Reject:

Make an informed decision to approve or reject the trigger action. Approving an action will execute the API call, while
rejecting it will prevent the call from being made.

### Update Status:

Once an action is approved or rejected, its status is updated to reflect the new state.

## Action Buttons

### Approve:

Click this button to authorize the API call.

### Reject:

Click this button to decline the API call.

## Historical Actions

### Viewing Past Actions

### Access Historical Data:

The lower section of the Actions tab displays a history of all API trigger actions that have been taken.

Analyse Past Decisions: Review the Approval ID, Request Summary, Final State, Updated At, and the full list of requests
and responses to understand the outcome of past decisions.

### Historical Insights

Gain insights into the decision-making process over time.
Assess the consistency and accuracy of past approvals or rejections.

### Example of the Actions Tab

Trigger Name: social-media
Trigger Group: social-media-approvals
Trigger Env: social-media-engagement

Action Details:

- Workflow Result ID: 170665375241213000
- Approval ID: 1706653756335363475
- Approval State: Pending
- Request Summary: Requesting approval for trigger action [...]

Historical Actions:

- Approval ID: 170664805335778953
- Request Summary: Finished approval for trigger action
- Final State: Finished
- Updated At: 1/30/2024, 10:00:05 PM

![Scre](https://github.com/zeus-fyi/zeus/assets/17446735/c1451545-4ea7-4c32-9415-01d079b43b39)
8 changes: 8 additions & 0 deletions apps/docusaurus/zeusfyi/docs/mockingbird/api/_category_.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
{
"label": "API",
"position": 5,
"link": {
"type": "generated-index",
"description": "How to use the mockingbird API."
}
}
9 changes: 9 additions & 0 deletions apps/docusaurus/zeusfyi/docs/mockingbird/api/api.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
---
sidebar_position: 2
displayed_sidebar: mockingbird
---

# Docs

### Coming Soon (Expected Mid-Feb 2024)

34 changes: 34 additions & 0 deletions apps/docusaurus/zeusfyi/docs/mockingbird/builder/aggregations.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,4 +9,38 @@ Use this to tell the AI how to aggregate the results of your analysis chunks int
aggregating on a single analysis, the aggregation cycle count sets how many base analysis cycles to aggregate on. If
aggregating on multiple analysis, it will aggregate whenever the the underlying analysis is run.

#### Diagram: Analysis->Aggregation Flow

```mermaid
flowchart TD
A[Analysis Stage] -->|Completion Choices| B[Aggregation Stage]
B --> C{Aggregation Decision}
C -->|Single Analysis| D[Aggregate Once]
C -->|Multiple Analyses| E[Wait for All Analyses]
E --> F[All Completed]
F --> G[Time Window Check]
G -->|Within Window| H[Aggregate Results]
```

## Text Aggregations

![gg](https://github.com/zeus-fyi/zeus/assets/17446735/7fa3a1f2-cf92-4442-8b7f-50841dca7f02)

## JSON Aggregations

![ScreenshotPM](https://github.com/zeus-fyi/zeus/assets/17446735/e7d816a2-439e-40e9-b732-4dacfe27e09d)

### Example Run: Analysis->Aggregation Flows

This shows an analysis stage feeding the aggregation stage. Notice how the Completion Choices output from the
analysis stage are then used in the aggregation stage as the prompt body, which is how the aggregation stages work.
By default it will use the last analysis stage in the workflow, relative to the aggregation stage, and
for multiple analysis stages it will aggregate once every analysis stage has completed at least once
and then aggregate all the results from the analysis stages that are within the time window since last aggregation.

### Start::End SQL Search Window

The start and end fields in the report are the generated SQL window query that the AI uses to aggregate the results
when fetching from a time series indexed datastore.

![Screens](https://github.com/zeus-fyi/zeus/assets/17446735/da52e86d-d24b-4ee0-9f49-37e2c3b6e7a8)
14 changes: 13 additions & 1 deletion apps/docusaurus/zeusfyi/docs/mockingbird/builder/analysis.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,4 +11,16 @@ pieces and aggregate them into a final analysis result. Truncate will simply tru
length it can support. If you set the max tokens field greater than 0, it becomes the maximum number of tokens to spend
per task request.

![zzz](https://github.com/zeus-fyi/zeus/assets/17446735/8340c438-5b19-4684-9b19-1cb0c973a5fe)
![zzz](https://github.com/zeus-fyi/zeus/assets/17446735/8340c438-5b19-4684-9b19-1cb0c973a5fe)

![Screens](https://github.com/zeus-fyi/zeus/assets/17446735/da52e86d-d24b-4ee0-9f49-37e2c3b6e7a8)

### Example Run: Analysis->Aggregation Flows

This shows an analysis stage feeding the aggregation stage. Notice how the Completion Choices output from the
analysis stage are then used in the aggregation stage as the prompt body, which is how the aggregation stages work.
By default it will use the last analysis stage in the workflow, relative to the aggregation stage, and
for multiple analysis stages it will aggregate once every analysis stage has completed at least once
and then aggregate all the results from the analysis stages that are within the time window since last aggregation.

![Screens](https://github.com/zeus-fyi/zeus/assets/17446735/da52e86d-d24b-4ee0-9f49-37e2c3b6e7a8)
63 changes: 63 additions & 0 deletions apps/docusaurus/zeusfyi/docs/mockingbird/builder/intro.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,3 +9,66 @@ The builder section allows you to build tasks (analysis, aggregation), retrieval
then mix and match them to quickly create new workflows, which is ideal for rapid prototyping and experimentation.
We tried to keep things as simple as possible to use via the UI to make it as accessible as possible. Let us know if
anything is confusing or you have any questions.

## Key Components of the Workflow Builder

### Retrievals

Purpose: This stage is responsible for the initial data gathering. Users define the retrieval by naming it and
specifying the group and platform it associates with.
Functionality: Set up multiple retrieval stages to collect data from different sources or services like databases, APIs,
or social media platforms.

### Analysis

Function: Once data is retrieved, the analysis stage interprets and processes the information using selected AI models.
Configuration: Assign a name, group, and the AI model to be used for analysis, along with setting the analysis cycle
count, determining how frequently the analysis should occur.
Aggregation

### Aggregation

Aggregation stages combine the results from multiple analysis cycles into a cohesive summary.
Execution Logic: Define aggregation cycles relative to analysis cycles, ensuring aggregation happens after all necessary
analyses are completed.
Automated Evals
Purpose: Automated evals are used to evaluate the results of analyses or aggregations against a set of criteria to
validate or trigger further actions.
Cycles: Configure eval cycles in tandem with analysis cycles to ensure evaluations are timely and relevant.

### Fundamental Time Period

Definition: This is the base time period against which all cycles are referenced, determining the workflow's execution
intervals.
Application: Set the fundamental time step and unit to establish how often the workflow runs, which can range from
minutes to hours, depending on the needs of the task.

## Workflow Builder View

![Screen](https://github.com/zeus-fyi/zeus/assets/17446735/15ad98ff-ed0c-482c-9ced-9f4c08f222f7)

## Creating a Workflow

Define Workflow Name and Group: Start by giving your workflow a distinctive name and group for organizational purposes.

### Add Stages

Sequentially add retrieval, analysis, aggregation, and eval stages, specifying each stage's parameters and settings.

### Connect Stages

Link retrieval stages to analysis, ensuring data flows correctly through the workflow.

### Set Time Periods

Adjust the fundamental time period to control the overall pacing of the workflow execution.

### Save Workflow

Once all stages are correctly configured, save the workflow to activate it within the system.

## Example Usage

Consider a scenario where you are monitoring social media engagement. You would set up retrieval stages to pull data
from social media APIs, analyze the sentiment and relevance using an AI model, aggregate the analysis results to track
engagement trends over time, and evaluate these trends to decide on marketing strategies.
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
sidebar_position: 7
sidebar_position: 8
displayed_sidebar: mockingbird
---

Expand Down
10 changes: 5 additions & 5 deletions apps/docusaurus/zeusfyi/docs/mockingbird/builder/retrievals.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,18 +3,18 @@ sidebar_position: 5
displayed_sidebar: mockingbird
---

# Retrievals
# Inputs/Outputs

For generic retrievals we recommend you setup an API that can be called from the load balancer group for quick starting.

## Web APIs via Load Balancer
## APIs via Load Balancer

It will get data from each route in the load balancer group you reference here and then aggregate the results into a
single response.

![Sc](https://github.com/zeus-fyi/zeus/assets/17446735/6c9f79e8-4ff7-4fc3-bc57-f4d425dbff46)
![ScreensM](https://github.com/zeus-fyi/zeus/assets/17446735/f8011006-e92c-413a-9267-ea69165b91e9)

The website extraction and keywords feature is still in development, but coming soon you'll be able to use it, but you
can
save instructions for now and it will be available in the future for you to update your retrievals.
can save instructions for now and it will be available in the future for you to update your retrievals.

See the RPC Load Balancer Documentation for adding routes, and creating routing groups that you can use here.
66 changes: 66 additions & 0 deletions apps/docusaurus/zeusfyi/docs/mockingbird/builder/triggers.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
---
sidebar_position: 7
displayed_sidebar: mockingbird
---

# Triggers and Actions

## Intro

The Triggers and Actions system is a critical component of our automation framework.
It links evaluation functions to API triggers to streamline the execution of AI-driven workflows.
Upon evaluation, this system generates a request payload for user approval or rejection.
If approved, the corresponding API call is executed. Rejected requests are archived and not executed.
This mechanism ensures that automated decisions are overseen and validated by human judgment when necessary.

![Scr](https://github.com/zeus-fyi/zeus/assets/17446735/893cb682-b82d-4cbc-9cdf-742e41f7142b)

## Trigger Action Procedures

![ScreensM](https://github.com/zeus-fyi/zeus/assets/17446735/f8011006-e92c-413a-9267-ea69165b91e9)

### Setting Up Triggers

- **Define a New Trigger**: Assign a name and group to a new trigger, identify the trigger source, and specify the
trigger action.
- **Link to Eval Function**: Connect the trigger to an eval function that will analyze AI outputs to decide on API call
initiation.

### API Trigger Settings

- **Retry Mechanisms**: Configure the maximum retry attempts for API calls and set up backoff coefficients for retry
intervals.
- **Save Trigger**: Save the configured trigger so it can start listening for outputs from the linked eval function.

## Evaluation Functions and API Calls

![ScreensM](https://github.com/zeus-fyi/zeus/assets/17446735/f8011006-e92c-413a-9267-ea69165b91e9)

### Connecting Eval Functions

- **Select the Eval Function**: Pick the appropriate evaluation function that will provide criteria for API call
triggers.

### Automated Payload Creation

- **Payload Generation**: Once the eval function criteria are met, the system automatically generates an API call
request payload.

## Actions Menu

![Scree](https://github.com/zeus-fyi/zeus/assets/17446735/8f340b39-5c42-4813-9ca2-7e94334cafdf)

### Review and Decision

- **Approve/Reject Payloads**: Use the actions menu to review generated payloads and make decisions to approve or reject
them.

### Historical Tracking

- **Action Records**: The system maintains a log of all actions taken, which can be accessed for review or auditing
purposes.

## Conclusion

This triggers and actions framework empowers users to have final control over automated processes, ensuring that every
action taken by the AI is deliberate and in line with organizational standards.
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"label": "Runs",
"position": 2,
"position": 3,
"link": {
"type": "generated-index",
"description": "How to manage mockingbird workflow runs."
Expand Down
17 changes: 16 additions & 1 deletion apps/docusaurus/zeusfyi/docs/mockingbird/runs/reports.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
sidebar_position: 2
sidebar_position: 3
displayed_sidebar: mockingbird
---

Expand All @@ -9,3 +9,18 @@ Shows the outputs of each stage within the workflow. You can click on the expand
mockingbird run.

![ddd](https://github.com/zeus-fyi/zeus/assets/17446735/c77d77c4-0023-47f1-8196-66fdd93eeefc)

### Example Run: Eval Results

![ScreenshotPM](https://github.com/zeus-fyi/zeus/assets/17446735/648dbda2-ce22-4eea-b698-5c012844d190)

### Example Run: Analysis->Aggregation Flows

This shows an analysis stage feeding the aggregation stage. Notice how the Completion Choices output from the
analysis stage are then used in the aggregation stage as the prompt body, which is how the aggregation stages work.
By default it will use the last analysis stage in the workflow, relative to the aggregation stage, and
for multiple analysis stages it will aggregate once every analysis stage has completed at least once
and then aggregate all the results from the analysis stages that are within the time window since last aggregation.

![Screens](https://github.com/zeus-fyi/zeus/assets/17446735/da52e86d-d24b-4ee0-9f49-37e2c3b6e7a8)

Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
{
"label": "Setup",
"position": 1,
"link": {
"type": "generated-index",
"description": "How to manage mockingbird setup options."
}
}
Loading

0 comments on commit 4255995

Please sign in to comment.