Tutorial 2. Hello World

The next two tutorials illustrate the process of building and running workflows with RestFlow. This tutorial begins with a simple "Hello World" example, and the next chapter extends this example to highlight key system capabilities.

Creating and running a 'Hello World' workflow

Copy the contents below into a new text file, and save the file with the name hello1.yaml.

imports:

- classpath:/common/groovy/actors.yaml
- classpath:/common/directors.yaml

components:

- id: HelloWorld
  type: Workflow
  properties:
    director: !ref MTDataDrivenDirector
    nodes:
    - !ref CreateGreeting
    - !ref RenderGreeting

- id: CreateGreeting
  type: Node
  properties:  
    actor: !ref ConstantSource
    constants:
      value: Hello World!
    outflows:
      value: /messages/greeting/
 
- id: RenderGreeting
  type: Node
  properties:
    actor: !ref PrintStreamWriter
    inflows:
      message: /messages/greeting/

To run the workflow, provide the path to hello1.yaml using the -f option to RestFlow. For example:

$ restflow -f hello1.yaml 
Hello World!
$ 

If you have not set up the restflow alias as described in Tutorial 1. Setting up Restflow, then the command will look more like this:

$ java -jar ~/bin/RestFlow-0.1.jar -f hello1.yaml
Hello World!
$

Understanding the workflow specification

The above workflow seems fairly complicated given its simple function. Starting in the next chapter, however, we will begin to increase the complexity of the workflow in very useful ways that are sometimes difficult to achieve in a traditional scripting language. We will also see ways to specify workflows more concisely. But first, it is important to understand what the various parts of this workflow specification mean.

The first thing to note is that unlike most scripts the workflow specification does not contain many verbs. A RestFlow specification is declarative, not procedural. It does not tell RestFlow what to do, but rather what to make. A workflow specification is like a parts list for a machine along with assembly instructions. Telling RestFlow to run a workflow is like telling a robot to (1) "Build the following machine from these components (assemble this workflow)", and then to (2) "Turn the machine on (run the workflow)."

Workflow components

The hello1.yaml file is split into two sections named imports and components, respectively. The parts list for the workflow is what follows the components heading. In this example, three components are listed, and each is described by a block of text. The components are identified by the first line of each block of text, the lines that begin with id. Note that the hello1 workflow comprises a HelloWorld component, a CreateGreeting component, and a RenderGreeting component.

The lines following the id of each block of text describes the characteristics of the component. These characteristics are provided in name:value pairs with the name of the characteristic on the left side of a colon, and the value of that characteristic on the right. Starting a value on the following line and indented two spaces indicates that a list of one or more values is associated with the characteristic. And the values of characteristics can be additional characteristics (i.e. characteristics can be nested) as indicated by indentation.

Consider the CreateGreeting component:

- id: CreateGreeting
  type: Node
  properties:  
    actor: !ref ConstantSource
    constants:
      value: Hello World!
    outflows:
      value: /messages/greeting/

This component has three chief characteristics, its id, its type, and its properties. The id distinguishes the component from the others and enables other components to refer specifically to it. (Note that in the hello1.yaml listing, CreateGreeting is listed as one of values for the _nodes _property of the HelloWorld component). A component's type indicates what kind of component it is. CreateGreeting is a Node. There can be any number of instances of any type of component in a workflow. Finally, CreateGreeting has a list of properties. These properties are characteristics specific to the type of component that CreateGreeting is. Because CreateGreeting is a Node, it has additional characteristics including actor, parameters, and outflows.

Now look at the HelloWorld component (the first block of text in the components section).

- id: HelloWorld
  type: Workflow
  properties:
    director: !ref MTDataDrivenDirector
    nodes:
    - !ref CreateGreeting
    - !ref RenderGreeting

This is an instance of the Workflow type of component. And indeed, this component represents the workflow as a whole. One of the properties of a Workflow component is a list of nodes. Workflow nodes represent the steps carried out by a workflow. Here, the two nodes comprising the HelloWorld workflow are the CreateGreeting and RenderGreeting components defined in the same file. The !ref operator indicates that the string of text following it is the id of another component. It allows one component to refer to another.

The overall meaning of the components section of the hello1 workflow specification should now be clear. RestFlow is meant to assemble a workflow, represented overall by the HelloWorld component. HelloWorld in turn comprises the CreateGreeting and RenderGreeting workflow nodes.

Inflows, outflows, and flow expressions

A workflow is more than a parts list. Besides the list of nodes, RestFlow needs to know how data is meant to flow between components when the workflow is run. Put another way, RestFlow must be given instructions specifying how to wire together the nodes to create a runnable workflow. The inflows and outflows properties of workflow nodes provides these wiring instructions.

Note that the CreateGreeting node has a _outflows _property, which in turn has an value property with the value /messages/greeting/. What this means is that when the workflow is run, the CreateGreeting component will produce data of some kind, and that it will places this data in an outward data flow identified by the flow expression /messages/greeting/. The precise meaning of flow expressions will be discussed in a later chapter. For now, the important thing to note is that the /messages/greeting/ outflow expression on CreateGreeting exactly matches the message inflow expression (listed under inflows) of the RenderGreeting component. Because the inflow expression of RenderGreeting matches that of the outflow of CreateGreeting, any data that CreateGreeting puts into the /messages/greeting flow during the workflow run subsequently will be provided as input to the RenderGreeting component.

Thus, matching flow expressions indicate to RestFlow how data is to flow between components when the workflow is run.

The imports section

So far we have only discussed the components section of the workflow specification. What about the imports section of the file? Imports are used to include descriptions of workflow components defined in other files. These then may be referred to from within the workflow specification (using the !ref operator). Components in hello1.yaml refer to the ConstantSource and PrintStreamWriter components defined in the /common/groovy/actors.yaml file, and to the MTDataDrivenDirector component provided in the /common/directors.yaml file. The classpath: qualifiers preceding the paths to the imported files indicate that the files are packaged within the RestFlow jar file. Actor and director components will be described in later chapters.

A run of Hello World

The following sequence of events occurs when RestFlow runs the hello1.yaml workflow:

1. RestFlow loads into the computer's memory executable code for each of the HelloWorld, CreateGreeting, and RenderGreeting workflow components and for the components they refer to.
2. RestFlow finds the component that is an instance of Workflow and tells it to run. In this case, this is the HelloWorld component.
3. RestFlow begins to trigger the components listed in the HelloWorld nodes property. By trigger, we mean that the component is given a chance to step, i.e. carry out whatever step it is designed to perform, including receiving data from other components via its inflows, computing new data, and sending received or newly computed data to other components via its outflows. Workflow nodes may be triggered repeatedly during a workflow run.
4. When CreateGreeting steps, it produces the value "Hello World!" and outputs it on the /messages/greeting/ outflow.
5. When RenderGreeting subsequently steps, it receives the value "Hello World!" from CreateGreeting on its inflow. RenderGreeting outputs this value to the terminal.
6. RestFlow detects that the CreateGreeting and RenderGreeting nodes have completed their operations for this run of the workflow.
7. RestFlow exits.

In Tutorial 3. Extended Example, we will expand on this example.