Oms web service

What is openM++ web-service

OpenM++ web-service (oms) is a JSON web-service written in Go and used from openM++ UI JavaScript. Today most of popular development platforms (.NET, Java, Python, Perl, R, JavaScript, etc.) with only few lines of code allow to create HTTP client and send-receive JSON data. That makes integration with openM++ very easy.

How to start openM++ web-service

OpenM++ web-service does not required any installation. It can be run with default settings from command-line prompt.

To start openM++ web-service on Windows:

• download and unzip openM++ https://github.com/openmpp/main/releases/latest binaries into C:\SomeDir\
• run oms from command-line:

C:
cd \SomeDir\openmpp_win_20190508\
bin\oms.exe

2022-09-14 15:51:30.477 Models directory:    models\bin
2022-09-14 15:51:30.565 HTML UI directory:   html
2022-09-14 15:51:30.567 Etc directory:       etc
2022-09-14 15:51:30.567 Oms instance name:   localhost_4040
2022-09-14 15:51:30.574 Listen at localhost:4040
2022-09-14 15:51:30.574 To start open in your browser: http://localhost:4040
2022-09-14 15:51:30.574 To finish press Ctrl+C

OpenM++ UI is a client of oms web-service, after above command you can open UI in browser at http://localhost:4040

To start openM++ web-service on Linux:

• download and unpack openM++, i.e.:

wget https://github.com/openmpp/main/releases/download/v1.2.0/openmpp_debian_20190508.tar.gz
tar xzf openmpp_debian_20190508.tar.gz

• run oms executable:

cd openmpp_debian_20190508/
bin/oms

2022-09-14 15:51:30.477 Models directory:    models/bin
2022-09-14 15:51:30.565 HTML UI directory:   html
2022-09-14 15:51:30.567 Etc directory:       etc
2022-09-14 15:51:30.567 Oms instance name:   localhost_4040
2022-09-14 15:51:30.574 Listen at localhost:4040
2022-09-14 15:51:30.574 To start open in your browser: http://localhost:4040
2022-09-14 15:51:30.574 To finish press Ctrl+C

Note: We recommend to use normal Windows command line cmd.exe. If you are using Windows PowerShell then it may be necessary to put "quotes" around command line options, e.g:

oms.exe "-oms.ApiOnly"

Oms as "pure" web-service vs "full" web-UI

By default oms.exe started in "full" web-UI mode. That means it handles web-service requests and web-UI content from ./html sub-directory. If you want only "pure" web-service mode without UI then use:

oms -oms.ApiOnly

How to use oms: arguments of web-service methods

Following arguments most often used in web-service methods:

:model - model digest or model name

Example of method:

GET /api/model/:model

Call example:

http://localhost:4040/api/model/f5024ac32c4e8abfc696a0f925141c95
http://localhost:4040/api/model/modelOne

Model can be identified by digest or by model name. It is recommended to use digest because it is uniquely identifies model. It is possible to use model name, which is more human readable than digest, but if there are multiple models with same name in database than result is undefined.

:run - model run or model task run

Example of method:

GET /api/model/:model/run/:run/status
GET /api/model/:model/task/:task/run-status/run/:run

Call example:

http://localhost:4040/api/model/modelOne/run/modelOne_first_run/status
http://localhost:4040/api/model/modelOne/run/d06f4a0a45a9514c22593025e489f933/status
http://localhost:4040/api/model/modelOne/task/taskOne/run-status/run/First Task Run

This argument is used to identify model run or modeling task run.

Modeling task run can be identified by task run stamp or task run name.

Model run can be identified by run digest, run stamp or run name. It is recommended to use run digest because it is uniquely identifies model run. Run stamp can be explicitly specified as command line option when you run the model. If run stamp not specified then it is automatically generated as timestamp string, ex.: 2016_08_17_21_07_55_123. It is also possible to use run name, which is more human readable than digest, but if there are multiple runs with same name in database than result is undefined.

:lang - language code

Example of method:

GET /api/model/:model/text/lang/:lang

Call example:

http://localhost:4040/api/model/modelOne/text/lang/EN
http://localhost:4040/api/model/modelOne/text/lang/en_US

Language code can be a model language (ex.: EN, FR) or any MIME language (see BCP47 or RFC3282). If no language explicitly specified then Accept-Language header is used (supplied by browser).

Result returned in best matched language supported by model. For example for en_US result is model language EN, if model supported EN language. If no such language then result is in default model language or can be empty.

:set - set of input data (a.k.a. workset)

Method examples:

GET  /api/model/:model/workset/:set/status
POST /api/model/:model/workset/:set/readonly/:val

Call examples:

http://localhost:4040/api/model/modelOne/workset/modelOne_set/status
curl -v -X POST http://localhost:4040/api/model/modelOne/workset/modelOne_set/readonly/1

Workset is a set of model input parameters (a.k.a. "scenario" input) and it used to run the model Each model workset uniquely identified by name.

:task - modelling task

Method examples:

GET /api/model/:model/task/:task/text/lang=FR

Call examples:

http://localhost:4040/api/model/modelOne/task/taskOne/text
curl -v http://localhost:4040/api/model/modelOne/task/taskOne/text/lang=fr_CA

Modelling task consists of multiple input data sets (a.k.a. worksets or scenarios in Modgen). Task can be used to run the model in batch mode.

Results of web-service methods

Run status

Model run status and task run status may contain one of the following values:

 i = initial state, not running yet
 p = run in progress
 w = task wait for additional input
 s = completed successfully
 x = completed by exit (reserved fro internal use)
 e = completed with error

Important: if model run failed with exception (e.g. database write exception) then status may not be updated and still p=in progress.

Oms web-service configuration

Oms default configuration options can be overwritten by command-line arguments or ini-file. For example:

• listen from any host on port 7070:

oms -l :7070

• serve only API calls and not html for openM++ UI:

oms -oms.ApiOnly

• listen from localhost port 4044 only and read more oms run options from oms.ini file:

oms -l localhost:4044 -ini oms.ini

• models directory relative path is: ../some/dir

oms -oms.ModelDir ../some/dir

• typical log settings for remote server:
  • log user request
  • log into the file instead of cosole by default
  • log files rotation: create new log file every day

oms -l localhost:4044 -oms.LogRequest -OpenM.LogToConsole false -OpenM.LogToFile -OpenM.LogUseDailyStamp

• typical settings for model user in cloud:
  • allow user home directory with downloads and uploads
  • use model run jobs to manage back-end computational servers resources

oms -l localhost:4044 -oms.HomeDir models/home -oms.AllowDownload -oms.AllowUpload -oms.JobDir job

It is recommended to use oms.ini file to avoid long command lines, especially for cloud environment where you may want to combine log options and user options from two examples above.

Get and use oms web-service configuration

Clients of oms web-service can retrieve configuration by calling GET web-service configuration or simply by open http://localhost:4040/api/service/config in the browser. Response to that call may also contain client environment variables which names started from OM_CFG_ prefix (oms web-service does not use any of OM_CFG_ environment variables, it only passes it to clients).

For example openM++ UI uses following server variables:

OM_CFG_LOGIN_URL=/public/login_required.html
OM_CFG_LOGOUT_URL=/login?logout=true
OM_CFG_DEFAULT_RUN_TMPL=run.Win32.Debug.template.txt
OM_CFG_INI_ALLOW=true
OM_CFG_INI_ANY_KEY=true
OM_CFG_TYPE_MAX_LEN=366

OpenM++ UI is using above variables as follow:

• OM_CFG_LOGIN_URL : display user login button linked to the URL
• OM_CFG_LOGOUT_URL : display user logout button linked to the URL
• OM_CFG_DEFAULT_RUN_TMPL : use this template to run the model, e.g.: to debug from IDE
• OM_CFG_INI_ALLOW : allow user to run the model with ini-file, e.g.: RiskPaths.ini
• OM_CFG_INI_ANY_KEY : allow to use model development options from ini-file
• OM_CFG_TYPE_MAX_LEN : type size limit, if dimension type exceed this size then UI would not show parameter or output table with that dimension

Note: Model ini-files and model development options described at: Model Run Options and ini file.

Oms run options

Following options supported by oms:

-oms.Listen:         address to listen, default: localhost:4040
-l:                  address to listen (short form of -oms.Listen)
-OpenM.IniFile:      path to ini-file
-ini ini-file:       path to ini-file (short of OpenM.IniFile)
-oms.ApiOnly:        if true then API only web-service, no web UI
-oms.RootDir:        oms root directory, default: current directory
-oms.ModelDir:       models directory, if relative then must be relative to oms root directory, default: models/bin
-oms.ModelLogDir:    models log directory, if relative then must be relative to oms root directory: default: models/log
-oms.ModelDocDir:    models documentation directory, if relative then must be relative to oms root directory: default: models/doc
-oms.HomeDir:        user personal home directory, if relative then must be relative to oms root directory
-oms.AllowDownload:  if true then allow download from user home/io/download directory
-oms.AllowUpload:    if true then allow upload to user home/io/upload directory
-oms.FilesDir:       user files directory, if relative then must be relative to oms root directory, if home directory specified then it is: home/io
-oms.AllowMicrodata: if true then allow model run microdata
-oms.HtmlDi:         front-end UI directory, if relative then must be relative to oms root directory, default: html
-oms.EtcDir:         configuration files directory, if relative then must be relative to oms root directory, default: etc
-oms.JobDir:         model run jobs directory, if relative then must be relative to oms root directory
-oms.Name:           oms instance name, used model run by jobs, automatically generated if empty
-oms.UrlSaveTo:      file path to save oms URL in form of: http://localhost:4040, if relative then must be relative to oms root directory
-oms.Languages:      comma-separated list of supported languages, default: en
-oms.CodePage:       code page to convert source file into utf-8, e.g.: windows-1252
-oms.DoubleFormat:   format to convert float or double value to string, default: %.15g
-oms.AdminAll        if true then allow global administrative routes: /admin-all/
-oms.NoAdmin         if true then disable local administrative routes: /admin/
-oms.NoShutdown      if true then disable shutdown route: /shutdown/

-OpenM.LogToConsole:     if true then log to standard output (default true)
-v:                      if true then log to standard output (short of OpenM.LogToConsole)
-OpenM.LogToFile:        if true then log to file
-OpenM.LogFilePath:      path to log file, default = current/dir/oms.log
-OpenM.LogUseDailyStamp: if true then use daily-stamp in log file name
-OpenM.LogUsePidStamp:   if true then use process id stamp in log file name
-OpenM.LogUseTimeStamp:  if true then use time-stamp in log file name
-OpenM.LogSql:           if true then log sql statements into log file
-oms.LogRequest:         if true then log HTTP requests

There are many common options, e.g.: -OpenM.LogToFile which can be used with any openM++ executable: models, compiler, dbcopy and oms.

Example of oms.ini

; This is a comment
# This is also a comment

; Ini file can be supplied to oms.exe as command line option "-ini" or "-OpenM.IniFile"
; "-ini" is a short form of "-OpenM.IniFile", command lines below are equal:
;
# oms.exe -ini           path/to/oms.ini
# oms.exe -OpenM.IniFile path/to/oms.ini

; "-l"   is a short form of "-oms.Listen", command lines below are equal:
;
# oms.exe -l          localhost:4040
# oms.exe -oms.Listen localhost:4040

; boolean options can be "true" or "false" or empty value
; boolean empty value is the same as "true"
; for example both command lines below are equal:
;
# oms -oms.ApiOnly
# oms -oms.ApiOnly true

[oms]
;
; Listen         = localhost:4040 # address to listen, default: localhost:4040
; RootDir        =                # oms "root" directory, expected to have log subfolder
; ModelDir       = models/bin     # models executable and model.sqlite directory, if relative then must be relative to oms root directory
; ModelLogDir    = models/log     # models log directory, if relative then must be relative to oms root directory
; ModelDocDir    = models/doc     # models documentation directory, default: models/doc, if relative then must be relative to oms root directory
; HomeDir        = models/home    # user personal home directory, if relative then must be relative to oms root directory
; AllowDownload  = false          # if true then allow download from user home sub-directory: home/io/download
; AllowUpload    = false          # if true then allow upload to user home sub-directory: home/io/upload
; FilesDir       =                # user files directory, if relative then must be relative to oms root directory, if home directory specified then it is: home/io
; AllowMicrodata = false          # if true then allow model run microdata
; UrlSaveTo      =                # file path to save oms URL, if relative then must be relative to oms root directory
; LogRequest     = false          # if true then log HTTP requests
; ApiOnly        = false          # if true then API only web-service, no web UI
; HtmlDir        = html           # front-end web UI directory, if relative then must be relative to oms root directory
; EtcDir         = etc            # configuration files directory, if relative then must be relative to oms root directory
; JobDir         =                # jobs control directory, if empty then jobs control disabled
; Name           =                # instance name, used for job control
; Languages      = en             # comma-separated list of supported languages
; CodePage       =                # code page to convert source file into utf-8, e.g.: windows-1252
; DoubleFormat   = %.15g          # format to convert float or double value to string, e.g. %.15g
; AdminAll       = false          # if true then allow global administrative routes: /admin-all/
; NoAdmin        = false          # if true then disable local administrative routes: /admin/
; NoShutdown     = false          # if true then disable shutdown route: /shutdown/

[OpenM]
;
; LogToConsole = true      # if true then log to standard output
; LogToFile = false        # if true then log to file
; LogFilePath = oms.log    # log file path, default = current/dir/exeName.log
; LogUseTimeStamp = false  # if true then use time-stamp in log file name
; LogUsePidStamp = false   # if true then use pid-stamp in log file name
; LogUseDailyStamp = false # if true then use daily-stamp in log file name
; LogSql = false           # if true then log sql statements into log file

; "-v" is a short form of "-OpenM.LogToConsole"

; log settings:
; log can be enabled/disabled for 3 independent streams:
;   console             - standard output
;   "current" log file  - log file with specified name, overwritten on every model run
;   "stamped" log file  - log file with unique name, created for every model run
;
; "stamped" name produced from "current" name by adding time-stamp and/or pid-stamp, i.e.:
;   oms.log => oms.2012_08_17_16_04_59_148.123456.log
#
; LogUseDailyStamp creates new log file every day
; by default LogUseDailyStamp:
;   = false if log file disabled (default)
;   = false if "stamped" log file enabled
;   = true  if log file enabled and "stamped" log file disabled

Oms directory structure: user home and jobs directories

Following directory structure expected by default:

./        -> oms "root" directory, by default it is current directory
    html/    -> web-UI directory with HTML, js, css, images...
    etc/     -> config files directory, contain template(s) to run models
    log/     -> recommended log files directory
    models/
          bin/  -> default model.exe and model.sqlite directory
          log/  -> default directory for models run log files
          doc/  -> models documentation directory

If you don't don't want web-UI or don't have html directory then start oms as:

oms -oms.ApiOnly

You can explicitly specify oms log files location, models and models log directory, e.g.:

oms -oms.ModelDir /my-models -oms.ModelLogDir /my-models-log -oms.ModelDocDir /my-models/doc2

If you want to use log file and no console messages:

oms -OpenM.LogToConsole=false -OpenM.LogToFile
oms -OpenM.LogToConsole=false -OpenM.LogFilePath log/oms.log

If you want to use "daily" log files:

oms -OpenM.LogUseDailyStamp -OpenM.LogToFile 
oms -OpenM.LogUseDailyStamp -OpenM.LogFilePath log/oms.log

User home directory

You can enable user home directory to store home directory for user personal settings, model ini-files, downloads of model model run results or upload input scenarios:

oms -oms.HomeDir models/home -oms.AllowDownload
oms -oms.HomeDir models/home -oms.AllowDownload -oms.AllowUpload

Important: Please keep in mind, if you allow users to download or upload files on your server then it may create a security risk.

Above command assume directory structure with home, download and upload sub-folders of models:

./        -> oms "root" directory, by default it is current directory
    html/    -> web-UI directory with HTML, js, css, images...
    etc/     -> config files directory, contain template(s) to run models
    log/     -> recommended log files directory
    models/
          bin/  -> default model.exe and model.sqlite directory
          log/  -> default directory for models run log files
          doc/  -> models documentation directory
          home/ -> user personal home directory
              io/download  -> user directory for download files
              io/upload    -> user directory to upload files

Note: openM++ dbcopy utility is required for download and upload, it must be located in the same directory where oms executable is.

Model run jobs directory structure

If you want to have model runs queue, or using openM++ in cloud and want automatically scale up and down cloud resources, e.g. start and stop virtual machines for model runs then start oms with job control option:

oms -oms.JobDir job

Following directory structure expected:

./        -> oms "root" directory, by default it is current directory
    html/    -> web-UI directory with HTML, js, css, images...
    etc/     -> config files directory, contain template(s) to run models
             disk.ini  -> (optional) disk usage control settings to set storage quotas
    log/     -> recommended log files directory
    models/
          bin/  -> default model.exe and model.sqlite directory
          log/  -> default directory for models run log files
          doc/  -> models documentation directory
          home/ -> user personal home directory
              io/download  -> user directory for download files
              io/upload    -> user directory to upload files
    job/  -> model run jobs control directory
          job.ini   -> (optional) job control settings to restrict storage size
          active/   -> active model run state files
          history/  -> model run history files
          past/     -> (optional) shadow copy of history folder, invisible to the end user
          queue/    -> model run queue files
          state/    -> jobs state and computational servers state files
               jobs.queue.paused      -> if such file exists then jobs queue is paused
               jobs.queue.all.paused  -> if such file exists then all jobs in all queues are paused

Please visit following page to find out how to use oms in cloud and manage model runs queue.