v7.2.0

Highlights

• Expedited clean shutdown for workflows that run many consequtive states.
• Add support for custom logic when nFlow engine thinks the workflow state processing may be stuck.
• Convert all exceptions to proper error response messages in all nFlow REST API endpoints. Applies to both nflow-rest-api-jax-rs and nflow-rest-api-spring-web.
• Support for Guice.

Details

• nflow-engine
  • When shutdown is requested, stop processing workflows immediately after the current state has been executed.
  • Add WorkflowExecutorLister.handlePotentiallyStuck(Duration processingTime) to support custom handling when nFlow engine thinks the workflow state processing may be stuck. If any registered listener implementation returns true from this method, nFlow will interrupt the processing thread. The default implementation returns false.
  • Throw IllegalArgumentException instead of IllegalStateException when trying to update workflow instance state to an invalid value.
  • Throw IllegalArugmentException instead of RuntimeException when trying to insert workflow instance with unknown type or with a state that is not a start state.
  • Make StateVariableTooLongException extend IllegalArgumentException instead of RuntimeException.
  • Fix SQL deadlocks in workflow instance polling for PostgreSQL with skip locked.
  • Add EngineEnvironmentModule and EngineModule for Guice support. Call NflowController.start() and NflowController.stop() to start and stop nFlow engine, as nflow.autostart and nflow.autoinit configuration options are not supported with Guice.
  • MySQL/MariaDB: use compressed table format for archives.
  • PostgreSQL/SQLServer: use partial (not null) indices for parent hierarchy.
  • PostgreSQL: tune fillfactors of tables and indices.
  • PostgreSQL: reorder table column order to minimize space lost on padding. Only affects new tables.
  • Drop (type, external_id, executor_group) unique constraint from nflow_archive_workflow table. Allows archiving workflow instances with same values multiple times. Only one instance is allowed in production table though.
  • Dependency updates:
    • spring 5.2.5
    • jackson 2.10.3
    • javassist 3.27.0
    • jetty 9.4.28
    • cxf 3.3.6
    • metrics 4.1.6
    • netty 0.9.6
    • swagger 1.6.1
    • lang3 3.10
    • mariadb 2.6.0
    • mssql 8.2.2
    • postgresql 42.2.10
    • junit 5.6.2
    • mockito 3.3.3
    • spotbugs 4.0.2
    • hibernate 6.1.4
    • commons-lang3 3.10
• nflow-rest-api-jax-rs and nflow-rest-api-spring-web
  • Convert IllegalArgumentException to HTTP 400 Bad Request with a valid JSON response body in all endpoints.
  • Convert NflowNotFoundException to HTTP 404 Not Found with a valid JSON response body in all endpoints.
  • Convert all other throwables to HTTP 500 Internal Server Error with a valid JSON response body in all endpoints.
• nflow-explorer
  • Dependency updates:
    • swagger-ui 2.2.10
    • es5-shim 4.5.14
    • npm 6.14.4
    • autoprefixer 9.7.5
    • grunt 1.1.0
    • grunt-contrib-imagemin 4.0.0

v7.1.0

Highlights

• nflow-engine
  • Add support for marking exceptions and workflow states as non-retryable. If processing the state method throws an exception, nFlow engine will check if it
    is allowed to retry. If not, the workflow instance will go directly to a failure/error state. See NoRetryWorkflow for example.

Details

• nflow-engine
  • Exceptions can be annotated with @NonRetryable to prevent retrying the state processing.
  • Add isRetryable(Throwable thrown) method to the WorkflowState interface with a default implementation that returns true when the thrown exception is not
    annotated with @NonRetryable.
  • Add default implementation for WorkflowState.getDescription() that returns the name of the state.

v7.0.0

BREAKING CHANGES

• Drop nflow_workflow.root_workflow_id column.
• Remove, add and update foreign key constraints and indices in nFlow database tables.
• ArchiveService and ArchiveResource (/v1/archive) are replaced by MaintenanceService and MaintenanceResource (/v1/maintenance).
• Change WorkflowSettings.setHistoryDeleteableAfterHours(Integer) to WorkflowSettings.setHistoryDeleteableAfter(ReadablePeriod) for more flexible configuration.
• StateExecution.getAllChildWorkflows does not return current state variables of child workflows anymore.
• WorkflowStateProcessor does not read child workflow IDs of processed workflow instances anymore.
• QueryWorkflowInstances.Builder does not query for current state variables by default anymore.
• BulkWorklow moved to io.nflow.engine.workflow.curated java package.

Highlights

• nflow-engine
  • Improve workflow instance archiving performance. Archiving has been in practice unusable in some scenarios.
  • Add support for deleting workflow instances, actions and state variables from production and archive tables.
  • Remove support for nflow.executor.fetchChildWorkflowIds configuration property.
  • Fix bug in stuck workflow instance processing detection when multiple states are processed consecutively by the same processor thread. Now the time counter is reset between each state.
  • Added abstract CronWorkflow that can be used to periodically execute a task.
  • Built-in support for periodically running maintenance to clean up workflow instances.

Details

• nflow-engine
  • nflow_workflow.root_workflow_id was only used in old archiving code, so it was removed as the new archiving logic does not need it anymore.
  • Removed unnecessary indices and foreign keys and added missing indices to improve nFlow database performance. See database update scripts for details.
  • Added name for all existing and new constraints in create scripts, if they did not have one yet. This is to make modify operations easier in future. All of these may not be covered in database update scripts.
  • See MaintenanceService and MaintenanceConfiguration for details on how to archive and delete workflow instances. The maintenance operations can now be limited by workflow type as well.
  • As ArchiveService is removed, the old functionality of ArchiveService.archiveWorkflows(DateTime olderThan, int batchSize) can now be achieved with
    MaintenanceService.cleanupWorkflows(new MaintenanceConfiguration.Builder() .setArchiveWorkflows(new ConfigurationItem.Builder() .setOlderThanPeriod(new Period(olderThan, DateTime.now()) .setBatchSize(batchSize) .build()) .build)
  • For example, WorkflowSettings.setHistoryDeleteableAfterHours(12) can now be achieved by WorkflowSettings.setHistoryDeleteableAfter(Period.hours(12)).
  • To get child workflows with state variables in a state method, use StateExecution.queryChildWorkflows.
  • To get all child workflow IDs in a state method, use StateExecution.getAllChildWorkflows.
  • New MaintenanceWorkflow that can be enabled by setting nflow.maintenance.insertWorkflowIfMissing property to true. On the first run the workflow is inserted and it will start to run periodically. Further configuration must be done by editing the two state variables of the workflow: cron for the scheduling and config for the MaintenanceConfiguration to execute.
    • The first time configuration can also be tuned with nflow.maintenance.initial.cron, nflow.maintenance.initial.delete.olderThan, nflow.maintenance.initial.archive.olderThan and nflow.maintenance.initial.deleteArchived.olderThan
  • Dependency updates:
    • spring 5.2.4
    • jersey 2.30.1
    • metrics 4.1.3
    • netty 0.9.5
    • mockito 3.3.0
    • spotbugs 4.0.0
• nflow-explorer
  • Dependency updates:
    • angular extra libraries 1.7.9
    • angular-ui-router 1.0.25
    • dagre-d3 0.6.4
    • node-sass 9.7.4
    • karma-jasmine 3.1.1
    • autoprefixer 9.7.4
    • node v12.16.1
    • npm 6.13.7

v6.2.0

Highlights

• nflow-engine
  • Improve child workflow final state execution speed by caching the parent workflow type. See configuration details below.
  • Add optional performance improvement to state execution. See configuration details below.
  • Optimize workflow instance polling code.

Details

• nflow-engine
  • Workflow type cache size can be set using nflow.db.workflowInstanceType.cacheSize property. Default is 10000.
  • Set property nflow.executor.fetchChildWorkflowIds to false (recommended) to avoid reading child workflow IDs when executing a state. If access to child workflow instances is needed, they can be fetched using StateExecution interface. This will be the default in 7.0.0 release.
  • Automatically disable batch updates if batch update returns unsupported values.
  • Fetch optimistic locked list of workflow instance IDs in a separate transaction from the modify phase. This should reduce the risk of deadlocks in databases.
  • Tune batch status handling to consider status a failure if more than one row were modified.
• Dependency updates:
  • spotbugs-annotations 4.0.0-RC3
  • mssql 8.2.0.jre8
  • jetty 9.4.26.v20200117
  • mariadb 2.5.4
  • postgresql 42.2.10
  • hibernate-validator 6.1.2.Final
• Maven plugin updates:
  • spotbugs 4.0.0-RC3

v6.1.0

Highlights

• nflow-engine
  • Check that state variable value fits into the database column
  • Fix performance problems when querying workflows by only fetching state variables when requested

Details

• nflow-engine
  • Verify on startup that connections returned by DataSource used by nFlow have auto commit enabled. nFlow may not work correctly without it.
  • Throw StateVariableValueTooLongException if a state variable value that does not fit into the database column is detected. Checked in StateExecution.setVariable, StateExecution.addWorkflows, StateExecution.addChildWorkflows, WorkflowInstanceService.insertWorkflowInstance and when creating a new or updating an existing instance via REST API.
    If the exception is thrown during state processing and not handled by the state implementation, nFlow engine will catch the exception and retry state processing after delay configured by property nflow.executor.stateVariableValueTooLongRetryDelay.minutes (default is 60).
  • Fix honoring of includeCurrentStateVariables flag in WorkflowInstanceService.listWorkflowInstances. This caused major slowness when using bulk workflows.
    To preserve the existing (incorrect) default behaviour in backwards compatible way the default value in QueryWorkflowInstances.Builder is changed to true. The REST API is unaffected.
    Especially in workflows with many children that use the StateExecution.getAllChildWorkflows method the performance impact can be high. Before 7.0.0 release, it is recommended to use StateExecution.queryChildWorkflows(new QueryWorkflowInstances.Builder().setIncludeCurrentStateVariables(false).build()) if state variables are not needed.
  • Dependency updates:
    • jetty 9.4.25.v20191220
    • junit4 4.13
    • mariadb 2.5.3
    • spotbugs 4.0.0-beta4
    • spotbugs-annotations 4.0.0-beta4
    • swagger 1.6.0
    • hikaricp 3.4.2
    • jackson 2.10.2
    • jersey 2.30
    • mockitor-junit-jupiter 3.2.4
    • metrics 4.1.2
    • mockito 3.2.4
    • mysql 8.0.19
    • postgresql 42.2.9
    • netty 0.9.4.RELEASE
    • reflections 0.9.12
    • spring 5.2.3.RELEASE
    • junit5 5.6.0
    • apache-cxf 3.3.5
• nflow-rest-api
  • REST API returns HTTP 400 in case the state variable value is too long when inserting a new or updating an existing workflow instance
  • Exception mappers return JSON objects instead of raw strings
• nflow-explorer
  • Dependency updates:
    • angular 1.7.9
    • chart.js 2.9.3
    • karma 4.4.1
    • node-sass 4.13.0
    • node v12.13.1
    • npm 6.13.1
• Maven plugins
  • Dependency updates:
    • frontend 1.9.0
    • assembly 3.2.0
    • asciidoctor 2.0.0-RC.1
    • deploy 3.0.0-M1
    • enforcer 3.0.0-M3
    • install 3.0.0-M1
    • jar 3.2.0
    • site 3.8.2
    • source 3.2.1
    • surefire 3.0.0-M4

v6.0.0

Highlights

• Add priority to workflow instances
• Use constructor injection instead of field or setter injection in nFlow classes
• Separate workflow definition scanning from WorkflowDefinitionService
• Remove deprecated WorkflowInstanceInclude.STARTED enum value
• Remove deprecated AbstractWorkflowExecutorListener, use WorkflowExecutorListener instead
• Remove deprecated WorkflowInstance.setStarted, use WorkflowInstance.setStartedIfNotSet instead
• Add MariaDB support
• Add Kotlin example using nFlow with Spring Boot, and integrated nFlow Explorer
• Expose wakeup via REST-API
• Update database indices to match workflow instance polling code
• Add new configuration option nflow.db.disable_batch_updates

Details

• nflow-engine
  • Add priority two byte integer to the nflow_workflow table. When the dispatcher chooses from many available scheduled workflow instances it primarily (unfairly) picks the workflow instances with the largest priority values, and for workflows with the same priority, the ones with oldest next_activation timestamp. Priority defaults to 0 and can also be negative. Default priority value for the new workflow instances can be set per workflow definition (WorkflowSettings.Builder.setDefaultPriority), and overridden per workflow instance (WorkflowInstance.Builder.setPriority). Requires database migration, see database update scripts for details.
  • Separate workflow definition scanning from WorkflowDefinitionService by introducing WorkflowDefinitionSpringBeanScanner and WorkflowDefinitionClassNameScanner. This allows breaking the circular dependency when a workflow definition uses WorkflowInstanceService (which depends on WorkflowDefinitionService, which depended on all workflow definitions). This enabled using constructor injection in all nFlow classes.
  • Add disableMariaDbDriver to default MySQL JDBC URL so that in case there are both MySQL and MariaDB JDBC drivers in the classpath then MariaDB will not steal the MySQL URL.
  • Add support for nflow.db.mariadb profile.
  • Update database indices to match current workflow instance polling code.
  • Create indices for foreign keys in MS SQL database.
  • Fix create database scripts to work with empty database.
  • Add nflow.db.disable_batch_updates (default false) configuration parameter to make it possible to force use of multiple updates even if batch updates are supported by the database.
  • Dependency updates:
    • reactor.netty 0.9.1.RELEASE
    • jackson 2.10.1
    • mysql-connector-java 8.0.18
    • mariadb jdbc 2.4.4
    • postgresql jdbc 42.2.8
    • mssql-jdbc 7.4.1.jre8
    • metrics 4.1.1
    • junit5 5.5.2
    • hikaricp 3.4.1
    • jetty 9.4.20.v20190813
    • apache-cxf 3.3.4
    • slf4j 1.7.29
    • spring 5.2.1.RELEASE
    • hibernate-validator 6.1.0.Final
    • joda-time 2.10.5
    • swagger 1.5.24
    • mockito 3.1.0
    • hamcrest 2.2
    • h2 1.4.200
    • javassist 3.26.0-GA
    • jetty 9.4.21.v20190926
• nflow-explorer
  • Dependency updates
    • nodejs 10.16.3
    • npm 6.11.3
    • lodash 4.7.15
• nflow-examples
  • Update Spring Boot examples' dependencies
    • Spring Boot 2.1.7.RELEASE
    • nFlow 5.7.0
    • Gradle 5.5.1
• New REST-API endpoint to wake up workflow instance sleeping in specified states:
  PUT /nflow/api/v1/workflow-instance/5/wakeup
• Improve error logging in WorkflowStateProcessor.
• Replace FindBugs with SpotBugs.
• Drop index from main table that was only used for archiving purposes.

v5.7.0

Highlights

• Added started timestamp to workflow instance table (requires database update)
• Deprecated WorkflowInstanceInclude.STARTED enum value
• Deprecated AbstractWorkflowExecutorListener, use WorkflowExecutorListener instead
• Allow easily starting nflow-engine for embedding via NflowEngine class
• Added WorkflowLogContextListener for setting generic workflow properties to SLF4J log context and logging state variables when processing any state of any workflow instance

Details

• nflow-engine
  • Add started timestamp to workflow instance table. This makes the instance queries much faster when instances have lots of actions, as there is no need to join the nflow_workflow_action table to the query anymore.
  • Deprecated WorkflowInstanceInclude.STARTED enum value. This is not needed anymore, since the started timestamp is always read from the database when the instance is loaded.
  • Moved default implementations for WorkflowExecutorListener interface methods from the abstract class to the interface.
  • Fixed bug with dropping non-existent index in PostgreSQL create script.
• Dependency updates:
  • reactor.netty 0.8.8.RELEASE
  • jetty 9.4.18.v20190429
  • javassist 3.25.0-GA
  • mysql-connector-java 8.0.16
  • mssql-jdbc 7.2.2.jre8
  • metrics 4.1.0
  • spring 5.1.7.RELEASE
  • hibernate.validator 6.0.15.Final
  • cxf 3.3.2
  • joda-time 2.10.2
  • commons-lang3 3.9
  • jackson 2.9.9
  • junit 5.4.1
  • mockito 2.27.0
• nflow-explorer
  • Dependency updates

v5.6.0

Highlights

• Support non-enum WorkflowStates to enable, for example, Kotlin workflow definitions by extending AbstractWorkflowDefinition.

Details

• Dependency and plugin updates:
  • spring 5.1.6.RELEASE
  • reactor.netty 0.8.6.RELEASE
  • jetty 9.4.17.v20190418
• nflow-engine
  • Retry workflow state processing until all steps in nFlow-side are executed successfully. This will prevent workflow instances from being locked in executing status, if e.g. database connection fails after locking the instance and before querying the full workflow instance information (WorkflowStateProcessor).
  • Fix #306: create empty ArrayList with default initial size
  • Log more executor details on startup
  • Fix #311: Replace references to WorkflowDefinition with AbstractWorkflowDefinition to support non-enum WorkflowStates
  • Use name() instead of toString() when getting workflow instance initial state name

v5.5.0

CRITICAL BUG
This release contains a bug (#306, introduced in 5.4.1) which may cause OutOfMemory errors while fetching child workflow IDs. We recommend to update to 5.6.0 as soon as possible.

Highlights

• Introduce possibility to temporarily stop polling for new workflow instances by invoking pause() on WorkflowLifecycle, continue polling with resume(), and check pause status with isPaused().

Details

• Update libraries for nFlow Explorer. Includes fix for morgan library security issue.
  • https://github.com/NitorCreations/nflow/network/alert/nflow-explorer/package-lock.json/morgan/open
• Fix travis build to actually run unit tests for nflow-explorer module.
• Add pause(), resume() and isPaused() to WorkflowLifecycle, to pause and resume workflow instance polling in a running nFlow.

v5.4.1

CRITICAL BUG
This release introduced issue #306 which may cause OutOfMemory errors while fetching child workflow IDs. We recommend to update to 5.6.0 as soon as possible.

Highlights

• Introduce BulkWorkflow which can be used or extended to handle mass of child workflows without overloading the system.
• Introduce new workflow instance state type wait. Child workflow instances automatically wake up the parent when the parent is in a wait state and the child enters an end state.
• Allow creating workflows via REST API with null activation time (by setting activate = false).
• Allow creating child workflows via REST API (by setting parentWorkflowId).

Details

• See BulkWorkflowTest and DemoBulkWorkflow for examples on how to use bulk workflows
• Support boxed primitives (Integer, Float etc) with @StateVar
• nFlow Explorer: Library updates to lodash 4.17.11, moment 2.24.0 and extend 3.0.2
  Earlier lodash versions had this security vulnerability: https://nvd.nist.gov/vuln/detail/CVE-2018-16487
• Use select distinct when getting preserved actions while cleaning workflow instance history
• Dependency and plugin updates:
  • slf4j 1.7.26
  • spring 5.1.5.RELEASE
  • hamcrest 2.1
  • reactor.netty 0.8.5.RELEASE
  • swagger 1.5.22
  • mockito 2.24.5
  • io.dropwizard.metrics 4.0.5
  • mysql-connector-java 8.0.15
  • mssql-jdbc 7.2.1.jre8
  • hikaricp 3.3.1
  • maven-surefire 2.22.1
  • jetty 9.4.15.v20190215
  • h2 1.4.199
• Fix workflow history cleanup to keep the actions that hold the latest values of state variables
• nFlow Explorer: Custom content to workflow definition and workflow instance pages.
• nFlow Explorer: Executors page to use standard time formatting in tooltips
• nFlow netty: Add support for registering Spring ApplicationListeners
• nFlow jetty: Replace deprecated NCSARequestLog with CustomRequestLog
• Fix WorkflowLifecycle.stop() blocking forever if nflow.autostart=false and WorkflowLifecycle.start() not called