Components

The components are split up in components delivered by Flowable, and third party components.

Flowable

  • Flowable Engage Platform

  • Flowable Engage Web Socket Handler and Engage UI

Third party systems

  • Message Queue like ActiveMQ

  • Oracle Database v11, MySQL or Postgresql

  • Elastic Search v6.2

  • SMTP server

Software requirements

The following components are required in order to run the Flowable components:

  • Oracle JDK 8 or OpenJDK 8

  • Tomcat 8.0 or 8.5 (only when not ran with Java directly)

Flowable Platform

Installation

  1. Download a recent stable version of Apache Tomcat

  2. Download flowable-engage-app.war distribution

  3. Make sure the Tomcat Server is stopped

  4. Copy the war file to the Tomcat webapps folder

  5. Start the Tomcat Server

The Flowable Platform app should now be running with an H2 in-memory database and when accessing the http service on port 8080 the login screen should be shown.

Follow the next chapter which will show how to connect the app to the third party components.

Configuration

Flowable Application Properties

The Flowable auto configuration is leveraging the Spring Boot properties and configuration mechanism.

Here is a list of configuration properties that the Flowable Spring Boot support consumes with the default values and a brief explanation.

flowable.project-name= # The project name
flowable.notification.enabled=true # Whether the Notification engine needs to be started
flowable.action.enabled=true # Whether the Engage engine needs to be started
flowable.action.deployment-name=SpringBootAutoDeployment # The name of the deployment for the action definition resources
flowable.action.resource-location=classpath*:/actions/ # The location where the Action resources are located
flowable.action.deploy-resources=true # Whether to perform deployment of resources
flowable.action.deploy-default-resources=true # Whether to perform the loading of the default resources. The default resources are loaded from a fix location and are provided by the dependencies. In case overwriting of the defaults is needed, then this should be set to false.
flowable.engage.enabled=true # Whether the Engage engine needs to be started
flowable.engage.deployment-name=SpringBootAutoDeployment # The name of the deployment for the action definition resources
flowable.engage.resource-location=classpath*:/actions/ # The location where the action definition resources are located
flowable.engage.deploy-resources=true # Whether to perform deployment of resources
flowable.technical.admin-username=technical-admin # The username for the in memory technical admin
flowable.technical.admin-password= # The password for the in memory technical admin
flowable.websocket.broker-type=simple # The type of the broker that needs to be configure.
  # It has two options available:
   * simple # A simple message broker.
   * stomp # Enable a STOMP broker relay.
flowable.websocket.destination-prefixes= # The prefixes to filter destinations targeting the broker (e.g. destinations prefixed with "/topic"). In case of STOMP broker type check the STOMP documentation of the message broker for supported destinations
flowable.websocket.application-destination-prefixes=/app # A comma separated list of prefixes to filter destinations targeting application annotated methods. For example destinations prefixed with "/app" may be processed by annotated methods while other destinations may target the message broker (e.g. "/topic", "/queue"). -prefixes that do not have a trailing slash will have one automatically appended.
flowable.websocket.stomp-broker-addresses=127.0.0.1:61613 # A comma separated list of the locations of the STOMP broker relay. They need to be in a hostname + port format.
flowable.websocket.user-destination-broadcast=/topic/unresolved.user.dest # Set a destination to broadcast messages to user destinations that remain unresolved because the user appears not to be connected. In a multi-application server scenario this gives other application servers a chance to try.
flowable.websocket.user-registry-broadcast=/topic/registry.broadcast # Set a destination to broadcast the content of the local user registry to and to listen for such broadcasts from other servers. In a multi-application server scenarios this allows each server's user registry to be aware of users connected to other servers.
flowable.websocket.user-status-baseUrl=http://localhost:8090/engage-api/technical/users # The base url for the update of the technical status of users
flowable.indexing.enabled=true # Whether the Indexing service needs to be started
flowable.indexing.elasticsearch-addresses=http://localhost:9200 # Separated comma list of elasticsearch addresses
flowable.indexing.max-retry-timeout=10000 # Expressed in milliseconds
flowable.indexing.index-name-prefix= # Will be used as a prefix for every index
flowable.indexing.enable-default-mappings=true # Whether the default mapping needs to be used
flowable.indexing.converter.converter-date-pattern=dd.MM.yyyy dd MM yyyy dd.MM MM.yyyy # Pattern to convert Date values
flowable.indexing.converter.converter-date-time-pattern=dd.MM.yyyy HH:mm:ss # Pattern to converte DateTime values
flowable.indexing.number-of-shards=5
flowable.indexing.number-of-replicas=1
flowable.indexing.refresh-interval=1s
flowable.indexing.attachment-indexed-chars=-1
flowable.indexing.ngram-min-char=2
flowable.indexing.ngram-max-char=8
flowable.indexing.ngram-edge-min-char=1
flowable.indexing.ngram-edge-max-char=8
flowable.indexing.ngram-max-diff=7
flowable.async-history.enabled= # Whether the async history needs to be started
flowable.async-history.mode=THREADPOOL # It can be set also to MESSAGING
flowable.async-history.threadpool.core-poll-size=1
flowable.async-history.threadpool.max-poll-size=10
flowable.async-history.threadpool.queue-capacity=1024
flowable.async-history.threadpool.thread-name-prefix=flowable-async-
flowable.async-history.threadpool.await-termination-secords=30
flowable.async-executor.enabled= # Whether the async executor needs to be started
flowable.async-executor.mode=THREADPOOL # It can be set also to MESSAGING
flowable.async-executor.threadpool.core-poll-size=1
flowable.async-executor.threadpool.max-poll-size=10
flowable.async-executor.threadpool.queue-capacity=1024
flowable.async-executor.threadpool.thread-name-prefix=flowable-async-
flowable.async-executor.threadpool.await-termination-secords=30

Database

Usually, the data of the apps should be stored in a persistent way and not in the default H2 in-memory database. To achieve this a change of the configuration to a MySQL or Postgres (or other persistent database). You can do this per app by changing the application.properties file in the WEB-INF/classes/ directory of the app.

spring.datasource.url=jdbc:mysql://127.0.0.1:3306/flowable?characterEncoding=UTF-8
spring.datasource.username=flowable
spring.datasource.password=flowable

This configuration will expect a database named flowable to be present in the MySQL server and the UI apps will automatically generate the necessary database tables.

For Postgres the following changes are necessary:

spring.datasource.url=jdbc:postgresql://localhost:5432/flowable
spring.datasource.username=flowable
spring.datasource.password=flowable

For Oracle database the following changes are necessary:

spring.datasource.url=jdbc:oracle:thin:@//localhost:1521/flowable
spring.datasource.username=flowable
spring.datasource.password=flowable

For more information about Oracle data sources and URLs check the Oracle Database JDBC Developer’s Guide and Reference.

In addition to changing the configuration, make sure the database driver is available on the classpath. Please do so by adding the driver JAR file to the WEB-INF/lib folder, but you can also copy the JAR file once to the Tomcat lib folder. For MySQL and Postgres the database drivers can be downloaded from:

SMTP

Add some or all of these SMTP properties to connect to your SMTP service:

flowable.mail.server.default-from=flowable@localhost # The default from address that needs to be used when sending emails.
flowable.mail.server.host=localhost # The host of the mail server.
flowable.mail.server.password= # The password for the mail server authentication.
flowable.mail.server.port=1025 # The port of the mail server.
flowable.mail.server.use-ssl=false # Sets whether SSL/TLS encryption should be enabled for the SMTP transport upon connection (SMTPS/POPS).
flowable.mail.server.use-tls=false # Set or disable the STARTTLS encryption.
flowable.mail.server.username= # The username that needs to be used for the mail server authentication. If empty no authentication would be used.

Message Queue

You can use any Message Queue that supports STOMP, like ActiveMQ. Add some or all of these MQ properties to connect to your MQ service:

spring.activemq.broker-url= # URL of the ActiveMQ broker. Auto-generated by default.
spring.activemq.close-timeout=15s # Time to wait before considering a close complete.
spring.activemq.in-memory=true # Whether the default broker URL should be in memory. Ignored if an explicit broker has been specified.
spring.activemq.non-blocking-redelivery=false # Whether to stop message delivery before re-delivering messages from a rolled back transaction. This implies that message order is not preserved when this is enabled.
spring.activemq.password= # Login password of the broker.
spring.activemq.send-timeout=0ms # Time to wait on message sends for a response. Set it to 0 to wait forever.
spring.activemq.user= # Login user of the broker.
spring.activemq.packages.trust-all= # Whether to trust all packages.
spring.activemq.packages.trusted= # Comma-separated list of specific packages to trust (when not trusting all packages).
spring.activemq.pool.block-if-full=true # Whether to block when a connection is requested and the pool is full. Set it to false to throw a "JMSException" instead.
spring.activemq.pool.block-if-full-timeout=-1ms # Blocking period before throwing an exception if the pool is still full.
spring.activemq.pool.create-connection-on-startup=true # Whether to create a connection on startup. Can be used to warm up the pool on startup.
spring.activemq.pool.enabled=false # Whether a PooledConnectionFactory should be created, instead of a regular ConnectionFactory.
spring.activemq.pool.expiry-timeout=0ms # Connection expiration timeout.
spring.activemq.pool.idle-timeout=30s # Connection idle timeout.
spring.activemq.pool.max-connections=1 # Maximum number of pooled connections.
spring.activemq.pool.maximum-active-session-per-connection=500 # Maximum number of active sessions per connection.
spring.activemq.pool.reconnect-on-exception=true # Reset the connection when a "JMSException" occurs.
spring.activemq.pool.time-between-expiration-check=-1ms # Time to sleep between runs of the idle connection eviction thread. When negative, no idle connection eviction thread runs.
spring.activemq.pool.use-anonymous-producers=true # Whether to use only one anonymous "MessageProducer" instance. Set it to false to create one "MessageProducer" every time one is required.

Other spring properties

As the Flowable apps are Spring Boot applications you can use all the properties Spring Boot provides. In order to provide custom configuration for the apps have a look at the Externalized Configuration section of the Spring Boot documentation.

Health check

You can check the status of the application using the health check page:

curl http://localhost:8080/flowable-engage-app/actuator/health

You will receive a JSON string where status should be set to UP.

In addition, check the tomcat catalina.out whether the database connection has been established successful. If so you will see liquibase issuing SQL commands for creating the default scaffold in the database.

Architecture

ElasticSearch (ES) is used in the Engage platform to make conversation, message, process, case and task data available in a performant manner, also when performing complex queries. The Engage platform works against version 6.x of ElasticSearch. A recent version, such as 6.2.4, is recommended.

Elasticsearch Schema

The proposed configration for an Elasticsearch cluster (ESC) to use with flowable engage platform consist as the graphic above shows of a minimum of nine nodes: 3 master-eligible nodes, 2 http nodes and a minimum of 4 data nodes. Because of the separation between client, master and data nodes, this environment is ready for scaling-out to a larger setup if that is required by the usage of the Engage platform.

Terminology
Name Description

Master Eligible Node

The master node is responsible for lightweight cluster-wide actions such as creating or deleting an index, tracking which nodes are part of the cluster, and deciding which shards to allocate to which nodes. It is important for cluster health to have a stable master node.

Data Node

Data nodes hold the shards that contain the documents you have indexed. Data nodes handle data related operations like CRUD, search, and aggregations. These operations are I/O-, memory-, and CPU-intensive. It is important to monitor these resources and to add more data nodes if they are overloaded.

Http Node

A Http node (coordinating node) can only route requests, handle the search reduce phase, and distribute bulk indexing. Essentially, coordinating only nodes behave as smart load balancers.

Further information concerning the purpose, configuration and more are to be found on the official elasticsearch documentation.

Technical Details of the Virtual Machines

Name CPU RAM Disk

Master Eligible Nodes

16

64G

500G

Data Nodes

24

64G

2000G

Http Nodes (coordinating nodes)

8

16G

50G

JVM Parameters

Java parameters: 31GB assigned to the java heap (maximum) for Client & Master nodes.

Oracle Database

The Oracle database will be used by the Engage platform to transactionally handle conversation, message, process, case and task data. When either the current version of an Engage data entity is needed or transactional guarantees are required, the Oracle database will be used to persist the data. The stored data will be indexed to ElasticSearch through the Engage platform using asynchronous jobs.

The following Oracle database setup is recommended for Engage platform.

Elasticsearch Schema

Technical Detail of the Vitual Machines

The following table shows the recommend VM specifications per node type of the Oracle DB servers when no file persistence is necessary for Oracle.

CPU RAM Disk (if no BLOB)

Minimum

16

64GB

1TB

Recommended

32

96GB

1TB

When file persistence is necessary for Oracle, the following specifications are applicable.

CPU RAM Disk (if BLOB)

Minimum

16

128GB

10TB

Recommended

32

196GB

10TB

Storage Growth Estimation

The following calculations describe the minimum and recommended sizing for the Oracle database and ElasticSearch nodes, based on the estimated number of conversations and messages. The numbers are strongly dependent on the number of people using the Engage platform. In addition to the message data, there is also case, process instance and task data to be stored. A similar amount of data is expected on a daily basis.

Example

These values are completely imaginary, please do not use them as they are for your project

Data:

Description Values

Users

100 Client Advisors with 30 Customers each

Conversations

5 conversations a day with average of 10 messages

Messagesize

expected average size with 120 characters per message of 1920 bytes = 1.92 kb

Period

1 Year ( 260 working days)

Calculations:

[(100 Client Advisors * 30 Clients) * (5 conversations * 10 messages)] * 1.92 KB message size
= 288000 KB ≈ 288 MB/day =>  288MB/day * 260 days/year = 7488 MB ≈ 75 GB

The data for case, process instances and task is, as already mentioned above, about the same size. This would make a total of about 150 GB of yearly data.

Appendix

Appendix A. Service files

You can use these service files as a template for registering a Linux service.

systemd

[Unit]
Description=Service Name
After=syslog.target network.target
[Service]
Environment=PATH=/opt/tomcat/bin/
Type=forking
User=service
Group=service
ExecStart=startup.sh
ExecStop=shutdown.sh
[Install]
WantedBy=multi-user.target

Disclaimer

Flowable AG may change, amend or delete information contained in its documents at any time and without formal arrangement. Documents and other informational items concerning a release of Flowable Engage that has not yet been officially released by Flowable AG are non-binding and may be incomplete or contain errors. Flowable AG strives to provide complete and exact information in its documents. If permitted by applicable law, Flowable AG takes no responsibility for the accuracy of the issued informational items and documentation and does not provide any warranty for their content. Flowable AG assumes no liability for any direct, indirect or incidental damages incurred by the use of its documents and informational items. Documents or informational items do not change or amend the contractual terms and conditions regarding Flowable Engage. For improved readability, the documentation uses male pronouns for all persons. It does, however, address both male and female readers to the same extent.