Skip to main content

Flowable Incident Reporting

v3.17.0+

Introduction​

During the runtime of Flowable there can be high load on the system, due to difference reasons. From 3.17.0 Flowable automatically can create incidents if certain threshold are met. These incidents can be seen in Flowable Control, and they are deleted automatically after 90 days. The deletion period is configurable using the property flowable.incident.report.cleaning-after with a duration value. e.g. in order to delete incidents created more than 30 days ago the property can be set to

flowable.incident.report.cleaning-after=30d

Incident Reporting​

By default, the system is checked every hour for incidents. This can be configured using the property flowable.incident.report.step with a duration value. For the saturation checks, the system is checked every minute. This can be configured using the property flowable.incident.report.gauge-metrics-check-step with a duration value.

The following incidents are created:

  • Slow jobs - If the mean execution time of a job group is above 5 seconds within the check interval
  • Slow service executions - If the mean execution time of a service operation is above 5 seconds within the check interval
  • Large Elasticsearch index requests - If there was a single indexing request above 5MB within the check interval
  • Task Executor queue saturation - If the Task Executor queue is filled above 80% within the check interval
  • Datasource connection pool saturation - If the Datasource connection pool is filled above 80% within the check interval

The following incidents have the following groupings:

  • Slow jobs
    • Type (Indexing / Timer / Async)
    • Job Handler
    • Scope Type (BPMN / CMMN, Content, etc.)
    • Scope Definition ID
    • Tenant ID
  • Slow Service Executions
    • Service Type
    • Service Key
    • Operation Key
    • Tenant ID
  • Large Elasticsearch index requests
    • Tenant ID
  • Task Executor queue saturation
    • Type (default, async, history, etc.)
  • Datasource connection pool saturation
    • Type (default, async, history, etc.)

The following properties exist for the incident reporting:

# Whether incident reporting is enabled
flowable.incident.report.enabled=true
# How often to check for incidents
flowable.incident.report.step=1h
# How often to check the saturation
flowable.incident.report.gauge-metrics-check-step=1m

# Whether job incidents are enabled
flowable.incident.report.job.enabled=true
# The threshold for async jobs above which an incident will be reported.
flowable.incident.report.job.async-threshold=5s
# The threshold for history jobs above which an incident will be reported.
flowable.incident.report.job.history-threshold=5s

# Whether indexing incidents are enabled
flowable.incident.report.indexing.enabled=true
# The threshold for the indexing bulk request size above which an incident will be reported.
flowable.incident.report.indexing.bulk-request-size-threshold=5MB


# Whether service registry incidents are enabled
flowable.incident.report.service-registry.enabled=true
# The threshold for service execution above which an incident will be reported.
flowable.incident.report.service-registry.threshold=5s

# Whether executor incidents are enabled
flowable.incident.report.executor.<executorName || all>.enabled=true
# The saturation threshold for the queue of the executor.
flowable.incident.report.executor.<executorName || all>.queue-saturation-threshold=0.8

# Whether datasource connection pool incidents are enabled
flowable.incident.report.db-connection-pool.enabled=true
# DB connection saturation threshold.
# If the connection pool saturation is above this threshold, an incident will be reported.
flowable.incident.report.db-connection-pool.saturation-threshold=0.8

The available names for the executors are:

  • spring - The out-of-the-box Task Executor configured by Spring Boot
  • platformAsync - The Task Executor used for executing async jobs in Flowable
  • platformHistoryAsync - The Task Executor used for executing indexing jobs in Flowable
  • contentAsync - The Task Executor used for generating content renditions in Flowable
  • flowableTaskInvoker - The Task Executor used for executing true parallel service task in Flowable (e.g. HTTP Task)