Metrics Lifecycle and Deletion after a Period of Time

Target audience: System administrators

When you enable Metrics in Flowable Work, there will be no housekeeping done. If you would like to periodically remove old indices use the lifecycle management feature from Elasticsearch. Therefore, you need to configure your Elasticsearch instance accordingly. This how-to explains how you can setup the housekeeping.

Create a Lifecycle Template

First, it is required to create a lifecycle template. The lifecycle policy below will remove indices after 90 days.

Execute a PUT request against your Elasticsearch with the endpoint /_ilm/policy/flowable-metrics-lifecycle (e.g. http://localhost:9200/_ilm/policy/flowable-metrics-lifecycle):

{
    "policy": {
        "_meta": {
            "description": "Used for Flowable Metrics"
        },
        "phases": {
            "delete": {
                "min_age": "90d",
                "actions": {
                    "delete": {
                        "delete_searchable_snapshot": true
                    }
                }
            }
        }
    }
}

Create an Index Template

The lifecycle policy now needs to be applied to your indices. For new indices this can be done by creating an index template.

Therefore, another PUT needs to be executed against /_index_template/flowable-metrics-template of your Elasticsearch instance (e.g. http://localhost:9200/_index_template/flowable-metrics-template):

{
  "index_patterns": ["flowable-metrics-*"], 
  "template": {
    "settings": {
      "index.lifecycle.name": "flowable-metrics-lifecycle"
    }
  }
}

note

To apply it to an existing index, you can update the settings of the index. For further details please check out the Elasticsearch documentation.

Conclusion

Flowable does not provide out-of-the-box housekeeping for the metrics in Elasticsearch. However, since Elasticsearch is made for time series based data, it is easy to maintain the indices with Elasticsearch. This allows you to keep the size of the Elasticsearch indices on a consistent size and still have metrics enabled.