Deploying Flowable infrastructure on Azure

This document describes deploying a Flowable Kubernetes cluster and it's required components on Azure using Terraform.

The example deployment configuration files can be downloaded here.

Terraform

Terraform is an open source Infrastructure-As-Code (IAC) tool. Such tools allow you to manage infrastructure with configuration files rather than through a graphical user interface. More information regarding Terraform can be found here.

In order to provision the required Flowable infrastructure a collection of cloud platform specific Terraform modules are provided; including for Azure. These modules can be executed with a certain configuration that can be tailored to specific needs. For example; sizing, database type, permissions etc.

A complete example containing a configuration of a development environment and a production / live environment is provided. Deploying this example is described below.

Architecture Overview

Azure resource group organization

Resource groups

When managing multiple environments (dev, qa, prod) in the cloud there are several options on how to organize this. This guide follows the approach of having separate resource groups for each environment and a centralized security resource group. This account will contain;

1. The Terraform state
2. A key vault containing credentials and other secrets

Terraform state

Terraform must store state about your managed infrastructure and configuration. This state is used by Terraform to map real world resources to your configuration, keep track of metadata, and to improve performance for large infrastructures. By default, Terraform stores state locally in a file. When working with Terraform in a team, use of a local file makes Terraform usage complicated because each user must make sure they always have the latest state data before running Terraform and make sure that nobody else runs Terraform at the same time. With remote state, Terraform writes the state data to a remote data store (Azure storage account container), which can then be shared between all members of a team.

Key vault

The infrastructure components that we are going to deploy will make use of some security sensitive data. For example the required credentials to access the Flowable Helm and Docker repository. But also credentials for components like the database. To avoid storing these credentials in the deployment configuration files (and perhaps in a code repository) they will be stored as secrets in a Azure key vault. Terraform will be able to access and use them when required during the provisioning on the infrastructure.

Environment Resource Groups

Each Flowable environment (dev, qa, prod) will be under a separate Azure resource group. This approach provides a separation of resources.

Prerequisites

Before we can start deploying the infrastructure we need to complete some one-time initial steps. For this we will make use of the Azure Command Line Interface (Azure CLI) tooling. In addition Terraform and Terragrunt need to be installed. Terragrunt is a thin Terraform wrapper that provides some additional features like templating. More information on Terragrunt can be found here.

Setup CLI tooling

Both Azure CLI and Terraform need to authenticate with Azure. There are several ways to do this. Have look here and to read about the different options. More info about the authentication options with Terraform is available here.

For this guide we are going to use a Service Principal. You can create a Service Principal as described here or ask your administrator to provide you with the required info. The data needed for authenticating using a Service Principal is;

• app ID
• secret
• tenant ID

Homebrew

On MacOS systems the package manager Homebrew can be used.

Install Homebrew.

/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

For more information about Homebrew and how to install can be found here.

Azure CLI

brew install azure-cli

For more information about Azure CLI and how to install can be found here.

Test logging in using the Service Principal.

az login --service-principal -u <app ID> -p <secret> --tenant <tenant ID>

If the login succeeds the output will display the Service Principal identity information.

Terragrunt and Terraform

brew install terragrunt

this will also install Terraform. Please make sure Terraform is installed when Terragrunt is installed using another method.

More information about Terragrunt and how to install can be found here. More information about Terraform and how to install can be found here.

Central security resource group setup

A central resource group will be created to contain security sensitive data that will be used while provisioning the environments using Terraform. A key vault will be used to store his sensitive data. In addition to this a storage account will be create to store the Terraform state.

Two separate keyvaults will be created for each environment that will be deployed later. One for the development environment. And one for the production environment. This provides the possibility to define different access policies for each vault.

Execute the commands below to create the required resources. Provide your input where required.

The example below will create 2 separate storage containers (Terraform backend) and key vaults; one for the development and one for the production environment. When there is no need to have separate resources it's also possible create a storage container and key vault that will be used for both environments.

If you choose your own resources names or identifiers make sure you use these consistently executing the commands below and when updating the Terragrunt configuration later on.

List available regions

az account list-locations -o table

Set default region

Set northeurope as the default region

az configure --defaults location=northeurope

Create resource group

Create a resource group (in the default region) with the name rg-flowable-iac-sec

az group create -n rg-flowable-iac-sec

Create storage account and storage container

Create a storage account. The name used in this example is stflowableiactfstate. The storage account must however have a unique name within Azure. Update the chosen name in commands below and in the deployment configuration files (env.hcl) to match it.

az storage account create --resource-group rg-flowable-iac-sec --name stflowableiactfstate \
--sku Standard_LRS --encryption-services blob

List the keys for the newly created storage account

az storage account keys list --resource-group rg-flowable-iac-sec --account-name stflowableiactfstate

Create a storage container with the name scflowableiactfstate with one of the storage account keys (use the output from the previous command)

az storage container create --name scflowableiactfstate --account-name stflowableiactfstate \
--account-key <storrage account key>

Create key vault and secrets for development environment

Create a key vault that will contain the secrets for the development environment. The name used in this example is kv-flowable-iac-dev. The key vault name must however have a unique name within Azure. Update the chosen name in commands below and in the deployment configuration files (env.hcl) to match it.

az keyvault create --name kv-flowable-iac-dev --resource-group rg-flowable-iac-sec

This will create a keyvault with a default access policy making it accessible only to the identity which created it; the service principal in this case. When other identities need to be added, for example enabling access from the Azure Portal (web), the following command be used. The available secret-permissions are; all, backup, create, delete, deleteissuers, get, getissuers, import, list, listissuers, managecontacts, manageissuers, purge, recover, restore, setissuers, update.

Optional: add additional access policy to keyvault

az keyvault set-policy --name kv-flowable-iac-dev --object-id <identity object id> \
--secret-permissions <permission list>

Create a secret with the name flowable-repo-username with the value repo_user. Update the value with the username that has access to the Flowable Docker and Helm registry.

az keyvault secret set --name flowable-repo-username \
--vault-name kv-flowable-iac-dev --value <repo_user>

Create a secret with the name flowable-repo-password. Update the value with the password belonging to the user that has access to the Flowable Docker and Helm registry.

az keyvault secret set --name flowable-repo-password \
--vault-name kv-flowable-iac-dev --value <repo_password>

Create a secret with the name flowable-db-username. The username will be used when provisioning the database(s) and deploying the Flowable application(s).

az keyvault secret set --name flowable-db-username \
--vault-name kv-flowable-iac-dev --value flowable

Create a secret with the name flowable-db-password. The password will be used when provisioning the database(s) and deploying the Flowable application(s).

az keyvault secret set --name flowable-db-password \
--vault-name kv-flowable-iac-dev --value <db_password>

Create a secret with the name flowable-license. Provide the path to a valid Flowable license file. az keyvault secret set --name flowable-license --vault-name kv-flowable-iac-dev --file /path/to/license

Create key vault and secrets for production environment

Create a key vault that will contain the secrets for the production environment The name used in this example is kv-flowable-iac-prod. The key vault name must however have a unique name within Azure. Update the chosen name in commands below and in the deployment configuration files (env.hcl) to match it.

az keyvault create --name kv-flowable-iac-prod --resource-group rg-flowable-iac-sec

This will create a keyvault the a default access policy making it accessible only to the identity which created it; the service account in this case. When other identities need to be added, for example enabling access from the Azure Portal, the following command be used. The available secret-permissions are; all, backup, create, delete, deleteissuers, get, getissuers, import, list, listissuers, managecontacts, manageissuers, purge, recover, restore, setissuers, update.

Optional: add additional access policy to keyvault

az keyvault set-policy --name kv-flowable-iac-prod --object-id <identity object id> \
--secret-permissions <permission list>

Create a secret with the name flowable-repo-username with the value repo_user. Update the value with the username that has access to the Flowable Docker and Helm registry.

az keyvault secret set --name flowable-repo-username \
--vault-name kv-flowable-iac-prod --value <repo_user>

Create a secret with the name flowable-repo-password. Update the value with the password belonging to the user that has access to the Flowable Docker and Helm registry.

az keyvault secret set --name flowable-repo-password \
--vault-name kv-flowable-iac-prod --value <repo_password>

Create a secret with the name flowable-db-username. The username will be used when provisioning the database(s) and deploying the Flowable application(s).

az keyvault secret set --name flowable-db-username \
--vault-name kv-flowable-iac-prod --value flowable

Create a secret with the name flowable-db-password. The password will be used when provisioning the database(s) and deploying the Flowable application(s).

az keyvault secret set --name flowable-db-password \
--vault-name kv-flowable-iac-prod --value <db_password>

Create a secret with the name flowable-license. Provide the path to a valid Flowable license file. az keyvault secret set --name flowable-license --vault-name kv-flowable-iac-prod --file /path/to/license

Environment Architecture

Both quick start Flowable infrastructures (prod and dev) consist of the following Azure services and resources.

• Azure Kubernetes Service (AKS). This service provides a managed Kubernetes control plane and a autoscaling nodegroup.

• Azure Database for PostgreSQL server. This service provides a managed PostgreSQL database.

The provided environment configurations have a generic sizing and architecture. There is a differentation in non-prod and prod configuration. But depending on the actual use case additional configuration may be needed. In terms of sizing, the used Kubernetes node vm types, the used Postgres database server vm types. In terms of making use of in cluster services or managed SaaS offerings (Elasticsearch). And in terms of application configuration (JVM sizing, thread pool configuration, etc).

For these environments some practical choices were made with the focus on demonstrating the ease of deploying a complete Flowable environment end-to-end. For a real production environment we strongly recommend some additional improvements. One of them is to make use of the Elastic Cloud SaaS for Elasticsearch or an existing production ready Elasticsearch. Also, when running Flowable applications multi instance a central session store, for example Azure Cache for Redis, is advised.

Development

Production

Deploy Environment

Quick start projects structure

The Flowable Quick start can be downloaded here.

The archive consists of 2 folders;

• flowable-iac-deployments
• flowable-iac-modules

Flowable IAC Deployments

This folder contains the Terragrunt configuration files that, along with the Terraform modules in flowable-iac-modules, can be used to deploy a complete infrastructure running Flowable on various cloud platforms.

The code in this repo uses the following folder hierarchy:

account
└ region
  └ environment
    └ layer
      └ resource

Where:

• Account: For each provider account. In this case non-prod and prod.

• Region: Within each account, there will be one or more Azure regions, such as eastus, westeurope, and switzerlandwest, where you've deployed resources.

• Environment: Within each region, there will be one or more "environments", such as dev, stage, prod, etc. Typically, an environment will correspond to a Resource Group, which groups the resources for that environment in a logical container.

• Layer: This is a logical set of resources. F.e. foundation which consist of the base networking, security and Kubernetes cluster resources.

• Resource: This are the actual Azure services and resources. F.e. Azure Kubernetes Service, Azure Database Server or Flowable App.

Flowable IAC Modules

This folder contains a selection of Terraform modules that, along with the Terragrunt configuration files in flowable-iac-deployments can be used to deploy a complete infrastructure running Flowable on various cloud platforms. Some static configuration is present in these module definitions but most will be provided by the environment specific configuration defined in flowable-iac-deployments.

Update deployment configuration

During the initial setup of the Azure cloud environment; the security resource group rg-flowable-iac-sec; some resources were created. The default configuration corresponds to the initial configuration steps. When other names or regions where chosen during the initial setup some configuration files need to be updated.

Configuration files;

filename	description
/flowable-iac-deployments/<account>/<region>/<environment>/account.hcl	Configuration of account wide properties; for example the name of the workload; used in several places for naming resources
/flowable-iac-deployments/<account>/<region>/<environment>/region.hcl	Configuration of the region wide properites; for example the region name
/flowable-iac-deployments/<account>/<region>/<environment>/env.hcl	Containing properties related to the security resource group and resources. Also contains the property configuring the target environment resource group and naming properties
/flowable-iac-deployments/<account>/<region>/<environment>/app/app.hcl	Containing properties related to the Flowable application, Database and Elastisearch components
/flowable-iac-deployments/<account>/<region>/<environment>/app/flowable-app/values.yml	The Flowable Helm chart values file containing the configuration options

Configure Terraform authentication

We want Terraform to use the Service Principal that was used during the initial setup. When using regular user identities Terraform can make use of an authenticated Azure CLI sessie to perform it's actions. When using a Service Principal identity this is not possible. In this case we need to provide the credentials in another way. There are several ways to do this. For this example we are going to set environment variables.

update the values below with Service Principal account data

export ARM_CLIENT_ID="00000000-0000-0000-0000-000000000000"
export ARM_CLIENT_SECRET="00000000-0000-0000-0000-000000000000"
export ARM_SUBSCRIPTION_ID="00000000-0000-0000-0000-000000000000"
export ARM_TENANT_ID="00000000-0000-0000-0000-000000000000"

Deploying the infrastructure

There are several options for deploying the infrastructure depending on 'where' (in what folder) you execute the terragrunt commands.

You can deploy multiple resources / layers / environments using the terragrunt run-all apply command depending. Or you can deploy a single resource using the command terragrunt apply.

Another option is to preview te change that will be deployed comparing the current configuration against the live infrastructure using the command terragrunt plan. Mind: this will not work for modules that have dependencies to other modules that have not been 'applied' yet. More info on this can be found here.

The next section shows some examples on how to deploy or undeploy complete environments or parts of the infrastructure.

Deploy (complete) development and production environment

This will deploy the complete development and production environment.

cd flowable-iac-deployments
terragrunt run-all apply

Deploy (complete) development environment

This will deploy the complete development environment.

cd flowable-iac-deployments/non-prod/westeurope/dev
terragrunt run-all apply

Undeploy (complete) production environment

This will remove the resource group containing all deployed production resources and services.

cd flowable-iac-deployments/prod/westeurope/prod/foundation/resource-group
terragrunt run-all destroy

Deploy foundation layer development environment

This will deploy the Virtual Private Cloud (VPC), the Elastic File Service (EFS), the Elastic Kubernetes Service (EKS) with the required components. It will also create a Resource Group grouping all deployed services and resources.

cd flowable-iac-deployments/non-prod/westeurope/dev/foundation
terragrunt run-all apply

Review (plan) and deploy (execute) Flowable development environment

This will deploy the Flowable Helm chart using a development environment specific configuration defined in a custom values.yaml file.

cd flowable-iac-deployments/non-prod/westeurope/dev/app/flowable-app
terragrunt plan

Have a look at the output and review the planned changes to the live infrastructure.

cd flowable-iac-deployments/non-prod/westeurope/dev/app/flowable-app
terragrunt apply

The default configuration will result in a dev environment containing the following applications

DEV:

• Flowable Design (https://flowable-iac-dev.northeurope.cloudapp.azure.com/design)
• Flowable Work (https://flowable-iac-dev.northeurope.cloudapp.azure.com/work)

PROD:

• Flowable Control (https://flowable-iac-prod.northeurope.cloudapp.azure.com/control)
• Flowable Work (https://flowable-iac-prod.northeurope.cloudapp.azure.com/work)

Advanced scenarios

The following section describes some more advanced use cases with a more in depth explanation.

Change Flowable datasource

The quick start environments dev and prod deploy and configure Azure Database for PostgreSQL service and databases. When there is the need to connect to an existing database, for example Oracle, some modifications to the deployment configuration need to be done.

If an existing database server is already in place there is no need to deploy one. Therefore we need to remove the deployment configuration for PostgreSQL.

Let's have look on how to configure this for the development environment.

Remove the flowable-iac-deployments/dev/northeurope/dev/foundation/postgres-flexible-server and the flowable-iac-deployments/dev/northeurope/dev/app/postgres-flexible-databases folders.

The Flowable Docker images do not contain the Oracle JDBC driver. However it is possible to download these drivers during the deployment of the Helm chart (while initialization the pods). In order to do this the following needs to be configured.

flowable-iac-deployments/dev/northeurope/dev/app/flowable-app/values.yaml

work:
  enabled: ${flowable_work_enabled}
  ...
  envVariables:
    spring.datasource.url=jdbc:oracle:thin:@...
    spring.datasource.username=username
    spring.datasource.password=password

...

  jdbc: 
    fetchDriver: true
    mvnDependency:
      groupId: com.oracle.database.jdbc
      artifactId: ojdbc8
      version: 19.3.0.0
  
...

postgres:
  enabled: false

Modify the configuration for each needed Flowable application.