AI Service Definition
Introduction
Flowable has a service registry where you can define any sort of service definition within Flowable Design to be invoked and executed with the service engine at runtime.
With version v3.17.0+ of Flowable, there is a new type available for a service definition, named AI to define an AI-based service and use it within a process or case model or even directly out of the UI.
A really great way to model such an AI-based service, is to use the integrated testing feature within Design to test the prompting, but already using the very same structure as being used at runtime and adjust it, if needed.
AI Service Definition
When creating a new service definition, choose the service type AI to create an AI-based service:
For an AI-based service definition, you can add as many operations as you want. They are basically not connected other than being in the same service definition.
How the AI service works
Before we go into details on how to model an AI-based service, we need to get an understanding on how it works.
You probably already know AI tools like ChatGPT and are familiar with creating a prompt, asking questions and getting answers accordingly, rephrasing and
going back and forth until you get the result you were looking for, hopefully.
The AI service works similarly, but it adds a very important piece to the equation: structure! Instead of getting a text-based response, you can define the
exact data structure you expect and want the answer to be.
The very same way you can define input and output parameters for any other type of service, you can do so for the AI service too, and the output parameters are going to be used to tell AI what exact data structure we want our answer to be.
As you can see, having very clear structure on input and output, gives us the opportunity to use AI-based services fully automatically within a process or case, providing structure input, but also getting structured output back and continue processing it as needed.
First simple example
Let's assume we are dealing with large content like documents or email content or whatever, and it would be very beneficial to just get a brief summary.
We can create an AI-based service to input content and get back a summary of it.
Add operation for content summary
Click the Add operation (plus) button and fill in the following information in the first tab, where we can give the operation its name, key and an optional
description:
Define the input parameters
In this first, very simple AI-service, we only need one input parameter providing the content:
Define the output parameters
Although we only request a simple return value containing the summary, it is still nice to put it into a structured way by defining the name of the output
we expect and its type of course:
As you can see, we can define quite a lot for output parameters like what happens, if there is no value provided or is empty and so on. For our example, we only have one and that one is required as otherwise the service does not make any sense, but as we will see later, we can also play with default values, if AI is not providing any outcome.
Defining the prompt
The most important part of an AI-based service is the engineered prompt. Within the prompt tab, you can define the user- system-message, most likely the user message containing the input you get with the service and the system message gives the necessary context and instructing AI on how to process the user message.
Our example could look something like this:
Testing the AI prompt with example input
Testing is very essential when developing a prompt, so you can go back and forth until you have the prompt and the resulting answer in a way according to the
requirements.
Adding more details
Maybe we want to limit the summary to a maximum number of characters, and maybe we additionally want to get the sentiment of the content as well.
With the structured input and output parameters, this is very straightforward.
We can add another input parameter containing the maximum size of the summary like this:
And another output parameter, containing the sentiment from 1 (very bad) to 5 (very exciting):
Afterwards we can make use of the new parameters within the prompt:
When you test the prompt and play with the length, you can see the effect of the additional parameter. You will also notice that you can let AI return more than one output value in exactly the format you specified.
Using the service within a process or case
An AI-based service definition can be used in exactly the same way as any other service definition as we also have input and output parameters and everything else is kind of transparent to the invoking model like a process- or case-model, which makes the AI assisted service even more useful within any type of process.
It allows to wire in structured data as well as process the outcome data in a structured way as well.
Second example
We learned how we can structure input and especially output data which makes the usage of AI-assisted services really easy when used in automated processes and cases.
Now we want to look at an example where we want to extract structured information from unstructured content like an email text or document.
Assume you receive emails with address data somewhere in the content and you need to extract it and put it into a structured form. With an AI service, this can be fully automated.
Create the AI service definition
Pretty similar to the first example, we can create a new AI service definition or just add a new operation to the existing one.
We also use content as the input parameter and the following list of output parameters with their types:
The prompt could look something like this:
When testing with the prompt and various samples of input content, you can see that AI is even able to complement incomplete address data like looking up and adding the zip code or country, if there is enough data available.
Conclusion
The AI service definition is a very useful and easy way to embed AI-assisted services into your process or case automation as it supports a very structured way to interact with an AI service and use your structured data of a case or process within such a service and especially afterwards with the provided output parameters.
Developing an AI service is really all about prompt engineering! So take your time to come up with a prompt suitable to your requirements or even look up similar examples to create the prompt you need.
If you want to use a step-by-step tutorial on how to create a comprehensive AI service and use it within a process or even directly within the UI, have a look at the Create an AI service HowTo.