Automating Automation: BPMN ⨉ OpenAPI

The goal I am working towards is a common language for designers and developers. The latest footstep is OpenAPI integration with BPMN diagrams, making it simpler to build human/service workflows by importing APIs from public specifications. Here is a snapshot of the latest demo prototype.

What is BPMN?

What you see is the Camunda Modeling programs view of a BPMN diagram. It's a visual front end for an industry standard called Business Process Modeling Notation (BPMN 2.0). BPMN is powerful, it's a flexible language for representing human/system interactions, with a number of implementations from different vendors (i.e. no vendor lock in). I am building upon Camunda's work in particular because it is MIT licensed. BPMN is used to express generic business processes in a way that is also executable. It can serve both design and development needs.

To get a feel of the target scope of BPMN. Here is an example of BPMN workflow of a two person approval processes.

There is a ton of literature on the best way to model business processes using BPMN. The point though is that these diagrams are executable by a process engine. It's both a neat high level graphical expression of a workflow (design), with technical details hidden within each box within the tool (development). BPMN is an XML document that is both code and a diagram.

Camunda makes an excellent open source (Apache license) process engine you can serverlessly self host. It's mature, and reliable. I have been playing with it a lot lately as I think it's the kind of generic thing that fits a wide variety of business needs. However, one pain point is that it's really tedious trying to make JSON based REST calls, which is what I have set out to solve here.

What is OpenAPI?

OpenAPI is a standard for describing web service endpoints (previously known as Swagger). Mostly my job as a developer is calling other people's service endpoints, and occasionally writing an endpoint myself. Slack bots, Github hooks, starting an AWS resource, creating a Google doc are all web service calls. It's such a common thing we have a standard for describing the endpoints (OpenAPI).

There are lots of resources on the web about them. Including public directories like apis.guru. Almost all mainstream 3rd party services have OpenAPI specifications. Once you have an OpenAPI specification, you have all the information needed to call that service. OpenAPI specifications are machine readable, so the core idea is exploiting the public OpenAPI descriptions to prepopulate the tedious parts in BPMN, allowing BPMN processes to use 3rd party services trivially.

Camunda Modeler Element Templates

First, super kudos to the foresight of Camunda. They added an extension point to the BPMN Modeler tool which allows you to customize the UI for BPMN diagram boxes, called an Element Template. This is driven by an external config file, so to customize the development experience we "just" have to generate that JSON config from an OpenAPI spec.

Element Templates provide a drop down that you can customize. I started by populating it with the HTTP method + path extracted from the OpenAPI spec.

When applied to the Google Tasks API this gives me the following options.

What these do is explained by Google's documentation.

An individual API call needs to be customized in a variety of places though:

• The URL path
• The URL query string
• The request body
• HTTP headers

It's this mapping the BPMN process variables to a concrete API call and then doing the reverse when the service responds that is very error prone and tedious.

With Element Templates we are able to generate a Custom UI that allows configuration of the specific API parameters in a BPMN native way. What's really cool is that we can pass through the OpenAPI description too, so not only do we get a box to fill in, but also a human written description describing what the parameter does.

So specifically when creating a new Google task, we need to pass the title in the request body.

Typically we don't want hard coded values, so you can also use process variables:

To read a response from an API call, we bind it to a parameter. Here we assign the process value "items" to value of the response of getting the user's tasklists.

Put all these techniques together, we can chain the output of one API call to the input of another API call.

The Demo

When making calls to Google APIs you need an Oauth 2.0 token for the individual. The Camunda engine ships with a REST API, allowing you to start a workflow and pass in a token in a single request. The demo workflow can be run yourself but you need a token. The Google Developer's Oauth playground provides a convenient and trustworthy method to generating tokens for all Google APIs.

1. Goto the Oauth playground
2. Authorize the Tasks API v1 (https://www.googleapis.com/auth/tasks)
3. Exchange authorization code for tokens and copy the "Access Token"

4. Call the engine API, replacing the TOKEN with the result of step 3.

   curl -X POST https://camunda-flxotk3pnq-ew.a.run.app/engine-rest/message/ \     --header "Content-Type: application/json" \     --data '{"messageName":"add_task_test", \ "processVariables": { \         "token": {"type":"String", "value":"TOKEN"} \ } \ }'

Voila! The workflow will run and place a task in your first task list, right in the Gmail UI. (Note if the container is scaled down you might get a 419, just wait 2 mins and try again).

Next Steps

Hopefully you find this example inspiring. Using public OpenAPI specifications it becomes very easy to integrate with third party services such as G Suite, and build workflows around these existing services, with no serious coding necessary.

Given many clients are already using G Suite, this would allow organizations to provide custom services to employees very easily and embedded within the tools employees already use.

Discussion on Reddit.