Botium Connectors

Supported technologies

For a list of all supported technologies see Botium Wiki

Generic HTTP(S)/JSON Connector

Lots of chatbots connect to a simple HTTP/JSON based API in the backend to handle user requests and prepare the responses. This Botium connector mode enables Botium to connect to such an API.

Features

  • Sync API supported. (API is sync if the API messages are coming as HTTP response)

  • Async API is supported (API is async if the API messages are coming via a webhook) - see below

  • Polling API is supported (Botium will continue polling an endpoint for chatbot responses) - see below

  • All HTTP methods are supported, JSON and Non-JSON content (only JSON content can be processed further)

  • Session tracking is supported (“Context”). The context can be initialized and will be updated after each request/response and can be used for example to track a session-id (often called “conversation id”) or additional conversation state information.

  • Using Mustache templates to dynamically construct HTTP Headers, Body, URL, ConversationId, and StepId.

  • You can use in Mustache the context (“context” scope in the mustache template) and message content (“msg” scope). You can reach scripting memory variables too (“msg.scripting”) and all the functions as in scripting memory.

  • The mustache templates are fed with a unique “conversationId” (by conversation) and a unique “stepId” (for each sent message) in the “botium” scope

Important: In case you don’t want Mustache to do URL/HTML encoding with your texts (which you typically don’t want for HTTP POST endpoints), use triple braces {{{ … }}} as placeholders instead of two

Mustache Variables

For building up URLs, HTTP Headers, HTTP Body there are Mustache templates used. Several mustache variables are available, this is the Mustache view object constructed:

{
  container: <current-container>,
  context: <session-context>,
  msg: <botium-message>,
  botium: {
    conversationId: <uniq-id-for-conversation>,
    stepId: <uniq-id-for-conversation-step>
  },
  fnc: {
    <scripting-memory-functions>,
    jsonify: <json-escape string>
  }
}

context.*

This is the current session context, as initialized with the capability SIMPLEREST_INIT_CONTEXT and updated on each conversation step from the HTTP response with the capability SIMPLEREST_CONTEXT_JSONPATH (see below for details).

For example, if the session context holds something like a sessionId to be part of the message body, this can be done by initializing the session context with:

...
"SIMPLEREST_INIT_CONTEXT": { "sessionId": "" },
"SIMPLEREST_CONTEXT_JSONPATH": "$.session",
"SIMPLEREST_BODY_TEMPLATE": {
  "text": "{{msg.messageText}}",
  "sessionId": "{{context.sessionId}}"
},
...

msg.*

This is the Botium message in its internal representation. You can use the full structure of the message, see here for details.

  • Message Text

  • Test Project Name

  • Test Session Name

  • Convo Name

botium.*

In this scope for convenience there are unique ids for the conversation and for each conversation step available. A possible use case are chatbot endpoints where the session identifier is to be generated by the client, not by the server.

You can use them in this way:

...
"SIMPLEREST_BODY_TEMPLATE": {
  "text": "{{msg.messageText}}",
  "stepId": "{{botium.stepId}}",
  "conversationId": "{{botium.conversationId}}"
},
...

In case there are special formatting requirements for those, you can use the capabilities SIMPLEREST_CONVERSATION_ID_TEMPLATE and SIMPLEREST_STEP_ID_TEMPLATE (see below) as template for generating those values.

fnc.*

Scripting memory functions are available under this scope, see section Using Scripting Memory within Mustache Templates

Connecting Generic HTTP(S)/JSON chatbot to Botium

Create a botium.json with this URL in your project directory:

{
  "botium": {
    "Capabilities": {
      "PROJECTNAME": "<whatever>",
      "CONTAINERMODE": "simplerest",
      "SIMPLEREST_URL": "...",
      ...
    }
  }
}

Supported Capabilities

SIMPLEREST_URL *

This points to the URL of your endpoint. On each request, the URL can be adapted to current context and message by using Mustache rendering.

Example: Constructing the URL from context and message content:

https://my-api-website/api/{{context.conversation_id}}/{{msg.messageText}}

The URL is constructed by a base URL (https://my-api-website/api/) and extended by a context variable (“conversation_id”) which has been retrieved previously and by the text of the current message to send to the chatbot.

SIMPLEREST_METHOD

Default: GET

Either GET or POST

SIMPLEREST_TIMEOUT

HTTP Timeout (default 10 seconds)

SIMPLEREST_HEADERS_TEMPLATE

If you require HTTP headers to be sent to the endpoint (for example for authorization), this is the capability to configure. The headers are constructed as Mustache template and can be extended with current context or message variables.

It is a JSON structure which contains key/value pairs for HTTP headers and values.

Example: Sending an API Token:

{ "apiToken": "testapitoken" }

SIMPLEREST_BODY_TEMPLATE

For POST endpoints, the HTTP body to be sent to the endpoint. Again, Mustache rendering is applied.

Example: Sending the message text and the current conversation id in the HTTP body:

{ "text": "{{msg.messageText}}", "conversation_id": {{context.conversation_id}}" }

SIMPLEREST_BODY_RAW

By default, a JSON structure is sent to the HTTP endpoint. If you want to send raw data (for example: x-www-form-urlencoded), set this capability to prevent JSON formating.

SIMPLEREST_BODY_JSONPATH

Available since Botium Core 1.6.2

If your endpoint is delivering multiple independent responses to be shown to the user, this JSONPath Expression can be used to split the HTTP response body into multiple junks to be handled by the other JSONPath Expressions below individually.

Example: this response contains multiple text messages:

{
  responses: [
    {
      text: 'text 1',
      media: 'http://botium.at/1.jpg'
    },
    {
      text: 'text 2',
      media: 'http://botium.at/2.jpg'
    },
    {
      text: 'text 3',
      media: 'http://botium.at/3.jpg'
    }
  ]
}

Example: Set of capabilities to handle this response:

...
"SIMPLEREST_BODY_JSONPATH": "$.responses[*]",
"SIMPLEREST_RESPONSE_JSONPATH": "$.text",
"SIMPLEREST_MEDIA_JSONPATH": "$.media"
...

You can see in this example that the SIMPLEREST_BODY_JSONPATH capability splits the response into multiple chunks, and the other JSONPath expressions are evaluated relative to them, 3 times.

SIMPLEREST_RESPONSE_JSONPATH(*)

This capability is for extracting the actual response texts from the HTTP response body of the endpoint. This is given as a JSONPath Expression (JSONPath online evaluator) and in case your endpoint returns more than one response, there can be given additional capabilities starting with the SIMPLEREST_RESPONSE_JSONPATH-prefix. Every single capability is evaluated against the HTTP response body of the endpoint, yielding one chatbot response message per expression.

Example: the “text” attribute of the JSON response contains the message content:

...
"SIMPLEREST_RESPONSE_JSONPATH": "$.text.*"
...

Example: there are additional “quick response” elements to be extracted as text:

...
"SIMPLEREST_BUTTONS_JSONPATH": "$.quick_response.*"
...

Example: add multiple JSONPath expressions as array:

The two examples from above can be combined like this:

...
"SIMPLEREST_RESPONSE_JSONPATH": [ "$.text.*", "$.quick_response.*"]
...

Example: add multiple JSONPath expressions with separator:

Or like this:

...
"SIMPLEREST_RESPONSE_JSONPATH_SOMETHING_COMPLETELY_DIFFERENT": "$.text.*,$.quick_response.*"
...

SIMPLEREST_IGNORE_EMPTY

Make Botium skip empty messages from processing (no text, no attachments, no buttons, no nlp …)

Default empty content is ignored.

SIMPLEREST_CONTEXT_JSONPATH(*)

The session variables you have to store in the current session context are extracted from the HTTP response body of the endpoint. This is a JSONPath Expression, just use “$” to use the full HTTP response body as session context (default: use full body). Can be specified multiple times, all found sections will be merged.

SIMPLEREST_MEDIA_JSONPATH(*)

JSONPath Expression(s) for retrieving media attachments from response body.

See SIMPLEREST_RESPONSE_JSONPATH how to use it with multiple JSONPath expressions.

SIMPLEREST_BUTTONS_JSONPATH(*)

JSONPath Expression(s) for retrieving buttons from response body.

See SIMPLEREST_RESPONSE_JSONPATH how to use it with multiple JSONPath expressions.

SIMPLEREST_INIT_TEXT

Some chatbots require an introductory “trigger” text from the user to start working (and maybe present a welcome message). To get the conversation rolling, the text in this capability is sent to the endpoint before actually starting the conversation. The context is evaluated (see SIMPLEREST_CONTEXT_JSONPATH), but the text response is ignored.

SIMPLEREST_INIT_CONTEXT

The initial value for the session context.

Example: init context variable “conversation_id”:

{ "conversation_id": "none" }

SIMPLEREST_CONVERSATION_ID_TEMPLATE

Optional Mustache template. If it is not set, then an uuid will be generated.

Example: generating 13 digit long timestamp:

{{fnc.timestamp}}

SIMPLEREST_STEP_ID_TEMPLATE

Optional Mustache template. If it is not set, then an uuid will be generated.

Example: generating 8 digit long random number:

{{#fnc.random}}8{{/fnc.random}}

Plugging in Custom Functionality

UPDATE_CUSTOM ADD_QUERY_PARAM

The generic UPDATE_CUSTOM logic hook can be used to add additional query parameters to the URI from the convo file. You can use Mustache variables as well:

#me
Hello, World!
UPDATE_CUSTOM ADD_QUERY_PARAM|paramname1|paramvalue
UPDATE_CUSTOM ADD_QUERY_PARAM|paramname2|{{msg.location}}

#bot
Cool, I received additional query parameters from you!

The URI will now look something like this:

http://my-host/endpoint/msgText?paramname1=paramvalue&paramname2=mylocation

UPDATE_CUSTOM ADD_HEADER

The generic UPDATE_CUSTOM logic hook can be used to add additional headers to the HTTP request from the convo file. You can use Mustache variables as well:

#me
Hello, World!
UPDATE_CUSTOM ADD_HEADER|headername1|paramvalue
UPDATE_CUSTOM ADD_HEADER|headername2|{{msg.location}}

#bot
Cool, I received additional HTTP headers from you!

To add the additional headers for all requests in the current convo file, use the #begin section to set the additional headers:

#begin
UPDATE_CUSTOM ADD_HEADER|headername1|paramvalue
UPDATE_CUSTOM ADD_HEADER|headername2|{{msg.location}}

#me
Hello, World!

#bot
Cool, I received additional HTTP headers from you!

SIMPLEREST_REQUEST_HOOK

This is a dynamic way to change the request options right before it is sent out. You can use the same variables (context, msg, botium) as in Mustache. You can even change them, but usually you wont.

The format of the request options is described in request npm package

You can use this capability many ways:

  • use JavaScript code as string in botium.json

"SIMPLEREST_REQUEST_HOOK": "requestOptions.body = { bodyField: 'val', bodyField2: context.contextField }; context.contextField = 'new value'"
  • reference a JavaScript module in botium.json

"SIMPLEREST_REQUEST_HOOK": "my-custom-module"

the module must export a single function:

module.exports = ({ requestOptions, context }) => {
  requestOptions.body = { bodyField: 'val', bodyField2: context.contextField }
  context.contextField = 'new value'
}
  • reference a Javascript file in botium.json

"SIMPLEREST_REQUEST_HOOK": "connectors/simple/my-request-hook.js"

Again, the file has to export a single function (see above)

  • add direct Javascript function code, only when using botium-core as API

SIMPLEREST_REQUEST_HOOK: ({ requestOptions, context }) => {
  requestOptions.body = { bodyField: 'val', bodyField2: context.contextField }
  context.contextField = 'new value'
}

Hook Function Arguments:

  • requestOptions: full HTTP request including method, uri, body, headers (see here for details)

  • container: the current container instance

  • context: current session context

  • msg: input message

  • botium: conversationId, stepId (see Moustache variables)

SIMPLEREST_RESPONSE_HOOK

Connector extracts data from response like this if you use SIMPLEREST_RESPONSE_JSONPATH, and SIMPLEREST_BUTTONS_JSONPATH Capabilities:

{
  messageText: 'Choose please!',
  buttons: [
    'button1',
    'button2'
  ]
}

(See all extractable fields here)

Response hook is a general purpose way to read the response, and update this message object, using JavaScript.

Use them if you want:

  • Set a field which as no Capability.

  • Set a field which cant be set by its Capability. (Response hook is javascript code, so very flexible)

  • Process the response some way

  • Update the Mustache contexts (context, msg, botium). What you put there, you can use in Mustache templates later.

  • Overwrite a field is set already by other Response-Capability depending on a condition.

You can use this capability same way as SIMPLEREST_REQUEST_HOOK, just with botMsg instead of requestOptions

  • all configuration options apply for this hook as well (reference your own custom module and your own custom Javascript code, …)

"SIMPLEREST_RESPONSE_HOOK": "botMsg.nlp = {intent: {name: botMsg.sourceData.intent}};"

Most likely you want to use it to extract some custom values from the HTTP/JSON response body. You can access this JSON data as part of botMsg - botMsg.sourceData contains the full JSON response body.

Hook Function Arguments:

  • botMsg: add extracted values here

  • botMsgRoot: part of the JSON response pointing to the extracted section for this bot message

    • in case there are multiple botMsg extracted from one JSON response

    • available since Botium Box 2.1

  • messageTextIndex

    • available since Botium Box 2.1

  • container: the current container instance

  • context: current session context

  • msg: input message

  • botium: conversationId, stepId (see Moustache variables)

SIMPLEREST_START_HOOK/SIMPLEREST_STOP_HOOK

Those hooks work like the other hooks, they are called before starting any Botium conversation and after a conversation is finished and can be used to do some setup and teardown tasks.

Hook Function Arguments:

  • container: the current container instance

  • context: current session context

  • msg: input message

  • botium: conversationId, stepId (see Moustache variables)

HTTP Session Setup (“Ping” Request)

Botium will only start as soon as this URL is available (returns a non-error response) - for example, to wait until the bot service in the background is up and running. If there is JSON response returned, it will be added to the session context and can be used in the following Mustache templates.

Common scenario is that the ping request returns a JSON response containing a session id. This can be used in the following HTTP requests for session tracking. The payload of the HTTP response is otherwise ignored.

SIMPLEREST_PING_URL

HTTP-Url for the Ping call (can use Moustache template variables)

SIMPLEREST_PING_VERB

HTTP-Method for the Ping call - GET/POST/PUT/…

SIMPLEREST_PING_BODY

HTTP-Body for the Ping call (can use Moustache template variables)

SIMPLEREST_PING_BODY_RAW

Set to “false” to use plain text body instead of JSON for Ping call.

SIMPLEREST_PING_HEADERS

HTTP-Headers for the Ping call (can use Moustache template variables)

SIMPLEREST_PING_RETRIES

Number of times calling the Ping URL for a valid response (default 6)

SIMPLEREST_PING_TIMEOUT

HTTP Timeout and ping retry interval (default 10 seconds)

SIMPLEREST_PING_UPDATE_CONTEXT

Flag if ping response should be used to update the session context (default true)

SIMPLEREST_PING_PROCESS_RESPONSE

Flag if the JSON response should be handled as chatbot welcome message. If disabled, the response will be ignored. (default false)

HTTP Session Welcome (Start Request)

When starting a conversation, Botium will use this URL to send a first welcome message.

SIMPLEREST_START_URL

HTTP-Url for the Start call (can use Moustache template variables)

SIMPLEREST_START_VERB

HTTP-Method for the Start call - GET/POST/PUT/…

SIMPLEREST_START_BODY

HTTP-Body for the Start call (can use Moustache template variables)

SIMPLEREST_START_BODY_RAW

Set to “false” to use plain text body instead of JSON for Start call.

SIMPLEREST_START_HEADERS

HTTP-Headers for the Start call (can use Moustache template variables)

SIMPLEREST_START_RETRIES

Number of times calling the Start URL for a valid response (default 6)

SIMPLEREST_START_TIMEOUT

HTTP Timeout and retry interval (default 10 seconds)

SIMPLEREST_START_PROCESS_RESPONSE

Flag if the JSON response should be handled as chatbot welcome message. If disabled, the response will be ignored. (default true)

HTTP Session Teardown (“Stop” Request)

When ending a conversation, Botium will use this URL to teardown the session.

Commen scenario is that the stop request is used for ending a server-side session.

SIMPLEREST_STOP_URL

HTTP-Url for the Stop call (can use Moustache template variables)

SIMPLEREST_STOP_VERB

HTTP-Method for the Stop call - GET/POST/PUT/…

SIMPLEREST_STOP_BODY

HTTP-Body for the Stop call (can use Moustache template variables)

SIMPLEREST_STOP_BODY_RAW

Set to “false” to use plain text body instead of JSON for Stop call.

SIMPLEREST_STOP_HEADERS

HTTP-Headers for the Stop call (can use Moustache template variables)

SIMPLEREST_STOP_RETRIES

Number of times calling the Stop URL for a valid response (default 6)

SIMPLEREST_STOP_TIMEOUT

HTTP Timeout and retry interval (default 10 seconds)

HTTP(S) Inbound Messages

For chatbots delivering messages asynchronously, that means, not as response to an HTTP call, but by doing outbound calls to another HTTP endpoint, it is possible to connect it to Botium as well. Botium has to launch an additional HTTP endpoint where the chatbot has to post it’s responses.

There are capabilities to define what inbound messages are currently accepted by the Botium script running. Most likely, you will have some kind of session identifier included to let Botium know what messages belong together.

Response handling with JSONPaths for texts, media and buttons is the same as for the synchronous mode, see above.

When using Botium Box, the endpoint is integrated, you won’t have to launch any external service. The endpoint is reachable at this url, and a valid API Key has to be appended as query parameter:

http(s)://your-botium-box-url/api/inbound?APIKEY=…

In all other cases, you will have to either make Botium core launch it’s own internal API endpoint (using the SIMPLEREST_INBOUND_ENDPOINT and SIMPLEREST_INBOUND_PORT capabilities), or use Botium CLI to launch the endpoint (botium-cli inbound-proxy)

SIMPLEREST_INBOUND_REDISURL

Url of the Redis service to distribute

Not required with Botium Box, it always uses the default Redis service

SIMPLEREST_INBOUND_ENDPOINT and SIMPLEREST_INBOUND_PORT

Port and endpoint path to launch the endpoint

Not required with Botium Box, it launches it’s own endpoint

SIMPLEREST_INBOUND_SELECTOR_JSONPATH

A valid JSONPath selector for extracting the session identifier from the received message. The received message will be handed in, including the full URL of the HTTP request (for identifying session identifiers included in the URL), the HTTP method used and the full message body:

{
  "originalUrl": "/api/inbound/xxxx",
  "originalMethod": "POST",
  "body": {
    ...
  }
}

The result of the JSONPath selector is compared with the SIMPLEREST_INBOUND_SELECTOR_VALUE to decide if the message belongs to the current Botium session.

SIMPLEREST_INBOUND_SELECTOR_VALUE

A Mustache template for comparing the inbound selector result.

Most likely you will have something like a unique session id or a unique user id, maybe generated by Botium in the Mustache element botium.conversationId - so this configuration capability will be something like “{{botium.conversationId}}”

SIMPLEREST_INBOUND_UPDATE_CONTEXT

Flag if inbound responses should be used to update the session context (default true)

SIMPLEREST_INBOUND_ORDER_UNSETTLED_EVENTS_JSONPATH

Filling this capability a debounce wrapped function is activated. All inbound requests are collecting in a certain timeout (by default 500 ms) and order these requests in the debounce function by using the JSONPath set in the capability. This has to be valid JSONPath selector for extracting the values for ordering from the received messages. The received messages will be handed in, including the full URL of the HTTP request, the HTTP method used and the full message body:

{
  "originalUrl": "/api/inbound/xxxx",
  "originalMethod": "POST",
  "body": {
    ...
  }
}

The result of the JSONPath selector is used to order the request ascending. Normally it should be e.g. a timestamp in the body:

$.body.timestamp

SIMPLEREST_INBOUND_DEBOUNCE_TIMEOUT

A timeout for the debounce wrapped function described in the previous SIMPLEREST_INBOUND_ORDER_UNSETTLED_EVENTS_JSONPATH capability section. By default it is 500 millisec.

HTTP(S) Polling

Chatbots can deliver messages asynchronously by expecting the client to continuously poll for new messages available for a specific user or channel. Botium can do this polling and processes the response messages the same way as if received synchronously.

Response handling with JSONPaths for texts, media and buttons is the same as for the synchronous mode, see above.

SIMPLEREST_POLL_URL

Botium will poll this URL. If there is JSON response returned, this will be processed the same way as synchronous responses.

SIMPLEREST_POLL_VERB

HTTP-Method for the Poll call - GET/POST/PUT/…

SIMPLEREST_POLL_BODY

HTTP-Body for the Poll call (can use Moustache template variables)

SIMPLEREST_POLL_BODY_RAW

Set to “false” to use plain text body instead of JSON for Poll call.

SIMPLEREST_POLL_HEADERS

HTTP-Headers for the Poll call (can use Moustache template variables)

SIMPLEREST_POLL_TIMEOUT

HTTP Timeout for Poll request (default 10 seconds)

SIMPLEREST_POLL_INTERVAL

Polling interval (default 1 second)

SIMPLEREST_POLL_UPDATE_CONTEXT

Flag if polling responses should be used to update the session context (default true)

User Authentication

HTTP Basic Authentication can be done by adding username/passwort to the URLs - this works for all sections (ping, polling, request/response):

...
"SIMPLEREST_URL": "http://my-username:my-password@myhost.com/endpoint"
"SIMPLEREST_POLL_URL": "http://my-username:my-password@myhost.com/polling"
...

Token-based authentication can be done by adding HTTP headers - for example, from an environment variable:

...
"SIMPLEREST_HEADERS_TEMPLATE": {
  "Authorization": "Bearer {{fnc.env}}MY_TOKEN{{/fnc.env}}
},
...

A common scheme is to first generate a session token with an initial request, and use this for subsequent calls:

...
"SIMPLEREST_PING_URL": "some url",
"SIMPLEREST_PING_VERB": "POST",
"SIMPLEREST_PING_HEADERS": {
  "token": "{{#fnc.env}}MY_TOKEN{{/fnc.env}}"
},
"SIMPLEREST_PING_BODY": { some json content for the body },
...
"SIMPLEREST_URL": "...",
"SIMPLEREST_HEADERS_TEMPLATE": {
  "sessionid":"{{context.sessionid}}"
},
...

HTTP(S) Proxy Support

Set a HTTP(S) proxy by setting the capability SIMPLEREST_PROXY_URL to the full url of the proxy:

...
"SIMPLEREST_PROXY_URL": "http://myproxy.com:3128"
...

You can use proxy authentication as well:

...
"SIMPLEREST_PROXY_URL": "http://my-username:my-password@myproxy.com:3128"
...

Dealing with SSL Certificates

Setting the capability SIMPLEREST_STRICT_SSL to false (default: true) will disable the SSL certificate validity check (accepting outdated certificates)

In order to accept self-signed certificates or certificates not signed by an accepted CA, set the system environment variable (not Botium capability) NODE_TLS_REJECT_UNAUTHORIZED to 0.

Using Scripting Memory within Mustache Templates

You can use all Scripting Memory features of Botium in the Mustache templates. Scripting memory variables are available in the msg.scriptingMemory namespace, Scripting Memory functions are available in the fnc namespace.

Some Mustache examples

  • Using length scripting memory variable: {{msg.scriptingMemory.length}}

  • Using year function: {{fnc.year}}

  • Using random function with parameter: {{#fnc.random}}5{{/fnc.random}}

  • Using random function with parameter from scripting memory: {{#fnc.random}}{{msg.scriptingMemory.length}}{{/fnc.random}}

  • Using environment variable: {{#fnc.env}}MY_PERSONAL_TOKEN{{/fnc.env}}

    • Useful for handing over secrets like authentication headers

  • Executing code: {{#fnc.func}} 1 + 2 {{/fnc.func}}

  • Executing code from scripting memory: {{#fnc.func}}{{msg.scriptingMemory.javaScript}}{{/fnc.func}}