- 25 Mar 2024
- 9 Minutes To Read
- Print
- DarkLight
- PDF
Streaming API
- Updated On 25 Mar 2024
- 9 Minutes To Read
- Print
- DarkLight
- PDF
The Streaming API is a system to system communication tool that uses a pull-based communication strategy to allow you to stream large amounts of data out of Mambu.
There are two stages to working with the Streaming API. You must first create your event streaming notification templates in the Mambu UI. Then the Streaming API acts as an intermediary storage location. When an event happens in Mambu for a topic that you have set an event streaming notification for, Mambu stores it in the Streaming API.
Afterwards, your client applications can use the Streaming API endpoints to establish connections with the Streaming API in order to read the event as many times as they need. For more detailed documentation about the API endpoints, see Streaming API in our API Reference.
Webhooks are another system-to-system communication tool. However, they use a push-based communication strategy and therefore their infrastructure does not support the same amount of data as the Streaming API. If you have multiple client applications that need to consume events from Mambu then we recommend using the Streaming API as opposed to webhooks. For more information, see Webhooks Overview.
In this article we will use the term client application to refer to the application or consumer that consumes events from the Streaming API.
Creating event streaming notification templates
You must create event streaming notification templates in order to define the payload for each event type.
To create an event streaming notification template:
- On the main menu, go to Administration > Events Streaming.
- Select Add Notification.
- Enter all the necessary information. More information on fields below.
- Select Save Changes.
Fields for event streaming notification templates
Name
The human readable name identifying this event in your list. It must be unique and must be a maximum of 255 characters.
Subscription option
Defines whether individual clients or groups are automatically subscribed to trigger a notification when a certain event occurs.
If you select Opt-Out, then individual clients and groups are automatically subscribed to trigger notifications. This is the most common option so that you are informed of actions that happen for all your individual clients and groups.
If you select Opt-In, then you must manually subscribe individual clients and groups to trigger notifications. For more information, see Subscribing or unsubscribing a client from an event stream notification.
Target
Defines what entity is being monitored to trigger this event streaming notification. This will have an effect on what events are available for triggering this notification. The entities available are individual clients, groups, end of day processing, and data access.
If you have the Payments capability enabled, then you will also have the payment order target available. For more information, see Payment events.
Event trigger
The available event triggers depend on the selected Target. For more information on each event, see Event Triggers for Notifications.
Conditions
You may create additional conditions that need to be met in order for an event streaming notification to be created.
Most conditions are made up of a field or piece of information that is monitored, an operator, and a value.
To set up a condition:
- Select the field or piece of information to monitor. The available options will depend on the target, recipient, and event trigger selected.
- Select the operator. The available options will depend on the option you selected in step 1.
- If appropriate, enter the value(s).
To remove conditions, select the Delete icon .
To create more conditions, select Add Condition.
In the dropdown, selecting Match All sends a notification if all of the conditions are met. Selecting Match Any sends a notification if at least one of the conditions is met.
For example, you may want to create an event stream notification when a deposit is made but only when the deposit amount is greater than USD1,000.
Topic
The topic field is automatically generated by Mambu and it represents the name of the channel to which Mambu notifications are submitted. It consists of the Mambu instance and the event trigger chosen. The topic is synonymous to the event type and this is the value that you must enter in the event_types
array when creating a subscription. For more information, see Create a subscription.
Contents
The content to be used as the payload for the event streaming notification. For JSON or XML content types you are advised to validate your payload using an external service such as JSON Lint before saving as there is no validation done on this field by the Mambu system.
You use placeholders to build your payloads. For more information, see Placeholders.
Set up API key for Streaming API requests
To use the Streaming API you must create an API consumer that has the Manage Events Streaming (MANAGE_EVENTS_STREAMING
) permission assigned to it and create API keys using this API consumer. For more information, see API Consumers and Streaming API - Authentication.
Using the Streaming API
Once you have set up the event streaming notification templates you may start consuming the events from the Streaming API.
There are four steps to working with the Streaming API:
- Create a subscription
- Open an event stream
- Commit cursor
- Delete a subscription
Below we go into more detail into each step and later we provide an example.
Create a subscription
To create a subscription the client application must make a POST
request to the /subscriptions
endpoint. For more information, see Create a Subscription in the API Reference.
In the event_types
array you must define the events you would like to listen to for this subscription. The value you must enter in the event_types
array is the topic value that Mambu generated when you set up the event streaming notification template. For more information, see Topic above.
One subscription may contain multiple topics. You may have three client applications consume events per topic.
Once you create a subscription, you will receive subscription ID in the id
property of the response payload. This value is necessary in the next step, when your client application opens an event stream.
Open an event stream
To open an event stream the client application must make a GET
request to the /subscriptions/{subscription_id}/events
endpoint specifying the subscription ID it received in the response payload when it made the subscription. Additionally, there are query parameters that allow your client application to specify the behaviour of your batch generation. For more information, see Start Event Stream in the API Reference.
Once your client applications open a stream, Mambu is going to generate batches containing events and a cursor. The default behaviour is that if no events occur for the topics specified in the subscription then Mambu will send empty batches in order to keep the connection between the Streaming API and the client application alive. To change this behaviour you may set another value for the stream_keep_alive_limit
query parameter.
Whenever events occur in the Mambu application that your client application is listening for, those events will be included in an event batch. Your client application may go to each event in the batch and carry out any processing it needs to. Once it has completed the processing it may proceed to the next step to commit the cursor.
Committing the cursor
To commit the cursor the client application must make a POST
request to the /subscriptions/{subscription_id}/cursors
endpoint. In the body of the request payload it must send the cursor object that it received in the event batch. This communicates back to Mambu that the client application has completed the processing of the batch and Mambu no longer needs to send this batch. For more information, see Commit Cursors in the API Reference.
Example
Create a subscription
Create a subscription specifying the event types you want to read. The event type value is represented by the topic value in the notification template form. The client application may subscribe to one or multiple topics.
Single event type
In the example below the client application is creating a subscription to just one event topic.
POST: https://TENANT_NAME.[env].mambu.com/api/v1/subscriptions
```json
{
"owning_application": "streaming-api-",
"event_types": ["mrn.event.testapp.streamingapi.savings_deposit"]
}
Response: Status: 200 OK
{
"owning_application": "streaming-api-",
"event_types": ["mrn.event.testapp.streamingapi.savings_deposit"
],
"consumer_group": "default",
"read_from": "end",
"initial_cursors": [],
"authorization": null,
"id": "[subscriptionID]",
"created_at": "2019-03-05T09:59:30.057Z",
"updated_at": "2019-03-05T09:59:30.057Z"
}
```
Multiple event types (single subscription ID is generated for the event types):
In the example below the client application is creating a subscription to two topics.
POST: https://TENANT_NAME.[env].mambu.com/api/v1/subscriptions
```json
{
"owning_application": "streaming-api-",
"event_types": ["mrn.event.testapp.streamingapi.savings_deposit", "mrn.event.testapp.streamingapi.client_rejected"]
}
Response: Status: 200 OK
{
"owning_application": "streaming-api-",
"event_types": [
"mrn.event.testapp.streamingapi.client_rejected",
"mrn.event.testapp.streamingapi.savings_deposit"
],
"consumer_group": "default",
"read_from": "end",
"initial_cursors": [],
"authorization": null,
"id": "[subscriptionID]",
"created_at": "2019-03-11T11:32:19.107Z",
"updated_at": "2019-03-11T11:32:19.107Z"
}
```
Start an event stream
In the example below, we start an event stream and receive an event batch with an object containing one event that was triggered for the events that were part of the subscription as well as the cursor.
GET https://[TENANT_NAME.environment]/api/v1/subscriptions/{subscription_id}/events
```json
{
"cursor": {
"event_type": "mrn.event.testapp.streamingapi.savings_deposit",
"cursor_token": "[cursorToken]",
"offset": "001-0001-000000000000000002",
"partition": "1"
},
"events": [
{
"metadata": {
"eid": "5119bd1b-973f-44ae-8d2d-314169619282",
"occurred_at": "2019-03-11T13:30:30.497Z",
"content_type": "text/plain; charset=UTF-8",
"category": "DATA",
"event_type": "mrn.event.testapp.streamingapi.savings_deposit"
},
"template_name": "group-deposit",
"body": ""
}
]
}
```
Commit the cursor
Next, to communicate that the events in the batch have been processed, the client application commits the cursors found in the event batch back to Mambu, which will store the offsets.
To commit the cursor, the generated ID is passed as a request header with the name X-Mambu-StreamId
.
X-Mambu-StreamId: [fieldValue]
POST: https://TENANT_NAME.[env].mambu.com/api/v1/subscriptions/[subscriptionID]/cursors
{
"items": [
{
"event_type": "mrn.event.testapp.streamingapi.savings_deposit",
"cursor_token": "[cursorToken]",
"offset": "001-0001-000000000000000001",
"partition": "1"
}
]
}
Once a subscription is read and the cursor is committed it is no longer displayed in the logs. The cursor should be committed in max 60 seconds, otherwise the POST
request should be re-done.
If the connection is closed, and later restarted, client applications will get events from the point of the last cursor commit.
Subscribing or unsubscribing a client from an event stream notification
The subscription option of event stream notifications determines whether a client is automatically subscribed to trigger a notification. For more information, see Subscription option.
To edit which event stream notifications a client is subscribed to trigger:
- Go to the profile of the client.
- In the top right-hand corner select More > Set Notifications.
- In the Notification Requests dialog, select the notification that you would like the client to be subscribed to.
- Select Save Changes.
The notifications in the Notification Requests dialog are listed by the name of the event trigger and in parentheses they list whether they are SMS, email, webhook, or event stream notifications.
Control over which notifications a client or group is subscribed to trigger is managed at the level of the event trigger. For example, if you three different event stream notifications set up for the Client Activity event trigger, only one Client Activity (Event Stream) checkbox will appear in the Notification Requests dialog. You have to select whether you want to subscribe a client or group to trigger all of them or none of them.