Watchers
A Siren Alert watcher is created using the following structure:
➔ Trigger Schedule
When and How to run the Watcher
➔ Input Query
What Query or Join Query to Execute
➔ Condition
How to conditionally Analyze Response
➔ Transform
How to Adapt or Post-Process data
➔ Actions
How to Notify users about this event
Trigger schedule
The schedule defines a set of constraints that must be met to execute a saved watcher. Any number of constraints can be added to a single schedule, and multiple rules can be combined to achieve complex intervals, programmed using simple text expressions using the Node.JS later module.
Interval exceptions can also be defined as follows:
every 2 hours except after 20th hour
Input query
The input parameter is the key element of a watcher, and defines a dynamic range index query feeding the circuit. The input field accepts any standard Elasticsearch query including server side scripts in supported languages and fully supports the Siren Join capabilities out of the box.
"input": {
"search": {
"request": {
"index": [
"<mos-{now/d}>",
"<mos-{now/d-1d}>"
],
"body": {}
}
}
}
Condition
The condition block is the "entry gate" into the processing pipeline of a Watcher and determines its triggered status.
-
On
true
condition, the pipeline will proceed further. -
On
false
condition, the pipeline will stop (no action will be executed) until its next invocation.
Never condition
Use the never
condition to set the condition to false
. This
means the watch actions are never executed when the watch is triggered.
Nevertheless, the watch input is executed. This condition is used for
testing. There are no attributes to specify for the never
condition.
condition: { "never" : {} }
Compare condition
Use the compare
condition to perform a simple comparison against a
value in the watch payload.
condition: { "compare" : { "payload.hits.total" : { "gte" : 5 } }
Comparison operators (apply to numeric, string and date).
eq |
Returns true when the resolved value equals the given one |
---|---|
|
Returns true when the resolved value does not equal the given one |
|
Returns true when the resolved value is less than the given one |
|
Returns true when the resolved value is less/equal than/to the given one |
|
Returns true when the resolved value is greater than the given one |
|
Returns true when the resolved value is greater/equal than/to the given one |
Array compare condition
Use array_compare
to compare an array of values. For example, the
following array_compare condition returns true
if there is at least
one bucket in the aggregation that has a doc_count
greater than or
equal to 25:
"condition": { "array_compare": { "payload.aggregations.top_amounts.buckets" : { "path": "doc_count" , "gte": { "value": 25, } } } }
Options
|
The path to the array in the execution context, specified in dot notation. |
|
The path to the field in each array element that you want to evaluate. |
|
How many matches are required for
the comparison to evaluate to true: |
|
The value to compare against. |
Script condition
A condition that evaluates a script. The scripting language is JavaScript. It can be as simple as a function expecting a Boolean condition or counter.
condition: { "script": { "script": "payload.hits.total > 100" } }
Or it can be as complex as an aggregation parser to filter buckets.
condition: { "script": { "script": "payload.newlist=[];var match=false;var threshold=10;var start_level=2;var finish_level=3;var first=payload.aggregations[start_level.toString()];function loop_on_buckets(element,start,finish,upper_key){element.filter(function(obj){return obj.key;}).forEach( function ( bucket ) { if (start == finish - 1) { if (bucket.doc_count >= threshold) { match=true;payload.newlist.push({line: upper_key + bucket.key + ' ' + bucket.doc_count}); } } else { loop_on_buckets(bucket[start + 1].buckets, start + 1, finish, upper_key + ' ' + bucket.key); } }); } var upper_key = ''; loop_on_buckets(first.buckets, start_level, finish_level, upper_key);match;" } }
Anomaly detection
Simple anomaly finder based on the three-sigma rule.
-
Dynamic detection of outliers/peaks/drops:
{ "script": { "script": "payload.hits.total > 0" }, "anomaly": { "field_to_check": "fieldName" } }
-
Static detection for known ranges/interrupts:
{ "script": { "script": "payload.hits.total > 0" }, "anomaly": { "field_to_check": "fieldName", "normal_values": [ 5, 10, 15, 20, 25, 30 ] } }
Range filtering
Use for getting documents which have a value in between some values. For
example, get only the documents which have values from 45 to 155 inside
Amount
field.
{ "script": { "script": "payload.hits.total > 0" }, "range": { "field_to_check": "Amount", "min": 50, "max": 150, "tolerance": 5 } }
Transform
A transform processes and changes the payload in the watch execution context to prepare it for the watch actions. No actions are executed in the case that the payload is empty after transform processing.
Search transform
A transform that executes a search on the cluster and replaces the current payload in the watch execution context with the returned search response.
"transform": { "search": { "request": { "index": [ "credit_card" ], "body": { "size": 300, "query": { "bool": { "must": [ { "match": { "Class": 1 } } ] } } } } } }
Script transform
A transform that executes a script (JavaScript) on the current payload and replaces it with a newly generated one. Its uses include:
-
Converting format types.
-
Generating new payload keys.
-
Interpolating data
Create new payload property:
"transform": { "script": { "script": "payload.outliers = payload.aggregations.response_time_outlier.values['95.0']" } }
Filter aggregation buckets:
"transform": { "script": { "script": "payload.newlist=[]; payload.payload.aggregations['2'].buckets.filter(function( obj ) { return obj.key; }).forEach(function(bucket){ console.log(bucket.key); if (doc_count.length > 1){ payload.newlist.push({name: bucket.key }); }});" } }
Chain transform
A transform that executes an ordered list of configured transforms in a chain, where the output of one transform serves as the input of the next transform in the chain.
"transform": { "chain": [ { "search": { "request": { "index": [ "credit_card" ], "body": { "size": 300, "query": { "bool": { "must": [ { "match": { "Class": 1 } } ] } } } } } }, { script: { script: "payload.hits.total > 100" } } ] }
Actions
Actions are used to deliver any results obtained by a Watcher to users, APIs or new documents in the cluster. Multiple Actions and Groups can be defined for each.
Actions use the {{ mustache }}
logic-less template syntax, and work
by iterating arrays and expanding tags in a template using values
provided in the response payload.
For more details, see Supported actions.
Full watcher example
{
"_index": "watcher",
"_type": "watch",
"_id": "new",
"_source": {
"trigger": {
"schedule": {
"later": "every 5 minutes"
}
},
"input": {
"search": {
"request": {
"index": [
"<mos-{now/d}>",
"<mos-{now/d-1d}>"
],
"body": {}
}
}
},
"condition": {
"script": {
"script": "payload.hits.total > 100"
}
},
"transform": {
"script": {
"script": "payload.hits.total += 100"
}
},
"actions": {
"email_admin": {
"throttle_period": "15m",
"email": {
"to": "alarm@localhost",
"subject": "Siren Alert Alarm",
"priority": "high",
"body": "Found {{payload.hits.total}} Events"
}
},
"slack_admin": {
"throttle_period": "15m",
"slack": {
"channel": "#kibi",
"message": "Siren Alert Alert! Found {{payload.hits.total}} Events"
}
}
}
}
}
Supported actions
Following are the currently supported "actions" for Siren Alert watchers.
Note that the actions which have active: true
in their config are not enabled unless the active property is set to true.
Send Query results and message using Email/SMTP.
Requires action settings in Siren Investigate configuration.
"email" : {
"active": true,
"to" : "root@localhost",
"from" : "sirenalert@localhost",
"subject" : "Alarm Title",
"priority" : "high",
"body" : "Series Alarm {{ payload._id}}: {{payload.hits.total}}",
"stateless" : false
}
Email HTML
Send Query results and message using Email/SMTP using HTML body.
Requires action settings in Siren Investigate configuration.
"email_html" : {
"to" : "root@localhost",
"from" : "sirenalert@localhost",
"subject" : "Alarm Title",
"priority" : "high",
"body" : "Series Alarm {{ payload._id}}: {{payload.hits.total}}",
"html" : "<p>Series Alarm {{ payload._id}}: {{payload.hits.total}}</p>",
"stateless" : false
}
webHook
Deliver a POST request to a remote web API
"webhook" : {
"active": true,
"method" : "POST",
"host" : "remote.server",
"port" : 9200,
"path": ":/{{payload.watcher_id}}",
"body" : "{{payload.watcher_id}}:{{payload.hits.total}}",
"create_alert" : true
"cacert" : "ca-certificate.pem",
"cert" : "certificate.pem",
"key" : "key.pem"
}
Deliver a GET request to a remote web API
"webhook" : {
"active": true,
"method" : "GET",
"host" : "remote.server",
"port" : 9200,
"path" : "/trigger",
"params" : {
"watcher": "{{watcher.title}}",
"query_count": "{{payload.hits.total}}"
},
"cacert" : "ca-certificate.pem",
"cert" : "certificate.pem",
"key" : "key.pem"
}
webHook using Proxy
Deliver message to remote API using Proxy - Telegram example:
"webhook": {
"active": true,
"method": "POST",
"host": "remote.proxy",
"port": "3128",
"path": "https://api.telegram.org/bot{botId}/sendMessage",
"body": "chat_id={chatId}&text=Count+total+hits:%20{{payload.hits.total}}",
"headers": {
"Content-Type": "application/x-www-form-urlencoded"
},
"create_alert" : true
}
Slack
Delivery Message to #Slack channel.
Requires action settings in Siren Investigate configuration.
"slack" : {
"active": true,
"channel": "#channel",
"message" : "Series Alarm {{ payload._id}}: {{payload.hits.total}}",
"stateless" : false
}
Report
Take a website snapshot using PhantomJS and send it using Email/SMTP.
-
Requires action settings in Siren Investigate configuration.
-
Requires Pageres/PhantomJS:
npm install -g pageres
. -
Requires Chromium browser to be installed. If not, then Siren downloads chromium provided the user has the permission to do so.
"report" : {
"active": true,
"to" : "root@localhost",
"from" : "kaae@localhost",
"subject" : "Report Title",
"priority" : "high",
"body" : "Series Report {{ payload._id}}: {{payload.hits.total}}",
"snapshot" : {
"res" : "1280,900",
"url" : "http://127.0.0.1/app/kibana#/dashboard/Alerts",
"path" : "/tmp/",
"params" : {
"username" : "username",
"password" : "password",
"delay" : 5000,
"crop" : false
}
},
"stateless" : false
}
Watcher controllers
The following controls are presented when listing existing Watchers:
-
Expand or edit a Watcher.
-
Manually execute a Watcher.
-
Remove a Watcher.
-
Toggle a Watcher on or off.
Watcher visibility
When security is enabled, users can define the visibility of a watcher, alarm or report for different groups of users.
As a watcher is a saved object its visibilty can be configured in the saved objects configuration page (see here).
The visibility of alarms and reports can be restricted in the Watcher editor using the privacy section. They can be set as follows:

To restrict the permissions of alarms and reports, you must use document level security (DLS) by adding the following settings to the sg_roles.yml
file of your searchguard security and running the sgadmin.sh
script:
alert_viewer_role:
indices:
'watcher_alarms-*':
'*':
- READ
- VIEW_INDEX_METADATA
_dls_: '{
"bool": {
"should": [
{
"term": {
"is_private": false
}
},
{
"bool": {
"must_not": [
{
"exists": {
"field": "is_private"
}
}
]
}
},
{
"terms": {
"roles": [${user.roles}]
}
},
{
"term": {
"created_by": "${user.name}"
}
}
]
}
}'
Examples
Watchers can be as simple or complex as the query and aggregations they use. Here are some examples to get started with.
Hit watcher
{
"_index": "watcher",
"_type": "watch",
"_id": "new",
"_source": {
"trigger": {
"schedule": {
"later": "every 5 minutes"
}
},
"input": {
"search": {
"request": {
"index": [
"<mos-{now/d}>",
"<mos-{now/d-1d}>"
],
"body": {}
}
}
},
"condition": {
"script": {
"script": "payload.hits.total > 100"
}
},
"transform": {},
"actions": {
"email_admin": {
"throttle_period": "15m",
"email": {
"to": "alarm@localhost",
"from": "sirenalert@localhost",
"subject": "Siren Alert Alarm",
"priority": "high",
"body": "Found {{payload.hits.total}} Events"
}
},
"slack_admin": {
"throttle_period": "15m",
"slack": {
"channel": "#kibi",
"message": "Siren Alert Alert! Found {{payload.hits.total}} Events"
}
}
}
}
}
Transform (Elasticsearch 2.x)
{
"_index": "watcher",
"_type": "watch",
"_id": "95th",
"_score": 1,
"_source": {
"trigger": {
"schedule": {
"later": "every 5 minutes"
}
},
"input": {
"search": {
"request": {
"index": [
"<access-{now/d}>",
"<access-{now/d-1d}>"
],
"body": {
"size": 0,
"query": {
"filtered": {
"query": {
"query_string": {
"analyze_wildcard": true,
"query": "*"
}
},
"filter": {
"range": {
"@timestamp": {
"from": "now-5m"
}
}
}
}
},
"aggs": {
"response_time_outlier": {
"percentiles": {
"field": "response_time",
"percents": [
95
]
}
}
}
}
}
}
},
"condition": {
"script": {
"script": "payload.aggregations.response_time_outlier.values['95.0'] > 200"
}
},
"transform": {
"script": {
"script": "payload.myvar = payload.aggregations.response_time_outlier.values['95.0']"
}
},
"actions": {
"email_admin": {
"throttle_period": "15m",
"email": {
"to": "username@mycompany.com",
"from": "sirenalert@mycompany.com",
"subject": "Siren Alert ALARM {{ payload._id }}",
"priority": "high",
"body": "Series Alarm {{ payload._id}}: {{ payload.myvar }}"
}
}
}
}
}
Siren Alert: insert back to Elasticsearch bulk (using nginx or direct).
{
"_index": "watcher",
"_type": "watch",
"_id": "surprise",
"_score": 1,
"_source": {
"trigger": {
"schedule": {
"later": "every 50 seconds"
}
},
"input": {
"search": {
"request": {
"index": "my-requests-*",
"body": {
"query": {
"filtered": {
"query": {
"query_string": {
"query": "*",
"analyze_wildcard": true
}
},
"filter": {
"range": {
"@timestamp": {
"from": "now-5m"
}
}
}
}
},
"size": 0,
"aggs": {
"metrics": {
"terms": {
"field": "first_url_part"
}
}
}
}
}
}
},
"condition": {
"script": {
"script": "payload.hits.total > 1"
}
},
"transform": {},
"actions": {
"ES_bulk_request": {
"throttle_period": "1m",
"webhook": {
"method": "POST",
"host": "elasticsearch.foo.bar",
"port": 80,
"path": ":/_bulk",
"body": "{{#payload.aggregations.metrics.buckets}}{\"index\":{\"_index\":\"aggregated_requests\", \"_type\":\"data\"}}\n{\"url\":\"{{key}}\", \"count\":\"{{doc_count}}\", \"execution_time\":\"tbd\"}\n{{/payload.aggregations.metrics.buckets}}",
"headers": {
"Content-Type": "text/plain; charset=ISO-8859-1"
},
"create_alert": true
}
}
}
}
}
Wizard
Siren Alert provides a wizard to help create watchers using a step-by-step sequence.
Give the watcher a name and choose an execution frequency. For example, run every day.
Specify the input query parameters and the condition to trigger on (based on a date histogram aggregation. For example, trigger an alert when there are more than two articles in an hour during the day.
To send an alert, set up a variety of actions to happen when your condition is met. For example, send a HTML-formatted email injected with data from the watcher and query response using the mustache templating language.
Custom watchers
Together with standard watchers, Siren Alert supports an additional list of use case-specific watcher types that are created from the dashboard. These types of watchers are called custom watchers.
Dashboard button
Custom watchers created from a dashboard inherit its search criteria, including the search pattern, search query, and filters. Each custom watcher type applies specific processing and trigger conditions.
Custom watcher types
-
New results: Alerts when there are new results for the given search.
-
Geo fence: Alerts when there are new results within a geographical area.
-
Proximity: Alerts when two entities are closer or further than a specified distance. This type must be enabled manually as it is not applicable to all data sets. Siren provides support for enabling this.
Creating custom watchers
Custom watchers are managed in the Script tab of the Management section. Creating a custom watcher requires writing a script that provides an object with the attributes described below. Siren provides support for creating these scripts.
Attribute | Type | Description |
---|---|---|
|
|
Used to place watcher type in dashboard box. |
|
|
Tests whether to show the watcher type in the dashboard box. |
|
|
The collection of parameters and default values required by the watcher. |
|
|
Executed in the backend that searches Elasticsearch using the
|
|
|
Evaluates the response from the search function and determines if the watcher should perform the alert actions. |
|
|
Deprecated: Use attributes below. The HTML template presenting inputs for the
watcher parameters. The |
|
|
The HTML template presenting the configuration panel. |
|
|
The HTML template presenting inputs for the
watcher parameters. The |
|
|
Converts the time that will be searched. Moment.js objects are passed in to facilitate relative date manipulation (e.g. |
|
|
Changes the time field used in the search query. If this is set, |
|
|
The global time range set on a dashboard by a watcher’s dashboard link. This is a mandatory field if |
|
|
Default: |
|
|
Defines the
If this is an empty array, the "Add action" button will be hidden. |
|
|
Allows passing of default values via custom watcher script. Currently only
or using data provided from the dashboard
|
|
|
Enables users to create bespoke validation rules for a watcher’s parameters. Throw an error if a parameter value is invalid, for example:
|
Specific custom watcher information
Additional watcher information is provided in the watcher variable and can be accessed on watcher actions.
Property | Type | Description |
---|---|---|
|
|
Dashboard link with the 'filtered by watcher' results |
|
|
ID for associated dashboard |
|
|
ID for associated index pattern |