Siren Platform User Guide

Anomaly detection

The Siren Alert anomaly detection mechanism is based on the three-sigma rule. In short, anomalies are the values which lie outside a band around the mean in a normal distribution with a width of two, four and six standard deviations (68.27%, 95.45% and 99.73%).

  1. Create a new watcher.
  2. In watcher editor, inside Input tab insert Elasticsearch query to get the credit card transactions data set.

    {
      "search": {
        "request": {
          "index": [
            "credit_card"
          ],
          "body": {
            "size": 10000,
            "query": {
              "bool": {
                "must": [
                  {
                    "exists": {
                      "field": "Amount"
                    }
                  }
                ]
              }
            }
          }
        }
      }
    }
  3. In the Condition tab specify a minimum number of results to look for payload.hits.total > 0 and a field name in which to look for anomalies, Amount in our example.

    {
      "script": {
    "script": "payload.hits.total > 0"
      },
      "anomaly": {
    "field_to_check": "Amount"
      }
    }
  4. In Action tab create email html action. In Body HTML field render all the anomalies you have in the payload.anomaly using mustache syntax.

    <h1 style="background-color:DodgerBlue;color:white;padding:5px">Anomalies</h1>
    <div style="background-color:Tomato;color:white;padding:5px">
    <ul>
    {{#payload.anomaly}}
    <li><b>id:</b> {{_id}} <b>Amount</b>: {{_source.Amount}}</li>
    {{/payload.anomaly}}
    </ul>
    </div>

As a result, we have an email with a list of anomaly transactions.

Anomaly detection

Also, the list of anomalies was indexed in today’s alert index watcher_alarms-{year-month-date}.

Watcher alarms