Siren Platform User Guide

X-Pack security integration

In order to use Investigate against an Elasticsearch cluster secured by X-Pack, you will need to create the following roles:

  • investigate_system: a role that allows Investigate to store user generated content.

  • investigate_admin: a role that designates users with administrative privileges on an Investigate installation.

  • investigate_user: a role that designates users with read access to specific indices.

  • federate_system: a role used by the Siren Federate plugin to perform privileged operations in the cluster.

The roles can be created by saving the bash script below as initroles.sh and executing it with bash initiroles.sh; the script will ask for the following information when executed:

  • Elasticsearch username: the username of an Elasticsearch user with administrative privileges (defaults to elastic).

  • Elasticsearch password: the password of the Elasticsearch user in the previous step (defaults to changeme).

  • Elasticsearch URL: the URL of your Elasticsearch cluster (defaults to http://localhost:9200).

  • Investigate index prefix: the prefix on Investigate indices (defaults to .siren).

  • Index pattern readable by Investigate users: an index pattern matching indices that will be readable by users having the investigate_user role.

  • curl flags: any custom curl flag that should be set in requests to Elasticsearch (for example -k to ignore validation of private CA certificates or --cacert <ca.pem> to validate certificates signed by a private CA)

#!/bin/bash
set -e

read -p "Elasticsearch username: [elastic]: " ES_USERNAME
ES_USERNAME=${ES_USERNAME:-elastic}

read -p "Elasticsearch password: " ES_PASSWORD
ES_PASSWORD=${ES_PASSWORD:-changeme}

read -p "Elasticsearch URL [http://localhost:9200]: " ES_URL
ES_URL=${ES_URL:-http://localhost:9200}

read -p "Investigate index prefix [.siren]: " INVESTIGATE_PREFIX
INVESTIGATE_PREFIX=${INVESTIGATE_PREFIX:-.siren}

read -p "Index pattern readable by Investigate users [data-*]:" DATA_INDICES
DATA_INDICES=${DATA_INDICES:-"data-*"}

read -p "curl flags: " CURL_FLAGS

echo "Creating investigate_system role..."

curl $CURL_FLAGS -XPUT -u $ES_USERNAME:$ES_PASSWORD $ES_URL/_xpack/security/role/investigate_system -H "Content-Type: application/json" -d '{
    "cluster": [
      "cluster:internal/federate/*",
      "cluster:admin/federate/*",
      "cluster:monitor/*",
      "manage_index_templates"
    ],
    "indices": [
      {
        "names": [
          "/\\'${INVESTIGATE_PREFIX}'.*/"
        ],
        "privileges": [
          "all"
        ]
      },
      {
        "names": [
          "watcher",
          "/watcher_alarms.*/"
        ],
        "privileges": [
          "all"
        ]
      },
      {
        "names": [
          "*"
        ],
        "privileges": [
          "indices:data/read*",
          "indices:admin/template/get",
          "indices:admin/aliases/get",
          "indices:admin/aliases/exists",
          "indices:admin/get",
          "indices:admin/exists",
          "indices:admin/mappings/fields/get*",
          "indices:admin/mappings/get*",
          "indices:admin/mappings/federate/connector/get*",
          "indices:admin/mappings/federate/connector/fields/get*",
          "indices:admin/types/exists",
          "indices:admin/validate/query",
          "indices:monitor/settings/get"
        ]
      }
    ]
  }
}'

echo
echo

echo "Creating federate_system role..."

curl $CURL_FLAGS -XPUT -u $ES_USERNAME:$ES_PASSWORD $ES_URL/_xpack/security/role/federate_system -H "Content-Type: application/json" -d '{
  "cluster": [
      "cluster:internal/federate/*",
      "cluster:admin/federate/*",
      "cluster:monitor/*"
    ],
    "indices": [
      {
        "names": [
          "/\\'${INVESTIGATE_PREFIX}'.*/"
        ],
        "privileges": [
          "all"
        ]
      },
      {
        "names": [
          "*"
        ],
        "privileges": [
          "indices:monitor/*",
          "indices:admin/*",
          "indices:data/read*",
          "indices:data/write*"
        ]
      }
    ]
}'

echo
echo

echo "Creating investigate_user role"

curl $CURL_FLAGS -XPUT -u $ES_USERNAME:$ES_PASSWORD $ES_URL/_xpack/security/role/investigate_user -H "Content-Type: application/json" -d '{
  "cluster": [
    "cluster:internal/federate/*"
  ],
  "indices": [
    {
      "names": [
        "$DATA_INDICES"
      ],
      "privileges": [
        "indices:data/read*",
        "indices:admin/aliases/get",
        "indices:admin/aliases/exists",
        "indices:admin/get",
        "indices:admin/exists",
        "indices:admin/mappings/fields/get*",
        "indices:admin/mappings/get*",
        "indices:admin/mappings/federate/connector/get*",
        "indices:admin/mappings/federate/connector/fields/get*",
        "indices:admin/types/exists",
        "indices:admin/validate/query",
        "indices:monitor/settings/get",
        "indices:admin/template/get"
      ]
    }
  ]
}'

echo
echo

echo "Creating investigate_admin role"

curl $CURL_FLAGS -XPUT -u $ES_USERNAME:$ES_PASSWORD $ES_URL/_xpack/security/role/investigate_admin -H "Content-Type: application/json" -d '{
  "cluster": [
    "cluster:internal/federate/*",
    "cluster:admin/federate/*",
    "cluster:monitor/*",
    "cluster:admin/xpack/security/*"
  ],
  "indices": [
    {
      "names": [
        "*"
      ],
      "privileges": [
        "indices:monitor/*",
        "indices:admin/*",
        "indices:data/read*"
      ]
    }
  ]
}'

echo
echo

Once the roles have been created, you'll need to create the following users:

  • sirenserver: a user with the investigate_system role.

  • sirenadmin: a user with the investigate_admin role.

  • sirenuser: a user with the investigate_user role.

  • federate: a user with the federate_system role.

The following script can be used to create the above users interactively:

#!/bin/bash
set -e

read -p "Elasticsearch username: [elastic]: " ES_USERNAME
ES_USERNAME=${ES_USERNAME:-elastic}

read -p "Elasticsearch password: " ES_PASSWORD
ES_PASSWORD=${ES_PASSWORD:-changeme}

read -p "Elasticsearch URL [http://localhost:9200]: " ES_URL
ES_URL=${ES_URL:-http://localhost:9200}

read -p "Password of sirenserver user [password]:" SIRENSERVER_PASSWORD
SIRENSERVER_PASSWORD=${SIRENSERVER_PASSWORD:-"password"}

read -p "Password of federate user [password]:" FEDERATE_PASSWORD
FEDERATE_PASSWORD=${FEDERATE_PASSWORD:-"password"}

read -p "Password of sirenuser user [password]:" SIRENUSER_PASSWORD
SIRENUSER_PASSWORD=${SIRENUSER_PASSWORD:-"password"}

read -p "Password of sirenadmin user [password]:" SIRENADMIN_PASSWORD
SIRENADMIN_PASSWORD=${SIRENADMIN_PASSWORD:-"password"}

read -p "curl flags: " CURL_FLAGS

echo "Creating sirenserver user"

curl $CURL_FLAGS -XPUT -u $ES_USERNAME:$ES_PASSWORD $ES_URL/_xpack/security/user/sirenserver -H "Content-Type: application/json" -d '{
  "password" : "'$SIRENSERVER_PASSWORD'",
  "full_name": "Siren Server",
  "roles": [ "investigate_system" ]
}'

echo
echo

echo "Creating sirenadmin user"

curl $CURL_FLAGS -XPUT -u $ES_USERNAME:$ES_PASSWORD $ES_URL/_xpack/security/user/sirenadmin -H "Content-Type: application/json" -d '{
  "password" : "'$SIRENADMIN_PASSWORD'",
  "full_name": "Siren Admin",
  "roles": [ "investigate_admin" ]
}'

echo
echo

echo "Creating sirenuser user"

curl $CURL_FLAGS -XPUT -u $ES_USERNAME:$ES_PASSWORD $ES_URL/_xpack/security/user/sirenuser -H "Content-Type: application/json" -d '{
  "password" : "'$SIRENUSER_PASSWORD'",
  "full_name": "Siren User",
  "roles": [ "investigate_user" ]
}'

echo
echo

echo "Creating federate system user"

curl $CURL_FLAGS -XPUT -u $ES_USERNAME:$ES_PASSWORD $ES_URL/_xpack/security/user/federate -H "Content-Type: application/json" -d '{
  "password" : "'$FEDERATE_PASSWORD'",
  "full_name": "Federate System",
  "roles": [ "federate_system" ]
}'

echo
echo

Once the users have been created, investigate.yml will have to be modified as follows.

1. Set elasticsearch.username and elasticsearch.password to the credentials of the sirenserver user, for example:

elasticsearch.username: sirenserver
elasticsearch.password: password

2. If HTTPS is enabled for the Elasticsearch REST API, ensure that the elasticsearch.url setting contains a URL starting with https, for example:

elasticsearch.url: 'https://localhost:9220'

3. If the certificate is not signed by a public authority, you will also need to set the elasticsearch.ssl.certificateAuthorities to the path of the CA chain bundle in PEM format, for example:

elasticsearch.ssl.certificateAuthorities: 'pki/xpack.pem'

4. To enable certificate verification, set elasticsearch.ssl.verificationMode to full, for example:

elasticsearch.ssl.verificationMode: full

5. Set the backend parameter of the investigate_access_control section of the investigate.yml to xpack:

investigate_access_control:
  admin_role: investigate_admin
  enabled: true
  backend: xpack
  acl:
    enabled: true
  cookie:
    secure: true
    password: '12345678123456781234567812345678'

If you are running Investigate with https disabled, remember to set investigate_access_control.cookie.secure to false, as otherwise the cookie won't be send to the browser.

You should now be able start Investigate and login as either sirenadmin or sirenuser using the password set previously.

Granting existing users the ability to use Federate on data indices

In order to allow existing users/role to use Siren Federate in Investigate or Elasticsearch queries in general, they will need to have the following cluster privileges:

  • cluster:internal/federate/*

In addition, users will need the following index privileges:

  • indices:data/read*

  • indices:admin/aliases/exists

  • indices:admin/get

  • indices:admin/exists

  • indices:admin/mappings/fields/get*

  • indices:admin/mappings/get*

  • indices:admin/mappings/federate/connector/get*

  • indices:admin/mappings/federate/connector/fields/get*

  • indices:admin/types/exists

  • indices:admin/validate/query

  • indices:monitor/settings/get

  • indices:admin/template/get

The following is an example role definition for a user that has access to all indices starting with data-:

{
    "cluster": [
      "cluster:internal/federate/*"
    ],
    "indices": [
      {
        "names": [
          "data-*"
        ],
        "privileges": [
          "indices:data/read*",
          "indices:admin/aliases/get",
          "indices:admin/aliases/exists",
          "indices:admin/get",
          "indices:admin/exists",
          "indices:admin/mappings/fields/get*",
          "indices:admin/mappings/get*",
          "indices:admin/mappings/federate/connector/get*",
          "indices:admin/mappings/federate/connector/fields/get*",
          "indices:admin/types/exists",
          "indices:admin/validate/query",
          "indices:monitor/settings/get",
          "indices:admin/template/get"
        ]
      }
    ]
}
Configuring Federate for virtual indices support

To enable support for Virtual Indices, you'll need to set the following configuration options in elasticsearch.yml:

  • siren.connector.username: the username of the Federate system user

  • siren.connector.password: the password of the Federate system user.

For additional information about Virtual Indices, please check the section Working with JDBC datasources .Working with JDBC datasources