Siren Platform User Guide

Search Guard integration and Siren Investigate access control

This section offers an overview of how to integrate Search Guard with Siren Investigate; for further reference and detailed options, consult the Search Guard documentation.

Before proceeding, ensure that:

  • Siren Investigate is either running with HTTPS enabled or behind a reverse SSL proxy.
  • The Gremlin Server is running with HTTPS enabled.

For information on how on to enable HTTPS in both components, see Authentication and access control.

SSL certificates

All the Elasticsearch nodes in a cluster secured by Search Guard are required to use SSL to encrypt all network traffic.

In addition, changing the Search Guard configuration requires the use of a client SSL certificate to perform administrative actions.

To set up a Search Guard cluster, you must generate the following files:

  • truststore: Common to all nodes, containing the CA certificate chain.
  • keystore: For each node, containing the certificate for the node.
  • keystore: For each administrative user, containing a certificate bundle that identifies the user.
  • keystore: Containing an SSL certificate for the Elasticsearch HTTP server.

These files can be either Java KeyStore files or PKCS12 bundles.

Issuing certificates in an existing PKI infrastructure

If your organization has a PKI infrastructure in place, you can generate Java KeyStore files from a PEM bundle by using the keytool command, for example:

$ keytool  \
  -importcert \
  -file ca.pem  \
  -keystore truststore.jks

The command will store the contents of ca.pem into a file named truststore.jks in the current folder.

The same command can be used to convert certificates signed by your CA for nodes, administrative users and the REST API.

Node certificates must include oid:1.2.3.4.5.5 as a Subject Alternative Name entry to work correctly with Search Guard; for details on how to customize the OID, consult the Search Guard documentation.

If you want to enable hostname verification, ensure that at least one Subject Alternative Name is equal to the DNS name of the node.

Client certificates for administrative users must contain a unique Distinguished Name to identify the user, for example:

CN=admin,DC=siren,DC=solutions

Certificates for the Elasticsearch HTTP server can be used across multiple nodes by setting multiple hostnames in the Subject Alternative Name attribute or by using a wildcard certificate.

Issuing certificates using the TLS certificate generator

Floragunn GmbH provides a TLS certificate generation service which can be used to create a bundle of certificates for evaluation purposes.

To try the certificates in a single node setup, it is possible to specify localhost as the first hostname and submit the form.

The bundle contains:

  • README.txt: Provides an overview of the bundle and the passwords for all the keystores.
  • truststore.jks: the CA certificate chain in KeyStore format.
  • node-certificates: the transport certificates for the nodes in several formats; these certificates can also be used for the Elasticsearch HTTP server.
  • client-certificates: client certificates and private keys.
  • root-ca: the root CA bundle in PEM format.
  • signing-ca: the signing CA bundle in PEM format.

In addition to the online generator, Floragunn provides a TLS tool which can be used to manage a private certification authority.

Search Guard installation

Install the Search Guard plugin on each node in the Elasticsearch cluster by changing to the node folder and running the following commands (with the appropriate version number):

$ bin/elasticsearch-plugin install -b com.floragunn:search-guard-6:<version>

To find the most recent version of the plugins for your Elasticsearch version, consult the Search Guard version matrix.

Note

Elasticsearch requires a matching version of Search Guard. For example, Elasticsearch 6 requires Search Guard 6.

Then, copy the following files to the config folder of each node:

  • The truststore file (for example truststore.jks).
  • The keystore file containing the node certificate (for example CN=localhost-keystore.jks).
  • The keystore file containing the certificate for the Elasticsearch HTTP server, only if different from the node certificate.

Open the config/elasticsearch.yml file and set the following Search Guard options:

Node to node transport options

  • searchguard.ssl.transport.enabled: Must be set to true for Search Guard to work.
  • searchguard.ssl.transport.keystore_filepath: The filename of the keystore file that contains the node certificate.
  • searchguard.ssl.transport.keystore_password: The password of the keystore file that contains the node certificate.
  • searchguard.ssl.transport.truststore: The filename of the truststore file that contains the root certificate chain.
  • searchguard.ssl.transport.truststore_password: The password of the truststore file that contains the root certificate chain.
  • searchguard.ssl.transport.enforce_hostname_verification: Set to true to enable hostname verification, false otherwise.

REST API options:

  • searchguard.ssl.http.enabled: Set to true to enable SSL on the HTTP interface.
  • searchguard.ssl.http.keystore_filepath: The filename of the keystore file that contains the certificate for the HTTP interface.
  • searchguard.ssl.http.keystore_password: The password of the keystore file that contains the certificate for the HTTP interface.
  • searchguard.ssl.http.truststore: The filename of the truststore file that contains the root certificate chain for the HTTP certificate.
  • searchguard.ssl.http.truststore_password: The password of the truststore file that contains the root certificate chain for the HTTP certificate.

Administrative user options:

  • searchguard.authcz.admin_dn: A list of Distinguished Names in SSL client certificates which are authorized to submit administrative requests.

Client certificate authentication options:

  • searchguard.ssl.http.clientauth_mode: Set to OPTIONAL to enable optional client certificate authentication on the REST endpoint.

For example:

searchguard.ssl.transport.enabled: true
searchguard.ssl.transport.truststore_filepath: truststore.jks
searchguard.ssl.transport.truststore_password: <password>
searchguard.ssl.transport.keystore_filepath: CN=localhost-keystore.jks
searchguard.ssl.transport.keystore_password: <password>
searchguard.ssl.transport.enforce_hostname_verification: false
searchguard.ssl.http.enabled: true
searchguard.ssl.http.keystore_filepath: CN=localhost-keystore.jks
searchguard.ssl.http.keystore_password: <password>
searchguard.ssl.http.truststore_filepath: truststore.jks
searchguard.ssl.http.truststore_password: <password>
searchguard.authcz.admin_dn:
  - CN=sgadmin
searchguard.ssl.http.clientauth_mode: OPTIONAL

Note

Ensure that all the files in the configuration folder and the certificate files are readable only by the user running Elasticsearch.

Start Elasticsearch:

$ bin/elasticsearch

If either a certificate or a password is incorrect, Elasticsearch will not start.

Access control configuration

Access control configuration (users, roles and privileges) is stored in an Elasticsearch index which can be modified through the sgadmin.sh script.

The script reads the configuration from a local folder containing YAML files and uploads it to the index; the request is authenticated through a client SSL certificate.

After the configuration has been uploaded, it will be available to all the nodes in the cluster, so it is not necessary to copy the Search Guard configuration folder to all the Elasticsearch nodes, just on the node from where sgadmin is run.

sgadmin.sh is available in the plugins/search-guard-<version>/tools folder in each Elasticsearch instance in which Search Guard has been installed; a standalone version (sgadmin-standalone.zip) can be downloaded from https://github.com/floragunncom/search-guard/wiki#search-guard-admin-standalone.

After a Search Guard enabled cluster has been initialized, sgadmin can be used to upload new configurations.

Siren Investigate certificates

Here we give an example of where to store client certificates and keystores on Siren Investigate. Note: These are examples for a fresh install using the TLS certificate generator.

In siren-investigate/pki (which was created earlier for https support) a new folder searchguard was created with the following:

  • CN=sgadmin.crtfull.pem: a certificate bundle with administrative privileges over the Search Guard Management REST API. Copied from client-certificates in TLS certificate generator bundle.
  • CN=sgadmin.key.pem: the key of the administrative certificate. Copied from client-certificates in TLS certificate generator bundle.
  • ca.pem: the cluster CA certificate chain in PEM format. Copy of root-ca.pem from top level folder TLS certificate generator bundle.
  • CN=sgadmin-keystore.jks: Keystore containing the admin certificate. Used with sgadmin. Copied from client-certificates in TLS certificate generator bundle.

The password of all Java keystores can be found in the README.txt from top level folder of TLS certificate generator bundle.

Search Guard configuration

A Search Guard configuration folder contains the following files:

  • sg_config.yml: General configuration.

  • sg_action_groups.yml: Named groups of permissions.

  • sg_roles.yml: Definition of roles.

  • sg_internal_users.yml: Search Guard internal user database.

  • sg_roles_mapping.yml: Mapping between users and roles.

The following sample configuration is used for Elasticsearch with no data, for example your own Elasticsearch or using our no-data-no-security package. Further examples are available in the config/sgconfig folder in the Elasticsearch instance included in the demonstration distribution.

Note

There are differences between the following documented examples and the Search Guard configurations provided in the different Siren distributions. For example, Siren Platform with Demo Data has security but it is permissive to allow easy deployment. The examples can be used as general guidelines.

For additional configuration options, refer to the Search Guard documentation.

General configuration

sg_config.yml

searchguard:
  dynamic:
    respect_request_indices_options: true
    kibana:
      do_not_fail_on_forbidden: true
    http:
      anonymous_auth_enabled: false
      xff:
        enabled: false
    authc:
      transport_auth_domain:
        enabled: true
        order: 2
        http_authenticator:
          type: basic
        authentication_backend:
          type: internal
      basic_internal_auth_domain:
        enabled: true
        http_authenticator:
          type: basic
          challenge: true
        authentication_backend:
          type: intern

The sg_config.yml file contains the configuration of the authentication mechanisms and backends; this configuration:

  • Switches off the anonymous role (anonymous_auth_enabled: false).

  • Switches off support for external proxies (xff.enabled: false).

  • Enables HTTP basic authentication on the internal Search Guard user database.

Action groups

sg_action_groups.yml

UNLIMITED:
  - "*"

###### INDEX LEVEL ######

INDICES_ALL:
  - "indices:*"

# for backward compatibility
ALL:
  - INDICES_ALL

MANAGE:
  - "indices:monitor/*"
  - "indices:admin/*"

# for backward compatibility
MONITOR:
  - MANAGE

WRITE:
  - "indices:data/write*"
  - "indices:admin/mapping/put"

READ:
  - "indices:data/read*"

VIEW_INDEX_METADATA:
  - "indices:admin/aliases/get"
  - "indices:admin/aliases/exists"
  - "indices:admin/get"
  - "indices:admin/exists"
  - "indices:admin/mappings/fields/get*"
  - "indices:admin/mappings/get*"
  - "indices:admin/mappings/federate/connector/get*"
  - "indices:admin/mappings/federate/connector/fields/get*"
  - "indices:admin/types/exists"
  - "indices:admin/validate/query"
  - "indices:monitor/settings/get"

###### CLUSTER LEVEL ######

CLUSTER_ALL:
  - "cluster:*"

CLUSTER_MONITOR:
  - "cluster:monitor/*"

CLUSTER_COMPOSITE_OPS:
  -  CLUSTER_COMPOSITE_OPS_RO
  - "indices:data/write/bulk"

CLUSTER_COMPOSITE_OPS_RO:
  - "indices:data/read/mget"
  - "indices:data/read/msearch"
  - "indices:data/read/mtv"
  - "indices:data/read/scroll*"

CLUSTER_MANAGE:
  - CLUSTER_INTERNAL_FEDERATE
  - "cluster:admin/federate/*"
  - "indices:admin/aliases*"

CLUSTER_INTERNAL_FEDERATE:
  - "cluster:internal/federate/*"

The file sg_action_groups.yml contains named groups of permissions which can be used in the role configuration file including Search Guard’s default groups and Siren Investigate specific groups. Groups are divided into cluster and indices levels.

Cluster-level groups
  • CLUSTER_ALL: all cluster-level actions.

  • CLUSTER_MONITOR: monitoring actions.

  • CLUSTER_COMPOSITE_OPS: Groups all the permissions to execute composite requests not recognized by Search Guard; the group has to be granted on all indices to roles that have access only to a subset of indices .

  • CLUSTER_INTERNAL_FEDERATE: actions specific to the internal workflow of Federate.

  • CLUSTER_MANAGE: in addition to CLUSTER_INTERNAL_FEDERATE this includes actions to manage its own internal indices.

Indices-level groups
  • INDICES_ALL: all indices-level actions.

  • MONITOR: all actions regarding index monitoring, e.g., recovery, segments info, index stats and status.

  • MANAGE: all monitor and index administration actions.

  • WRITE: actions to modify an index.

  • READ: actions to read and search an index.

  • VIEW_INDEX_METADATA: any action to retrieve metadata about an index, e.g., list of fields, get a setting.

Roles

sg_roles.yml

# Permissions for the investigate system user
investigate_system:
  cluster:
    - CLUSTER_COMPOSITE_OPS
    - CLUSTER_MANAGE
    - CLUSTER_MONITOR
  indices:
    '?siren*':
      '*':
        - INDICES_ALL
    '*':
      '*':
        - READ
        - VIEW_INDEX_METADATA

# Permissions for a Investigate administrator
investigate_admin:
  cluster:
    - CLUSTER_COMPOSITE_OPS_RO
    - CLUSTER_MANAGE
    - CLUSTER_MONITOR
  indices:
    '*':
      '*':
        - MANAGE
        - READ
        - VIEW_INDEX_METADATA
# Permissions for a Investigate regular user.
investigate_user:
  cluster:
    - CLUSTER_COMPOSITE_OPS_RO
    - CLUSTER_INTERNAL_FEDERATE
  indices:
    'data-*':
      '*':
        - READ
        - VIEW_INDEX_METADATA

A permission is defined by the following syntax:

<username>:
  <indices or cluster>:
    '<index name or regular expression>':
      '<type name or regular expression>':
        - <list of permissions or action group names>

The index name can contain the simple expansion characters * and ? to match any sequence of character/any single character; for further information about defining permissions, refer to the Search Guard configuration documentation.

This sample configuration defines the following roles:

  • investigate_system: Defines the permissions for the Siren Investigate server process, with read/write access to the internal Siren Investigate indices.

  • investigate_admin: Defines the permissions for a Siren Investigate user that has the administrator permissions. This user has permissions to upload the Siren Investigate license, get monitoring information from the cluster and managed JDBC datasources.

  • investigate_user: Defines the permissions for a Siren Investigate user with read only access to all indices whose name starts with data-*.

Users

sg_internal_users.yml

# Internal user database
# The hash value is a bcrypt hash and can be generated with plugin/tools/hash.sh
sirenserver:
  hash: $2a$12$zMeFc6Xi.pcgDVHsvtCV9ePNteVwTE5uGxcKdf7XQcKB9.VkD8iOy
sirenadmin:
  hash: $2a$12$zMeFc6Xi.pcgDVHsvtCV9ePNteVwTE5uGxcKdf7XQcKB9.VkD8iOy
sirenuser:
  hash: $2a$12$zMeFc6Xi.pcgDVHsvtCV9ePNteVwTE5uGxcKdf7XQcKB9.VkD8iOy

The file defines the credentials for Search Guard internal users; passwords are stored as hashes in the hash attribute beneath each username.

The password for all the accounts in the example is password.

To change the password of a user, you must generate the corresponding hash; this can be done by executing the plugins/search-guard-<version>/tools/hash.sh script as follows:

$ bash plugins/search-guard-6/tools/hash.sh -p password

The script will output the hash for the password specified after the -p switch.

It is also possible to change passwords for internal users from the Access Control application in the Siren Investigate UI once configured.

Role mappings

sg_roles_mapping.yml

investigate_system:
  users:
    - sirenserver

investigate_admin:
  users:
    - sirenadmin

investigate_user:
  users:
    - sirenuser

The file defines the list of users assigned to each role using the following form:

<role name>:
  users:
    - <username>
    - <username>
Uploading the configuration to the cluster

To upload the configuration defined in the previous steps, go to the Elasticsearch folder and execute the plugins/search-guard-<version>/tools/sgadmin.sh script as follows:

$ bash plugins/search-guard-6/tools/sgadmin.sh \
  -cd config/sgconfig \
  -cn siren-distribution \
  -ts config/truststore.jks \
  -tspass password \
  -ks ../siren-investigate/pki/searchguard/CN\=sgadmin-keystore.jks \
  -kspass password \
  -h localhost \
  -p 9330 \
  -nhnv

To reload the configuration you have to use the same command with the -rl flag instead of -cd, for example:

$ bash plugins/search-guard-6/tools/sgadmin.sh \
  -rl
  -cn siren-distribution \
  -ts config/truststore.jks \
  -tspass password \
  -ks ../siren-investigate/pki/searchguard/CN\=sgadmin-keystore.jks \
  -kspass password \
  -h localhost \
  -p 9330 \
  -nhnv

You must specify the following arguments based on your environment configuration:

  • -cd: Path to the folder containing the Search Guard access control configuration.
  • -cn: Name of the Elasticsearch cluster.
  • -ts: Path to the truststore file.
  • -tspass: Password of the truststore file.
  • -ks: Path to the administrative client certificate keystore.
  • -kspass: Password of the client certificate keystore file.
  • -h: Hostname of a node in the cluster.
  • -p: Transport port of the node specified in the -h option.
  • -nhnv: Switches off host name verification; remove this option if you installed node certificates with the correct hostname (recommended in production).
  • -rl: Reloads the configuration and flushes the internal cache.

By default, the number of replicas for the searchguard index will be set at creation time to the number of data nodes - 1.

For additional information on how to set replication settings and sgadmin in general, refer to http://floragunncom.github.io/search-guard-docs/sgadmin.html.

If the command is executed successfully, a list of the actions executed and their outcome will be printed on screen, for example:

Clustername: elasticsearch
Clusterstate: YELLOW
Number of nodes: 1
Number of data nodes: 1
searchguard index does not exists, attempt to create it ... done
Populate config from /elasticsearch/sg_config
Will update 'config' with sg_config/sg_config.yml
   SUCC: Configuration for 'config' created or updated
Will update 'roles' with sg_config/sg_roles.yml
   SUCC: Configuration for 'roles' created or updated
Will update 'rolesmapping' with sg_config/sg_roles_mapping.yml
   SUCC: Configuration for 'rolesmapping' created or updated
Will update 'internalusers' with sg_config/sg_internal_users.yml
   SUCC: Configuration for 'internalusers' created or updated
Will update 'actiongroups' with sg_config/sg_action_groups.yml
   SUCC: Configuration for 'actiongroups' created or updated
Done with success

You can then verify that SSL and authentication are enabled by making an authenticated request with wget, for example:

$ wget --ca-certificate=../siren-investigate/pki/searchguard/ca.pem --http-user=sirenserver --http-password=password -qO - https://localhost:9220

To display information about the certificate as seen by a client you can execute the following command:

$ echo | openssl s_client -servername localhost -connect localhost:9220 -showcerts | openssl x509 -text -inform pem -text -noout
Siren Investigate access control configuration

Edit config/investigate.yml and specify the credentials of the sirenserver user, for example:

elasticsearch.username: 'sirenserver'
elasticsearch.password: 'password'

If HTTPS is enabled for the Elasticsearch REST API, ensure that the elasticsearch.url setting contains a URL starting with https, for example:

elasticsearch.url: 'https://localhost:9220'

If the certificate is not signed by a public authority, you will also need to set the elasticsearch.ssl.certificateAuthorities to the path of the CA chain bundle in PEM format, for example:

elasticsearch.ssl.certificateAuthorities: 'pki/searchguard/ca.pem'

If you are using the certificates generated by the TLS generator service, the PEM file containing the certification bundles is available in root-ca/root-ca.pem.

To enable certificate verification, set elasticsearch.ssl.verificationMode to full, for example:

elasticsearch.ssl.verificationMode: full

If you want to validate the certificate but not the hostname, set elasticsearch.ssl.verificationMode to certificate, for example:

elasticsearch.ssl.verificationMode: certificate

Set the investigate_core.elasticsearch.auth_plugin option to searchguard:

investigate_core:
  elasticsearch:
    auth_plugin: searchguard

To enable the Siren Investigate access control plugin, specify the following configuration values in the investigate_access_control section:

  • enabled: Set to true to enable the Siren Investigate access control plugin. Defaults to false.
  • backend: The authentication backend installed in the cluster; valid values are searchguard and xpack. Defaults to searchguard.
  • cookie.password: A 32 characters long alphanumeric string used to derive the key used to encrypt and sign cookies.
  • cookie.secure: If set to true, the cookie will be transmitted only if the request is being served over HTTPS. You must set this to false if Siren Investigate is behind an SSL proxy or if you are running Siren Investigate without HTTPS (which is not advised). Defaults to true.
  • admin_role: The name role that will have access to the access control management UI. This user will not be subject to any permission check by Siren Investigate, but will still be subject to permission checks when issuing queries to Elasticsearch. Defaults to sirenadmin.
  • acl.enabled: Set to true to switch off access control on saved objects. Defaults to false.

Example minimal configuration:

investigate_access_control:
  enabled: true
  acl:
    enabled: true
  cookie:
    secure: true
    password: '12345678123456781234567812345678'

Make sure to personalize the session cookie password.

Additional configuration options:

  • session.ttl: The lifetime of the session in milliseconds. If not set, the session will last as long as the session cookie is valid. Defaults to 3600000 (1 hour).
  • session.keepAlive: If set to true, every time a request is received within the session lifetime, the session lifetime will be extended by session.ttl. Defaults to true.
  • cookie.password: A 32 characters long alphanumeric string used to derive the key used to encrypt and sign cookies.
  • cookie.ttl: The lifetime of the session cookie in milliseconds. If not set, the cookie will expire when the browser is closed, which is the recommended setting. Note that browsers may not remove session cookies when a tab is closed or even across restarts, so you should set session.ttl for additional protection. Defaults to null.
  • cookie.name: The name of the session cookie. Defaults to kac.
  • acl.index: The Elasticsearch index in which access control rules and saved objects metadata will be stored (.sirenaccess by default).

If Siren Investigate is running behind a reverse SSL proxy like Nginx, remember to set cookie.secure to false otherwise the cookie will not be sent, for example:

investigate_access_control:
  enabled: true
  acl:
    enabled: true
  cookie:
    password: '12345678123456781234567812345678'
    secure: false

If you want to use the Siren Alert plugin, you must specify the Siren Alert user credentials in the investigate_access_control.sirenalert section, For example:

investigate_access_control:
  enabled: true
  acl:
    enabled: true
  cookie:
    password: '12345678123456781234567812345678'
    secure: false
  sirenalert:
    elasticsearch:
      username: sirenalert
      password: password

If Siren Alert credentials are not specified, Siren Alert will use the backend credentials to execute the watchers.

Restart Siren Investigate after changing the configuration file; if the configuration is correct, you should see an authentication dialog when browsing to Siren Investigate.

Authentication dialog
Saved objects access control

Siren Investigate features an access control system on saved objects that enables you to filter dashboards and visualizations visible to end users; this was enabled in the previous section by setting investigate_access_control.acl.enabled to true.

When the system is enabled, normal users should not have access to the .siren and .sirenaccess indices, as these will be managed by the backend user (.sirenserver).

Search Guard management user interface configuration 

Siren Investigate includes an optional user interface for the Search Guard REST Management API add-on; to use it, the Siren Investigate back end has to connect to the Elasticsearch cluster using a PEM client certificate with administrative privileges.

For Elasticsearch 6, this module comes already bundled with Search Guard 6, so there is no need to download it separately.

For older Elasticsearch and Search Guard versions, follow the Add-on installation instructions.

Add-on installation

To install the Search Guard REST Management API add-on it is required to download the correct jar for your Elasticsearch / Search Guard version from https://github.com/floragunncom/search-guard-rest-api/wiki and copy it to the plugins/search-guard-<version> folder of each node in the cluster.

To access the API it is required to use a client certificate with administrative privileges; to enable optional client certificate authentication on the REST interface, ensure that the following option is present in elasticsearch.yml:

searchguard.ssl.http.clientauth_mode: OPTIONAL

After the plugin has been copied and the configuration updated, the nodes must be restarted; a rolling restart is enough to install the add-on.

Note

When using this add-on, ensure that the sgadmin configuration folder contains only the sg_config.yml file, otherwise sgadmin will replace users, roles, action groups and mappings that may have been modified through the API.

Siren Investigate configuration

Copy the client certificate and its key to a folder readable by Siren Investigate (for example pki); then add the following parameters to the investigate_access_control configuration section:

  • admin_role: The Search Guard role that has access to the Search Guard management UI (investigate_admin by default).

  • backends.searchguard.admin.ssl.cert: Path to the administrative client certificate bundle in PEM format.

  • backends.searchguard.admin.ssl.key: Path to the administrative client certificate key in PEM format.

  • backends.searchguard.admin.ssl.keyPassphrase: The passphrase of the administrative client certificate key. Not required if the key is not encrypted.

For example:

investigate_access_control:
  enabled: true
  acl:
    enabled: true
  admin_role: investigate_admin
  cookie:
    password: '12345678123456781234567812345678'
    secure: false
  backends:
    searchguard:
      admin.ssl.cert: pki/searchguard/CN=sgadmin.crtfull.pem
      admin.ssl.key: pki/searchguard/CN=sgadmin.key.pem
      admin.ssl.keyPassphrase: password

Note that the administrative client certificate bundle must contain both the full CA chain and the client certificate; if using certificates generated by the TLS generation service, the file name will be CN=sgadmin.crtfull.pem, otherwise it is possible to generate the bundle manually by using cat, for example:

$ cat user.crt.pem ca-chain.pem > user.crtfull.pem
Access control: authentication

After the certificate is set up, restart Siren Investigate, login with a user having an administrative role, click the apps button, then click Access control and finally Authentication.

The Access control app

The Authentication section enables you to browse, edit and create the following Search Guard resources:

  • Internal users
  • Roles
  • Role mappings
  • Action groups

To verify that the application is working correctly, click Roles then click Open; you should see the list of roles defined during the initial Search Guard setup or an authorization error if the certificate is incorrect:

Browsing Search Guard roles

If you get an error upon opening the Authentication app, most probably the client certificate does not contain the full CA chain or the add-on has not been installed correctly, check Elasticsearch and Siren Investigate logs for related errors.

If you experience a Siren Investigate crash when opening the application, ensure that the option investigate_access_control.backends.searchguard.admin.ssl.keyPassphrase is set to the correct password.

Access Control: ACL
The ACL section

The ACL Roles panel in the ACL section enables you to define Siren Investigate roles, which are collections of permissions on saved objects and UI elements. The main purpose of this system is to hide and block access to end users and avoid unauthorized changes to configuration objects or use of certain parts of the system:

  • UI elements - applications, for example: Timelion, Access control, Siren Alert
  • UI elements - specific functionalities, for example: export CSV feature.
  • UI elements - Siren Investigate sections, for example: discover, management.
  • Saved objects on unauthorized indices, for example: dashboards, searches.

There are two kinds of rules:

  • Rules: Set permissions for saved objects.
  • UI rules: Set permissions to view different user interface elements.

The everyone role defines permissions for all the users in the system, and is mapped by default to any user logged in Siren Investigate; by default it grants all users read only access to the Siren Investigate configuration (Advanced settings), saved searches and index patterns as well as permission to view all applications and UI elements.

The everyone role

Denying access to certain saved objects like saved search using the first sets of rules is usually transparent to the user which means that for them, those objects are not visible anywhere in Siren Investigate.

Usually it is not required to create explicit UI rules for the dashboard application as access to specific dashboards can be restricted through saved object rules.

Denying access to an application like Timelion or a Siren Investigate section like management will hide the navigation menu elements, block access at the root level and display an error.

Blocked Timelion application and Siren Investigate management section

When the user tries to access app/timelion, the following error is shown.

Blocked timelion error

When the user tries to access /app/kibana#/management, the following error is shown.

Blocked Siren Investigate management section error

For most setups it makes sense to grant view permissions on visualizations as well, then set specific permissions on dashboards and dashboard groups for each role.

To define a new role, click Create role, then set the following parameters:

  • Role ID: The ID of the role (for example sirenuser); must be a lowercase alphanumeric string.
  • Backend roles: A list of Search Guard roles that will be mapped to this Siren Investigate role (for example sirenuser).
  • Rules: A list of rules on saved object types.

Each rule is defined by three parameters:

  • Action: Allow or deny.
  • Permission: The permission to allow or deny.
  • Context: The saved object type on which the permission must be enforced.
The Create role button
Saving a role
Object permissions

In addition to role level permissions, it is possible to define permissions on specific objects by going to ManagementSave Objects and clicking the permissions button next to an object:

The object permissions button

The object permissions form enables you to set the owner of the object and custom access rules.

By default, the owner is set to the user that created the object; the owner has all permissions on the created object; it is possible to unset the owner of an object by leaving the field blank and clicking the Save button.

Custom access rules can be used to grant access to an object that would be otherwise hidden; for example, if everyone is not granted to display dashboards but you want to display the Overview dashboard to all users, visit the object permissions form for the Overview dashboard and set the View permission for everyone to Allow.

If everyone can see dashboards but you would like to hide the IT dashboard to users, set the View permission for everyone to Deny.

The object permissions form
Notes

Users are not allowed to view or edit the following types unless they have permission to do so. But they will be retrieved and executed by the backend if used by a visualization:

  • Query
  • Query templates
  • Datasource
Logstash configuration

To enable authentication in Logstash, set the following parameters in the output.elasticsearch section:

  • user: The username of the user having the Logstash role.
  • password: The password of the user having the Logstash role.
  • ssl: Set to true to enable SSL.
  • truststore: The path to the CA truststore file.
  • truststore_password: The password of the CA truststore file.

For example:

output {
    elasticsearch {
       hosts => ['localhost:9220']
       user => logstash
       password => password
       ssl => true
       truststore => '/etc/pki/logstash/truststore.jks'
       truststore_password => password
    }
}

The truststore file must be copied on all nodes running Logstash.

Beats configuration

To enable authentication in a beat which connects directly to Elasticsearch, set the following parameters in the output.elasticsearch section:

  • protocol: Set to https.
  • username: The username of the user having the Logstash role.
  • password: The password of the user having the Logstash role.
  • tls.certificate_authorities: An array containing the path to the CA truststore file in PEM format.

For example:

output:

  elasticsearch:
    hosts: ['localhost:9220']

    protocol: 'https'
    username: 'logstash'
    password: 'password'

    tls:
      certificate_authorities: ['/etc/pki/filebeat/ca.pem']

The root certification authority in PEM format must be copied to all nodes running one or more beats.

Console configuration

To successfully submit queries from console to a cluster secured by Search Guard set the following parameters in config/investigate.yml:

console.proxyConfig:
  - match:
      protocol: 'https'

    ssl:
      ca: 'pki/searchguard/ca.pem'

console.proxyConfig.ssl.ca must point to the CA certificate bundle, so it can be set to the same value as the elasticsearch.ssl.ca parameter.

X-Pack monitoring configuration

To store monitoring data in a cluster secured by Search Guard it is required to configure agent exporters to submit data over an authenticated HTTPS connection.

The exporter configuration in elasticsearch.yml must include the following parameters:

  • type: http.
  • host: An array of URLs that will be contacted by the exporter.
  • auth.username: The username of the Marvel agent user.
  • auth.password: The password of the Marvel agent user.
  • ssl.truststore.path: The path to the CA certificate truststore (this will usually be the same as the one specified in the Search Guard configuration).
  • ssl.truststore.password: The password of the CA certificate truststore.

For example, the following configuration defines an exporter which sends data to the cluster at https://localhost:9220, authenticating as the monitoring user:

xpack.monitoring.exporters:
  id1:
    type: http
    host: ['https://localhost:9220']

    auth:
      username: monitoring
      password: password

    ssl:
      truststore.path: truststore.jks
      truststore.password: password
X-Pack security integration

Create a investigate_system role with the following definition and map it to a sirenserver user:

{
  "investigate_system": {
    "cluster": [
      "cluster:internal/federate/*",
      "cluster:admin/federate/*",
      "cluster:monitor/*"
    ],
    "indices": [
      {
        "names": [
          "/\\.siren.*/"
        ],
        "privileges": [
          "all"
        ]
      },
      {
        "names": [
          "*"
        ],
        "privileges": [
          "indices:data/read*",
          "indices:admin/aliases/get",
          "indices:admin/aliases/exists",
          "indices:admin/get",
          "indices:admin/exists",
          "indices:admin/mappings/fields/get*",
          "indices:admin/mappings/get*",
          "indices:admin/mappings/federate/connector/get*",
          "indices:admin/mappings/federate/connector/fields/get*",
          "indices:admin/types/exists",
          "indices:admin/validate/query",
          "indices:monitor/settings/get"
        ]
      }
    ]
  }
}

If using a custom configuration, replace the configuration index name (.siren by default) and access control index name (.sirenaccess by default) with the correct names.

Set elasticsearch.username and elasticsearch.password to the credentials of the sirenserver user, for example:

elasticsearch.username: sirenserver
elasticsearch.password: password

If HTTPS is enabled for the Elasticsearch REST API, ensure that the elasticsearch.url setting contains a URL starting with https, for example:

elasticsearch.url: 'https://localhost:9220'

If the certificate is not signed by a public authority, you will also need to set the elasticsearch.ssl.certificateAuthorities to the path of the CA chain bundle in PEM format, for example:

elasticsearch.ssl.certificateAuthorities: 'pki/xpack.pem'

To enable certificate verification, set elasticsearch.ssl.verificationMode to full, for example:

elasticsearch.ssl.verificationMode: full

Set the investigate_core.elasticsearch.auth_plugin option to xpack:

investigate_core:
  elasticsearch:
    auth_plugin: xpack

Then, set the backend parameter of the investigate_access_control section of the investigate.yml to xpack:

investigate_access_control:
  enabled: true
  backend: xpack
  acl:
    enabled: true
  cookie:
    secure: true
    password: '12345678123456781234567812345678'

All users with access to Siren Investigate should have the following role definition.

Example standard user role with access to all indices starting with data- and to all virtual indices starting with db-:

{
  "investigate_user": {
    "cluster": [
      "cluster:internal/federate/*"
    ],
    "indices": [
      {
        "names": [
          "data-*", "db-*"
        ],
        "privileges": [
          "indices:data/read*",
          "indices:admin/aliases/get",
          "indices:admin/aliases/exists",
          "indices:admin/get",
          "indices:admin/exists",
          "indices:admin/mappings/fields/get*",
          "indices:admin/mappings/get*",
          "indices:admin/mappings/federate/connector/get*",
          "indices:admin/mappings/federate/connector/fields/get*",
          "indices:admin/types/exists",
          "indices:admin/validate/query",
          "indices:monitor/settings/get",
          "indices:admin/template/get"
        ]
      }
    ]
  }
}

For administrative user, ensure you have admin_role configured in the investigate_access_control section in investigate.yml. e.g

investigate_access_control:
  admin_role: investigate_admin

Example administrative user with access to all indices starting with data-, to all virtual indices starting with db-, license management, and permission to manage external datasources and virtual indices starting with db-:

{
  "investigate_admin": {
    "cluster": [
      "cluster:internal/federate/*",
      "cluster:admin/federate/*",
      "cluster:monitor/*",
      "cluster:admin/xpack/security/*"
    ],
    "indices": [
      {
        "names": [
          "data-*", "db-*"
        ],
        "privileges": [
          "indices:monitor/*",
          "indices:admin/*",
          "indices:data/read*"
        ]
      }
    ]
  }
}

For additional information on datasources configuration, check the section.