Configuring security for shared indices

When you create indices with the Global or Local scope and save them as import templates, all write operations on the indices are performed by the Siren Investigate back-end user. The default back-end user is sirenserver and it must have View permission on the selected template.

If you create indices with the global scope, they are prefixed with siren-import-. If you create indices with the local scope, they are prefixed with siren-import-<dataspace shortcode>-, for example, siren-import-home-.

Prerequisites

Ensure that the latest version of Siren Federate is installed in your cluster, which must be compatible with Elasticsearch 7.9.2 or a later version.

Configuring write access

To take advantage of shared indices, grant the investigate_system role full access to all indices with names starting with siren-.

If you are setting up authentication as described in the Authentication and access control section, the role already has the required permissions.

The following is an excerpt from the investigate_system role definition for Elastic Stack security:

{
  "cluster": [
    ...
  ],
  "indices": [
    {
      "names": [
        "/siren-.*/",
      ],
      "privileges": [
        "all"
      ]
    }
  ]
}

The following is an excerpt from the investigate_system role definition for Search Guard:

investigate_system:
  cluster_permissions:
    ...
  index_permissions:
  # Grant all permissions to indices with names that start with siren-*
  - index_patterns:
    - 'siren-*'
    allowed_actions:
    - INDICES_ALL
  ...

Configuring read access

To restrict access to the documents that are associated with the current dataspace in a shared index, when a user sends a request to Elasticsearch, Siren Investigate puts the current dataspace in the X-Opaque-Id header; this information is then used in a document-level security query to filter the index.

The default security configuration allows Siren Investigate users to have read access to all of the documents in shared indices that are not segmented by dataspace. If users require more granular read access, you can manually customize the DLS rules.

To avoid tampering, the header is encoded and signed by using an RSA private key, which is then verified by using the corresponding public key on Elasticsearch.

Generating and configuring the signature key

To generate a new signature key, execute the following command from the directory where Siren Investigate is installed:

bin/investigate generate_signature_key

The command asks for a passphrase to encrypt the key and create two files:

investigate.pem: the private key used to sign the header.
investigate.pub: the public key to be set in the Elasticsearch configuration.

In addition, the command will output two snippets:

One snippet must be added to the investigate.yml file.
One snippet must be added to the Elasticsearch configuration file on all of the nodes in the cluster.

To use the generated key, append the first snippet to the investigate.yml file and make sure to:

Set the passphrase attribute to the passphrase that was set during the execution of the command.
Specify the correct path to the investigate.pem file in case you want to copy it to a custom directory.

Example of the parameters in the investigate.yml file:

shared_index_components:
  dataspaceSignature:
    # The algorithm that is used to sign the header
    algorithm: 'RS256'
    privateKey:
      # The path to the private key. The path can be relative to the working directory of the Investigate process.
      key: 'pki/investigate.pem'
      # The passphrase of the private key
      passphrase: 'password'

After you apply the changes, restart Siren Investigate.

Next, append the second snippet to the configuration file, which lists all of the Elasticsearch nodes in the cluster, for example:

siren.dataspaces.jws.pubkey: 'MIIBI...'

After you apply the changes, restart Elasticsearch.

Configuring document-level security

To regulate access to shared indices, you must define an Elasticsearch role that has:

Read access to indices that start with siren-import-.
A document-level security query that contains a clause to return only documents that are associated to the dataspace set in the request by Siren Investigate.

If you are setting up authentication as described in the Authentication and access control section, the role already has the required permissions.

The field where the dataspace associated to a document is specified is named _siren.sic.namespace.

The following Elasticsearch query is used to implement the above criteria:

{
  "siren_dataspaces": {
    "field": "_siren.sic.namespace"
  }
}

The siren_dataspaces query, which is provided by Siren Federate, works by checking if the request has an X-Opaque-Id header. It extracts the current dataspace from it after validating the header signature using the public key that is specified in the siren.dataspaces.jws.pubkey parameter.

If the header is valid and signed, the query matches all of the documents that contain the _siren.sic.namespace field matching the current dataspace and documents that do not contain a _siren.sic.namespace field. Otherwise, it will not match any document.

If a user is given unrestricted access to the siren-import- prefix by another role, the restrictions might not be enforced. Be aware of how your security configuration handles the union of multiple document-level security queries.

It is recommended that your system contains one role only that controls access to indices that are created by Siren Investigate.

Examples

The following is an example of an Elastic Stack security role with the document-level security query configured:

{
  "cluster": [
    "cluster:internal/federate/*"
  ],
  "indices": [
    {
      "names": [
        "/siren-.*/"
      ],
      "privileges": [
        "indices:data/read*",
        "indices:admin/aliases/get",
        "indices:admin/aliases/exists",
        "indices:admin/get",
        "indices:admin/exists",
        "indices:admin/mappings/fields/get*",
        "indices:admin/mappings/get*",
        "indices:admin/types/exists",
        "indices:admin/validate/query"
      ],
      "query": {
        "bool": {
          "should": [
            {
              "siren_dataspaces": {
                "field": "_siren.sic.namespace"
              }
            },
            {
              "bool": {
                "must_not": {
                  "exists": {
                    "field": "_siren.sic.namespace"
                  }
                }
              }
            }
          ]
        }
      }
    }
  ]
}

The following is an example of a Search Guard Classic role with the document-level security query configured:

# Permissions on shared index component indices for all users
sic_user:
  cluster_permissions:
  - CLUSTER_COMPOSITE_OPS_RO
  - CLUSTER_INTERNAL_FEDERATE
  index_permissions:
  - index_patterns:
    - 'siren-import-*'
    allowed_actions:
    - READ
    - VIEW_INDEX_METADATA
    dls: '{
      "bool": {
        "should": [
          {
            "siren_dataspaces": {
              "field": "_siren.sic.namespace"
            }
          },
          {
            "bool": {
              "must_not": {
                "exists": {
                  "field": "_siren.sic.namespace"
                }
              }
            }
          }
        ]
      }
    }'

Customizing the prefix of shared indices

You can customize the first part of the prefix by changing the shared_index_components.indexPrefix value from siren to the desired value in the investigate.yml file. However, you must also ensure that the document-level security rules and permissions on the new prefix are set correctly in the security configuration of your Elasticsearch cluster.

If you change the shared_index_components.indexPrefix, existing entity tables that rely on the older prefix are broken. To avoid this, ensure that you choose a prefix before you create shared entity tables.