Authentication and access control

sg_config.yml

The sg_config.yml file contains the configuration of the authentication mechanisms and backends; this configuration:

sg_action_groups.yml

The file sg_action_groups.yml contains named groups of permissions which can be used in the role configuration file including Search Guard’s default groups and Siren Investigate specific groups. Groups are divided into cluster and indices levels.

sg_roles.yml

A permission is defined by the following syntax:

The index name can contain the simple expansion characters * and ? to match any sequence of character/any single character; for further information about defining permissions, refer to the Search Guard configuration documentation.

This sample configuration defines the following roles:

sg_internal_users.yml

The file defines the credentials for Search Guard internal users; passwords are stored as hashes in the hash attribute beneath each username.

The password for all the accounts in the example is password.

To change the password of a user, you must generate the corresponding hash; this can be done by executing the plugins/search-guard-<version>/tools/hash.sh script as follows:

The script will output the hash for the password specified after the -p switch.

It is also possible to change passwords for internal users from the Access Control application in the Siren Investigate UI once configured.

sg_roles_mapping.yml

The file defines the list of users assigned to each role using the following form:

To upload the configuration defined in the previous steps, go to the Elasticsearch folder and execute the plugins/search-guard-<version>/tools/sgadmin.sh script as follows:

To reload the configuration you have to use the same command with the -rl flag instead of -cd, for example:

You must specify the following arguments based on your environment configuration:

By default, the number of replicas for the searchguard index will be set at creation time to the number of data nodes - 1.

For additional information on how to set replication settings and sgadmin in general, refer to Using sgadmin to configure Search Guard.

If the command is executed successfully, a list of the actions executed and their outcome will be printed on screen, for example:

You can then verify that SSL and authentication are enabled by making an authenticated request with wget, for example:

To display information about the certificate as seen by a client you can execute the following command:

To enable OpenID Connect support in Siren Investigate, you will need to set the following variables under the investigate_access_control section:

A minimal configuration follows:

Restart Investigate after applying the changes to the investigate_access_control section.

If the configuration is correct, when opening Investigate you should be redirected to the Identity Provider login page and after entering your credentials be redirected to Investigate and logged in.

When logged in, the user information tooltip should:

In order to create a new App Registration, login to the Azure portal and switch to the Active Directory blade; then, select App Registrations and click on New registration:

Enter an appropriate display name (e.g. Investigate) and select the supported account types, which will be Accounts in this organization directory only for most use cases.

The Redirect URI must be specified as <public Investigate URL>/investigate-access-control/openid/callback; for example, if users are opening Investigate at https://investigate.local, the Redirect URI will be https://investigate.local/investigate-access-control/openid/callback, e.g. :

Take a note of the App Registration parameters, in particular the Application (client) ID; this is the client identifier that Investigate will use to retrieve the information about a logged in user.

Then, click on Endpoints and take a note of the OpenID Connect metadata document URL, which is the discovery URL for the App Registration:

In order to put Active Directory groups in the ID token, open the App Registration and click on Token Configuration:

Then click on Add groups claim and add the Group ID to the ID token:

Depending on your authorization needs, you can also add Directory Roles or All Groups instead of just groups.

Finally, you’ll need to create a set of credentials to grant Investigate the permission to initiate the authentication flow; in order to create the credentials, click on Certificates and secrets and then on New client secret.

Take a note of the secret as it won’t be displayed anymore and make sure to set a reminder when the secret expires as authentication will stop working.

To configure Siren Investigate to work with Azure Active Directory, you will need to set the following variables under the investigate_access_control section:

Example configuration:

Restart Investigate after applying the changes to the investigate_access_control section.

If the configuration is correct, when opening Investigate you should be redirected to a login page on microsoft.com:

After approval, the user should be logged into Investigate and should have the expected Search Guard roles (displayed as Elasticsearch roles) - The GUIDs of the Active Directory groups should be listed in the list of DLS roles.

With both basic authentication and OpenID Connect enabled, several versions of Search Guard will log many warnings such as:

To silence those warnings, you can add the following lines to the log4j2.properties file in the Elasticsearch configuration directory on each node:

If the OpenID Connect discovery URL is incorrect, Investigate will display a status page with a discovery error. If that happens, head to the Endpoints page of the App Registration blade and double check it has been pasted correctly, then restart Investigate.

If the OpenID client_id is incorrect, you will get an "Application not found" error from microsoft.com; if the client ID is correct but the secret is not, Investigate won’t be able to initiate the authentication flow and will display a generic 401 error instead of redirecting to microsoft.com.

If you get an authorization error after approving the login request on microsoft.com, first check that the value of openid_connect_url in the Search Guard configuration is correct by looking at Elasticsearch logs; if the URL is incorrect, you will see a KeySetProvider threw error message and details about the request failure.

If there are no errors related to the OpenID Connect discovery URL, a common error is that the subject_key option is set to a claim name that is not present in the ID token issues by Active Directory; when that happens, you will see a warning like the following:

If the user is authenticated but it does not have the expected roles, make sure that:

It is required that Siren Investigate and the container application are published on the same domain to enable cross frame communication. This can be achieved by implementing a proxy to Siren Investigate in the container application routes or configuring a reverse proxy on a path in the application server configuration.

The application that embeds Siren Investigate is responsible for generating JWT tokens; jwt.io provides a good overview of the technology, a browser based debugging tool and a list of libraries for several platforms.

The Search Guard documentation provides an overview of all the claims supported by the add-on and a list of all the configuration options.

Note that the application must specify an expiration date claim (exp) to avoid creating tokens with unlimited duration.

Each node must be restarted after the setting has been added.

It is possible to specify user roles in a token claim by setting the roles_key attribute in the authenticator configuration to the desired claim name, for example:

After the attribute is set and the configuration is updated, it is possible to assign backend roles to the user by setting the claim defined in http_authenticator.config.roles_key in the token payload, for example :

To enable JWT support in Siren Investigate, set the investigate_access_control.backends.searchguard.authenticator option in investigate.yml to http-jwt, for example:

Then restart Siren Investigate and open it in a browser; you should get a blank page and the URL should end with login.

To test JWT authentication, open your browser console (Ctrl+Shift+I on Chrome and Firefox) and call setJWTToken of the sireninvestigate object, for example:

After the token is set, Siren Investigate will store it in an encrypted cookie and send it in every request to the backend; the backend will then forward the JWT token to Search Guard to authenticate the user.

After the token is set, you can switch to the desired Siren Investigate URL by changing location.href.

When the user is logged out from the main application, sessionStorage and localStorage should be cleared.

For more information on how to call set JWTToken from the parent frame, see Legacy REST datasources.

To enable Kerberos authentication, you need to create a service Principal to identify the Elasticsearch REST interface; usually the principal name is HTTP/<public DNS name of the cluster> (for example HTTP/es.ad.local).

After the service Principal is defined, you need to generate a keytab file that will be used by the Kerberos add-on to authenticate with the KDC.

To verify that the keytab works correctly, copy it to a different machine with access to the KDC / Domain controller; the keytab contains the credentials of the service principal user so it should be removed from any intermediate machine used to transfer the file the transfer and from the target machine after the test is complete.

Create a file named krb5.conf in the same folder as the keytab with the contents below; replace AD.LOCAL with your domain name and DC.AD.LOCAL with the name or IP address of your KDC or domain controller, keeping the case of domains as in the example:

Kerberos authentication support requires the installation of the commercial Search Guard Kerberos HTTP Authentication add-on; to install it, download the correct jar for your Search Guard version from https://github.com/floragunncom/search-guard-auth-http-kerberos/wiki and copy it to the plugins/search-guard-<version> folder on each node.

Create a file named krb5.conf in the config folder of each node with the following contents; replace AD.LOCAL with your domain name and DC.AD.LOCAL with the name or IP address of your KDC/domain controller, keeping the case of domains as in the example:

Copy the keytab file for the service principal to the configuration folder of each Elasticsearch node.

Add the following options to the elasticsearch.yml file of each node:

Example configuration:

To switch off the Kerberos replay cache in Search Guard, you must set the sun.security.krb5.rcache JVM property to none; this can be done by setting the following line in config/jvm.options:

For information on where to set/modify this variable, refer to Running as a service on Linux or Running as a service on Windows.

After the previous steps have been completed on all nodes, perform a rolling restart of the cluster.

To complete the Kerberos configuration you need to modify your sg_config.yml file and upload it to the cluster using sgadmin; if you are using the Search Guard management API make sure you include only the sg_config.yml in the sgadmin configuration folder or you will overwrite internal users, actiongroups, roles and mappings defined through the API.

To enable Kerberos authentication over HTTP, you must:

Examplesg_config.yml:

With this configuration, if the user is not authenticated Search Guard will reply with a 401 challenge; SPNEGO compatible browsers will then repeat the request automatically with Kerberos credentials if the cluster is in a trusted network or display an authentication popup where the user can enter its domain credentials.

If an HTTP request to the cluster contains an HTTP Basic authorization header, it will still be authenticated by the HTTP authenticator defined in basic_internal_auth_domain; it is necessary to leave this enabled as the Siren Investigate backend uses this method to authenticate with the cluster.

It is possible to enable only a single HTTP challenge; if your browser is configured to automatically send Kerberos credentials in a trusted zone it is possible to switch off the challenge attribute by setting kerberos_auth_domain.http_authenticator.challenge to false.

For more details about configuring Search Guard authenticator, refer to the Search Guard documentation.