Kerberos/SPNEGO authentication support

This section offers an overview of how to enable Kerberos/SPNEGO authentication in Siren Investigate.

Prerequisites

Before you begin, ensure that you have completed the following steps:

Limitations

The current implementation requires disabling the Kerberos replay cache in Search Guard Classic, as the Siren Investigate backend needs to make multiple requests to the Elasticsearch cluster on behalf of the user in several places without the ability to generate new service tickets.

As long as all the traffic to Siren Investigate is encrypted and the service ticket lifetime is short (the default in most system is five to 10 minutes) this should not pose a significant security risk.

Prerequisites

Service Principal

To enable Kerberos authentication, you need to create a service Principal to identify the Elasticsearch REST interface; usually the principal name is HTTP/<public DNS name of the cluster> (for example HTTP/es.ad.local).

Active Directory

On an Active Directory domain controller it is possible to use the setspn command to set a Service Principal Name for a domain user; for example, the following command run in an elevated command prompt associates the Service Principal Name HTTP/es.ad.local to a user named elasticsearch:

setspn -A HTTP/es.cluster.local elasticsearch

Refer to the Active Directory documentation for more details about setspn and Kerberos integration.

Keytab

After the service Principal is defined, you need to generate a keytab file that will be used by the Kerberos add-on to authenticate with the KDC.

Active Directory

On an Active Directory domain controller you can generate a keytab by running the ktpass command in an elevated command prompt as follows:

ktpass -out es.keytab -princ <principal name>@<domain> /mapuser <principal user> /pass "<principal user password>" /kvno 0

For example, to generate a keytab for the SPN HTTP/es.ad.local, associated to elasticsearch user in the AD.LOCAL domain, you need to run the following command:

ktpass -out es.keytab -princ HTTP/es.ad.local@AD.LOCAL /mapuser elasticsearch /pass "password" /kvno 0

Verification

This verification step is optional but it is useful to ensure that the keytab is correct before configuring Search Guard Classic.

To verify that the keytab works correctly, copy it to a different machine with access to the KDC / Domain controller; the keytab contains the credentials of the service principal user so it should be removed from any intermediate machine used to transfer the file the transfer and from the target machine after the test is complete.

Create a file named krb5.conf in the same folder as the keytab with the contents below; replace AD.LOCAL with your domain name and DC.AD.LOCAL with the name or IP address of your KDC or domain controller, keeping the case of domains as in the example:

[libdefaults]
default_realm = AD.LOCAL
forwardable=true
default_tkt_enctypes = rc4-hmac,aes256-cts-hmac-sha1-96,aes128-cts-hmac-sha1-96
default_tgs_enctypes = rc4-hmac,aes256-cts-hmac-sha1-96,aes128-cts-hmac-sha1-96

[realms]
AD.LOCAL = {
kdc = dc.ad.local:88
default_domain = ad.local
}

[domain_realm]
.ad.local = AD.LOCAL
ad.local = AD.LOCAL

Linux

On Linux, set the KRB5_CONFIG variable temporarily to point to the absolute path of the file created before and run kinit -t <keytab> <principal>, for example:

KRB5_CONFIG=./krb5.conf kinit -t es.keytab HTTP/es.ad.local

If the keytab is correct, kinit should exit immediately and not show a password prompt; to verify that the ticket has been issued, execute the klist -v command and check that it outputs the details of the ticket:

klist -v
Credentials cache: API:123
        Principal: HTTP/es.ad.local@ES.AD.LOCAL
    Cache version: 0

Server: krbtgt/AD.LOCAL@AD.LOCAL
Client: HTTP/es.ad.local@AD.LOCAL
Ticket etype: aes256-cts-hmac-sha1-96, kvno 2
Session key: arcfour-hmac-md5
Ticket length: 1194
Auth time:  May 12 19:59:10 2017
End time:   May 13 05:59:10 2017
Ticket flags: enc-pa-rep, pre-authent, initial, forwardable
Addresses: addressless

You can then destroy the ticket by executing the kdestroy command.

Windows systems

If you are running Elasticsearch nodes on Windows, you can use the Kerberos tools bundled with the Java Runtime Environment to verify the keytab.

If the JRE folder is not in the system path, prepend it to each command.

Execute kinit <principal> -t <keytab> -J-Djava.security.krb5.conf=<path to krb5.conf> to get a ticket, for example:

kinit HTTP/es.ad.local -t es.keytab -J-D"java.security.krb5.conf=C:\Users\test\krb5.conf"

If the keytab is correct kinit will print the path to the file where the ticket has been saved, for example:

New ticket is stored in cache file C:\Users\test\krb5cc_test

Execute klist to see the details of the ticket; to destroy the ticket you can simply remove the file create by kinit.

Configuration

Kerberos configuration file

Create a file named krb5.conf in the config folder of each node with the following contents; replace AD.LOCAL with your domain name and DC.AD.LOCAL with the name or IP address of your KDC/domain controller, keeping the case of domains as in the example:

[libdefaults]
default_realm = AD.LOCAL
forwardable=true
default_tkt_enctypes = rc4-hmac,aes256-cts-hmac-sha1-96,aes128-cts-hmac-sha1-96
default_tgs_enctypes = rc4-hmac,aes256-cts-hmac-sha1-96,aes128-cts-hmac-sha1-96

[realms]
AD.LOCAL = {
kdc = dc.ad.local:88
default_domain = ad.local
}

[domain_realm]
.ad.local = AD.LOCAL
ad.local = AD.LOCAL

Keytab

Copy the keytab file for the service principal to the configuration folder of each Elasticsearch node.

Elasticsearch configuration

Add the following options to the elasticsearch.yml file of each node:

  • searchguard.kerberos.krb5_filepath: the path to the Kerberos configuration file, usually krb5.conf.

  • searchguard.kerberos.acceptor_keytab_filepath: the path to the keytab file relative to the configuration folder of the Elasticsearch node. It is mandatory to store the keytab in this folder.

  • searchguard.kerberos.acceptor_principal: the name of the principal stored in the keytab (for example HTTP/es.ad.local).

Example configuration:

searchguard.kerberos.krb5_filepath: 'krb5.conf'
searchguard.kerberos.acceptor_keytab_filepath: 'es.keytab'
searchguard.kerberos.acceptor_principal: 'HTTP/es.ad.local'

To switch off the Kerberos replay cache in Search Guard, you must set the sun.security.krb5.rcache JVM property to none; this can be done by setting the following line in config/jvm.options:

-Dsun.security.krb5.rcache=none

For information on where to set/modify this variable, refer to Running as a service on Linux or Running as a service on Windows.

Cluster restart

After the previous steps have been completed on all nodes, perform a rolling restart of the cluster.

Search Guard Classic authenticator configuration

To complete the Kerberos configuration you need to modify your sg_config.yml file and upload it to the cluster using sgadmin; if you are using the Search Guard Classic management API make sure you include only the sg_config.yml in the sgadmin configuration folder or you will overwrite internal users, actiongroups, roles and mappings defined through the API.

To enable Kerberos authentication over HTTP, you must:

  • Add a Kerberos authenticator stanza to searchguard.authc.

  • Switch off challenge in the existing HTTP Basic authenticator if enabled.

Examplesg_config.yml:

searchguard:
  dynamic:
    http:
      anonymous_auth_enabled: false
      xff:
        enabled: false
    authc:
      kerberos_auth_domain:
        enabled: true
        order: 2
        http_authenticator:
          type: kerberos
          challenge: true
          config:
            krb_debug: false
            strip_realm_from_principal: true
        authentication_backend:
          type: noop
      basic_internal_auth_domain:
        enabled: true
        order: 1
        http_authenticator:
          type: basic
          challenge: false
        authentication_backend:
          type: intern

With this configuration, if the user is not authenticated Search Guard Classic will reply with a 401 challenge; SPNEGO compatible browsers will then repeat the request automatically with Kerberos credentials if the cluster is in a trusted network or display an authentication popup where the user can enter its domain credentials.

If an HTTP request to the cluster contains an HTTP Basic authorization header, it will still be authenticated by the HTTP authenticator defined in basic_internal_auth_domain; it is necessary to leave this enabled as the Siren Investigate backend uses this method to authenticate with the cluster.

It is possible to enable only a single HTTP challenge; if your browser is configured to automatically send Kerberos credentials in a trusted zone it is possible to switch off the challenge attribute by setting kerberos_auth_domain.http_authenticator.challenge to false.

Verification

After sg_config.yml has been loaded you can verify if the authentication is working by mapping a username in the Active Directory / Kerberos domain to a Search Guard Classic role mapping, for example:

sirenuser:
  users:
    - sirenuser
    - domainuser

After the mapping is loaded to the cluster, logon to a machine in the domain with the domain user and open the cluster URL in a Kerberos enabled browser (for example Chrome on Windows).

If everything is set up correctly you should see the default JSON response of Elasticsearch in the browser without having to enter credentials, for example:

{
  "name" : "Node",
  "cluster_name" : "cluster",
  "cluster_uuid" : "nimUDAyBQWSskuHoAQG06A",
  "version" : {
    "number" : "5.4.0",
    "build_hash" : "fcbb46dfd45562a9cf00c604b30849a6dec6b017",
    "build_timestamp" : "2017-01-03T11:33:16Z",
    "build_snapshot" : false,
    "lucene_version" : "5.5.2"
  },
  "tagline" : "You Know, for Search"
}

If you are getting an authentication popup, ensure that the Elasticsearch cluster URL is in a trusted zone.

To add a site to the trusted zone on Windows:

  1. Open Internet Explorer and click Internet options.

  2. Click the Security tab.

  3. Click Local Intranet.

  4. Click Sites.

  5. Click Advanced.

  6. Add the URL of the cluster to the list (the port can be omitted).

After the cluster is in the trusted zone, try to open the cluster URL again.

Internet Explorer options are also used by Chrome on Windows.

Troubleshooting

To check why a request is not authenticated you can check the Elasticsearch logs of the client node serving the REST API.

The most common issues are:

  • Cluster URL not present in the trusted sites list.

  • A keytab containing an incorrect Service Principal Name and/or a wrong password for the user account associated to the SPN.

  • An incorrect address of the domain controller / KDC in the krb5.conf file.

To get additional debugging information you can set krb_debug to true temporarily in sg_config.yml and upload it to the cluster using sgadmin.

Siren Investigate SPNEGO configuration

To enable SPNEGO support in Siren Investigate, set the investigate_access_control.backends.searchguard.authenticator option to http-negotiate, in investigate.yml, for example:

investigate_access_control:
  #... existing options
  backends:
    searchguard:
      #... existing options
      authenticator: 'http-negotiate'

Then restart Siren Investigate and verify that you can log in from a browser in the domain using a user defined in Search Guard Classic.

When SPNEGO support is enabled, cookie based authentication will be switched off; if you need to provide both authentications for different networks, it is possible to start an additional Siren Investigate instance with investigate_access_control.backend.searchguard.authenticator set to http-basic or not set at all.

Next steps

Configuring ACL in Siren Investigate