Siren Platform User Guide

Settings

In order to send queries to virtual indices the Elasticsearch cluster must contain at least one node enabled to issue queries over JDBC; it is advised to use a coordinating only node for this role, although this is not a requirement for testing purposes.

JDBC node settings

In order to enable JDBC on a node where the Siren Federate plugin is installed, add the following setting to elasticsearch.yml:

node.attr.connector.jdbc: true

Then, create a directory named jdbc-drivers inside the configuration directory of the node (e.g. elasticsearch/config or /etc/elasticsearch).

Finally, copy the JDBC driver for your remote datasource and its dependencies to the jdbc-drivers directory created above and restart the node; see the JDBC driver installation and compatibility section for a list of compatible drivers and dependencies.

Common configuration settings
Encryption

JDBC passwords are encrypted by default using a predefined 128 bit AES key; before creating datasources, it is advised to generate a custom key by running the keygen.sh script included in the siren-federate plugin directory as follows:

bash plugins/siren-federate/tools/keygen.sh -s 128

The command will output a random base64 key; it is also possible to generate keys longer than 128 bit if your JVM supports it.

To use the custom key, the following parameters must be set in elasticsearch.yml on master nodes and on all the JDBC nodes:

  • siren.connector.encryption.enabled: true by default, can be set to false to disable JDBC password encryption.
  • siren.connector.encryption.secret_key: a base64 encoded AES key used to encrypt JDBC passwords.

Example elasticsearch.yml settings for a master node with a custom encryption key:

siren.connector.encryption.secret_key: "1zxtIE6/EkAKap+5OsPWRw=="

Example elasticsearch.yml settings for a JDBC node with a custom encryption key:

siren.connector.encryption.secret_key: "1zxtIE6/EkAKap+5OsPWRw=="
node.attr.connector.jdbc: true

Restart the nodes after changing the configuration to apply the settings.

Cluster wide settings

The following parameters can be set in elasticsearch.yml on JDBC nodes or by using the Elasticsearch cluster update settings API:

  • siren.connector.siren.timeout.connection: the maximum amount of seconds to wait when establishing or acquiring a JDBC connection (30 by default).
  • siren.connector.timeout.query: the maximum execution time for JDBC queries, in seconds (30 by default).
  • siren.connector.enable_union_aggregations: true by default, can be set to false to disable the use of unions in nested aggregations.
  • siren.connector.query.max_result_rows: the maximum number of rows that will be retrieved from a resultset when performing a join across datasources. Defaults to 50000.
  • siren.connector.query.max_bucket_queries: the maximum number of JDBC queries that will be generated to compute aggregation buckets. Defaults to 500.
Additional node settings

The following settings can be used to tune query processing on JDBC enabled nodes:

  • siren.connector.pool.size: the number of threads that will be allocated to process the execution of queries to remote datasources; by default it is set to int((number of available_processors * 3) / 2) + 1.
  • siren.connector.pool.queue: the maximum number of requests that should be queued if all the threads are busy. Defaults to 40.