Troubleshooting guide

Installation error when extracting the plugin ZIP file

When you install the Siren Federate plugin , you might get the following error:

On Windows

elasticsearch-plugin install PATH-TO-SIREN-FEDERATE-PLUGIN\siren-federate-8.14.3-36.2.zip
> Downloading PATH-TO-SIREN-FEDERATE-PLUGIN\siren-federate-8.14.3-36.2.zip
[=================================================] 100%
Exception in thread "main" java.nio.file.NoSuchFileException: PATH-TO-ELASTICSEARCH\plugins.installing-3603227438462114792\plugin-descriptor.properties
at sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:79)
at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97)
at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:102)
at sun.nio.fs.WindowsFileSystemProvider.newByteChannel(WindowsFileSystemProvider.java:230)
at java.nio.file.Files.newByteChannel(Files.java:361)
at java.nio.file.Files.newByteChannel(Files.java:407)
at java.nio.file.spi.FileSystemProvider.newInputStream(FileSystemProvider.java:384)
at java.nio.file.Files.newInputStream(Files.java:152)
at org.elasticsearch.plugins.PluginInfo.readFromProperties(PluginInfo.java:162)
at org.elasticsearch.plugins.InstallPluginCommand.loadPluginInfo(InstallPluginCommand.java:724)
at org.elasticsearch.plugins.InstallPluginCommand.installPlugin(InstallPluginCommand.java:803)
at org.elasticsearch.plugins.InstallPluginCommand.install(InstallPluginCommand.java:786)
at org.elasticsearch.plugins.InstallPluginCommand.execute(InstallPluginCommand.java:232)
at org.elasticsearch.plugins.InstallPluginCommand.execute(InstallPluginCommand.java:217)
at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86)
at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124)
at org.elasticsearch.cli.MultiCommand.execute(MultiCommand.java:77)
at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124)
at org.elasticsearch.cli.Command.main(Command.java:90)
at org.elasticsearch.plugins.PluginCli.main(PluginCli.java:47)

On Linux

Exception in thread "main" java.nio.file.NoSuchFileException: PATH-TO-ELASTICSEARCH/plugins/.installing-2425832270248497263/plugin-descriptor.properties
at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
at sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:214)
at java.nio.file.Files.newByteChannel(Files.java:361)
at java.nio.file.Files.newByteChannel(Files.java:407)
at java.nio.file.spi.FileSystemProvider.newInputStream(FileSystemProvider.java:384)
at java.nio.file.Files.newInputStream(Files.java:152)
at org.elasticsearch.plugins.PluginInfo.readFromProperties(PluginInfo.java:162)
at org.elasticsearch.plugins.InstallPluginCommand.loadPluginInfo(InstallPluginCommand.java:724)
at org.elasticsearch.plugins.InstallPluginCommand.installPlugin(InstallPluginCommand.java:803)
at org.elasticsearch.plugins.InstallPluginCommand.install(InstallPluginCommand.java:786)
at org.elasticsearch.plugins.InstallPluginCommand.execute(InstallPluginCommand.java:232)
at org.elasticsearch.plugins.InstallPluginCommand.execute(InstallPluginCommand.java:217)
at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86)
at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124)
at org.elasticsearch.cli.MultiCommand.execute(MultiCommand.java:77)
at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124)
at org.elasticsearch.cli.Command.main(Command.java:90)
at org.elasticsearch.plugins.PluginCli.main(PluginCli.java:47)

This error occurs because the command is being run on the distribution ZIP file, rather than on the plugin ZIP file inside it.

To resolve this error, complete the following steps:

Extract the Siren Federate distribution ZIP file into a local directory.
In the extracted directory, locate the plugin ZIP file named siren-federate-8.14.3-36.2-proguard-plugin.zip. The path to this plugin ZIP file is represented by PATH-TO-SIREN-FEDERATE-PLUGIN in the command that follows.
Run the installation command: $ ./bin/elasticsearch-plugin install file:///PATH-TO-SIREN-FEDERATE-PLUGIN/siren-federate-8.14.3-36.2-proguard-plugin.zip

Cannot start the buffer allocator service

The following error message is displayed:

elasticsearch.bootstrap.StartupException: BufferAllocatorException[Cannot start the buffer allocator service];

This error occurs when the memory root limit is higher than the direct memory size (-XX:MaxDirectMemorySize) of the JVM. The default root memory limit is 1GB, or the maximum direct memory size of the JVM if lower than 1GB.

You can resolve this in one of the following ways:

Open the config/elasticsearch.yml file and reduce the siren.memory.root.limit parameter to a value in bytes that is lower than the direct memory limit.
Open the jvm.options file and increase the direct memory limit. For more information, see Setting off-heap memory.

Out of memory exception

The following error message is displayed:

out_of_memory_exception: Unable to allocate buffer of size.

This message indicates that off-heap memory allocation is used up. A join might require more memory than that which is available.

Open the config/elasticsearch.yml file and increase the siren.memory.root.limit parameter to a value in bytes for the root allocator.

Changing the thread pool queue size

You can modify the thread pools in Siren Federate.

To change the thread pool queue size, follow the instructions in the Elasticsearch documentation.

To verify how much thread pool queue is in use, follow the instructions in the Elasticsearch documentation.

Supported data types in a join

Siren Federate supports all primitive data types, however you must ensure that the data type of the joined fields across index patterns is the same.

For example, if you try to join a field from the pattern index*, but the field is an integer in index1 while it is a keyword in index2, an error will result.

For more information, see Configuring joins by type.

Support for joining on the document ID

When you add the on parameter to the join query, Siren Federate does support joining on the _id metadata field.

However, for system performance reasons - and because the _id metadata field does not have doc_values enabled - we recommend that you use a dedicated field with doc_values enabled, such as a keyword field.

One way to get around this issue is to index the _id metadata field in a secondary field that has doc_values enabled, and use that field in the join.

You can duplicate the content of the _id field into another field that has doc_values enabled on the client side or use a set ingest processor.

For more information about how doc_values are scanned during a join, see Configuring joins by type.

Minimum memory requirements

If you are uncertain about the minimum memory that is required to use Siren Federate, it depends on the size of your data set and the size of the join.

For more information, see Configuring off-heap memory.

System performance

If the response time of search requests that involve joins is too long, try the following options to improve the performance of the join:

Increase the number of nodes or shards in the index. For more information, see the Elasticsearch documentation.
Configure a task timeout. For more information, see Search Request.