Release notes

8.4.3-29.2

Improved off-heap memory management of data packets by leveraging the Elasticsearch API for tracking references to a resource.

Fixed execution of a join query with a query referencing a runtime field at the same level.
Fixed a waiting thread issue when a packet upload is rejected by the thread_pool.federate.data thread pool.
Fixed execution of a join query with a script query at the same level.

Added ROUTING_JOIN as a new join algorithm optimized for use cases when joining on the parent set’s _id field.
Added the Siren Consolidator, an ingest processor for consolidating data from different indices, as part of the Siren Federate plugin.
Added a new REST API _siren/nodes/benchmark that can be used to evaluate different aspects of the Federate plugin.

Implemented compatibility with Elasticsearch 8.4.3.
Improved the performance of joins on the _id field.
Upgrade to Arrow 9.0.0.
Adopt JPMS (Java Platform Module System).
Update architecture diagram.
Improved networking performance of the search-project task.
Updated default values for the following settings:
- The setting siren.io.pipeline.hash.partitions_per_node that sets the number of partitions per node used in the HASH_JOIN algorithm, now defaults to the number of CPUs - 1.
- The setting siren.io.pipeline.max_packet_size that sets the size of data packets sent between nodes of the cluster now defaults to 1MB.

Fixed dead links in the documentation.
Fixed an issue that could cause a node to cache failed statistics.
Fixed a race condition issue when executing a multi-search request.
Fixed ClassCastException and properly handle unsupported cases of projecting fields on virtual indices.

Remove deprecated and unused actions over virtual indices.
Removed deprecated settings siren.io.tuple.collector.packet_size and siren.io.tuple.collector.hash.partitions_per_node.