Release notes

8.18.4-38.1

Improvements

  • Upgraded to Elasticsearch 8.18.4.

  • Added support for comma-separated indices in the join clause.

  • Removed the preference parameter for fingerprint computation.

  • Added ignore_query_cache query parameter to skip join query cache pruning strategy.

  • Improved the search-project task when the query is rewritten into a match_all.

  • Improved the initialization and cleanup job responsibilities.

  • Improved the concurrent execution of queries.

  • Removed unnecessary usage of the Guava EventBus.

  • Improved asynchronous execution in multi-search requests.

  • Added support for named join queries.

Bug fixes

  • Fixed a concurrent modification issue when creating task nodes.

  • Fixed an issue with dangling tasks in case of task cancellation failure.

  • Fixed a race condition when notifying task completion.

  • Fixed a validation rule in the static planner when joining on _index.

  • Fixed the took response time value to reflect the planner’s took_in_millis value.

  • Fixed request failures when shard allocation was disabled.

  • Fixed a race condition in the data upload queue.

  • Fixed a concurrent modification error when logging task metrics.

  • Fixed a missing entitlement entry for the json-ws processor.

8.18.0-38.0

New features

  • Set Adaptive Query Planner (AQP) as a default planner.

  • Introduced a Federate transport layer to detect rolling upgrades and support retro-compatible requests during the upgrade process.

    • Introduced a Federate Transport Version.

    • Enriched Request and Response objects with the Federate Transport Version of the sending and receiving nodes.

  • Implemented an input swap optimization for multi-conditional and spatial index joins.

  • Added join capabilities to KNN (K-Nearest Neighbors) Elasticsearch Query.

  • Added preliminary support for Search Guard FLX.

Improvements

  • Upgraded to Elasticsearch 8.18.0.

  • Aggregated shard-level operations into a single request per-node (i.e. leverage TransportBroadcastByNodeAction).

  • Improved the physical data signature and DataId in AQP requests.

  • Reduced the amount of data shared if the debug is disabled.

  • Enhanced memory error messages.

  • Optimized lock operations in LRU Cache.

  • Removed the usage of Class.forName for obfuscated classes.

  • Improved resource cleanup when a node becomes isolated.

  • Protected cached nested joins from eviction.

  • Improved the performance of the fingerprint computation.

Deprecations

  • Deprecated the Static Query Planner. Note: In versions 38.0 or later, the Adaptive Query Planner (AQP) is the default planner.

    • The federate.planner.type settings has been changed to "adaptive" by default. If any issues arise, it can be set to "static" and Siren should be contacted for troubleshooting.

Bug fixes

  • Reintroduced the json-ws processor, which was previously removed as part of the connector’s deprecation.

  • Fixed projection issues with floating-point values in nested joins.

  • Fixed pagination of join queries when the join was cached.

  • Fixed index ordering to ensure deterministic request cache keys.

  • Fixed a fingerprint computation issue that ignored the segment life cycle.

  • Fixed incorrect behavior in Boolean queries using the minimum_should_match parameter on AQP requests.

  • Fixed an issue retrieving debug responses when the track_total_hits parameter was disabled.

  • Fixed a caching issue in Point-In-Time requests.

  • Fixed a bug with cached join queries using function_score in combination with the min_score parameter.

  • Fixed an issue with cached queries using runtime fields.

Breaking change

  • This version introduced a Federate transport layer to allow retro-compatibility in case of rolling upgrade for the following versions. This version is not compatible with previous Federate versions during a rolling upgrade.