Siren Platform User Guide

Cluster APIs

The cluster APIs enables the retrieval of cluster and node level information, such as statistics about off-heap memory allocation.

Nodes Statistics

The cluster nodes stats API allows to retrieve one or more (or all) of the cluster nodes statistics.

GET /_siren/nodes/stats
GET /_siren/nodes/nodeId1,nodeId2/stats

The first command retrieves stats of all the nodes in the cluster. The second command selectively retrieves nodes stats of only nodeId1 and nodeId2

By default, all stats are returned. You can limit this by combining any of the following stats:

memory
Memory allocation statistics
planner
Statistics about the planner job and task pools.

Memory Information

The memory flag can be set to retrieve information about the memory allocation:

GET /_siren/nodes/stats/memory

The response includes memory allocation statistics for each node node as follows:

{
  "se6baEC9T4K7-14yuG2qgA": {
    "memory" : {
      "allocated_direct_memory_in_bytes" : 0,
      "allocated_root_memory_in_bytes": 0,
      "root_allocator_dump": "Allocator(ROOT) 0/0/3750232064/17179869184 (res/actual/peak/limit)"
    }
  },
  "sKnVUBo9ShGzkl4GYih7BA": {
    "memory" : {
      "allocated_direct_memory_in_bytes" : 0,
      "allocated_root_memory_in_bytes": 0,
      "root_allocator_dump": "Allocator(ROOT) 0/0/0/17179869184 (res/actual/peak/limit)"
    }
  }
}
allocated_direct_memory_in_bytes
The actual direct memory allocated by Netty in bytes
allocated_root_memory_in_bytes
The actual direct memory allocated by the root allocator in bytes
allocator_dump
Dump of the root allocator including the actual direct memory allocated, the peak and the limit.

Planner Information

The planner flag can be set to retrieve information about the planner job and task pools:

GET /_siren/nodes/stats/planner

The response includes memory allocation statistics for each node node as follows:

{
  "se6baEC9T4K7-14yuG2qgA": {
    "planner": {
      "thread_pool": {
        "job": {
          "permits": 1,
          "queue": 0,
          "active": 0,
          "largest": 1,
          "completed": 538
        },
        "task": {
          "permits": 3,
          "queue": 0,
          "active": 0,
          "largest": 3,
          "completed": 3955
        }
      }
    }
  },
  "sKnVUBo9ShGzkl4GYih7BA": {
    "planner": {
      "thread_pool": {
        "job": {
          "permits": 1,
          "queue": 0,
          "active": 0,
          "largest": 1,
          "completed": 537
        },
        "task": {
          "permits": 3,
          "queue": 0,
          "active": 0,
          "largest": 3,
          "completed": 3863
        }
      }
    }
  }
}

Query Cache Information

The query_cache flag can be set to retrieve information about the Siren’s query cache:

GET /_siren/nodes/stats/query_cache

The response includes statistics about the query_cache on each node:

{
  "_nodes": {
    "total": 2,
    "successful": 2,
    "failed": 0
  },
  "cluster_name": "my_cluster",
  "nodes": {
    "tEwWYjpbQzSYghVJVt87QQ": {
      "timestamp": 1545408407569,
      "name": "node_s0",
      "transport_address": "127.0.0.1:41639",
      "host": "127.0.0.1",
      "ip": "127.0.0.1:41639",
      "roles": [
        "master",
        "data",
        "ingest"
      ],
      "query_cache": {
        "memory_size_in_bytes": 0,
        "total_count": 0,
        "hit_count": 0,
        "miss_count": 0,
        "cache_size": 0,
        "cache_count": 0,
        "evictions": 0
      }
    },
    "Dw06QS6oRbS3fEMazn5llQ": {
      "timestamp": 1545408407569,
      "name": "node_s1",
      "transport_address": "127.0.0.1:42841",
      "host": "127.0.0.1",
      "ip": "127.0.0.1:42841",
      "roles": [
        "master",
        "data",
        "ingest"
      ],
      "query_cache": {
        "memory_size_in_bytes": 0,
        "total_count": 0,
        "hit_count": 0,
        "miss_count": 0,
        "cache_size": 0,
        "cache_count": 0,
        "evictions": 0
      }
    }
  }
}
memory_size_in_bytes
The size in bytes of the cache
total_count
The total number of lookups in the cache
hit_count
The number of successful lookups in the cache
miss_count
The number of lookups in the cache that failed to retrieve data
cache_size
The number of entries in the cache
cache_count
The number of entries that have been cached
evictions
The number of entries that have been evicted from the cache

Optimizer Statistics Cache

The cluster optimizer cache API allows to retrieve a snaphshot of the query optimizer cache for a list of the cluster nodes.

GET /_siren/cache
GET /_siren/nodeId1,nodeId2/cache
GET /_siren/cache/clear
GET /_siren/nodeId1,nodeId2/cache/clear

The first command retrieves the state of the optimizer cache for all the nodes in the cluster, while the second only for the desired list of node IDs. The third command invalidates the optimizer cache on every node, while the last command does so for only the selected nodes.

The response includes statistics about the cache use on each node:

{
  "aQAf0tIwRtq_n4mBr9SLTw": {
    "size": 92,
    "hit_count": 32,
    "miss_count": 92,
    "eviction_count": 42,
    "load_exception_count": 0,
    "load_success_count": 92,
    "total_load_time_in_millis": 68004
  }
}
size
The estimated number of entries in the cache.
hit_count
The number of cache hits.
miss_count
The number of cache misses.
eviction_count
The number of evicted entries.
load_exception_count
The number of times a request failed to execute as its response was to be put in the cache.
load_success_count
The number of times a request was executed successfully as its response was to be put in the cache.
total_load_time_in_millis
The time spent in milliseconds to load request responses into the cache.