Siren Platform User Guide

Creating an index pattern search

Click Create Index Pattern Search.

Here, you can choose a name for the entity, add a long and short descriptions and, using an icon picker and color picker, choose an icon and color that will be associated with that entity in graph views, for example when you select the Graph View tab.

For more details on creating an index pattern search, see Index pattern searches.

Editing an index pattern search

You can view and edit multiple properties. They are grouped into tabs:

  • Data: Browse data and create child searches

  • Data model graph: Explore the relations graph.

  • Fields: View fields information.

  • Info: Modify search properties such as name, icon, label and description.

  • Relations: Add relationships.

  • Scripted Fields: Create scripted fields.

  • Source Filters: Add source filters.

To save your changes, click Save.

Data

From this tab you can:

  • Generate a dashboard and populate it with visualizations created automatically from the currently selected fields.

  • Create a child search.

  • Add filters.

data-tab.png
Data model graph
graph-tab.png
Fields

The fields tab displays properties for each field. A check mark indicates the presence of a property:

  • Name.

  • Type.

  • Format.

  • Searchable: Can be used in the filter bar.

  • Aggregateable: Can be used in visualization aggregations.

  • Excluded: Excluded from _source when it is fetched.

  • Primary key: The primary key of an entity identifier.

  • Single value: A field that is not an array.

To configure the field, click the Edit icon in the field’s row:

  • Format: Enables you to control the way that specific values are displayed. It can also cause values to be completely changed and prevent highlighting in Discover from working. The following options are available:

    • Boolean

    • color

    • string (default)

    • truncated string

    • URL

  • Popularity.

  • Primary key.

  • Single value.

Click Update field to save your changes. Alternatively, click Cancel to abandon your changes.

fields-tab.png
Info

The Info tab enables you to set the following properties:

  • Name of the search: The search name, typically describing returned entity types, for example: "users", "logs firewall 1", "financial articles".

  • Icon.

  • Color.

  • Label when visualized in the graph browser: How to compose the label used when records are visualized on the graph. This can either be a Scripted Label or a Document Field.

  • Short description.

  • Index pattern used by this search: The pattern used to select the indexes to receive queries from the search. By changing the pattern you can change the index or even select multiple indexes (* is a wildcard). For example, logstash-*. Changing this setting will impact this search, the searches derived from this and all the associated visual components.

  • Time Filter field name: Used to filter events with the global time filter.

info-tab.png
Relations

The Relations tab enables you to define relations between entities:

  • Source entity: Select a Field.

  • Labels: Select or create a new label for each direction of the relation.

  • Target entity: Select a Search and a Field or select an entity identifier.

relations-tab.png
Scripted fields

Scripted fields are computed in real time from your data. They can be used in visualizations and displayed in your documents. However, they cannot be searched.

Caution

Familiarize yourself with script fields (www.elastic.co/guide/en/elasticsearch/reference/5.6/search-request-script-fields.html) and scripts in aggregations (www.elastic.co/guide/en/elasticsearch/reference/5.6/search-aggregations.html#_values_source) before using scripted fields.

Scripted fields can be used to display and aggregate calculated values. As such, they can be very slow, and when configured incorrectly, can cause Siren Investigate to become unusable. There is no protection from unexpected exceptions caused by script errors.

By default, scripted fields use Painless (www.elastic.co/guide/en/elasticsearch/reference/5.6/modules-scripting-painless.html), a simple and secure scripting language designed specifically for use with Elasticsearch. To access values in the document use the following format:

doc['some_field'].value

Painless is powerful but easy to use. It provides access to many native Java APIs(www.elastic.co/guide/en/elasticsearch/reference/5.6/modules-scripting-painless.html#painless-api) and has an easy to learn syntax.

Currently, Siren Investigate does not support named functions in Painless scripts.

Alternatively you can use Lucene Expressions (www.elastic.co/guide/en/elasticsearch/reference/5.6/modules-scripting-expression.html). These are a lot like JavaScript, but limited to basic arithmetic, bitwise and comparison operations.

Lucene Expressions have the following limitations:

  • Only numeric, boolean, date and geo_point fields may be accessed.

  • Stored fields are not available.

  • If a field is sparse (only some documents contain a value), documents missing the field will have a value of 0.

Lucene Expressions support the following operators and functions:

  • Arithmetic operators:

    • +

    • -

    • *

    • /

    • %

  • Bitwise operators:

    • |

    • &

    • ^

    • ~

    • <<

    • >>

    • >>>

  • Boolean operators (including the ternary operator):

    • &&

    • ||

    • !

    • ?:

  • Comparison operators:

    • <

    • ==

    • >=

    • >

  • Common mathematic functions:

    • abs

    • ceil

    • exp

    • floor

    • ln

    • log10

    • logn

    • max

    • min

    • sqrt

    • pow

  • Trigonometric library functions:

    • acosh

    • acos

    • asinh

    • asin

    • atanh

    • atan

    • atan2

    • cosh

    • cos

    • sinh

    • sin

    • tanh

    • tan

  • Distance functions:

    • haversin

  • Miscellaneous functions:

    • min

    • max

Scripted fields have the following properties:

  • Name.

  • Language:

    • expression

    • painless (default)

  • Type:

    • Boolean

    • date

    • number

    • string

  • Format: Enables you to control the way that specific values are displayed. It can also cause values to be completely changed and prevent highlighting in Discover from working. The following options are available:

    • Boolean

    • bytes

    • color

    • duration

    • number (default)

    • percentage

    • string

    • URL

  • Popularity.

  • Primary key.

  • Single value.

  • Script.

Click Create field to save your changes. Alternatively, click Cancel to abandon your changes.

Source filters

Source filters can be used to exclude one or more fields when fetching the document source. This happens when viewing a document in Discover, or with a table displaying results from a saved search in Dashboard. Each row is built using the source of a single document. If you have documents with large or unimportant fields you may benefit from filtering those out at this lower level.

Note that multi-fields will incorrectly appear as matches in the table. These filters only apply to fields in the original source document, so that matching multi-fields are actually not filtered.

Enter a string in the Source Filter in the box and click Add. Filters accept wildcards, for example user* will return fields starting with user.

Removing an index pattern search
  1. Select the index pattern search from the left menu.

  2. Click Delete.

Creating a child search

After you have created an index pattern search, you can create more specific searches. For example, if your main index pattern search is Companies you can now create a narrower selection such as Companies from New York:

  1. Select the Companies index pattern search on the left menu.

  2. Go to the Data tab.

  3. Search for New York and press Enter.

  4. Click Create Child Search.

  5. Enter Companies from New York and click Save.

The child search appears nested under Companies on the left side.

Editing a child search

To save your changes, click Save.

Removing a child search
  1. Select the child search from the left menu.

  2. Click Delete.