Siren Platform User Guide

Managing fields

The fields for the index pattern are listed in a table. Click a column header to sort the table by that column. Click Controls in the rightmost column for a given field to edit the field’s properties. You can manually set the field’s format from the Format box. Format options vary based on the field’s type.

You can also set the field’s popularity value in the Popularity text entry box to any desired value. Click Update Field to confirm your changes or Cancel to return to the list of fields.

Siren Investigate has field formatters for the following field types:

String field formatters

String fields support the String and URL formatters.

The String field formatter can apply the following transformations to the field’s contents:

  • Convert to lowercase.
  • Convert to uppercase.
  • Convert to title case.
  • Apply the short dots transformation, which replaces the content before a . character with the first character of that content, as in the following example:

Original

Becomes

com.organizations.project.ClassName

c.o.p.ClassName

The URL field formatter can take on the following types:

  • The Link type turn the contents of the field into a URL.
  • The Image type can be used to specify an image folder where a specified image is located.

You can customize either type of URL field formats with templates. A URL template enables you to add specific values to a partial URL. Use the string {{value}} to add the contents of the field to a fixed URL.

For example, when:

The resulting URL replaces {{value}} with the user ID from the field.

The {{value}} template string URL-encodes the contents of the field. When a field encoded into a URL contains non-ASCII characters, these characters are replaced with a % character and the appropriate hexadecimal code. For example, field contents users/admin result in the URL template adding users%2Fadmin.

When the formatter type is set to Image, the {{value}} template string specifies the name of an image at the specified URI.

To pass unescaped values directly to the URL, use the {{rawValue}} string.

A Label Template enables you to specify a text string that displays instead of the raw URL. You can use the {{value}} template string normally in label templates. You can also use the {{url}} template string to display the formatted URL.

Date field formatters

Date fields support the Date, Url, and String formatters.

The Date formatter enables you to choose the display format of date stamps using the moment.js standard format definitions.

The String field formatter can apply the following transformations to the field’s contents:

  • Convert to lowercase
  • Convert to uppercase
  • Convert to title case
  • Apply the short dots transformation, which replaces the content before a . character with the first character of that content, as in the following example:

Original

Becomes

com.organizations.project.ClassName

c.o.p.ClassName

The URL field formatter can take on the following types:

  • The Link type turn the contents of the field into a URL.
  • The Image type can be used to specify an image folder where a specified image is located.

You can customize either type of URL field formats with templates. A URL template enables you to add specific values to a partial URL. Use the string {{value}} to add the contents of the field to a fixed URL.

For example, when:

The resulting URL replaces {{value}} with the user ID from the field.

The {{value}} template string URL-encodes the contents of the field. When a field encoded into a URL contains non-ASCII characters, these characters are replaced with a % character and the appropriate hexadecimal code. For example, field contents users/admin result in the URL template adding users%2Fadmin.

When the formatter type is set to Image, the {{value}} template string specifies the name of an image at the specified URI.

To pass unescaped values directly to the URL, use the {{rawValue}} string.

A Label Template enables you to specify a text string that displays instead of the raw URL. You can use the {{value}} template string normally in label templates. You can also use the {{url}} template string to display the formatted URL.

Geographic point field formatters

Geographic point fields support the String formatter.

The String field formatter can apply the following transformations to the field’s contents:

  • Convert to lowercase
  • Convert to uppercase
  • Convert to title case
  • Apply the short dots transformation, which replaces the content before a . character with the first character of that content, as in the following example:

Original

Becomes

com.organizations.project.ClassName

c.o.p.ClassName

Numeric field formatters

Numeric fields support the URL, Bytes, Duration, Number, Percentage, String, and Color formatters.

The URL field formatter can take on the following types:

  • The Link type turn the contents of the field into a URL.
  • The Image type can be used to specify an image folder where a specified image is located.

You can customize either type of URL field formats with templates. A URL template enables you to add specific values to a partial URL. Use the string {{value}} to add the contents of the field to a fixed URL.

For example, when:

The resulting URL replaces {{value}} with the user ID from the field.

The {{value}} template string URL-encodes the contents of the field. When a field encoded into a URL contains non-ASCII characters, these characters are replaced with a % character and the appropriate hexadecimal code. For example, field contents users/admin result in the URL template adding users%2Fadmin.

When the formatter type is set to Image, the {{value}} template string specifies the name of an image at the specified URI.

To pass unescaped values directly to the URL, use the {{rawValue}} string.

A Label Template enables you to specify a text string that displays instead of the raw URL. You can use the {{value}} template string normally in label templates. You can also use the {{url}} template string to display the formatted URL.

The String field formatter can apply the following transformations to the field’s contents:

  • Convert to lowercase
  • Convert to uppercase
  • Convert to title case
  • Apply the short dots transformation, which replaces the content before a . character with the first character of that content, as in the following example:

Original

Becomes

com.organizations.project.ClassName

c.o.p.ClassName

The Duration field formatter can display the numeric value of a field in the following increments:

  • Picoseconds
  • Nanoseconds
  • Microseconds
  • Milliseconds
  • Seconds
  • Minutes
  • Hours
  • Days
  • Weeks
  • Months
  • Years

You can specify these increments with up to 20 decimal places for both input and output formats. The default number of decimals for the Number format is 3, i.e. 0,0.[000]. If there are values smaller than this, but larger than 1e-7, they will be rounded to 0. The fix is to change the Numeral.js format pattern to: 0,0.[0000000]

The Color field formatter enables you to specify colors with specific ranges of values for a numeric field.

When you select the Color field formatter, Siren Investigate displays the Range, Font Color, Background Color, and Example fields.

Click Add Color to add a range of values to associate with a particular color. You can click in the Font Color and Background Color fields to display a color picker. You can also enter a specific hex code value in the field. The effect of your current color choices are displayed in the Example field.

colorformatter.png

The Bytes, Number, and Percentage formatters enable you to choose the display formats of numbers in this field using the numeral.js standard format definitions.

Scripted fields

Scripted fields compute data on the fly from the data in your Elasticsearch indices. Scripted field data is shown on the Discover tab as part of the document data, and you can use scripted fields in your visualizations. Scripted field values are computed at query time so they are not indexed and cannot be searched. Note that Siren Investigate cannot query scripted fields.

Caution

Computing data on the fly with scripted fields can be very resource intensive and can have a direct impact on Siren Investigate’s performance. Keep in mind that there’s no built-in validation of a scripted field. If your scripts are buggy, you will get exceptions whenever you try to view the dynamically generated data.

When you define a scripted field in Siren Investigate, you have a choice of scripting languages. Starting with 5.0, the default options are Lucene expressions and Painless. While you can use other scripting languages if you enable dynamic scripting for them in Elasticsearch, this is not recommended because they cannot be sufficiently sandboxed.

Note

Use of Groovy, Javascript, and Python scripting is deprecated starting in Elasticsearch 5.0, and support for those scripting languages will be removed in the future.

You can reference any single value numeric field in your expressions, for example:

doc['field_name'].value

For more background on scripted fields and additional examples, refer to Using Painless in Kibana scripted fields.

Creating a scripted field

  1. Go to Settings > Indices.
  2. Select the index pattern you want to add a scripted field to.
  3. Go to the pattern’s Scripted Fields tab.
  4. Click Add Scripted Field.
  5. Enter a name for the scripted field.
  6. Enter the expression that you want to use to compute a value on the fly from your index data.
  7. Click Save Scripted Field.

For more information about scripted fields in Elasticsearch, see Scripting.

Modifying a scripted field

  1. Go to Settings > Indices
  2. Click Edit for the scripted field you want to change.
  3. Make your changes and then click Save Scripted Field to update the field.

Deleting a scripted field

  1. Go to Settings > Indices.
  2. Click Delete for the scripted field you want to remove.
  3. Confirm that you really want to remove the field.