Setting up record editing

Siren Investigate provides the ability to make manual changes to the records of your entity tables and searches. These changes can then be stored on the same index or in a separate index called a "revision index".

Enabling record editing helps in the process of data curation by providing a way to manually correct small errors in some records and test the fixes immediately.

Applied changes can be analyzed later by a data curator, to be applied at scale when importing a dataset.

Enabling record editing

Before you begin editing records in Siren Investigate, ensure that the required security settings are configured.

Changes made to a record of an entity table can be saved only if the index pattern contains a single index, which means that neither the characters that indicate multiple indices (,) nor a wildcard (*) are accepted in the index name.

  1. In the Data model app, select the Editing tab.

  2. Select Editing is allowed on <name>.

  3. Specify how the edited records will be stored:

    • Use the same index: Configure the same index to store the edited records by overwriting the original data. The applied changes are permanent, and disabling record editing does not restore the original data.

    • Use a revision index: Configure a separate revision index. This does not overwrite the original index. You can easily revert all changes and restore the original data by disabling record editing.

      Storing edited records in a revision index is subject to the following limitations:

      • The revision index is joined with the original indices to produce the final view of the entity table. The speed of the join, therefore, depends on how many records are edited. Only the records in the user’s current dataspace are considered in the join. If you expect the number of edited records to exceed 15,000 records, it is recommended that you store the edited records in the same index to avoid impacting the system’s performance.

      • All edited records will score lower in search results, unless you specify a more accurate (but slower) search type in the siren:scoringSearchType advanced setting. This is due to the method that Elasticsearch uses to calculate scores across separate shards.

  4. Specify the fields that are allowed or denied for editing. Enter each value on a new line.

Use of an Elasticsearch index with no scoping is now deprecated in favor of Siren-managed dataspace scoped indices. For more information, see Configuring security for shared indices.

Visibility of edits

If you are using the original index for storage, then visibility of record edits is the same as that of the original data. However, if you are configuring a separate revision index, you can choose between the following options:

  • Private: Edited data remains private to the dataspace that a record is revised in.

  • Shared: Edited data is visible from all dataspaces.

  • Shared edits on a legacy index: This option is visible only for entity tables that have revision indices from Siren Investigate version 11 or earlier. It is selected when the revision index is a standard (not Siren-managed) Elasticsearch index.

The visibility of edited data is bound by the scope of the original data. This means that you cannot create shared edited records for data that is private to the dataspace, but you can create private edited records of shared data.

Revision index settings

If you select Use a revision index, you must complete the following fields:

  1. Name: Specify a name for the revision index.

    A revision index name must be unique - it cannot match the name of the original entity table. Edited records must also be stored in a single index. Therefore, the use of the wildcard character * and the comma , to indicate multiple indices is not supported. Index aliases are not supported either.

  2. Identifier field: Select a field in the entity table or search that contains unique values; this field becomes the identifier that is used to track records. You can view or discover the unique value in your data set on the Fields tab.

  3. Click Create index.

Editing records

After record editing is configured, additional controls to edit records are available on all record tables and Record Table visualizations on that entity table or search.

For more information, see Editing records in record tables.

Adding data to entity tables that contain edited records

It is recommended that you do not attempt to overwrite edited records by importing data.

If you want to import data into an entity table that is enabled for editing and has an identifier defined, be aware of the following considerations:

  • Using the same index: If you selected this option for the current entity table, the original field values are overwritten.

  • Using a revision index: The original record gets overwritten but remains hidden if the value of the Identifier field on the Editing tab is the same. Only edited records are displayed in the Data tab.

Next step

Adding relations between entities