To import data into Insight you need to configure an instance of one of the import types that are available in your JIRA environment. The core implementation of the Import functionality is the same for all import types to make the import scalable, robust and below you can read about common configuration concepts.
The type of import to be used in Insight. The number of import types can vary depending on installed import modules. Below you will find the configuration that you can specify for the import type. Some of these configuration items can be override by the Object Type Mapping configuration.
There are 4 parts that needs to be performed when creating an import.
- First you need to select the import type to use for the import.
- "General Fields" defines general configuration and are common for all import types
- "Module Fields" are the import type depended fields (different for each import type)
- "Scheduling" fields are common to all import types and is where you specify synchronization details.
So here are the details of each step:
|Name||The name of the import|
|Description||The description for your convenience.|
You can specify a default concatenator. When joining multiple data locators into one Insight attribute, this will be the default concatenator. One example could be to join two columns like "First name" and "Last name" into one attribute. So "Mathias" (first name) and "Edblom" (last name) will be concatenated as "Mathias Edblom" if using \s as concatenator.
Enter \s for space-concatenated. To include a concatenate character, place the value between double quotes (i.e "\s").
|Empty Values||Defines what should happen when a Data Locator is empty. Should the import remove the attribute value or just ignores it and leave the current value as is.|
|Defines what should happen if a Data Locator is unknown to Insight. This could happen with attribute types like "Status" and "Select". The value can be added as an option or just ignore the value.|
Format for date fields in import source to convert dates into Insight. If no one provided, Insight will automatically try to find correct format.
|Format for date/time fields in import source to convert dates into Insight. If no one provided, Insight will automatically try to find correct format. |
The format should be specified according to the Java SimpleDateFormat guidelines.
The import type are responsible of defining and validating the module field configuration. However, it's optional for the Import Type module to implement any specific module fields. If no one is implemented, this section will be ignored and the import will move to the "Scheduling Fields". A list of built in import types and the description of the module field configuration can be found here: Available Import Types
|Synchronizing Account||The JIRA user to use when synchronize data into Insight|
|Automatically Synchronize||If the import should be scheduled for automatic synchronization.|
|Cron Expression||The interval for the automatic synchronization.|
This is an example of an created CSV import. To see the information on the import, click on the header of the import to expand more information. This is also where you can create Object Type Mappings. Not that it's marked as invalid due to that no Object Type Mappings have been added.
Import Object Type Mapping
You can select what part of the data source that should be imported in an object type. You can configure as many object type mappings as you like. You can not specify more the one object type mapping to the same object type within the same import instance. However, if you create more then one import, you can let the imports updated the same objects from different sources.
|Object Type||The Insight object type to import the data into. This value can not be changed, you need to create a new object type mapping to configure another object type.|
|Selector||This is the selector of the import type that specifies what data from the data source that should be imported. This will be different to different import types. Read more on each import type documentation|
You can filter the data to be imported by IQL before it actually is imported into Insight
This is a video demonstrating how to split your import source into different object types using IQL
There are three options when an object is missing in the data source.
Update the object with a predefined value to a specified attribute. The Threshold configuration will apply.
The object will be deleted in Insight. The Threshold configuration will apply.
The import will just ignores it
|Threshold Type||When we have missing objects in the data source you can specify a threshold for the "Update" and "Delete" configuration. This means that even if an object is missing from the source, you may wait until the import actually will consider doing any action upon it. You can specify if the type should be "Days" or "Synchronizations".|
|Threshold Number||The number of (days or synchronizations) before acting on the missing objects|
|Empty Values||This will override the general field configuration of the import type, for this specific object type mapping. What should happen if an Data Locator is empty? Should the import remove the attribute value or just ignore it and leave the current value as is?|
|This will override the general field configuration of the import type, for this specific object type mapping. Defines what should happen if a Data Locator is unknown to Insight. This could happen with attribute types like "Status" and "Select". The value can be added as an option or just ignore the value.|
At least one object type mapping needs to be enabled in order to enable the import.
Each Object Type Mapping needs to have at least one attribute configured.
If a specific attribute should be consider as a part of the object identifier. If multiple attributes is used as identifiers the combined value will be used to identify the object and identify duplicates.
The identifiers are used to construct an internal IQL responsible for identifying the object.
To exemplify: If two import attributes are configured for Insight attribute First Name and Last Name and specified to be identifier the following IQL will be used to identify object(s) in Insight
|Data Locator||Each import type has the responsibility to provide the available data locators for the import type. An example the CSV import type, where all "column" in the CSV will be the Data Locators.|
|Insight Attribute||The Insight attribute for the specified Object Type to import the Data Locator values|
|Object Mapping (IQL)|
When importing data, and using Insight attributes of type "Object" (reference) you need to provide an IQL to fetch the objects to set, depending on the data provided by the Data Locator. In the examples that follows, Name is a reference to the value in Insight that the referenced object has, in the attribute Name.
You can use place holders in the IQL like:
|Description||You can specify a description for the attribute mapping, for you convenience.|
|Concatenator||When joining multiple Data Locators into one Insight attribute, this will be the default concatenator.|
You can use regular expressions to edit the value from the import source.
For the regular expression we use Java syntax https://docs.oracle.com/javase/7/docs/api/java/util/regex/Pattern.html. If the regular expression does not find any match the attribute is left empty. It is possible to extract from the value given by the Data Locator by making what you wish to extract the first group in the regular expression, for example using the regular expression "(\w*)\W" (quotation marks excluded) on the value "Blue Whale" yields "Blue" to the corresponding attribute. If there is no group present and the regex finds a match the whole value is given.
Each import module can (but is not required to) specify a predefined structure. The concept is basically that the import module will create an Insight object type structure together with attributes based on the external data source. It is a default Insight structure that can be used to represent the external data. This is just a suggestion and attributes, object types and the order of the same can be configured to your liking after creation.
You can use a custom created Insight structure or reuse the one you already have.
The predefined configuration is often used in conjunction with the Predefined Structure. The Predefined Configuration is a suggested configuration for the external data to your Insight structure. It will create the suggested object type configuration part(s) to your Insight structure.
The creation is done by using information provided by the import module how object types and attributes should be mapped with the data locators, the IQL references and all above mentioned configuration. Based on the name of the object type as well as zero or more attribute names supplied from the import module Insight will determine which object types(s) that should be added to the configuration.
To better understand what is happening the following simplified flow illustrates how each import is handled.
Before using the importers
You should configure the JVM memory to at least 4Gb maximum memory, and at least 2Gb in "normal" environments. But in very large instances you need to increase this even more, sometimes up to 32Gb maximum.
The more CPU cores the importers can use, the faster the import will complete. We recommend at least 8 cores.