Data mapping

Introduction to data mappings¶

Once a data source has been configured, it must be mapped to a Brinqa data model. Data sources will provide information on one or more types of objects that will also exist within the Brinqa application, like vulnerabilities and hosts. Mapping allows the Bapplication to determine what data from the source should be used to build what type of object in the system.

It also allows for the normalization and transformation of data from different sources. For example, host objects from two different data sources may have slightly different attributes, or call those attributes different things even though they refer to essentially the same data. So, an attribute storing data about how important a host is to business functions might be called "Criticality" when it comes from one source and "Importance" when it comes from another. Those data sources could have those respective attributes mapped to one corresponding attribute on the Brinqa host data model (e.g. "Host Importance"). Then, host objects created by syncs would populate that attribute with values from either the "Importance" or "Criticality" fields of the data source, making information from the differing sources easier to compare.

Administration > Data Integration > Mappings will take you to the Manage Data Mappings page, which displays a list view of all existing mappings.

Table 1: List view contents

Create a new data mapping¶

Procedure

1. Navigate to Administration > Data Integration > Mappings
2. Click Create Data Mapping and fill in the following:

Table 2: New data mapping properties

Field	Description
Title	Title of the data mapping. The title will be displayed wherever the data mapping appears in the UI.
Name	Reference name for queries and scripts. The name can contain only letters, numbers, and underscores.
Description	Description of the data mapping. Description is displayed on the mapping list view and can be searched from the list view search bar.
Order	Identifies the ordering of this mapping. If multiple mappings for this data source exist (e.g. both a host and a vulnerability mapping), this field determines the order they’ll be synced in when the data source is synced. Owning objects like hosts should always be synced first, since vulnerabilities cannot be associated to a host until that host exists.
Active	Whether the mappings are active. Inactive mappings will not be used and are in effect archived.
Run business rules	Whether a manual sync should trigger business rules. Business rules can be set under Administration > Rules > Business Rules and allow you to specify additional operations to be performed at the time of sync.
Validate attributes	Requires system to validate that every source attribute is associated with a target attribute.
Copy blank values	Copies blank/empty values to a Brinqa attribute when the source attribute is empty, overwriting any existing non-empty value.
Coalesce	If datasets are coalesced with multiple attributes, this setting determines whether all of the attributes must be a match to coalesce, or whether just one attribute matching is sufficient.
Data Source	Data source the data will be mapped from.
Source	Specific subset of data wanted from the data source.
Target	Data model the data will be associated with.

Once a data source, source, and target have been selected, attribute mapping options will appear near the middle of the form.

3. Add attribute mappings individually with the Add Attribute Mapping button or automatically with the Auto Map button. Not all attributes can be automapped, but it is generally better to start by automapping and then add additional mappings as needed.

Table 3: Add/edit attribute properties

Field	Description
Source Attribute	The attribute from the original dataset in the source.
Target Attribute	The attribute in the Brinqa data model that the source attribute should populate data to.
Direction	Whether values should move from the Brinqa dataset to the source dataset (outbound), from the source dataset to the Brinqa dataset (inbound), or both (bidirectionally).
Coalesce	Selects this attribute as a unique identifier for this record to coalesce on (e.g. ticket number). New datasets will only be created when coalesce attributes are unqiue.
Use source script	Use a script to provide a value (inbound or outbound) instead of the target or source object. E.g. if you want to apply a calculation to a value before placing it in the target attribute.


Once attribute mappings have been added, they will appear on a list view in the middle of the form.

Table 4: Attribute Mapping list view contents

Columns	Description
Order	Referenced for evaluation order when using a source script. Mapped attribute order can be changed by dragging and dropping entries on the mapped attribute list view.
Source Attribute	Name of the attribute in the data source.
Source Type	Source attribute type (e.g. string).
Target Attribute	Name of the corresponding attribute in the Brinqa data model.
Target Type	Target attribute’s type (e.g. text). Type is set during data model configuration.
References	The data model that reference type attributes refer to. E.g. the host owner attribute would reference the user data model, because host owners will be users.
Source Format	Format of the value in the source.
Coalesce	Whether this is the unique identifier.
Actions	Where the Actions button appears on mouseover.

4. (Optional) Transform scripts can also be added to mappings by clicking the Add Transform Script button. The modal that opens allows you to specify when the script runs, enter a Groovy script for the action, and choose whether the script is active. Source scripts will only run on sync, whereas transform scripts can be set to run at other times or reference multiple attributes simultaneously.
5. Click Create.

Edit or delete a data mapping¶

Individual attribute mappings can be edited or deleted by clicking the Actions button that appears to the right on mouseover of the entry on the create/edit mappings form.

Data mappings (groups of mapped attributes) can be edited or deleted by clicking the Actions button that appears on mouseover of the entry on the Manage Data Mappings list view.

Sync a data mapping¶

An individual mapping can be synced rather than syncing the data from an entire data source. If a mapping is synced only data from the source objects of that mapping (e.g. host) will be imported.

You can manually sync data from a mapping by clicking the refresh icon that appears next to the Last Synced date when moused over. A modal window will open and allow you to select the time period for which you want data, as well as filtering options for that data. Filtering options are specific to each connector.

TUTORIAL: Qualys Vulnerability Management host mapping¶

This tutorial covers how to map your Qualys Vulnerability Management data source to the host data model in Brinqa. To complete this tutorial, the Qualys Vulnerability Management data source must have already been created in your system.

1. Navigate to Administration > Data Integration > Mappings
2. Click Create Data Mapping
3. Enter "Qualys Host" for the Title
4. Enter "1" for the Order. This step is important because it means the host mapping will be synced before the vulnerability mapping when the data source is synced. This is relevant because if new hosts are created in Qualys, they must be created in Brinqa before vulnerabilities can be associated to them. Reversing the order of these syncs would mean that vulnerabilities on the new hosts would be created with the host field empty, because Brinqa would not know a host of that name existed yet.
5. Leave the Options and Coalesce settings as their defaults.
6. Select your Qualys data source for the Data Source
7. Select "Host" for the Source
8. Select "Host" for the Target
9. Click Automap. Additional attributes can be mapped if they exist on the Brinqa host data model. A table of all available Qualys host attributes is below.
10. Click Create

Table 5. Attributes on the Qualys host source

TUTORIAL: Qualys Vulnerability Management vulnerability mapping¶

This tutorial covers how to map your Qualys Vulnerability Management data source to the vulnerability data model in Brinqa. To complete this tutorial, the Qualys Vulnerability Management data source must have already been created in your system.

1. Navigate to Administration > Data Integration > Mappings
2. Click Create Data Mapping
3. Enter "Qualys Vulnerability" for the Title
4. Enter "2" for the Order. This step is important because it means the vulnerability mapping will be synced after the host mapping when the data source is synced. This is relevant because if new hosts are created in Qualys, they must be created in Brinqa before vulnerabilities can be associated to them. Reversing the order of these syncs would mean that vulnerabilities on the new hosts would be created with the host field empty, because Brinqa would not know a host of that name existed yet.
5. Leave the Options and Coalesce settings as their defaults.
6. Select your Qualys data source for the Data Source
7. Select "Vulnerability" for the Source
8. Select "Vulnerability" for the Target
9. Click Automap. This will automatically map the majority of attributes.
10. Click Add Attribute Mapping. The next few steps will add a mapping for host on vulnerabilities, associating vulnerabilities to the host they were found on.
11. Select "Host ID" for the Source
12. Select "Host (Master Detail)" for the target. This makes the host a parent/owning object for vulnerabilities associated with it.
13. Select "Sys ID" for the Reference attribute. This will use the incoming vulnerability’s host ID to look up an existing host in the Brinqa database using the host’s SYS ID attribute to create a relationship between vulnerability and host records.
14. Click Create
15. (Optional) Additional attributes can be mapped if they exist on the Brinqa vulnerability data model. A table of all available Qualys vulnerability attributes is below.
16. Click Create

Table 6. Attributes on the Qualys vulnerability source

TUTORIAL: Rapid7 Nexpose host mapping¶

This tutorial covers how to map your Rapid7 Nexpose data source to the host data model in Brinqa. To complete this tutorial, the Rapid7 Nexpose data source must have already been created in your system.

1. Navigate to Administration > Data Integration > Mappings
2. Click Create Data Mapping
3. Enter "Nexpose Host" for the Title
4. Enter "1" for the Order. This step is important because it means the host mapping will be synced before the vulnerability mapping when the data source is synced. This is relevant because if new hosts are created in Qualys, they must be created in Brinqa before vulnerabilities can be associated to them. Reversing the order of these syncs would mean that vulnerabilities on the new hosts would be created with the host field empty, because Brinqa would not know a host of that name existed yet.
5. Leave the Options and Coalesce settings as their defaults.
6. Select your Nexpose data source for the Data Source
7. Select "Host" for the Source
8. Select "Host" for the Target
9. Click Automap. Additional attributes can be mapped if they exist on the Brinqa host data model. A table of all available Nexpose host attributes is below.
10. Click Create

Table 7. Attributes on the Nexpose host source

TUTORIAL: Rapid7 Nexpose Vulnerability Management vulnerability mapping¶

This tutorial covers how to map your Rapid7 Nexpose data source to the vulnerability data model in Brinqa. To complete this tutorial, the Rapid7 Nexpose data source must have already been created in your system.

1. Navigate to Administration > Data Integration > Mappings
2. Click Create Data Mapping
3. Enter "Nexpose Host" for the Title
4. Enter "2" for the Order. This step is important because it means the host mapping will be synced before the vulnerability mapping when the data source is synced. This is relevant because if new hosts are created in Nexpose, they must be created in Brinqa before vulnerabilities can be associated to them. Reversing the order of these syncs would mean that vulnerabilities on the new hosts would be created with the host field empty, because Brinqa would not know a host of that name existed yet.
5. Leave the Options and Coalesce settings as their defaults.
6. Select your Nexpose data source for the Data Source
7. Select "Vulnerability" for the Source
8. Select "Vulnerability" for the Target
9. Click Automap. This will automatically map the majority of attributes.
10. Click Add Attribute Mapping. The next few steps will add a mapping for host on vulnerabilities, associating vulnerabilities to the host they were found on.
11. Select "Host ID" for the Source
12. Select "Host (Master Detail)" for the target. This makes the host a parent/owning object for vulnerabilities associated with it.
13. Select "Sys ID" for the Reference attribute. This will use the incoming vulnerability’s host ID to look up an existing host in the Brinqa database using the host’s SYS ID attribute to create a relationship between vulnerability and host records.
14. Click Create
15. (Optional) Additional attributes can be mapped if they exist on the Brinqa vulnerability data model. A table of all available Nexpose vulnerability attributes is below.
16. Click Create

Table 8. Attributes on the Nexpose vulnerability source