Data mapping

Introduction to data mappings

Once a data source has been configured, it must be mapped to a Brinqa data model. Data sources will provide information on one or more types of objects that will also exist within the Brinqa application, like vulnerabilities and hosts. Mapping allows the Bapplication to determine what data from the source should be used to build what type of object in the system.

It also allows for the normalization and transformation of data from different sources. For example, host objects from two different data sources may have slightly different attributes, or call those attributes different things even though they refer to essentially the same data. So, an attribute storing data about how important a host is to business functions might be called "Criticality" when it comes from one source and "Importance" when it comes from another. Those data sources could have those respective attributes mapped to one corresponding attribute on the Brinqa host data model (e.g. "Host Importance"). Then, host objects created by syncs would populate that attribute with values from either the "Importance" or "Criticality" fields of the data source, making information from the differing sources easier to compare.

Administration > Data Integration > Mappings will take you to the Manage Data Mappings page, which displays a list view of all existing mappings.

Table 1: List view contents

Columns Description
Title Title of data mapping
Description Description of data mapping
Data Source Data source mapped
Last Synced Last time the data mapping was synced. Note: this may be different from the last time the data source as a whole was synced.

Create a new data mapping

Procedure

  1. Navigate to Administration > Data Integration > Mappings
  2. Click Create Data Mapping and fill in the following:
  3. Table 2: New data mapping properties

    Field Description
    Title Title of the data mapping. The title will be displayed wherever the data mapping appears in the UI.
    Name Reference name for queries and scripts. The name can contain only letters, numbers, and underscores.
    Description Description of the data mapping. Description is displayed on the mapping list view and can be searched from the list view search bar.
    Order Identifies the ordering of this mapping. If multiple mappings for this data source exist (e.g. both a host and a vulnerability mapping), this field determines the order they’ll be synced in when the data source is synced. Owning objects like hosts should always be synced first, since vulnerabilities cannot be associated to a host until that host exists.
    Active Whether the mappings are active. Inactive mappings will not be used and are in effect archived.
    Run business rules Whether a manual sync should trigger business rules. Business rules can be set under Administration > Rules > Business Rules and allow you to specify additional operations to be performed at the time of sync.
    Validate attributes Requires system to validate that every source attribute is associated with a target attribute.
    Copy blank values Copies blank/empty values to a Brinqa attribute when the source attribute is empty, overwriting any existing non-empty value.
    Coalesce If datasets are coalesced with multiple attributes, this setting determines whether all of the attributes must be a match to coalesce, or whether just one attribute matching is sufficient.
    Data Source Data source the data will be mapped from.
    Source Specific subset of data wanted from the data source.
    Target Data model the data will be associated with.

    Once a data source, source, and target have been selected, attribute mapping options will appear near the middle of the form.

  4. Add attribute mappings individually with the Add Attribute Mapping button or automatically with the Auto Map button. Not all attributes can be automapped, but it is generally better to start by automapping and then add additional mappings as needed.
  5. Table 3: Add/edit attribute properties

    Field Description
    Source Attribute The attribute from the original dataset in the source.
    Target Attribute The attribute in the Brinqa data model that the source attribute should populate data to.
    Direction Whether values should move from the Brinqa dataset to the source dataset (outbound), from the source dataset to the Brinqa dataset (inbound), or both (bidirectionally).
    Coalesce Selects this attribute as a unique identifier for this record to coalesce on (e.g. ticket number). New datasets will only be created when coalesce attributes are unqiue.
    Use source script Use a script to provide a value (inbound or outbound) instead of the target or source object. E.g. if you want to apply a calculation to a value before placing it in the target attribute.


    Once attribute mappings have been added, they will appear on a list view in the middle of the form.

    Table 4: Attribute Mapping list view contents

    Columns Description
    Order Referenced for evaluation order when using a source script. Mapped attribute order can be changed by dragging and dropping entries on the mapped attribute list view.
    Source Attribute Name of the attribute in the data source.
    Source Type Source attribute type (e.g. string).
    Target Attribute Name of the corresponding attribute in the Brinqa data model.
    Target Type Target attribute’s type (e.g. text). Type is set during data model configuration.
    References The data model that reference type attributes refer to. E.g. the host owner attribute would reference the user data model, because host owners will be users.
    Source Format Format of the value in the source.
    Coalesce Whether this is the unique identifier.
    Actions Where the Actions button appears on mouseover.
  6. (Optional) Transform scripts can also be added to mappings by clicking the Add Transform Script button. The modal that opens allows you to specify when the script runs, enter a Groovy script for the action, and choose whether the script is active. Source scripts will only run on sync, whereas transform scripts can be set to run at other times or reference multiple attributes simultaneously.
  7. Click Create.

Edit or delete a data mapping

Individual attribute mappings can be edited or deleted by clicking the Actions button that appears to the right on mouseover of the entry on the create/edit mappings form.

Data mappings (groups of mapped attributes) can be edited or deleted by clicking the Actions button that appears on mouseover of the entry on the Manage Data Mappings list view.

Sync a data mapping

An individual mapping can be synced rather than syncing the data from an entire data source. If a mapping is synced only data from the source objects of that mapping (e.g. host) will be imported.

You can manually sync data from a mapping by clicking the refresh icon that appears next to the Last Synced date when moused over. A modal window will open and allow you to select the time period for which you want data, as well as filtering options for that data. Filtering options are specific to each connector.

TUTORIAL: Qualys Vulnerability Management host mapping

This tutorial covers how to map your Qualys Vulnerability Management data source to the host data model in Brinqa. To complete this tutorial, the Qualys Vulnerability Management data source must have already been created in your system.

  1. Navigate to Administration > Data Integration > Mappings
  2. Click Create Data Mapping
  3. Enter "Qualys Host" for the Title
  4. Enter "1" for the Order. This step is important because it means the host mapping will be synced before the vulnerability mapping when the data source is synced. This is relevant because if new hosts are created in Qualys, they must be created in Brinqa before vulnerabilities can be associated to them. Reversing the order of these syncs would mean that vulnerabilities on the new hosts would be created with the host field empty, because Brinqa would not know a host of that name existed yet.
  5. Leave the Options and Coalesce settings as their defaults.
  6. Select your Qualys data source for the Data Source
  7. Select "Host" for the Source
  8. Select "Host" for the Target
  9. Click Automap. Additional attributes can be mapped if they exist on the Brinqa host data model. A table of all available Qualys host attributes is below.
  10. Click Create

Table 5. Attributes on the Qualys host source

Attribute Description
DNS DNS address of the host
IP Address IP address of the host
Last Scanned Last scanned date
NetBIOS NetBIOS of the host
OS Operating system of the host
Owner Owner of the host
ID Unique identifier for the host
Tags Tags for the host

TUTORIAL: Qualys Vulnerability Management vulnerability mapping

This tutorial covers how to map your Qualys Vulnerability Management data source to the vulnerability data model in Brinqa. To complete this tutorial, the Qualys Vulnerability Management data source must have already been created in your system.

  1. Navigate to Administration > Data Integration > Mappings
  2. Click Create Data Mapping
  3. Enter "Qualys Vulnerability" for the Title
  4. Enter "2" for the Order. This step is important because it means the vulnerability mapping will be synced after the host mapping when the data source is synced. This is relevant because if new hosts are created in Qualys, they must be created in Brinqa before vulnerabilities can be associated to them. Reversing the order of these syncs would mean that vulnerabilities on the new hosts would be created with the host field empty, because Brinqa would not know a host of that name existed yet.
  5. Leave the Options and Coalesce settings as their defaults.
  6. Select your Qualys data source for the Data Source
  7. Select "Vulnerability" for the Source
  8. Select "Vulnerability" for the Target
  9. Click Automap. This will automatically map the majority of attributes.
  10. Click Add Attribute Mapping. The next few steps will add a mapping for host on vulnerabilities, associating vulnerabilities to the host they were found on.
  11. Select "Host ID" for the Source
  12. Select "Host (Master Detail)" for the target. This makes the host a parent/owning object for vulnerabilities associated with it.
  13. Select "Sys ID" for the Reference attribute. This will use the incoming vulnerability’s host ID to look up an existing host in the Brinqa database using the host’s SYS ID attribute to create a relationship between vulnerability and host records.
  14. Click Create
  15. (Optional) Additional attributes can be mapped if they exist on the Brinqa vulnerability data model. A table of all available Qualys vulnerability attributes is below.
  16. Click Create

Table 6. Attributes on the Qualys vulnerability source

Attribute Description
Access Complexity Vulnerability CVSSv2 access complexity
Access Vector Vulnerability CVSSv2 access vector
Authentication Whether the vulnerability scan was authenticated
Availability Impact Vulnerability availability impact
CVE Vulnerability CVE ID
CVSS Base Score Vulnerability CVSSv2 base score
CVSS Temporal Score Vulnerability CVSSv2 temporal score
Category Vulnerability CVSSv2 category
Confidentiality Impact Vulnerability CVSSv2 confidentiality impact
Consequence Vulnerability CVSSv2 consequence
Diagnosis Vulnerability diagnosis
Exploitability Vulnerability exploitability
Exploits Exploits available, if any
First Found Date first found
IP Address Host IP Address
Integrity Impact Vulnerability integrity impact
Last Fixed Date last fixed
Last Found Date last found
Last Scanned Date last scanned
Last Updated Date last updated
Patchable Vulnerability patchability
Port Vulnerability port, if any
Protocol Vulnerability protocol, if any
Remediation Level Vulnerability remediation level
Report Confidence Vulnerability report confidence
Results Scan results
SYS ID Unique identifier
Severity Vulnerability severity
Software Affected software
Solution Vulnerability solution
Status Vulnerability status
Title Vulnerability title
Type Vulnerability type
Vendor Vendor
Vendor Reference Vendor vulnerability reference number
BugTraq ID SecurityFocus ID
CVSSV3 Base Score Vulnerability CVSS v3 base score
CVSSV3 Temporal Score Vulnerability CVSS v3 temporal score
Discovery Discovery method
DNS DNS address of host
FQDN Fully Qualified Domain Name
Host ID Host ID
Last Modified Last modified date
Malware Associated malware
NetBIOS NetBIOS of the host
Published Date published
QID ID assigned by the PCI compliance service

TUTORIAL: Rapid7 Nexpose host mapping

This tutorial covers how to map your Rapid7 Nexpose data source to the host data model in Brinqa. To complete this tutorial, the Rapid7 Nexpose data source must have already been created in your system.

  1. Navigate to Administration > Data Integration > Mappings
  2. Click Create Data Mapping
  3. Enter "Nexpose Host" for the Title
  4. Enter "1" for the Order. This step is important because it means the host mapping will be synced before the vulnerability mapping when the data source is synced. This is relevant because if new hosts are created in Qualys, they must be created in Brinqa before vulnerabilities can be associated to them. Reversing the order of these syncs would mean that vulnerabilities on the new hosts would be created with the host field empty, because Brinqa would not know a host of that name existed yet.
  5. Leave the Options and Coalesce settings as their defaults.
  6. Select your Nexpose data source for the Data Source
  7. Select "Host" for the Source
  8. Select "Host" for the Target
  9. Click Automap. Additional attributes can be mapped if they exist on the Brinqa host data model. A table of all available Nexpose host attributes is below.
  10. Click Create

Table 7. Attributes on the Nexpose host source

Attribute Description
Sys ID Unique identifier for the host
IP Address IP address of the host
MAC Address MAC address of the host
Operating System Operating system of the host
Name Name or title of the host
Risk Score Host risk score (1-1000)
Site Name Host site name
Open ports List of open ports
Service List of services
Site Importance Host site importance
Software List of software found on host
Status Host status

TUTORIAL: Rapid7 Nexpose Vulnerability Management vulnerability mapping

This tutorial covers how to map your Rapid7 Nexpose data source to the vulnerability data model in Brinqa. To complete this tutorial, the Rapid7 Nexpose data source must have already been created in your system.

  1. Navigate to Administration > Data Integration > Mappings
  2. Click Create Data Mapping
  3. Enter "Nexpose Host" for the Title
  4. Enter "2" for the Order. This step is important because it means the host mapping will be synced before the vulnerability mapping when the data source is synced. This is relevant because if new hosts are created in Nexpose, they must be created in Brinqa before vulnerabilities can be associated to them. Reversing the order of these syncs would mean that vulnerabilities on the new hosts would be created with the host field empty, because Brinqa would not know a host of that name existed yet.
  5. Leave the Options and Coalesce settings as their defaults.
  6. Select your Nexpose data source for the Data Source
  7. Select "Vulnerability" for the Source
  8. Select "Vulnerability" for the Target
  9. Click Automap. This will automatically map the majority of attributes.
  10. Click Add Attribute Mapping. The next few steps will add a mapping for host on vulnerabilities, associating vulnerabilities to the host they were found on.
  11. Select "Host ID" for the Source
  12. Select "Host (Master Detail)" for the target. This makes the host a parent/owning object for vulnerabilities associated with it.
  13. Select "Sys ID" for the Reference attribute. This will use the incoming vulnerability’s host ID to look up an existing host in the Brinqa database using the host’s SYS ID attribute to create a relationship between vulnerability and host records.
  14. Click Create
  15. (Optional) Additional attributes can be mapped if they exist on the Brinqa vulnerability data model. A table of all available Nexpose vulnerability attributes is below.
  16. Click Create

Table 8. Attributes on the Nexpose vulnerability source

Attribute Description
SYS ID Unique identifier
Title Vulnerability title
CVE Vulnerability CVE ID
Status Vulnerability status
CVSS Base Score Vulnerability CVSS v2 base score
CVSS Vector Vulnerability CVSS v2 vector
Exploits Exploits available, if any
First Found Date first found
IP Address Host IP Address
Port Vulnerability port, if any
Protocol Vulnerability protocol, if any
Severity Vulnerability severity
Solution Vulnerability solution, if any
Risk Score Vulnerability risk score
Description Vulnerability description
References Additional references
PCI Status 0 or 1 to indicate PCI flag
PCI Severity Vulnerability PCI severity value
Key Vulnerability key
OVAL OVAL reference
Malwares Malwares if any
Service Service vulnerability found on