BigID API/DSAR Tutorial

This page is a work in progress. The information here is most likely incorrect and not intended for public viewing.

In this article, you'll learn:

• Find a DSAR profile using the BigID API
• Search for individuals using attributes using the BigID API
• Run a DSAR scan using the BigID API
• Get a DSAR report using the BigID API

scenarioYou already have your own account management portal that also manages other tasks for your business. You already have BigID for your security and data governance teams and they've mentioned that it can perform DSARs too. Run the DSAR calls required to integrate your management portal with BigID's DSAR capabilities

In this tutorial, we'll use SAMPLE as our session token. This is unique to the training sandbox and will not work in other environments. See BigID API/Tutorial for information on authenticating with BigID.

Getting DSAR Profiles

BigID uses DSAR profiles to specify which databases to look for users. You can create these using the APIs, but creating them via the UI is preferred since the UI will provide suggestions as you work. In our case, we already have a few data sources within our system.

From this API call we can see a list of DSAR profiles. This also gives us insight into why organizations use DASR profiles. Different groups of systems can have users with the same unique ID (employee number 1 and customer number 1 are probably different people). Profiles allow us to segment those user groups. Below we see a different profile for US customers to illustrate that.

{
  "profiles": [
    {
      "_id": "5d93e8431810782ce9173ae0",
      "name": "Default Profile",
      "allEnabledEs": true,
      "allEnabledDs": true,
      "scopes": [
        "root"
      ],
      "isCustom": false
    },
    {
      "_id": "614d7794dfa3fdf8bd71cacb",
      "allEnabledDs": true,
      "allEnabledEs": false,
      "name": "ALL PI for US Customer",
      "scopes": [
        "root"
      ],
      "shouldAttributesEnrichment": true,
      "isCustom": true
    }
  ]
}

In our case, we know a user with the email [email protected] is present in the default profile. However, we need to figure out what attributes we can use to execute a DSAR with this profile so we get the names correct.

We can see that there's an idsor_attribute named Email that has an identifiability of 1. This means that 100% of the sampled users have a different Email which is perfect for our DSAR. Let's initiate a DSAR on this user using that Email attribute.

Initiating a DSAR

Because the amount of data scanned as part of a DSAR can be well into the petabytes, the DSAR API is asynchronous. The below request will start a DSAR and return immediately with an ID you can use to track its progress. Execute the below request to start your DSAR.

Checking DSAR Status

Let's check the status of your DSAR. Replace REQUESTID in the URL of the below request with the requestId you received above when you initiated the DSAR.

When checking the status, we can see the state of our DSAR overall as well as each individual data source. The sample below has 17/18 data sources in the scan completed so the overall state is still "Started". Rerun the above request until you have a state of "Complete".

{
    "status": "success",
    "statusCode": 200,
    "data": {
        "requestId": "61784c9b1a1d3e0ea3b21a53",
        "userId": "DSAR",
        "state": "Started",
        "statuses": {
            "Completed": 17,
            "Queued": 1,
            "InProgress": 1
        }
        ...
    },
    "message": null
}

Retrieving DSAR Results

After your DSAR has completed, there's a variety of ways we can get and display the results. For a full view of the options, visit the BigID Docs We now know the API calls we need and can use our programming language of choice to prepare our report. Below are some samples.

NodeJSJavascriptPython

import fetch from 'node-fetch';

let credentials = { username: "bigid", password: "learner" };
let env = "https://sandbox.mybigid.com/";

async function getDataSources(credentials, env) {
    // Request API Key using user/pass authentication
    const sessionRequest = fetch(env + 'api/v1/sessions', {
        method: 'POST',
        body: JSON.stringify(credentials),
        headers: { 'Content-Type': 'application/json' }
    });
    const sessionData = await request.json();

    const dsRequest = fetch(env + 'api/v1/ds-connections', {
        method: 'GET',
        headers: {
            'Content-Type': 'application/json',
            'Authorization': sessionData.auth_token
        }
    });
    return await dsRequest.json();
}

let credentials = { username: "bigid", password: "bigid111" };
let env = "https://sandbox.mybigid.com/";

async function getDataSources(credentials, env) {
    // Request API Key using user/pass authentication
    const sessionRequest = window.fetch(env + 'api/v1/sessions', {
        method: 'POST',
        body: JSON.stringify(credentials),
        headers: { 'Content-Type': 'application/json' }
    });
    const sessionData = await request.json();

    const dsRequest = window.fetch(env + 'api/v1/ds-connections', {
        method: 'GET',
        headers: {
            'Content-Type': 'application/json',
            'Authorization': sessionData.auth_token
        }
    });
    return await dsRequest.json();
}

import requests

credentials = {'username':'bigid', 'password':'bigid111'}
env = 'https://sandbox.mybigid.com/'

def getDataSources(credentials, env):
    sessionRequest = requests.post(env+'api/v1/sessions', data = credentials)
    sessionData = sessionRequest.json()
    dsRequest = requests.get(env+'api/v1/ds-connections', headers = {'Authorization':sessionData.get('auth_token')})
    return dsRequest.json()

Data Subject Access Request - allows employees and individuals to both request and receive a copy of all the personal data that a company or organization has collected about them