Introduction to Connectors

Connectors vs BigID Applications

BigID has two main ways to extend your system’s functionality: connectors and applications. While both connectors and applications have the same goal of giving you more value from BigID, they do that very differently.

Applications

Applications allow you to add new screens and business logic using the BigID application framework. While these applications can contact external systems, an application cannot be a data source or entity source within BigID. Applications are primarily for adding additional screens or logic, such as reports with additional statistics or synchronization with external tools.

Connectors

Connectors serve as the translator between your data source’s language and the language of BigID. BigID ships with over 50 connectors for common enterprise systems, such as AWS DynamoDB, Google Drive, and Microsoft Exchange.

BigID Scanning Process

The scanning process involves:

User executes a scan.
BigID sends a message to the scanner.
Scanner contacts the data source via the connector.
Data source returns information to the scanner.
Scanner builds a search tree and returns results to BigID.
User views results in the Scan Result Details page.

Connector Types

BigID supports two types of connectors:

**Internal (Java-based Connectors):** Customizable but complex, installed manually.
**External (Generic REST API Connectors):** Easier to develop, supports both structured and unstructured data.

Exercise: Finding Connectors

Instructions for finding and reviewing connectors in BigID documentation for specific use cases.

Developing a Structured Connector

The Connector Supermarket

Structured connectors organize data hierarchically:

Departments are objects (e.g., "Customers").
Items in departments are records.
Each record contains fields with names, types, and values.

Connector Services

Structured connectors must implement the following services:

List fields for objects.
List objects available.
List records within an object.
Retrieve specific record details.
Search for records using specific fields.

Exercise: List Object Fields

Walkthrough for creating a "Customer" object and testing field responses using sample data.

Developing an Unstructured Connector

Unstructured Connectors

Unstructured connectors focus on systems like file storage where data cannot be neatly categorized into fields. Components include:

Containers (e.g., directories or buckets).
Objects (e.g., individual files).
Metadata (e.g., creation dates, owner).

Exercise: Implement Unstructured Connector Services

Implementation steps include listing containers, retrieving metadata, and handling content streams.

Distributing a Connector

Distributing Your Connector

Connectors can be distributed privately (e.g., for internal use) or publicly through platforms like the BigID Community or Marketplace.

Exercise: Packaging Your Connector

Instructions for creating a Docker container for the connector, testing it locally, and preparing it for deployment.

Exercise: Deploy Your Connector

Steps for deploying the connector in a BigID environment and enabling external connector support.

Connector Setup and Testing

Connector Setup

Adding a connector involves: 1. Populating configuration parameters. 2. Testing the connection. 3. Saving the data source.

Testing Your Connector

Steps for running scans and ensuring functionality, including viewing scan results in the BigID UI.

Connectors/Learn

Contents