Connectors: Difference between revisions
| (13 intermediate revisions by the same user not shown) | |||
| Line 1: | Line 1: | ||
{{Box/start}} | |||
In this article, you'll learn: | |||
* What do connectors add to your BigID system | |||
* What types of connectors exist | |||
{{Box/end}} | |||
BigID Connectors allow your BigID system to provide insights about new types of data. Whether that's a known data type like CSV from a new type of data source or something completely new to the BigID ecosystem, a connector will allow you to bring BigID's data discovery capabilities to that system. | BigID Connectors allow your BigID system to provide insights about new types of data. Whether that's a known data type like CSV from a new type of data source or something completely new to the BigID ecosystem, a connector will allow you to bring BigID's data discovery capabilities to that system. | ||
===Why do we need connectors? === | |||
Every data source has its own way of communicating with third parties. Some data sources return information nicely organized, others return it as a jumbled mess. In order for BigID to give you the insights you expect, data needs to be fed to BigID in a consistent way. Connectors work as translators between the multitude of formats that data sources have adapted to the standard format BigID expects. Note that even if the data format is the same (REST JSON, REST XML, GraphQL, etc) small differences make it difficult to reuse connectors. Think of a connector as a way to interface with a single system. | |||
There are two types of connectors: Internal (Java) and External (REST) | There are two types of connectors: Internal (Java) and External (REST) | ||
| Line 6: | Line 14: | ||
== Internal Connectors == | == Internal Connectors == | ||
Internal connectors are written in Java and installed on BigID Scanners. These connectors are developed by BigID and shipped with the product for you to use out of the box. A full list of the connectors is available at https://www.docs.bigid.com/bigid/docs/bigid-supported-data-sources | Internal connectors are written in Java and installed on BigID Scanners. These connectors are usually developed by BigID and shipped with the product for you to use out of the box. A full list of the connectors is available at https://www.docs.bigid.com/bigid/docs/bigid-supported-data-sources | ||
== External Connectors == | == External Connectors == | ||
External connectors are developed by you. They are implemented as a REST API, | External connectors are developed by you. They are implemented as a REST API, which means you can use whatever programming language you feel most comfortable in to make them. | ||
External connectors come in two different flavors depending on the underlying data type. Unstructured connectors are for data sources that contain files. Structured connectors are for data sources that contain database tables. | External connectors come in two different flavors depending on the underlying data type. Unstructured connectors are for data sources that contain files. Structured connectors are for data sources that contain database tables. | ||
The Swagger file for unstructured connectors is located | ===Additional Resources=== | ||
*The Swagger file for unstructured connectors is located [https://developer.bigid.com/images/6/69/ConnectorGenericApiUnstructured.yaml here.] | |||
The Swagger file for structured connectors is located | *The Swagger file for structured connectors is located [https://developer.bigid.com/images/4/45/Structured.yaml here.] | ||
*To learn how to install a generic connector, follow our instructions [[Connectors/Installing | here.]] | |||
*For a checklist of items to figure out before you build a connector go [[Connectors/Checklist | here.]] | |||
Latest revision as of 11:50, 22 June 2021
- What do connectors add to your BigID system
- What types of connectors exist
BigID Connectors allow your BigID system to provide insights about new types of data. Whether that's a known data type like CSV from a new type of data source or something completely new to the BigID ecosystem, a connector will allow you to bring BigID's data discovery capabilities to that system.
Why do we need connectors?
Every data source has its own way of communicating with third parties. Some data sources return information nicely organized, others return it as a jumbled mess. In order for BigID to give you the insights you expect, data needs to be fed to BigID in a consistent way. Connectors work as translators between the multitude of formats that data sources have adapted to the standard format BigID expects. Note that even if the data format is the same (REST JSON, REST XML, GraphQL, etc) small differences make it difficult to reuse connectors. Think of a connector as a way to interface with a single system.
There are two types of connectors: Internal (Java) and External (REST)
Internal Connectors
Internal connectors are written in Java and installed on BigID Scanners. These connectors are usually developed by BigID and shipped with the product for you to use out of the box. A full list of the connectors is available at https://www.docs.bigid.com/bigid/docs/bigid-supported-data-sources
External Connectors
External connectors are developed by you. They are implemented as a REST API, which means you can use whatever programming language you feel most comfortable in to make them.
External connectors come in two different flavors depending on the underlying data type. Unstructured connectors are for data sources that contain files. Structured connectors are for data sources that contain database tables.