BigID API/Metadata Export Tutorial: Difference between revisions
No edit summary |
No edit summary |
||
| (3 intermediate revisions by the same user not shown) | |||
| Line 11: | Line 11: | ||
{{:InformationTemplates:BigID_Catalog}} | {{:InformationTemplates:BigID_Catalog}} | ||
<html><center><img class="diagram" src="https:// | <html><center><img class="diagram" src="https://wiki-images.bigid.tools/cdn/objects.svg"/></center></html> | ||
For unstructured data, we discover information directly about and inside the files. For example, we can see that there are phone numbers within a file, or that a file matches a machine learning model for an invoice. | For unstructured data, we discover information directly about and inside the files. For example, we can see that there are phone numbers within a file, or that a file matches a machine learning model for an invoice. | ||
For structured data we discover data about the data inside of the columns as well as the columns themselves. For example we would know that a database ''Users'' has a column named State and that column contains values of the type US State Abbreviation. This means that if we want information about specific columns we will need to perform an extra API request. The below API request shows how to retrieve the information about a specific table. The column parameter follows the format of | For structured data we discover data about the data inside of the columns as well as the columns themselves. For example we would know that a database ''Users'' has a column named State and that column contains values of the type US State Abbreviation. This means that if we want information about specific columns we will need to perform an extra API request. The below API request shows how to retrieve the information about a specific table. The column parameter follows the format of Data Source Name.schema name.table name. | ||
<html> | <html> | ||
<iframe style="border:0px; width:100%; height:400px; border-radius:10px;" src="https:// | <iframe style="border:0px; width:100%; height:400px; border-radius:10px;" src="https://apiexplorer.bigid.tools/?url=data-catalog%2Fobject-details%2Fcolumns%3Fobject_name%3DDevices.rockstream.Devices&method=GET&headers=%5B%7B%22name%22%3A%22Authorization%22%2C%22value%22%3A%22SAMPLE%22%7D%5D"></iframe> | ||
</html> | </html> | ||
| Line 24: | Line 24: | ||
<html> | <html> | ||
<iframe style="border:0px; width:100%; height:400px; border-radius:10px;" src="https:// | <iframe style="border:0px; width:100%; height:400px; border-radius:10px;" src="https://apiexplorer.bigid.tools/?url=data-catalog%2Fobject-details%2Fattributes%3Fobject_name%3DSales%2520Drive.devops%2540bigiddemo.com%252FEducation%252FProspects4-Restricted.docx&method=GET&headers=%5B%7B%22name%22%3A%22Authorization%22%2C%22value%22%3A%22SAMPLE%22%7D%5D"></iframe> | ||
</html> | </html> | ||
By using these three API calls you can find out information about any element being scanned by your BigID system. | |||
[[Category:Tutorial]][[Category:API]] | |||
Latest revision as of 19:43, 4 March 2022
- What the BigID data catalog can be used for
- What is the structure of the BigID Catalog
- How to retrieve data from each part of the BigID Catalog Structure
The BigID Catalog
The BigID catalog provides a view into all of your data. It allows you to see the types of data being held in each column, the access rights for that data and how it relates to other information across your systems. Below is a demo of the catalog through the BigID UI:
Since we care about the data within the API as opposed to the UI, let's see what the API response that powers this page looks like.
In the results section of the response you'll see a listing much like that in the UI. Each of these items is an object. An object is:
- A database table in a structured data source
- A file in an unstructured data source
For unstructured data, we discover information directly about and inside the files. For example, we can see that there are phone numbers within a file, or that a file matches a machine learning model for an invoice.
For structured data we discover data about the data inside of the columns as well as the columns themselves. For example we would know that a database Users has a column named State and that column contains values of the type US State Abbreviation. This means that if we want information about specific columns we will need to perform an extra API request. The below API request shows how to retrieve the information about a specific table. The column parameter follows the format of Data Source Name.schema name.table name.
For unstructured data sources, you can use the information returned from the top level catalog call or use the attributes call to get more detailed information like below:
By using these three API calls you can find out information about any element being scanned by your BigID system.