Connection to a dataspot. data catalog
The dataspot. connection links the DQC Platform with your data catalog in both directions. You connect tables to data objects and rules to data quality indicators. After every ruleset check, the DQC Platform records a measurement on each linked indicator, so data stewards see current quality figures in the place where they govern the data. The connected objects in dataspot. link back to the table, the data source and the rule.
How the objects map
dataspot. | DQC Platform | What the connection does |
|---|---|---|
Data object | Table | You connect a table with the URL of its data object. The data object receives a link to the table |
System | Data source | The system the data object is deployed in receives a link to the data source of the table |
Data quality indicator | Rule | You link a rule to an existing indicator or let the DQC Platform create one. The indicator receives a link to the rule |
Measurement | Result of a rule in a ruleset check | Every finished check records one measurement per linked rule |
The links are stored in dataspot. in the custom property dqc_url of each object.
Prerequisites
The DQC Platform signs in as an application through Microsoft Entra ID and acts in dataspot. as a technical user. Ask your dataspot. and Entra administrators for:
An app registration in Microsoft Entra ID for the DQC Platform, with a client secret or a certificate
The token endpoint of your Entra tenant, in the form
https://login.microsoftonline.com/<tenant-id>/oauth2/v2.0/tokenThe scope of the dataspot. application in Entra ID, in the form
<application-id>/.defaultA technical user in dataspot. with an access key
The custom property
dqc_urlin dataspot. for data objects, systems and data quality indicators. The DQC Platform stores its links thereA collection in dataspot. in which the DQC Platform creates indicators
The import link in dataspot., if data stewards create rules from indicators. See "Create a rule from an indicator" below
If access to your dataspot. instance is restricted by IP address, allow the static IP of the DQC Platform: 3.123.94.228
Add the connection
Go to Connectors → Catalogs and click Add data catalog
Choose Dataspot as the type
Fill in the fields and save
Field | Value |
|---|---|
Name | A name for the connection |
Default indicator collection | The URL of the collection in which indicators are created, copied from dataspot. in the form |
dataspot. Auth URL | The token endpoint of your Entra tenant. It has to contain the tenant ID |
Client ID | The application (client) ID of the app registration |
dataspot. scope | The scope of the dataspot. application, ending in |
Authentication Method | Client Secret, or Client Certificate with the PEM-encoded private key and the SHA-1 thumbprint of the certificate |
dataspot. access key | The access key of the technical user. It can stay empty if the application was granted an app role for dataspot. in Entra ID |
A tenant has one dataspot. connection. Adding and editing catalog connections is reserved for owners.
Connect a table to a data object
Open the table and choose Add data catalog in the menu of the table
Paste the URL of the data object from dataspot. into Object URL, in the form
https://<server>/web/<database>/classifiers/<id>Click Save
The table is now connected. The menu entry changes to View data catalog and shows the connected object with a link to dataspot. To disconnect the table, open View data catalog and click Delete.
The data object receives the link to the table, and the system it is deployed in receives the link to the data source
Connecting a table needs the Edit permission on the table
Link a rule to a data quality indicator
Open the rule and choose Data catalog in the menu of the rule editor. The entry is available once the tenant has a dataspot. connection. There are two ways to link the rule:
Link an existing indicator: Paste the URL of the indicator into Object URL, in the form
https://<server>/web/<database>/indicators/<id>, and click SaveLet the DQC Platform create the indicator: Click Create in dataspot. The indicator is created in the default indicator collection with the name and the description of the rule. If the table is connected to a data object, the indicator is assigned to it: to the attribute named like the column when the rule checks exactly one column, otherwise to the data object itself
What linking does:
The indicator receives the link to the rule, and its measurement type is set to amount, because a rule reports a number of rows out of a total
A rule is linked to one indicator. To link another one, click Unlink first. Unlinking also removes the link in dataspot.
Deleting the rule removes its link from the indicator. The indicator and its measurements stay in dataspot.
Linking a rule needs the Coordinate permission on the ruleset
After linking, the DQC Platform checks whether results can be recorded. If not, the message "Results will not reach the catalog" names the reason.
Measurements
After every finished ruleset check, the DQC Platform records one measurement for each linked rule:
Measurement in dataspot. | Value |
|---|---|
Amount | The number of rows without an issue: rows checked minus issues |
Total | The number of rows checked |
Computed on | The system the measurement belongs to, see below |
Date | The day of the check |
Comment | The name of the rule with a link to it, the number of issues and the failure rate |
dataspot. counts the rows that are fine, the DQC Platform counts the issues. A check of 15 rows with 3 issues is recorded as 12 of 15, with the comment "Issues 3 (failure rate 20%)"
An indicator holds one measurement per system and day. A later check on the same day replaces the measurement of that day
A check that cannot report its results still finishes normally
The system of a measurement
dataspot. requires every measurement to name the system it was computed on. The DQC Platform looks for it in this order:
The system in which the attribute or data object assigned to the indicator is deployed
A deployment of the indicator itself
The system of the data object the table is connected to
If none of them names a system, no measurement is recorded. The Notification Center then shows "Results not reported to the catalog" with the rules concerned. To fix it, assign the indicator in dataspot. to an attribute or data object that is deployed in a system.
Create a rule from an indicator
A data steward can start in dataspot. and turn an indicator into a rule through a link on the indicator. Your dataspot. administrator stores this link in dataspot., in this form, with the ID of the indicator at the end:
https://<your DQC Platform address>/apiV2/integrations/dataspot/?indicator_id=<indicator-id>
The DQC Platform reads the indicator and the objects it applies to, finds the connected table, builds a rule from the description of the indicator with AI and links the rule to the indicator. The page shows each step and opens the new rule at the end. If a step fails, it says what to change and where.
For the import to work:
The indicator is assigned to a data object, or to attributes of one, that is connected to exactly one table in the DQC Platform
The table has a ruleset
You can view the table and hold the Coordinate permission on its ruleset
An AI model is available in the tenant
The description of the indicator states the condition to check and the fields concerned
An indicator that already has a rule is not imported a second time. The name and the description of the new rule are written in the tenant language.
💡 Notes
If the sign-in is rejected, check the client secret or certificate, the access key and their expiry dates, and that the Auth URL contains your tenant ID
Learn more: Connection to an Alation data catalog, Notification Center, User management and access rights