Data Quality Pipelines¶
Quality Rules Execution¶
Start segment: [RECORD_UPSERT_QUALITY_START] - defines an entity/lookup entity for which the quality rules will be executed.
Segment of the Point type: [RECORD_UPSERT_QUALITY_POINT] - perfoms the assigned quality rules for all validity periods of the record.
Segment of the Gate type: [RECORD_UPSERT_QUALITY_GATE] - indexes the calculated quality rule errors and applies the enrichment results to the record.
Finish segment: [RECORD_UPSERT_QUALITY_FINISH] - returns the result of quality rule execution.
When Saving Record Changes¶
Start segment: [RECORD_UPSERT_START] - saves record changes.
Segment of the Connector type: [RECORD_QUALITY_CONNECTOR] - checks quality rule sets assigned to the entity/lookup entity and starts the quality rules execution pipeline for the records (if the records have assigned rule sets).
Finish segment: [RECORD_UPSERT_FINISH] - returns the result of saving record changes.
Forming Quality Rule Errors for Records¶
Start segment: [RECORD_GET_START].
Segment of the Connector type: [RECORD_GET_QUALITY_CONNECTOR] - allows you to retrieve the calculated quality rule errors for the requested record.
Finish segment: [RECORD_GET_FINISH].
Figure 1. Example of completed pipeline