RCA Type Scans
Root Cause Analysis Type Scans
Some scans are simple tests that show whether a metric is above or below certain thresholds set by the user. Other scans are more complex and look more in depth at what could be causing the issue. We call this RCA - root cause analysis. These scans will help you discover segments of customers or groups of records for which the model has a lower than expected accuracy or groups for which bias thresholds are not met.
You should expect a level of variability of your model for different parts of your dataset. However, if the variability is too high, with larger segments failing you might want to understand the causes of this, and address them. These scans help you find out exactly which segment has an issue, which should help you fix it sooner.
Imagine if only a part of the data drifted or only a segment is underperforming, your overall tests might not pick up on it, but this test would. While you can pre-set segments you are interested in, if you just run the scan as is, it will discover problematic segments on its own.
This pipeline is experimental.
RCA type tests are also very useful when it comes to the drivers that impact a model’s decision. We don’t include explainability scans in our public release. However if this is something you need, email us [email protected].
Last modified 1yr ago