# Accuracy RCA Scan

`scan_accuracy_metrics_rca` is the typical RCA scan. You can find an example notebook [here](https://github.com/ETIQ-AI/ml-testing/tree/main/RCA).

An example config file is as below:

```json
{
    "dataset": {
        "label": "income",
        "bias_params": {
            "protected": "gender",
            "privileged": 1,
            "unprivileged": 0,
            "positive_outcome_label": 1,
            "negative_outcome_label": 0
        },
        "train_valid_test_splits": [0.0, 1.0, 0.0],
        "remove_protected_from_features": true
    },
    "scan_accuracy_metrics": {
        "thresholds": {
            "accuracy": [0.8, 1.0],
            "true_pos_rate": [0.75, 1.0],
            "true_neg_rate":  [0.7, 1.0]           
        }
    },
    "scan_accuracy_metrics_rca": {
        "thresholds": {
            "accuracy": [0.8, 1.0],
            "true_pos_rate": [0.7, 1.0]          
        },
        "metric_filter": ["accuracy", "true_pos_rate"], 
        "minimum_segment_size": 1000
    }
}
```

The syntax to run the scan after logging the model and dataset is the following:

```python
snapshot.scan_accuracy_metrics_rca()
```

Like with all RCA scans the principle behind the scan is that it searches through different combinations of records and it finds those combinations for which the metric is outside the thresholds. As per the usual scans, you can set the thresholds for what constitutes an issue for your use case. You can also filter out the metrics you want/do not want RCA for, using, for example,:

`"metric_filter": ["accuracy", "true_pos_rate"]` 

To make the metrics per group meaningful, it assigns a minimum number of records that constitutes a group, but you can change this by using the following syntax/parameter as per config example below:

`"minimum_segment_size": 1000` 

{% hint style="warning" %}
The minimum segment size will impact the results. We recommend setting the minimum segment size at 2% of the sample size. However if 2% is less than a significant segment size for your sample (e.g. less than 1000), please increase it. By default the scans use 2% of your sample size.  
{% endhint %}

At the moment we only have results retrieval through the IDE and by snapshot using the following syntax and then call each of the elements.

```python
(segments_accuracy, issues_accuracy, issue_summary_accuracy)  = snapshot.scan_accuracy_metrics_rca()
```

The end results give business rules to the segments to help you understand the records you’re having an issue with.

We are working to add more retrieval methods.

Out of the box you can scan for the following metrics:

1. **Accuracy** - % correct out of total
2. **True positive rate** - the proportion positive outcome labels that are correctly classified out of all positive outcome labels
3. **True negative rate** - the proportion negative outcome labels that are correctly classified out of all negative outcome labels


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.etiq.ai/etiq-1.x-documentation/rca/accuracy-rca-scan.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.