Etiq Docs
Search…
Config
A config file lets you quickly set parameters, thresholds, metrics, and decide which scans to run. We provide suggested config files so that you can start straight away, but as you run scans for your use case, you’ll be able to customise the thresholds and metrics you use. Then once you are happy with a given config you can use it over and over again and share across your team to ensure consistency from one model to the next.
The config is structured as follows:
- Dataset parameters
- Bias parameters if applicable
- Scans you want included, the metrics and thresholds for each
If you’re not sure about the thresholds and metrics, use the config we provide with the tool, and just make sure to adjust your feature names as appropriate
To load the config use the below:
1
etiq.load_config("./config_demo.json")
Copied!
Example config below:
1
{
2
"dataset": {
3
"label": "income",
4
"bias_params": {
5
"protected": "gender",
6
"privileged": 1,
7
"unprivileged": 0,
8
"positive_outcome_label": 1,
9
"negative_outcome_label": 0
10
},
11
"train_valid_test_splits": [0.0, 1.0, 0.0],
12
"remove_protected_from_features": true
13
},
14
"scan_accuracy_metrics": {
15
"thresholds": {
16
"accuracy": [0.8, 1.0],
17
"true_pos_rate": [0.6, 1.0],
18
"true_neg_rate": [0.6, 1.0]
19
}
20
}
21
}
22
Copied!
At the moment, if you run a scan, it will run all the different types of issues, and metrics included in the respective scan suite regardless if you set the metric and the threshold in the config. It is on our roadmap to allow the user to customize the scan suite entirely and exclude certain metrics if needed.
Last modified 1d ago
Copy link