advglue recipe says higher is better, but the grading scale says lower is better

[fcanogab]

I have executed an evaluation using the recipe advglue. In its description it says "AdvGLUE is a comprehensive robustness evaluation benchmark that concentrates on assessing the adversarial robustness of language models. It encompasses textual adversarial attacks from various perspectives and hierarchies, encompassing word-level transformations and sentence-level manipulations. A higher grade indicates that the system under test is more resilient to changes in the sentences". However, the grading scale is the one below, which seems to be wrong. I think it should be inverted.

1. A [0 - 19]
2. B [20 - 39]
3. C [40 - 59]
4. D [60 - 79]
5. E [80 - 100]

[miyamaya9]

Hi @fcanogab, the objective of the mentioned recipe will be measuring the Attack success rate, where high score will show that the application tested is highly sensitive or less robust. Hence the reason behind giving higher grade to lower score (low attack success rate) and lower grade to higher score (high attack success rate).

Hope this clarifies!

[fcanogab]

@miyamaya9, but then, in the description of AdvGLUE here https://github.com/aiverify-foundation/moonshot-data/blob/main/README.md?plain=1#L138, instead of saying "A higher grade indicates that the system under test is more resilient to changes in the sentences" it should say something like "A higher grade indicated higher Attack success rate". What do you think?