You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
This repository has been archived by the owner on Feb 17, 2024. It is now read-only.
MLQA only have eval/test sets, could you please assist me and clarify on which dataset you have trained the model for the zero-shot experience? The result reported in the paper, are they computed on the eval set or the test set?
thank you.
The text was updated successfully, but these errors were encountered:
It says so in the paper. The English zero-shot evaluation on MLQA was done using SQuAD and the multilingual one was conducted using SQuAD + translations of SQuAD in the target languages, i.e. Arabic, Spanish, English, German, Hindi, Vietnamese and simplified Chinese. The MLQA set used is most likely the test one since its intended use is benchmarking.
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Hi
MLQA only have eval/test sets, could you please assist me and clarify on which dataset you have trained the model for the zero-shot experience? The result reported in the paper, are they computed on the eval set or the test set?
thank you.
The text was updated successfully, but these errors were encountered: