Supervised Shuffle Test Model Release

We release the roberta-large model that was finetuned on the Is Shuffle task and achieves a >97% accuracy at the task on the standard Wall Street Journal-based test set.

The file name is: cls_roberta-large_supervised_shuffle.bin and an example of usage is provided in run_supervised.py.
The model can be loaded through the HuggingFace library:

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

tokenizer = AutoTokenizer.from_pretrained("roberta-large")
model = AutoModelForSequenceClassification.from_pretrained("roberta-large")

coherence_model.load_state_dict(torch.load("cls_roberta-large_supervised_shuffle.bin"))