Are trg_mask and src_mask correct in dataset script

[anesh-ml]

in the Bert4Rec_dataset script, I see that trg_mask and src_mask outputs are same. Are they supposed to be same or different. It would be helpful if you could confirm it @vatsalsaglani

[vatsalsaglani]

It's been a long time since I worked on this. But as far as I remember here, the transformer treats source and target sequence as one sequence instead of source and target. The masked positions depict the valid positions for attention and the model can learn from the entire sequence.

As this architecture is encoder-only we can use the same masks given we use the same during prediction/inference.

[anesh-ml]

thanks, the accuracy of the model is stuck around 15%. Does it take lot of training time to improve? @vatsalsaglani