There was no increase in speed after installing flash-attn and xformer.

[kongwanbianjinyu]

Hello,

I installed the flash-attn and xformer packages to accelerate the training process, but noticed no speed improvement compared to slow_attn. I'm using 4 RTX 6000 GPUs to train VAR-d16 for 200 epochs, with a global_batch_size of 96 due to memory constraints. The whole training process would take roughly 1000 hours. Could you kindly share your hardware settings and the time required to train VAR-d16?

Best,
Jiachen