Implementing support for embeddings

[someshsingh22]

This module needs to be planned well as we will need this for all our future implementations and will be required at all steps.
Currently, there are a lot of libraries available for this, common ones including Flair and Torchtext. We need a pipeline that will convert any textual input to vectors/matrices for further computation.

[Sharad24]

Torchtext has a module, no? Doesn't have transformers though, I think

…

On Fri, 15 May 2020, 11:06 Somesh Kumar Singh, ***@***.***> wrote: This module needs to be planned well as we will need this for all our future implementations and will be required at all steps. Currently, there are a lot of libraries available for this, common ones including Flair and Torchtext. We need a pipeline that will convert any textual input to vectors/matrices for further computation. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#62>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AH72FJZDGZOFFHURNKFZQE3RRTIG3ANCNFSM4NBIYT6Q> .

[someshsingh22]

Yes, Yes, you can load transformers embeddings using torchtext easily but it's not in the library bas of now. So we will need a separate module that has torchtext's own as well as other methods.