Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Incorporating item and user biases in LightFM embeddings for accurate recommendations in Vespa #696

Open
Homa-Radaei opened this issue Jul 4, 2023 · 1 comment

Comments

@Homa-Radaei
Copy link

I am working on deploying a recommender system using LightFM and Vespa. I have trained a LightFM model and saved the item embeddings in Vespa for deployment. However, I am facing an issue with prediction accuracy as I can only use the dot product of user and item embeddings for prediction in Vespa. The LightFM model's prediction formula includes item_biases and user_biases, which are crucial for accurate recommendations (user_e . item_e + user_b + item_b). Unfortunately, Vespa does not support entering bias information for user and item embeddings, resulting in reduced prediction accuracy.

I would appreciate any guidance or suggestions on how to train the model without biases or how to collect the information of embeddings and biases together, so that I can make accurate predictions in Vespa.
Unfortunately, ignoring biases or concatenating them to embeddings led to worsened prediction accuracy.
Thank you for your help.

@Homa-Radaei Homa-Radaei changed the title Need guidance on incorporating item and user biases in LightFM embeddings for accurate recommendations in Vespa Incorporating item and user biases in LightFM embeddings for accurate recommendations in Vespa Jul 5, 2023
@Oussamahadder01
Copy link

Why do you only can use the dot product and not use biases too in Vespa ? As i don't know much about Vespa i don't know how it works. Because you can access the item embeddings and biases by writing the following code :
item_biases,item_embeddings=model.get_item_representations(features=item_feature_matrix)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants