Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improving GPT's instruction following ability of non english prompts #440

Open
wants to merge 10 commits into
base: main
Choose a base branch
from

Conversation

Raghavan1988
Copy link
Contributor

GPT 3.5's response quality in non english prompt is usually poorer compared to english atleast for south asian and middle eastern languages due to lesser representation in training data.

In this tutorial, i showed a simple translation trick improves the response quality anecdotally.

Showing GPT 3.5's gap in non english prompts and simple technique on how to improve response quality
@Raghavan1988
Copy link
Contributor Author

requesting review @ezzcodeezzlife @OlesiaZinchenko

@DonGuillotine DonGuillotine self-assigned this Jan 28, 2024
Copy link
Collaborator

@DonGuillotine DonGuillotine left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hello @Raghavan1988,

This is a really interesting tutorial, I'm sure people from non English backgrounds will benefit a lot from this.

I have a few suggestions to make your tutorial even more effective and compliant with the guidelines:

  • You have effectively used one # header and several ## H2 headings, Kindly add a specific H2 heading for "Topic Input" and another one to summarize all previously discussed points.
  • You've provided code snippets showing how to implement the translation layer. To make the tutorial more beginner-friendly, I suggest expanding the code implementation section. Specifically, it would be helpful to include a more complete example of how GPT-3.5 processes the translated prompt.
    A possible enhancement could be a full implementation example that includes:
    • A function for translating non-English prompts to English.
    • Processing the translated prompt with GPT-3.5.
    • Translating the response back to the original language.
    • A simple example demonstrating the entire flow from input to output.

These suggestions will make your tutorial more accessible to beginners but also give them a practical tool to experiment with.

Thank you for your contribution to the community 🚀

@Raghavan1988
Copy link
Contributor Author

Thanks @DonGuillotine for the review. I made changes to create 3 functions

- Updated Header Structure
- Corrected Markdown format for URLs with a custom text label
@DonGuillotine
Copy link
Collaborator

DonGuillotine commented Feb 5, 2024

Thanks @DonGuillotine for the review. I made changes to create 3 functions

Hello @Raghavan1988,

It's great to see the updates you have made 🎉, I made some updates to the Header and Hyperlinks, kindly note it and apply to future tutorials.

Here are the final finishing touches to this great tutorial:

  • Remove the flask installation as it was not used
  • Use the latest version of langchain and openai. For example in the latest version of langchain from langchain.llms import OpenAI is no longer supported, rather use: from langchain_community.chat_models import ChatOpenAI
  • When I tested your code, especially the translate_prompt_from_language_x_to_english function I got errors suggesting the library fails to parse the response it receives from the Google Translate API. The documentation clearly states this:

Due to limitations of the web version of google translate, this API does not guarantee that the library would work properly at all times. (so please use this library if you don’t care about stability.)

Please consider using an alternative library that provides similar functionality such as translate or pydeepl

Looking forward to the updates, great work so far @Raghavan1988 🥇

Copy link
Contributor Author

@Raghavan1988 Raghavan1988 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @DonGuillotine for very thoughtful review.
I made the following changes.

  1. I removed langchain and used only OpenAI API since latest langchain APIs are different.
  2. I removed googletrans and used "translate" package as per your review.
  3. Added the end to end flask code with pointers to working github repository that i tested

Apologies for the delay in addressing your review. Please let me know if it looks good for merge or any further changes needed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants