Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEATURE] Create a custom ArgillaNode to monitor Nodes/Components #3

Open
kursathalat opened this issue Jan 5, 2024 · 0 comments
Open

Comments

@kursathalat
Copy link
Contributor

Is your feature request related to a problem? Please describe.
With the current structure of Deepset Haystack, there is the possibity to use callback only with Agents. This prevents obtaining inner processes of the workflow. This PR #2 adds an ArgillaCallbackHandler to log LLM results and some metadata into Argilla with no other info.

Describe the solution you'd like
A custom ArgillaNode or a wrapper can be introduced to account for other processes as proposed here.

Describe alternatives you've considered
It is possible that this solution might not be ideal for some cases. For such a case, if a general CallbackManager is introduced by Deepset, then we can make use of it to further improve the integration between Argilla and Haystack.

Additional context
One thing to keep in mind is that Deepset is planning to introduce Haystack 2.0, which is currently in beta version. The changes here might be relevant for our work, like the change from Nodes to Components. More info on this page.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant