Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[IX] Add all the relevant statuses in the Suggestions column of the extractor table #6977

Closed
Tracked by #6791
aphilop opened this issue Jul 5, 2024 · 5 comments
Closed
Tracked by #6791

Comments

@aphilop
Copy link

aphilop commented Jul 5, 2024

At the moment we display only a few statuses (eg. 54 values | 54 suggestions | 54 mismatching).
We need to add more statuses that are also displayed in the Stats & Filters panel

@txau
Copy link
Collaborator

txau commented Jul 5, 2024

I think we need clarification on what the different numbers in stats panel mean:

  • LABELED: it means the conditions for being labeled are met. But these conditions depend on the type of information you are extracting:

    • text, number, date: it has a value and a text selection
    • selects/relationships: it has a value, no need for text selection
    • all of them: something marking the "empty value" as the labeled data
  • Accuracy: calculation based on matches/mismatches for labeled data

  • Match: there is labeled data, there is a suggestion and they match

  • Mismatch: there is labeled data, there is a suggestion and they do not match

  • NON-LABELED: they do not meet the criteria for labeled

    • Pending: I don't know what this represents at this moment. We should take a look to the code
    • No suggestion: not sure about this one too. I guess it means, it is not obsolete, error or no context. The model returned something but it is empty? Should we rather say that the suggestion is "empty"?
    • No context: the model could not figure out a text? the context returned by the model is empty?
    • Obsolete: the model is training and extracting suggestions. This is a transitional state that only should happen WHILE training. So this should be always CERO after training
    • Others: I do not know if there is any other possible state apart from error. Probably we could change this to just "error". Make sure that errors and obsoletes are not the same (an error should still update the suggestion to the latest model, so it is no longer obsolete). Check the code to see if there is any other possible meaning for error.

We are also still waiting for a new category in non-labeled that is "Suggestion".

After reviewing these descriptions, we should update the side panel to reflect this, then the status could be shown in the table as well.

@txau
Copy link
Collaborator

txau commented Jul 26, 2024

Pending: I don't know what this represents at this moment. We should take a look to the code

Pending means they are non-labeled vs the labeled amount. We should clarify.

No suggestion: not sure about this one too. I guess it means, it is not obsolete, error or no context. The model returned something but it is empty? Should we rather say that the suggestion is "empty"?

No suggestions as of today is the default value before processing the suggestions. It means that either the model didn't return anything for that particular suggestion, or it returned empty (array, string, number). This needs revision fi we introduce the concept of "empty" as a valid labeled data.

No context: the model could not figure out a text? the context returned by the model is empty?

Rafa reported an incorrect logic in this setting in issue #7060

We are also still waiting for a new category in non-labeled that is "Suggestion".

This has already been implemented.

@txau
Copy link
Collaborator

txau commented Jul 29, 2024

Related to: #6804

We need to rationalize all these statuses.

@txau
Copy link
Collaborator

txau commented Jul 29, 2024

Also related: #6804

@aphilop
Copy link
Author

aphilop commented Aug 8, 2024

Will be handed with #7085

@aphilop aphilop closed this as completed Aug 8, 2024
@aphilop aphilop added this to the Information Extraction milestone Aug 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants