You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem? Please describe.
When using argilla for crowdsourced annotation workflows on large datasets, it would be necessary to enforce per-user limits on how many annotations can be submitted.
Describe the solution you'd like
I would like a way to enforce an upper limit on how many instances each user can annotate. This could be added as part of a DistributionStrategy, featuring a max_annotations_per_user parameter.
As a bonus, if an upper limit is set, the progress tracker could then reflect the personal progress of the user instead of the dataset progress.
Describe alternatives you've considered
A workaround is sharding the dataset into slices of max annotations size and selectively adding users to each; this creates huge overhead.
A different approach is having secondary app running that monitors user stats and removes them from the workspace if their upper limit is reached. Again, overhead.
The text was updated successfully, but these errors were encountered:
Is your feature request related to a problem? Please describe.
When using argilla for crowdsourced annotation workflows on large datasets, it would be necessary to enforce per-user limits on how many annotations can be submitted.
Describe the solution you'd like
I would like a way to enforce an upper limit on how many instances each user can annotate. This could be added as part of a
DistributionStrategy
, featuring amax_annotations_per_user
parameter.As a bonus, if an upper limit is set, the progress tracker could then reflect the personal progress of the user instead of the dataset progress.
Describe alternatives you've considered
A workaround is sharding the dataset into slices of max annotations size and selectively adding users to each; this creates huge overhead.
A different approach is having secondary app running that monitors user stats and removes them from the workspace if their upper limit is reached. Again, overhead.
The text was updated successfully, but these errors were encountered: