Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add scoring types GroupSum, GroupSumCond, GroupSumCheck #1251

Open
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

prandla
Copy link
Contributor

@prandla prandla commented Nov 20, 2023

These are 3 scoring types that are frequently used in Estonian olympiads, especially for more beginner-oriented tasks.

GroupSum gives points for each test proportionally for each subgroup. It doesn't have many functional advantages over Sum (or even just GroupMin and splitting each testcase into its own group), but it makes the contestant-facing UI consistent with GroupMin tasks (we generally have a few GroupSum tasks and then GroupMin for the rest). Also, it allows assigning different weights to different testcases in a somewhat intuitive manner (i.e. you give it the amount of points for the entire group, and the points get distributed evenly between the testcases).

GroupSumCond is useful for tasks where the question is of the form "find X, or if no such exist, print NONE". In these tasks, with GroupSum it'd be easy to get a lot of points by just printing NONE unconditionally. GroupSumCond allows designating some groups as "conditional", with the contestant only getting points for such groups if they also got some points from the unconditional groups. This allows moving all testcases where the answer is NONE into a conditional group, with the contestant only getting points if they also solve at least one of the non-NONE test cases correctly.

GroupSumCheck allows setting the whole group's points to zero in case of some particularly egregious error in one testcase. Admittedly less useful than the other 2 (I think we only needed it for BOI once).

I am aware that this PR is missing tests and documentation, but I'd want to first get feedback as to whether this feature is desired for CMS master.

@wil93
Copy link
Member

wil93 commented Jan 18, 2024

Related prior discussions: #639 #620

@prandla
Copy link
Contributor Author

prandla commented Jan 18, 2024

regarding lack of demand, we use GroupSum at least a few times per year, and it really isn't that big of a maintenance burden IMO (well, GroupSumCheck might be a bit of a different story as far as demand goes, but it too is tiny code-wise). issue 639 is interesting, i assume it would allow some refactoring and simplification of the scoring logic, but i don't know the codebase well enough to say how useful that would be.

@wil93
Copy link
Member

wil93 commented Jan 19, 2024

I think regardless of whether these scoretypes are useful enough to ship by default, it would make sense to have a separate PR for each of them. I also think GroupSum probably now has enough demand, so it can be added (with tests).

@stefano-maggiolo what do you think about adding GroupSum now, and specifically about the idea in #639 to basically drop the standard Sum in favor of GroupSum?

@stefano-maggiolo
Copy link
Member

GroupSum: seems uncontroversial.
GroupSumCheck: can't this be implemented with GroupThreshold? Actually I think it would already work (as long as you set a very high threshold) as GT checks outcome > 0
GroupSomeCond: this seems quite involved to me (hence not generic enough to be in the repo); the idea to deal with those tasks was to use GroupMin/Mul to make sure that each subtask has a representation of None and non-None answers.

Regarding #639, the only objection I have is whether we can keep a simple contestant interface for "tasks that are effectively Sum tasks" even if we represent as GroupSum.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

3 participants