Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow setting separate time limit for interpreted languages #1247

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

prandla
Copy link
Contributor

@prandla prandla commented Nov 19, 2023

This allows setting a different (presumably higher) per-task time limit for some languages. This is mainly intended to make competing in Python viable without giving C++ users an advantage.

If a task doesn't have the interpreted language time limit set, the default time limit will be used instead. The set of interpreted languages is currently hardcoded as Python (2/3) and PHP.

I made this as a pull request directly because it's what we already use for Estonian olympiads, but admittedly it's not as configurable as it could be (mainly the list of which languages count as "interpreted"). The set of interpreted languages needs to be carefully chosen to make sure all such languages have roughly the same performance, otherwise competing in one language might be impossible, or in another language asymptotically bad solutions might get through the tests. Realistically it should be all of the slow languages that the jury has written example solutions for, with the time limit set accordingly to make all of them pass.

This makes me think that perhaps the set of languages should be a contest-specific setting, or at the very least something in cms.conf. But feedback is of course welcome.

@gollux
Copy link
Contributor

gollux commented Nov 29, 2023

Admitting that different solutions could be subject to different time/memory limits opens a whole can of worms :-)

First, there are fairness issues (described to some extent in my paper on security of grading systems). But I agree that it's not a significant problem in some contests (e.g., first rounds solved at home).

Second, once you admit the possibility, it is no longer clear that it makes sense to have just two categories of languages and that the only difference should be in time limits. Perhaps you can also set a different memory limit for Java solutions? Or to give C# and Java a little more time, but as much as to Python?

I think it would be much better (even though more complicated) to allow setting of limits per language in every task.

@prandla
Copy link
Contributor Author

prandla commented Nov 29, 2023

I am aware of the fairness issues, I tried that (embedding C code in Python using ctypes) once myself and only got points deducted after the end of a week-long contest, which i am still a little salty about :)
However, in my opinion, clearly forbidding such schemes in the contest rules would be enough.

I agree that setting per-language limits would be better. However, I haven't thought of any good way to show these in CWS. The "task overview" page lists each task as a single row, so if there's more than 2 sets of limits I think it'd get quite unwieldy. An individual task's statement page has a separate row for the time limit, this could either become multiple rows or have something like "C++: 1 sec, Python: 3 sec" as the value. And yes, memory limit should be configurable per-language too if we're already going that far.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

2 participants