Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Insert benchmark defaults for params in run_one() #129

Open
alistaire47 opened this issue Feb 1, 2023 · 0 comments
Open

Insert benchmark defaults for params in run_one() #129

alistaire47 opened this issue Feb 1, 2023 · 0 comments

Comments

@alistaire47
Copy link
Contributor

Currently run_one() does not call get_default_parameters(), so while benchmark parameter defaults are used when running, if not specified they will not show up in tags as they should.

One way to deal with this would be to call get_default_parameters() on the params and ensure the result has exactly one row. (Note we can't do this right now, because voltrondata-labs/benchmarks is sending cpu_count = NULL to run_one(), which run_one() is currently interpreting as "do nothing about CPU count in the script", whereas get_default_params() will turn that into c(1L, parallel::detectCores()). For Arrow in particular, arrow:::GetCpuThreadPoolCapacity() appears equal to parallel::detectCores(), but changing the way we set it may set the cpu_count tag and break histories. We probably should, but will need to go clean them up.)

Alternately we could validate based on formals(bm$setup) and insert or error if any are not specified, but maintaining an alternate version of get_default_params() would increase the complexity and maintenance burden.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant