You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This can lead to accidental breakages since we don't have explicit tests for the arrow-benchmarks-ci use case in the conbench repo. An example is that we recently merged a change to conbench that used list[str]. Since arrow-benchmarks-ci is on python 3.8, all benchmarks started failing.
If we pegged the conbench dependency to a version from PyPI (and maybe if conbench were a little better about versioning 🙂) we could trust that arrow-benchmarks-ci is at least opting in to changes in conbench. That way, whoever bumps the conbench dependency peg on the arrow-benchmarks-ci side could be looking out for possible failures.
The text was updated successfully, but these errors were encountered:
we could trust that arrow-benchmarks-ci is at least opting in to changes in conbench. That way, whoever bumps the conbench dependency peg on the arrow-benchmarks-ci side could be looking out for possible failures.
Exactly. That's the idea of pinning versions. :-)
IMO the important aspect is to specify a specific build/checkout, i.e. clone is also fine as long as we say which commit to check out.
And as I noted in conbench/conbench#618 (comment) a CLI should not rely on system Python's properties if little surprises are the goal (should bring its own interpreter).
We currently git clone conbench:
arrow-benchmarks-ci/buildkite/benchmark/utils.sh
Line 76 in 14ef68d
This can lead to accidental breakages since we don't have explicit tests for the
arrow-benchmarks-ci
use case in the conbench repo. An example is that we recently merged a change to conbench that usedlist[str]
. Sincearrow-benchmarks-ci
is on python 3.8, all benchmarks started failing.If we pegged the conbench dependency to a version from PyPI (and maybe if conbench were a little better about versioning 🙂) we could trust that
arrow-benchmarks-ci
is at least opting in to changes in conbench. That way, whoever bumps the conbench dependency peg on thearrow-benchmarks-ci
side could be looking out for possible failures.The text was updated successfully, but these errors were encountered: