Install conbench CLI from pypi, not git clone

[austin3dickey]

We currently git clone conbench:

arrow-benchmarks-ci/buildkite/benchmark/utils.sh

Line 76 in 14ef68d

git clone https://github.com/conbench/conbench.git

This can lead to accidental breakages since we don't have explicit tests for the arrow-benchmarks-ci use case in the conbench repo. An example is that we recently merged a change to conbench that used list[str]. Since arrow-benchmarks-ci is on python 3.8, all benchmarks started failing.

If we pegged the conbench dependency to a version from PyPI (and maybe if conbench were a little better about versioning 🙂) we could trust that arrow-benchmarks-ci is at least opting in to changes in conbench. That way, whoever bumps the conbench dependency peg on the arrow-benchmarks-ci side could be looking out for possible failures.

[jgehrcke]

we could trust that arrow-benchmarks-ci is at least opting in to changes in conbench. That way, whoever bumps the conbench dependency peg on the arrow-benchmarks-ci side could be looking out for possible failures.

Exactly. That's the idea of pinning versions. :-)

IMO the important aspect is to specify a specific build/checkout, i.e. clone is also fine as long as we say which commit to check out.

And as I noted in conbench/conbench#618 (comment) a CLI should not rely on system Python's properties if little surprises are the goal (should bring its own interpreter).