fix(update): don't reuse a previously locked dependency if multiple major versions are compatible

[stormshield-kg]

What does this PR try to resolve?

Fixes #5529.
Fixes #14115.
Fixes #14446.

How should we test and review this PR?

This first commit adds a test documenting the current behavior, and the second commit implements the fix and updates the test.

Additional information

[rustbot]

Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @epage (or someone else) some time within the next two weeks.

Please see the contribution instructions for more information. Namely, in order to ensure the minimum review times lag, PR authors and assigned reviewers should ensure that the review label (S-waiting-on-review and S-waiting-on-author) stays updated, invoking these commands when appropriate:

• @rustbot author: the review is finished, PR author should check the comments and take action accordingly
• @rustbot review: the author is ready for a review, this PR will be queued again in the reviewer's queue

[epage]

As a heads up, none of these issues are marked as A-accepted but instead S-needs-design (see Issues in contrib docs). It would generally be best to explore the high level ideas in the Issue before moving on to a PR (even if playing with an implementation locally) so we keep the solution space exploration in one place and leave the PR to focus on the implementation. Sometimes PRs come and go for the same issue and if we explore designs in PRs, it fractures the conversation.

@Eh2406 can better speak to whether we can shortcircuit that and move forward with this.

r? @Eh2406

btw another issue with multi-major version reqs is #9029 with #13535 as a proposed change for it.

[epage]

For myself, I feel uncomfortable how much resolver logic lives in PackageRegistry, I feel even more uncomfortable adding VersionPreferences to it and to trait Registry as well.

[bors]

☔ The latest upstream changes (presumably #14569) made this pull request unmergeable. Please resolve the merge conflicts.

[Eh2406]

My instinct has long been that this heuristic should just be removed. Apparently I tried that when I was a little baby contributor new to open source in #5718. A lot has changed since then, so it might work now.

Another approach discussed in that PR is to remove this heuristic and double our calls to resolve. We try resolving with --offline and only try normal resolution after the --offline did not work. But now that I say that (I think, without having try it), that would lead to cargo update randomly updating unrelated GIT dependencies. In addition to the performance impacts.

[Eh2406]

After some more investigation, this heuristic seems to be loadbearing for cleaning up different ways to refer to the same git package. For example a dependency might require the default branch (or the master branch) but what's available in the lock file is a particular checkout. This heuristic identifies that the particular hash could match and uses it. This seems accidental to me. I think there must be a better way of handling this. If we had a matches_with_compatible_git_source when iterating over locked_deps things could be solved much more elegantly. In the time I had available I did not find one laying around, but this still seems a promising direction.

[stormshield-kg]

This test checks this heuristic and is not related to git dependencies:

cargo/tests/testsuite/registry.rs

Line 2487 in 43e3fa9

fn add_dep_dont_update_registry_http() {

So passing a matches_with_compatible_git_source parameter will not be enough.

[Eh2406]

You are correct. I forgot to mention that case. That test was added at the same time as the heuristic. Based on the PR that added it was to allow use of cargo with fewer network requests. This was a big issue at the time because we did not yet have --offline. So if cargo decided that you needed a network request and you did not have network access you were stuck. I have not formally discussed it with the team, but my vote would be that we just remove that test.