Compatibility across forks, or where the version number comes from?

[malininae]

Hello fellow ESMValToolers,

I have a very specific issue. I am working usually on my fork malininae:ESMValTool , I've been doing that for 2.5 years now, and I am trying to keep my main up-to-date and when there is a new release, I am merging the updates from ESMValGroup:ESMValTool/main into my fork. I never had an issue with that workflow until recently.

In late August this year, I created a branch sia_so in malininae:ESMValTool from the up-to-date at the time main with core being v2.6 and tool v2.5. Yesterday, I made a stupid mistake and accidentally updated all packages in my conda environment and had to re-create the environment and the tool. In between, I decided that it would be a good idea to update the tool to the newest version and merged latest commits from ESMValGroup:ESMValTool/main to malininae:ESMValTool/main and malininae:ESMValTool/sia_so. After that I tried to install the tool from scratch in sia_so, but I got the core version v2.6 and tool v2.0. I also tried to install the tool from malininae:ESMValTool/main, but the result was the same. I even reinstalled conda and mamba, but result the versions were still odd. Then, I copied ESMValGroup:ESMValTool/main, and that installs core v2.6 and tool v2.6.

Then I though I dreamed about the tool being v2.5 to begin with, and I tried to install the tool from my malininae:ESMValTool/extremes_bc branch, which was updated some time this summer. There I am getting tool version v2.5 as it should be. I did diff on malininae:ESMValTool/main and ESMValGroup:ESMValTool/main folders, and the differences seem to be only in the git origin there as it should be. git log also shows that there were indeed the changes from the ESMValGroup:ESMValTool/main pushed in. So it seems to be solely version number problem, which for some very weird reason got screwed up in the latest merge. I was wondering if someone else had a similar issue? Or if not, where does the version number come from? Is it hidden in some file? I tried to debug it but didn't go too far.

I know I can go to the earlier commits, I just don't see a need for that now. So any help or advise would be very appreciated! Thanks for reading this long opus!

[remi-kazeroni]

Hi @malininae,
Thanks for reporting this issue. I think your problem could (partly?) be related to the fact that the handling of version numbers has been changed between v2.5.0 and v2.6.0. While this used to be hard-coded in previous versions (see here for the Tool v2.5.0), this is now automatically computed using the setuptools-scm (see here for the Tool v2.6.0). This change was implemented in #2663 (see Description for some explanations). The same goes for the Core (see here). With v2.6.0, you get more accurate version numbers (e.g. 2.6.1.dev1+g03a42c8fd) for the Tool/Core if these were installed in developer mode (pip install --editable '.[develop]') and standard versions otherwise (e.g. 2.6.0). I wonder if you encountered an issue related to this change while trying to update your installation.

As for good practices on how to update an installation from a fork, I can't really say because I'm only using ESMValGroup:ESMValTool/main. Perhaps other developers would be able to advise here.

Then I though I dreamed about the tool being v2.5 to begin with

What was your motivation to stick to v2.5.0 despite the latest v2.6.0 release?

Finally, you may be interested in ongoing discussion on backwards-incompatible changes for recipe/diagnostic developers. You can find some more info in ESMValGroup/ESMValTool_Tutorial#263 and ESMValGroup/Community#7

[bouweandela]

Hi @malininae

The setuptools-scm package that @remi-kazeroni referred to gets its version numbers from git. Maybe there is some issue with the way you have that set up? What is the output of the command

git remote -v

?

You can also have a look at this document for more information on working with forks.

[zklaus]

Specifically, setuptools-scm will need the right tags, so make sure that you fetch the tags by adding --tags to your git fetch line.

[malininae]

Hi @malininae

The setuptools-scm package that @remi-kazeroni referred to gets its version numbers from git. Maybe there is some issue with the way you have that set up? What is the output of the command

git remote -v

?

You can also have a look at this document for more information on working with forks.

Hi @bouweandela,

the output is
origin https://github.com/malininae/ESMValTool.git (fetch)
origin https://github.com/malininae/ESMValTool.git (push)
I personally don't see any issue.

@zklaus, thanks for the tip! I will try to look, if it works soon!

[bouweandela]

The usual setup when working with a fork is to add two remotes, have a look at the post linked in my previous comment for more information. The result of git remote -v would then for example like this:

origin https://github.com/ESMValGroup/ESMValTool.git (fetch)
origin https://github.com/ESMValGroup/ESMValTool.git (push)
malininae https://github.com/malininae/ESMValTool.git (fetch)
malininae https://github.com/malininae/ESMValTool.git (push)

Like that git will retrieve the required version information from the main repository when you do when you do git fetch origin. You can edit the remotes that you have configured with the git remote command.

[malininae]

A small update for those who might have the same issue: adding upstream helped. Now my git remote -v looks like

origin  https://github.com/malininae/ESMValTool.git (fetch)
origin  https://github.com/malininae/ESMValTool.git (push)
upstream        https://github.com/ESMValGroup/ESMValTool.git (fetch)
upstream        https://github.com/ESMValGroup/ESMValTool.git (push)

with me updating the tool to 2.8 grabbed the correct tags, so I'm all golden :-)