-
Notifications
You must be signed in to change notification settings - Fork 253
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
NuGet pack ignores nuspec versions [unwanted version normalization?] #3050
Comments
@timotei would you share more about your scenarios here? Are you expecting the version string to be an exact match as part of other scripts or tools? |
Well, we have a ton of internal libraries that are used throughout the end products (private NuGet repo based on Artifactory). Some scripts/tools generate them directly via the nuget pack, others are using some more legacy build systems and we managed to make some bridge between them (not possible/feasible to migrate them to the newer NuGet-based process). The issue comes from the fact that newer NuGet automatically normalizes the version, making the interop between the two types of projects hard to almost impossible. Basically, we have interop between Java and .NET libraries (via IKVM), and thus we have a specific format for the versions, which we must respect. That's why, it's very important (for us and I think in general) that tools that generate stuff, do it in a expected way, based on the .nuspec, because that impacts the entire dependency tree. So, I'd expect the |
I'd like to understand how this is working, are your scripts resolving nuget dependencies without nuget? Does IKVM consume nupkgs directly? Which parts of your build process read the nupkgs without nuget? |
Yeah, up until NuGet 3.4, we had in place a simple mechanism which would translate 1-to-1 from Gradle dependencies, to NuGet packages (which are basically IKVM-compiled assemblies - Java to .NET). We were able to create the Once NuGet 3.4+ went in, we found the issue that the versions wouldn't be mapped properly, all depending on whether the restored NuGet packages were created with NuGet 3.4+ or not. The thing is, nuget places the restored packages in folder with the full version, not the normalized one, which means, if a package was created with NuGet prior to 3.4 it will have |
This has been fixed in 3.5.0, you can find it here: https://dist.nuget.org/win-x86-commandline/v3.5.0-beta2/NuGet.exe
With normalization there is only one possible string, making this deterministic now. Paths can be determined upfront without opening every nupkg to check if it is the expected version. Here is how it works:
Examples: |
Hmm, yeah, but AFAICT, the 3 or 4 digits folder name will depend whether the NuGet package was generated with NuGet <3.4 or NuGet 3.4+, no? About the normalization part, isn't possible to make nuget pack not normalize the version if explicitly not wanting? Or is the whole 3.4+ Normalized Versions Saga about normalizing the behavior (pun intended) of all NuGet behavior so it uses normalized versions everywhere? (pack + install + restore) How I see this is, that NuGet is kinda forcing people to use normalized versions even if they don't want it, right :) ? |
Normalizing solves many issues within NuGet. I think your scenario is valid, but less common than the problems normalizing fixes. You can work around this problem by finding the normalized version from the package after packing and using that for your mappings instead of the -Version input. You could also modify the nuspec in the package after pack to un-normalize it. |
Does this mean that a PR that enables the NuGet Pack to generate the NuGet package "as is" from the NuSpec (without normalizing versions) would not be accepted into the client? IIRC, there was once a warning when doing a pack for 4 digits (that the version doesn't satisfy SemVer 2?), but that warning does not show up now, and instead it directly normalizes the version (not sure why normalize it only when ending in '0', since SemVer2 is 3 digits only nevertheless the last 4th digit) |
@timotei - yes, 3.4.* does this behavior during pack. Earlier versions didn't. |
Well, in the end we managed to get a post-NuGet pack step that denormalizes the version, to keep the previous behavior. So far it seems to work. But this is not a very satisfactory solution :( I'd rather have NuGet not change things I specifically set in the nuspec. |
This is now causing problems downstream as tools now start upgrading their nuget versions. Octopack now uses 3.4.x which is dropping the 4th part of the version number if it's zero, and breaking templated builds in our TeamCity. Please consider command line options |
@slaneyrw can you explain how TeamCity uses this? Which part of the build breaks due to this change? |
Octopack generates an nupkg file which we are expecting to be in a specific format {package name}.{version number}.nupkg that we are publishing to our internal ProGet feed. The version number is TC's system build number which is a 4 part number. The file name we expect is not present when it has been normalised. The TC configuration template is shared by many individual build projects. All build projects run off a dedicated nuspec file in each project, this file is patched with TC's system build number before an Octopack build. |
Today, I received a report from a user about NuGet mangling the version within the nuspec file that gets embedded into the nupkg file. Apparently, this "normalization" completely broke the package restore feature, with an error complaining about the package "not being found" until he manually re-added the final trailing zero. Linked to ticket [https://system.data.sqlite.org/index.html/info/e9573e2d12387877]. |
@mistachkin restore could only break if the user is changing the package and re-packing it using an existing id/version. That scenario is not supported since packages are cached based on the id/version, having different copies of the same id/version will cause other problems. |
Sorry, but no. The user stated the package would not restore until he manually "un-normalized" the version to add the trailing zero. To be perfectly honest, this is going to be a huge headache. The packages I manage have a total of around 3.3M downloads at this point. If even a small fraction of the users end up with issues like this, that will be a major problem. Looks like the solution is to stick with NuGet version 2.x until this is fixed. Meanwhile, I'll have to look into manually fixing the current production packages. Of course, that won't work either because I cannot ever delete anything to re-upload them with the same version. I'm very frustrated now. |
@mistachkin would you open a new issue for what you are seeing with restore along with some repro steps? This issue is around package creation, it sounds like you are seeing issues on the consuming side. |
Respectfully, I disagree. The real issue is that what ended up in the embedded nuspec file was not what I originally authored. Subsequently, this "normalization" ended up breaking package restore (and other features?). |
@mistachkin NuGet supports all both normalized and non-normalized versions during restore. The question is what part of the process is treating the nuspec version as a string instead of a version. |
That part of the process, whatever it is, by definition, does not "support" the version normalization. Meanwhile, some of the most popular non-Microsoft packages on NuGet.org are effectively broken. |
@mistachkin please open a new issue for the restore issues you are seeing. Any extra details on how this affects existing packages would be extremely helpful and I would like to understand it better to help you out. Normalizing the version at pack time only, as this issue covers, only affects new packages from what I can see. Users installing those new packages will have the version form used in the nuspec file from the start, so there will be no changes on the consuming side. |
I did not discover the issue myself and I do not know the reproduction steps. I will contact the original user and ask them to file a new issue. Done, email sent to original user. |
Do we have any direction on what NuGet will be doing in regards to normalization. Are we doing to be able to turn OFF this behaviour during pack ? |
I agree with @slaneyrw, there needs to be a -verbatim option (or similar) to the NuGet pack command that retains the originally authored nuspec file exactly as authored. |
We are having issues as a result of this too - We use a build counter version number throughout our build, deploy, and test pipelines and having a mismatch causes failures when using the package version in a file system path. Example scenario:
If a version is specified in the nuspec file then the output package should contain that exact version. We have had to roll back to a specific older version of NuGet because of this. The behaviour of using nuget pack should remain the same as previous versions, and a -Normalize switch should be present if you want Nuget to automatically normalize the version number. |
I like the idea of -verbatim or possibly -verbatimversion for pack to keep nuget from trying to help out here. Making non-normalized opt-in instead of by default makes sense for most users I think. It helps avoid duplicates having in the same folder on disk and the confusing builds that can result from that. |
Nuget version 4.0.0.2283 is also removing leading zeros: Do I have to fork my own repo to fix this? I hope not. |
Microsoft's response to this problem they've created and maintained has been a clear lesson on how NOT to behave (toward one's customers). |
Just asked on the powershell issue: |
@rrelyea - "put it back the way it was" should cover the necessary work for all of those affected instead of just for one group of your choosing. Then you can add your nuget.org specific restrictions |
@StingyJack is correct. |
@rrelyea Is the forced SemVer behavior going to be rolled back or not? |
Also tagging @aortiz-msft since this item has had a Triage discussion |
I see this issue has dragged on for nearly 4 years, without an end in sight. There must be some powerful forces wanting to ram SemVer down everyone's throats. |
I'm also interested in resolving this issue in 2021. |
NuGet client normalizes the version number passed by nugetPush and removes revision when it's equal to zero which causes nugetPush to predict package filename incorrectly, so we instead do this inside nugetPush and always remove zero revision before passing the version to NuGet client. Reference: NuGet/Home#3050 (comment) More examples: 0.3.211.0 -> 0.3.211 1.2.0.0 -> 1.2.0 1.0.2 -> 1.0.2 1.2.1.1 -> 1.2.1.1
@StormRider01 - ...and the stick was so big and heavy (caber toss?) they didnt have enough strength left to increment the major version when releasing this breaking change. Or for some other reason. The irony of coercing everyone* to use a specific versioning scheme and at the same time violating the most important part of that same versioning scheme should have been enough to revert the change or actually bump the major version. They dont even think that they did something wrong by trying to dictate version numbers, or that saying something like "we will only take a PR for it if the result requires the breaking behavior to be default and a cmd arg to be used to get previous behavior" is salting the wound. * everyone - the intended audience for this was only users of nuget.org, but it affected everyone that uses private package feeds |
The only thought that I have after reading this thread is best represented as an emoji, here you are — 🤡. SemVer is an unnecessary overkill for me and I'd like my package manager to not change the version of the package that I'm publishing without me asking it to do so. This issue is still applicable 6 years after it was firstly submitted, please stop ignoring it. |
Versions are strings that can be Semantic Versions. This remains an active issue in 2023. |
Hello,
It seems latest NuGet pack ignores the version set in the nuspec (or via the
-Version
override flag) and the dependencies' versions, by normalising them, even if nobody asked for it :(It seems this was "fixed" for this #2039, but to me this seems a super counter-intuitive change, that breaks (at least for us) the daily workflows.
For the meantime, we had to do a workaround to "fix" the versions in the generated nupkg. I'm hoping that this can be fixed either by removing this weird logic, or allowing the nuget pack command to generate nuget package which contains the same versions as in the nuspec.
I'm using
NuGet Version: 3.4.4.1321
.With this nuspec:
executing
nuget pack mylib.nuspec -Version 1.9.0.0-SNAPSHOT
generates this .nuspec:As we can see:
1.9.0-SNAPSHOT
instead of1.9.0.0-SNAPSHOT
(even if we specifically overrode it via the command line) - same happens with a non-SNAPSHOT version3.16.0
instead of3.16.0.0
mylib.1.9.0.0-SNAPSHOT.nupkg
The text was updated successfully, but these errors were encountered: