-
Notifications
You must be signed in to change notification settings - Fork 253
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Proposal of snupkg restore functionality and debug improvement #10899
base: dev
Are you sure you want to change the base?
Conversation
Thank you so much for the contribution @BlackGad! We will keep this proposal open for the next couple of weeks to gather upvotes and feedback from the community. We'll then review it as a team and get back to you shortly. Thanks again in the meantime! If you're seeing this message, please 👍 or provide your feedback on this proposal in this PR as to why we should or shouldn't do this. Thank you everyone! |
Is it reasonable to advertise this PR for community? Maybe you can invite some via the twitter? |
@BlackGad I literally just tweeted a minute before your comment :) https://twitter.com/_JonDouglas/status/1399788019851407374 |
Well :) Is it time to continue or no? |
@BlackGad I have scheduled a review for this week & we will update this post shortly! Sorry for the delay as we're slightly behind this time of year. |
Nice to hear this. If you need any explanations or details fill free to ask in any form |
Did you manage to do a review? |
@JonDouglas congratulations once again! :) When you will be able to review this proposal? |
@loic-sharma will you be able to help drive this forward and share thoughts about the review with @BlackGad while I'm out? |
@JonDouglas , @loic-sharma guys? |
@JonDouglas traditional Monday ping. |
Hello, apologies for the late response. Thank you for creating this proposal! I definitely agree that our .NET debugging story is incomplete and needs work 😅 I'm not sure I fully understand your scenario just yet. Could you explain why you need to manually fetch your symbols instead of letting the debugger get them for you? Is it for improved exceptions at runtime? I'm also curious as to the benefits of adding "symbol gathering" to NuGet restore. If we provided a separate standalone tool, would that work for you? |
Details described in issue but there is a brief overview Debugging
Abstract exampleAs far as I understand you are Web developer so will try to explain in this domain (not accurate example I must say). Web example diagram
Blobs - byte array Generic application developmentNow lets change labels on the diagram
So what NuGet currently can do:
Basically it is artifacts split and publish mechanisms. Also there is only half of delivery mechanism implementation right now (nupkg only). What NuGet missing
Important
NotesIt is a brief overview without technical details. This PR and initial issue have additional information about technical implementation, API definitions, problem areas so must refer you to them :) This PR: #10899 And the answer
Current debugging mechanism assumes that you MUST have remote NuGet feed which also hosting Symbol server. Local file system feeds are not supported. There is no easy way to fill symbols manually explanation on real case (We are the legion that requires this %))
One more spike?) snupkg is the same nupkg. All procedures that can be applied to nupkg can be applied to snupkg (discovery, security, install etc). Also NuGet already have API to create and publish snupkg so will be at least strange decision to use external tool for install and use. |
@loic-sharma Simplest solution - application have no additional component references so it uses required nuget packages directly. Complex solutions requires additional packed modules and this modules require another internal modules so on. Real build often 1 hour CI pipeline build. End product bug means bug in some separate component and to debug and fix you need to have ability rapidly change and merge components locally. I already said at the very beginning that this functionality is first step to manage above situation %) |
This seems like an awesome feature to improve user's debugging/error experiences. I created a prototype for this proposal: https://github.com/loic-sharma/symbols How this works:
My findings:
Some open questions:
I'll ping some folks at Microsoft to get their thoughts on this idea. |
This will brake security for signed snupkg's :( #10793 (comment)
Symbols restore option must be opt-in only. Symbols restore process must not impact nupkg restore option and also must be controllable. Also you faced with one of issues that I collect earlier in source issue. #10793 (comment) I insist on same behavior for snupkg as for the nupkg. Current internal protocol has implementation for web and local resources (feeds) discovery with solved transfer/auth/security features. If snupkg will behave in different way - you will be forced to solve same features separately. snupkg is the same nupkg with some content and metadata filters. Also there are already implemented caching and restore mechanisms (for VS old and new project structures). There is no easy road for this feature because different parts of it impact NuGet, VS, NuGet registry teams :) |
Why would it break security? This is how your debugger works today. The assembly contains a checksum for its corresponding symbols (LINK). So once you download symbols, you can check it hasn't been tampered by recalculating its checksum. FYI, nuget.org does not verify .snupkg's signatures today. We do, however, check that the uploaded symbols match assemblies previously uploaded to nuget.org.
Yup, that's my thinking as well.
I see where you're coming from. That said, symbols have been around for decades and changing protocol details would mean losing decades worth of tooling and integrations. This is not a decision we should make lightly. The following problems are already solved for symbols:
That only leaves authentication. NuGet doesn't bring much to the table here as out of the box it only supports HTTP basic authentication and client certificate authentication. Hence, I don't think there's a strong case yet to move away from the existing symbol server protocol. |
Ok, I already mention earlier that it is first step for local debugging. If we add support for snupkg into the protocol - than we can manage main thing that I need for proper local debugging of composite applications. All features above cover more than I need, but purpose is organic for current implementation state of protocol. Idea behind all of this is to call command line to end product for proper assemblies and symbols substitution. Because nupkg become not only assemblies blob. Can't find my initial case, it is possible that it was missed somewhere in other messages. Real caseLets imagine we have multilayered product. Every layer it is some kind of nupkg.
Now client report a bug. After the research we understand that bug can be in Base.dll. To determine this we need restore symbols and debug in code issue line by line. Our nuget feed do not hosts symbol server. Ok we download symbols as an artifact and copy pdb files into proper location near the assemblies (file by file for every platform) then determine that bug there, so we need to fix it. We clone repo for base.dll and trying to fix an issue. To test, that issue was fixed we compile new base nuget package. Current behavior
Note: Package version will be available only after CI build. Base NuGet package content cannot be copied to output folder because of automatic build output files replacement. We are using 3.5, 4 and 5.0 net frameworks in same time. Now imagine that bug was inside package with device driver logic because some issues inside base.dll. We will receive dependency tree. And now we need to work with 3 repos waterfall substitution. Wished behavior
This is simplified developer flow. Ideally it would be 2 command lines. But, hope, it is a future. Not relevant to this purpose. |
And again. You insist that symbols restore handled by SSQP. What to do when you have no symbol server? Current builds already have assemblies and symbols in right place for debug. NuGet selects binaries and pack them to package, NuGet selects symbols and pack them to package as well, NuGet can install packed nupkg, but Nuget cannot install packed symbols! From my prospective it is the same situation like brief scheme above for web but at stage css unpack we tell browser - hey bro your css in another castle. Decide yourself how to fetch and link it yourself ;) and it is not my business that technically it similar to already delivered HTML |
My company manages shared dependencies for our desktop apps using NuGet. We like to ship symbols to our users so that automated error reports contain line numbers in crash stacks. This requires downloading symbols at build time. So we would find this feature useful, although most of our products have migrated to the new . We're using this hack when we collect the files to copy to our output directory:
|
This PR has been automatically marked as stale because it has no activity for 30 days. It will be closed if no further activity occurs within another 330 days of this comment. If it is closed, you may reopen it anytime when you're ready again, as long as you don't delete the branch. |
Hi, we have removed our "proposals" folder, so please move this proposal to the "accepted" folder. |
This PR has been automatically marked as stale because it has no activity for 30 days. It will be closed if no further activity occurs within another 330 days of this comment. If it is closed, you may reopen it anytime when you're ready again, as long as you don't delete the branch. |
Design for #10793