Detect if restore used NuGetAudit or not for PackageReference projects

[zivkan]

NuGet Product(s) Involved

MSBuild.exe, dotnet.exe

The Elevator Pitch

Engineering teams responsible for CI build compliance may want to enforce that NuGetAudit runs during restore. Therefore, there should be a way to detect if NuGetAudit was actually used when the last restore ran.

Note

The first version of this will not work when RestoreUseStaticGraphEvaluation is true

Additional Context and Details

No response

[zivkan]

from: dotnet/arcade#15019 (comment)

When <auditSources> is not empty, but a vulnerability database couldn't be downloaded, NuGet will raise a NU1905 warning. This can be elevated to an error. However, NuGetAudit will not run if the MSBuild property NuGetAudit is set to false. So, you could enforce by 1. checking that nuget.config has at least one audit source (no easy way, needs xml parsing, or use dotnet nuget config get all, and parse the output) and 2. make sure NuGetAudit isn't set to false (dotnet sln list and dotnet msbuild -getProperty:NuGetAudit, or search all project.asset.json files for $.project.restore.restoreAuditProperties.enableAudit)

[zivkan]

from dotnet/arcade#15019 (comment)

If we're making changes to NuGet/MSBuild targets to make auditing behavior more detectable, we should just make those changes in such a way that we can detect desired behavior during the build, rather than after.

I think it needs to be detectable afterwards as well, because not all external customers are going to want to script their builds in MSBuild.

@mmitche do you have suggestions for how you'd like it to work within msbuild? I don't consider myself very experienced in MSBuild (I'm sure I am compared to external customers, but not internal customers). Before this week I didn't know that targets even had the Returns="..." syntax. So, I'm probably not the right person to design something that can actually be actioned by build scripts.

I'm guessing the best option would be for the task & target to return one item per project that was restored, and they'd have metadata saying if audit ran or not. This way you can detect if there are any projects where audit did not run, but also detect if there are any projects in the solution that were not restored.

Any other advice?