Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WebSocket exception with LLVM and full AOT Enabled in .NET Android App #104599

Open
kcrg opened this issue Jul 7, 2024 · 31 comments
Open

WebSocket exception with LLVM and full AOT Enabled in .NET Android App #104599

kcrg opened this issue Jul 7, 2024 · 31 comments

Comments

@kcrg
Copy link

kcrg commented Jul 7, 2024

Android framework version

net8.0-android, net9.0-android

Affected platform version

VS 17.11 Preview and 17.10 Stable, .NET 8.0.6 and .NET 9.0.0-preview.5.24306.8

Description

I am encountering an issue with WebSocket functionality in a .NET Android application when LLVM and full AOT is enabled. The application fails to establish WebSocket connections.

Configuration:

<PropertyGroup Condition="'$(Configuration)|$(TargetFramework)|$(Platform)'=='Release|net8.0-android|AnyCPU'">
  <EnableLLVM>True</EnableLLVM>
  <AndroidEnableProfiledAot>False</AndroidEnableProfiledAot>
  <RunAOTCompilation>True</RunAOTCompilation>
</PropertyGroup>

When running the application with the above configuration, WebSocket connections throws exception. The same code works without issues when LLVM is disabled.

Expected Behavior:
The WebSocket connection should be established, messages should be sent and received without errors, and the connection should be properly closed.

Actual Behavior:
WebSocket connection throws exception when LLVM is enabled with the above configuration.

Stack Trace:

Message: net_webstatus_ConnectFailure
StackTrace: at System.Net.WebSockets.WebSocketHandle.ConnectAsync(Uri , HttpMessageInvoker , CancellationToken , ClientWebSocketOptions )
   at System.Net.WebSockets.ClientWebSocket.ConnectAsyncCore(Uri , HttpMessageInvoker , CancellationToken )
   at WebsocketLLVMIssue.WebSocketClient.ConnectAsync(String uri)
   at WebsocketLLVMIssue.MainActivity.ConnectAsync()

Additional Notes:

  • Disabling LLVM resolves the issue, indicating a potential conflict between WebSocket operations and LLVM optimizations.
  • A repro project is attached to this issue for further investigation.

Steps to Reproduce

  1. Create a new .NET Android project.
  2. Implement WebSocket client functionality.
  3. Add UI elements for connecting, sending messages, and disconnecting.
  4. Handle user interactions to manage WebSocket operations.
  5. Configure the project to use LLVM without AOT.
  6. Build and run the project on an Android device or emulator.
  7. Observe the behavior when trying to connect using WebSocket.

Or download this minimal project: WebsocketLLVMIssue.zip

Did you find any workaround?

nope

Relevant log output

logcat.txt

@grendello
Copy link
Contributor

grendello commented Jul 8, 2024

@kcrg Could you please collect more info for us by issuing the following commands from the VS Developer Prompt:

> adb shell setprop debug.mono.log default,assembly,mono_log_level=debug,mono_log_mask=all
> adb logcat -G 65M
> adb logcat -c
rem Start the crashing application here, wait for it to crash, then wait 5 more seconds and...
> adb logcat -d > logcat.txt

Then attach the resulting logcat.txt file to this issue, thanks!

@kcrg
Copy link
Author

kcrg commented Jul 8, 2024

@grendello Done, I just added the logcat.txt to the issue.
I had problem with adb shell debug.mono.log default,assembly,mono_log_level=debug,mono_log_mask=all
it returns /system/bin/sh: debug.mono.log: inaccessible or not found

Issue occures only in Release. Enabling AOT and LLVM in Debug mode seems wrong.

@grendello
Copy link
Contributor

@grendello Done, I just added the logcat.txt to the issue. I had problem with adb shell debug.mono.log default,assembly,mono_log_level=debug,mono_log_mask=all it returns /system/bin/sh: debug.mono.log: inaccessible or not found

Oops, my bad, sorry about that. The first command should be:

> adb shell setprop debug.mono.log default,assembly,mono_log_level=debug,mono_log_mask=all

And please attach logcat.txt directly here, to your next comment, thanks!

@kcrg
Copy link
Author

kcrg commented Jul 8, 2024

@grendello Here it is: logcat.txt

@grendello
Copy link
Contributor

@kcrg it looks like a runtime issue, I can't find anything that's specific to .NET for Android. @steveisok I don't have permissions to transfer this to dotnet/runtime, would you mind doing that and assigning somebody to take a look at this? Thanks!

@akoeplinger akoeplinger transferred this issue from dotnet/android Jul 9, 2024
@dotnet-policy-service dotnet-policy-service bot added the untriaged New issue has not been triaged by the area owner label Jul 9, 2024
Copy link
Contributor

Tagging subscribers to this area: @dotnet/ncl
See info in area-owners.md if you want to be subscribed.

@grendello grendello removed their assignment Jul 9, 2024
@vitek-karas
Copy link
Member

Looking into the log, this seems network related:

07-08 15:32:19.972  1492  5110 D VpnManagerService: Exception: java.lang.NullPointerException: Attempt to invoke interface method 'int com.samsung.android.knox.zt.networktrust.filter.IKnoxNetworkFilterService.getKnoxNwFilterHttpProxyPort(int, java.lang.String)' on a null object reference
07-08 15:32:19.972  1492  5110 D VpnManagerService: 	at com.android.server.VpnManagerService.getKnoxNwFilterHttpProxyPort(VpnManagerService.java:1945)
07-08 15:32:19.972  1492  5110 D VpnManagerService: 	at android.net.IVpnManager$Stub.onTransact(IVpnManager.java:1075)
07-08 15:32:19.972  1492  5110 D VpnManagerService: 	at android.os.Binder.execTransactInternal(Binder.java:1375)
07-08 15:32:19.972  1492  5110 D VpnManagerService: 	at android.os.Binder.execTransact(Binder.java:1311)

@simonrozsival could you please take a quick look if this is something we've seen before or it's new...

@simonrozsival simonrozsival self-assigned this Jul 9, 2024
@grendello
Copy link
Contributor

grendello commented Jul 9, 2024

@vitek-karas @simonrozsival that exception is a red herring, it comes from a different (system) process. The one that is relevant is the following:

07-08 15:32:19.814 15061 15061 D AndroidRuntime: Shutting down VM
--------- beginning of crash
07-08 15:32:19.814 15061 15061 E AndroidRuntime: FATAL EXCEPTION: main
07-08 15:32:19.814 15061 15061 E AndroidRuntime: Process: com.companyname.WebsocketLLVMIssue, PID: 15061
07-08 15:32:19.814 15061 15061 E AndroidRuntime: android.runtime.JavaProxyThrowable: [System.Net.WebSockets.WebSocketException]: net_webstatus_ConnectFailure
07-08 15:32:19.814 15061 15061 E AndroidRuntime:        at System.Net.WebSockets.WebSocketHandle+<ConnectAsync>d__22.MoveNext(Unknown Source:0)
07-08 15:32:19.814 15061 15061 E AndroidRuntime:        at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw(Unknown Source:0)
07-08 15:32:19.814 15061 15061 E AndroidRuntime:        at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Unknown Source:0)
07-08 15:32:19.814 15061 15061 E AndroidRuntime:        at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Unknown Source:0)
07-08 15:32:19.814 15061 15061 E AndroidRuntime:        at System.Net.WebSockets.ClientWebSocket+<ConnectAsyncCore>d__21.MoveNext(Unknown Source:0)
07-08 15:32:19.814 15061 15061 E AndroidRuntime:        at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw(Unknown Source:0)
07-08 15:32:19.814 15061 15061 E AndroidRuntime:        at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Unknown Source:0)
07-08 15:32:19.814 15061 15061 E AndroidRuntime:        at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Unknown Source:0)
07-08 15:32:19.814 15061 15061 E AndroidRuntime:        at WebsocketLLVMIssue.WebSocketClient+<ConnectAsync>d__4.MoveNext(Unknown Source:0)
07-08 15:32:19.814 15061 15061 E AndroidRuntime:        at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw(Unknown Source:0)
07-08 15:32:19.814 15061 15061 E AndroidRuntime:        at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Unknown Source:0)
07-08 15:32:19.814 15061 15061 E AndroidRuntime:        at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Unknown Source:0)
07-08 15:32:19.814 15061 15061 E AndroidRuntime:        at WebsocketLLVMIssue.MainActivity+<ConnectAsync>d__5.MoveNext(Unknown Source:0)
07-08 15:32:19.814 15061 15061 E AndroidRuntime:        at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw(Unknown Source:0)
07-08 15:32:19.814 15061 15061 E AndroidRuntime:        at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Unknown Source:0)
07-08 15:32:19.814 15061 15061 E AndroidRuntime:        at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Unknown Source:0)
07-08 15:32:19.814 15061 15061 E AndroidRuntime:        at WebsocketLLVMIssue.MainActivity+<<OnCreate>b__4_0>d.MoveNext(Unknown Source:0)
07-08 15:32:19.814 15061 15061 E AndroidRuntime:        at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw(Unknown Source:0)
07-08 15:32:19.814 15061 15061 E AndroidRuntime:        at System.Threading.Tasks.Task+<>c.<ThrowAsync>b__128_0(Unknown Source:0)
07-08 15:32:19.814 15061 15061 E AndroidRuntime:        at Android.App.SyncContext+<>c__DisplayClass2_0.<Post>b__0(Unknown Source:0)
07-08 15:32:19.814 15061 15061 E AndroidRuntime:        at Java.Lang.Thread+RunnableImplementor.Run(Unknown Source:0)
07-08 15:32:19.814 15061 15061 E AndroidRuntime:        at Java.Lang.IRunnableInvoker.n_Run(Unknown Source:0)
07-08 15:32:19.814 15061 15061 E AndroidRuntime:        at Android.Runtime.JNINativeWrapper.Wrap_JniMarshal_PP_V(Unknown Source:0)
07-08 15:32:19.814 15061 15061 E AndroidRuntime:        at mono.java.lang.RunnableImplementor.n_run(Native Method)
07-08 15:32:19.814 15061 15061 E AndroidRuntime:        at mono.java.lang.RunnableImplementor.run(RunnableImplementor.java:31)
07-08 15:32:19.814 15061 15061 E AndroidRuntime:        at android.os.Handler.handleCallback(Handler.java:958)
07-08 15:32:19.814 15061 15061 E AndroidRuntime:        at android.os.Handler.dispatchMessage(Handler.java:99)
07-08 15:32:19.814 15061 15061 E AndroidRuntime:        at android.os.Looper.loopOnce(Looper.java:230)
07-08 15:32:19.814 15061 15061 E AndroidRuntime:        at android.os.Looper.loop(Looper.java:319)
07-08 15:32:19.814 15061 15061 E AndroidRuntime:        at android.app.ActivityThread.main(ActivityThread.java:8919)
07-08 15:32:19.814 15061 15061 E AndroidRuntime:        at java.lang.reflect.Method.invoke(Native Method)
07-08 15:32:19.814 15061 15061 E AndroidRuntime:        at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:578)
07-08 15:32:19.814 15061 15061 E AndroidRuntime:        at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:1103)
...
07-08 15:32:19.818 15061 15061 I MonoDroid: Android.Runtime.JavaProxyThrowable: Exception_WasThrown, Android.Runtime.JavaProxyThrowable
07-08 15:32:19.818 15061 15061 I MonoDroid: 
07-08 15:32:19.818 15061 15061 I MonoDroid:   --- End of managed Android.Runtime.JavaProxyThrowable stack trace ---
07-08 15:32:19.818 15061 15061 I MonoDroid: android.runtime.JavaProxyThrowable: [System.Net.WebSockets.WebSocketException]: net_webstatus_ConnectFailure
07-08 15:32:19.818 15061 15061 I MonoDroid:     at System.Net.WebSockets.WebSocketHandle+<ConnectAsync>d__22.MoveNext(Unknown Source:0)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw(Unknown Source:0)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Unknown Source:0)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Unknown Source:0)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at System.Net.WebSockets.ClientWebSocket+<ConnectAsyncCore>d__21.MoveNext(Unknown Source:0)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw(Unknown Source:0)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Unknown Source:0)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Unknown Source:0)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at WebsocketLLVMIssue.WebSocketClient+<ConnectAsync>d__4.MoveNext(Unknown Source:0)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw(Unknown Source:0)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Unknown Source:0)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Unknown Source:0)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at WebsocketLLVMIssue.MainActivity+<ConnectAsync>d__5.MoveNext(Unknown Source:0)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw(Unknown Source:0)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Unknown Source:0)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Unknown Source:0)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at WebsocketLLVMIssue.MainActivity+<<OnCreate>b__4_0>d.MoveNext(Unknown Source:0)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw(Unknown Source:0)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at System.Threading.Tasks.Task+<>c.<ThrowAsync>b__128_0(Unknown Source:0)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at Android.App.SyncContext+<>c__DisplayClass2_0.<Post>b__0(Unknown Source:0)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at Java.Lang.Thread+RunnableImplementor.Run(Unknown Source:0)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at Java.Lang.IRunnableInvoker.n_Run(Unknown Source:0)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at Android.Runtime.JNINativeWrapper.Wrap_JniMarshal_PP_V(Unknown Source:0)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at mono.java.lang.RunnableImplementor.n_run(Native Method)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at mono.java.lang.RunnableImplementor.run(RunnableImplementor.java:31)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at android.os.Handler.handleCallback(Handler.java:958)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at android.os.Handler.dispatchMessage(Handler.java:99)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at android.os.Looper.loopOnce(Looper.java:230)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at android.os.Looper.loop(Looper.java:319)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at android.app.ActivityThread.main(ActivityThread.java:8919)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at java.lang.reflect.Method.invoke(Native Method)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:578)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:1103)
07-08 15:32:19.818 15061 15061 I MonoDroid: 
07-08 15:32:19.818 15061 15061 I MonoDroid:   --- End of managed Android.Runtime.JavaProxyThrowable stack trace ---
07-08 15:32:19.818 15061 15061 I MonoDroid: android.runtime.JavaProxyThrowable: [System.Net.WebSockets.WebSocketException]: net_webstatus_ConnectFailure
07-08 15:32:19.818 15061 15061 I MonoDroid:     at System.Net.WebSockets.WebSocketHandle+<ConnectAsync>d__22.MoveNext(Unknown Source:0)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw(Unknown Source:0)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Unknown Source:0)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Unknown Source:0)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at System.Net.WebSockets.ClientWebSocket+<ConnectAsyncCore>d__21.MoveNext(Unknown Source:0)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw(Unknown Source:0)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Unknown Source:0)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Unknown Source:0)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at WebsocketLLVMIssue.WebSocketClient+<ConnectAsync>d__4.MoveNext(Unknown Source:0)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw(Unknown Source:0)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Unknown Source:0)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Unknown Source:0)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at WebsocketLLVMIssue.MainActivity+<ConnectAsync>d__5.MoveNext(Unknown Source:0)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw(Unknown Source:0)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Unknown Source:0)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Unknown Source:0)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at WebsocketLLVMIssue.MainActivity+<<OnCreate>b__4_0>d.MoveNext(Unknown Source:0)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw(Unknown Source:0)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at System.Threading.Tasks.Task+<>c.<ThrowAsync>b__128_0(Unknown Source:0)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at Android.App.SyncContext+<>c__DisplayClass2_0.<Post>b__0(Unknown Source:0)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at Java.Lang.Thread+RunnableImplementor.Run(Unknown Source:0)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at Java.Lang.IRunnableInvoker.n_Run(Unknown Source:0)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at Android.Runtime.JNINativeWrapper.Wrap_JniMarshal_PP_V(Unknown Source:0)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at mono.java.lang.RunnableImplementor.n_run(Native Method)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at mono.java.lang.RunnableImplementor.run(RunnableImplementor.java:31)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at android.os.Handler.handleCallback(Handler.java:958)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at android.os.Handler.dispatchMessage(Handler.java:99)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at android.os.Looper.loopOnce(Looper.java:230)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at android.os.Looper.loop(Looper.java:319)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at android.app.ActivityThread.main(ActivityThread.java:8919)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at java.lang.reflect.Method.invoke(Native Method)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:578)
07-08 15:32:19.818 15061 15061 I MonoDroid:     at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:1103)
07-08 15:32:19.818 15061 15061 I MonoDroid: 

It is also related to networking, and the weird thing is that AOT+LLVM somehow makes the connection fail.

@vitek-karas
Copy link
Member

Thanks a lot @grendello - learning :-)

@simonrozsival
Copy link
Member

simonrozsival commented Jul 12, 2024

I can reproduce the issue locally, this is the exception stack trace I'm getting:

System.Net.WebSockets.WebSocketException (0x80004005): net_webstatus_ConnectFailure
 ---> System.Net.Http.HttpRequestException: Arg_InvalidOperationException (echo.websocket.org:443)
 ---> System.InvalidOperationException: Arg_InvalidOperationException
   at System.Threading.Tasks.Sources.ManualResetValueTaskSourceCore`1[[System.Boolean, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].GetStatus(Int16 )
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.GetStatus(Int16 )
   at System.Net.Sockets.Socket.ConnectAsync(EndPoint , CancellationToken )
   at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String , Int32 , HttpRequestMessage , Boolean , CancellationToken )
   Exception_EndOfInnerExceptionStack
   at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String , Int32 , HttpRequestMessage , Boolean , CancellationToken )
   at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage , Boolean , CancellationToken )
   at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage , Boolean , CancellationToken )
   at System.Net.Http.HttpConnectionPool.AddHttp11ConnectionAsync(QueueItem )
   at System.Threading.Tasks.TaskCompletionSourceWithCancellation`1.<WaitWithCancellationAsync>d__1[[System.Net.Http.HttpConnection, System.Net.Http, Version=8.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a]].MoveNext()
   at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage , Boolean , Boolean , CancellationToken )
   at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage , Boolean , CancellationToken )
   at System.Net.WebSockets.WebSocketHandle.ConnectAsync(Uri , HttpMessageInvoker , CancellationToken , ClientWebSocketOptions )
   at System.Net.WebSockets.WebSocketHandle.ConnectAsync(Uri , HttpMessageInvoker , CancellationToken , ClientWebSocketOptions )
   at System.Net.WebSockets.ClientWebSocket.ConnectAsyncCore(Uri , HttpMessageInvoker , CancellationToken )
   at WebsocketLLVMIssue.WebSocketClient.ConnectAsync(String uri)
   at WebsocketLLVMIssue.MainActivity.ConnectAsync()

I get the same failure if I keep the same LLVM configuration, but I simply to connect a Socket:

try
{
    var socket = new Socket(SocketType.Stream, ProtocolType.Tcp) { NoDelay = true };
    await socket.ConnectAsync(IPEndPoint.Parse("1.1.1.1:443"), CancellationToken.None);
}
catch (Exception ex)
{
    Console.WriteLine(ex);
}

I get the following exception:

System.InvalidOperationException: Arg_InvalidOperationException
  at System.Threading.Tasks.Sources.ManualResetValueTaskSourceCore`1[[System.Boolean, System.Private.CoreLib, Version=9.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].GetStatus(Int16 )
  at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.GetStatus(Int16 )
  at System.Net.Sockets.Socket.ConnectAsync(EndPoint , CancellationToken )
  at TestSocketsHttpHandlerLlvm.MainActivity.<>c.<<OnCreate>b__1_0>d.MoveNext()

This means that using SocketsHttpHandler with LLVM won't work.

@wfurt do you have an idea what could be going on?

@wfurt
Copy link
Member

wfurt commented Jul 12, 2024

no, I don't know why AOT would make any difference. (assuming same code runs OK on Android without AOT) This seems to come from the ManualResetValueTaskSourceCore implementation.

        private void ValidateToken(short token)
        {
            if (token != _version)
            {
                ThrowHelper.ThrowInvalidOperationException();
            }
        }

perhaps you can look at it in debugger and check the token version(s). cc @stephentoub for any more recommendations.

@vitek-karas vitek-karas added this to the 9.0.0 milestone Jul 24, 2024
@jeffschwMSFT jeffschwMSFT removed the untriaged New issue has not been triaged by the area owner label Jul 24, 2024
@matouskozak
Copy link
Member

Investigation update

The failures does reproduce in dotnet new android and dotnet new maui (net9.0-android) apps with code from #104599 (comment) with AOT-llvm:

  <EnableLLVM>True</EnableLLVM>
  <AndroidEnableProfiledAot>False</AndroidEnableProfiledAot>
  <RunAOTCompilation>True</RunAOTCompilation>

However, I was not able to reproduce the same issue on iOS (which uses AOT-llvm as well) or with a Mono sample app on arm64 desktop in AOT-llvm configuration. What's more, the Android sample app in dotnet/runtime also doesn't reproduce the issue (setting Makefilevariables USE_LLVM=true and AOT=true).

I'm thinking that there might be a conflict between dotnet/runtime and dotnet/android LLVMs which is causing some unexpected behavior. @jonathanpeppers do you know who would be the best person to discuss this issue with?

@jonathanpeppers
Copy link
Member

@grendello or I might be able to help, depending what the problem is. Do you know if there is some AOT compiler flag/option we are missing that iOS has?

Offhand, -p:EnableLLVM=true will require the Android NDK, while other builds can succeed without the NDK used at all.

@grendello
Copy link
Contributor

One thing that might be wrong is the newer LLVM linkers tend to rewrite code at link time and that breaks Mono's AOT since it expects to find the exact same code at run time as it generated at compile time. To prevent lld from doing this, we pass the --no-relax flag to it. Perhaps this flag is somehow missing when LLVM AOT is enabled? Even if it's there, it might be a good idea to compare disassembled source of the involved AOT shared libraries around the WebSocket code, when built with and without LLVM?

@matouskozak
Copy link
Member

matouskozak commented Aug 20, 2024

One thing that might be wrong is the newer LLVM linkers tend to rewrite code at link time and that breaks Mono's AOT since it expects to find the exact same code at run time as it generated at compile time. To prevent lld from doing this, we pass the --no-relax flag to it. Perhaps this flag is somehow missing when LLVM AOT is enabled? Even if it's there, it might be a good idea to compare disassembled source of the involved AOT shared libraries around the WebSocket code, when built with and without LLVM?

I checked the binlogs and the --no-relax flag seems to be applied: [System.Net.Sockets.dll] Exec (with response file contents expanded) in .../WebSocketLLVMExceptionMAUI: MONO_PATH=.../WebSocketLLVMExceptionMAUI/obj/Release/net9.0-android/android-x86/linked: MONO_ENV_OPTIONS= /usr/local/share/dotnet/packs/Microsoft.NETCore.App.Runtime.AOT.osx-arm64.Cross.android-x86/9.0.0-preview.4.24266.19/Sdk/../tools/mono-aot-cross --debug --llvm "--aot=asmwriter,temp-path=obj/Release/net9.0-android/android-x86/aot/x86/System.Net.Sockets,nodebug,llvm-path=/usr/local/share/dotnet/packs/Microsoft.NETCore.App.Runtime.AOT.osx-arm64.Cross.android-x86/9.0.0-preview.4.24266.19/Sdk/../tools,mtriple=i686-linux-android,tool-prefix=/usr/local/share/dotnet/packs/Microsoft.Android.Sdk.Darwin/34.99.0-preview.4.272/tools/Darwin/binutils/bin/i686-linux-android-,outfile=obj/Release/net9.0-android/android-x86/aot/System.Net.Sockets.dll.so,llvm-outfile=obj/Release/net9.0-android/android-x86/aot/System.Net.Sockets.dll-llvm.o,temp-path=obj/Release/net9.0-android/android-x86/System.Net.Sockets.dll.tmp,ld-name=ld,ld-flags=\"/usr/local/share/dotnet/packs/Microsoft.Android.Sdk.Darwin/34.99.0-preview.4.272/tools/libstubs/android-x86/libc.so\";\"/usr/local/share/dotnet/packs/Microsoft.Android.Sdk.Darwin/34.99.0-preview.4.272/tools/libstubs/android-x86/libm.so\" --no-relax -s" "obj/Release/net9.0-android/android-x86/linked/System.Net.Sockets.dll" (here is the full binlog).

@grendello or I might be able to help, depending what the problem is. Do you know if there is some AOT compiler flag/option we are missing that iOS has?

Offhand, -p:EnableLLVM=true will require the Android NDK, while other builds can succeed without the NDK used at all.

What I've noticed that is that on iOS, we use by default MONO_AOT_MODE_FULL, whereas on Android we set only MONO_AOT_MODE_NORMAL (allows JIT). Not sure if that could play a role as well.

I'll try to get the disassembled source of System.Net.Sockets dll and I'll report the findings here.

@BrzVlad any thoughts on how to investigate this?

@DigitApps2
Copy link

We have the same issue using latest Azure.Storage.Blobs nuget for the azure file upload functions

@vitek-karas vitek-karas modified the milestones: 9.0.0, 10.0.0 Sep 5, 2024
@alexyakunin
Copy link

Experiencing the same issue - we noticed it a while ago (maybe 6mo or so), didn't have a chance to report.

The exception we get is identical to this one: #104599 (comment)

@alexyakunin
Copy link

Just captured our own exception:
image

@alexyakunin
Copy link

Just checked the sample from .zip on .NET 9 Preview 7: it works.

That was the only modification (+ install whatever is required):

    <TargetFramework>net9.0-android</TargetFramework>

@kcrg
Copy link
Author

kcrg commented Sep 11, 2024

@alexyakunin By changing only target framework to .NET 9, you disable the below PropertyGroup with Condition="'$(Configuration)|$(TargetFramework)|$(Platform)'=='Release|net8.0-android|AnyCPU'"

If u want to test if it works on .NET 9, update Condition in this PropertyGroup to .NET 9.

Here's how csproj looks like on .NET 9:

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <TargetFramework>net9.0-android</TargetFramework>
    <SupportedOSPlatformVersion>21</SupportedOSPlatformVersion>
    <OutputType>Exe</OutputType>
    <Nullable>enable</Nullable>
    <ImplicitUsings>enable</ImplicitUsings>
    <ApplicationId>com.companyname.WebsocketLLVMIssue</ApplicationId>
    <ApplicationVersion>1</ApplicationVersion>
    <ApplicationDisplayVersion>1.0</ApplicationDisplayVersion>
  </PropertyGroup>

  <PropertyGroup Condition="'$(Configuration)|$(TargetFramework)|$(Platform)'=='Release|net9.0-android|AnyCPU'">
    <EnableLLVM>True</EnableLLVM>
    <AndroidEnableProfiledAot>False</AndroidEnableProfiledAot>
    <RunAOTCompilation>True</RunAOTCompilation>
  </PropertyGroup>
</Project>

The issue still persists in .NET 9 rc1.

@alexyakunin
Copy link

alexyakunin commented Sep 13, 2024

@kcrg Oh, you're right - I forgot to change the framework identifier in Condition part.

Which means we also really need this fix.

@guygodin
Copy link

guygodin commented Sep 14, 2024

Also experiencing this issue when attempting to bring a Xamarin.Android app to .NET 8.0/9.0 while using regular TCP sockets. Same exception as described here: #104599 (comment)

@alexyakunin
Copy link

alexyakunin commented Oct 2, 2024

Hi guys, are there any updates on that? In particular, will the issue with WebSockets and sockets be addressed @ .NET 9 release?

@matouskozak
Copy link
Member

Hi guys, are there any updates on that? In particular, will the issue with WebSockets and sockets be addressed @ .NET 9 release?

The issue was moved to .NET 10 so the fix likely won't be part of the .NET 9 release but will be backported to .NET 9 servicing when ready.

We don't have any update yet as we are prioritizing .NET 9 issues before the release. In the meantime, could you try setting disabling LLVM to work around this issue (<EnableLLVM>False</EnableLLVM>)?

@alexyakunin
Copy link

In the meantime, could you try setting disabling LLVM to work around this issue (<EnableLLVM>False</EnableLLVM>)?

It works in this case - both in our app and in the demo app above.

@matouskozak
Copy link
Member

In the meantime, could you try setting disabling LLVM to work around this issue (<EnableLLVM>False</EnableLLVM>)?

It works in this case - both in our app and in the demo app above.

Thank you for the update. Could you please share the reasons behind enabling LLVM on Android? I don't think it is enabled by default.

@kcrg
Copy link
Author

kcrg commented Oct 12, 2024

Thank you for the update. Could you please share the reasons behind enabling LLVM on Android? I don't think it is enabled by default.

https://learn.microsoft.com/en-us/dotnet/maui/whats-new/dotnet-9?view=net-maui-8.0#llvm-marshalled-methods

Low-level Virtual Machine (LLVM) marshalled methods are now enabled by default in .NET for Android 9 in non-Blazor apps.

@jonathanpeppers
Copy link
Member

@kcrg $(AndroidEnableMarshalMethods) is not the same feature as $(EnableLLVM). I'll look into updating this doc, it also seems like the MSBuild properties they mention here are not spelled correctly?!?

$(EnableLLVM) is a Mono AOT compiler feature that has been around since Xamarin, but it has some issues that prevent us from making it the default.

@kcrg
Copy link
Author

kcrg commented Oct 13, 2024

@jonathanpeppers so what is the difference between <AndroidEnableLLVM>True</AndroidEnableLLVM> and <EnableLLVM>True</EnableLLVM>?

This is confusing to me.

Also

... by default in .NET for Android 9 in non-Blazor apps.

First thought was that it applies to Android 9, second thought was that I'm reading the .NET for Android release notes, so it applies to .NET 9 itself. I know there was a name change Xamarin.Android -> .NET for Android, I think a better option is to put the version number right after ".NET" -> .NET 9 for Android

@jonathanpeppers
Copy link
Member

@jonathanpeppers so what is the difference between <AndroidEnableLLVM>True</AndroidEnableLLVM> and <EnableLLVM>True</EnableLLVM>?

$(AndroidEnableLLVM) looks to be an AI hallucination, it does not exist:

@alexyakunin
Copy link

alexyakunin commented Oct 18, 2024

Okey, a quick update: you can workaround this issue by adding this piece to .csproj:

  <!--
  LLVM fails in runtime if it processes System.Net.Sockets.dll.
  See https://github.com/dotnet/runtime/issues/104599
  -->
  <Target Name="_FixAndroidAotInputs" DependsOnTargets="_AndroidAotInputs" BeforeTargets="_AndroidAotCompilation">
    <ItemGroup Condition="$(EnableLLVM)">
      <_AndroidAotInputs Remove="$(IntermediateLinkDir)**\System.Net.Sockets.dll" />
    </ItemGroup>
  </Target>

More sophisticated fix idea is to generate AOT build of System.Net.Sockets.dll w/o LLVM. I don't have time to work on it now, + I suspect it won't change much in terms of perf. So if someone else comes up w/ the modifications needed, please share.

CC @kcrg

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests