[FR] support TSan

[vlovich]

Description

Source:

#include <future>

int main() {
  while (true) {
    auto f = std::async([] { return 123; });
    if (!f.valid()) {
      return 1;
    }
    if (f.get() != 123) {
      return 1;
    }
    if (f.valid()) {
      return 1;
    }
  }
}

Build it with TSAN on r19c:

toolchains/llvm/prebuilt/windows-x86_64/bin/aarch64-linux-android24-clang++ -g -Og ../MyTest.cpp -fsanitize=thread toolchains/llvm/prebuilt/windows-x86_64/lib64/clang/8.0.2/lib/linux/libclang_rt.tsan-aarch64-android.a
adb push a.out
adb shell "TSAN_OPTIONS=no_huge_pages_for_shadow=0 ./a.out"

The following crash is observed:

FATAL: ThreadSanitizer CHECK failed: /usr/local/google/buildbot/src/android/llvm-toolchain/toolchain/compiler-rt/lib/tsan/rtl/tsan_rtl_thread.cc:304 "((tid)) < ((kMaxTid))" (0xffffffffffffffff, 0x1fc0)

Running under gdb the following symbolicated backtrace is observed:

#0  0x0000007fb7e5bed8 in tgkill () from target:/system/lib64/libc.so
#1  0x0000007fb7e58c84 in pthread_kill () from target:/system/lib64/libc.so
#2  0x00000055555dd7b8 in __interceptor_pthread_kill ()
    at /usr/local/google/buildbot/src/android/llvm-toolchain/toolchain/compiler-rt/lib/tsan/rtl/tsan_interceptors.cc:2027
#3  0x0000007fb7e06ac4 in raise () from target:/system/lib64/libc.so
#4  0x00000055555dd378 in __interceptor_raise ()
    at /usr/local/google/buildbot/src/android/llvm-toolchain/toolchain/compiler-rt/lib/tsan/rtl/tsan_interceptors.cc:1999
#5  0x0000007fb7dfedd0 in abort () from target:/system/lib64/libc.so
#6  0x00000055555dbdb8 in __interceptor_abort ()
    at /usr/local/google/buildbot/src/android/llvm-toolchain/toolchain/compiler-rt/lib/tsan/rtl/tsan_interceptors.cc:1736
#7  0x00000055556385c4 in __sanitizer::Abort() ()
    at /usr/local/google/buildbot/src/android/llvm-toolchain/toolchain/compiler-rt/lib/sanitizer_common/sanitizer_posix_libcdep.cc:153
#8  0x0000005555636820 in __sanitizer::Die() ()
    at /usr/local/google/buildbot/src/android/llvm-toolchain/toolchain/compiler-rt/lib/sanitizer_common/sanitizer_termination.cc:59
#9  0x0000005555620190 in TsanCheckFailed () at /usr/local/google/buildbot/src/android/llvm-toolchain/toolchain/compiler-rt/lib/tsan/rtl/tsan_rtl_report.cc:49#10 0x000000555563689c in CheckFailed ()
    at /usr/local/google/buildbot/src/android/llvm-toolchain/toolchain/compiler-rt/lib/sanitizer_common/sanitizer_termination.cc:79
#11 0x0000005555623e50 in __tsan::ThreadJoin(__tsan::ThreadState*, unsigned long, int) ()
    at /usr/local/google/buildbot/src/android/llvm-toolchain/toolchain/compiler-rt/lib/tsan/rtl/tsan_rtl_thread.cc:304
#12 0x00000055555d6b18 in __interceptor_pthread_join ()
    at /usr/local/google/buildbot/src/android/llvm-toolchain/toolchain/compiler-rt/lib/tsan/rtl/tsan_interceptors.cc:1027
#13 0x00000055555d6d1c in __interceptor_pthread_detach ()
    at /usr/local/google/buildbot/src/android/llvm-toolchain/toolchain/compiler-rt/lib/tsan/rtl/tsan_interceptors.cc:1037
#14 0x00000055556953f4 in std::__ndk1::__libcpp_thread_detach (__t=0x7ffffff548)
    at /Volumes/Android/buildbot/src/android/ndk-release-r18/external/libcxx/include/__threading_support:348
#15 std::__ndk1::thread::detach (this=0x7ffffff548) at /Volumes/Android/buildbot/src/android/ndk-release-r18/external/libcxx/src/thread.cpp:68
#16 0x00000055555768ec in std::__ndk1::__make_async_assoc_state<int, std::__ndk1::__async_func<AsyncTest_GetCanOnlyBeCalledOnce_Test::TestBody()::$_0> >(std::__ndk1::__async_func<AsyncTest_GetCanOnlyBeCalledOnce_Test::TestBody()::$_0>&&) (__f=<optimized out>)
    at third-party/toolchains/android-ndk/r18b/sources/cxx-stl/llvm-libc++/include/future:2297
#17 0x0000005555576810 in std::__ndk1::async<AsyncTest_GetCanOnlyBeCalledOnce_Test::TestBody()::$_0>(std::__ndk1::launch, AsyncTest_GetCanOnlyBeCalledOnce_Test::TestBody()::$_0&&) (__policy=std::__ndk1::launch::any, __f=<optimized out>)
---Type <return> to continue, or q <return> to quit---
    at third-party/toolchains/android-ndk/r18b/sources/cxx-stl/llvm-libc++/include/future:2346
#18 0x0000005555576510 in std::__ndk1::async<AsyncTest_GetCanOnlyBeCalledOnce_Test::TestBody()::$_0>(AsyncTest_GetCanOnlyBeCalledOnce_Test::TestBody()::$_0&&) (
    __f=<optimized out>) at third-party/toolchains/android-ndk/r18b/sources/cxx-stl/llvm-libc++/include/future:2364
#19 AsyncTest_GetCanOnlyBeCalledOnce_Test::TestBody (this=<optimized out>) at MyTest.cpp:6
#20 0x00000055555ce334 in testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void> (object=<optimized out>, method=<optimized out>,
    location=<optimized out>) at googletest/googletest\src/gtest.cc:2447
#21 0x00000055555c0b68 in testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void> (object=0x7fb5000b90,
    method=&virtual testing::Test::TestBody(), location=0x55556c0b66 "the test body") at googletest/googletest\src/gtest.cc:2483
#22 0x00000055555a7328 in testing::Test::Run (this=0x7fb5000b90) at googletest/googletest\src/gtest.cc:2522
#23 0x00000055555a7dac in testing::TestInfo::Run (this=0x7fb4003930) at googletest/googletest\src/gtest.cc:2703
#24 0x00000055555a84b4 in testing::TestCase::Run (this=0x7fb4003840) at googletest/googletest\src/gtest.cc:2825
#25 0x00000055555b1a28 in testing::internal::UnitTestImpl::RunAllTests (this=0x7fb5296e00) at googletest/googletest\src/gtest.cc:5214
#26 0x00000055555cf600 in testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool> (object=<optimized out>,
    method=<optimized out>, location=<optimized out>) at googletest/googletest\src/gtest.cc:2447
#27 0x00000055555c2f34 in testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool> (object=0x7fb5296e00,
    method=(bool (testing::internal::UnitTestImpl::*)(testing::internal::UnitTestImpl * const)) 0x55555b1738 <testing::internal::UnitTestImpl::RunAllTests()>,
    location=0x55556c1362 "auxiliary test code (environments or event listeners)") at googletest/googletest\src/gtest.cc:2483
#28 0x00000055555b16ec in testing::UnitTest::Run (this=0x555573c240 <testing::UnitTest::GetInstance()::instance>)
    at googletest/googletest\src/gtest.cc:4822
#29 0x000000555557a52c in RUN_ALL_TESTS () at googletest/googletest/include\gtest/gtest.h:2369
#30 0x000000555557a484 in main (argc=1, argv=0x7ffffff990) at gmock_with_gflags\main.cc:61

NOTE: This reproduces on r18b too but that has a race condition in std::future so for the purposes of the experiment you backport the header fix for the race condition (https://reviews.llvm.org/D51170) since the test doesn't use std::future<void>.

On r20b the program crashes with a segfaul:

toolchains/llvm/prebuilt/windows-x86_64/bin/aarch64-linux-android24-clang++ ../MyTest.cpp -fsanitize=thread toolchains/llvm/prebuilt/windows-x86_64/lib64/clang/8.0.7/lib/linux/libclang_rt.tsan-aarch64-android.a -static-libstdc++ -g -Og

Backtrace:

#0  0x0000005555569358 in ScopedInterceptor () at /usr/local/google/buildbot/src/android/llvm-toolchain/toolchain/compiler-rt/lib/tsan/rtl/tsan_interceptors.cc:247
#1  0x0000005555581bd8 in __interceptor_pthread_mutex_lock ()
    at /usr/local/google/buildbot/src/android/llvm-toolchain/toolchain/compiler-rt/lib/tsan/../sanitizer_common/sanitizer_common_interceptors.inc:4088
#2  0x0000007fb7e38b34 in __cxa_atexit () from target:/system/lib64/libc.so
#3  0x0000005555591ce4 in InitializeInterceptors () at /usr/local/google/buildbot/src/android/llvm-toolchain/toolchain/compiler-rt/lib/tsan/rtl/tsan_interceptors.cc:2748
#4  0x00000055555a9090 in Initialize () at /usr/local/google/buildbot/src/android/llvm-toolchain/toolchain/compiler-rt/lib/tsan/rtl/tsan_rtl.cc:376
#5  0x000000555556934c in ScopedInterceptor () at /usr/local/google/buildbot/src/android/llvm-toolchain/toolchain/compiler-rt/lib/tsan/rtl/tsan_interceptors.cc:246
#6  0x0000005555573ef4 in __interceptor_strcmp ()
    at /usr/local/google/buildbot/src/android/llvm-toolchain/toolchain/compiler-rt/lib/tsan/../sanitizer_common/sanitizer_common_interceptors.inc:424
#7  0x0000007fb7dce6e0 in __libc_init_vdso(libc_globals*, KernelArgumentBlock&) () from target:/system/lib64/libc.so
#8  0x0000007fb7dd2264 in __libc_init_globals(KernelArgumentBlock&) () from target:/system/lib64/libc.so
#9  0x0000007fb7dcc3b4 in __libc_preinit() () from target:/system/lib64/libc.so
#10 0x0000007fb7f2c458 in __dl__ZN6soinfo10call_arrayEPKcPPFvvEmb () from target:/system/bin/linker64
#11 0x0000007fb7f2a300 in __dl__ZN6soinfo17call_constructorsEv () from target:/system/bin/linker64
#12 0x0000007fb7f2a300 in __dl__ZN6soinfo17call_constructorsEv () from target:/system/bin/linker64
#13 0x0000007fb7f300e0 in __dl__ZL29__linker_init_post_relocationR19KernelArgumentBlocky () from target:/system/bin/linker64
#14 0x0000007fb7f2f094 in __dl___linker_init () from target:/system/bin/linker64
#15 0x0000007fb7f260e8 in __dl__start () from target:/system/bin/linker64

Can't test on r21 due to linker bug #1170.

Environment Details

Not all of these will be relevant to every bug, but please provide as much
information as you can.

• NDK Version:
  r18b
  Pkg.Desc = Android NDK
  Pkg.Revision = 18.1.5063045

r19c
Pkg.Desc = Android NDK
Pkg.Revision = 19.2.5345600

r20b
Pkg.Desc = Android NDK
Pkg.Revision = 20.1.5948944

• Build system: Standalone toolchain
• Host OS: Windows & Mac
• ABI:
• NDK API level: N/A
• Device API level: N/A

[vlovich]

Reported upstream as well: google/sanitizers#1183

[DanAlbert]

TSan isn't supported on Android AFAIK. Leaving this open bug converting to a FR.

[bjacob]

please, please prioritize this. It's hard to understand threaded code without TSan. Code out there is becoming more threaded, in more bug-prone ways (C++11 weakly ordered atomics...). We need TSan!

[DanAlbert]

It sounds like we may not have anyone free to look at this for a while, but I'm going to triage it to r23 for now so we keep thinking about it.

[DanAlbert]

Duplicate of #1041

Sorry for not noticing that sooner :)