[sanitizers] Make it possible to XFAIL on the effective target, not just the default.

Summary:
The triple is not the right thing to XFAIL on since LIT only sees the default
triple and not the effective triple chosen by any -target option in the RUN
directives. This discrepancy is shown in the table below:

  Default Triple   | Options                           | XFAIL  | LIT's expected result | Desired expectation
  =================+===================================+========+=======================+====================
  mips-linux-gnu   | -target mips-linux-gnu            |        | Pass                  | Pass  
  mips-linux-gnu   | -target mips64-linux-gnu -mabi=64 |        | Pass                  | Pass  
  mips-linux-gnu   | -target mips-linux-gnu            | mips   | Fail                  | Fail  
  mips-linux-gnu   | -target mips64-linux-gnu -mabi=64 | mips   | Fail                  | Fail/Pass* (debatable**)
  mips-linux-gnu   | -target mips-linux-gnu            | mips-  | Fail                  | Fail  
  mips-linux-gnu   | -target mips64-linux-gnu -mabi=64 | mips-  | Fail                  | Pass* 
  mips-linux-gnu   | -target mips-linux-gnu            | mips64 | Pass                  | Pass  
  mips-linux-gnu   | -target mips64-linux-gnu -mabi=64 | mips64 | Pass                  | Fail* 
  mips64-linux-gnu | -target mips-linux-gnu            |        | Pass                  | Pass  
  mips64-linux-gnu | -target mips64-linux-gnu -mabi=64 |        | Pass                  | Pass  
  mips64-linux-gnu | -target mips-linux-gnu            | mips   | Fail                  | Fail* 
  mips64-linux-gnu | -target mips64-linux-gnu -mabi=64 | mips   | Fail                  | Fail/Pass (debatable**)
  mips64-linux-gnu | -target mips-linux-gnu            | mips-  | Pass                  | Fail* 
  mips64-linux-gnu | -target mips64-linux-gnu -mabi=64 | mips-  | Pass                  | Pass  
  mips64-linux-gnu | -target mips-linux-gnu            | mips64 | Fail                  | Pass* 
  mips64-linux-gnu | -target mips64-linux-gnu -mabi=64 | mips64 | Fail                  | Fail  
  x64_64-linux-gnu | -target i386-linux-gnu            |        | Pass                  | Pass
  x64_64-linux-gnu | -target x86_64-linux-gnu          |        | Pass                  | Pass
  x64_64-linux-gnu | -target i386-linux-gnu            | i386   | Pass                  | Fail*
  x64_64-linux-gnu | -target x86_64-linux-gnu          | i386   | Pass                  | Pass
  x64_64-linux-gnu | -target i386-linux-gnu            | x86_64 | Fail                  | Pass
  x64_64-linux-gnu | -target x86_64-linux-gnu          | x86_64 | Fail                  | Fail*
  * These all differ from LIT's current behaviour.
  ** People's expectations vary depending on whether they know that LIT does a
   substring match on the default triple or think it's an exact match on an
   architecture.

This patch adds "target-is-${target_arch}" to the available features list and
updates the mips XFAIL's to use them. XFAIL'ing on these features will
correctly account for the target being tested. Making the table:

  Options                           | XFAIL            | LIT's expected result
  ==================================+==================+======================
  -target mips-linux-gnu            |                  | Pass
  -target mips64-linux-gnu -mabi=64 |                  | Pass
  -target mips-linux-gnu            | target-is-mips   | Fail
  -target mips64-linux-gnu -mabi=64 | target-is-mips   | Pass
  -target mips-linux-gnu            | target-is-mips64 | Pass
  -target mips64-linux-gnu -mabi=64 | target-is-mips64 | Fail
  -target i386-linux-gnu            |                  | Pass
  -target x86_64-linux-gnu          |                  | Pass
  -target i386-linux-gnu            | target-is-i386   | Fail
  -target x86_64-linux-gnu          | target-is-i386   | Pass
  -target i386-linux-gnu            | target-is-x86_64 | Pass
  -target x86_64-linux-gnu          | target-is-x86_64 | Fail

Reviewers: probinson

Subscribers: probinson, kubabrecka, llvm-commits, samsonov

Differential Revision: https://reviews.llvm.org/D22802

git-svn-id: https://llvm.org/svn/llvm-project/compiler-rt/trunk@278116 91177308-0d34-0410-b5e6-96231b3b80d8

11 files changed, 76 insertions, 9 deletions

diff --git a/docs/TestingGuide.rst b/docs/TestingGuide.rst
new file mode 100644
index 000000000..4edda6738
--- /dev/null
+++ b/docs/TestingGuide.rst
@@ -0,0 +1,66 @@
+========================================
+Compiler-rt Testing Infrastructure Guide
+========================================
+
+.. contents::
+   :local:
+
+Overview
+========
+
+This document is the reference manual for the compiler-rt modifications to the
+testing infrastructure. Documentation for the infrastructure itself can be found at
+:ref:`llvm_testing_guide`.
+
+LLVM testing infrastructure organization
+========================================
+
+The compiler-rt testing infrastructure contains regression tests which are run
+as part of the usual ``make check-all`` and are expected to always pass -- they
+should be run before every commit.
+
+Quick start
+===========
+
+The regressions tests are in the "compiler-rt" module and are normally checked
+out in the directory ``llvm/projects/compiler-rt/test``. Use ``make check-all``
+to run the regression tests after building compiler-rt.
+
+REQUIRES, XFAIL, etc.
+---------------------
+
+Sometimes it is necessary to restrict a test to a specific target or mark it as
+an "expected fail" or XFAIL. This is normally achieved using ``REQUIRES:`` or
+``XFAIL:`` with a substring of LLVM's default target triple. Unfortunately, the
+behaviour of this is somewhat quirky in compiler-rt. There are two main
+pitfalls to avoid.
+
+The first pitfall is that these directives perform a substring match on the
+triple and as such ``XFAIL: mips`` affects more triples than expected. For
+example, ``mips-linux-gnu``, ``mipsel-linux-gnu``, ``mips64-linux-gnu``, and
+``mips64el-linux-gnu`` will all match a ``XFAIL: mips`` directive. Including a
+trailing ``-`` such as in ``XFAIL: mips-`` can help to mitigate this quirk but
+even that has issues as described below.
+
+The second pitfall is that the default target triple is often inappropriate for
+compiler-rt tests since compiler-rt tests may be compiled for multiple targets.
+For example, a typical build on an ``x86_64-linux-gnu`` host will often run the
+tests for both x86_64 and i386. In this situation ``XFAIL: x86_64`` will mark
+both the x86_64 and i386 tests as an expected failure while ``XFAIL: i386``
+will have no effect at all.
+
+To remedy both pitfalls, compiler-rt tests provide a feature string which can
+be used to specify a single target. This string is of the form
+``target-is-${arch}`` where ``${arch}}`` is one of the values from the
+following lines of the CMake output::
+
+  -- Compiler-RT supported architectures: x86_64;i386
+  -- Builtin supported architectures: i386;x86_64
+
+So for example ``XFAIL: target-is-x86_64`` will mark a test as expected to fail
+on x86_64 without also affecting the i386 test and ``XFAIL: target-is-i386``
+will mark a test as expected to fail on i386 even if the default target triple
+is ``x86_64-linux-gnu``. Directives that use these ``target-is-${arch}`` string
+require exact matches so ``XFAIL: target-is-mips``,
+``XFAIL: target-is-mipsel``, ``XFAIL: target-is-mips64``, and
+``XFAIL: target-is-mips64el`` all refer to different MIPS targets.
diff --git a/test/asan/TestCases/Linux/local_alias.cc b/test/asan/TestCases/Linux/local_alias.cc
index d941ff2f9..23512138c 100644
--- a/test/asan/TestCases/Linux/local_alias.cc
+++ b/test/asan/TestCases/Linux/local_alias.cc
@@ -6,7 +6,7 @@
 //
 // FIXME: https://github.com/google/sanitizers/issues/316
 // XFAIL: android
-// XFAIL: mips64
+// XFAIL: target-is-mips64
 //
 // RUN: %clangxx_asan -DBUILD_INSTRUMENTED_DSO=1 -fPIC -shared -mllvm -asan-use-private-alias %s -o %t-INSTRUMENTED-SO.so
 // RUN: %clangxx -DBUILD_UNINSTRUMENTED_DSO=1 -fPIC -shared %s -o %t-UNINSTRUMENTED-SO.so
diff --git a/test/asan/TestCases/Linux/odr-violation.cc b/test/asan/TestCases/Linux/odr-violation.cc
index 143fb6e14..0938ef631 100644
--- a/test/asan/TestCases/Linux/odr-violation.cc
+++ b/test/asan/TestCases/Linux/odr-violation.cc
@@ -1,6 +1,6 @@
 // FIXME: https://code.google.com/p/address-sanitizer/issues/detail?id=316
 // XFAIL: android
-// XFAIL: mips64
+// XFAIL: target-is-mips64
 //
 // We use fast_unwind_on_malloc=0 to have full unwinding even w/o frame
 // pointers. This setting is not on by default because it's too expensive.
diff --git a/test/lit.common.configured.in b/test/lit.common.configured.in
index 35aa78c98..4472f59dd 100644
--- a/test/lit.common.configured.in
+++ b/test/lit.common.configured.in
@@ -28,6 +28,7 @@ set_default("emulator", "@COMPILER_RT_EMULATOR@")
 set_default("sanitizer_can_use_cxxabi", @SANITIZER_CAN_USE_CXXABI_PYBOOL@)
 set_default("has_lld", @COMPILER_RT_HAS_LLD_SOURCES_PYBOOL@)
 set_default("can_symbolize", @CAN_SYMBOLIZE@)
+config.available_features.add('target-is-%s' % config.target_arch)
 
 # LLVM tools dir can be passed in lit parameters, so try to
 # apply substitution.
diff --git a/test/tsan/ignore_lib4.cc b/test/tsan/ignore_lib4.cc
index 193df11d2..1c94a51d9 100644
--- a/test/tsan/ignore_lib4.cc
+++ b/test/tsan/ignore_lib4.cc
@@ -4,7 +4,7 @@
 // RUN: %env_tsan_opts=suppressions='%t.supp' %run %t 2>&1 | FileCheck %s
 
 // Longjmp assembly has not been implemented for mips64 yet
-// XFAIL: mips64
+// XFAIL: target-is-mips64
 // powerpc64 big endian bots failed with "FileCheck error: '-' is empty" due
 // to a segmentation fault.
 // UNSUPPORTED: powerpc64-unknown-linux-gnu
diff --git a/test/tsan/longjmp.cc b/test/tsan/longjmp.cc
index 61d285c11..73e4e4dc2 100644
--- a/test/tsan/longjmp.cc
+++ b/test/tsan/longjmp.cc
@@ -1,7 +1,7 @@
 // RUN: %clang_tsan -O1 %s -o %t && %run %t 2>&1 | FileCheck %s
 
 // Longjmp assembly has not been implemented for mips64 yet
-// XFAIL: mips64
+// XFAIL: target-is-mips64
 
 #include <stdio.h>
 #include <stdlib.h>
diff --git a/test/tsan/longjmp2.cc b/test/tsan/longjmp2.cc
index 2b2775a8c..971f69c79 100644
--- a/test/tsan/longjmp2.cc
+++ b/test/tsan/longjmp2.cc
@@ -1,7 +1,7 @@
 // RUN: %clang_tsan -O1 %s -o %t && %run %t 2>&1 | FileCheck %s
 
 // Longjmp assembly has not been implemented for mips64 yet
-// XFAIL: mips64
+// XFAIL: target-is-mips64
 
 #include <stdio.h>
 #include <stdlib.h>
diff --git a/test/tsan/longjmp3.cc b/test/tsan/longjmp3.cc
index 197b91e1c..ac3f3ef3d 100644
--- a/test/tsan/longjmp3.cc
+++ b/test/tsan/longjmp3.cc
@@ -1,7 +1,7 @@
 // RUN: %clang_tsan -O1 %s -o %t && %deflake %run %t 2>&1 | FileCheck %s
 
 // Longjmp assembly has not been implemented for mips64 yet
-// XFAIL: mips64
+// XFAIL: target-is-mips64
 
 #include <pthread.h>
 #include <stdio.h>
diff --git a/test/tsan/longjmp4.cc b/test/tsan/longjmp4.cc
index 3785a0f07..cdf406920 100644
--- a/test/tsan/longjmp4.cc
+++ b/test/tsan/longjmp4.cc
@@ -1,7 +1,7 @@
 // RUN: %clang_tsan -O1 %s -o %t && %deflake %run %t 2>&1 | FileCheck %s
 
 // Longjmp assembly has not been implemented for mips64 yet
-// XFAIL: mips64
+// XFAIL: target-is-mips64
 
 #include <pthread.h>
 #include <stdio.h>
diff --git a/test/tsan/map32bit.cc b/test/tsan/map32bit.cc
index 0411f29a9..cec91a4fa 100644
--- a/test/tsan/map32bit.cc
+++ b/test/tsan/map32bit.cc
@@ -8,7 +8,7 @@
 // https://github.com/google/sanitizers/issues/412
 
 // MAP_32BIT flag for mmap is supported only for x86_64.
-// XFAIL: mips64
+// XFAIL: target-is-mips64
 // XFAIL: aarch64
 // XFAIL: powerpc64
 
diff --git a/test/tsan/signal_longjmp.cc b/test/tsan/signal_longjmp.cc
index 45e24626c..290056b15 100644
--- a/test/tsan/signal_longjmp.cc
+++ b/test/tsan/signal_longjmp.cc
@@ -4,7 +4,7 @@
 // https://github.com/google/sanitizers/issues/482
 
 // Longjmp assembly has not been implemented for mips64 yet
-// XFAIL: mips64
+// XFAIL: target-is-mips64
 // This test fails on powerpc64 BE (VMA=44), a segmentation fault
 // error happens at the second assignment
 // "((volatile int *volatile)mem)[1] = 1".