summaryrefslogtreecommitdiff
path: root/docs/TestingGuide.rst
diff options
context:
space:
mode:
Diffstat (limited to 'docs/TestingGuide.rst')
-rw-r--r--docs/TestingGuide.rst66
1 files changed, 66 insertions, 0 deletions
diff --git a/docs/TestingGuide.rst b/docs/TestingGuide.rst
new file mode 100644
index 000000000..4edda6738
--- /dev/null
+++ b/docs/TestingGuide.rst
@@ -0,0 +1,66 @@
+========================================
+Compiler-rt Testing Infrastructure Guide
+========================================
+
+.. contents::
+ :local:
+
+Overview
+========
+
+This document is the reference manual for the compiler-rt modifications to the
+testing infrastructure. Documentation for the infrastructure itself can be found at
+:ref:`llvm_testing_guide`.
+
+LLVM testing infrastructure organization
+========================================
+
+The compiler-rt testing infrastructure contains regression tests which are run
+as part of the usual ``make check-all`` and are expected to always pass -- they
+should be run before every commit.
+
+Quick start
+===========
+
+The regressions tests are in the "compiler-rt" module and are normally checked
+out in the directory ``llvm/projects/compiler-rt/test``. Use ``make check-all``
+to run the regression tests after building compiler-rt.
+
+REQUIRES, XFAIL, etc.
+---------------------
+
+Sometimes it is necessary to restrict a test to a specific target or mark it as
+an "expected fail" or XFAIL. This is normally achieved using ``REQUIRES:`` or
+``XFAIL:`` with a substring of LLVM's default target triple. Unfortunately, the
+behaviour of this is somewhat quirky in compiler-rt. There are two main
+pitfalls to avoid.
+
+The first pitfall is that these directives perform a substring match on the
+triple and as such ``XFAIL: mips`` affects more triples than expected. For
+example, ``mips-linux-gnu``, ``mipsel-linux-gnu``, ``mips64-linux-gnu``, and
+``mips64el-linux-gnu`` will all match a ``XFAIL: mips`` directive. Including a
+trailing ``-`` such as in ``XFAIL: mips-`` can help to mitigate this quirk but
+even that has issues as described below.
+
+The second pitfall is that the default target triple is often inappropriate for
+compiler-rt tests since compiler-rt tests may be compiled for multiple targets.
+For example, a typical build on an ``x86_64-linux-gnu`` host will often run the
+tests for both x86_64 and i386. In this situation ``XFAIL: x86_64`` will mark
+both the x86_64 and i386 tests as an expected failure while ``XFAIL: i386``
+will have no effect at all.
+
+To remedy both pitfalls, compiler-rt tests provide a feature string which can
+be used to specify a single target. This string is of the form
+``target-is-${arch}`` where ``${arch}}`` is one of the values from the
+following lines of the CMake output::
+
+ -- Compiler-RT supported architectures: x86_64;i386
+ -- Builtin supported architectures: i386;x86_64
+
+So for example ``XFAIL: target-is-x86_64`` will mark a test as expected to fail
+on x86_64 without also affecting the i386 test and ``XFAIL: target-is-i386``
+will mark a test as expected to fail on i386 even if the default target triple
+is ``x86_64-linux-gnu``. Directives that use these ``target-is-${arch}`` string
+require exact matches so ``XFAIL: target-is-mips``,
+``XFAIL: target-is-mipsel``, ``XFAIL: target-is-mips64``, and
+``XFAIL: target-is-mips64el`` all refer to different MIPS targets.