[globalisel][tablegen] Generate rule coverage and use it to identify untested rules

Summary:
This patch adds a LLVM_ENABLE_GISEL_COV which, like LLVM_ENABLE_DAGISEL_COV,
causes TableGen to instrument the generated table to collect rule coverage
information. However, LLVM_ENABLE_GISEL_COV goes a bit further than
LLVM_ENABLE_DAGISEL_COV. The information is written to files
(${CMAKE_BINARY_DIR}/gisel-coverage-* by default). These files can then be
concatenated into ${LLVM_GISEL_COV_PREFIX}-all after which TableGen will
read this information and use it to emit warnings about untested rules.

This technique could also be used by SelectionDAG and can be further
extended to detect hot rules and give them priority over colder rules.

Usage:
* Enable LLVM_ENABLE_GISEL_COV in CMake
* Build the compiler and run some tests
* cat gisel-coverage-[0-9]* > gisel-coverage-all
* Delete lib/Target/*/*GenGlobalISel.inc*
* Build the compiler

Known issues:
* ${LLVM_GISEL_COV_PREFIX}-all must be generated as a manual
  step due to a lack of a portable 'cat' command. It should be the
  concatenation of all ${LLVM_GISEL_COV_PREFIX}-[0-9]* files.
* There's no mechanism to discard coverage information when the ruleset
  changes

Depends on D39742

Reviewers: ab, qcolombet, t.p.northover, aditya_nandakumar, rovka

Reviewed By: rovka

Subscribers: vsk, arsenm, nhaehnle, mgorny, kristof.beyls, javed.absar, igorb, llvm-commits

Differential Revision: https://reviews.llvm.org/D39747

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@318356 91177308-0d34-0410-b5e6-96231b3b80d8

1 files changed, 67 insertions, 0 deletions

diff --git a/utils/llvm-gisel-cov.py b/utils/llvm-gisel-cov.py
new file mode 100644
index 00000000000..a74ed10f864
--- /dev/null
+++ b/utils/llvm-gisel-cov.py
@@ -0,0 +1,67 @@
+#!/usr/bin/env python
+"""
+Summarize the information in the given coverage files.
+
+Emits the number of rules covered or the percentage of rules covered depending
+on whether --num-rules has been used to specify the total number of rules.
+"""
+
+import argparse
+import struct
+
+class FileFormatError(Exception):
+  pass
+
+def backend_int_pair(s):
+  backend, sep, value = s.partition('=')
+  if (sep is None):
+    raise argparse.ArgumentTypeError("'=' missing, expected name=value")
+  if (not backend):
+    raise argparse.ArgumentTypeError("Expected name=value")
+  if (not value):
+    raise argparse.ArgumentTypeError("Expected name=value")
+  return backend, int(value)
+
+def main():
+  parser = argparse.ArgumentParser(description=__doc__)
+  parser.add_argument('input', nargs='+')
+  parser.add_argument('--num-rules', type=backend_int_pair, action='append',
+                      metavar='BACKEND=NUM',
+                      help='Specify the number of rules for a backend')
+  args = parser.parse_args()
+
+  covered_rules = {}
+
+  for input_filename in args.input:
+    with open(input_filename, 'rb') as input_fh:
+      data = input_fh.read()
+      pos = 0
+      while data:
+        backend, _, data = data.partition('\0')
+        pos += len(backend)
+        pos += 1
+
+        if len(backend) == 0:
+          raise FileFormatError()
+        backend, = struct.unpack("%ds" % len(backend), backend)
+
+        while data:
+          if len(data) < 8:
+            raise FileFormatError()
+          rule_id, = struct.unpack("Q", data[:8])
+          pos += 8
+          data = data[8:]
+          if rule_id == (2 ** 64) - 1:
+            break
+          covered_rules[backend] = covered_rules.get(backend, {})
+          covered_rules[backend][rule_id] = covered_rules[backend].get(rule_id, 0) + 1
+
+  num_rules = dict(args.num_rules)
+  for backend, rules_for_backend in covered_rules.items():
+    if backend in num_rules:
+      print "%s: %3.2f%% of rules covered" % (backend, (float(len(rules_for_backend.keys())) / num_rules[backend]) * 100)
+    else:
+      print "%s: %d rules covered" % (backend, len(rules_for_backend.keys()))
+
+if __name__ == '__main__':
+  main()