[scudo] Quarantine overhaul

Summary:
First, some context.

The main feedback we get about the quarantine is that it's too memory hungry.
A single MB of quarantine will have an impact of 3 to 4MB of PSS/RSS, and
things quickly get out of hand in terms of memory usage, and the quarantine
ends up disabled.

The main objective of the quarantine is to protect from use-after-free
exploitation by making it harder for an attacker to reallocate a controlled
chunk in place of the targeted freed chunk. This is achieved by not making it
available to the backend right away for reuse, but holding it a little while.

Historically, what has usually been the target of such attacks was objects,
where vtable pointers or other function pointers could constitute a valuable
targeti to replace. Those are usually on the smaller side. There is barely any
advantage in putting the quarantine several megabytes of RGB data or the like.

Now for the patch.

This patch introduces a new way the Quarantine behaves in Scudo. First of all,
the size of the Quarantine will be defined in KB instead of MB, then we
introduce a new option: the size up to which (lower than or equal to) a chunk
will be quarantined. This way, we only quarantine smaller chunks, and the size
of the quarantine remains manageable. It also prevents someone from triggering
a recycle by allocating something huge. We default to 512 bytes on 32-bit and
2048 bytes on 64-bit platforms.

In details, the patches includes the following:
- introduce `QuarantineSizeKb`, but honor `QuarantineSizeMb` if set to fall
  back to the old behavior (meaning no threshold in that case);
  `QuarantineSizeMb` is described as deprecated in the options descriptios;
  documentation update will follow;
- introduce `QuarantineChunksUpToSize`, the new threshold value;
- update the `quarantine.cpp` test, and other tests using `QuarantineSizeMb`;
- remove `AllocatorOptions::copyTo`, it wasn't used;
- slightly change the logic around `quarantineOrDeallocateChunk` to accomodate
  for the new logic; rename a couple of variables there as well;

Rewriting the tests, I found a somewhat annoying bug where non-default aligned
chunks would account for more than needed when placed in the quarantine due to
`<< MinAlignment` instead of `<< MinAlignmentLog`. This is fixed and tested for
now.

Reviewers: alekseyshl, kcc

Reviewed By: alekseyshl

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D35694

git-svn-id: https://llvm.org/svn/llvm-project/compiler-rt/trunk@308884 91177308-0d34-0410-b5e6-96231b3b80d8

2 files changed, 70 insertions, 9 deletions

diff --git a/test/scudo/overflow.cpp b/test/scudo/overflow.cpp
index d12824578..82ea10fd1 100644
--- a/test/scudo/overflow.cpp
+++ b/test/scudo/overflow.cpp
@@ -1,6 +1,6 @@
 // RUN: %clang_scudo %s -o %t
-// RUN:                                  not %run %t malloc     2>&1 | FileCheck %s
-// RUN: SCUDO_OPTIONS=QuarantineSizeMb=1 not %run %t quarantine 2>&1 | FileCheck %s
+// RUN:                                   not %run %t malloc     2>&1 | FileCheck %s
+// RUN: SCUDO_OPTIONS=QuarantineSizeKb=64 not %run %t quarantine 2>&1 | FileCheck %s
 
 // Tests that header corruption of an allocated or quarantined chunk is caught.
 
@@ -29,7 +29,7 @@ int main(int argc, char **argv)
     ((char *)p)[-(offset + 2)] ^= 1;
     // Trigger the quarantine recycle
     for (int i = 0; i < 0x100; i++) {
-      p = malloc(1U << 16);
+      p = malloc(1U << 8);
       free(p);
     }
   }
diff --git a/test/scudo/quarantine.cpp b/test/scudo/quarantine.cpp
index 39ce1bd91..cfa9aa919 100644
--- a/test/scudo/quarantine.cpp
+++ b/test/scudo/quarantine.cpp
@@ -1,10 +1,15 @@
 // RUN: %clang_scudo %s -o %t
-// RUN: SCUDO_OPTIONS="QuarantineSizeMb=0:ThreadLocalQuarantineSizeKb=0" %run %t zeroquarantine 2>&1
-// RUN: SCUDO_OPTIONS=QuarantineSizeMb=1                                 %run %t smallquarantine 2>&1
+// RUN: SCUDO_OPTIONS="QuarantineSizeMb=1:QuarantineSizeKb=64"           not %run %t unused 2>&1
+// RUN: SCUDO_OPTIONS="QuarantineSizeMb=1:QuarantineChunksUpToSize=256"  not %run %t unused 2>&1
+// RUN: SCUDO_OPTIONS="QuarantineSizeKb=0:ThreadLocalQuarantineSizeKb=0"     %run %t zeroquarantine 2>&1
+// RUN: SCUDO_OPTIONS=QuarantineSizeKb=64                                    %run %t smallquarantine 2>&1
+// RUN: SCUDO_OPTIONS=QuarantineChunksUpToSize=256                           %run %t threshold 2>&1
+// RUN: SCUDO_OPTIONS="QuarantineSizeMb=1"                                   %run %t oldquarantine 2>&1
 
 // Tests that the quarantine prevents a chunk from being reused right away.
 // Also tests that a chunk will eventually become available again for
-// allocation when the recycling criteria has been met.
+// allocation when the recycling criteria has been met. Finally, tests the
+// threshold up to which a chunk is quarantine, and the old quarantine behavior.
 
 #include <assert.h>
 #include <malloc.h>
@@ -16,9 +21,16 @@
 int main(int argc, char **argv)
 {
   void *p, *old_p;
-  size_t allocated_bytes, size = 1U << 16;
+  size_t allocated_bytes, size = 1U << 8, alignment = 1U << 8;
 
   assert(argc == 2);
+  // First, warm up the allocator for the classes used.
+  p = malloc(size);
+  assert(p);
+  free(p);
+  assert(posix_memalign(&p, alignment, size) == 0);
+  assert(p);
+  free(p);
 
   if (!strcmp(argv[1], "zeroquarantine")) {
     // Verifies that a chunk is deallocated right away when the local and
@@ -44,13 +56,62 @@ int main(int argc, char **argv)
 
     // Eventually the chunk should become available again.
     bool found = false;
-    for (int i = 0; i < 0x100 && found == false; i++) {
+    for (int i = 0; i < 0x200 && !found; i++) {
       p = malloc(size);
       assert(p);
       found = (p == old_p);
       free(p);
     }
-    assert(found == true);
+    assert(found);
+  }
+  if (!strcmp(argv[1], "threshold")) {
+    // Verifies that a chunk of size greater than the threshold will be freed
+    // right away. Alignment has no impact on the threshold.
+    allocated_bytes = __sanitizer_get_current_allocated_bytes();
+    p = malloc(size + 1);
+    assert(p);
+    assert(__sanitizer_get_current_allocated_bytes() > allocated_bytes);
+    free(p);
+    assert(__sanitizer_get_current_allocated_bytes() == allocated_bytes);
+    assert(posix_memalign(&p, alignment, size + 1) == 0);
+    assert(__sanitizer_get_current_allocated_bytes() > allocated_bytes);
+    free(p);
+    assert(__sanitizer_get_current_allocated_bytes() == allocated_bytes);
+    // Verifies that a chunk of size lower or equal to the threshold will be
+    // quarantined.
+    p = malloc(size);
+    assert(p);
+    assert(__sanitizer_get_current_allocated_bytes() > allocated_bytes);
+    free(p);
+    assert(__sanitizer_get_current_allocated_bytes() > allocated_bytes);
+    allocated_bytes = __sanitizer_get_current_allocated_bytes();
+    assert(posix_memalign(&p, alignment, size) == 0);
+    assert(__sanitizer_get_current_allocated_bytes() > allocated_bytes);
+    free(p);
+    assert(__sanitizer_get_current_allocated_bytes() > allocated_bytes);
+  }
+  if (!strcmp(argv[1], "oldquarantine")) {
+    // Verifies that we quarantine everything if the deprecated quarantine
+    // option is specified. Alignment has no impact on the threshold.
+    allocated_bytes = __sanitizer_get_current_allocated_bytes();
+    p = malloc(size);
+    assert(p);
+    assert(__sanitizer_get_current_allocated_bytes() > allocated_bytes);
+    free(p);
+    assert(__sanitizer_get_current_allocated_bytes() > allocated_bytes);
+    allocated_bytes = __sanitizer_get_current_allocated_bytes();
+    assert(posix_memalign(&p, alignment, size) == 0);
+    assert(p);
+    assert(__sanitizer_get_current_allocated_bytes() > allocated_bytes);
+    free(p);
+    assert(__sanitizer_get_current_allocated_bytes() > allocated_bytes);
+    // Secondary backed allocation.
+    allocated_bytes = __sanitizer_get_current_allocated_bytes();
+    p = malloc(1U << 19);
+    assert(p);
+    assert(__sanitizer_get_current_allocated_bytes() > allocated_bytes);
+    free(p);
+    assert(__sanitizer_get_current_allocated_bytes() > allocated_bytes);
   }
 
   return 0;