| Age | Commit message (Collapse) | Author |
|---|
|
Summary:
There is currently too much redundancy in the class/variable/* names in Scudo:
- we are in the namespace `__scudo`, so there is no point in having something
named `ScudoX` to end up with a final name of `__scudo::ScudoX`;
- there are a lot of types/* that have `Allocator` in the name, given that
Scudo is an allocator I figure this doubles up as well.
So change a bunch of the Scudo names to make them shorter, less redundant, and
overall simpler. They should still be pretty self explaining (or at least it
looks so to me).
The TSD part will be done in another CL (eg `__scudo::ScudoTSD`).
Reviewers: alekseyshl, eugenis
Reviewed By: alekseyshl
Subscribers: delcypher, #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D49505
git-svn-id: https://llvm.org/svn/llvm-project/compiler-rt/trunk@337557 91177308-0d34-0410-b5e6-96231b3b80d8
|
|
when building with an IDE so that header files show up in the UI.
This massively improves the development workflow in IDEs.
To implement this a new function `compiler_rt_process_sources(...)` has
been added that adds header files to the list of sources when the
generator is an IDE. For non-IDE generators (e.g. Ninja/Makefile) no
changes are made to the list of source files.
The function can be passed a list of headers via the
`ADDITIONAL_HEADERS` argument. For each runtime library a list of
explicit header files has been added and passed via
`ADDITIONAL_HEADERS`. For `tsan` and `sanitizer_common` a list of
headers was already present but it was stale and has been updated
to reflect the current state of the source tree.
The original version of this patch used file globbing (`*.{h,inc,def}`)
to find the headers but the approach was changed due to this being a
CMake anti-pattern (if the list of headers changes CMake won't
automatically re-generate if globbing is used).
The LLVM repo contains a similar function named `llvm_process_sources()`
but we don't use it here for several reasons:
* It depends on the `LLVM_ENABLE_OPTION` cache variable which is
not set in standalone compiler-rt builds.
* We would have to `include(LLVMProcessSources)` which I'd like to
avoid because it would include a bunch of stuff we don't need.
Differential Revision: https://reviews.llvm.org/D48422
git-svn-id: https://llvm.org/svn/llvm-project/compiler-rt/trunk@336663 91177308-0d34-0410-b5e6-96231b3b80d8
|
|
Summary:
Namely, set the abort message, and allow to write the message to syslog if the
option is enabled.
Reviewers: alekseyshl
Reviewed By: alekseyshl
Subscribers: delcypher, #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D48902
git-svn-id: https://llvm.org/svn/llvm-project/compiler-rt/trunk@336445 91177308-0d34-0410-b5e6-96231b3b80d8
|
|
Summary:
The C interceptors were using `SIZE_T` defined in the interception library as
a `__sanitizer::uptr`. On some 32-bit platforms, this lead to the following
warning:
```
warning: declaration of ‘void* malloc(SIZE_T)’ conflicts with built-in declaration ‘void* malloc(unsigned int)’ [-Wbuiltin-declaration-mismatch]
INTERCEPTOR_ATTRIBUTE void *malloc(SIZE_T size) {
```
`__sanitizer::uptr` is indeed defined as an `unsigned long` on those.
So just include `stddef.h` and use `size_t` instead.
Reviewers: alekseyshl
Reviewed By: alekseyshl
Subscribers: delcypher, llvm-commits, #sanitizers
Differential Revision: https://reviews.llvm.org/D48885
git-svn-id: https://llvm.org/svn/llvm-project/compiler-rt/trunk@336221 91177308-0d34-0410-b5e6-96231b3b80d8
|
|
Summary:
It would be useful for Flutter apps, especially, to be able to use
malloc hooks to debug memory leaks on Fuchsia. They're not able to do
this right now, so it'd be a nice bonus to throw in with the Scudo
switchover.
Reviewers: cryptoad, alekseyshl
Reviewed By: cryptoad
Differential Revision: https://reviews.llvm.org/D48618
git-svn-id: https://llvm.org/svn/llvm-project/compiler-rt/trunk@336139 91177308-0d34-0410-b5e6-96231b3b80d8
|
|
Summary:
This patch follows D48373.
The point is to be able to use Scudo with `-fsanitize-minimal-runtime`. For that
we need a runtime that doesn't embed the UBSan one. This results in binaries
that can be compiled with `-fsanitize=scudo,integer -fsanitize-minimal-runtime`.
Reviewers: eugenis
Reviewed By: eugenis
Subscribers: mgorny, delcypher, llvm-commits, #sanitizers
Differential Revision: https://reviews.llvm.org/D48377
git-svn-id: https://llvm.org/svn/llvm-project/compiler-rt/trunk@335296 91177308-0d34-0410-b5e6-96231b3b80d8
|
|
Summary:
Mark `isRssLimitExceeded` as `NOINLINE`, and move it's definition as well as
the one of `performSanityChecks` out of the class definition, as requested.
Reviewers: filcab, alekseyshl
Reviewed By: alekseyshl
Subscribers: delcypher, #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D48228
git-svn-id: https://llvm.org/svn/llvm-project/compiler-rt/trunk@335054 91177308-0d34-0410-b5e6-96231b3b80d8
|
|
Summary:
The current `FailureHandler` mechanism was fairly opaque with regard to the
failure reason due to using `CHECK(0)`. Scudo is a bit different from the other
Sanitizers as it prefers to avoid spurious processing in its failure path. So
we just `dieWithMessage` using a somewhat explicit string.
Adapted the tests for the new strings.
While this takes care of the `OnBadRequest` & `OnOOM` failures, the next step
is probably to migrate the other Scudo failures in the same failes (header
corruption, invalid state and so on).
Reviewers: alekseyshl
Reviewed By: alekseyshl
Subscribers: filcab, mgorny, delcypher, #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D48199
git-svn-id: https://llvm.org/svn/llvm-project/compiler-rt/trunk@334843 91177308-0d34-0410-b5e6-96231b3b80d8
|
|
Summary:
As a follow up to D48142 for Scudo, switch the `SpinMutex` to its static
counterpart, and ensure zero-initialization by memset'ing the whole class.
Reviewers: alekseyshl
Reviewed By: alekseyshl
Subscribers: delcypher, #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D48148
git-svn-id: https://llvm.org/svn/llvm-project/compiler-rt/trunk@334716 91177308-0d34-0410-b5e6-96231b3b80d8
|
|
Summary:
This CL adds support for aligned new/delete operators (C++17). Currently we
do not support alignment inconsistency detection on deallocation, as this
requires a header change, but the APIs are introduced and are functional.
Add a smoke test for the aligned version of the operators.
Reviewers: alekseyshl
Reviewed By: alekseyshl
Subscribers: delcypher, #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D48031
git-svn-id: https://llvm.org/svn/llvm-project/compiler-rt/trunk@334505 91177308-0d34-0410-b5e6-96231b3b80d8
|
|
Summary:
The shared TSD model in its current form doesn't scale. Here is an example of
rpc2-benchmark (with default parameters, which is threading heavy) on a 72-core
machines (defaulting to a `CompactSizeClassMap` and no Quarantine):
- with tcmalloc: 337K reqs/sec, peak RSS of 338MB;
- with scudo (exclusive): 321K reqs/sec, peak RSS of 637MB;
- with scudo (shared): 241K reqs/sec, peak RSS of 324MB.
This isn't great, since the exclusive model uses a lot of memory, while the
shared model doesn't even come close to be competitive.
This is mostly due to the fact that we are consistently scanning the TSD pool
starting at index 0 for an available TSD, which can result in a lot of failed
lock attempts, and touching some memory that needs not be touched.
This CL attempts to make things better in most situations:
- first, use a thread local variable on Linux (intead of pthread APIs) to store
the current TSD in the shared model;
- move the locking boolean out of the TSD: this allows the compiler to use a
register and potentially optimize out a branch instead of reading it from the
TSD everytime (we also save a tiny bit of memory per TSD);
- 64-bit atomic operations on 32-bit ARM platforms happen to be expensive: so
store the `Precedence` in a `uptr` instead of a `u64`. We lose some
nanoseconds of precision and we'll wrap around at some point, but the benefit
is worth it;
- change a `CHECK` to a `DCHECK`: this should never happen, but if something is
ever terribly wrong, we'll crash on a near null AV if the TSD happens to be
null;
- based on an idea by dvyukov@, we are implementing a bound random scan for
an available TSD. This requires computing the coprimes for the number of TSDs,
and attempting to lock up to 4 TSDs in an random order before falling back to
the current one. This is obviously slightly more expansive when we have just
2 TSDs (barely noticeable) but is otherwise beneficial. The `Precedence` still
basically corresponds to the moment of the first contention on a TSD. To seed
on random choice, we use the precedence of the current TSD since it is very
likely to be non-zero (since we are in the slow path after a failed `tryLock`)
With those modifications, the benchmark yields to:
- with scudo (shared): 330K reqs/sec, peak RSS of 327MB.
So the shared model for this specific situation not only becomes competitive but
outperforms the exclusive model. I experimented with some values greater than 4
for the number of TSDs to attempt to lock and it yielded a decrease in QPS. Just
sticking with the current TSD is also a tad slower. Numbers on platforms with
less cores (eg: Android) remain similar.
Reviewers: alekseyshl, dvyukov, javed.absar
Reviewed By: alekseyshl, dvyukov
Subscribers: srhines, kristof.beyls, delcypher, llvm-commits, #sanitizers
Differential Revision: https://reviews.llvm.org/D47289
git-svn-id: https://llvm.org/svn/llvm-project/compiler-rt/trunk@334410 91177308-0d34-0410-b5e6-96231b3b80d8
|
|
Summary:
It turns out that the previous code construct was not optimizing the allocation
and deallocation of batches. The class id was read as a class member (even
though a precomputed one) and nothing else was optimized. By changing the
construct this way, the compiler actually optimizes most of the allocation and
deallocation away to only work with a single class id, which not only saves some
CPU but also some code footprint.
Reviewers: alekseyshl, dvyukov
Reviewed By: dvyukov
Subscribers: dvyukov, delcypher, llvm-commits, #sanitizers
Differential Revision: https://reviews.llvm.org/D46961
git-svn-id: https://llvm.org/svn/llvm-project/compiler-rt/trunk@332502 91177308-0d34-0410-b5e6-96231b3b80d8
|
|
Summary:
This adds `__scudo_print_stats` as an interface function to display the Primary
and Secondary allocator statistics for Scudo.
Reviewers: alekseyshl, flowerhack
Reviewed By: alekseyshl
Subscribers: delcypher, llvm-commits, #sanitizers
Differential Revision: https://reviews.llvm.org/D46016
git-svn-id: https://llvm.org/svn/llvm-project/compiler-rt/trunk@330857 91177308-0d34-0410-b5e6-96231b3b80d8
|
|
Summary:
Fuchsia uses zx_system_get_features in lieu of getauxval.
Use this call when checking for CRC32 support.
Reviewers: cryptoad
Reviewed By: cryptoad
Subscribers: delcypher, llvm-commits, #sanitizers, kristof.beyls, chrib
Differential Revision: https://reviews.llvm.org/D45896
git-svn-id: https://llvm.org/svn/llvm-project/compiler-rt/trunk@330598 91177308-0d34-0410-b5e6-96231b3b80d8
|
|
Summary:
Host symbolizer & stacktraces related code in their own RT:
`RTSanitizerCommonSymbolizer`, which is "libcdep" by nature. Symbolizer &
stacktraces specific code that used to live in common files is moved to a new
file `sanitizer_symbolizer_report.cc` as is.
The purpose of this is the enforce a separation between code that relies on
symbolization and code that doesn't. This saves the inclusion of spurious code
due to the interface functions with default visibility, and the extra data
associated.
The following sanitizers makefiles were modified & tested locally:
- dfsan: doesn't require the new symbolizer RT
- esan: requires it
- hwasan: requires it
- lsan: requires it
- msan: requires it
- safestack: doesn't require it
- xray: doesn't require it
- tsan: requires it
- ubsan: requires it
- ubsan_minimal: doesn't require it
- scudo: requires it (but not for Fuchsia that has a minimal runtime)
This was tested locally on Linux, Android, Fuchsia.
Reviewers: alekseyshl, eugenis, dberris, kubamracek, vitalybuka, dvyukov, mcgrathr
Reviewed By: alekseyshl, vitalybuka
Subscribers: srhines, kubamracek, mgorny, krytarowski, delcypher, llvm-commits, #sanitizers
Differential Revision: https://reviews.llvm.org/D45457
git-svn-id: https://llvm.org/svn/llvm-project/compiler-rt/trunk@330131 91177308-0d34-0410-b5e6-96231b3b80d8
|
|
Summary:
In the same spirit of SanitizerToolName, allow the Primary & Secondary
allocators to have names that can be set by the tools via PrimaryAllocatorName
and SecondaryAllocatorName.
Additionally, set a non-default name for Scudo.
Reviewers: alekseyshl, vitalybuka
Reviewed By: alekseyshl, vitalybuka
Subscribers: kubamracek, delcypher, #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D45600
git-svn-id: https://llvm.org/svn/llvm-project/compiler-rt/trunk@330055 91177308-0d34-0410-b5e6-96231b3b80d8
|
|
Summary:
Now that common options are propagated again for runtimes build with D45507,
the -f{data,function}-sections flags are now duplicates, remove them.
Reviewers: alekseyshl
Reviewed By: alekseyshl
Subscribers: mgorny, delcypher, #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D45575
git-svn-id: https://llvm.org/svn/llvm-project/compiler-rt/trunk@329925 91177308-0d34-0410-b5e6-96231b3b80d8
|
|
Summary:
Fuchsia requires its Scudo shared runtime to not be C++ dependant. Since they
don't use UBSan in conjunction with Scudo, we can just remove the runtime,
and add the extra `nostdinc++` and `nostdlib++` flags. No need for Coverage
either. This allows to keep things going while working on additional splits
of sanitizer_commong and a more minimal runtime.
Reviewers: phosek, flowerhack, alekseyshl
Reviewed By: phosek, alekseyshl
Subscribers: mgorny, delcypher, #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D44791
git-svn-id: https://llvm.org/svn/llvm-project/compiler-rt/trunk@328625 91177308-0d34-0410-b5e6-96231b3b80d8
|
|
Summary:
`sanitizer_common`'s coverage support is fairly well separated, and libcdep by
default. Several sanitizers don't make use of coverage, and as far as I can
tell do no benefit from the extra dependencies pulled in by the coverage public
interface functions.
The following sanitizers call `InitializeCoverage` explicitely: MSan, ASan,
LSan, HWAsan, UBSan. On top of this, any sanitizer bundling RTUBSan should
add the coverage RT as well: ASan, Scudo, UBSan, CFI (diag), TSan, MSan, HWAsan.
So in the end the following have no need: DFSan, ESan, CFI, SafeStack (nolibc
anyway), XRay, and the upcoming Scudo minimal runtime.
I tested this with all the sanitizers check-* with gcc & clang, and in
standalone on Linux & Android, and there was no issue. I couldn't test this on
Mac, Fuchsia, BSDs, & Windows for lack of an environment, so adding a bunch of
people for additional scrunity. I couldn't test HWAsan either.
Reviewers: eugenis, vitalybuka, alekseyshl, flowerhack, kubamracek, dberris, rnk, krytarowski
Reviewed By: vitalybuka, alekseyshl, flowerhack, dberris
Subscribers: mgorny, delcypher, #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D44701
git-svn-id: https://llvm.org/svn/llvm-project/compiler-rt/trunk@328204 91177308-0d34-0410-b5e6-96231b3b80d8
|
|
Summary:
Using `getActuallyAllocatedSize` from the Combined resulting in mediocre
compiled code, as the `ClassId != 0` predicament was not propagated there,
resulting in additional branches and dead code. Move the logic in the frontend,
which results in better compiled code. Also I think it makes it slightly easier
to distinguish between the size the user requested, and the size that was
actually allocated by the allocator.
`const` a couple of things as well.
This has no functional impact.
Reviewers: alekseyshl
Reviewed By: alekseyshl
Subscribers: delcypher, #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D44444
git-svn-id: https://llvm.org/svn/llvm-project/compiler-rt/trunk@327525 91177308-0d34-0410-b5e6-96231b3b80d8
|
|
Summary:
The need for this change stems from the fact that Windows doesn't support
partial unmapping (`MEM_RELEASE` implies the entire allocated region). So we
now have to keep track of the reserved region and the committed region, so that
we can function without the trimming we did when dealing with larger alignments.
Instead of just having a `ReservedAddressRange` per chunk, we introduce a
`LargeChunkHeader` (and `LargeChunk` namespace) that additionally holds the
committed size and the usable size. The former is needed for stats purposes,
the latter is used by the frontend. Requiring both is debatable, we could only
work with the usable size but then be off by up to a page per chunk when
dealing with stats.
Additionally, we introduce more stats since they turned out to be useful for
experiments, and a `PrintStats` function that will be used by the combined
allocator in later patch.
Reviewers: alekseyshl, flowerhack
Reviewed By: alekseyshl
Subscribers: delcypher, #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D43949
git-svn-id: https://llvm.org/svn/llvm-project/compiler-rt/trunk@327321 91177308-0d34-0410-b5e6-96231b3b80d8
|
|
Summary:
A few changes related to logging:
- prepend `Scudo` to the error messages so that users can identify that we
reported an error;
- replace a couple of `Report` calls in the RSS check code with
`dieWithMessage`/`Print`, mark a condition as `UNLIKELY` in the process;
- change some messages so that they all look more or less the same. This
includes the `CHECK` message;
- adapt a couple of tests with the new strings.
A couple of side notes: this results in a few 1-line-blocks, for which I left
brackets. There doesn't seem to be any style guide for that, I can remove them
if need be. I didn't use `SanitizerToolName` in the strings, but directly
`Scudo` because we are the only users, I could change that too.
Reviewers: alekseyshl, flowerhack
Reviewed By: alekseyshl
Subscribers: mgorny, delcypher, llvm-commits, #sanitizers
Differential Revision: https://reviews.llvm.org/D44171
git-svn-id: https://llvm.org/svn/llvm-project/compiler-rt/trunk@326901 91177308-0d34-0410-b5e6-96231b3b80d8
|
|
Summary:
If not using `-Wl,--gc-sections`, a whole lot of unused `sanitizer_common` code
and related static variables are pulled into the shared library.
Keep the binary size smaller, and its memory footprint smaller as well, by
using the compiler flags `-ffunction-section` & `-fdata-sections` by default,
as well as the linker flags `-Wl,--gc-sections`.
Current experiments show a large discrepency between binary sizes generated
by gcc (big) and clang (small). I am not sure yet how I can make a test that
would encompass both, so it's an outstanding work item.
Reviewers: alekseyshl, flowerhack
Reviewed By: alekseyshl
Subscribers: mgorny, delcypher, llvm-commits, #sanitizers
Differential Revision: https://reviews.llvm.org/D44121
git-svn-id: https://llvm.org/svn/llvm-project/compiler-rt/trunk@326833 91177308-0d34-0410-b5e6-96231b3b80d8
|
|
Summary:
Instead of using `AlignedChunkHeaderSize`, introduce a `constexpr` function
`getHeaderSize` in the `Chunk` namespace. Switch `RoundUpTo` to a `constexpr`
as well (so we can use it in `constexpr` declarations). Mark a few variables
in the areas touched as `const`.
Overall this has no functional change, and is mostly to make things a bit more
consistent.
Reviewers: alekseyshl
Reviewed By: alekseyshl
Subscribers: delcypher, #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D43772
git-svn-id: https://llvm.org/svn/llvm-project/compiler-rt/trunk@326206 91177308-0d34-0410-b5e6-96231b3b80d8
|
|
Summary:
Allow for options to be defined at compile time, like is already the case for
other sanitizers, via `SCUDO_DEFAULT_OPTIONS`.
Reviewers: alekseyshl, dberris
Reviewed By: alekseyshl, dberris
Subscribers: kubamracek, delcypher, llvm-commits, #sanitizers
Differential Revision: https://reviews.llvm.org/D42980
git-svn-id: https://llvm.org/svn/llvm-project/compiler-rt/trunk@324620 91177308-0d34-0410-b5e6-96231b3b80d8
|
|
Summary:
Few changes to the secondary:
- mark `const` variables as such;
- change some `CHECK` to `DCHECK`: I don't feel we need to be as conservative as
we were with out checks, as they are the results of our own computation.
- mark a condition as `UNLIKELY`.
Reviewers: alekseyshl
Reviewed By: alekseyshl
Subscribers: delcypher, #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D42696
git-svn-id: https://llvm.org/svn/llvm-project/compiler-rt/trunk@323997 91177308-0d34-0410-b5e6-96231b3b80d8
|
|
Summary:
This is in preparation for platforms where `SANITIZER_SUPPORTS_WEAK_HOOKS` is 0.
They require a default implementation.
Reviewers: alekseyshl
Reviewed By: alekseyshl
Subscribers: delcypher, llvm-commits, #sanitizers
Differential Revision: https://reviews.llvm.org/D42557
git-svn-id: https://llvm.org/svn/llvm-project/compiler-rt/trunk@323795 91177308-0d34-0410-b5e6-96231b3b80d8
|
|
Summary:
This is a follow-up to D42506.
There are a few of things that bothered me about `scudo_interceptors.cpp`:
- the filename is a misnomer: it intercepts some functions, but the rest (C++)
is actually in `scudo_new_delete.cpp`. I feel like `scudo_malloc.cpp` is more
appropriate (ASan uses the same naming scheme);
- we do not need "full" interceptors, since we are never accessing the
unsanitized version of the functions, we just need the
`extern "C" INTERCEPTOR_ATTRIBUTE` part of it to just call our functions;
- a couple of functions where duplicated while they could just be `ALIAS`'d;
- use the `SANITIZER_INTERCEPT_*` defines to hide the unneeded interceptors;
- use `SIZE_T` instead of `uptr`: while it's the same behind the curtain,
the former is meant for this use case.
In the end there is no functional change on the currently supported platforms
(Linux, Android).
Reviewers: alekseyshl
Reviewed By: alekseyshl
Subscribers: mgorny, hintonda, delcypher, #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D42546
git-svn-id: https://llvm.org/svn/llvm-project/compiler-rt/trunk@323464 91177308-0d34-0410-b5e6-96231b3b80d8
|
|
Summary:
Currently all platforms are using the `scudo_interceptors.cpp` interceptors.
We might to come up with platform specific interceptors when/if we get Apple &
Windows, but as of now, that allows for Fuchsia to use them.
`scudo_new_delete.cpp` didn't have the `#if SANITIZER_LINUX` so it's good to go.
Reviewers: alekseyshl, flowerhack
Reviewed By: flowerhack
Subscribers: delcypher, #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D42506
git-svn-id: https://llvm.org/svn/llvm-project/compiler-rt/trunk@323386 91177308-0d34-0410-b5e6-96231b3b80d8
|
|
Summary:
Hooks in the allocation & deallocation paths can be a security risk (see for an
example https://scarybeastsecurity.blogspot.com/2016/11/0day-exploit-advancing-exploitation.html
which used the glibc's __free_hook to complete exploitation).
But some users have expressed a need for them, even if only for tests and
memory benchmarks. So allow for `__sanitizer_malloc_hook` &
`__sanitizer_free_hook` to be called if defined, and gate them behind a global
define `SCUDO_CAN_USE_HOOKS` defaulting to 0.
Reviewers: alekseyshl
Reviewed By: alekseyshl
Subscribers: #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D42430
git-svn-id: https://llvm.org/svn/llvm-project/compiler-rt/trunk@323278 91177308-0d34-0410-b5e6-96231b3b80d8
|
|
Summary:
We somehow never did it, and it raised no issue until now, when trying to
enable Fuchsia as a supported Scudo platform in the cmake config.
So propagate SANITIZER_COMMON_LINK_FLAGS for now.
Reviewers: alekseyshl, flowerhack
Reviewed By: flowerhack
Subscribers: mgorny, #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D42314
git-svn-id: https://llvm.org/svn/llvm-project/compiler-rt/trunk@322999 91177308-0d34-0410-b5e6-96231b3b80d8
|
|
Summary:
A forgotten include in `scudo_allocator.cpp` made the symbol only local :/
Before:
```
nm ./lib/clang/7.0.0/lib/linux/libclang_rt.scudo-i686-android.so | grep rss
00024730 t __scudo_set_rss_limit
```
After:
```
nm ./lib/clang/7.0.0/lib/linux/libclang_rt.scudo-i686-android.so | grep rs
00024760 T __scudo_set_rss_limit
```
And we want `T`!
This include also means that we can get rid of the `extern "C"` in the C++
file, the compiler does fine without it (note that this was already the case
for all the `__sanitizer_*` interface functions.
Reviewers: alekseyshl, eugenis
Reviewed By: eugenis
Subscribers: #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D42199
git-svn-id: https://llvm.org/svn/llvm-project/compiler-rt/trunk@322782 91177308-0d34-0410-b5e6-96231b3b80d8
|
|
Summary:
jemalloc on Android currently uses 2 arenas
(https://android.googlesource.com/platform/external/jemalloc/+/master/Android.bp#64).
Since the Android toolchain absorbs compiler-rt and compiles it as is, we have
to enforce the same limit to somehow stay competitive in terms of memory usage.
The changes could either go in:
- `scudo_platform.h` with a default for Android of 2 (this is the solution
implemented here);
- in `CMakeLists.txt` adding -DSCUDO_SHARED_TSD_POOL_SIZE=2 for Android.
- something else?
I don't have a strong opinion on how to do it, but it has to be done upstream
anyway.
Reviewers: alekseyshl, eugenis
Reviewed By: alekseyshl, eugenis
Subscribers: srhines, #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D42194
git-svn-id: https://llvm.org/svn/llvm-project/compiler-rt/trunk@322764 91177308-0d34-0410-b5e6-96231b3b80d8
|
|
Summary:
This is needed for the shared runtime since we are pulling RTUbsan in.
Otherwise some builds might fail with errors such as:
`error: undefined reference to '__dynamic_cast'`
Reviewers: alekseyshl, srhines
Reviewed By: srhines
Subscribers: kongyi, pirama, chh, mgorny, llvm-commits, #sanitizers
Differential Revision: https://reviews.llvm.org/D41995
git-svn-id: https://llvm.org/svn/llvm-project/compiler-rt/trunk@322389 91177308-0d34-0410-b5e6-96231b3b80d8
|
|
Summary:
`__scudo_set_rss_limit`'s `LimitMb` should really be a `size_t`. Update
accordingly the prototype. To avoid the `NOLINT` and conform with the other
Sanitizers, use the sanitizers types for the internal definition. This should
have no functional change.
Additionally, capitalize a variable name to follow the LLVM coding standards.
Reviewers: alekseyshl, flowerhack
Reviewed By: alekseyshl
Subscribers: #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D41704
git-svn-id: https://llvm.org/svn/llvm-project/compiler-rt/trunk@321803 91177308-0d34-0410-b5e6-96231b3b80d8
|
|
Summary:
The initial implementation used an ASan like Chunk class that was deriving from
a Header class. Due to potential races, we ended up working with local copies
of the Header and never using the parent class fields. ScudoChunk was never
constructed but cast, and we were using `this` as the pointer needed for our
computations. This was meh.
So we refactored ScudoChunk to be now a series of static functions within the
namespace `__scudo::Chunk` that take a "user" pointer as first parameter (former
`this`). A compiled binary doesn't really change, but the code is more sensible.
Clang tends to inline all those small function (in -O2), but GCC left a few not
inlined, so we add the `INLINE` keyword to all.
Since we don't have `ScudoChunk` pointers anymore, a few variables were renamed
here and there to introduce a clearer distinction between a user pointer
(usually `Ptr`) and a backend pointer (`BackendPtr`).
Reviewers: alekseyshl, flowerhack
Reviewed By: alekseyshl
Subscribers: #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D41200
git-svn-id: https://llvm.org/svn/llvm-project/compiler-rt/trunk@320745 91177308-0d34-0410-b5e6-96231b3b80d8
|
|
Summary:
The first and only function to start with allows to set the soft or hard RSS
limit at runtime. Add associated tests.
Reviewers: alekseyshl
Reviewed By: alekseyshl
Subscribers: mgorny, #sanitizers, llvm-commits
Differential Revision: https://reviews.llvm.org/D41128
git-svn-id: https://llvm.org/svn/llvm-project/compiler-rt/trunk@320611 91177308-0d34-0410-b5e6-96231b3b80d8
|
|
Summary:
See D40657 & D40679 for previous versions of this patch & description.
A couple of things were fixed here to have it not break some bots.
Weak symbols can't be used with `SANITIZER_GO` so the previous version was
breakin TsanGo. I set up some additional local tests and those pass now.
I changed the workaround for the glibc vDSO issue: `__progname` is initialized
after the vDSO and is actually public and of known type, unlike
`__vdso_clock_gettime`. This works better, and with all compilers.
The rest is the same.
Reviewers: alekseyshl
Reviewed By: alekseyshl
Subscribers: srhines, kubamracek, krytarowski, llvm-commits, #sanitizers
Differential Revision: https://reviews.llvm.org/D41121
git-svn-id: https://llvm.org/svn/llvm-project/compiler-rt/trunk@320594 91177308-0d34-0410-b5e6-96231b3b80d8
|
|
Summary:
getScudoChunk function is implicitly inlined for optimized builds on
clang, but not on gcc. It's a small enough function that it seems
sensible enough to just inline it by default.
Reviewers: cryptoad, alekseyshl
Reviewed By: cryptoad
Differential Revision: https://reviews.llvm.org/D41138
git-svn-id: https://llvm.org/svn/llvm-project/compiler-rt/trunk@320592 91177308-0d34-0410-b5e6-96231b3b80d8
|
|
Summary: D40679 broke a couple of builds, reverting while investigating.
Reviewers: alekseyshl
Reviewed By: alekseyshl
Subscribers: srhines, kubamracek, krytarowski, llvm-commits, #sanitizers
Differential Revision: https://reviews.llvm.org/D41088
git-svn-id: https://llvm.org/svn/llvm-project/compiler-rt/trunk@320417 91177308-0d34-0410-b5e6-96231b3b80d8
|
|
allocator [redo]
Summary:
Redo of D40657, which had the initial discussion. The initial code had to move
into a libcdep file, and things had to be shuffled accordingly.
`NanoTime` is a time sink when checking whether or not to release memory to
the OS. While reducing the amount of calls to said function is in the works,
another solution that was found to be beneficial was to use a timing function
that can leverage the vDSO.
We hit a couple of snags along the way, like the fact that the glibc crashes
when clock_gettime is called from a preinit_array, or the fact that
`__vdso_clock_gettime` is mangled (for security purposes) and can't be used
directly, and also that clock_gettime can be intercepted.
The proposed solution takes care of all this as far as I can tell, and
significantly improve performances and some Scudo load tests with memory
reclaiming enabled.
@mcgrathr: please feel free to follow up on
https://reviews.llvm.org/D40657#940857 here. I posted a reply at
https://reviews.llvm.org/D40657#940974.
Reviewers: alekseyshl, krytarowski, flowerhack, mcgrathr, kubamracek
Reviewed By: alekseyshl, krytarowski
Subscribers: #sanitizers, mcgrathr, srhines, llvm-commits, kubamracek
Differential Revision: https://reviews.llvm.org/D40679
git-svn-id: https://llvm.org/svn/llvm-project/compiler-rt/trunk@320409 91177308-0d34-0410-b5e6-96231b3b80d8
|
|
Summary:
It looks like clang was generating somewhat weird assembly with the current
code. `FromPrimary`, even though `const`, was replaced every time with the code
generated for `size <= SizeClassMap::kMaxSize` instead of using a variable or
register, and `FromPrimary` didn't induce `ClassId != 0` for the compiler, so a
dead branch was generated for `getActuallyAllocatedSize(Ptr, ClassId)` since
it's never called for `ClassId = 0` (Secondary backed allocations) [this one
was more wishful thinking on my side than anything else].
I rearranged the code bit so that the generated assembly is less clunky.
Also changed 2 whitespace inconsistencies that were bothering me.
Reviewers: alekseyshl, flowerhack
Reviewed By: flowerhack
Subscribers: llvm-commits, #sanitizers
Differential Revision: https://reviews.llvm.org/D40976
git-svn-id: https://llvm.org/svn/llvm-project/compiler-rt/trunk@320160 91177308-0d34-0410-b5e6-96231b3b80d8
|
|
Summary:
This wasn't noticed: `RoundUpTo` doesn't produce a constant expression, so the
sizes were not constant either. Enforce them to be static const, replace
`RoundUpTo` by its expression. The compiler can now optimize the associated
computations accordingly.
Also looking at the produced assembly, `PageSize` was fetched multiple times
during `Allocate`, so keep a local value of it. As a result it's fetched once
and kept in a register.
Reviewers: alekseyshl, flowerhack
Reviewed By: alekseyshl
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D40862
git-svn-id: https://llvm.org/svn/llvm-project/compiler-rt/trunk@319903 91177308-0d34-0410-b5e6-96231b3b80d8
|
|
Summary:
It was deemed that the salt in the chunk header didn't improve security
significantly (and could actually decrease it). The initial idea was that the
same chunk would different headers on different allocations, allowing for less
predictability. The issue is that gathering the same chunk header with different
salts can give information about the other "secrets" (cookie, pointer), and that
if an attacker leaks a header, they can reuse it anyway for that same chunk
anyway since we don't enforce the salt value.
So we get rid of the salt in the header. This means we also get rid of the
thread local Prng, and that we don't need a global Prng anymore as well. This
makes everything faster.
We reuse those 8 bits to store the `ClassId` of a chunk now (0 for a secondary
based allocation). This way, we get some additional speed gains:
- `ClassId` is computed outside of the locked block;
- `getActuallyAllocatedSize` doesn't need the `GetSizeClass` call;
- same for `deallocatePrimary`;
We add a sanity check at init for this new field (all sanity checks are moved
in their own function, `init` was getting crowded).
Reviewers: alekseyshl, flowerhack
Reviewed By: alekseyshl
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D40796
git-svn-id: https://llvm.org/svn/llvm-project/compiler-rt/trunk@319791 91177308-0d34-0410-b5e6-96231b3b80d8
|
|
Summary:
With this change, we allow someone to chose the `SizeClassMap` they want to use
at compile time via a define.
I feel somewhat unimaginative with the name of the defines, so if someone has a
better idea, let me know. I have been alternating between those and
`SCUDO_USE_xxx_SIZECLASSMAP` which is clearer but also longer. The issue with
those is that it wouldn't be consistent with `SCUDO_TSD_EXCLUSIVE` that should
probably become `SCUDO_USE_EXCLUSIVE_TSD` maybe?
Anyway, naming is hard, and I am not sure what makes more sense!
Reviewers: alekseyshl, flowerhack
Reviewed By: alekseyshl
Subscribers: llvm-commits, srhines
Differential Revision: https://reviews.llvm.org/D40521
git-svn-id: https://llvm.org/svn/llvm-project/compiler-rt/trunk@319350 91177308-0d34-0410-b5e6-96231b3b80d8
|
|
Summary:
Bionic doesn't initialize its globals early enough. This causes issues when
trying to access them from a preinit_array (b/25751302) or from another
constructor called before the libc one (b/68046352). __progname is initialized
after the other globals, so we can check its value to know if calling
`getauxval` is safe.
Reviewers: alekseyshl
Reviewed By: alekseyshl
Subscribers: srhines, llvm-commits
Differential Revision: https://reviews.llvm.org/D40504
git-svn-id: https://llvm.org/svn/llvm-project/compiler-rt/trunk@319099 91177308-0d34-0410-b5e6-96231b3b80d8
|
|
Summary:
This patch aims at condensing the hardware CRC32 feature detection and making
it slightly more effective on Android.
The following changes are included:
- remove the `CPUFeature` enum, and get rid of one level of nesting of
functions: we only used CRC32, so we just implement and use
`hasHardwareCRC32`;
- allow for a weak `getauxval`: the Android toolchain is compiled at API level
14 for Android ARM, meaning no `getauxval` at compile time, yet we will run
on API level 27+ devices. The `/proc/self/auxv` fallback can work but is
worthless for a process like `init` where the proc filesystem doesn't exist
yet. If a weak `getauxval` doesn't exist, then fallback.
- couple of extra corrections.
Reviewers: alekseyshl
Reviewed By: alekseyshl
Subscribers: kubamracek, aemerson, srhines, kristof.beyls, llvm-commits
Differential Revision: https://reviews.llvm.org/D40322
git-svn-id: https://llvm.org/svn/llvm-project/compiler-rt/trunk@318859 91177308-0d34-0410-b5e6-96231b3b80d8
|
|
Summary:
This change allows Fuchsia to boot properly using the Scudo allocator.
A first version of this commit was reverted by rL317834 because it broke Android
builds for toolchains generated with older NDKs. This commit introduces a
fall back to solve that issue.
Reviewers: cryptoad, krytarowski, rnk, alekseyshl
Reviewed By: cryptoad, krytarowski, alekseyshl
Subscribers: llvm-commits, srhines, kubamracek, krytarowski
Differential Revision: https://reviews.llvm.org/D40121
git-svn-id: https://llvm.org/svn/llvm-project/compiler-rt/trunk@318802 91177308-0d34-0410-b5e6-96231b3b80d8
|
|
Summary:
This implements an opportunistic check for the RSS limit.
For ASan, this was implemented thanks to a background thread checking the
current RSS vs the set limit every 100ms. This was deemed problematic for Scudo
due to potential Android concerns (Zygote as pointed out by Aleksey) as well as
the general inconvenience of having a permanent background thread.
If a limit (soft or hard) is specified, we will attempt to update the RSS limit
status (exceeded or not) every 100ms. This is done in an opportunistic way: if
we can update it, we do it, if not we return the current status, mostly because
we don't need it to be fully consistent (it's done every 100ms anyway). If the
limit is exceeded `allocate` will act as if OOM for a soft limit, or just die
for a hard limit.
We use the `common_flags()`'s `hard_rss_limit_mb` & `soft_rss_limit_mb` for
configuration of the limits.
Reviewers: alekseyshl
Reviewed By: alekseyshl
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D40038
git-svn-id: https://llvm.org/svn/llvm-project/compiler-rt/trunk@318301 91177308-0d34-0410-b5e6-96231b3b80d8
|
|
Summary:
This is mostly some cleanup and shouldn't affect functionalities.
Reviewing some code for a future addition, I realized that the complexity of
the initialization path was unnecessary, and so was maintaining a structure
for the allocator options throughout the initialization.
So we get rid of that structure, of an extraneous level of nesting for the
`init` function, and correct a couple of related code inaccuracies in the
flags cpp.
Reviewers: alekseyshl
Reviewed By: alekseyshl
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D39974
git-svn-id: https://llvm.org/svn/llvm-project/compiler-rt/trunk@318157 91177308-0d34-0410-b5e6-96231b3b80d8
|
