| Age | Commit message (Collapse) | Author |
|---|
|
Summary:
This change allows for registration of multiple logging implementations
through a central mechanism in XRay, mapping an implementation to a
"mode". Modes are strings that are used as keys to determine which
implementation to install through a single API. This mechanism allows
users to choose which implementation to install either from the
environment variable 'XRAY_OPTIONS' with the `xray_mode=` flag, or
programmatically using the `__xray_select_mode(...)` function.
Here, we introduce two API functions for the XRay logging:
__xray_log_register_mode(Mode, Impl): Associates an XRayLogImpl to a
string Mode. We can only have one implementation associated with a given
Mode.
__xray_log_select_mode(Mode): Finds the associated Impl for Mode and
installs it as if by calling `__xray_set_log_impl(...)`.
Along with these changes, we also deprecate the xray_naive_log and
xray_fdr_log flags and encourage users to instead use the xray_mode
flag.
Reviewers: kpw, dblaikie, eizan, pelikan
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D40703
git-svn-id: https://llvm.org/svn/llvm-project/compiler-rt/trunk@319759 91177308-0d34-0410-b5e6-96231b3b80d8
|
|
This change is the first in a series of changes to get the XRay runtime
building on macOS. This first allows us to build the minimal parts of
XRay to get us started on supporting macOS development. These include:
- CMake changes to allow targeting x86_64 initially.
- Allowing for building the initialisation routines without
`.preinit_array` support.
- Use __sanitizer::SleepForMillis() to work around the lack of
clock_nanosleep on macOS.
- Deprecate the xray_fdr_log_grace_period_us flag, and introduce
the xray_fdr_log_grace_period_ms flag instead, to use
milliseconds across platforms.
Reviewers: kubamracek
Subscribers: llvm-commits, krytarowski, nglevin, mgorny
Differential Review: https://reviews.llvm.org/D39114
git-svn-id: https://llvm.org/svn/llvm-project/compiler-rt/trunk@319165 91177308-0d34-0410-b5e6-96231b3b80d8
|
|
Summary:
Before this change, the FDR mode implementation relied on at thread-exit
handling to return buffers back to the (global) buffer queue. This
introduces issues with the initialisation of the thread_local objects
which, even through the use of pthread_setspecific(...) may eventually
call into an allocation function. Similar to previous changes in this
line, we're finding that there is a huge potential for deadlocks when
initialising these thread-locals when the memory allocation
implementation is also xray-instrumented.
In this change, we limit the call to pthread_setspecific(...) to provide
a non-null value to associate to the key created with
pthread_key_create(...). While this doesn't completely eliminate the
potential for the deadlock(s), it does allow us to still clean up at
thread exit when we need to. The change is that we don't need to do more
work when starting and ending a thread's lifetime. We also have a test
to make sure that we actually can safely recycle the buffers in case we
end up re-using the buffer(s) available from the queue on multiple
thread entry/exits.
This change cuts across both LLVM and compiler-rt to allow us to update
both the XRay runtime implementation as well as the library support for
loading these new versions of the FDR mode logging. Version 2 of the FDR
logging implementation makes the following changes:
* Introduction of a new 'BufferExtents' metadata record that's outside
of the buffer's contents but are written before the actual buffer.
This data is associated to the Buffer handed out by the BufferQueue
rather than a record that occupies bytes in the actual buffer.
* Removal of the "end of buffer" records. This is in-line with the
changes we described above, to allow for optimistic logging without
explicit record writing at thread exit.
The optimistic logging model operates under the following assumptions:
* Threads writing to the buffers will potentially race with the thread
attempting to flush the log. To avoid this situation from occuring,
we make sure that when we've finalized the logging implementation,
that threads will see this finalization state on the next write, and
either choose to not write records the thread would have written or
write the record(s) in two phases -- first write the record(s), then
update the extents metadata.
* We change the buffer queue implementation so that once it's handed
out a buffer to a thread, that we assume that buffer is marked
"used" to be able to capture partial writes. None of this will be
safe to handle if threads are racing to write the extents records
and the reader thread is attempting to flush the log. The optimism
comes from the finalization routine being required to complete
before we attempt to flush the log.
This is a fairly significant semantics change for the FDR
implementation. This is why we've decided to update the version number
for FDR mode logs. The tools, however, still need to be able to support
older versions of the log until we finally deprecate those earlier
versions.
Reviewers: dblaikie, pelikan, kpw
Subscribers: llvm-commits, hiraditya
Differential Revision: https://reviews.llvm.org/D39526
git-svn-id: https://llvm.org/svn/llvm-project/compiler-rt/trunk@318733 91177308-0d34-0410-b5e6-96231b3b80d8
|
|
The build got broken after D39277 (and rL316816) deleted <cerrno>.
git-svn-id: https://llvm.org/svn/llvm-project/compiler-rt/trunk@316821 91177308-0d34-0410-b5e6-96231b3b80d8
|
|
Summary:
This change removes dependencies on STL types:
- std::aligned_storage -- we're using manually-aligned character
buffers instead for metadata and function records.
- std::tuple -- use a plain old struct instead.
This is an incremental step in removing all STL references from the
compiler-rt implementation of XRay (llvm.org/PR32274).
Reviewers: dblaikie, pelikan, kpw
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D39277
git-svn-id: https://llvm.org/svn/llvm-project/compiler-rt/trunk@316816 91177308-0d34-0410-b5e6-96231b3b80d8
|
|
Summary:
In FDR Mode, when we set up a new buffer for a thread that's just
overflowed, we must place the CPU identifier with the TSC record as the
first record. This is so that we can reconstruct all the function
entry/exit with deltas rooted on a TSC record for the CPU at the
beginning of the buffer.
Without doing this, the tools are rejecting the log for cases when we've
overflown and have different buffers that don't have the CPU and TSC
records as the first entry in the buffers.
Reviewers: pelikan
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D38995
git-svn-id: https://llvm.org/svn/llvm-project/compiler-rt/trunk@315987 91177308-0d34-0410-b5e6-96231b3b80d8
|
|
Summary:
When the XRay user calls the API to finish writing the log, the thread
which is calling the API still hasn't finished and therefore won't get
its trace written. Add a test for only the main thread to check this.
Reviewers: dberris
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D38493
git-svn-id: https://llvm.org/svn/llvm-project/compiler-rt/trunk@314875 91177308-0d34-0410-b5e6-96231b3b80d8
|
|
Summary:
Write out records about logged function call first arguments. D32840
implements the reading of this in llvm-xray.
Reviewers: dberris
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D32844
git-svn-id: https://llvm.org/svn/llvm-project/compiler-rt/trunk@314378 91177308-0d34-0410-b5e6-96231b3b80d8
|
|
handlers
Summary:
Before this change, the recursion guard for the flight data recorder
(FDR) mode handlers were independent. This change makes it so that when
a handler is already in the process of running and somehow the same or
another handler starts running -- say in a signal handler, while the
XRay handler is executing -- then we can use the same thread-local
recursion guard to stop the second handler from running.
Reviewers: kpw, eizan
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D37612
git-svn-id: https://llvm.org/svn/llvm-project/compiler-rt/trunk@312992 91177308-0d34-0410-b5e6-96231b3b80d8
|
|
function.
Summary:
This change hides all the initialization of thread_local variables used
by the XRay FDR mode implementation behind a function call. This makes
initialization of thread-local data to be done lazily, instead of
eagerly when they're done as globals. It also gives us an isolation
mechanism if/when we want to change the TLS implementation from using
the C++ thread_local keyword, for something more ad-hoc (potentialy
using pthread directly) on some platforms or set-ups where we cannot use
the C++ thread_local variables.
Reviewers: kpw, eizan
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D37248
git-svn-id: https://llvm.org/svn/llvm-project/compiler-rt/trunk@311997 91177308-0d34-0410-b5e6-96231b3b80d8
|
|
scope.
Summary:
This change attempts to remove all the dependencies we have on
std::mutex and any std::shared_ptr construction in global variables. We
instead use raw pointers to these objects, and construct them on the
heap. In cases where it's possible, we lazily initialize these pointers.
While we do not have a replacement for std::shared_ptr yet in
compiler-rt, we use this work-around to avoid having to statically
initialize the objects as globals. Subsequent changes should allow us to
completely remove our dependency on std::shared_ptr and instead have our
own implementation of the std::shared_ptr and std::weak_ptr semantics
(or completely rewrite the implementaton to not need these
standard-library provided abstractions).
Reviewers: dblaikie, kpw, pelikan
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D36078
git-svn-id: https://llvm.org/svn/llvm-project/compiler-rt/trunk@309792 91177308-0d34-0410-b5e6-96231b3b80d8
|
|
Summary:
This change implements support for the custom event logging sleds and
intrinsics at runtime. For now it only supports handling the sleds in
x86_64, with the implementations for other architectures stubbed out to
do nothing.
NOTE: Work in progress, uploaded for exposition/exploration purposes.
Depends on D27503, D30018, and D33032.
Reviewers: echristo, javed.absar, timshen
Subscribers: mehdi_amini, nemanjai, llvm-commits
Differential Revision: https://reviews.llvm.org/D30630
git-svn-id: https://llvm.org/svn/llvm-project/compiler-rt/trunk@302857 91177308-0d34-0410-b5e6-96231b3b80d8
|
|
determine the CPU frequency
A problem arises if a machine supports the rdtscp instruction, but the processor
frequency cannot be determined by the function getTSCFrequency(). In this case,
we want to use the emulated TSC instead. This patch implements that by adding a
call to getTSCFrequency() from probeRequiredCPUFeatures(), and the function only
returns true if both the processor supports rdtscp and the CPU frequency can be
determined.
This should fix PR32620.
Reviewers: dberris
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D32067
git-svn-id: https://llvm.org/svn/llvm-project/compiler-rt/trunk@300525 91177308-0d34-0410-b5e6-96231b3b80d8
|
|
git-svn-id: https://llvm.org/svn/llvm-project/compiler-rt/trunk@300041 91177308-0d34-0410-b5e6-96231b3b80d8
|
|
emulation is needed
This patch applies a work-around to the XRay FDR tests when TSC emulation is
needed because the processor frequency cannot be determined.
This fixes PR32620 using the suggestion given by Dean in comment 1.
Reviewers: dberris
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D31967
git-svn-id: https://llvm.org/svn/llvm-project/compiler-rt/trunk@300017 91177308-0d34-0410-b5e6-96231b3b80d8
|
|
xray_fdr_logging.cc
Previously in r297800, a work-around was created to use TSC emulation on x86_64 when RDTSCP was not available on the host. A similar change was needed in the file xray_fdr_logging.cc which this patch ports over to that file.
Eventually the code should be refactored as there will be 3 locations with the same code, but that can be done as a separate step. This patch is just to keep the test from failing on my machine due to an illegal instruction since RDTSCP is not available on my x86_64 linux VM.
Reviewers: dberris
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D31909
git-svn-id: https://llvm.org/svn/llvm-project/compiler-rt/trunk@299922 91177308-0d34-0410-b5e6-96231b3b80d8
|
|
Summary:
Currently the FDR log writer, upon flushing, dumps a sequence of buffers from
its freelist to disk. A reader can read the first buffer up to an EOB record,
but then it is unclear how far ahead to scan to find the next threads traces.
There are a few ways to handle this problem.
1. The reader has externalized knowledge of the buffer size.
2. The size of buffers is in the file header or otherwise encoded in the log.
3. Only write out the portion of the buffer with records. When released, the
buffers are marked with a size.
4. The reader looks for memory that matches a pattern and synchronizes on it.
2 and 3 seem the most flexible and 2 does not rule 3 out.
This is an implementation of 2.
In addition, the function handler for fdr more aggressively checks for
finalization and makes an attempt to release its buffer.
Reviewers: pelikan, dberris
Reviewed By: dberris
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D31384
git-svn-id: https://llvm.org/svn/llvm-project/compiler-rt/trunk@298982 91177308-0d34-0410-b5e6-96231b3b80d8
|
|
Summary:
This change exercises the end-to-end functionality defined in the FDR
logging implementation. We also prepare for being able to run traces
generated by the FDR logging implementation from being analysed with the
llvm-xray command that comes with the LLVM distribution.
This also unblocks D31385, D31384, and D31345.
Reviewers: kpw, pelikan
Subscribers: llvm-commits, mgorny
Differential Revision: https://reviews.llvm.org/D31452
git-svn-id: https://llvm.org/svn/llvm-project/compiler-rt/trunk@298977 91177308-0d34-0410-b5e6-96231b3b80d8
|
|
Instead of std::atomic APIs for atomic operations, we instead use APIs
include with sanitizer_common. This allows us to, at runtime, not have
to depend on potentially dynamically provided implementations of these
atomic operations.
Fixes http://llvm.org/PR32274.
git-svn-id: https://llvm.org/svn/llvm-project/compiler-rt/trunk@298833 91177308-0d34-0410-b5e6-96231b3b80d8
|
|
Summary:
Separated the IO and the thread local storage state machine of logging
from the writing of log records once the contents are deterministic.
Finer granularity functions are provided as inline functions in the same
header such that stack does not grow due to the functions being separated.
An executable utility xray_fdr_log_printer is also implemented to use the
finest granularity functions to produce binary test data in the FDR format
with a relatively convenient text input.
For example, one can take a file with textual contents layed out in rows
and feed it to the binary to generate data that llvm-xray convert can then
read. This is a convenient way to build a test suite for llvm-xray convert
to ensure it's robust to the fdr format.
Example:
$cat myFile.txt
NewBuffer : { time = 2 , Tid=5}
NewCPU : { CPU =1 , TSC = 123}
Function : { FuncId = 5, TSCDelta = 3, EntryType = Entry }
Function : { FuncId = 5, TSCDelta = 5, EntryType = Exit}
TSCWrap : { TSC = 678 }
Function : { FuncId = 6, TSCDelta = 0, EntryType = Entry }
Function : { FuncId = 6, TSCDelta = 50, EntryType = Exit }
EOB : { }
$cat myFile.txt | ./bin/xray_fdr_log_printer > /tmp/binarydata.bin
$./bin/llvm-xray convert -output-format=yaml -output=- /tmp/binarydata.bin
yaml format comes out as expected.
Reviewers: dberris, pelikan
Reviewed By: dberris
Subscribers: llvm-commits, mgorny
Differential Revision: https://reviews.llvm.org/D30850
git-svn-id: https://llvm.org/svn/llvm-project/compiler-rt/trunk@297801 91177308-0d34-0410-b5e6-96231b3b80d8
|
|
Summary:
Functions with the LOG_ARGS_ENTRY sled kind at their beginning will be handled
in a way to (optionally) pass their first call argument to your logging handler.
For practical and performance reasons, only the first argument is supported, and
only up to 64 bits.
Reviewers: javed.absar, dberris
Reviewed By: dberris
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D29703
git-svn-id: https://llvm.org/svn/llvm-project/compiler-rt/trunk@297000 91177308-0d34-0410-b5e6-96231b3b80d8
|
|
Summary:
Use a common definition of a "this variable is unused" annotation for useless
variables only present for their lambda global initializers, to silence gcc's
warning.
Reviewers: dberris
Reviewed By: dberris
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D29860
git-svn-id: https://llvm.org/svn/llvm-project/compiler-rt/trunk@296449 91177308-0d34-0410-b5e6-96231b3b80d8
|
|
GCC has a warning about enum bitfields that cannot be disabled. Do the
ugly thing and go through uint8_t for all the values.
git-svn-id: https://llvm.org/svn/llvm-project/compiler-rt/trunk@295967 91177308-0d34-0410-b5e6-96231b3b80d8
|
|
Fix -Wsign-compare - this might not be quite right, but preserves behavior
git-svn-id: https://llvm.org/svn/llvm-project/compiler-rt/trunk@294868 91177308-0d34-0410-b5e6-96231b3b80d8
|
|
Revert "Fix -Wsign-compare - this might not be quite right, but preserves behavior"
Revert "[XRay] Implement powerpc64le xray."
This reverts commit r294826.
This reverts commit r294781.
git-svn-id: https://llvm.org/svn/llvm-project/compiler-rt/trunk@294842 91177308-0d34-0410-b5e6-96231b3b80d8
|
|
git-svn-id: https://llvm.org/svn/llvm-project/compiler-rt/trunk@294826 91177308-0d34-0410-b5e6-96231b3b80d8
|
|
Summary:
The implementation, however, is in different arch-specific files, unless it's emulated.
Reviewers: dberris, pelikan, javed.absar
Subscribers: aemerson, llvm-commits
Differential Revision: https://reviews.llvm.org/D29796
git-svn-id: https://llvm.org/svn/llvm-project/compiler-rt/trunk@294777 91177308-0d34-0410-b5e6-96231b3b80d8
|
|
logging mode.
Summary:
Fixing a bug I found when testing a reader for the FDR format. Function ID is
now correctly packed into the 28 bits which are documented for it instead of being
masked to all ones.
Reviewers: dberris, pelikan, eugenis
Reviewed By: dberris
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D29698
git-svn-id: https://llvm.org/svn/llvm-project/compiler-rt/trunk@294563 91177308-0d34-0410-b5e6-96231b3b80d8
|
|
Summary:
As pointed out in casual reading of the XRay codebase, that we had
some interesting named functions that didn't quite follow the LLVM coding
conventions.
Reviewers: chandlerc, dblaikie
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D29625
git-svn-id: https://llvm.org/svn/llvm-project/compiler-rt/trunk@294373 91177308-0d34-0410-b5e6-96231b3b80d8
|
|
Summary:
This was pointed out that FDR mode didn't quite put the thread ID in the
buffers, but instead would write down the parent process ID.
Reviewers: pelikan, rSerge
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D29484
git-svn-id: https://llvm.org/svn/llvm-project/compiler-rt/trunk@294166 91177308-0d34-0410-b5e6-96231b3b80d8
|
|
Summary: Change from `compare_exchange_weak()` to `compare_exchange_strong()` where appropriate, because on ARM ( http://lab.llvm.org:8011/builders/clang-cmake-armv7-a15-full/builds/3190 , http://lab.llvm.org:8011/builders/clang-cmake-armv7-a15-full/builds/3191 ) and AArch64 ( http://lab.llvm.org:8011/builders/clang-cmake-aarch64-42vma/builds/3900 ) it fails even in single-threaded scenarios.
Reviewers: dberris, rengolin
Reviewed By: rengolin
Subscribers: aemerson, llvm-commits, iid_iunknown
Differential Revision: https://reviews.llvm.org/D29286
git-svn-id: https://llvm.org/svn/llvm-project/compiler-rt/trunk@293505 91177308-0d34-0410-b5e6-96231b3b80d8
|
|
Summary:
In this change we introduce the notion of a "flight data recorder" mode
for XRay logging, where XRay logs in-memory first, and write out data
on-demand as required (as opposed to the naive implementation that keeps
logging while tracing is "on"). This depends on D26232 where we
implement the core data structure for holding the buffers that threads
will be using to write out records of operation.
This implementation only currently works on x86_64 and depends heavily
on the TSC math to write out smaller records to the inmemory buffers.
Also, this implementation defines two different kinds of records with
different sizes (compared to the current naive implementation): a
MetadataRecord (16 bytes) and a FunctionRecord (8 bytes). MetadataRecord
entries are meant to write out information like the thread ID for which
the metadata record is defined for, whether the execution of a thread
moved to a different CPU, etc. while a FunctionRecord represents the
different kinds of function call entry/exit records we might encounter
in the course of a thread's execution along with a delta from the last
time the logging handler was called.
While this implementation is not exactly what is described in the
original XRay whitepaper, this one gives us an initial implementation
that we can iterate and build upon.
Reviewers: echristo, rSerge, majnemer
Subscribers: mehdi_amini, llvm-commits, mgorny
Differential Revision: https://reviews.llvm.org/D27038
git-svn-id: https://llvm.org/svn/llvm-project/compiler-rt/trunk@293015 91177308-0d34-0410-b5e6-96231b3b80d8
|
|
This reverts rL290852 as it breaks aarch64 and arm.
git-svn-id: https://llvm.org/svn/llvm-project/compiler-rt/trunk@290854 91177308-0d34-0410-b5e6-96231b3b80d8
|
|
Summary:
In this change we introduce the notion of a "flight data recorder" mode
for XRay logging, where XRay logs in-memory first, and write out data
on-demand as required (as opposed to the naive implementation that keeps
logging while tracing is "on"). This depends on D26232 where we
implement the core data structure for holding the buffers that threads
will be using to write out records of operation.
This implementation only currently works on x86_64 and depends heavily
on the TSC math to write out smaller records to the inmemory buffers.
Also, this implementation defines two different kinds of records with
different sizes (compared to the current naive implementation): a
MetadataRecord (16 bytes) and a FunctionRecord (8 bytes). MetadataRecord
entries are meant to write out information like the thread ID for which
the metadata record is defined for, whether the execution of a thread
moved to a different CPU, etc. while a FunctionRecord represents the
different kinds of function call entry/exit records we might encounter
in the course of a thread's execution along with a delta from the last
time the logging handler was called.
While this implementation is not exactly what is described in the
original XRay whitepaper, this one gives us an initial implementation
that we can iterate and build upon.
Reviewers: echristo, rSerge, majnemer
Subscribers: mehdi_amini, llvm-commits, mgorny
Differential Revision: https://reviews.llvm.org/D27038
git-svn-id: https://llvm.org/svn/llvm-project/compiler-rt/trunk@290852 91177308-0d34-0410-b5e6-96231b3b80d8
|
