summaryrefslogtreecommitdiff
path: root/utils/lit
AgeCommit message (Collapse)Author
2017-12-05Disable detect_leaks in the ASanified build of LLVM when using Apple LLVM. ↵Kuba Mracek
The released Apple LLVM versions don't support LSan. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@319738 91177308-0d34-0410-b5e6-96231b3b80d8
2017-12-04[NFC][lit] Use proper semantic versioning names for variablesJonas Devlieghere
The variable named `minor` was actually pointing to the patch part of the version. While I was changing this I also made the check for Apple clang more robust by checking both patch and minor rather than just minor. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@319656 91177308-0d34-0410-b5e6-96231b3b80d8
2017-12-01[lit] Don't enable LSan on Darwin for Apple clang 9.0.0Jonas Devlieghere
The latest clang that ships with Xcode (clang 900 or 9.0.0) does not support LSan. This fixes the lit configuration to reflect that. Differential revision: https://reviews.llvm.org/D40672 git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@319530 91177308-0d34-0410-b5e6-96231b3b80d8
2017-12-01[lit] Implement non-pipelined ‘mkdir’, ‘diff’ and ‘rm’ commands ↵Ying Yi
internally Summary: The internal shell already supports 'cd', ‘export’ and ‘echo’ commands. This patch adds implementation of non-pipelined ‘mkdir’, ‘diff’ and ‘rm’ commands as the internal shell builtins. Reviewed by: Zachary Turner, Reid Kleckner Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D39567 git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@319528 91177308-0d34-0410-b5e6-96231b3b80d8
2017-11-29Make check-lit tests respect LLVM_LIT_TOOLS_DIRGreg Bedwell
Differential Revision: https://reviews.llvm.org/D40520 git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@319329 91177308-0d34-0410-b5e6-96231b3b80d8
2017-11-28lit: Bring back -Dtool=xxx feature lost in r313928Matthias Braun
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@319139 91177308-0d34-0410-b5e6-96231b3b80d8
2017-11-23Reverted rL318911 since it broke the sanitizer-windows.Ying Yi
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@318914 91177308-0d34-0410-b5e6-96231b3b80d8
2017-11-23[lit] Implement non-pipelined ‘mkdir’, ‘diff’ and ‘rm’ commands ↵Ying Yi
internally Summary: The internal shell already supports 'cd', ‘export’ and ‘echo’ commands. This patch adds implementation of non-pipelined ‘mkdir’, ‘diff’ and ‘rm’ commands as the internal shell builtins. Reviewers: Zachary Turner, Reid Kleckner Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D39567 git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@318911 91177308-0d34-0410-b5e6-96231b3b80d8
2017-11-21Re-revert "Refactor debuginfo-tests."Zachary Turner
This is still breaking greendragon. At this point I give up until someone can fix the greendragon bots, and I will probably abandon this effort in favor of using a private github repository. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@318722 91177308-0d34-0410-b5e6-96231b3b80d8
2017-11-20Resubmit "Refactor debuginfo-tests" again.Zachary Turner
This was reverted due to the tests being run twice on some build bots. Each run had a slightly different configuration due to the way in which it was being invoked. This fixes the problem (albeit in a somewhat hacky way). Hopefully in the future we can get rid of the workflow of running debuginfo-tests as part of clang, and then this hack can go away. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@318697 91177308-0d34-0410-b5e6-96231b3b80d8
2017-11-19[LIT] Fix testing out-of-tree Clang buildsEric Fiselier
Summary: Currently, LIT configures the LLVM binary path before the Clang binary path. However this breaks testing out-of-tree Clang builds (where the LLVM binary path includes a copy of Clang). This patch reverses the order of the paths when looking for Clang, putting the Clang binary directory first. Reviewers: zturner, beanz, chapuni, modocache, EricWF Reviewed By: EricWF Subscribers: mgorny, cfe-commits, llvm-commits Differential Revision: https://reviews.llvm.org/D40217 git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@318607 91177308-0d34-0410-b5e6-96231b3b80d8
2017-11-17[lit] Try to improve Ctrl-C behavior on WindowsReid Kleckner
This functionality was broken during a refactor a while back because 'pool' is no longer in scope. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@318572 91177308-0d34-0410-b5e6-96231b3b80d8
2017-11-17Fix an issue with llvm lit tool substitutions.Zachary Turner
When using an installed clang with an in-tree llvm, we were not searching in the right paths for the tools. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@318564 91177308-0d34-0410-b5e6-96231b3b80d8
2017-11-17Re-revert "Refactor debuginfo-tests"Zachary Turner
This is still broken because it causes certain tests to be run twice with slightly different configurations, which is wrong in some cases. You can observe this by running: ninja -nv check-all | grep debuginfo-tests And seeing that it passes clang/test and clang/test/debuginfo-tests to lit, which causes it to run debuginfo-tests twice. The fix is going to involve either: a) figuring out that we're running in this "deprecated" configuration, and then deleting the clang/test/debuginfo-tests path, which should cause it to behave identically to before, or: b) make lit smart enough that it doesn't descend into a sub-suite if that sub-suite already has a lit.cfg file. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@318486 91177308-0d34-0410-b5e6-96231b3b80d8
2017-11-16Resubmit "Refactor debuginfo-tests"Zachary Turner
This was reverted due to some failures on specific darwin buildbots, the issue being that the new lit configuration was not setting the SDKROOT environment variable. We've tested a fix locally and confirmed that it works, so this patch resubmits everything with the fix applied. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@318435 91177308-0d34-0410-b5e6-96231b3b80d8
2017-11-13Revert "Update test_debuginfo.pl script to point to new tree location."Zachary Turner
This reverts the aforementioned patch and 2 subsequent follow-ups, as some buildbots are still failing 2 tests because of it. Investigation is ongoing into the cause of the failures. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@318112 91177308-0d34-0410-b5e6-96231b3b80d8
2017-11-10[debuginfo-tests] Make debuginfo-tests work in a standard configuration.Zachary Turner
Previously, debuginfo-tests was expected to be checked out into clang/test and then the tests would automatically run as part of check-clang. This is not a standard workflow for handling external projects, and it brings with it some serious drawbacks such as the inability to depend on things other than clang, which we will need going forward. The goal of this patch is to migrate towards a more standard workflow. To ease the transition for build bot maintainers, this patch tries not to break the existing workflow, but instead simply deprecate it to give maintainers a chance to update the build infrastructure. Differential Revision: https://reviews.llvm.org/D39605 git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@317925 91177308-0d34-0410-b5e6-96231b3b80d8
2017-11-01Add system-linux to allow tests run with llvm-lit to restrict themselves to ↵Jake Ehrlich
linux I need a test that only runs in a reasonable amount of time on systems that have sparse files. The broadest class of systems that support sparse files are linux systems. So restricting my test to linux systems should suffice. This change adds the system-linux feature to llvm-lit so that it can be required. Differential Revision: https://reviews.llvm.org/D39482 git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@317055 91177308-0d34-0410-b5e6-96231b3b80d8
2017-10-17lit: Improve %: normalization.Peter Collingbourne
The new scheme should match the normalization of embedded paths in linkrepro tar files. Differential Revision: https://reviews.llvm.org/D39023 git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@316044 91177308-0d34-0410-b5e6-96231b3b80d8
2017-10-14Revert "lit.py: Add new %{shared_output(LABEL)} substitution"Jordan Rose
This reverts r315697 and my ill-fated attempts to fix it on Windows. I'll try again when I get access to a Windows machine. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@315793 91177308-0d34-0410-b5e6-96231b3b80d8
2017-10-14lit.py: Give up and disable the new shared-output.py test on WindowsJordan Rose
"No such file or directory: C:\\...\\tests\\Output\\shared-output.py.tmp/Output/Shared/SHARED.tmp" And yet other forward-slashes don't seem to be causing the same problem. I'll see if I can get ahold of a Windows machine to poke at this directly later. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@315792 91177308-0d34-0410-b5e6-96231b3b80d8
2017-10-14lit.py: Previous test fix was a red herring; backslashes are escapesJordan Rose
I don't have access to a Windows machine at the moment, so if this doesn't fix it I'll just revert for now. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@315782 91177308-0d34-0410-b5e6-96231b3b80d8
2017-10-14lit.py: Fix new test harder for systems that don't use / as os.path.sepJordan Rose
I didn't think about '%{inputs}' having the same problem. This one should be a fully Windows path name. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@315779 91177308-0d34-0410-b5e6-96231b3b80d8
2017-10-13lit.py: Fix new test for systems that don't use / as os.path.sepJordan Rose
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@315773 91177308-0d34-0410-b5e6-96231b3b80d8
2017-10-13lit.py: Add new %{shared_output(LABEL)} substitutionJordan Rose
This refers to a temporary path that can be shared across all tests, identified by a particular label. This can be used for things like caches. At the moment, the character set for the LABEL is limited to C identifier characters, plus '-', '+', '=', and '.'. This is the same set of characters currently allowed in REQUIRES clause identifiers. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@315697 91177308-0d34-0410-b5e6-96231b3b80d8
2017-10-12[lit] Raise the logic for enabling clang & lld substitutions to llvm.Zachary Turner
This paves the way for other projects which might /use/ clang or lld but not necessarily need to the full set of functionality available to clang and lld tests to be able to have a basic set of substitutions that allow a project to run the clang or lld executables. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@315627 91177308-0d34-0410-b5e6-96231b3b80d8
2017-10-10[lit] Only enable LSan on darwin when clang supports itFrancis Ricci
Summary: LSan on darwin doesn't exist on older versions of clang, causing non-boostrapped sanitized buildbots to fail Reviewers: kubamracek, qcolombet, sqlbyme, zturner, modocache Reviewed By: zturner Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D38703 git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@315333 91177308-0d34-0410-b5e6-96231b3b80d8
2017-10-06[lit] Improve tool substitution in lit.Zachary Turner
This addresses two sources of inconsistency in test configuration files. 1. Substitution boundaries. Previously you would specify a substitution, such as 'lli', and then additionally a set of characters that should fail to match before and after the tool. This was used, for example, so that matches that are parts of full paths would not be replaced. But not all tools did this, and those that did would often re-invent the set of characters themselves, leading to inconsistency. Now, every tool substitution defaults to using a sane set of reasonable defaults and you have to explicitly opt out of it. This actually fixed a few latent bugs that were never being surfaced, but only on accident. 2. There was no standard way for the system to decide how to locate a tool. Sometimes you have an explicit path, sometimes we would search for it and build up a path ourselves, and sometimes we would build up a full command line. Furthermore, there was no standardized way to handle missing tools. Do we warn, fail, ignore, etc? All of this is now encapsulated in the ToolSubst class. You either specify an exact command to run, or an instance of FindTool('<tool-name>') and everything else just works. Furthermore, you can specify an action to take if the tool cannot be resolved. Differential Revision: https://reviews.llvm.org/D38565 git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@315085 91177308-0d34-0410-b5e6-96231b3b80d8
2017-10-06Run pyformat on lit code.Zachary Turner
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@315084 91177308-0d34-0410-b5e6-96231b3b80d8
2017-10-04[test] Fix append_path in the empty caseFrancis Ricci
Summary: normpath() was being called on an empty string and appended to the environment variable in the case where the environment variable was unset. This led to ":." being appended to the path, since normpath() of an empty string is '.', presumably to represent cwd. Reviewers: zturner, sqlbyme, modocache Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D38542 git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@314915 91177308-0d34-0410-b5e6-96231b3b80d8
2017-10-01[lit] Fix running lit tests in unconfigured source dirMichal Gorny
Fix llvm_tools_dir attribute access not to fail when the variable is not present. This directory is not really necessary to run lit tests, and the code already accounts for it being None. The reference was added in r313407, and it breaks the stand-alone lit package in Gentoo. Differential Revision: https://reviews.llvm.org/D38442 git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@314620 91177308-0d34-0410-b5e6-96231b3b80d8
2017-09-29[test] Enable LeakSanitizer on 64-bit Darwin ASan llvm buildsFrancis Ricci
Summary: Also disables leak checking on lto tests, due to many leaks reported in the system's ld64. Reviewers: kcc, pcc, bogner, kubamracek Subscribers: mehdi_amini, llvm-commits Differential Revision: https://reviews.llvm.org/D37781 git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@314535 91177308-0d34-0410-b5e6-96231b3b80d8
2017-09-21Resubmit "[lit] Refactor out some more common lit configuration code."Zachary Turner
There were two issues, one Python 3 specific related to Unicode, and another which is that the tool substitution for lld no longer rejected matches where a / preceded the tool name. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@313928 91177308-0d34-0410-b5e6-96231b3b80d8
2017-09-21Revert "[lit] Refactor out some more common lit configuration code."Zachary Turner
This is breaking several bots. I have enough information to investigate, so I'm reverting to green until I get it figured out. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@313922 91177308-0d34-0410-b5e6-96231b3b80d8
2017-09-21[lit] Refactor out some more common lit configuration code.Zachary Turner
debuginfo-tests has need to reuse a lot of common configuration from clang and lld, and in general it seems like all of the projects which are tightly coupled (e.g. lld, clang, llvm, lldb, etc) can benefit from knowing about one other. For example, lldb needs to know various things about how to run clang in its test suite. Since there's a lot of common substitutions and operations that need to be shared among projects, sinking this up into LLVM makes sense. In addition, this patch introduces a function add_tool_substitution which handles all the dirty intricacies of matching tool names which was previously copied around the various config files. This is now a simple straightforward interface which is hard to mess up. Differential Revision: https://reviews.llvm.org/D37944 git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@313919 91177308-0d34-0410-b5e6-96231b3b80d8
2017-09-21[lit] Actually do normalize the case of files in the config map.Zachary Turner
This has gone back and forth, but it seems this is necessary after all. realpath is not sufficient because if you have a file named 'C:\foo.txt', then both realpath('c:\foo.txt') and realpath(C:\foo.txt') return the string that was passed to them exactly as is, meaning the case of the drive-letter won't match. The problem before was not that we were normalizing the case of items going into the config map, but rather that we were normalizing the case of something we needed to print. The value that is used to key on the config map should never be printed. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@313918 91177308-0d34-0410-b5e6-96231b3b80d8
2017-09-21[lit] Don't norm case when inserting into the config map.Zachary Turner
This makes all paths lowercase on Windows, which seemed like a good idea at the time, but it means that tests can't properly use FileCheck to match expected path names. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@313889 91177308-0d34-0410-b5e6-96231b3b80d8
2017-09-21[lit] Add a test for the builtin config map.Zachary Turner
Config map is not exposed through the command line, so testing this is somewhat tricky. But basically we need a test that if a custom driver builds a config map and passes it to main, it gets respected. A config map allows config files in the source tree to be mapped to alternate config files in the build tree. This particular test works by having two config files in separate directories, and setting up a config map to have that redirects A/lit.site.cfg to B/altconfig. Then, we print a message in A/lit.site.cfg and B/altconfig and check that we do see the output from B but don't see the output from A. Additionally we test that the test suite specified by A's config map is properly discovered. Differential Revision: https://reviews.llvm.org/D38105 git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@313887 91177308-0d34-0410-b5e6-96231b3b80d8
2017-09-21[lit/Win] Check if a path was found before attempting to use it.David L. Jones
Summary: This appears to break some bots, when getToolsPath fails to find some or all of the tools (for example, an incomplete GnuWin32 installation). Reviewers: zturner, modocache Subscribers: sanjoy, llvm-commits Differential Revision: https://reviews.llvm.org/D38115 git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@313854 91177308-0d34-0410-b5e6-96231b3b80d8
2017-09-21[lit] Make lit support config files with .py extension.Zachary Turner
Many editors and Python-related diagnostics tools such as debuggers break or fail in mysterious ways when python files don't end in .py. This is especially true on Windows, but still exists on other platforms. I don't want to be too heavy handed in changing everything across the board, but I do want to at least *allow* lit configs to have .py extensions. This patch makes the discovery process first look for a config file with a .py extension, and if one is not found, then looks for a config file using the old method. So for existing users, there should be no functional change. Differential Revision: https://reviews.llvm.org/D37838 git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@313849 91177308-0d34-0410-b5e6-96231b3b80d8
2017-09-20[lit] Undo the patch to stop writing pyc files.Zachary Turner
The problems on the bots appear to be resolved and this was determined to not be the culprit. Removing this. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@313807 91177308-0d34-0410-b5e6-96231b3b80d8
2017-09-20[lit] Reverse path list when updating environment vars.Zachary Turner
Bug pointed out by EricWF. This would construct a path where items would be added in the wrong order, potentially leading to using the wrong tools for testing. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@313765 91177308-0d34-0410-b5e6-96231b3b80d8
2017-09-19Make lit stop writing pyc files.Zachary Turner
Many svn-based buildbots seem to be getting stuck continually in tree conflicts due to the output of pyc files. I'm disabling these as a temporary measure in an attempt to get everything stable again. I'll try to remove this code once I understand the problem better. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@313698 91177308-0d34-0410-b5e6-96231b3b80d8
2017-09-18[lit] Use realpath when adding to the config map.Zachary Turner
Since the path a user specifies to the llvm-lit script might be different than the source tree they built from (since they could be behind different symlinks), we need to use realpath to make sure that path comparisons work as expected. Even better would be to use a custom dictionary comparison with actual file equivalence comparison semantics, but this is the least friction to unbreak things for now. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@313594 91177308-0d34-0410-b5e6-96231b3b80d8
2017-09-18Fix inverted regex search.Zachary Turner
I was using the pattern as the source string and vice versa causing strange regular expression errors. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@313590 91177308-0d34-0410-b5e6-96231b3b80d8
2017-09-18[lit] Fix a Python 3 compatibility issue.Zachary Turner
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@313580 91177308-0d34-0410-b5e6-96231b3b80d8
2017-09-18[lit] Update clang and lld to use new config helpers.Zachary Turner
NFC intended here, this only updates clang and lld's lit configs to use some helper functionality in the lit.llvm submodule. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@313579 91177308-0d34-0410-b5e6-96231b3b80d8
2017-09-16Try to fix some failing bots.Zachary Turner
It doesn't make sense to me why these bots are failing as the traceback does not agree with the source code. It's possible something is stale or there is some other mysterious error, but in any case hopefully this fixes it. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@313469 91177308-0d34-0410-b5e6-96231b3b80d8
2017-09-16Resubmit "Add a shared llvm.lit module that all test suites can use."Zachary Turner
There were some issues surrounding Py2 / Py3 compatibility, but I've now tested with both Py2 and Py3 and everything seems to work. git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@313467 91177308-0d34-0410-b5e6-96231b3b80d8
2017-09-16[lit] Fix some Python 3 compatibility issues.Zachary Turner
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@313466 91177308-0d34-0410-b5e6-96231b3b80d8