Commit e3a7c771 authored by Jonas Devlieghere's avatar Jonas Devlieghere
Browse files

[lldb/Lit] Change the lldbtest format to behave more like shell test. 

The current lldbtest format has a number of shortcomings, all related to
how we omit information based on why the test fails. For example, a
successful test would print nothing, even when `-a` is passed to lit.
It's not up to the test format to decide whether to print something or
not, that's handled by lit itself. For other test results we would
sometimes print stdout & stderr, but not always, such as when a timeout
was reached or we couldn't parse the dotest output.

This patch changes the lldbtest format and makes it behave more like
lit. We now always print the dotest invocation, the exit code, the
output to stdout & stderr. If you're used to dealing with ShTests in
lit, this will feel all very familiar.

Differential revision: https://reviews.llvm.org/D73384
parent 632ba9fc

lldb/test/API/lldbtest.py

+22 −9
Original line number Diff line number Diff line
@@ -86,33 +86,46 @@ class LLDBTest(TestFormat): 
                shutil.copy(python, copied_python)

            cmd[0] = copied_python



        timeoutInfo = None

        try:

            out, err, exitCode = lit.util.executeCommand(

                cmd,

                env=test.config.environment,

                timeout=litConfig.maxIndividualTestTime)

        except lit.util.ExecuteCommandTimeoutException:

            return (lit.Test.TIMEOUT, 'Reached timeout of {} seconds'.format(

                litConfig.maxIndividualTestTime))

            timeoutInfo = 'Reached timeout of {} seconds'.format(

                litConfig.maxIndividualTestTime)



        output = """Script:\n--\n%s\n--\nExit Code: %d\n""" % (

            ' '.join(cmd), exitCode)

        if timeoutInfo is not None:

            output += """Timeout: %s\n""" % (timeoutInfo,)

        output += "\n"



        if out:

            output += """Command Output (stdout):\n--\n%s\n--\n""" % (out,)

        if err:

            output += """Command Output (stderr):\n--\n%s\n--\n""" % (err,)



        if timeoutInfo:

            return lit.Test.TIMEOUT, output



        if exitCode:

            # Match FAIL but not XFAIL.

            for line in out.splitlines() + err.splitlines():

                if line.startswith('FAIL:'):

                    return lit.Test.FAIL, out + err

                    return lit.Test.FAIL, output



            if 'XPASS:' in out or 'XPASS:' in err:

                return lit.Test.XPASS, out + err

                return lit.Test.XPASS, output



        has_unsupported_tests = 'UNSUPPORTED:' in out or 'UNSUPPORTED:' in err

        has_passing_tests = 'PASS:' in out or 'PASS:' in err

        if has_unsupported_tests and not has_passing_tests:

            return lit.Test.UNSUPPORTED, out + err

            return lit.Test.UNSUPPORTED, output



        passing_test_line = 'RESULT: PASSED'

        if passing_test_line not in out and passing_test_line not in err:

            msg = ('Unable to find %r in dotest output (exit code %d):\n\n%s%s'

                   % (passing_test_line, exitCode, out, err))

            return lit.Test.UNRESOLVED, msg

            return lit.Test.UNRESOLVED, output



        return lit.Test.PASS, ''
 
        return lit.Test.PASS, output