Manual: add missing test tag descriptions, update text here and there

$ ctest -R test_performance

\end{verbatim}

A list of all tests performed with this version of \apsq is provided in the Appendix~\ref{app:testlist}.

A list of all tests performed with this version of \apsq is provided in Appendix~\ref{app:testlist}.

\section{Test Configurations}

Test configuration files consist of regular \apsq configuration files for a simulation, invoking the desired behavior to be tested.

\begin{description}

  \item[Passing a test] The expression marked with the tag \parameter{#PASS} has to be found in the output in order for the test to pass. If the expression is not found, the test fails.

  \item[Failing a test] If the expression tagged with \parameter{#FAIL} is found in the output, the test fails. If the expression is not found, the test passes.

  \item[Depending on another test] The tag \parameter{#DEPENDS} can be used to indicate dependencies between tests. For example, the module test 09 described below implements such a dependency as it uses the output of module test 08-1 to read data from a previously produced \apsq data file.

  \item[Depending on another test] The tag \parameter{#DEPENDS} can be used to indicate dependencies between tests. For example, module test \parameter{ROOTObjectReader/01-reading} implements such a dependency as it uses the output of module test \parameter{ROOTObjectWriter/01-write} to read data from a previously produced \apsq data file.

  \item[Defining a timeout] For performance tests the runtime of the application is monitored, and the test fails if it exceeds the number of seconds defined using the \parameter{#TIMEOUT} tag.

  \item[Adding additional CLI options] Additional module command line options can be specified for the \parameter{allpix} executable using the \parameter{#OPTION} tag, following the format found in Section~\ref{sec:allpix_executable}. The \parameter{-o} flag will be added automatically. Multiple options can be supplied by repeating the \parameter{#OPTION} tag in the configuration file, only one option per tag is allowed. In exactly the same way options for the detectors can be set as well using the \parameter{#DETOPION} tag, where \parameter{-g} will be added automatically.

  For all other command line options to be passed to the executable, the \parameter{#CLIOPTION} can be used. Here, the complete flag and possible value needs to be passed, e.g.\ \parameter{-j9}.

  \item[Defining a test case label] Tests can be grouped and executed based on labels, e.g.\ for code coverage reports. Labels can be assigned to individual tests using the \parameter{#LABEL} tag.

  \item[Describing the test] Every test should bear a short description of its goal. This descriptive text can be provided via the \parameter{#DESC} tag and is required for every test. CMake will print warnings for every test missing this tag.

  \item[Running scripts] Some tests require additional input which needs to be generated by a script. For this propose, commands to be executed \emph{before} the tests starts can be provided via the \parameter{#BEFORE_SCRIPT} tag, e.g.

  \begin{minted}[frame=single,framesep=3pt,breaklines=true,tabsize=2,linenos]{bash}

  #BEFORE_SCRIPT python @PROJECT_SOURCE_DIR@/etc/scripts/create_deposition_file.py --type a --detector mydetector --events 2 --steps 1 --seed 0

  \end{minted}

  to run a Python script that generates an input file read by the test.

  \item[Requiring external data] Some tests require external data which needs to be downloaded before executing the test. For this purpose, the \parameter{#DATA} tag is available, which can contain file paths which will be set as required files for the test. If registered with CMake's ExternalData module, they will be downloaded automatically.

\end{description}

\section{Interpretation of Pass and Fail Conditions}

Multiple pass or fail conditions can be separated by a semicolon or by adding multiple \parameter{#PASS} or \parameter{#FAIL} expressions.

It should however be noted that test passes or fails \emph{if any of these conditions is met}, i.e.\ the conditions are combined with a logical \parameter{OR}.

At least one pass or one fail conditions must be present in every test.

A noteworthy exception to this are line breaks.

To ease matching of multi-line expressions, the newline escape sequence \parameter{\n} of any test expression is automatically expanded to \parameter{[\r\n\t ]*} to match any new line, carriage return, tab and whitespace characters following the line break.

\section{Warning and Error Messages During Testing}

If no explicit fail conditions are specified, the test will fail if any \parameter{WARNING}, \parameter{ERROR} or \parameter{FATAL} appears in the output log unless it is already part of the pass condition.

For example, if a test is supposed to pass in case of an error provoked