Re #0 Beginning the unscripted testing page

dev-docs/source/Testing/ErrorReporter-ProjectRecovery/ErrorAndRecovery.rst

+.. _error_recovery_testing:
+
+Error Reporter and Project Recovery Testing
+===========================================
+
+.. contents::
+  :local:
+
+Error Reporter test
+-------------------
+
+*Preparation*
+
+-  Ensure that `Report usage data` is checked in the First Time Setup screen
+-  Files ``SANS2D00005512.nxs``
+-  Make sure the location of these files is included in your search
+   directories. Full instructions `are available
+   online <http://www.mantidproject.org/MBC_Getting_set_up#MantidPlot_First-Time_Setup>`__.
+- These tests should be run with someone who has access to the error reports database. Please
+contact Keith Butler when you are going to run these tests.
+
+**Time required 10 - 20  minutes**
+
+--------------
+
+1. Open MantidPlot
+
+- Cause a crash by running the `Segfault` algorithm
+- The error report dialog should pop up
+- Make sure the hyperlink to the privacy policy works
+- Enter some text in the `Name` box; make sure the `Share non-identifiable information` button gets greyed out
+- In the `Name` box enter `Private` and in the `Email` box enter `private`
+- Click the `Don't share any information` box
+
+---------------
+
+2. Open MantidPlot
+
+- Cause a crash by running the `Segfault` algorithm
+- The error report dialog should pop up
+- In the `Name` box enter `Public` and in the `Email` box enter `public`
+- Click the `Yes, share information` box
+
+---------------
+
+3. Open MantidPlot
+
+- Load the file `SANS2D00005512.nxs`
+- Click on the new workspace group, this should give a list of workspaces
+- Right-click one of the workspaces and select `Sample Logs` 
+- This should cause an error box saying Mantid has thrown an unexpected exception
+- Make sure the hyperlink to the privacy policy works
+- Enter some text in the `Name` box; make sure the `Share non-identifiable information` button gets greyed out
+- In the `Name` box enter `Private2` and in the `Email` box enter `private2`
+- Make sure the `Continue` radio button is checked
+- Click the `Don't share any information` box
+- You should be returned to the main Mantid window
+
+---------------
+
+4. Open MantidPlot
+
+- Load the file `SANS2D00005512.nxs`
+- Click on the new workspace group, this should give a list of workspaces
+- Right-click one of the workspaces and select `Sample Logs` 
+- This should cause an error box saying Mantid has thrown an unexpected exception
+- Make sure the `Continue` radio button is checked
+- Click the `Share non-identifiable information` box
+- You should be returned to the main Mantid window
+
+---------------
+
+5. Open MantidPlot
+
+- Load the file `SANS2D00005512.nxs`
+- Click on the new workspace group, this should give a list of workspaces
+- Right-click one of the workspaces and select `Sample Logs` 
+- This should cause an error box saying Mantid has thrown an unexpected exception
+- Make sure the hyperlink to the privacy policy works
+- Enter some text in the `Name` box; make sure the `Share non-identifiable information` button gets greyed out
+- In the `Name` box enter `Public2` and in the `Email` box enter `public2`
+- Make sure the `Continue` radio button is checked
+- Click the `Yes, share information` box
+- You should be returned to the main Mantid window
+
+---------------
+
+6. Open MantidPlot
+
+- Load the file `SANS2D00005512.nxs`
+- Click on the new workspace group, this should give a list of workspaces
+- Right-click one of the workspaces and select `Sample Logs` 
+- This should cause an error box saying Mantid has thrown an unexpected exception
+- Make sure the hyperlink to the privacy policy works
+- Enter some text in the `Name` box; make sure the `Share non-identifiable information` button gets greyed out
+- In the `Name` box enter `Public3` and in the `Email` box enter `public3`
+- Make sure the `Terminate` radio button is checked
+- Click the `Yes, share information` box
+- Mantid should shut down
+
+---------------
+
+Project Recovery test
+---------------------
+
+*Preparation*
+
+
+**Time required XX - YY  minutes**
+
+--------------

dev-docs/source/Testing/MuonInterface/MuonTesting.md

+# Muon Testing
+
+
+
+## Ionic Diffusion test
+
+*Preparation*
+
+* Instrument `EMU`
+* Files `51341.nxs`, `51342.nxs` and `51343.nxs`
+* Make sure the location of these files is included in your search directories. Full instructions [are available online](http://www.mantidproject.org/MBC_Getting_set_up#MantidPlot_First-Time_Setup).
+
+**Time required 5 - 10 minutes**
+
+---
+
+1. Make sure that the `.nxs` files are in one of the directories on Mantid's search path.
+* Open Muon Analysis
+* Choose `Machine`: `EMU`
+* In `Load Run` enter `51341` 
+* Under `Settings` tab, ensure the `Enable Multiple Fitting` is checked
+* In `Data Analysis Tab`
+	* Beside `Runs` enter `51341-3` and check `Simultaneous` below
+	* Right click the `Fit Function` table area; select `Add Function`
+	* Add `Background` > `Flat Background`
+	* Add `Muon` > `DynamicKuboToyabe`
+	* Check the `Global` box for: `A0`, `Asym`, `Delta`, `Nu`
+	* Click on the value `0.0000` for `Field` - a `...` box should appear, click it
+		* On the `Set` dropdown, select `Fix All`
+		* Enter values of `0`, `5`, `10` for the three runs
+	* Set `A0 = 0.05`, `Asym = 0.15`, `Delta = 0.2`, `Nu = 0.1`
+	* Click `Fit` > `Fit`
+* In the `Results Table` tab
+	* Under `Fitting Results` check the `Simultaneous` button
+	* Click `Create Table`
+	* A table should appear in the main Mantid work area
+* In the `Results Table` workspace
+	* Expected values (similar to within 50%) ..
+	* f0.A0: 0.18235938124844153
+	* f1.Asym: 0.0050597127151507
+	* f1:Delta: 0.3301445010124604
+	* f1:Nu: 0.8574644977974730220884
+
+* In the workspaces section of the main Mantid window:
+	* Click of the `MuonSimulFit_5141-3` group
+	* Within this there should be three workspaces
+	* Right-click each workspace
+	* Choose `Plot spectrum`
+	* Choose `1D plot` and click `Plot All`
+	* The results should look like the image below:
+	
+![CuResult.png](CuResult.png)
+
+	
+
+
+#### Possible problems
+
+* Loading local data for 51341-3; these may not be in recognised directories - make sure that this is the case.
+
+
+## Superconducting Copper Test
+
+*Preparation*
+
+* Files `EMU00020882.nxs` -- `EMU00020900.nxs`
+* Make sure the location of these files is included in your search directories
+
+**Time required 5 - 10 minutes**
+
+---
+
+1. Make sure that the `.nxs` files are in one of the directories on Mantid's search path.
+* Open Muon Analysis
+* Choose `Machine`: `EMU`
+* In `Load Run` enter `20882`
+* Click the `>` tab next to `Load Run` a few times, this should produce new plots each time
+* Under `Settings` tab, ensure the `Enable Multiple Fitting` is un-checked
+* After loading `20884` switch to the `Data Analysis` tab
+* Right click under `Property` and select `Add Function`
+* Choose `Muon` > `ExpDecayMuon`
+* Click `Fit` > `Fit`
+* Click `Fit` > `Sequential Fit`
+* Beside `Runs` enter `20889-20900`
+* Click `start`
+* Close the `Sequential Fits` window
+* Switch to the `Results Table` tab
+* In the `Fitting Results` table check the `Sequential` radio button
+* In the `Values` table check `Run Number` and `Field_Danfysik`
+* Ensure that all runs are checked in the `Fitting Results` table
+* Click `Create Table`
+* In the main Mantid GUI select the `ResultsTable` in the `Workspaces` tab, this is the one you just created
+* Right click and select `Convert to MatrixWorkspace`
+* In the resulting dialog select `Column X` as `Field_Danfysik` and `Column Y` as `lambda`
+* Click `Run`
+* Right click the new workspace created in the `Workspaces` tab
+* Select `Plot Spectrum`
+* You should get something like the following plot:
+![](Cu-fitting.png)
+
+## HIFI Transverse Field Simultaneous Fitting
+
+*Preparation*
+
+* Files `HIFI00134028` -- `HIFI00134039.nxs`
+* Make sure the location of these files is included in your search directories
+* Make sure that the instrument in the Muon Interface is set to `HIFI`
+
+**Time required 5 - 10 minutes**
+
+---
+
+1. Open the Muon interface, load the first file `HIFI00134028`
+* Using the `>` to the right of the `Load Data` box, step through to `134034`
+* Go to the `Grouping Options` tab
+* Click `Guess Alpha` - in the `Pair Table` box a value of Alpha close to 1.3 should appear
+* Return to `Home` tab
+* Click `>` until all data sets up to `134039` have been loaded
+* In the `Settings` tab ensure that `Enable Multiple Fitting` is checked
+* Go to the `Data Analysis` tab
+* Right-click in the `Fit Functions` box
+* Select `Add function`
+* Choose `Background`>`Flat Background`
+* Right-click again and `Add function`
+* Choose `Muon`>`ExpDecayOsc`
+* In the `Fit Function` window select all parameters except `Frequency` to be `Global`
+* In the `Runs` box enter `134028-39`
+* Check the `Simultaneous` radio button
+* Select `Fit`>`Fit`
+* After some time the fir should fail to converge, with a large `Chi` value (>100) reported in the `Fit Function` box.
+* So we want to re-start to see if the fit works a different way
+* In the main Mantid `Workspaces` dialog select all the workspaces and delete them
+* Back in the Muon interface load up `HIFI00134034`
+* Go to `Grouping Options` and select `Guess Alpha`
+* Go to the `Data Analysis` tab
+* In the `Runs` box enter `134028-39`
+* Check the `Simultaneous` radio button
+* Add the same functions as above (`Flat Background` and `ExpDecayOsc`)
+* In the `Fit Function` window select all parameters except `Frequency` to be `Global`
+* Click on the value of the `Frequency` parameter; a `...` box should appear, click it
+* Enter values for each of the rows in the table that opens, values from `0.0 - 1.1` in steps of `0.1`
+* Click `OK`
+* Select `Fit`>`Fit`
+* You should now get a significantly lower `Chi-sq` value (<10)
+* In the main Mantid window
+* Click on the `MuonSimulFit_134028-39`
+* Scroll to the workspaces
+* Right-click on one, choose `Plot spectrum`
+* Choose `1D Plot` and select `Plot All`
+* You should get some plots like this:
+
+![HIFI-TF-Result.png](HIFI-TF-Result.png)
+
+## Transverse Field Asymmetry Test
+
+*Preparation*
+
+* Files `MUSR00062260.nxs `
+* Make sure the location of these files is included in your search directories
+* Make sure that the instrument in the Muon Interface is set to `MUSR`
+
+
+**Time required 5 - 10 minutes**
+
+---
+
+1. Open the Muon Data Analysis interface
+* Go to the home tab and select instrument MUSR
+* Ensure that you have access to the archive
+* Load run `62260`
+* * Add a “Flat Background” and “GausOsc” functions
+* In the Data Analysis tab “TF Asymmetry Mode” should be ticked
+* Set the “Groups/Pairs to fit” to “Custom”
+* A pop-up should appear and make sure that only “fwd” is ticked
+* Close the pop-up
+* Run a fit
+
+

dev-docs/source/Testing/MuonInterface/MuonTesting.rst

+.. _muon_testing:
+
+Muon Testing
+============
+
+.. contents::
+  :local:
+
+Ionic Diffusion test
+--------------------
+
+*Preparation*
+
+-  Instrument ``EMU``
+-  Files ``51341.nxs``, ``51342.nxs`` and ``51343.nxs``
+-  Make sure the location of these files is included in your search
+   directories. Full instructions `are available
+   online <http://www.mantidproject.org/MBC_Getting_set_up#MantidPlot_First-Time_Setup>`__.
+
+**Time required 5 - 10 minutes**
+
+--------------
+
+1. Make sure that the ``.nxs`` files are in one of the directories on
+   Mantid’s search path.
+
+-  Open Muon Analysis
+-  Choose ``Machine``: ``EMU``
+-  In ``Load Run`` enter ``51341``
+-  Under ``Settings`` tab, ensure the ``Enable Multiple Fitting`` is
+   checked
+-  In ``Data Analysis Tab``
+
+   -  Beside ``Runs`` enter ``51341-3`` and check ``Simultaneous`` below
+   -  Right click the ``Fit Function`` table area; select
+      ``Add Function``
+   -  Add ``Background`` > ``Flat Background``
+   -  Add ``Muon`` > ``DynamicKuboToyabe``
+   -  Check the ``Global`` box for: ``A0``, ``Asym``, ``Delta``, ``Nu``
+   -  Click on the value ``0.0000`` for ``Field`` - a ``...`` box should
+      appear, click it
+
+      -  On the ``Set`` dropdown, select ``Fix All``
+      -  Enter values of ``0``, ``5``, ``10`` for the three runs
+
+   -  Set ``A0 = 0.05``, ``Asym = 0.15``, ``Delta = 0.2``, ``Nu = 0.1``
+   -  Click ``Fit`` > ``Fit``
+
+-  In the ``Results Table`` tab
+
+   -  Under ``Fitting Results`` check the ``Simultaneous`` button
+   -  Click ``Create Table``
+   -  A table should appear in the main Mantid work area
+
+-  In the ``Results Table`` workspace
+
+   -  Expected values (similar to within 50%) ..
+   -  f0.A0: 0.18235938124844153
+   -  f1.Asym: 0.0050597127151507
+   -  f1:Delta: 0.3301445010124604
+   -  f1:Nu: 0.8574644977974730220884
+
+-  In the workspaces section of the main Mantid window:
+
+   -  Click of the ``MuonSimulFit_5141-3`` group
+   -  Within this there should be three workspaces
+   -  Right-click each workspace
+   -  Choose ``Plot spectrum``
+   -  Choose ``1D plot`` and click ``Plot All``
+   -  The results should look like the image below:
+
+.. figure:: ../../images/CuResult.png
+   :alt: CuResult.png
+
+
+Possible problems
+^^^^^^^^^^^^^^^^^
+
+-  Loading local data for 51341-3; these may not be in recognised
+   directories - make sure that this is the case.
+
+Superconducting Copper Test
+---------------------------
+
+*Preparation*
+
+-  Files ``EMU00020882.nxs`` – ``EMU00020900.nxs``
+-  Make sure the location of these files is included in your search
+   directories
+
+**Time required 5 - 10 minutes**
+
+--------------
+
+1. Make sure that the ``.nxs`` files are in one of the directories on
+   Mantid’s search path.
+
+-  Open Muon Analysis
+-  Choose ``Machine``: ``EMU``
+-  In ``Load Run`` enter ``20882``
+-  Click the ``>`` tab next to ``Load Run`` a few times, this should
+   produce new plots each time
+-  Under ``Settings`` tab, ensure the ``Enable Multiple Fitting`` is
+   un-checked
+-  After loading ``20884`` switch to the ``Data Analysis`` tab
+-  Right click under ``Property`` and select ``Add Function``
+-  Choose ``Muon`` > ``ExpDecayMuon``
+-  Click ``Fit`` > ``Fit``
+-  Click ``Fit`` > ``Sequential Fit``
+-  Beside ``Runs`` enter ``20889-20900``
+-  Click ``start``
+-  Close the ``Sequential Fits`` window
+-  Switch to the ``Results Table`` tab
+-  In the ``Fitting Results`` table check the ``Sequential`` radio
+   button
+-  In the ``Values`` table check ``Run Number`` and ``Field_Danfysik``
+-  Ensure that all runs are checked in the ``Fitting Results`` table
+-  Click ``Create Table``
+-  In the main Mantid GUI select the ``ResultsTable`` in the
+   ``Workspaces`` tab, this is the one you just created
+-  Right click and select ``Convert to MatrixWorkspace``
+-  In the resulting dialog select ``Column X`` as ``Field_Danfysik`` and
+   ``Column Y`` as ``lambda``
+-  Click ``Run``
+-  Right click the new workspace created in the ``Workspaces`` tab
+-  Select ``Plot Spectrum``
+-  You should get something like the following plot:
+
+.. figure:: ../../images/Cu-fitting.png
+
+HIFI Transverse Field Simultaneous Fitting
+------------------------------------------
+
+*Preparation*
+
+-  Files ``HIFI00134028`` – ``HIFI00134039.nxs``
+-  Make sure the location of these files is included in your search
+   directories
+-  Make sure that the instrument in the Muon Interface is set to
+   ``HIFI``
+
+**Time required 5 - 10 minutes**
+
+--------------
+
+1. Open the Muon interface, load the first file ``HIFI00134028``
+
+-  Using the ``>`` to the right of the ``Load Data`` box, step through
+   to ``134034``
+-  Go to the ``Grouping Options`` tab
+-  Click ``Guess Alpha`` - in the ``Pair Table`` box a value of Alpha
+   close to 1.3 should appear
+-  Return to ``Home`` tab
+-  Click ``>`` until all data sets up to ``134039`` have been loaded
+-  In the ``Settings`` tab ensure that ``Enable Multiple Fitting`` is
+   checked
+-  Go to the ``Data Analysis`` tab
+-  Right-click in the ``Fit Functions`` box
+-  Select ``Add function``
+-  Choose ``Background``>\ ``Flat Background``
+-  Right-click again and ``Add function``
+-  Choose ``Muon``>\ ``ExpDecayOsc``
+-  In the ``Fit Function`` window select all parameters except
+   ``Frequency`` to be ``Global``
+-  In the ``Runs`` box enter ``134028-39``
+-  Check the ``Simultaneous`` radio button
+-  Select ``Fit``>\ ``Fit``
+-  After some time the fir should fail to converge, with a large ``Chi``
+   value (>100) reported in the ``Fit Function`` box.
+-  So we want to re-start to see if the fit works a different way
+-  In the main Mantid ``Workspaces`` dialog select all the workspaces
+   and delete them
+-  Back in the Muon interface load up ``HIFI00134034``
+-  Go to ``Grouping Options`` and select ``Guess Alpha``
+-  Go to the ``Data Analysis`` tab
+-  In the ``Runs`` box enter ``134028-39``
+-  Check the ``Simultaneous`` radio button
+-  Add the same functions as above (``Flat Background`` and
+   ``ExpDecayOsc``)
+-  In the ``Fit Function`` window select all parameters except
+   ``Frequency`` to be ``Global``
+-  Click on the value of the ``Frequency`` parameter; a ``...`` box
+   should appear, click it
+-  Enter values for each of the rows in the table that opens, values
+   from ``0.0 - 1.1`` in steps of ``0.1``
+-  Click ``OK``
+-  Select ``Fit``>\ ``Fit``
+-  You should now get a significantly lower ``Chi-sq`` value (<10)
+-  In the main Mantid window
+-  Click on the ``MuonSimulFit_134028-39``
+-  Scroll to the workspaces
+-  Right-click on one, choose ``Plot spectrum``
+-  Choose ``1D Plot`` and select ``Plot All``
+-  You should get some plots like this:
+
+.. figure:: ../../images/HIFI-TF-Result.png
+   :alt: HIFI-TF-Result.png
+
+
+Transverse Field Asymmetry Test
+-------------------------------
+
+*Preparation*
+
+-  Files ``MUSR00062260.nxs``
+-  Make sure the location of these files is included in your search
+   directories
+-  Make sure that the instrument in the Muon Interface is set to
+   ``MUSR``
+
+**Time required 5 - 10 minutes**
+
+--------------
+
+1. Open the Muon Data Analysis interface
+
+-  Go to the home tab and select instrument MUSR
+-  Load run ``62260``
+-  
+
+   -  Add a “Flat Background” and “GausOsc” functions
+
+-  In the Data Analysis tab “TF Asymmetry Mode” should be ticked
+-  Set the “Groups/Pairs to fit” to “Custom”
+-  A pop-up should appear and make sure that only “fwd” is ticked
+-  Close the pop-up
+-  Run a fit
+
+

dev-docs/source/Testing/index.rst

+.. _Testing:
+
+=======
+Testing
+=======
+
+This is a series of tests for the different functionalities of Mantid. The tests are designed to 
+try to capture real-life use scenarios as closely as possible, whilst minimising the data and
+time required for each test. Hopefully if Mantid can pass these tests then it will be usable in
+*at least* a basic sense.
+
+When testing, please initially follow the instructions as presented here. After you have given
+a run through with the instructions try to play with the functionality that you are testing. For
+example, give nonsense values, like strings where floats are expected, or negative values
+where they should not be possible. Make sure that Mantid handles all situations gracefully.
+
+When encountering issues during testing, please create issues. The procedure for issue
+creation is outlined in :ref:`issue_tracking`.
+
+.. toctree::
+   :maxdepth: 1
+
+   MuonInterface/MuonTesting
+   ErrorReporter-ProjectRecovery/ErrorAndRecovery
+