Danny Hindson authored Nov 27, 2020 and Zhang, Chen committed May 05, 2021

Changes to the Unit class and ConvertUnits algorithm to allow
diffractometer constants (DIFA, DIFC, TZERO) stored in the instrument map
to be used in conversions between d spacing and TOF and v.v.

Rather than add 3 more parameters to lots of methods provided by the Unit
class I have changed some of the methods to take a std:map containing the
optional parameters relevant to only some units. I have included efixed
and delta in this new map

New diffractometerConstants method added to SpectrumInfo and DetectorInfo
to allow retrieval for a particular spectrum including averaging across
detectors. Also manages situations where some of the three constants are
missing.

New algorithm created called ApplyDiffCal to read the diffractometer
constants from a diffraction calibration table workspace and write them
into the instrument parameter map

Add diffractometer constants to the Show Detectors screen for elastic
workspaces

Remove the Diffraction.cpp unit and replace with calls to dSpacing unit class

Add ability to write difa, difc, tzero into a reduced instrument
geometry as part of EditInstrumentGeometry

* Changes to dSpacing to TOF in ConvertUnits

Changes to the Unit class and ConvertUnits algorithm to allow
diffractometer constants (DIFA, DIFC, TZERO) stored in the instrument map
to be used in conversions between d spacing and TOF and v.v.

Rather than add 3 more parameters to lots of methods provided by the Unit
class I have changed some of the methods to take a std:map containing the
optional parameters relevant to only some units. I have included efixed
and delta in this new map

New diffractometerConstants method added to SpectrumInfo and DetectorInfo
to allow retrieval for a particular spectrum including averaging across
detectors. Also manages situations where some of the three constants are
missing.

New algorithm created called ApplyDiffCal to read the diffractometer
constants from a diffraction calibration table workspace and write them
into the instrument parameter map

Add diffractometer constants to the Show Detectors screen for elastic
workspaces

Remove the Diffraction.cpp unit and replace with calls to dSpacing unit class

Add ability to write difa, difc, tzero into a reduced instrument
geometry as part of EditInstrumentGeometry

* Fix unit tests using old unit conversion logic

* Add overload to spectrumInfo::diffractometerConstants

Add overload that doesn't require the 2nd warning dets parameter

* Remove unused variable declarations

* Replace costly call to spectrumInfo() with spectrumInfo object

* Correct incorrect capitalisation in unit name

* Fix bug in CreateDetectorTable

The exception handler in populateTable assumed the column counter
was always at 1. Reset the column counter to 0 by calling colValues.row
and then fill in default values.
Also add if statement to manage diff constants for monitors which
was the specific exception that caused me to find problem with general
exception handler

* Update HRPD system test ref files to accommodate new unit conversion

Update HRPD system test reference files to accommodate changes to the
unit conversions between TOF and dSpacing. As part of this I have updated
the reference file HRPD66031_splined. This is used as an input to the two
system tests that run a focus and it should match the output of the create
vanadium tests but it had got out of sync over the last couple of years
eg changes to Bragg peak stripping didn't result in an update to
HRPD66031_splined. So some changes in HRPD66031_splined that aren't purely
from the unit conversion logic change

* Make EnggFitPeaks use ConvertUnits for focussed ws

* Update engineering diffraction system test ref files

* Update calls to convertSingleFromTOF in spectrum viewer code

* Remove unused variable in spectrum viewer code

* Backout change to Qt4 unit because it causes cppcheck problem

* Various changes

The ParameterMap::diff method has been updated to take an extra
parameter representing a tolerance to be used when comparing double
values between two parameter maps
The overloaded == operator has been modified to call diff with a
zero tolerance to avoid duplicating code
CompareWorkspaces also updated to pass the value of CheckAllData
into diff so diff can stop after first change rather than going
through the whole map
The diff logic has also been speeded up to cope with larger parameter
maps now that diff constants stored (esp for GEM)
Various updates to doc test and system test files
Shorten UnitConversionParameters enum class name
Remove use of EMPTY_DBL() from ConvertUnits::getDetectorValues in
preparation for moving into ExperimentInfo class and also fix
RemoveSpectraTest unit test that relied on this

* Refactor code to retrieve unit related quantities from workspace

* Improve performance of getDetectorValues

* Delete Kernel Diffraction code files

* Delete Kernel Diffraction code files

* Performance improvements to diff constant lookup

* Delete Kernel Diffraction code files

* Performance improvements to diff constant lookup

Restrict diff constant lookup to elastic conversions to or
from d spacing
Just look up diff constants on the specified detector rather
than recursive look through all parents (slow if workspace
doesn't have any diff constants in the pmap at all)
Make the main loop in ConvertUnits parallel (as was the case
in AlignDetectors)

* Replace std::map with std::unordered_map for better performance

* Remove some duplicated code to fetch detector values

* Readd blank line to avoid showing up in diff

* Readd blank line to avoid showing up in diff

* Fix clang format

* Reinstate Efixed from property logic

* Fix failing unit tests

Also defer more decisions on whether missing detector values are
a problem to the code inside the Unit class rather than
experimentInfo::getDetectorValues

* Remove default (empty) value for extra params argument

This will force users to supply the diff constants for dSpacing
conversions rather than assuming supplying L2 and twoTheta is sufficient

* Move all detectors parameters into the map

* Fix Doxygen warnings

* Fix compilation error and expose new unit conv signature to python

* Various changes

Update new case using Unit::Initialize from merge
Reinstate dropped getEFixedForIndirect method
Remove one of the duplicates for the WISH Powder system tests
Also replace duplicate code to retrieve detector info with a call to
getDetectorValues instead
Move getDetectorValues into API::SpectrumInfo

* Fix compilation error in common widgets

* Fix bug in SaveGDA relating to d to TOF spacing conversion

difa and difc were back to front. Rather than swap here I've
changed the code to call the official unit conversion logic
in the dSpacing unit class

* Make momentum transfer (q) unit conversion use diff constants

Had to make minor adjustment to dSpacing::singleFromTOF to avoid
rounding problem on conversion max\min
Also changed qsquared over since this also used v similar formula
Removed epsilon from the check ranges call in unit tests for qsquared
because doesn't seem to be needed any more

* Couples of changes

Modify method of calculating average difc for a spectrum if there's no
calibration present (to avoid changing behaviour for most instruments)
Remove changes to EditInstrumentGeometry to attach diff constants
because can do this using ApplyDiffCal after EditInstrumentGeometry has
been run

* Fix clang format issue

* Fix compilation error on Windows

* Fix failing tests and compilation error

* Fix compilation error from missing include

* Extra way of initialising dSpacing and MomentumTransfer

Enable initialising dSpacing and MomentumTransfer using L2 and two theta
as an alternative to supplying the diff constants. This might be more
user friendly and it also moves the difc formula into the Unit class

Add utility functions to manage extra params map

* Add feature to ApplyDiffCal to clear calibration

* Various changes

Update some more unit tests to cope with changes MomentumTransfer unit
to make it use the diff constants
Improve validation on the L2\ttheta params in dSpacing unit
Allow negative DIFCs in conversions to TOF -
do this by setting up validation on to FromTOF direction only
Revert some files now that I'm doing more "input" averaging in DIFC
calculation on spectra with more than one detector

* Fix failing docs tests

* Rework MomentumTransfer to depend on difc only

* Fix linux compilation error

* Reinstate validation on MomentumTransfer unit class

* Revert some more tests files now that average difc calc reverted

* Revert change to remove local difc calculation so can do in separate PR

* Revert some more enginx system test files

* Revert some WISH system test files

* Revert SNS Powder Reduction ref file

* Revert another SNS ref file

* Update reference file

The changes are due to the StripVanadiumPeaks step in the SNSPowderReduction
algorithm. This algorithm seems v sensitive to even tiny changes in the input
workspace. The input workspace has some v small changes in at the ~15 decimal
place level as a result of this PR which updates the TOF\dSpacing unit conversion.
This causes the fits on some of the V peaks to behave differently and result
in the output having some much bigger differences on a 5-10% level
There is a composite function fit used in StripVanadiumPeaks (in FitPeak to be
precise) that seems a bit unstable and v sensitive to the input data

* Revert conversion from dSpacing to TOF to happen without calibration for ISIS Powder

* Update reference file

The changes are due to the StripVanadiumPeaks step. This algorithm seems very
sensitive to even tiny changes in the input workspace. The workspace that is
input to this step has some v small changes in at the ~15 decimal place level
as a result of this PR which updates the TOF\dSpacing unit conversion. This
causes the fits on some of the V peaks to behave differently and result in the
output having some much bigger differences on a 5-10% level. There is a
composite function fit used in StripVanadiumPeaks (in FitPeak to be precise)
that seems a bit unstable and v sensitive to the input data

* Temporary fix to get Polaris FocusTest to pass

The Polaris FocusTest currently has a couple of problems:
1) the sample empty run it uses looks wrong. 98532 is a V sample run
2) the POLARIS00098532_splined workspace seems to have been generatedthe
using some wrong\old crop ranges. This results in the normalization process
dividing by zero which generates infinities in the output of the test. There
are also some v large values (not quite double max) that get output which makes
setting an absolute tolerance difficult

* Update various ISIS Powder system test files to include masked spectra

* Two changes

Turn off caching in the SNSPowderRedux.PG3Analysis system test so the result of
the system test doesn't depend on the previous job that ran on Jenkins
Load the calibration in from a GSAS file in addition to the other instrument
geometry information so any subsequent unit conversions in Mantid work properly

* Widen tolerance so that Polaris focus system test passes

* Turn off caching in SNSPowderRedux system test

* Make it possible to override calibration to use on focused datasets

This gets the changes from
https://github.com/mantidproject/mantid/pull/30052/ into ornl-next

This gets the changes from
https://github.com/mantidproject/mantid/pull/30052/ into ornl-next

Signed-off-by: Jose Borreguero <borreguero@gmail.com>

Signed-off-by: Jose Borreguero <borreguero@gmail.com>

Runs clang-tidy pass by ref across the codebase to fix-up various
warnings we are seeing in cppcheck

Also updated class_maker.py

Signed-off-by: Harriet Brown <harriet.brown@stfc.ac.uk>

This commit removes all uses of #ifndef in headers in Algorithms and replaces them with pragma once.

Update copyright headers in all files

Squashed commit of the following:

* First version of a script to manage copyright statements

re #23468

* Neatened up script, added comments

re #23488

* move script to tools directory

re #23488

* small script changes and a couple of manual file changes

re #23488

* Minor chnage to whitespace detection in regex

re #23488

* Add an excluded directory

re #23488

* remove a repeasted copyright statement in a file

re #23488

* Don't comsume the comment end if it is on the same line

re #23488

* fix error in new copright parsing

re #23488

* remove double copyrifght entry

re #23488

* Improve handling of old copyrights at the start of comments

re #23488

* remove empty comments re #23488

* exclude gsoapgenerated directories

re #23488

* Sort out greedy line matching re #23488

* improve empty comment removal

re #23488

* improve false positives

re #23488

* impressive speedup by limiting regex matching length

re #23488

* remove evil invisible non ascii character

Also upadte the copyright at the same time

re #23488

* resolve multiple copyrights in a single file

re #23488

* resolve an issue with new statement detection

re #23488

* another unprintable unicode character

re #23488

* pep updates and cmake the new copyright fit clang format

re #23488

* update already done new format headers

re #23488

* wrong type of bracket

re #23488

* Update class_maker and friends

re #23488

* Update all copyright statements

re #23488

* clang format re #23488

* flake8 warnings re #23488

* Flake8 warnings re #23488

* Exclude .cmake.in and rb.in files

re #23488

* replace missing line re #23488

* exclude .py.in files as they are flasely recognized as C++

re #23488

* another setp.py.in re #23488

* another .py.in correction re #23488

* Hopefully the last of the .py.in files re #23488

* resolve utf-8 encoding of python files and changed ABINS checksum

re #23488

* updates to unit tests that reference line numbers

re #23488

* remaining unit test files and other fixes

re #23488

Refs #22286

Some options were in place as maybe they'll be eventually needed.
This removes those and continues the cleanup. Refs #22286

Most of the work was resurrecting old code and renaming loop variables
to clarify what was actually being done. Refs #22286.

# Please enter a commit message to explain why this merge is necessary,
# especially if it merges an updated upstream into a topic branch.
#
# Lines starting with '#' will be ignored, and an empty message aborts
# the commit.

Merge done to enable documentation be be fixed, because of removal of
alias directive and addition of related algorithms directive.

Signed-off-by: Karl Palmen <karl.palmen@stfc.ac.uk>

re #22143

Merge to ensure CrystalFieldFunction is available and up to date.

This creates the starting calibration table using the difc calculated from the signal workspace.

Also select correct parameters when a small number of peaks are found.

It calculates the reduced chi-squared and uses the fit with the lowest value.