Unverified Commit 0063b501 authored by Guillaume Communie's avatar Guillaume Communie Committed by GitHub
Browse files

Drill samples refactoring (#32059)

* [DRILL] drill parameter refactoring

* [DRILL] model adds the samples

* [DRILL] refactor DrillSample class

* [DRILL] remove sample id from the parameters

* [DRILL] samples know the controller

* [DRILL] inject controller in parameter

* [DRILL] new sample presenter class

* [DRILL] new parameter presenter class

* [DRILL] create new parameter

* [DRILL] table keeps track of sample presenters

This is needed to avoid suppression of presenter references by the GC.

* [DRILL] new table item class

This item is now used in the drill table.

* [DRILL] connect modification signals in table/item

* [DRILL] propagate parameter changes to model

* [DRILL] avoid loop in item signals

* [DRILL] method to delete a parameter from a sample

* [DRILL] remove param checks from DrillModel

The parameters check themselves. The model does not know about them.

* [DRILL] samples signal their own processing state

* [DRILL] connect sample processing state

* [DRILL] special MVP for the custom options

CustomOptions could contain a list of several parameters. In this
situation, the presenter keeps track of all the parameter names and
rewritte all of them after each edit of the table item.

* [DRILL] use DrillTableItem in DrillTable

* [DRILL] cell background is moved to table items

* [DRILL] get processing parameters from samples

* [DRILL] sample MVP on add row

* [DRILL] samples signal on their own status

* [DRILL] add output name in samples

* [DRILL] samples take care of their own group

* [DRILL] table has to be enabled to get rows

* [DRILL] samples log their own processing status

* [DRILL] custom options in sample only

* [DRILL] useless function

Now, groups are known by samples only.

* [DRILL] model does not know the groups

* [DRILL] model does not know master samples

* [DRILL] model signals new samples

* [DRILL] samples signal on new parameter

* [DRILL] parameters signal on value changed

* [DRILL] inject controller in parameter

* [DRILL] properly remove cell background

* [DRILL] get mantid property info in parameter

* [DRILL] model keeps the processing parameters

* [DRILL] dedicated MVP for settings

* [DRILL] properly import the samples

* [DRILL] properly import the global settings

* [DRILL] properly import the groups

* [DRILL] sample MVPs take care of their data

* [DRILL] properly reset the drill table

* [DRILL] window modified at right moments

* [DRILL] use the correct item

* [DRILL] save numpy array property as python list

* [DRILL] properly save the global settings

* [DRILL] properly save the samples

* [DRILL] properly save the groups

* [DRILL] extract boolean from custom options

* [DRILL] model does know about columns

* [DRILL] delete empty parameters

When a cell is cleared, the corresponding parameter is deleted from the

* [DRILL] window not modified when loading a file

* [DRILL] task signals use string not int

* [DRILL] group index starts at 0

But displayed value starts at 1

* [DRILL] typo

* [DRILL] recompute group index when ungrouping

* [DRILL] new method to add samples to a group

* [DRILL] get group names in context menu

* [DRILL] protect against unsupported instruments

* [DRILL] add method to set the instrument in view

* [DRILL] set instrument when syncing the header

* [DRILL] add cancel button

When the popop asking for data saving appears, there is now a way to
cancel the action.

* [DRILL] propagate cancel action of data save popup

* [DRILL] master state reset when group change

* [DRILL] update sample tests

* [DRILL] add tests for drill parameter

* [DRILL] add drill parameter doc strings

* [DRILL] remove useless function

The cell tooltips are handled by the table items and sample presenters.

* [DRILL] remove useless function

* [DRILL] remove useless functions

* [DRILL] keep private member names coherent

* [DRILL] document _disabled variable

* [DRILL] update drill table tests

* [DRILL] update rundex io tests

* [DRILL] remove useless function

* [DRILL] master samples have to be in a group

* [DRILL] add view getter for the acquisition mode

* [DRILL] synchronize instrument/mode at startup

* [DRILL] column is a view concept

* [DRILL] formatting

* [DRILL] adaptation of model tests

* [DRILL] missing parameter

* [DRILL] adapt controller tests

* [DRILL] adapt view tests

* [DRILL] add a status in sample

The status of the sample informs on its processing state. This status is
updated by the processing engine but also when a parameter is added or

* [DRILL] typo

* [DRILL] check sample validity before processing

* [DRILL] simplification of parameter validation

* [DRILL] avoid signal repetition

* [DRILL] propagate checks on custom options

* [DRILL] delete parameter when empty

* [DRILL] add default empty error message

* [DRILL] increase sleep for more fluidity

* [DRILL] update sample status if not none

* [DRILL] connect to custom option check signals

* [DRILL] remove badly formatted custom options

* [DRILL] keep master sample when changing group

* [DRILL] add a way to unset the master sample

* [DRILL] labels are handled by the table

* [DRILL] row background is handled in DrillTable

* [DRILL] cell contents handled by DrillTable

* [DRILL] presenter needs a reference to the table

* [DRILL] get selected rows from table directly

* [DRILL] get all rows from table directly

* [DRILL] remove background of empty cells

* [DRILL] show only settings in settings dialog

* [DRILL] connect settings validation signals

* [DRILL] unused import

* [DRILL] unused import

* [DRILL] unused import

* [DRILL] parameters are valid by default

* [DRILL] set windowModified flag

* [DRILL] set output ws name from sample index

* [DRILL] update dril table tests

* [DRILL] process group mechanism

* [DRILL] unused variable

* [DRILL] bug when erasing cells

* [DRILL] not None check

* [DRILL] item MVP manage their own modofication

* [DRILL] datachanged emitted when MVP ready

* [DRILL] DrillTableItem::setData emit a signal

* [DRILL] clear columns list before setting it

* [DRILL] load visual settings

* [DRILL] adapt table widget tests

* [DRILL] missing documentation

* [DRILL] update presenter tests

* [DRILL] update model tests

* [DRILL] adapt view tests

* [DRILL] synchronize default visual settings

* [DRILL] adapt drill tests

* [DRILL] adapt sample tests

* [DRILL] unused statement

* [DRILL] exceptions are passed

* [DRILL] unused imports

* [DRILL] avoid too complex warning on load

* [DRILL] unused parameter

* [DRILL] unused import

* [DRILL] mock config

* [DRILL] typo in member name

* [DRILL] textChanegd signal works in system test

* [DRILL] import what is needed

* [DRILL] import what is needed

* [DRILL] update creation date

* [DRILL] update creation date

* [DRILL] use filter

* [DRILL] name consistency

* [DRILL] new sample group class

* [DRILL] group index and master in DrillSampleGroup

* [DRILL] order sample in group

* [DRILL] properly remove sample and update group

* [DRILL] check if the sample group is empty

* [DRILL] group samples with new DrillSampleGroup

* [DRILL] ungroup samples with new DrillSampleGroup

* [DRILL] obtain list of groups from model

* [DRILL] adapt context menu to DrillSampleGroup

* [DRILL) add to group with the new DrillSampleGroup

* [DRILL] update group name when adding a sample

* [DRILL] set master sample with DrillSampleGroup

* [DRILL] update old and new master sample state

* [DRILL] process parameter with DrillSampleGroup

* [DRILL] process group with DrillSampleGroup

* [DRILL] useless function

* [DRILL] adapt model tests with DrillSampleGroup

* [DRILL] adapt sample tests with DrillSampleGroup

* [DRILL] StringArray property type was missing

* [DRILL] convert only 1d array to list

* [DRILL] more readable

* [DRILL] sample group tests

* [DRILL] useless function

* [DRILL] more compact

* [DRILL] delete from group when samples are deleted

* [DRILL] properly save group

* [DRILL] sample groups is a dict

* [DRILL] add a getter for samples in group

* [DRILL] save group in rundex from dict

* [DRILL] load group in drill

* [DRILL] unused variable
parent 77e126ee
......@@ -49,7 +49,7 @@ class DrillProcessSANSTest(systemtesting.MantidSystemTest):
column (int): column index
text (str): string to be written in the cell
columnIndex = self.drill.table.columns.index(column)
columnIndex = self.drill.table._columns.index(column)
y = self.drill.table.rowViewportPosition(row) + 5
x = self.drill.table.columnViewportPosition(columnIndex) + 5
......@@ -472,6 +472,7 @@ public:
InstrumentSelector(QWidget *parent /TransferThis/ = nullptr);
void setTechniques(const QStringList &techniques);
void setCurrentIndex(int);
void setCurrentText(QString);
QString currentText();
int count();
# Mantid Repository : https://github.com/mantidproject/mantid
# Copyright © 2021 ISIS Rutherford Appleton Laboratory UKRI,
# NScD Oak Ridge National Laboratory, European Spallation Source
# & Institut Laue - Langevin
# SPDX - License - Identifier: GPL - 3.0 +
import numpy
from qtpy.QtCore import QObject, Signal
from mantid.kernel import StringPropertyWithValue, BoolPropertyWithValue, \
FloatArrayProperty, IntArrayProperty, \
from mantid.api import FileProperty, MultipleFileProperty, \
WorkspaceGroupProperty, MatrixWorkspaceProperty
class DrillParameter(QObject):
FILE_TYPE = "file"
FILES_TYPE = "files"
WORKSPACE_TYPE = "workspace"
COMBOBOX_TYPE = "combobox"
STRING_TYPE = "string"
BOOL_TYPE = "bool"
FLOAT_ARRAY_TYPE = "floatArray"
INT_ARRAY_TYPE = "intArray"
Name of the parameter.
_name = None
Value of the parameter.
_value = None
Parameter documentation.
_documentation = None
Parameter type.
_type = None
Parameter allowed values.
_allowedValues = None
Reference to the parameter controller.
_controller = None
Validation state of the parameter value.
_valid = True
Error message associated to an invalid state.
_validationErrorMsg = None
Sent when the parameter value has been checked.
checked = Signal()
Sent when the parameter value changed.
valueChanged = Signal()
def __init__(self, name):
self._name = name
self._validationErrorMsg = ""
def setController(self, controller):
Set the parameter controller.
controller (DrillParameterController): controller
self._controller = controller
def initFromProperty(self, mantidProperty):
Inititialize the parameter from a mantid property. This method populates
the type, documentation, allowed values of the parameters. It also sets
the value to the property default value.
mantidProperty (Property): the property
self._documentation = mantidProperty.documentation
self._allowedValues = mantidProperty.allowedValues
value = mantidProperty.value
if (isinstance(mantidProperty, FileProperty)):
self._type = self.FILE_TYPE
elif (isinstance(mantidProperty, MultipleFileProperty)):
self._type = self.FILES_TYPE
if not value:
value = ""
elif (isinstance(mantidProperty, (WorkspaceGroupProperty,
self._type = self.WORKSPACE_TYPE
elif (isinstance(mantidProperty, StringPropertyWithValue)):
if mantidProperty.allowedValues:
self._type = self.COMBOBOX_TYPE
self._type = self.STRING_TYPE
elif (isinstance(mantidProperty, BoolPropertyWithValue)):
self._type = self.BOOL_TYPE
elif (isinstance(mantidProperty, FloatArrayProperty)):
self._type = self.FLOAT_ARRAY_TYPE
elif (isinstance(mantidProperty, IntArrayProperty)):
self._type = self.INT_ARRAY_TYPE
elif (isinstance(mantidProperty, StringArrayProperty)):
self._type = self.STRING_TYPE
self._type = self.STRING_TYPE
if isinstance(value, numpy.ndarray):
if value.ndim == 1:
self._value = value.tolist()
elif value is None:
self._value = ""
self._value = value
def getName(self):
Get the name of the parameter.
name (str): name of the parameter
return self._name
def setValue(self, value):
Set the parameter value. If a controller is available, the value will
be checked.
value (any): value
self._value = value
if self._controller is not None:
def getValue(self):
Get the parameter value.
(any): value
return self._value
def setValidationState(self, valid, msg=""):
Set the parameter validation state.
valid (bool): True if the parameter is valid
msg (str): optional error message
self._valid = valid
self._validationErrorMsg = msg if not valid else None
def isValid(self):
Check if the parameter is valid.
bool: True if the parameter is valid
return self._valid
def getErrorMessage(self):
Get the error message associated with an invalid state.
str: error message
return self._validationErrorMsg
def getType(self):
Get the parameter type.
(str): type
return self._type
def getAllowedValues(self):
Get the allowed value.
(list(str)): allowed values
return self._allowedValues
def getDocumentation(self):
Get the parameter documentation.
(str): documentation
return self._documentation
......@@ -13,76 +13,7 @@ from qtpy.QtCore import QObject, Signal
import mantid.simpleapi as sapi
class DrillParameter:
Class that defines a parameter to be checked.
def __init__(self, name, value, sample):
Create a parameter by giving its name, value and the sample to which it
is associated.
name (str): parameter name
value (str): parameter value
sample (int): associated sample
self._name = name
self._value = value
self._sample = sample
self._errorMsg = str()
def name(self):
Get the parameter name
str: parameter name
return self._name
def value(self):
Get the parameter value
str: parameter value
return self._value
def sample(self):
Get the sample number assiociated with this parameter.
int: sample number
return self._sample
def errorMsg(self):
Get the error message of the parameter if its validation failed.
str: error message
return self._errorMsg
def errorMsg(self, msg):
Set the error message.
msg (str): error message
self._errorMsg = msg
from .DrillParameter import DrillParameter
class DrillControllerSignals(QObject):
......@@ -119,7 +50,7 @@ class DrillParameterController(threading.Thread):
def signals(self):
return self._signals
def addParameter(self, parameter):
def check(self, parameter):
Add a parameter for validation.
......@@ -145,17 +76,18 @@ class DrillParameterController(threading.Thread):
while self._running:
p = self._paramQueue.get(timeout=0.1)
defaultValue = self._alg.getProperty(p.name).getDefault
self._alg.setProperty(p.name, p.value)
pName = p.getName()
pValue = p.getValue()
defaultValue = self._alg.getProperty(pName).getDefault
self._alg.setProperty(pName, pValue)
self._alg.setProperty(p.name, defaultValue)
self._alg.setProperty(pName, defaultValue)
pass # in case of mandatory parameter
except Exception as e:
p.errorMsg = str(e)
p.setValidationState(False, str(e))
except Exception:
......@@ -10,7 +10,6 @@ import json
from mantid.kernel import *
from .configurations import RundexSettings
from .DrillSample import DrillSample
class DrillRundexIO:
......@@ -47,7 +46,7 @@ class DrillRundexIO:
return self._filename
def load(self):
def load(self): # noqa
Import data and set the associated model.
......@@ -88,7 +87,10 @@ class DrillRundexIO:
# global settings
if (RundexSettings.SETTINGS_JSON_KEY in json_data):
settings = json_data[RundexSettings.SETTINGS_JSON_KEY]
parameters = drill.getParameters()
for parameter in parameters:
if parameter.getName() in settings:
logger.warning("No global settings found when importing {0}. "
"Default settings will be used."
......@@ -105,23 +107,28 @@ class DrillRundexIO:
# samples
if ((RundexSettings.SAMPLES_JSON_KEY in json_data)
and (json_data[RundexSettings.SAMPLES_JSON_KEY])):
i = 0
for sampleJson in json_data[RundexSettings.SAMPLES_JSON_KEY]:
# for backward compatibility
if "CustomOptions" in sampleJson:
del sampleJson["CustomOptions"]
sample = DrillSample()
drill.addSample(-1, sample)
sample = drill.addSample(i)
for name, value in sampleJson.items():
param = sample.addParameter(name)
i += 1
logger.warning("No sample found when importing {0}."
# groups
if "SamplesGroups" in json_data and json_data["SamplesGroups"]:
for groupName, indexes in json_data["SamplesGroups"].items():
drill.groupSamples(indexes, groupName)
if "MasterSamples" in json_data and json_data["MasterSamples"]:
for groupName, index in json_data["MasterSamples"].items():
drill.setGroupMaster(index, groupName)
def save(self):
......@@ -145,9 +152,12 @@ class DrillRundexIO:
json_data[RundexSettings.VISUAL_SETTINGS_JSON_KEY] = visualSettings
# global settings
settings = drill.getSettings()
if settings:
json_data[RundexSettings.SETTINGS_JSON_KEY] = settings
parameters = drill.getParameters()
parametersJson = dict()
for parameter in parameters:
parametersJson[parameter.getName()] = parameter.getValue()
if parametersJson:
json_data[RundexSettings.SETTINGS_JSON_KEY] = parametersJson
# export settings
exportModel = drill.getExportModel()
......@@ -162,15 +172,17 @@ class DrillRundexIO:
json_data[RundexSettings.SAMPLES_JSON_KEY] = list()
for sample in samples:
# groups
groups = drill.getSamplesGroups()
groups = dict()
masters = dict()
for groupName, group in drill.getSampleGroups().items():
groups[groupName] = [sample.getIndex() for sample in group.getSamples()]
if group.getMaster() is not None:
masters[groupName] = group.getMaster().getIndex()
if groups:
json_data["SamplesGroups"] = dict()
for k,v in groups.items():
json_data["SamplesGroups"][k] = list(v)
masters = drill.getMasterSamples()
json_data["SamplesGroups"] = groups
if masters:
json_data["MasterSamples"] = masters
......@@ -5,8 +5,29 @@
# & Institut Laue - Langevin
# SPDX - License - Identifier: GPL - 3.0 +
from qtpy.QtCore import QObject, Signal
class DrillSample:
from mantid.kernel import logger
from .DrillParameter import DrillParameter
class DrillSample(QObject):
Status of sample when its processing is done.
STATUS_PROCESSED = "processing_done"
Status of the sample when its processing ended with an error.
STATUS_ERROR = "processing_error"
Status of the sample when its processing is running.
STATUS_PENDING = "processing"
Processing parameters.
......@@ -14,78 +35,266 @@ class DrillSample:
_parameters = None
Name of the output workspace.
Name of the sample.
_name = None
Name used for the output workspace.
_outputName = None
def __init__(self):
Index of the sample.
_index = None
Sample group, if the sample is in a group.
_group = None
_status = None
Controller for the parameters.
_controller = None
Sent if the group changed.
groupChanged = Signal()
Sent when a new parameter is added.
DrillParameter: the new parameter
newParameter = Signal(DrillParameter)
Signals that the status of the sample changed.
statusChanged = Signal()
def __init__(self, index):
Create an empty sample.
self._parameters = dict()
self._index = index
self._group = None
def setParameters(self, parameters):
def setController(self, controller):
Set the processing parameters.
Controller for the parameters.
parameters (dict(str:str)): parameter key:value pairs
controller (DrillParameterController): parameter controller
self._controller = controller
def setIndex(self, index):
self._parameters = {k:v for k,v in parameters.items()}
Set the sample index.
def getParameters(self):
index (int): index of the sample
Get the processing parameters.
self._index = index
def getIndex(self):
Get the sample index.
dict(str:str): parameter key:value pairs
int: index of the sample
return self._index
def setName(self, name):
Set the sample name.
name (str): name of the sample
return {k:v for k,v in self._parameters.items()}
self._name = name
def getName(self):
Get the sample name.
str: name of the sample