Zhang, Yuanpeng (e8aacc2c) at 04 Mar 15:38
Update README.md
Zhang, Yuanpeng (c9045448) at 04 Mar 15:38
Update README.md
The following dataset can be used for testing our implementation,
IPTS-24291,
CeO2_bulk_cryo_6x6_at_10K
.
Launch ADDIE 🠪 right click on table view 🠪 Table
🠪 Import from Database
🠪 Replace ...
Then a login window will pop up where we can use our UCAMS account to log in. I am not sure whether @halgh007 has the access to the example data we are going to use here. If not, we can always sort out a way to do this. Need offline discussion.
After logging in successfully, we can select IPTS-24291 either using the dropdown menu or the input box, followed by clicking on the Import All Runs
button to import all runs.
Then we need to remove all rows except CeO2_bulk_cryo_6x6_at_10K
.
Fill in all necessary information if they are not already there, e.g. background runs. Also, correct improper inputs, e.g., chemical formula.
We can refer to the json file attached below (that we are finally going to prepare) for reference about what proper input should be.
Clicking on the Launch Reduction
button, we need to tell the program to prepare the input json file, as attached below,
ceriaBulk_PAC06_at_10K_IDL_Calib_abs.json
For implementation, we need to first figure out where the
Launch Reduction
button is defined. Then to figure out the unique label for the button which will be used by the internal codes to recognize the button. Then to figure out what (i.e. a function, a method, or whatever) is linked to theLaunch Reduction
button, i.e., we want to know what the internal code is going to execute when theLaunch Reduction
button is clicked.
In terms of implementation, the json file preparation should be packaged up and implemented as a single function. For example, if the json file preparation function is called json_input_prep
, when the Launch Reduction
button is clicked, json_input_prep
will be called.
The updated version of example json file (with all the redundant entries removed, see item-3 above) should be in a one-to-one mapping with the table view in ADDIE.
The Output
and Cache
directory specification can be found by going to File
\rightarrow
Settings
menu.
Buttons in the Reduction Configuration
window is not working, like the two browse
buttons under Grouping
. Also, the refresh buttons in the Q-range and r-range section is not working. I checked the previous version and they seem to be working fine so I suspect it should be something to do with the adding in of the scroll bar.
In the Merging
entry of the json file, there should be a QBinning
entry whose value should be in the format of a list, e.g., [0.0, 0.005, 40.0]. The value should come from the Q-range section in the Reduction configuration
window. The value are Qmin, Qbinning and Qmax, respectively.
Again for the Merging
entry, there is a Grouping
sub-entry where we have Initial
and Output
sub-sub-entries. For the Initial
one, we need to go back to the Reduction Configuration
window - if From calibration file
is selected for Intermediate
, then the value for Initial
should be the full name of the calibration file (see bullet point 4 below). If Browse
is selected and file is specified through browsing, then the Initial
value should be the browsed file name. The same logic applies for the Output
sub-sub-entry.
Here is an example group file that can be imported by browsing, for both initial (i.e.
Intermediate
) and output grouping, outputgrouping.xml
Concerning the calibration file, it can be imported from the Calibration
browse button under the main table view. Here attached is an example calibration file that can be imported, NOM_144974_2020_06_04_shifter_cycle.h5.
This should be fine at this moment - users are expected to have the knowledge of basic workflow.
That's exactly what we want! Thank you!
In addition, if users fill out the reduction configuration tab before actually loading anything into the table, the calibration portion of the JSON file shows up as:
"Calibration": {},
"CacheDir": "./tmp",
"OutputDir": "./output",
"Merging": {
"QBinning": [
"0",
"0.02",
"40"
],
"Grouping": {
"Initial": "/SNS/users/3d0/Downloads/outputgrouping.xml",
"Output": ""
}
},
Is this a bug that needs fixing or just the logical result of users not performing the proper workflow, loading in table first before doing anything else.
In this case Initial
value was selected from browse
and Output
was left as from calibration file
This is the behavior you're seeking in the JSON file right? Calibration file is default grouping value if no file is selected from browse
in the reduction configuration window
{
"Facility": "SNS",
"Instrument": "NOM",
"Title": "CeO2_bulk_cryo_6x6_ramp_to_300K",
"Sample": {
"Runs": "143085",
"Material": "We have one bulk Ce O2 sample, with 300 mg in weight.",
"Density": {
"UseMassDensity": true,
"UseNumberDensity": false,
"UseMass": false
},
"PackingFraction": 1.0,
"Geometry": {
"Shape": "Cylinder"
}
},
"Normalization": {
"Density": {
"UseMassDensity": true,
"UseNumberDensity": false,
"UseMass": false
},
"Geometry": {
"Shape": "Cylinder"
}
},
"Calibration": {
"Filename": "/SNS/users/3d0/Downloads/NOM_144974_2020_06_04_shifter_cycle.h5"
},
"CacheDir": "./tmp",
"OutputDir": "./output",
"Merging": {
"QBinning": [
"0",
"0.02",
"40"
],
"Grouping": {
"Initial": "/SNS/users/3d0/Downloads/outputgrouping.xml",
"Output": "/SNS/users/3d0/Downloads/NOM_144974_2020_06_04_shifter_cycle.h5"
}
},
"AlignAndFocusArgs": {},
"self_scattering_levels": {
"lower": [
20.0,
20.0,
20.0,
30.0,
30.0,
10.0
],
"upper": [
30.0,
25.0,
30.0,
40.0,
40.0,
15.0
]
}
}
That's great! Thank you for the updates!
Could you send me the copy of the ui/reduction_configuration_dialog.ui
file in the version of addie where the buttons are working? I used the pyqt designer to generate the .ui
file with the scrollbar and it looks like it removed the majority of the <Connections>
section so the buttons aren't triggering anything. Copy pasting the <Connections>
section back in should fix the issue
Buttons in the Reduction Configuration
window is not working, like the two browse
buttons under Grouping
. Also, the refresh buttons in the Q-range and r-range section is not working. I checked the previous version and they seem to be working fine so I suspect it should be something to do with the adding in of the scroll bar.
In the Merging
entry of the json file, there should be a QBinning
entry whose value should be in the format of a list, e.g., [0.0, 0.005, 40.0]. The value should come from the Q-range section in the Reduction configuration
window. The value are Qmin, Qbinning and Qmax, respectively.
Again for the Merging
entry, there is a Grouping
sub-entry where we have Initial
and Output
sub-sub-entries. For the Initial
one, we need to go back to the Reduction Configuration
window - if From calibration file
is selected for Intermediate
, then the value for Initial
should be the full name of the calibration file (see bullet point 4 below). If Browse
is selected and file is specified through browsing, then the Initial
value should be the browsed file name. The same logic applies for the Output
sub-sub-entry.
Here is an example group file that can be imported by browsing, for both initial (i.e.
Intermediate
) and output grouping, outputgrouping.xml
Concerning the calibration file, it can be imported from the Calibration
browse button under the main table view. Here attached is an example calibration file that can be imported, NOM_144974_2020_06_04_shifter_cycle.h5.
Here are the codes
Thanks a lot for the updates. I will grab this file and run it with mantidtotalscattering
to see whether it works or not. Concerning those boolean values, I think we can keep them in.
{
"Facility": "SNS",
"Instrument": "NOM",
"Title": "CeO2_bulk_cryo_6x6_ramp_to_300K",
"Sample": {
"Runs": "143114,143115,143116,143117,143118,143119,143120,143121",
"Background": {
"Runs": "143230,143231,143232,143233",
"Background": {
"Runs": "143373,143374,143375,143376,143377,143377"
}
},
"Material": "Ce (O)2",
"Density": {
"MassDensity": "7.21",
"UseMassDensity": true,
"UseNumberDensity": false,
"UseMass": false
},
"PackingFraction": 0.7,
"Geometry": {
"Shape": "Cylinder",
"Radius": 0.3,
"Height": 1.8
},
"AbsorptionCorrection": {
"Type": "SampleOnly"
}
},
"Normalization": {
"Runs": "143379,143380,143381,143382,143383,143384",
"Background": {
"Runs": "143373,143374,143375,143376,143377,143377"
},
"Material": "(V)",
"Density": {
"MassDensity": "6.11",
"UseMassDensity": true,
"UseNumberDensity": false,
"UseMass": false
},
"PackingFraction": 1.0,
"Geometry": {
"Shape": "Cylinder",
"Radius": 0.2925,
"Height": 1.8
},
"AbsorptionCorrection": {
"Type": "SampleOnly"
}
},
"Calibration": {},
"CacheDir": "./tmp",
"OutputDir": "./output",
"Merging": {
"Grouping": {
"Initial": "",
"Output": ""
}
},
"AlignAndFocusArgs": {
"TMin": "300",
"TMax": "16799"
},
"self_scattering_levels": {
"lower": [
20.0,
20.0,
20.0,
30.0,
30.0,
10.0
],
"upper": [
30.0,
25.0,
30.0,
40.0,
40.0,
15.0
]
}
}
Here's the current output, Would you like me to delete all boolean values in the JSON file as well or will mantid use those as well?
That's great and thank you so much for the updates! We may want to chat a bit about the current situation and decide what we want to do next. Please feel free to call me on slack to chat. Thank you!
Deleted empty entries in table with new function in processing/mantid/master_table/master_table_exporter.py
def delete_empty_rows(self, dictionary):
del_set = set()
for key in dictionary:
try:
if dictionary[key] in self.NA_list:
del_set.add(key)
if len(dictionary[key]) == 0:
del_set.add(key)
except:
pass
if type(dictionary[key]) is dict:
for sub_key in dictionary[key]:
if dictionary[key][sub_key] in self.NA_list:
del_set.add(key)
try:
if len(dictionary[key]) == 0:
del_set.add(key)
except:
pass
print(del_set)
for entry in del_set:
dictionary.pop(entry)
return dictionary
JSON output working as expected now
That sounds perfect! Thank you so much for the updates!