Add version 1.0

README.md

+Instructions
+------------
+
+If you are a criticality safety expert who has used ICSBEP benchmarks, please
+help us and participate in this activity! If you are not, please forward this 
+to criticality safety experts in your organization.
+
+Download the sg8-v1.0-feedback-form.xlsx and add to it your ratings, comments,
+and suggestions for improvement for ICSBEP benchmarks you have used. Additional
+instructions are inside the excel sheet.
+
+Send the form to Will Wieselquist (SG-8 lead) at ww5@ornl.gov to be 
+incorporated into this repository.
+
+Forms are due December 5, 2021 to be incorporated in the final report. Each
+form will be placed with last name of the contributor in the feedback directory.
+
+Forms may be updated numerous times. Please don't hesitate to send me numerous
+updates. Any updates can be reincorporated. 
+
+
+Contents
+--------
+
+This repository will contain the data by individual in the `feedback` and then
+combined into one `summary.csv` file for analysis via `scripts/sg8-v1-post.py`.
+
+For example, to add the contents of two feedback forms to the summary.csv,
+one wold execute:
+
+```
+python3 scripts/sg8-v1-post.py feedback/Marshall.xlsx summary.csv
+python3 scripts/sg8-v1-post.py feedback/Percher.xlsx summary.csv
+```
+
+There is a directory with some test files at `scripts/tests` to practice
+using the sg-v1-post.py script.
+
+```
+$ python3 ../sg8-v1-post.py test-case1.xlsx practice.csv
+Reading data from: test-case1.xlsx
+Scanning for all cases: 4952it [00:00, 13275.35it/s]
+Processing rows: 4952it [00:00, 14081.22it/s]
+Updating existing: practice.csv
+Done.
+```
+
+This produces a simple CSV as follows. 
+```
+Case,Rev,Author,Rating,ModelIssue,BiasIssue,UncertaintyIssue,ImprovementsRecommended,AdditionalNotes
+HEU-COMP-FAST-001-001,1,J. Doe,3.0,True,False,True,,
+HEU-COMP-FAST-001-002,1,J. Doe,2.0,False,False,False,,
+HEU-COMP-FAST-002-001,2,J. Doe,4.0,False,True,False,,
+HEU-COMP-FAST-003-001,1,J. Doe,2.0,False,False,False,Do better,Try harder
+HEU-COMP-FAST-003-002,1,J. Doe,2.0,False,False,False,Do better,Try harder
+HEU-COMP-FAST-003-021,1,J. Doe,2.0,False,False,False,Do better,Try harder
+HEU-COMP-FAST-003-022,1,J. Doe,2.0,False,False,False,Do better,Try harder
+HEU-COMP-FAST-003-023,1,J. Doe,2.0,False,False,False,Do better,Try harder
+HEU-COMP-INTER-003-001,0,J. Doe,3.0,False,False,False,,
+HEU-COMP-INTER-003-002,0,J. Doe,3.0,False,False,False,,
+HEU-COMP-INTER-003-003,0,J. Doe,3.0,False,False,False,,
+```
+
+What is SG-8
+------------
+
+Subgroup 8 (SG-8) is an OECD/NEA Working Party for Nuclear Criticality Safety (WPNCS)
+sub group charged with developing a methodology for collecting and disseminating 
+feedback on evaluations from qualified experts to better serve users of the 
+ICSBEP benchmarks.
+
+Over twenty-five years of benchmarking activity, the expectations and review 
+rigor required for ICSBEP has evolved, the benchmarks are being used for 
+unanticipated scenarios, tools and computational power exist to solve more 
+complex problems, and new practitioners are entering the field. 
+
+A need has been recognized to preserve expert knowledge and judgement regarding 
+the suitability of ICSBEP evaluations to common uses such as modern code validation, 
+nuclear data evaluation, and nuclear data adjustment. 
+
+
+Technical Significance
+----------------------
+
+With the increasing rigor of the ICSBEP review process, there exists a 
+disparity between earlier and modern benchmarks in terms of uncertainty 
+quantification and more realistic modelling of the configurations. For example, 
+earlier benchmarks may quote unrealistic uncertainties that are then used to 
+set safety limits or assess nuclear data evaluations. Additionally, there are 
+benchmarks that have clear consistency problems, internally across cases or 
+compared to other, similar benchmarks.

sg8-v1-post.py

+import pandas as pd
+import sys
+import os
+from collections import defaultdict
+from tqdm import tqdm
+
+# set to true to generate debugging output
+debug = False
+
+# function to parse a full case name like
+# HEU-COMP-FAST-002-001 and return the last 001
+#                   ---
+def case_parse(full_case):
+    if full_case == "(REJECTED)":
+        case0 = -1
+    elif full_case == "":
+        case0 = 0
+    else:
+        case0 = int(full_case.split("-")[-1])
+    return case0
+
+
+# function to take the full case name like
+# HEU-COMP-FAST-002-001 and a special sequence of
+# cases to add like "5-9,13" and expand it to the
+# case list of 1,5,6,7,8,9,13. The keyword 'ALL'
+# implies all cases found in scanning the excel
+# sheet.
+def case_list(full_case, add_cases, all_cases):
+    case0 = case_parse(full_case)
+    cases = list()
+    if add_cases == "":
+        cases.append(case0)
+    else:
+        for c in add_cases.split(","):
+            if c == "ALL":
+                cases.extend(all_cases)
+            else:
+                c_list = c.split("-")
+                if len(c_list) > 1:
+                    cases.extend(list(range(int(c_list[0]), int(c_list[1]) + 1)))
+                else:
+                    cases.append(int(c_list[0]))
+    return list(set(cases))
+
+
+# reads a specially formatted Excel data file with benchmark feedback
+def process_excel_file(input_file):
+
+    # get excel sheet data
+    header = pd.read_excel(input_file, nrows=11, header=None)
+    c_author = header.loc[5, 2]
+    c_email = header.loc[5, 4]
+
+    # get ratings data
+    print("Reading data from:", input_file)
+    data = pd.read_excel(input_file, skiprows=13, usecols="B:I")
+    data = data.fillna("")
+
+    # print data for debugging
+    if debug:
+        print(data)
+
+    # create the ALL case range
+    all_cases = defaultdict(list)
+    for index, row in tqdm(
+        data.iterrows(), desc="Scanning for all cases", ascii=False, ncols=75
+    ):
+        iden = row["Identifier"]
+        full_case = row["Case"]
+        case0 = case_parse(full_case)
+        all_cases[iden].append(case0)
+
+    # print all cases for debugging
+    if debug:
+        print(all_cases)
+
+    # create a list element for each records row
+    records = list()
+    for index, row in tqdm(data.iterrows(), "Processing rows", ascii=False, ncols=75):
+        # only add data with a rating
+        c_rating = row["Rating"]
+        if c_rating != "":
+            # extract data
+            iden = str(row["Identifier"])
+            full_case = str(row["Case"])
+            c_rev = int(row["Rev."])
+            add_cases = str(row["Add Cases*"])
+            c_issue_keys = set(str(row["Issue Keys**"]).split(","))
+            c_improvements = str(row["Improvements Recommended"] or "")
+            c_notes = str(row["Additional Notes"] or "")
+
+            # expand to a case list (repeating the above data for each case)
+            for case in case_list(full_case, add_cases, all_cases[iden]):
+
+                # reconstruct full_case identifier
+                if case == -1:
+                    c_full_case = iden + "-REJ"  # rejected
+                else:
+                    c_full_case = "{}-{:03d}".format(iden, case)
+
+                # create a record with all the data
+                records.append(
+                    {
+                        "Case": str(c_full_case),
+                        "Rev": int(c_rev),
+                        "Author": str(c_author),
+                        "Rating": "{:2.1f}".format(c_rating),
+                        "ModelIssue": "M" in c_issue_keys,
+                        "BiasIssue": "B" in c_issue_keys,
+                        "UncertaintyIssue": "U" in c_issue_keys,
+                        "ImprovementsRecommended": str(c_improvements),
+                        "AdditionalNotes": str(c_notes),
+                    }
+                )
+    if debug:
+        print(*records, sep="\n")
+
+    # convert records data to a data frame
+    df = pd.DataFrame(data=records)
+
+    return df
+
+
+def main():
+
+    # input file from which we read data
+    input_file = sys.argv[1]
+
+    # archive file we merge new data into
+    archive_file = sys.argv[2]
+
+    if input_file.lower().endswith((".xlsx")):
+        df = process_excel_file(input_file)
+    else:
+        df = pd.read_csv(input_file)
+
+    # if the archive file exists, append to it.
+    if os.path.exists(archive_file):
+        print("Updating existing:", archive_file)
+
+        # use a custom index to ensure the combine happens as expected
+        df_index = ["Case", "Rev", "Author"]
+        df.set_index(df_index, inplace=True)
+
+        # read the old data and update the index as well
+        dfa = pd.read_csv(archive_file)
+        dfa.set_index(df_index, inplace=True)
+
+        # combine first (also called "upsert" which is a combination of
+        # updating existing and inserting). the data frame with priority
+        # should call combine first.
+        df = df.combine_first(dfa)
+
+        # reset the index so the CSV output is complete
+        df.reset_index(inplace=True)
+
+    updated = open(archive_file, "w")
+    updated.write(df.to_csv(index=False, header=True))
+    updated.close()
+    print("Done.")
+
+
+main()

tests/practice.csv

+Case,Rev,Author,Rating,ModelIssue,BiasIssue,UncertaintyIssue,ImprovementsRecommended,AdditionalNotes
+HEU-COMP-FAST-001-001,1,J. Doe,3.0,True,False,True,,
+HEU-COMP-FAST-001-002,1,J. Doe,2.0,False,False,False,,
+HEU-COMP-FAST-002-001,2,J. Doe,4.0,False,True,False,,
+HEU-COMP-FAST-003-001,1,J. Doe,2.0,False,False,False,Do better,Try harder
+HEU-COMP-FAST-003-002,1,J. Doe,2.0,False,False,False,Do better,Try harder
+HEU-COMP-FAST-003-021,1,J. Doe,2.0,False,False,False,Do better,Try harder
+HEU-COMP-FAST-003-022,1,J. Doe,2.0,False,False,False,Do better,Try harder
+HEU-COMP-FAST-003-023,1,J. Doe,2.0,False,False,False,Do better,Try harder
+HEU-COMP-INTER-003-001,0,J. Doe,3.0,False,False,False,,
+HEU-COMP-INTER-003-002,0,J. Doe,3.0,False,False,False,,
+HEU-COMP-INTER-003-003,0,J. Doe,3.0,False,False,False,,