Commit 431abbee authored by Bogdan Vacaliuc's avatar Bogdan Vacaliuc
Browse files

revised

parent 3e6ed059
Loading
Loading
Loading
Loading
+107 −58
Original line number Diff line number Diff line
@@ -11,21 +11,33 @@ can serve REF_L as well as REF_M; the `lr_reduction` "new workflow" branch
was endorsed as the shared reduction lineage to target; and a two-list
planning model (refactor-list + LR-wants-list) was agreed. A second
hack-a-thon / deep-dive week was committed for the back-end work proper,
to land **before the mid-to-late-June beam outage**. See **Consensus &
decisions** below.
to land **before the 2026-06-25 beam outage** (outage runs through
2026-08-04, per the SNS FY2026 Q1–Q3 official schedule). See
**Consensus & decisions** below.

Sources used for this summary:

- Bogdan's handwritten notes, `~/Desktop/Hack-A-Thon-2026/20260424071504.pdf`
  (30 pages; pages 4, 7, 9, 15, 19, 22, 23, 26 scanned inverted and rotated
  for reading).
- The shared Google-Doc notes prepared by Glass and circulated during the
  day (`2026-04-23-Notes-GoogleDoc.md`) — this is the most authoritative
  record of the morning-session tables (Mag / Shared / Liquids) and the Web
- Glass's live Confluence page in the Neutron Data Project Documentation
  space — *"Notes from hackathon discussion on Liquids Reflectometer
  requirements for QuickNXS 2026-04-22"* (downloaded PDF at
  `~/Desktop/Hack-A-Thon-2026/Notes+from+hackathon+discussion+on+Liquids+Reflectometer+requirements+for+QuickNXS.pdf`;
  same content as the local `2026-04-23-Notes-GoogleDoc.md`). This is the
  authoritative record of the Mag / Shared / Liquids table and the Web
  vs. Qt comparison matrix.
- Becky's `LR_reduction_workflow_config_parameters.xlsx` (linked in the
  Day-2 files manifest; not read directly — referenced as the forthcoming
  global-vs-per-run reference for the LR template).
- Becky's parameter inventory
  `~/Desktop/Hack-A-Thon-2026/LR_reduction_workflow_config_parameters.xlsx`
  — the canonical list of `lr_reduction` new-workflow config parameters,
  pre-sorted into seven tiers (per-run required, per-run often-changed,
  per-run-group required / often-changed / optional, per-experiment /
  reduction-method globals, instrument-based defaults, and a "not fully
  used" tier).
- SNS FY2026 Q1–Q3 official schedule
  (`~/Desktop/Hack-A-Thon-2026/sns-fy2026-q1-q3-official-updated-2026-03-01.jpg`),
  source: <https://neutrons.ornl.gov/sns/schedule> — used to pin the
  beam-outage dates.
- Meeting transcripts `2026-04-23-Part1-transcript.txt` and
  `Part2-transcript.txt` (Teams auto-transcripts; noisy — used for
  corroboration).
@@ -37,12 +49,11 @@ Sources used for this summary:
Scientists and software devs present or named: John Hetrick, Valeria
Lauter, Tim Charlton, Erik Watkins, Marie Backman, Becky Anderson, Jim
Browning, Glass Elsarboukh, Kevin Tactac, Volker Urban, **Maria Patrou**
(guest speaker, MVP tutorial), **Addi Malviya-Thakur** (joined in Session
3). Bogdan chaired. Asmaa Qdemat's presence on Day 2 is not clear from the
notes.
(guest speaker, MVP tutorial), **Addi Malviya-Thakur** (attending ad-hoc;
present for portions of Session 3). Bogdan chaired.

One housekeeping aside in the opening line: *"Bring tea for Becky for
Fri."*
Fri." ( *but she's not here for the rest of the hack-a-thon* )

---

@@ -74,9 +85,13 @@ afternoon) settle on an architectural pattern for the resulting code.

## 10:00 – 12:00 · Session 2 — Qt vs. Web, and the LiqRef requirements

This session ran the length of the slot. Glass maintained a live Google
Doc that tracked the room's consensus (see `2026-04-23-Notes-GoogleDoc.md`
for the authoritative tables — they're reproduced in summary form below).
This session ran the length of the slot. Glass maintained a live
Confluence page in the NDIP Managed-Software space (*"Notes from
hackathon discussion on Liquids Reflectometer requirements for
QuickNXS 2026-04-22"*) that tracked the room's consensus — the
`2026-04-23-Notes-GoogleDoc.md` file is a local mirror of the same
content. Authoritative tables live there; reproduced in summary form
below.

### Web vs. Qt — the morning's architectural debate

@@ -88,24 +103,26 @@ decision.
  easier to dev and maintain.
- **Erik:** agreed in principle.
- **Marie:** aligned with *institutional* direction — all effort at NSD
  is currently pushed toward NOIP/NOVA apps; the organisation "only way"
  is currently pushed toward NDIP/NOVA apps; the organisation "only way"
  is to deploy webapps long-term. QuickNXSv2 is production today
  (apprehension *and* excitement about touching it).
- **Glass / Tim:** a NOIP/NOVA "toy app" already exists and demonstrates
- **Glass / Tim:** a NDIP/NOVA "toy app" already exists and demonstrates
  pieces in place — deploy/replay are working.
- **Tim:** wants to track the effort point-by-point (Qt lane vs webapp
  lane). Note that **the web-app framing presupposes all business logic
  is already in the BE** — i.e. the refactor we were already doing is a
  precondition for either path.
- **?Nelld demo:** a browser demo got stuck loading a problem during the
  session. Tim: *"not all browsers are treated the same; some are not
  supported well."*
- **refl1d demo:** a browser demo of
  [refl1d](https://github.com/reflectometry/refl1d) (the precedent
  webapp from the reflectometry community) got stuck loading a problem
  during the session. Tim: *"not all browsers are treated the same;
  some are not supported well."*

#### NOIP/NOVA capacity / resource model (John's block)
#### NDIP/NOVA capacity / resource model (John's block)

- LDRD funding runs out ~FY26; operationalization falls under NSD;
  primary development on apps will transition. NSD fully commits to
  NOIP/NOVA but solution-probe testing is still unsure.
  NDIP/NOVA but solution-probe testing is still unsure.
- Spin-up of an app is Docker-based, so resources are allocated on
  demand. (refined uses all cores available today.)
- **Paradigm today:** all one character (node); if a job needs a super-
@@ -157,11 +174,15 @@ decision.
- Tim's users: give them a weblink and one click — benefit is a direct
  link.

### LiqRef requirements table (from the Google Doc)
### LiqRef requirements table (from Glass's Confluence page)

Glass's live table captured the Mag / Shared / Liquids breakdown. I have
not reproduced it here verbatim; see `2026-04-23-Notes-GoogleDoc.md` for
the authoritative version. Essentials only:
Glass's live table on the Confluence page — *"Notes from hackathon
discussion on Liquids Reflectometer requirements for QuickNXS
2026-04-22"* (NDIP-managed software documentation space) — captured the
Mag / Shared / Liquids breakdown. I have not reproduced it here
verbatim; see the Confluence page (or the locally saved copy and the
`2026-04-23-Notes-GoogleDoc.md` mirror) for the authoritative version.
Essentials only:

| Area | Key point |
|---|---|
@@ -204,9 +225,14 @@ single codebase, so the decision defers to UX preference.
### Lingering design questions (morning)

- Q: In `$mode`, do log values not fit? A: no, they exist.
- "Linkme / linear — *semantic triples* (subject-object-predicate)" —
  captured as an aside, context unclear from the notes; likely a
  reference to a metadata model discussion.
- **LinkML / semantic triples** (subject-object-predicate) — captured
  as an aside during Glass's discussion. This is the linked-data
  framing Glass pointed to for naming conventions in the code that
  "make more sense": see
  <https://linkml.io/linkml/data/rdf.html>. The idea is that
  parameter / metadata names should map cleanly onto subject-
  predicate-object triples so downstream tools can consume them
  without bespoke glue.

---

@@ -360,8 +386,8 @@ whiteboard tutorial; the notes follow it closely. Key content (pp.15–20):

### Addi Malviya-Thakur's monologue — technical-debt framing (p.21)

Addi chimed in (he had joined for Session 3) with the two-axis
debt framing he uses:
Addi chimed in (she was attending ad-hoc, present for this portion)
with the two-axis debt framing she uses:

- **Cognitive debt** — "must chase it in N places."
- **Intent debt** — fix one-by-one after you discover it.
@@ -378,8 +404,9 @@ debt framing he uses:

## 15:15 – 16:30 · Session 3 — Technical debt, test debt, and the big strategic question

With Addi present for the back half, the session moved to the hard
planning question: what, actually, does this hack-a-thon commit us to?
With Addi present ad-hoc for parts of the back half, the session
moved to the hard planning question: what, actually, does this
hack-a-thon commit us to?

### Technical debt — the concrete backlog (pp.22–23)

@@ -403,7 +430,7 @@ planning question: what, actually, does this hack-a-thon commit us to?

### Test debt (p.23)

- Takes a long time to set/forgery the tests.
- Takes a long time to set/forget the tests.
- So many tests rely on local axes — integration tests exercise the
  whole software (e.g. "button Z clicked" in GitLab-CI).
- GitHub Actions cannot do this — must verify behaviour. You have
@@ -489,11 +516,13 @@ everything `lr_reduction` needs?"*
- **Plan:** hack-a-thon → deep-dive (two-day session with Eric and the
  LR team) → **second Hack-A-Thon** dedicated to the underlying back-
  end refactor.
- The 2026 beam-cycle outage was discussed in the room: starts ~second
  week of June, ends ~second week of August (one team member noted the
  current schedule drops mid-to-end-of-June and picks back up the second
  week of August; SNS is running two megawatts a month ahead of
  schedule). This is the window the back-end plan must be in place for.
- **Beam outage window: 2026-06-25 → 2026-08-04** (per the SNS FY2026
  Q1–Q3 official schedule, updated 2026-03-01; source
  <https://neutrons.ornl.gov/sns/schedule>). SNS is running two
  megawatts a month ahead of schedule, which is why the pre-outage
  lead time is real. This is the window the back-end plan must be in
  place for — i.e. deep-dive and Hack-A-Thon #2 must both land before
  2026-06-25.

---

@@ -524,8 +553,8 @@ everything `lr_reduction` needs?"*
   this week that forecloses it.
9. **Follow-on cadence committed.** Current hack-a-thon → a two-day
   deep-dive session (Bogdan / Eric / LR team) → **second Hack-A-Thon
   on the underlying back-end** — all before the mid-June-to-mid-
   August beam outage.
   on the underlying back-end** — all before the 2026-06-25 → 2026-08-04
   SNS beam outage.
10. **Day 3 charge (tomorrow):**
    - Marie: deliver the LR-wants bullet document.
    - Kevin / Glass: Google-Doc folder is the working surface; the
@@ -556,11 +585,34 @@ everything `lr_reduction` needs?"*
- **Is the new-workflow LR branch feature-complete** relative to the
  production `lr_reduction` refined path? (Becky: "may be some stuff
  missing" — the separate-files structure limits merge risk.)
- **Global vs. per-run parameters** — Becky's xlsx is the forthcoming
  source of truth. List it, then wire the UI to read it.
- **Global vs. per-run parameters** — Becky's
  `LR_reduction_workflow_config_parameters.xlsx` is the source of
  truth; it pre-sorts parameters into seven tiers:
  1. *Per-run required*`method_per_run`, `DBname`, `RBnum`,
     `RB_Ymin`, `RB_Ymax`, `BkgROI`.
  2. *Per-run often-changed*`data_x_range`, `AutoScale`,
     `useCalcTheta`, `useBS`, `qmin`/`qmax`/`dqbin`,
     `LambdaMin`/`LambdaMax`, `tof_min`/`tof_max`/`tof_bin`,
     `ThetaShift`, `ScaleFactor`.
  3. *Per run-group required*`Sname`, `experiment_id` (IPTS).
  4. *Per run-group often-changed*`Normalize`, `Qline_threshold`,
     `Qnorm`.
  5. *Per run-group optional (good defaults)*`_Spath_override`,
     `_NEXUSpathRB_override`, `_DBpath_override`.
  6. *Per-experiment / reduction-method globals*`plotON`,
     `plotQ4`, `save8col`.
  7. *Instrument-based defaults*`mmpix`, `dSampDet`, `nx`/`ny`,
     `dMod`, `xi_ref`, `dS1Samp`, `IncidentTheta`,
     `emission_coefficients`, `use_emission_time`, `dead_time`,
     `dead_time_tof_step`, `DetResFn`, `DetSigma`, `peak_pad`,
     `peak_type`.
  8. *Not fully used*`_BINpath_override`, `DTCsubname`,
     `BINsubname`, `errBINsubname`, `useGravity`.
  Wire the UI against these tiers and the global-vs-per-run question
  answers itself.
- **Webreport / monitor.sns.gov integration** — adopt Plotly output
  for a diagnostic-plots tab? Blocker or already in flight?
- **NOIP/NOVA resource model for interactive reduction jobs** — John
- **NDIP/NOVA resource model for interactive reduction jobs** — John
  flagged that the *"need X cores / Y GB"* scaffolding exists, but
  neutron-apps haven't exercised it at interactive-GUI latency. Pilot
  in scope?
@@ -572,25 +624,22 @@ everything `lr_reduction` needs?"*

## Notes on fidelity

- The Google Doc prepared by Glass is the authoritative record for
  the Mag / Shared / Liquids breakdown and the Web-vs-Qt cost/benefit
  framing; I have summarised those tables rather than restating them
  verbatim. If you spot a divergence between the Google-Doc version
  and my summary, trust the Google Doc.
- Glass's Confluence page (*"Notes from hackathon discussion on Liquids
  Reflectometer requirements for QuickNXS 2026-04-22"*, in the Neutron
  Data Project Documentation space — "NDIP = Managed Software") is the
  authoritative record for the Mag / Shared / Liquids breakdown and the
  Web-vs-Qt cost/benefit framing; I have summarised those tables rather
  than restating them verbatim. If you spot a divergence between the
  Confluence version and my summary, trust Confluence.
- The Teams auto-transcripts are noisy (garbled, "LR-reduction" →
  "referend"/"LRRed"; "mr-reduction" sometimes → "demanded Albertson";
  speaker attribution missing), and the final 20 minutes of Session 3
  are where the decisions actually landed — I trusted the handwritten
  notes over the transcript here.
- A handful of tokens in the handwritten notes were ambiguous: "Nelld"
  on page 3 (during the bad webapp demo); "UDP = Managed Software" on
  p.11 (likely a NOIP term); "linkme/lineal — semantic triples" on
  p.12 (context unclear — appears to be a metadata-model aside).
  Please correct in review.
- Maria Patrou and Addi Malviya-Thakur were noted-present on
  handwritten notes but not listed in the invite-block of the agenda;
  I've treated them as guest / ad-hoc attendees for the MVP tutorial
  and Session 3, respectively.
- Maria Patrou (guest speaker for the MVP tutorial) and Addi
  Malviya-Thakur (ad-hoc attendee, present for parts of Session 3) were
  noted-present in the handwritten notes but not listed on the
  invite-block of the agenda.

---

+13 −0
Original line number Diff line number Diff line
@@ -71,4 +71,17 @@ Claude produced [Hack-A-Thon-2026-Day1-Summary.md](../Hack-A-Thon-2026-Day1-Summ

Please read ~/Desktop/Hack-A-Thon-2026/2026-04-23-Files.md. Please use that information to summarise the Day 2 events and write them to tasking/Hack-A-Thon-2026-Day2-Summary.md for me to review. If there is any other information that you would like to have, please ask me.

Claude produced [

### Prompt 1.2

Regarding LR_reduction_workflow_config_parameters.xlsx, I have downloaded it to ~/Desktop/Hack-A-Thon-2026/LR_reduction_workflow_config_parameters.xlsx if you want to inspect it. A few other notes:
1. "Nelld" is "refl1d" and which has a public github repository at: https://github.com/reflectometry/refl1d
2. "UDP = Managed Software" is "NDIP - Managed Software". It was a note to myself to obtain/extract [a page that Glass was working on during Day 2](https://ornl-neutrons.atlassian.net/wiki/spaces/NDPD/pages/785252365/Notes+from+hackathon+discussion+on+Liquids+Reflectometer+requirements+for+QuickNXS+2026-04-22). I have downloaded it to ~/Desktop/Hack-A-Thon-2026/Notes+from+hackathon+discussion+on+Liquids+Reflectometer+requirements+for+QuickNXS.pdf
3. "linkme/lineal semantic triples" is "LinkML/Semantic Triples" and was a note to myself obtain the [page that Glass referred to during his discussion on Day 2](https://linkml.io/linkml/data/rdf.html) where he related language terms such as subject-object-predicate to naming conventions in code that make more sense.
4. Addi is a female, please correct all references to her in the summary as necessary
5. Addi attended ad-hoc when she could
6. GenX was agreed to be dropped, I accept that resolution from the discussion.
7. Beam-outage dates are 2026-06-25 (begin) to 2026-08-04 (end). I have downloaded ~/Desktop/Hack-A-Thon-2026/sns-fy2026-q1-q3-official-updated-2026-03-01.jpg from https://neutrons.ornl.gov/sns/schedule for you to incorporate into the summary as appropriate