Commit 6ef7f4ad authored by Vacaliuc, Bogdan's avatar Vacaliuc, Bogdan
Browse files

slides: ship an editable .md companion for every live slide



Per user request — scientists want to edit slide content without
dealing with raw SVG.  Each slide now has a .md file carrying:
  - YAML front-matter: slide number, title, subtitle, file refs,
    companion slides, authoritative source
  - Body: purpose, layout notes, key messages, speaker notes,
    editing pointers (which SVG blocks to touch for typical changes)

These .md files are NOT generator inputs — the SVG / Python scripts
remain the canonical source of the rendered output.  The .md is
documentation / spec so a reviewer can:
  a) read the .md to understand what the slide is for
  b) edit the .md to propose changes, and have the author bring
     those changes back into the SVG

Files added:
  - slide-0-opening.md
  - slide-1-current-vs-ideal.md
  - slide-2-api-mapping.md
  - slide-3-silent-defaults.md
  - slide-3a-errorweighted-background.md     (cites scientist sketch)
  - slide-3b-crop-first-and-last-points.md
  - slide-3c-round-up-pixel.md
  - slide-3d-accept-null-reflectivity.md
  - slide-3e-use-sangle.md
  - slide-4-curves-disagree.md  (documents the two-stage pipeline
                                  and the real-reduction provenance)
  - slide-5-peak-drag-sequence.md
  - slide-5a-debt-accumulation.md
  - slide-end-closing.md

Co-Authored-By: default avatarClaude Opus 4.7 (1M context) <noreply@anthropic.com>
parent a88dc23d
Loading
Loading
Loading
Loading
+90 −0
Original line number Diff line number Diff line
---
slide_number: 1
title: "From Today's Coupling to a Modular Back-End"
subtitle: "quicknxsv2 + mr_reduction  the modularization target of the 2026 hack-a-thon"
file_svg: slide-1-current-vs-ideal.svg
file_png: slide-1-current-vs-ideal.png
audience: scientists + developers, Day 1
companion_slides:
  - slide-2-api-mapping (the two-instrument premise)
---

# Slide 1 — Today vs. Ideal

## Purpose

High-level argument for modularization.  Contrasts the current
tangled state (UI and reduction code interleave; Mantid called from
many places) with the target state (one back-end for two
instruments; GUIs sit on top).  A visual gate for the rest of the
deck.

## Left panel — TODAY (tangled)

Three boxes:

- **quicknxsv2 (Qt GUI).**  Sub-caption: "Qt widgets · reduction code
  · Mantid calls.  No clean boundary between UI and reduction."
- **mr_reduction.**  Sub-caption: "autoreduce · livereduce · library.
  Different defaults than the GUI."
- **Mantid.**  Sub-caption: "reduction engine.  Version pin conflict
  (6.14/6.15)."

Three red arrows:
- quicknxsv2 → mr_reduction (Calls)
- quicknxsv2 → Mantid (Calls)
- mr_reduction → Mantid (Calls)

Every arrow is labelled *Calls* to avoid the common misreading of
boxes/arrows as inheritance relations (scientists flagged that in
the 2026-04-20 review).

Consequences summary at the bottom:
- GUI and autoreduce produce different R(Q) for the same run
- Peak drag freezes the UI for 1–10 seconds
- Fixes in one path don't benefit the other
- Qt lifecycle bugs appear as "wrong reduction numbers"
- New staff need ~2 weeks to trace any parameter end-to-end

## Right panel — IDEAL (modular)

Two UI boxes side-by-side:

- **quicknxsv2 — UI only** (REF_M front-end)
- **REF_L front-end** (existing `lr_reduction` GUI; adopts at its own pace)

Both Call into:

- **ref_core** — one shared reduction back-end for both instruments.
  Public API: `reduce()`, `inspect()`, `stitch()`, `write()`.  "single
  parameter-builder · single dead-time · single DataInfo."

Below that:

- **Mantid (pinned)** — one version for the whole suite.
- **Wins** panel: GUI and autoreduce compute the same numbers · bug
  fix in ref_core benefits every caller on day one · REF_L team can
  adopt when ready.

## Key messages

1. The UI/reduction boundary already exists inside quicknxsv2 — the
   `data_handling/` tree has zero Qt imports.  Modularization is
   formalising what's already true.
2. `ref_core` is the target back-end name.  The scientists have
   agreed on this.
3. The modularization must support TWO instruments even if the
   hack-a-thon works on one.

## Speaker notes

- Walk left-to-right.
- At "what scientists see" — pause; this list is the motivation for
  the rest of the deck (each bullet maps to a later slide).
- On the right side, name the wins in plain language.

## Editing

Hand-edit the SVG directly.  The layout uses `<rect>` + `<text>`
blocks; each labelled comment identifies the section.  Re-render
with `./render.sh`.
+77 −0
Original line number Diff line number Diff line
---
slide_number: 2
title: "Functional API Mapping  mr_reduction  lr_reduction"
subtitle: "Two instruments, two Mantid-centric libraries  the functional shape already converges"
file_svg: slide-2-api-mapping.svg
file_png: slide-2-api-mapping.png
audience: scientists + developers
companion_doc: ../14-mr-lr-cross-reference.md
---

# Slide 2 — API Mapping

## Purpose

Show that the two standard Mantid-centric reduction libraries
(mr_reduction on REF_M, lr_reduction on REF_L) already have the
same functional shape.  Supports the two-instrument message from
slide 1 by demonstrating that a shared `ref_core` is feasible, not
aspirational.

## Scope

Left column: **mr_reduction (next branch)** — Mantid-centric · REF_M ·
polarized neutrons.

Right column: **lr_reduction (next branch)** — Mantid-centric · REF_L ·
unpolarized · the *standard* reduction.

**Explicitly NOT covered** (per scientist feedback 2026-04-20):
- `lr_reduction:new_workflow` branch (h5py+numpy migration in progress)
- `quicknxsv1:read_event_nexus` branch

These are both non-Mantid lineages and their inclusion here would
muddle the modularization argument.  Revisit later in the hack-a-thon
or in a follow-on effort.  Documented in `14-mr-lr-cross-reference.md`.

## Body

Three-column table with ten reduction phases mapped across the two
libraries.  Each cell names the concrete function/class that handles
that phase:

1. Top-level reduce  — `ReductionProcess(...).reduce()``workflow.reduce(...)`
2. Cross-section filter — `filter_events.split_events()` ↔ N/A (REF_L unpolarized)
3. Peak / ROI discovery — `MRInspectData` / `DataInfo``peak_finding.process_data` + `fit_signal_flat_bck`
4. Dead-time correction — `apply_dead_time_correction` + `SingleReadoutDeadTimeCorrection` (same in both)
5. Per-run reduction algorithm — `MagnetismReflectometryReduction``EventReflectivity.specular()`
6. Resolution dQ/Q — inside MRR ↔ `event_reduction.compute_resolution`
7. Multi-run stitch — `reflectivity_merge.combined_curves``output.RunCollection.merge()`
8. Output ASCII R(Q) — `reflectivity_output.write_reflectivity``output.RunCollection.save_ascii`
9. HTML report — `web_report.Report``web_report.assemble_report`
10. Reproducibility script — `script_output.write_partial_script``template.write_template`

## Key insight (bottom banner)

> Every reduction phase has a functional counterpart in both
> libraries.  Implementations differ (Mantid algorithm vs numpy/scipy
> in-module; kwargs vs template-dataclass), but the public-API shape
> already converges.  A unified `ref_core` can expose
> `reduce / inspect / stitch / write / report`; each instrument
> plugs in its own reduction algorithm below.

## Speaker notes

- Emphasise that row 5 (per-run algorithm) is genuinely different
  between MR and LR, and that's fine — `ref_core` shares
  infrastructure, not the algorithm core.
- Row 2 is explicitly N/A for REF_L (unpolarized).  Good example of
  "per-instrument concerns can be local."
- Point to `../14-mr-lr-cross-reference.md` for the authoritative
  details.

## Editing

Table rows are regular `<g>` groups in the SVG; each row has 3 or 4
`<rect>` + `<text>` pairs.  Rearranging rows means copy-paste with a
y-offset.  Re-render with `./render.sh`.
+72 −0
Original line number Diff line number Diff line
---
slide_number: 3
title: "Five Silent Default Disagreements in the MRR Call"
subtitle: "Same algorithm, same input  two callers pass different defaults for five properties (one slide per detail follows)"
file_svg: slide-3-silent-defaults.svg
file_png: slide-3-silent-defaults.png
audience: scientists + developers, Day 1
companion_slides:
  - slide-3a-errorweighted-background
  - slide-3b-crop-first-and-last-points
  - slide-3c-round-up-pixel
  - slide-3d-accept-null-reflectivity
  - slide-3e-use-sangle
companion_doc: ../10-mantid-algorithms-deep-dive.md §11
---

# Slide 3 — Silent Defaults Overview

## Purpose

Summarise the five MRR parameters where `quicknxs` (GUI) and
`mr_reduction` (autoreduce) silently pass different default values.
This is the concrete mechanism behind the "GUI and autoreduce
produce different R(Q)" bullet in slide 1.

The detail slides 3a–3e each elaborate one row.

## Layout — 4-column table

| Column | Content |
|---|---|
| Parameter | MRR property name (monospace) and one-line description |
| quicknxs (GUI) | large typewriter value (False/True) |
| mr_reduction (auto) | large typewriter value (True/False) |
| Effect (icon) | small inline SVG pictogram summarising the tension |

## The five rows

1. **ErrorWeightedBackground** — GUI `False` / auto `True` — icon
   shows off-specular contamination intruding on the background band
   (from scientist's sketch).
2. **CropFirstAndLastPoints** — GUI `False` / auto `True` — icon
   shows a reflectivity curve with endpoints crossed out on the auto side.
3. **RoundUpPixel** — GUI `False` / auto `True` — icon shows a
   number line with 126.9 (GUI) vs 127 (auto).
4. **AcceptNullReflectivity** — GUI `True` / auto `False` — icon
   shows GUI plotting zero line vs autoreduce stack-trace error box.
5. **UseSANGLE** — both send `True` but via an inverted GUI flag;
   icon shows the name-inversion annotation.

## Bottom insight (dark bar)

> The structural fix: one parameter-builder, shared by both callers.
> All five tensions disappear if `build_mrr_kwargs(configuration, …)`
> is the only place that constructs MRR parameters — a hack-a-thon
> stretch goal.

## Speaker notes

- Briefly point at the icon column; explicitly say "the detail
  slides that follow unpack each row."
- Don't try to explain all five here — that's what 3a-3e are for.
- Acknowledge that #5 (UseSANGLE) is not a current bug but a latent
  land-mine.

## Editing

Each row is 5 elements (param box, GUI value box, auto value box,
icon container, plus any inline icon geometry).  To change a row's
values, edit the corresponding `<text>` elements.  To swap an icon,
edit the `<g transform="translate(1210, Y)">` block for that row.
Re-render with `./render.sh`.
+72 −0
Original line number Diff line number Diff line
---
slide_number: 3a
title: "Silent Default 1 / 5 · ErrorWeightedBackground"
subtitle: "Background-pixel averaging mode inside RefRoi  the scientific meaning and the observable effect"
file_svg: slide-3a-errorweighted-background.svg
file_png: slide-3a-errorweighted-background.png
parent_slide: slide-3-silent-defaults
authoritative_source: ../10-mantid-algorithms-deep-dive.md §9a
scientist_sketch: /home/6ov/Downloads/slide3-ErrorWeightedBackground-Explanation.jpg
---

# Slide 3a — ErrorWeightedBackground detail

## Values

- **quicknxs (GUI):** `False` — explicitly hardcoded
- **mr_reduction (auto):** `True` — inherits MRR default

## Physics

MRR estimates a background value per TOF bin by averaging detector
pixels outside the specular peak.  Two averaging modes:

- `False` — uniform mean: `⟨b⟩ = (Σ b_i) / N`, errors
  `σ = √(Σ σ_i²) / N`.  Robust for Poisson-dominated counting data.
- `True` — inverse-variance weighted: `⟨b⟩ = Σ(b_i/σ_i²) / Σ(1/σ_i²)`.
  For REF_M counting data this mode is not used — it assumes
  independent estimates of a single background value, which does not
  match the detector's spatially-structured noise.

## Observable

> **Backgrounds used when high-intensity off-specular reflections
> are present unfairly bias low-count reflectivity.**

Diagram (based on scientist's hand sketch): a pixel-vs-TOF detector
view shows the specular peak band (shaded) and the background
bands above and below.  Bright off-specular features (orange ovals)
land inside the background band.  When those contaminated pixels
are weighted by `1/σ²`, they contribute disproportionately to the
average and the background estimate is biased upward.  Subtracting
that biased background from the signal produces an *unphysical dip*
in R(Q) — the innocent low-intensity reflectivity gets punished.

Second sub-diagram: a reflectivity curve cartoon with the expected
truth (dashed gray) and the autoreduce curve (solid red) showing the
unphysical dip.

## What scientists actually see

- A run with strong off-specular features has its specular R(Q)
  silently pushed down.
- The "innocent" low-intensity reflectivity is the one most
  affected — edges of Bragg peaks, high-Q tails.
- Side-by-side comparison of GUI and autoreduce on the same run
  shows different numbers.
- Today there is no notice, no warning, no record that the two
  paths disagree — the values just differ.

## Speaker notes

- Credit the scientist sketch as the origin of the diagram.
- Keep the inverse-variance math on screen briefly, then skip —
  scientists objected to its presence in the last round because they
  don't believe it's an appropriate statistic for REF_M data, and
  the slide says so explicitly.

## Editing

Diagram on the right is hand-drawn SVG: rects for detector bands,
ellipses for off-specular spots, paths for R(Q) curve.  Text blocks
on the left are straightforward `<text>` elements.
+57 −0
Original line number Diff line number Diff line
---
slide_number: 3b
title: "Silent Default 2 / 5 · CropFirstAndLastPoints"
subtitle: "Edge-bin trimming on the Q axis"
file_svg: slide-3b-crop-first-and-last-points.svg
file_png: slide-3b-crop-first-and-last-points.png
parent_slide: slide-3-silent-defaults
authoritative_source: ../10-mantid-algorithms-deep-dive.md §11 Tension 2
---

# Slide 3b — CropFirstAndLastPoints detail

## Values

- **quicknxs (GUI · NexusData path):** `False` — explicit
- **mr_reduction (auto):** `True` — inherits MRR default

## Physics

After MRR rebinning in Q, the reflectivity histogram's first and
last bins may have partial-bin statistics.  Cropping them is a
defensive safety margin, but `CleanupBadData` (always on in both
callers) already removes bins that are literally zero or NaN, so
cropping can throw away real data.

The GUI scientist's reasoning: "cropping throws away real data when
CleanupBadData has already handled the genuinely unusable bins."

## Observable

The two paths produce R(Q) curves with different Q ranges.  Slide 4
shows this on real data: GUI 67 Q points vs autoreduce 65 Q points —
two dropped at each end.

Multi-run stitching picks up a different overlap region → per-run
scale factors drift between the two paths.

## What scientists actually see

- GUI and autoreduce outputs differ at the first and last two Q
  points.
- Multi-run stitching disagrees on the overlap → scale factors
  disagree.
- Even within quicknxs: the CrossSectionData path inherits True,
  the NexusData path sets False — inconsistent.
- Resolution: pin a single value, `False` recommended
  (CleanupBadData already handles bad bins).

## Speaker notes

- Brief; this is one of the easier wins.
- Note that slide 4 demonstrates this on a real reduction.

## Editing

Diagram on the right is a simple polyline with X marks on two
endpoint circles.  Body text is `<text>` blocks on the left.
Loading