diff ref struct Rietveld refinement for the Mac · powder diffraction with X-rays, synchrotron and neutrons · started October 2026

A project of its own for how a refinement finds its answer: I05, I06, their sessions and the audit, deepened first and implemented after

A project of its own for how a refinement finds its answer: I05, I06, their sessions and the audit, deepened first and implemented after discussed

I07 · an idea of the seed-place, on its way to the project.

Said
2026-10-21, Josu
Status
discussed
Home
project (where it migrates)
Needs
I05, I06

The idea

Perhaps it is time to move the content of this topic, the ideas I05 and I06, the sessions they came out of and the audit of 21 October, to another website dedicated to it, to go deeper: all the references, what each of them claims, and how all of them could be taken into account, in a separate, independent project, first for deepening and only after for implementing, if possible and if it makes sense, in the app we are working on.

Assessment

Why a project of its own is right. Three reasons. The seed-place is a waiting room, not a library: an idea's page holds a conversation's elaboration, and I05's and I06's already hold more than that, with some ninety references between them. The app's streams are bound by L139 (every calculation in the core, bit-exact, under FullProf's tests), where the deepening needs to read, compare and try estimators and minimisers that are not Rietveld's, in whatever language is quickest, before anything is proposed to the core. And the precedent exists and works: the inverse-problems project of 14 October (20261014_inverse_problems_configuration_space), a line of thought on its own site, worked with the same protocol, code only when a proposal of its programme is taken up. Its CLAUDE.md, build.py, kit and publish Action can be copied in an hour; they were, the same day.

What the project is, said exactly. Its subject is how a refinement finds its answer: the estimator (least squares, maximum likelihood, the entropic and Bayesian forms of I05 and L286) and the minimiser (I06's path methods, the landscape, the energy, the learned pieces), kept apart and each with its test. Its matter in six parts, each a page generated from a data file:

  • The library (data/sources.yml): every reference of I05, I06, L286, the sessions and the audit, each with its status (read, checked by search on a date, not checked), its DOI or URL, and three fields: what it claims, in its own words where they were read; how it was tested there (data, program, numbers); what it means for a Rietveld engine. About ninety entries at the start; this is the page Josu asked for.
  • The method catalogue (data/methods.yml): A1–A7, B1–B5, C1–C5, D1–D7 of I06, A4d, A6′ and §5′ of the audit, I05's three uses, and whatever comes; each a card: the problem it attacks (i–vi), the mathematics in a paragraph, its precedents (links into the library), what it keeps (the reference minimum, or a result shown beside it: path or information), its test on the bench, and its status (idea, reading, prototype, benched, proposed to the app, declined, with the date of each).
  • The sessions, curated. The chats of 15 October (L286), 16 October (I05) and 19 October (I06) exported turn by turn from their transcripts into sessions/ (private, as sources/ is in the inverse project), by a tool of the project; and on the site, after Josu's review since the words are his, a reading of each: what was asked, what was answered, what was wrong, what was learned, what it led to.
  • The bench: I06's workshop §3–§4 as the design page (three kinds of start, four phases, the measures, the performance profiles), then its results when there are any; the data: FullProf's tutorials (their licence checked before any is published), GSAS-II's \(\mathrm{Al_{2}O_{3}}\) and \(\mathrm{PbSO_{4}}\) tutorials (Spotlight's cases), \(\mathrm{Y_{2}O_{3}}\) (BBO-Rietveld's and the core's), synthetic cases with known truth; the baselines: the core, FullProf headless, Spotlight's and BBO-Rietveld's published numbers, TOPAS where available.
  • The bridge to diff-ref-struct: one page listing what the project proposes to the app, each row with the test it passed, the numbers, and the stream it would go to (core, Explore, Publish); the app's seed-place keeps I05 and I06 with a pointer and marks them migrated when something lands. Nothing crosses without a bench result and a decision logged in both projects.
  • The brief as questions (Q01…: is the least-squares minimiser the best one can do; can a start far from the answer be captured without a human; can false minima be certified; which likelihood is honest; can the energy be used without presenting its result as Rietveld's; where may a learned piece sit), the roadmap, the log.

Prototypes, and the one rule for them. Code lives in named folders (p1-bench/, …), in Python first (numpy, scipy, JAX where a differentiable path is wanted), with every pattern computed by the core through its Python bindings (core 0.70.0, L381: drs.run, drs.open), so that the physics is the core's and only the search is the prototype's. What wins on the bench is proposed to the core in Rust; what loses is kept as a finding. This is the split L139 asks for: calculation in the core when in the app, experiments outside.

What it costs, and the risks. A fourth site to keep (the app's, Publish's figures, the inverse project's, this one): small, since the kit and the generator are shared. Duplication between the ideas pages and the project: the rule is that the project is the source once Josu names it; the ideas pages keep a summary and the pointer and are not extended further. Publishing the sessions needs Josu's review, and FullProf's tutorial data a licence check. The bench's compute on a loaded Mac (L. nice, CARGO_BUILD_JOBS): the bench counts evaluations, not seconds, for that reason. And the risk of every library: reading replaces doing; the roadmap puts the first bench run (phase 1) before the tenth reading is finished.

What exists on 21 October. The scaffold, local and not yet on GitHub, at ~/Desktop/2026/20261001/20261021_rietveld-minimisation (the working name is rietveld-minimisation; the name is Josu's): CLAUDE.md, README.md, ecosystem/ with the kit, data/sources.yml with the references and their claims, data/methods.yml with the catalogue, data/brief.yml as questions, data/nav.yml, data/log.yml from L001, the publish Action written for a site repository not yet created, the three sessions exported to sessions/ by tools/export_sessions.py, and the site rendered locally.

Proposal

  • Step 1 (done the day it was said). The scaffold above, local; nothing published.
  • Step 2 (Josu). The name; a private GitHub repository and a public -site repository with a deploy key, as the inverse project has; the first push, which renders and publishes.
  • Step 3. The reading programme. The references marked checked are read in full, ten first, each logged with what changes in the catalogue: Spotlight (Biwer et al. 2025), BBO-Rietveld (Ozaki et al. 2020), Gagin and Levin 2015, Transtrum and Sethna 2012, David 2004 (both papers), Antoniadis 1990, Coelho 2005, Toby 2024, Kerman 2012, Segal 2025; then the Agada–Biswas pair and AiMat-DRX.
  • Step 4. The bench's design frozen as a page, then its first run: the core through the bindings and FullProf headless from the same starts; the baseline (I06's L1), a finding with its numbers.
  • Step 5. The first path method prototyped and measured: the optimal-transport capture (A4d) or variable projection (A1), the bench deciding; the bridge page gets its first row.
  • In this project: I05 and I06 keep their pages with a line pointing to the project; new notes on the topic are written there; when something lands in the core, their log says so and their status moves to migrated.

For Josu to decide

  • The project's name (working name rietveld-minimisation; the subject is wider, how a refinement finds its answer, estimator and minimiser both).
  • Whether the site is public from the first push, as the inverse project's is, with the sessions private until Josu has reviewed their curated reading.
  • Whether the prototypes may use the core through its Python bindings, as proposed, so that every pattern is the core's.
  • Whether the ten first readings are the right ten, and in which order.

The idea's log

  • 2026-10-21 · said

    Said by Josu while the audit of I05 and I06 was being written: perhaps it is time to move the topic, the two ideas, their sessions and the audit, to a website of its own, to go deeper with all the references and what they claim, in an independent project, first for deepening and only after for implementing in the app, if it makes sense.

  • 2026-10-21 · discussion

    Assessed and designed: a project worked as the inverse-problems one, with a library of every reference and its claims, a method catalogue, the sessions curated, the bench, a bridge page to the app, the brief as questions; prototypes in Python on the core's bindings, Rust only for what wins. The scaffold made locally the same day, not published; the name, the repositories and the first readings are Josu's to decide. L394