Physics AI & Causal World Models

A learned atmospheric model has never once seen the atmosphere. It has seen instruments measuring transformations of the atmosphere. Radiances in a satellite channel. Reflectivity at a radar wavelength. Station reports with inconsistent siting and inconsistent quality control. Radiosondes launched twice a day into a field that varies continuously in space and in time. Everything a model knows about the sky arrives through a sensor, and every sensor imposes its own geometry, its own spectral response, its own drift, and its own silence about the places it does not look.

That sentence is the reason this series starts where it does. Welcome to an eight part walk through a literature map on physics AI and causal world models, following the eight themes of a curated explorer of about eighty papers. The question underneath all of it is not whether learned models can predict physical systems. That result is increasingly established, and we will spend the next episode on exactly how established it is. The question is what additional evidence is required before prediction can be interpreted as learned physical structure, as causal understanding, or as a model safe to use for counterfactual reasoning. Where do those claims separate? Which experiments discriminate between them? And which experiments still have not been run?

This is episode one, the sensor boundary. Observations, data assimilation, and the layer where the physical world becomes numbers. It comes first because it is the layer that most discussions skip, and skipping it is how a claim about a model quietly becomes a claim about an entire observing system that the model did not build and does not own.

Let us start with what the classical machinery does, because it is not a formality you can wave past on the way to the interesting part. The problem is an inverse problem. You have a state, the full three dimensional structure of the atmosphere at an instant, and you cannot measure it. You have observations, scattered, indirect, noisy, and far too few. You have a model that propagates a state forward and accumulates its own error while doing it. Data assimilation is the machinery that combines those into a best estimate.

The reference point for the modern version is Geir Evensen in 1994, the ensemble Kalman filter. The problem he was solving is that the extended Kalman filter evolves error covariances through a linearization around a trajectory, and for a nonlinear geophysical model that linearization degrades in exactly the situations you care about. Evensen replaced it with a Monte Carlo ensemble. Run many states forward through the actual nonlinear model, and sample the forecast error statistics from the model itself rather than approximating them. That is the paper that made state estimation in geophysics an explicitly probabilistic exercise, and it is the baseline every learned assimilation method is still measured against. If you take one structural fact from it, take this one. Operational weather has been doing uncertainty quantification, at scale, in production, for thirty years. This is not a field that needs to be taught that a point estimate is insufficient.

Now the part that matters most for machine learning, and it comes from Dick Dee in 2005, in a paper about bias and data assimilation. Suppose your observations are biased. Suppose your model is also biased. What happens? The analysis absorbs the bias into the state estimate. Variational bias correction, the standard operational tool, can only work when it is anchored by observations you have reason to believe are unbiased. And here is the sharp part. Model bias and observation bias are not separately identifiable from the residuals alone. The residual tells you they disagree. It does not tell you which one is wrong.

Sit with what that implies for the current generation of learned weather models. Almost all of them are trained on reanalysis, and scored against reanalysis. Reanalysis is not ground truth. Reanalysis is a model observation fusion, produced by running a physics based assimilation system over the historical observing record. It contains a model. So a learned system trained on it inherits the assimilating model’s biases, and a learned system scored against it is being graded by a product that already made the choices it is being tested on. This does not make the skill fake. It makes the skill conditional, and the condition is a seventy year investment in an observing system and an assimilation pipeline that the learned model treats as a free input.

There is a second measurement in this theme that I want to give real time to, because it answers an argument that circulates constantly right now. The argument goes like this. Language models are running into a data wall, but weather has no such wall, because the world collects an extraordinary volume of atmospheric observation every single week, more raw data than went into training a frontier language model. Therefore there is enormous headroom.

The trouble with that argument is that it treats observation volume and assimilable information as the same quantity, and the field has measured the difference. Zhiquan Liu and Florence Rabier, in 2002, worked the problem out in a one dimensional analysis system, carrying representativeness error and observation error correlation through to the analysis error. For an optimal scheme, denser observations keep helping. For the suboptimal schemes that operational centers actually run, density past a threshold stops paying, and thinning correlated observations improves the analysis. Adding data made the estimate worse. Not through anyone’s incompetence, but because correlated error that the scheme does not model gets treated as independent information.

Then Carla Cardinali in 2009 measured the marginal value of every observation in the ECMWF system directly, using adjoint forecast sensitivity to attribute twenty four hour forecast error reduction observation by observation, cross checked against observing system experiments over a summer month and a winter month. The assimilated observations reduce forecast error in aggregate. Specific subsets degrade it, traceable either to data quality or to suboptimality in the assimilation. So when someone argues from raw data abundance, there is already a per observation measurement waiting for them, and it has been running operationally for fifteen years. The right question is not how many bytes arrive per week. It is how much of that stream is informative after error correlation is accounted for, and nobody has published the curve showing where a learned model’s skill saturates as observing density rises.

So that is the classical layer. Now, what happens when learning is pushed into it?

The first move is to treat assimilation as a sampling problem instead of an optimization problem. François Rozet and Gilles Louppe, score based data assimilation, in 2023. Train a score model on short state trajectories, learn the distribution, then at inference time condition on whatever observations you happen to have and sample the posterior over the full trajectory. No linearization. No sequential filter structure. The posterior is something you generate rather than something you optimize toward. Most of the learned assimilation work since has followed that template, and you can see why it is attractive. It inherits the generative modeling toolchain, and it handles arbitrary observation configurations without rebuilding the machinery each time.

The second move closes the loop from the other side. Xiao and colleagues, FengWu 4D-Var, also 2023. Four dimensional variational assimilation is expensive largely because it needs an adjoint, the derivative of the forecast model, and maintaining an adjoint of an operational physics model is one of the most labor intensive tasks in the field. A learned forecast model is differentiable by construction. The adjoint comes free. So they coupled a learned forecaster to a 4D-Var scheme and cycled the whole system. That is the point at which AI weather models stopped structurally depending on somebody else’s analysis in order to run.

The third move removes the analysis entirely. Anna Vaughan and colleagues, Aardvark Weather, in 2024, replaced the full operational pipeline, assimilation and forecast model and post processing, with a single network trained from raw heterogeneous observations to station level forecasts, at a small fraction of the compute. And Mihai Alexe and colleagues, GraphDOP, trained a graph model purely on Earth system observations. No reanalysis anywhere in the pipeline. No physical state variables in the loss. It forecasts observed quantities directly in observation space, at skillful medium range, which is the cleanest existence proof I know of that useful forecast skill can be learned without ever seeing an analysis. Yumeng Zhao and colleagues in 2026 extend that line into the high resolution regime, where shedding analysis dependence has been hardest.

Then there is a follow up on GraphDOP that belongs in this episode and also foreshadows episode five. Lean and colleagues, in 2025, probed what GraphDOP’s latent state actually contains, and found structure corresponding to physical fields the model was never given. It was trained on instrument readings. Inside, there is something that lines up with the physics. I want to be precise about what that does and does not establish, because this is exactly the kind of result that gets over read. It establishes that the observation stream carries enough information to reconstruct physical structure, and that a model under prediction pressure will build some of that structure internally. It does not establish that the structure is load bearing for the model’s skill, and it does not establish that the model can use it the way a physical model uses a conservation law. We will come back to that distinction with the tools to test it.

The fourth move is the one closest to the instruments, and as someone who has spent time with spectroscopic data from spacecraft, it is the one I find most interesting. The observation operator is the map from model state to what the instrument would report. Historically it is hand derived physics. Radiative transfer for a satellite channel. Scattering assumptions for radar. It is also, historically, slow, and a known source of representativeness error.

Stefanelli and colleagues, in 2025, learned the operator for weather radar reflectivity, replacing a forward operator whose microphysical assumptions are a documented weak point in convective scale assimilation. Howard and colleagues emulated the Community Radiative Transfer Model with calibrated uncertainty on the emulated radiances, and that word calibrated is doing the work. A surrogate observation operator that reports where it disagrees with the physical forward model is a scientific instrument. A surrogate that silently substitutes for it is a liability. The useful artifact is the error distribution, not the speedup.

And then the failure mode that instrument people expect and machine learning evaluations rarely isolate. Li and colleagues, SPECTRA, in 2026, studied what breaks when a model pretrained on one satellite instrument’s spectral response and viewing geometry is transferred to another. Performance degrades, and it degrades in a way that looks exactly like model error unless your evaluation is designed to separate the two. Instrument bias, calibration drift, cross sensor generalization, provenance of the observation record. These are not administrative details. They are the mechanism by which a model can appear to misunderstand the physics when what it actually encountered was a different detector.

The last paper here makes truth itself an experimental variable. Jin and colleagues built WeatherReal, a quality controlled global in situ station dataset, and scored AI and physical models against real measurements rather than against reanalysis. The apparent skill ordering depends on which truth you use. That is not a subtle effect at the margins. It is a reminder that when two systems are compared on reanalysis, part of what is being measured is proximity to the assimilating model.

So what does this theme actually establish, standing back?

It establishes that assimilation is learnable, in several distinct senses. As generative sampling of the posterior. As a differentiable coupling where the learned forecaster supplies the adjoint. And as an end to end replacement in which the analysis step disappears and the model learns from instrument readings directly. All three exist, in published, reproducible work.

It establishes that the sensor boundary is where a large fraction of a learned model’s apparent error and apparent skill actually live. Change the truth dataset and the ranking moves. Change the instrument and the model degrades in a way that mimics physical error. Assimilate more data into a suboptimal scheme and the analysis can get worse.

And it establishes the standard for what a claim in this area has to answer. If you assert that the physical world provides effectively unlimited training signal, you are making a claim about assimilable information, and there is a measurement for it, running in production, per observation, since 2009.

What it does not settle is the attribution problem, and that gap runs through the whole series. No benchmark currently attributes a learned model’s error between the observing system, the assimilation, and the forecast model. So the model is wrong and the analysis is wrong right here remain indistinguishable in every score anyone reports. Everything in the next seven episodes inherits that ambiguity.

Next episode, we go to the result that made all of this urgent. Between 2022 and 2024, neural networks trained on reanalysis beat the operational deterministic forecast on the headline scores, under the forecasting center’s own verification, at a small fraction of the inference cost. That happened. It is real, it is narrow, and the interesting work is in the words the headline leaves out. That is episode two.

In 2022, a neural network beat the best physics based weather forecast in the world on its own headline scores. In 2023, another one did it more convincingly, and a third did it in the probabilistic setting where operational value actually lives. By 2024 the European Centre for Medium Range Weather Forecasts was running its own learned model in its own suite. That sequence is real, it took about two years, and if you work in numerical weather prediction it was the most disorienting stretch in a generation.

This is episode two of an eight part walk through a literature map on physics AI and causal world models. Last episode we stayed at the sensor boundary, where the physical world becomes numbers, and the takeaway was that a learned model inherits an observing system and an assimilation pipeline it did not build. Today we take the capability claim seriously and say exactly what was won, because the honest version of this result is narrower than the headline and more interesting than the headline.

Start with cost, because cost is what changed the conversation first. Jaideep Pathak and colleagues, FourCastNet, in 2022. An adaptive Fourier neural operator trained on ERA5 reanalysis, producing global forecasts at a quarter degree resolution, running a full forecast in seconds on a single GPU. The skill was competitive rather than dominant. The economics were not competitive, they were a different category. A physics based ensemble is a supercomputer scheduling problem. A learned ensemble at that speed makes large ensembles nearly free at the margin, and once large ensembles are free you can ask questions that were previously unaffordable. That is what put learned forecasting on operational centers’ agendas.

Then the skill argument closed, twice, within months. Kaifeng Bi and colleagues at Huawei, Pangu-Weather, a three dimensional Earth specific transformer with hierarchical temporal aggregation, beating the operational Integrated Forecast System on deterministic scores across most upper air and surface variables. And Remi Lam and colleagues at DeepMind, GraphCast, an encode process decode graph neural network on a multi mesh icosahedral grid, autoregressively rolled out during training, outperforming the operational system on the large majority of evaluated variable and lead time combinations, including several severe event proxies.

Notice something about that pair. One is a transformer on a latitude longitude grid. The other is message passing on an icosahedral mesh. Architecturally they have almost nothing in common, and they arrived at the same place at the same time. That is the signature of a result that is about the problem rather than about the method. The atmosphere at synoptic scale, over the range where reanalysis is dense and the dynamics are large and smooth, turns out to be extremely learnable. After that pair the architecture question mostly stopped being where the gains came from, and the gains came from training strategy, from rollout schedules, and from scale.

GraphCast also set the evaluation protocol as much as the score. It compared against the operational system on a defined variable and lead time grid, it reported severe event proxies rather than only continuous fields, and it published enough of the setup that later systems could be measured the same way. That matters more than it sounds. In a field where every group can choose its own truth dataset, its own initialization, and its own baseline, the paper that fixes a comparison protocol shapes what counts as progress for years. Much of the evaluation critique we will get to in episode four is an argument about how far that protocol goes and where it stops.

We now have measured exponents for that scale claim, which I appreciate, because scaling arguments in this area are usually asserted rather than fit. Yu and colleagues, in 2026, fit empirical scaling relations between validation loss and model size, dataset size, and compute across several learned global weather models. Three results worth carrying. Aurora shows the strongest data scaling, where a tenfold larger training set cuts validation loss by up to a factor of three point two. Under a fixed compute budget, spending on data beats spending on parameters. And these models favor width over depth, which is the opposite of the language model result. So when someone says physical prediction will improve with scale, the measured version of that statement is that it improves with training data, and the atmosphere’s training data is bounded by the observing record, which brings us right back to episode one.

Now the deterministic scores. There is a structural problem with them that the field walked into knowingly. If you train a model to minimize mean squared error over an ensemble of possible futures, the optimal output is the conditional mean. The conditional mean of a chaotic field is smooth. So a mean squared error trained forecast is systematically blurry, and blurriness is rewarded by the very metric being used to declare victory. A smoothed field has lower error against a noisy truth than a sharp field with the correct statistics but the wrong phase.

Two responses. Ilan Price and colleagues, GenCast, a conditional diffusion model over fifteen day trajectories, generating sharp physically plausible ensemble members and beating the European ensemble on the large majority of continuous ranked probability score targets. That is the move that took learned forecasting out of the regime where blurring wins and into the probabilistic setting where operational users live. And a parallel line at the European Centre optimizing the probabilistic score directly rather than through a sampling procedure. Both are the field conceding, in the design of the loss function, that a mean squared error forecast is a conditional mean and not a forecast.

Here is the part that gets flattened in popular accounts. The pure learned versus physics based framing did not survive contact with the results. Dmitrii Kochkov and colleagues, NeuralGCM, couples a differentiable dynamical core, actual fluid dynamics on the sphere, to learned parameterizations for the processes physics based models approximate anyway, and trains the whole thing end to end. It matches learned model skill at medium range. It stays numerically stable over decade long integrations. It reproduces emergent phenomena like tropical cyclone statistics that it was never explicitly trained to produce. That is not a compromise between two camps. The useful decomposition runs along a different axis, separating the parts of the system you have a reliable equation for from the parts you are approximating regardless.

And the convergence kept going from the other direction. Polichtchouk and colleagues, in 2026, nudge the large scales of a physical model toward a learned forecast while leaving the small scales to the physics, so each component runs where it is strongest. The interesting question stopped being who wins and became how you couple them.

Now the limits, and this is where I want to spend real time, because the limits are where the intellectual content is.

Zied Ben Bouallègue and colleagues at the European Centre ran the independent replication in 2024. Take Pangu-Weather and GraphCast, put them inside the center’s own operational verification framework, initialize them from operational analyses rather than from the reanalysis they were trained on, and score them against the metrics and baselines the center itself uses. The headline gains are real and they survive. They also come with deficiencies in near surface fields, at small scales, and in extremes. The advantage is not uniform. It is variable specific and regime specific, and it narrows when the initialization pipeline changes.

Massimo Bonavita, also at the European Centre, wrote the sharpest version of the critique in the same year. Learned models produce spatially over smoothed fields with deficient variance at small scales. Part of the apparent root mean square error advantage is an artifact of that smoothing rather than a reflection of better physics. And these systems depend on an assimilation system they neither model nor own. That last clause is the one to hold. In episode one we saw the pipeline being learned away, in Aardvark and GraphDOP and the high resolution follow ups, so the dependence is not permanent. But for the systems that produced the headline results, it was total.

Ashesh Chattopadhyay and Pedram Hassanzadeh explain the mechanism underneath the smoothing rather than treating it as an implementation quirk. When you train an emulator of a multiscale chaotic system and roll it out autoregressively, the small scale variance is the first thing to go, and the failure compounds because the model then feeds its own smoothed field back to itself. They analyze why, and what training and spectral interventions delay it.

Put those three together and you get a precise statement. These models can win at day five root mean square error while being wrong about how error grows. That combination is possible because aggregate error against a smooth truth and the propagation of uncertainty through scales are different quantities. One of them is what the leaderboard measures. The other is what a forecaster uses to decide whether to trust the forecast.

There is one more limit that has started to dissolve, and it is worth flagging because the objection was so central. For two years the standard rebuttal to learned forecasting was that these models only work because numerical weather prediction feeds them, and that the dependence would bite hardest at high resolution, where analysis quality and physical downscaling do the most work. Zhao and colleagues, in 2026, report skillful high resolution forecasting with no physics based model anywhere in the pipeline. That does not retire the objection for the systems that produced the 2022 and 2023 headlines, which were fully dependent. It does mean the dependence is a property of particular systems rather than a structural necessity, and that the interesting version of the objection has to be restated as a claim about the observing record rather than about the analysis product.

Two more entries round out the theme. Aurora, from Cristian Bodnar and colleagues, pretrains a single atmospheric model on a heterogeneous mixture of reanalyses, forecasts, and climate simulations, then fine tunes to tasks including air pollution and ocean waves, where dedicated operational systems are the baseline. That is the first credible demonstration that Earth system pretraining transfers across variables and domains rather than only across lead times, and it is the paper that earns the phrase foundation model in this space. We will pull on that thread properly next episode. And ECMWF’s own AIFS, developed inside the operational environment rather than against a published benchmark, marks the transition from external claims about beating the operational system to an operational center running a learned model in its own suite.

It is also worth being concrete about what operational like means in that replication, because the phrase is doing a lot of work. Training on reanalysis and evaluating on reanalysis is a closed loop. Reanalysis is produced after the fact, with the full observing record available in both directions in time, which is a luxury no real time system has. Initializing from an operational analysis means starting from a state estimate produced under a deadline, from observations that arrived in time, with whatever gaps that implies. Models trained in the retrospective setting and run in the real time one are being asked to generalize across a distribution shift that has nothing to do with the atmosphere and everything to do with how the data was assembled. Some of the narrowing that Ben Bouallegue and colleagues measured is exactly that.

So what did the forecast models actually win?

They won speed by orders of magnitude, and that is not a footnote, it changes which experiments are affordable. They won the deterministic headline scores at medium range, verified independently, inside the incumbent’s own framework. They won the probabilistic setting once the loss functions were fixed. And they won a real architectural insight, which is that at synoptic scale the atmosphere is learnable to a degree very few people predicted in 2020.

What they did not win is a claim about physical understanding, and nothing in this theme was designed to test one. The skill is concentrated in the smooth, large scale, well observed part of the problem. It is conditional on an initialization pipeline built by the systems being outperformed. It comes with under representation of small scales and of error growth. And the papers that document those limits were mostly written by the people whose model was beaten, which is worth noticing, because it means the critical literature here is unusually specific and unusually well instrumented.

Next episode, we take the generalization claim seriously. Operator learning promised something bigger than one good weather model. Learn the solution map of a family of equations rather than one solution, and transfer across initial conditions, across resolutions, and eventually across equations entirely. That promise has real results behind it and a documented baseline problem sitting next to it. Foundation models for physical systems, and what actually transfers. That is episode three.

There is a promise underneath most of the excitement about physics AI, and it is bigger than any single weather model. The promise is that you can learn the solution map of a whole family of equations rather than one solution, so a trained model transfers to new initial conditions, to new resolutions, and eventually to equations it has never seen. If that works, you do not have a fast weather model. You have something closer to a general purpose simulator for physical systems.

This is episode three of eight, and today we test that promise against what has actually been shown. Last episode ended with Aurora, a single atmospheric model pretrained on a mixture of reanalyses and forecasts and climate simulations, then fine tuned to air pollution and ocean wave tasks where dedicated operational systems are the baseline. That is the reference point for what an Earth system foundation model has to demonstrate. Today we go back to where the idea started and follow it forward, and we keep two things apart that get conflated constantly: what has been shown to transfer, and what has only been shown to fit.

The founding move is operator learning. A neural network normally maps vectors to vectors. An operator maps functions to functions. If you can learn the operator that takes an initial condition function to the solution function at a later time, you have learned the equation’s behavior rather than one trajectory through it.

Two canonical formulations arrived around the same time. Lu Lu and colleagues, DeepONet, in 2019, builds on a universal approximation theorem for operators, with a branch network that reads the input function and a trunk network that reads the query location, and the two combine to evaluate the output function anywhere. What I like about DeepONet is that the function space claim is explicit and theoretical rather than architectural. It says what class of thing is being approximated and proves it can be.

Zongyi Li and colleagues, the Fourier neural operator, in 2020, gets there differently. Parameterize the integral kernel directly in Fourier space. Because the parameters live in spectral space rather than on a grid, the learned operator is resolution invariant. You can train at low resolution and evaluate at high resolution, which is not something a convolutional network on a fixed grid does. They reported large speedups over spectral solvers on Navier Stokes. And two years later that architecture, adapted, became FourCastNet, the first global learned weather model from episode two. The lineage is direct.

The property to hold onto from both is resolution invariance, because it is the reusable one. Accuracy on one grid is a benchmark number. An operator that is defined in function space is a different kind of object, and the difference shows up the moment you change discretization.

It is worth being concrete about why that property is worth wanting. Numerical solvers have a hard coupling between accuracy and grid. Refine the mesh and cost rises, often steeply, and a solution computed on one mesh does not transfer to another without interpolation that introduces its own error. An operator learned in function space breaks that coupling in principle. You can train where data is cheap and evaluate where you need answers. In practice the invariance is approximate and degrades outside the spectral range the training data covered, which is a limitation worth stating plainly, because resolution invariance is often quoted as if it were unconditional. What survives is still substantial. The object being learned is defined independently of the discretization, so questions about it are questions about the physics rather than about the grid.

The third strand is the particle and mesh line. Alvaro Sanchez-Gonzalez and colleagues, Learning to Simulate Complex Physics with Graph Networks, in 2020. Represent the system as particles, connect neighbors into a graph, learn the dynamics as message passing over that graph. It generalizes across materials, across geometries, and to orders of magnitude more particles than it saw in training. That inductive bias, local interactions on a graph, is the one GraphCast later carried onto the sphere. If you want the intellectual genealogy of the model that beat the operational forecast, it runs through fluid simulation on particle graphs, not through weather.

That generalization result deserves a second look, because particle counts are a good stress test. A model trained on a few thousand particles and evaluated on hundreds of thousands is not interpolating within its training distribution in any ordinary sense. What makes it work is that the learned function is local. Each particle’s update depends on its neighbors, and the neighborhood statistics are similar whether the system has three thousand particles or three hundred thousand. That is a specific and checkable reason for generalization, tied to the structure of the physics rather than to model capacity, and it is the kind of explanation I want attached to every transfer claim in this field. When a model generalizes, the interesting question is what invariance it exploited.

Now the foundation model claim proper. Tung Nguyen and colleagues, ClimaX, in 2023, pretrains on heterogeneous CMIP6 climate simulations using variable tokenization and aggregation, then fine tunes to forecasting, downscaling, and climate projection, with different variables and different resolutions at each stage. It is the first paper to argue that Earth system tasks share a backbone, and it fixed the shape of the transfer evaluation that later work uses.

Michael McCabe and colleagues, Multiple Physics Pretraining, ran the most direct test I know of. Pretrain one transformer on several distinct partial differential equation systems at once through shared normalized field embeddings. Then transfer to physics that was not in the pretraining set. Does it beat training from scratch? Yes, in the low data regime. That is the honest form of the result, and I want to dwell on the qualifier rather than skip it, because the qualifier is the finding. Pretraining across physics buys sample efficiency. It gets you to a given accuracy with less data on the new system. It does not necessarily raise the accuracy ceiling once you have plenty of data on the new system. Those are different claims with different consequences, and only one of them is what people usually hear.

Maximilian Herde and colleagues, Poseidon, scales that idea with an operator transformer and a training scheme that exploits the semigroup structure of time dependent equations, which is a lovely detail. If the solution operator advances time, then applying it twice should be the same as advancing twice as far, and that structure lets you generate far more training pairs from the same trajectories. They report large sample efficiency gains on unseen equations. It is the strongest quantitative transfer result in operator learning, and it suggests the useful pretraining signal is time evolution structure rather than any particular physics.

So there is real transfer here. Now the correction, and it is a serious one.

Nick McGreivy and Ammar Hakim, in 2024, reviewed the literature claiming machine learning speedups over numerical solvers for fluid related partial differential equations. Not one paper, the literature. What they found is that the large majority of those comparisons run against baselines weaker than the domain standard. A coarse solver, an old scheme, an implementation not tuned the way a practitioner would tune it. And separately, that reporting biases suppress negative results, so the published record is not a sample of what was tried. Their conclusion is that a large fraction of the reported speedup does not survive a fair comparison.

I have opinions about why this happens, and none of them require anyone to be acting in bad faith. Implementing a strong numerical baseline is specialist work. It is slow, it is unglamorous, it makes your result look worse, and reviewers rarely demand it because reviewers in machine learning venues frequently cannot evaluate it. Meanwhile the incentive gradient points straight at the headline. The fix is structural rather than moral: the baseline has to be built by someone who wants it to win.

The other thing McGreivy and Hakim’s review does is make the physics AI literature legible as a literature. When you read papers one at a time, each speedup is plausible. When you read the distribution, you can see the selection effect. That is a general lesson for reading this whole field, and it is the reason the evaluation episode is the hinge of this series.

The last entry in this theme reframes what the bottleneck even is. Ruben Ohana and colleagues released The Well, fifteen terabytes of numerical simulations across sixteen physical systems, from biological flows to supernova explosions, with a common interface and baseline results. That is not a model. It is the substrate that makes cross physics transfer claims testable at all, and it is the most consequential artifact in this line for exactly that reason. The bottleneck moved from architecture to data, and once it does, datasets become the thing worth building.

Which brings me to the vocabulary problem, and this one matters because it is where a lot of hand waving hides. The phrase foundation model names two different things in this area. One is a pretrained backbone with demonstrated downstream transfer, where you can point at a task the model was not trained for and show that reusing the pretrained representation beats training from scratch. The other is a large task specific simulator with a label attached. Only the first is a testable claim. The second is a description of size.

There is a practical test you can apply while reading. Ask what the model was fine tuned to do, what the baseline on that downstream task was, and whether the downstream task differs from pretraining in kind or only in degree. Different lead times are degree. Different variables on the same grid are degree with a bit of kind. A different physical domain, evaluated against the production system that domain already has, is kind. The further a paper’s evidence sits from that last case, the more the word foundation is describing training set size rather than a demonstrated capability.

Aurora is the case that argues the first version credibly, because the fine tuned tasks include air pollution and ocean waves, and the baselines are dedicated operational systems rather than other neural networks. Beating a specialized production system on a task outside your pretraining objective is the kind of evidence the phrase is supposed to imply. Most systems described as physics foundation models have not shown that, and many have not attempted it.

The low data regime qualifier is also where I think the practical value of this line actually sits, and I do not mean that as a downgrade. Most physical sciences are permanently in the low data regime. A planetary spectrometer returns one observing campaign, not a benchmark corpus. A laboratory plasma experiment produces a few hundred shots. A field campaign happens once, in the weather it happened to get. If pretraining across many simulated systems buys sample efficiency on a new system with two hundred examples, that is worth more to those fields than a small accuracy gain on a system with two million examples. The framing of foundation models tends to emphasize the asymptotic claim. The measured result is about the small data end, and the small data end is where most science lives.

So what does this theme establish?

Resolution invariance is a real, reusable property with two clean formulations behind it, and it is the closest thing operator learning has to a general result. Multi physics pretraining transfers, measurably, in the low data regime, and the mechanism appears to be time evolution structure rather than shared physics per se. Earth system pretraining transfers across variables and domains, demonstrated once, convincingly, against operational baselines. And a large fraction of the surrounding speedup literature does not survive a fair comparison, which is a fact about the field’s evidence base rather than about any individual result.

What it does not establish is that a model trained on many physical systems has learned physics in any sense stronger than a reusable representation. Nothing in this theme tests whether the model can apply a conservation law it was not shown, or extrapolate outside the regime it was pretrained on, or reason about a system whose governing equations differ structurally rather than parametrically. Those tests exist, and they are the subject of episode five, and they mostly come back negative.

Before we get there, we need to talk about measurement, because every claim so far has been reported as a number, and the numbers are where the field is weakest. Next episode, evaluation, uncertainty, and physical validity. Where the headline score misleads, why aggregate skill and extreme event skill come apart, and what forecast verification worked out decades before machine learning arrived and then had to work out again. That is episode four.

In the autumn of 2023 a windstorm crossed the English Channel and did serious damage across northwest Europe. Afterwards, Andrew Charlton-Perez and colleagues ran the obvious comparison. They took the learned weather models that had by then been reported to beat the operational physics based system on aggregate scores, and they asked how those models did on that one storm. The answer was split. The learned models captured the synoptic evolution well. They knew where the low was going and roughly how deep it would get. What they under-represented was the near surface wind extremes, which is to say the specific quantity that broke things.

That is the shape of this entire theme. An aggregate score says one thing. The property a user actually needs says another. And the gap between them is structural, with a literature going back three decades that mostly predates machine learning and that machine learning largely had to rediscover.

This is episode four of an eight part walk through a literature map on physics AI and causal world models. Evaluation, uncertainty, and physical validity. I put this theme fourth rather than last on purpose, because it is the theme that decides whether the other seven are true. Every claim in the previous three episodes arrived as a number. This episode is about what those numbers can and cannot certify.

Start with Allan Murphy, writing in 1993, asking what a good forecast even is. His answer separates three things that get conflated constantly. Consistency is whether the forecast matches what the forecaster actually believes. Quality is correspondence between the forecast and what was observed. Value is the benefit the forecast produces for whoever used it. Murphy’s point, and it is a sharp one, is that these three can move in opposite directions. A forecaster who knows the truth but hedges toward the safe answer sacrifices consistency and often improves quality as measured by a squared error score. A forecast can improve in quality and lose value if the improvement lands in a regime nobody makes decisions in. Thirty years later the learned forecasting literature reports headline scores, which is one of Murphy’s three axes, measured partially. Almost every argument I have watched about whether an AI weather model is better than the operational system turns out on inspection to be an argument about which of Murphy’s three goodnesses the two sides are talking about.

Then there is the question of what a score is even allowed to do. Tilmann Gneiting and Adrian Raftery worked out the formal theory in 2007. A scoring rule is strictly proper when it is optimized only by reporting your true predictive distribution, which means you cannot improve your score by shading your forecast toward something you do not believe. The continuous ranked probability score, the logarithmic score, and the Brier score all fall out of that one framework as instances. This matters because the alternative is a score that rewards hedging, and a system optimized against a hedgeable score will hedge. Root mean square error is the canonical case. It is minimized by the conditional mean, so a deterministic model trained to minimize it learns to produce something like the average of the possible futures, which is smoother than any real atmospheric state and which is exactly the behavior that shows up later as missing extremes.

The same year, Gneiting, Fadoua Balabdaoui, and Raftery published the companion idea. Maximize sharpness subject to calibration. Calibration is whether the stated probabilities are honest, so that events you call thirty percent happen about thirty percent of the time. Sharpness is how concentrated the forecast is, how much it actually commits. A climatological forecast is perfectly calibrated and useless. A confident wrong forecast is sharp and dangerous. You want the most concentrated distribution that remains honest, and the two properties have to be measured separately, with tools like the probability integral transform histogram, because a single number cannot tell you which one you lost. That distinction is not a technicality. It is the axis along which the current arguments about diffusion based ensemble forecasting are actually being fought, and it is invisible to anyone reading a leaderboard.

Now the spatial problem. Beth Ebert’s 2008 review of neighborhood verification is the canonical treatment of what people call the double penalty. Imagine a forecast that predicts a squall line perfectly in structure and intensity but displaced by thirty kilometers. Point by point matching penalizes it twice, once for predicting rain where none fell and once for missing rain where it did. A blurred forecast that puts weak rain everywhere avoids both penalties and scores better while being less useful to anyone deciding whether to close a road. Ebert’s contribution is to organize the neighborhood based methods by the decision model each one implies, which reframes verification as a question about the user rather than about the field. It is also the reason you should be suspicious when a learned model’s advantage over a physical model is largest in exactly the variables where smoothing pays.

Which brings us to the papers that measure the gap directly. Noah Brenowitz and colleagues took learned models that the field had already declared roughly equivalent, because their deterministic scores matched, and asked what happens when you use them the way forecasts are actually used, as the seed of an ensemble. Models with equal root mean square error turned out to have substantially different spread and different probabilistic skill. They propose a lagged ensemble protocol that exposes this cheaply, without retraining anything. The result to hold onto is that deterministic scores do not rank models the way probabilistic use ranks them, demonstrated on the specific models whose equivalence the field had accepted.

Then the tails. Sun and colleagues ran what I think is the cleanest experiment in this theme. Take the most intense tropical cyclones out of the training set entirely, then test whether learned models can forecast storms of that intensity. They cannot. The models fail to extrapolate to unseen intensity even though the governing physics has not changed at all between the storms they saw and the storms they did not. That converts a suspicion into a mechanism. Extreme event failure is not vaguely about smoothing or about loss functions. It is out of distribution generalization failure, isolated by experiment. And it has a direct implication for the counterfactual question this whole series is about, because a model that cannot extrapolate to an unseen intensity of a phenomenon it has seen thousands of times is not a model you can interrogate about a world that differs from the one it was trained on.

Gabler and colleagues in 2026 do the systematic version, testing learned models on extremes across variables and regions and quantifying how much of the reported skill advantage survives when you restrict the evaluation to the tails. Much of it does not. The value of that paper is that it converts a widely repeated intuition into a variable by variable number, which is the form a claim has to be in before you can act on it.

Now the subtlest one, and my favorite in this theme. Landsberg and colleagues ask what happens when the climate a model was trained on drifts away from the climate it is forecasting. Learned models are trained on historical reanalysis. The atmosphere is not stationary. So the evaluation period differs from the training period in distribution, and part of any measured skill change is attributable to that drift rather than to the model. The consequence is uncomfortable. It means a benchmark number is not a fixed property of a model, it is a property of a model and an evaluation window jointly, and the confound grows silently over time. No standard protocol currently controls for it. When you see two systems compared across different evaluation years, some unknown fraction of the difference belongs to the calendar.

Against that background, WeatherBench 2, from Stephan Rasp and colleagues, is the piece of infrastructure that made the field legible at all. Fixed ground truth, fixed baselines, headline deterministic and probabilistic scores, every major learned model reported against the operational Integrated Forecast System on the same footing. Before it, cross paper comparison was mostly impossible, because everyone chose their own baseline and their own truth dataset. After it, comparison is routine. It is also, and this is the part worth sitting with, an artifact whose choices now shape what the field optimizes. Benchmarks reward what they measure. A scoreboard that leads with deterministic scores at fixed lead times produces models that are excellent at deterministic scores at fixed lead times. That is not a criticism of the benchmark, which is careful and explicit about its own limits. It is an observation about what happens when a community gets a shared objective.

Paul Ullrich and colleagues wrote the 2025 statement of what a real evaluation would have to include before a learned Earth system model can be trusted. Physical consistency checks, out of sample and out of distribution testing, and independence between the people who built the model and the people who evaluate it. That last requirement is the one I would underline. The field’s evidence base is largely self reported, against baselines and initializations and truth datasets chosen by the developers. Ullrich is the community saying out loud that this is not sufficient, which is a normal stage for a field to pass through and a healthy one.

Nick McGreivy and Ammar Hakim make the same argument with numbers, in the fluid dynamics setting rather than the weather setting. They reviewed the literature claiming machine learning speedups over numerical solvers for partial differential equations and found that the large majority compare against baselines weaker than the state of the art in the target field, and that reporting biases suppress the negative results that would balance the record. Their finding generalizes past fluids to any claim of the form a learned model beats the domain standard method, which is the claim structure this entire explorer is built around.

So what does the theme leave open? Two gaps, and both of them are the kind that would take real work rather than a clever idea.

The first is attribution. There is no benchmark that decomposes a learned model’s error into the part contributed by the observing system, the part contributed by the assimilation, and the part contributed by the forecast model. Which means the model is wrong and the analysis is wrong in this region remain indistinguishable in every score anyone reports. That is the loop closing back to episode one. The sensor boundary is not just a philosophical point about what a model has seen. It is a live measurement problem that makes every reported error number partially uninterpretable.

The second is statistical. Extreme event evaluation has no agreed power analysis. Nobody states how many events are required before an improvement in tail behavior counts as a result rather than as a sample. We have a field that has correctly identified extremes as the place where learned models are weakest, evaluating on event counts that are small by construction, without a convention for how small is too small. That one is fixable, and someone should fix it.

Here is where that leaves the argument. We have three episodes of evidence that learned models predict physical systems well, and one episode establishing that the measurements behind that evidence are weaker than they look, particularly in the tails, particularly under distribution shift, and particularly when the evaluator and the developer are the same person. None of this says the models are bad. Several of them are genuinely excellent. It says the evidence supports a narrower claim than the one usually made.

And it sets up the question the rest of the series is about. Everything so far has been about correspondence between a prediction and an observation. Next episode we ask whether any of it implies that the model has learned the physics. Probes of learned representations, models that predict orbits perfectly and cannot apply Newtonian mechanics, symbolic recovery of governing equations, and the difference between a model that fits a system and a model that understands it. That is episode five.

There is a story people tell about machine learning and physics, and it goes like this. Johannes Kepler stared at Tycho Brahe’s planetary observations and extracted three empirical laws describing how planets move. Kepler could predict. He could not explain. Then Isaac Newton came along and produced the law underneath, gravitation, from which Kepler’s rules fall out as consequences, and which also explains falling apples and tides and the orbits Kepler never measured. The hopeful version of the story says that today’s predictive models are at the Kepler stage, and that scale or the right probe will take them to Newton.

Keyon Vafa and colleagues ran that story as an experiment. They trained models on orbital trajectories. The models predict orbits accurately. Then they adapted those models to new physics tasks that require actually applying Newtonian mechanics, and measured what inductive bias transferred. The answer was that the models behaved as though they had learned task specific heuristics rather than the law. Excellent at the trajectories they were trained on. Unable to carry the mechanics forward.

This is episode five of an eight part walk through a literature map on physics AI and causal world models. Causal structure and mechanistic understanding. The theme where the central question of the series gets asked directly. Does prediction pressure force a model to learn mechanism?

The literature supports both answers, and the reason it supports both is that the word causal is being asked to carry five different claims with five different burdens of proof. It is worth separating them before going further, because almost every confused argument in this space is two people holding different items from this list.

The first claim is that there is a predictive relationship. Variable A helps forecast variable B. That is a statement about correlation structure in a distribution and it is the easiest to establish.

The second is that the model contains an internal representation encoding a physical variable. Somewhere in the activations there is something that tracks temperature, or vorticity, or a quantity nobody put in the training data. That is a representation claim, established by probing, and it is not the same as the first.

The third is that a causal structure has been identified. This variable causes that one, in a form you could write down as a graph and test.

The fourth is that the model supports a valid counterfactual. If this had been different, that would have followed.

And the fifth is that the model supports intervention. You can reach in, set something, and read out what actually happens rather than what typically co-occurs.

Those five are ordered roughly by strength, and evidence for one does not transfer to the next. Most claims that a model has learned physics are claims of type two being reported as though they established types four and five.

That ladder is not academic bookkeeping. Consider what each rung licenses in practice. Rung one lets you forecast. Rung two lets you say something about how the model works internally, which is useful for debugging and for trust but not for planning. Rung three lets you attribute, which is what an attribution study of a heatwave needs. Rung four lets you answer what would have happened without the aerosol forcing, or without the reservoir, which is what most climate policy questions actually are. Rung five lets you plan, because planning means evaluating actions you have not taken. When a company or a paper says its model supports counterfactual reasoning about physical systems, the useful response is to ask which rung the evidence sits on, and the answer is usually two.

Start with the inference machinery, because the causal literature for dynamical systems already exists and is explicit about its own assumptions. George Sugihara and colleagues introduced convergent cross mapping in 2012, which detects causal coupling in deterministic dynamical systems by testing whether the reconstructed attractor of one variable can recover the other. The important part of that paper for our purposes is the demonstration that Granger causality, the workhorse of time series causal inference, fails in exactly this regime. Coupled nonlinear dynamics needs different machinery than stochastic time series, and Earth system data is coupled nonlinear dynamics.

Jakob Runge and colleagues built the method the Earth system community actually runs, called P C M C I, a two stage conditional independence approach that controls false positives in high dimensional autocorrelated data where naive conditioning inflates them. It works, and it comes with assumptions: causal sufficiency, meaning no unobserved common driver, stationarity, and adequate sampling. Earth system data routinely violates all three. That is not a criticism of the method, which states its conditions honestly. It is a warning about what a discovered graph from that machinery licenses you to say.

Jonas Peters, Peter Bühlmann, and Nicolai Meinshausen supply what I think is the most useful operational definition available. Causal inference using invariant prediction. The idea is that the conditional distribution of a target given its true causal parents stays invariant across environments, while spurious relationships shift. So you turn causal discovery into a hypothesis test across environments, and you get confidence statements about the recovered parent set. Invariance is the property that survives distribution shift, which makes it the only definition that helps when a model is deployed outside the climate it was trained on. Bernhard Schölkopf and colleagues then make the programmatic argument in Towards Causal Representation Learning: the variables over which causal structure is defined have to themselves be learned from low level observations before any of this applies. That paper is the clearest statement of why the model learned physics is a representation claim rather than a prediction claim, and what it would take to establish it.

Now the evidence, which cuts both ways, and I want to give both sides properly because the field tends to cite whichever half suits the argument.

On the side of mechanism, the strongest current result comes from Lean and colleagues probing GraphDOP. GraphDOP, recall from episode one, is trained only on instrument measurements, with no reanalysis and no physical model in the loop. Probing its internal activations recovers structure aligned with physical variables it was never given and never directly observed. A model that has only ever seen radiances carries something inside it that tracks the fields underneath. That is a genuine result and it is the best evidence available that prediction pressure induces physically meaningful internal structure.

Baño Medina and colleagues take a different route, running gradient based sensitivity analysis on a learned model during Cyclone Xynthia, a well studied explosive cyclogenesis case, and comparing the influence patterns the model reveals against the physical precursors the meteorological literature already identified. Asking the model what it is using, rather than checking what it output. And Tempest and colleagues in 2026 build the tooling to push this further, inspecting internal circuits and attributing predictions to identifiable internal computations rather than to input regions. That is the move from saliency maps toward circuit level evidence, which is the kind of evidence a mechanism claim would actually require.

On the other side, the negative results are equally concrete and in some ways sharper, because they test properties that physical models satisfy by construction.

Tobias Selz and George Craig asked whether a learned weather model exhibits the butterfly effect. Apply a tiny localized perturbation to the initial state and watch how error grows. In the real atmosphere, and in physical models that resolve the relevant scales, error injected at convective scales propagates upscale, and that upscale propagation is what sets the intrinsic predictability limit. The learned model does not do it. Forecast skill is high. The error growth dynamics are wrong.

Gregory Hakim and Sanjit Masanam do the systematic version, subjecting a learned model to classical dynamical meteorology experiments where the answer is already known: steady tropical heating, extratropical cyclone development, geostrophic adjustment. The model reproduces some canonical responses and gets others qualitatively wrong. What I value most about that paper is methodological. It establishes the right instrument for mechanism claims, which is not held out forecast skill but controlled experiments the physics already has answers to. If you want to know whether a model knows something, do not test it on more of what it was trained on. Perturb it in a way the training distribution never contained and check the response against theory.

Then the direct tests, run outside weather, where the experiment can be clean.

We opened with Vafa. The orbital mechanics case is the sharpest because the ground truth is unambiguous and the model’s prediction quality is not in dispute. Bingyi Kang and colleagues run the complementary experiment on video generation. Train diffusion video models on two dimensional mechanics with an unlimited supply of simulated data, so data quantity is explicitly not the bottleneck, and separate three kinds of generalization: within the training distribution, combinatorial recombination of seen elements, and genuinely out of distribution. In distribution generalization is perfect. Combinatorial generalization improves measurably with scale. Out of distribution generalization fails. And the failure mode is diagnostic rather than mysterious. The models retrieve the nearest training example, prioritizing color first, then size, then velocity, then shape, which is a ranking of surface attributes and not a physical hierarchy at all. Unlimited clean data on simple mechanics did not produce the rule.

Pim de Haan, Dinesh Jayaraman, and Sergey Levine add the result from imitation learning, and it is the one I find most unsettling. Behavioral cloning fits a discriminative model with no access to the causal structure of how the expert and the environment interacted. The consequence is causal misidentification: giving the policy more information can make it worse, because confounded cues that correlate with expert actions are often easier to fit than the actual causes. More observations, worse policy. Resolving it requires targeted intervention, either interacting with the environment or querying the expert. Passive data will not do it, no matter how much of it there is.

Kori and colleagues, in 2026, write down the formal version of the gap. A unifying perspective on causal world models, setting out what a world model has to satisfy before its rollouts can be read as interventions rather than as conditional predictions. That distinction is the whole ballgame. A rollout answers the question, given what I have seen, what typically follows. An intervention answers, if I set this variable to this value, what follows. Those coincide only under conditions that a learned model has no reason to satisfy and that nobody currently checks.

It is also worth noticing what the Kepler to Newton story quietly assumes. Kepler did not become Newton. A different person, working eighty years later, with different tools and a different question, produced the law. The empirical regularities were necessary input and they were not sufficient, and the step from one to the other was not more data or more staring. It was a change in what was being asked. The optimistic reading of the analogy treats the transition as something a predictive system undergoes on its own given enough scale. Nothing in the history supports that, and nothing in the experiments does either. The more defensible reading is that learned models are useful Kepler stage instruments, producing regularities at a rate no human could, and that the Newton step still belongs to whoever designs the experiment that discriminates between explanations.

So where does the theme land? It leaves one gap open that I think is the most tractable important experiment in this entire explorer. We now have probing results showing that learned forecasters contain internal representations of physical variables. We do not know which of those representations are load bearing for the model’s skill and which are correlates that probing can find but ablation would not disturb. Probing tells you information is present and decodable. It does not tell you the model uses it. The experiment is ablation and causal mediation on internal representations of a learned weather model, and it is a matter of engineering effort rather than of new theory.

Put the two halves together and the honest summary is narrower than either camp’s version. Learned models develop internal structure aligned with physics nobody handed them, which is real and interesting. They fail dynamical tests that physical models pass by construction. And where the Kepler to Newton premise has been tested directly, on orbits and on simulated collisions, the model that predicted well did not carry the law. Prediction pressure produces representation. There is no current evidence that it produces mechanism in the sense required for counterfactuals.

Which raises the practical question. If a model cannot be trusted for counterfactual reasoning, what can it be trusted for? Quite a lot, as it turns out, and the answer runs through the people who actually use forecasts. Next episode, forecast to decision. Value versus skill, the users who need probabilities rather than point predictions, and the surprising finding that the link between a better forecast and a better outcome is weaker and stranger than anyone assumes. That is episode six.

Suppose you grow fruit, and a frost overnight will destroy the crop unless you run the wind machines and the heaters. Running them costs you something. Call that cost C. Losing the crop costs you rather more. Call that loss L. The only number that matters to your decision is the ratio between them. If protection costs one tenth of what the crop is worth, you should protect whenever the forecast probability of frost exceeds one tenth, because over a season that rule loses you less than any other rule. If protection costs nearly as much as the crop, your threshold is high and you protect only when the forecast is nearly certain.

Now here is the part that matters for this series. Two forecast systems can be compared, and which one is better for you depends on where your ratio sits. Not on which one has the lower error. On your ratio.

David Richardson published the quantitative version of that argument in 2000, computing the relative economic value of the European ensemble prediction system against the deterministic forecast across the full range of cost to loss ratios. The ensemble is more valuable for most users, including at lead times where its deterministic scores are unremarkable. Which means the ranking of two forecast systems is not a property of the forecasts. It is a property of the pair consisting of a forecast and a user, and it flips as you walk along the range of users.

This is episode six of an eight part walk through a literature map on physics AI and causal world models. Forecast to decision. A forecast has no value until something is decided with it, and the map from score to value is not monotone. You can reduce prediction error and change no decision at all. You can reduce prediction error and make decisions worse.

It is worth being precise about what that curve is, because it is simpler than its reputation. For a given user, you take three numbers. The expense of always acting on climatology, which is the naive strategy. The expense of acting on a perfect forecast, which is the floor. And the expense of acting on the forecast system under test, following the optimal threshold rule for that user’s ratio. Relative economic value is where the system sits between the naive strategy and the floor, scaled so that climatology is zero and perfection is one. Sweep the ratio from near zero to near one and you get a curve rather than a number. Two systems produce two curves, and the curves cross. Everything contentious about comparing forecast systems is visible in where they cross and how wide each one’s useful range is, and none of it is visible in a single score.

Tim Palmer extended the argument in 2002 across time scales, from weather out to seasonal and climate horizons, and made the stronger claim that the probabilistic forecast is the object carrying decision value. At a two week horizon the deterministic trajectory is a convenient fiction. At a seasonal horizon it has no defensible meaning at all, and the only honest product is a distribution. That is worth holding next to episode four, where we saw that the field’s headline scores are largely deterministic. The evaluation culture and the decision literature have been pointing in different directions for twenty years.

The value framing has been run at industrial scale, and the results are instructive about where value actually lives. Wang and colleagues in 2016 ran production cost simulation over four hundred and eighty scenarios on a standard one hundred and eighteen bus test system, configured to emulate the Midcontinent, California, and New England balancing authorities, and priced what a given improvement in wind power forecast accuracy is worth at each market timescale. The finding is that the value tracks generation mix and available ramp capability rather than forecast error alone. Improve the forecast for a grid that cannot ramp fast enough to exploit the information and you have created very little. The same improvement delivered to a system with flexible generation is worth substantially more. So the value of a forecast improvement belongs to the system it lands in, not to the forecast.

Matte and colleagues in 2017 attack a different assumption inside the same framework. The cost to loss ratio treats the decision maker as risk neutral, meaning they are indifferent between a certain loss and a gamble with the same expected loss. Operational users are not risk neutral. A reservoir operator does not weigh a small chance of catastrophic downstream flooding the way expected value says they should, and they are right not to. Matte and colleagues value ensemble streamflow forecasts under a constant absolute risk aversion utility instead, and the valuations change. The point generalizes past hydrology. The standard forecast value metric has a preference assumption baked into it, so a system tuned to maximize it is optimizing for a decision maker who may not exist.

That is the meteorological tradition. Machine learning arrived at the same uncomfortable place from the other direction, independently, and gave it a different name.

Priya Donti, Brandon Amos, and Zico Kolter published task based end to end model learning in stochastic optimization in 2017. The setup is that you have a prediction feeding an optimization problem, which is the structure of most operational decisions. The conventional approach trains the predictor on prediction error and then hands its output to the optimizer. Donti and colleagues differentiate through the optimization problem itself, so the predictive model is trained on the resulting task cost. Demonstrated on load forecasting for electricity scheduling and for battery storage. And the headline result is the one that should unsettle anyone who reports benchmark scores: a model with worse predictive accuracy can produce strictly better decisions. Not equal. Better. Because the errors it makes are in places the optimizer does not care about, and the errors the accurate model makes are in places it does.

Adam Elmachtoub and Paul Grigas gave the framing a loss function with theory behind it in the same year. Their smart predict then optimize loss measures decision regret rather than prediction error, and they supply a convex surrogate with consistency guarantees for linear optimization problems. That is why it became the reference formulation. It is a tractable objective rather than an argument.

Now the correction, because a decade of enthusiasm for decision focused learning needed one. Liu and colleagues in 2026 characterize when a plain accuracy trained predictor is already decision optimal, and when the decision focused objective is genuinely necessary. That work matters because train on the decision had started to circulate as an unconditional recommendation, and it is not one. The gain is conditional. Knowing the conditions is what turns it from a slogan into a method.

Here is the tension that this theme leaves standing, and I do not think anyone has resolved it.

The decision focused learning literature works on problems that are small and well specified. A scheduling problem with a known cost structure. A shortest path problem with a known objective. Under those conditions you can differentiate through the decision because you can write the decision down. Operational weather decisions are neither small nor well specified. There is no single objective function for a national meteorological service. Its outputs feed aviation, agriculture, energy trading, flood response, emergency management, and the public, and those users have different cost to loss ratios, different risk attitudes, different lead time requirements, and objectives that conflict with each other. Which decision do you train on?

That question has no clean answer, and I want to say plainly that it may not have one. But it does have a partial answer that the field is not using, and this is the practical thing I would take from this theme. The relative economic value framework does not require you to pick a single user. It sweeps the entire range of cost to loss ratios and reports value as a curve. You can compute that curve for a learned model exactly as easily as for a physical one. Richardson did it in 2000, with the computing resources of 2000. It would settle a large fraction of the current arguments about whether learned ensembles are better than operational ones, because it would show for which users they are and for which users they are not.

Almost none of the learned weather literature reports the downstream value of its own improvement. That absence is the gap in this theme, and it is not a research gap in the sense of requiring an invention. The instrument exists, it is twenty five years old, it is well understood, and nobody is picking it up. Reported skill improvements sit at some unknown place on a curve that nobody plots.

There is a second observation worth making, which connects this theme back to episode four. Recall the double penalty, where a blurred forecast scores better than a sharply placed one that is displaced. From a decision perspective the blurred forecast is often worse, because a user acting on a spatially specific threshold gets nothing from a field that has been smoothed toward its mean. The verification literature knew this. Beth Ebert’s neighborhood methods are explicitly organized by the decision model each one implies. So the tools for connecting scores to decisions were built inside the verification community, and they are the tools that got left out when the benchmark culture formed around aggregate deterministic error.

And there is a third, which is about what improvement even means at this point. A forecast that is more accurate by half a percent on a global average, for a user whose threshold sits in the tail of the distribution, may be worse for them than the previous system if the improvement came from smoothing. That is not a hypothetical. It is the mechanism connecting the missing extremes result from episode four to the value question here. Aggregate improvement and user relevant improvement can have opposite signs, and only one of the two gets reported.

So what does this theme establish? That forecast value is user specific and computable, with a framework in place since 2000. That the value of an improvement belongs to the system that acts on it, measured in money on named electricity systems. That the standard value metric encodes a risk neutrality assumption operational users do not satisfy. That training on the decision objective can beat training on accuracy, demonstrated end to end, and that the gain is conditional in ways now characterized. And that the learned weather literature has adopted none of it.

The experiment I would like to see is not complicated. Take two systems already evaluated head to head on a public benchmark, pick a handful of variables with genuine operational users behind them, ten meter wind, two meter temperature, precipitation, and compute relative economic value curves for both across the full ratio range at several lead times. Report where the curves cross. Report the ratio ranges where the learned system wins, where it loses, and where the difference is inside the noise. That is a paper someone could write this quarter from artifacts that already exist, and it would convert a long-running argument conducted in aggregate scores into a statement about which users are affected and by how much. The reason it has not been written, as far as I can tell, is that the two communities that would have to write it together do not share a venue.

The through line for this series is that skill is not evidence of value, in the same way that prediction is not evidence of mechanism. Each of the last three episodes has been a version of the same structure. A number gets reported. The number supports a narrower claim than the one made. And the instrument that would support the wider claim already exists in a neighboring literature that the field has not read.

Next episode we go a step further, to the case where the model is not just informing a decision but is the thing being planned inside. World models, planning, and physical control. Model based reinforcement learning worked out the failure mode precisely, which is that a planner will find the exact region where your model is wrong and optimistic and it will go there. We will look at what makes a learned model safe to act inside, at the two real deployments on physical systems, a stratospheric balloon and a tokamak plasma, and at what conditions those deployments needed that a continental atmosphere does not supply. That is episode seven.

Let an optimizer plan inside a learned dynamics model and watch what it does. The optimizer is not searching for good actions in the world. It is searching for good actions according to the model, and those are different objectives. Anywhere the model is wrong in the optimistic direction, predicting an outcome better than reality would deliver, the planner will find it and go there, because that region is exactly where the objective looks best. The failure is not a bug in any particular architecture. It is what optimization does to an imperfect model, and the model based reinforcement learning community named it early and precisely. They call it model exploitation.

That single dynamic organizes this entire theme, and it is the reason this episode is not about which model fits the dynamics best.

This is episode seven of an eight part walk through a literature map on physics AI and causal world models. World models, planning, and physical control. The place where a learned model stops informing a decision and starts being the thing decisions are made inside.

Start with the fix, because it is not what you would guess. Kurtland Chua, Roberto Calandra, Rowan McAllister, and Sergey Levine published the method usually called PETS in 2018, and the central move is epistemic. They use an ensemble of probabilistic dynamics models to separate two kinds of uncertainty that get conflated. Aleatoric uncertainty is noise inherent in the system, which more data will not remove. Epistemic uncertainty is ignorance, which more data will remove, and which tells you where the model does not know what it is talking about. Plan with trajectory sampling that respects that distinction and you match the asymptotic performance of model free methods with orders of magnitude fewer samples. The lesson worth carrying out of that paper is that what makes a learned model safe to plan against is its uncertainty representation rather than its one step accuracy. A model that is slightly worse at prediction and honest about where it is ignorant is more useful for control than a model that is slightly better and uniformly confident.

There is a corollary that took the field a while to accept. Rollout length is a safety parameter. Every step you roll forward inside a learned model compounds its error, and the theoretical bounds on how much a policy improved inside the model improves in the real world degrade with horizon. So the practical recipe that emerged is short rollouts from real states, repeatedly, rather than long imagined trajectories from a single starting point. That is a strange conclusion if you came in thinking of a world model as a simulator. A simulator is valuable in proportion to how far it can run. A learned dynamics model used for control is trustworthy in inverse proportion to how far you run it, and the engineering discipline is knowing where to cut.

The architectural origin point for this line is David Ha and Jürgen Schmidhuber’s World Models in 2018, which is where the phrase this explorer is partly named after entered circulation. Compress observations into a latent representation, learn a recurrent forward model of the latent dynamics, then train a small controller entirely inside that learned model and transfer it back to the real environment. Train the policy in the model’s imagination. It is a clean idea and it made the subsequent literature possible.

Then the strongest version of the argument. Julian Schrittwieser and colleagues built MuZero, which learns a latent dynamics model trained only to predict reward, policy, and value, and then runs tree search inside it. It matches AlphaZero at Go, chess, and shogi without ever being told the rules of those games, and sets a new state of the art on Atari at the same time. If you want a demonstration that planning inside a learned model can reach superhuman performance in a domain whose mechanics were never supplied, that is the paper.

It is also, read carefully, a statement of its own preconditions, and this is the part that gets skipped when it is cited in physical settings. MuZero requires a resettable environment, so you can return to a state and try something else. It requires cheap exact rollouts, so you can afford millions of them. And it requires a scalar reward, so the objective is unambiguous. Go supplies all three. Danijar Hafner and colleagues push the same architecture much further with the third version of Dreamer, a single world model and actor critic with fixed hyperparameters across more than one hundred and fifty tasks, spanning continuous control, Atari, and Minecraft, including collecting diamonds from scratch, which had resisted everything before it. That is the current reference point for how far world model planning generalizes when the environment is simulated, well instrumented, and resettable.

A continental scale atmosphere supplies none of those three conditions. It is not resettable. Its rollouts are neither cheap nor exact. And there is no scalar reward, as episode six established at length, because the objective is a family of conflicting user specific objectives. So when the reinforcement learning results are cited as evidence that we are close to planning inside learned physical world models, the honest reading is that they establish what is possible under conditions that the target domain does not meet.

Philipp Holl, Vladlen Koltun, and Nils Thuerey provide the cleanest result for systems governed by partial differential equations. They use a differentiable solver to train a predictor corrector control network end to end, planning through the solver rather than around it, and achieve long horizon control of fluid systems. That is a genuine demonstration of learned control on the kind of dynamics we care about. It is also a useful upper bound rather than an operating point, because it works with full state access and an exact simulator. Both of those are things the open atmosphere does not give you, and episode one was largely about why.

Now the two real deployments, and I want to spend time on what each of them required rather than on what each achieved, because the requirements are the transferable content.

Marc Bellemare and colleagues published autonomous navigation of stratospheric balloons using reinforcement learning in Nature in 2020. The controller steers a balloon by choosing altitude, riding wind layers that move in different directions, and it outperformed the hand engineered controller in flight over the Pacific under real wind uncertainty. That is learned control acting on a real atmospheric system, which makes it the closest thing in this literature to the thing people imagine when they talk about AI controlling weather. Notice how it works. The controller does not modify the atmosphere in any way. It exploits an atmospheric structure that already exists, selecting among wind layers rather than changing them. The action space is a single scalar, altitude. The state is a balloon. Success is measured by staying near a location. Every one of those properties is what made the problem tractable.

Jonas Degrave and colleagues published magnetic control of tokamak plasmas through deep reinforcement learning in 2022. Train a controller in a physics simulator of the TCV tokamak, deploy it on the real device, and sustain and shape plasma configurations including ones that had not been demonstrated before. As a sim to real result on a physical system this is the strongest in the literature and I do not want to undersell it. It depended on three things. A trusted simulator, meaning decades of plasma physics encoded in a model good enough to train against. Dense diagnostics, meaning the plant is instrumented to a degree that few physical systems are. And millisecond actuation, meaning the control loop closes fast enough to correct model error before it compounds.

That last one is the general principle and it deserves stating on its own. The two successful deployments both closed their loop faster than model error could accumulate. Model exploitation is bounded when you can observe the consequence of an action and correct quickly. It is unbounded when you act and then wait. For a continental atmosphere, the observation is partial, the actuation is undefined, and the feedback horizon is days. That combination inverts every condition that made the balloon and the tokamak work.

The third transfer argument in circulation comes from autonomous driving, where the perceive predict plan stack demonstrably works at scale, and its own literature is the best check on it. Mayank Bansal, Alex Krizhevsky, and Abhijit Ogale reported in ChauffeurNet that thirty million real expert driving examples were not sufficient on their own to produce a policy that could drive a real vehicle, even with a perception system upstream and a controller downstream. What was required was synthesizing failures the expert never made, including collisions and road departures, and adding explicit losses penalizing them. Expert data alone taught the model what good driving looks like and did not teach it what to do when things go wrong, because the expert never went wrong. The fix was to manufacture counterfactual experience.

Which is the same finding as the imitation learning result we met in episode five. Pim de Haan, Dinesh Jayaraman, and Sergey Levine showed that a policy cloned from demonstrations suffers causal misidentification, and that it can get worse as its observation set grows, because confounded cues correlating with expert actions are easier to fit than the actual causes. Only interventional data disambiguates. Two independent lines, one from industrial engineering practice and one from causal analysis, arriving at the identical conclusion: passive observation of expert behavior does not yield a policy, and the missing ingredient is intervention.

Put the theme together and it makes an argument that is more coherent than any individual paper in it.

Planning inside a learned model is real, it is powerful, and it has explicit preconditions. The preconditions are a resettable or fast correcting environment, a bounded and characterized action space, dense instrumentation, and a clear objective. Where those hold, results range from superhuman to operationally deployed. Where they do not hold, the central failure mode is a planner exploiting model error, and the mitigation is representing uncertainty honestly rather than fitting the dynamics better.

The open atmosphere fails the preconditions on every axis, and it is worth saying which axis is worst. It is not model quality. Learned atmospheric models are good, as episode two established. It is that there is no characterized actuation. Nobody has a transfer function from an intervention to an atmospheric response with error bars on it. The balloon case is instructive precisely because it sidesteps this: it acts on a balloon, not on the sky.

So the gap I would flag here is narrower and more useful than a general statement that we are far away. Every result in this theme that touches a physical system had a characterized actuator and a fast loop. The research question is what a control problem looks like when the actuation is uncharacterized and the feedback is slow, and whether uncertainty aware planning of the kind PETS established can be extended to that regime with any guarantee at all. That question is not being worked on inside the weather machine learning community, and the reinforcement learning community is not aiming at it either.

There is one more thing worth extracting from the two deployments, which is what they imply about evaluation. Neither the balloon controller nor the tokamak controller was validated by showing that its internal dynamics model predicted well. They were validated by running the closed loop on the real plant and measuring what happened. That is a different and much stronger form of evidence than anything in the first six episodes, because the environment gets a vote. A forecast benchmark cannot push back on a wrong model. A plasma can. If there is a single methodological transfer available from the control literature to the physics machine learning literature, it is that closing the loop on a real system is the evaluation that cannot be gamed, and that where you cannot close the loop, you should be correspondingly less confident in what your model knows.

Which brings us to the last episode, and to the claim that sits at the top of this whole literature map. If a learned model can be interrogated about interventions, then it is an instrument for asking what if, which is a different kind of object from a forecast system. Companies are being built on that premise right now. Next episode, intervention and the boundary of evidence. What the strongest claims in this space actually assert, what evidence would settle them, which experiments the literature has already run, and which ones nobody has run yet. That is episode eight, and it is where the whole series has been going.

In 2026 a group published a paper showing that a tropical cyclone can be steered by a small perturbation to the initial conditions. Change a modest amount of the atmospheric state in the right places, run the model forward, and the track moves. A second group did a more sophisticated version, targeting the perturbation at the dynamically unstable directions of the flow rather than searching by brute force gradient descent, and steered an extreme atmospheric river.

Both results are real. Both are entirely about the model. What they establish is that a learned weather model is differentiably sensitive to small changes in a targeted direction, which is a property of a function. Whether the atmosphere is sensitive in the same way, and whether any physically deliverable action could produce the perturbation in question, are separate questions that neither paper asks and that no experiment in the literature currently answers.

This is episode eight, the last of a walk through a literature map on physics AI and causal world models. Intervention and the boundary of evidence. The theme where the claims are largest and the evidence is oldest.

Because there is a seventy year record here, and the AI weather community has mostly not read it.

Project STORMFURY ran hurricane seeding experiments from 1962 to 1983. Hugh Willoughby and colleagues wrote the chronicle, and what killed the program is worth stating precisely, because it is not what people assume. It did not fail on execution or on funding or on nerve. It failed on two things. The physical premise turned out to be wrong: hurricanes contain far too little supercooled water for the hypothesized seeding mechanism to operate. And the apparent successes fell within the range of natural intensity fluctuation, so there was no way to attribute them.

Bernard Silverman’s 2001 assessment of glaciogenic seeding of convective clouds reaches a parallel conclusion by a different route. He reviews the randomized experiments against explicit criteria for statistical and physical evidence and concludes the accumulated results do not establish an effect, with several apparently positive results failing on replication. Decades of operational seeding activity coexisted with no demonstrated effect. Not with a demonstrated absence of effect. With no demonstration either way.

The National Research Council’s 2003 assessment is the authoritative version and it draws the distinction that matters more than any other in this theme. There is no convincing scientific proof of efficacy for intentional weather modification. The underlying questions remain scientifically legitimate. What limits them is experimental design, statistical power, and the counterfactual that does not exist, because you cannot run the unseeded version of the same storm.

Hold onto that. Not demonstrated is a different claim from disproven, and the reason for the difference is a measurement problem rather than a physics problem. Which is exactly the kind of problem that better prediction, better state estimation, and better causal inference could in principle change. That is the whole reason this theme belongs in a literature map about learned physical models.

There is one case where the chain has been closed. Jeffrey French and colleagues, working in the SNOWIE campaign, used airborne and ground based radar to track a glaciogenic seeding event end to end. Silver iodide released into an orographic cloud, ice initiation, particle growth, snowfall reaching the ground, with the seeding lines visible in the radar field as they propagated. Observed, not inferred from a statistical comparison across storms. It is a genuinely beautiful piece of measurement.

Katja Friedrich and colleagues then quantified it, estimating the mass of snow attributable to seeding over specific areas and durations. The number is on the order of fractions of a millimeter of water equivalent. That is the effect size for the one intervention whose mechanism has been observed working. Anyone reasoning about weather intervention should have both halves of that in mind at once: the mechanism is real and the demonstrated magnitude is very small, in the narrowest and most favorable configuration anyone has managed to instrument.

Robert Wood’s 2021 analysis of marine cloud brightening is the methodological model I would hold up here. Before proposing to test an intervention, he estimates its achievable radiative forcing with a transparent heuristic model, and shows the answer is dominated by uncertainties in aerosol cloud interaction rather than by deployment scale. Bound the plausible effect first. Find out what dominates the uncertainty. Then decide whether the experiment is worth designing. That discipline exists, it is standard in the climate intervention literature, and it is absent from the AI steering genre entirely.

So put the steering studies against that background. Huang and colleagues, and Liu and colleagues, produce optimization results inside a surrogate that was trained exclusively on unperturbed reanalysis. Which means the intervention regime is precisely the regime with no training support. Recall the finding from episode five, that models trained on trajectories fail to apply the mechanics underneath, and from episode four, that learned models fail to extrapolate to unseen intensities of phenomena they have seen thousands of times. A model with those documented properties is being asked what happens under a state it has never seen, and its answer is being reported as an intervention.

There is a piece of history that makes the steering genre look different, and it belongs here. Finding the perturbation that most efficiently changes a forecast is not a new capability. Operational centers have been computing singular vectors and running adjoint sensitivity since the 1990s, precisely to identify the fastest growing directions in the flow, and they use them for targeted observation, meaning they decide where to send an aircraft to sample because that is where uncertainty matters most. Episode one covered Carla Cardinali’s use of the same adjoint machinery to price individual observations. So the atmosphere’s sensitivity structure has been studied with physics based tools for thirty years, and the field that studied it drew the conclusion that this structure tells you where to measure. The learned model version rediscovers the sensitivity through a differentiable surrogate and draws the conclusion that it tells you where to act. The mathematics is similar. The inference is not, and the difference is that a targeted observation is an action whose effect you can verify while a targeted perturbation is not.

The two gaps in this theme follow directly and both are stated as experiments rather than as complaints.

No AI era steering study has been paired with a field testable hypothesis, an actuation model connecting the perturbation to something physically deliverable, or a power analysis for detecting the claimed effect. Those three are what would turn a gradient into a proposal. The perturbation the optimizer finds is a vector in state space. To be an intervention it has to be reachable by some physical action, and nobody has written down the map.

And there is no validation protocol for a learned model’s response to interventions absent from its training distribution. That one is embarrassingly tractable. Take the perturbation a learned model recommends. Apply it inside a physics based model. Compare the trajectories. Characterize how the disagreement grows with perturbation size, with lead time, and with distance from the training distribution. That experiment requires no new theory and no new data, only the willingness to check. Until somebody runs it, every steering result is a claim about a function.

It is worth asking what learned models could genuinely contribute to the counterfactual problem, because there is an answer and it is not steering. The reason STORMFURY and the convective seeding programs could not resolve their effect sizes is that natural variability swamped the signal and there was no control storm. Everything that reduces the variance of the expected outcome makes that detection problem easier. A better analysis reduces initial condition uncertainty. A larger and cheaper ensemble characterizes the natural spread more finely, and learned models made large ensembles cheap, which is not a small thing. A calibrated probabilistic forecast supplies the distribution of what would have happened, which is the closest available substitute for a control. None of that is intervention. All of it is the statistical infrastructure an intervention experiment would need, and it is the part of this literature that actually bears on the question the NRC identified. The contribution runs through improving the counterfactual estimate rather than through finding perturbations inside a surrogate.

Which is the right point to close the series, because the same structure has appeared in all eight episodes.

We started at the sensor boundary, with the fact that a learned atmospheric model has never seen the atmosphere, only instruments, and that credit for its skill is shared with an observing system and an assimilation chain it did not build and cannot reproduce. Then the prediction results, which are genuine: the learned models won that argument, reproducibly, and are now running operationally inside the centers they were once benchmarked against. Operator learning gave us one clean general property, resolution invariance, plus measurable transfer in the low data regime, alongside the finding that much of the surrounding speedup literature does not survive a fair comparison.

Episode four was the turn. The measurements behind all of it are weaker than they look, in the tails, under distribution shift, and when the developer and the evaluator are the same person. Episode five asked whether any of it implies mechanism, and found that prediction pressure does produce internal representations aligned with physics nobody supplied, while the direct tests of the Kepler to Newton premise come back negative. Episode six pointed out that forecast value is user specific, computable, and uncomputed. Episode seven found that planning inside a learned model works under conditions including a fast correcting loop and a characterized actuator, neither of which the open atmosphere supplies. And intervention, this episode, has a long empirical record whose limiting factor was always the counterfactual, which the current AI results have not engaged.

The pattern is consistent. At every step, a real and impressive capability is demonstrated, and then a claim is made one rung higher on the ladder than the evidence reaches. That ladder, from episode five, is worth repeating because it is the most portable thing in this map. Predictive relationship. Internal representation. Identified causal structure. Valid counterfactual. Support for intervention. Most claims about learned physical understanding are rung two evidence carrying rung four or rung five language.

I want to be clear that this is not a case for pessimism. Every one of those gaps is an experiment somebody can run, and several of them are cheap. Hold a model fixed and vary the observing system, dropping a satellite channel or degrading station density to a historical level, and you get the first attribution of forecast error between the sensors, the analysis, and the model. Build surrogate fields with matched autocorrelation and seasonality but no causal coupling, run them through the standard causal discovery pipeline, and you get the empirical false positive rate that the Earth system causal literature currently reports without. Probe a learned forecaster’s internal representations and then ablate them, and you find out which ones are load bearing rather than merely decodable. Compute a relative economic value curve for a modern ensemble system and a named user, and you convert a score improvement into a statement about outcomes. Publish a calibrated error bound over a reachable state set for one variable, one region, one lead time, and you produce the first honest interface between the learned modeling literature and the control theory that any intervention proposal would need.

None of those require a new architecture. Several of them are a few months of work on artifacts that already exist. What they have in common is that each one converts a claim into a measurement, and the field’s bottleneck right now is measurement rather than capability.

The last question is the one the NRC assessment left open in 2003 and that nobody has revisited with modern tools. Given current observing density and a learned model’s demonstrated sensitivity, what randomization design, what sample size, and what detection threshold would make a seeding effect of the SNOWIE magnitude measurable in the field? That is an answerable question. It might have a discouraging answer. Either way it would be the first time in twenty years that anyone checked whether the terms of the problem have changed.

That is the map. Eight themes, about eighty papers, and a set of experiments that would move claims from asserted to established. The explorer this series follows is on the site, with every paper linked, each with a takeaway and a note on why it belongs. If you want to argue with any of this, the papers are the place to start.