Additive Baseline Critique Flashcards (IPD reading)

Visani, Verma & DeWitt, bioRxiv April 2026. r > 0.999 vs ridge additive; MULTI-evolve learns addition, not epistasis. last 3 cards are figures to explain.

what is the additive baseline critique in one sentence?

a rebuttal showing that MULTI-evolve's neural network predictions correlate with a plain ridge additive model at r > 0.999, so the engineering wins came from stacking the best individual mutations, not from learned epistasis

who wrote the critique and why does that matter locally?

Gian Marco Visani (UW Allen School), Aayush Verma (UW Genome Sciences and Fred Hutch) and senior author William DeWitt. bioRxiv, posted April 24 2026, not peer reviewed. Visani is also first author on HERMES, and DeWitt is UW Genome Sciences, so this is a critique from the same campus

what is the methodological objection in one line?

an additive model cannot represent epistasis by construction, so it is the obvious null baseline for any epistasis learning claim, and the original study never ran it

what is an additive model here, concretely?

ridge regression on a binary "which mutations are present" vector, predicting a multi mutant by summing individual effects on a log2 scale. it literally cannot express interaction, which is exactly why it is the right null

is this a new experiment?

no. it is a like for like reanalysis using MULTI-evolve's own published code and the original authors' supplementary data

what were the four prediction tasks re-analyzed?

APEX peroxidase activity, dCasRx trans splicing, and HuABC2 antibody expression and binding. each trained on identical singles plus doubles data, then scored over the full combinatorial space

the headline number

Pearson r > 0.999 between the neural network's predictions and the additive model's, across all four tasks. the ranking of tens of thousands of untested variants is practically identical

what about the specific variants Tran et al. actually built?

the plain additive model ranks them just as highly. the additive baseline would have picked essentially the same winners

what happened to the antibody Pareto frontier argument?

MULTI-evolve framed antibody selection as navigating an expression versus binding tradeoff. the two models' Pareto frontiers are nearly identical, which undercuts the claim that the network learned distinct epistatic patterns for each property

blow two: does the network encode epistasis even in its training data?

no. predicted epistatic residuals for training doubles cluster at a constant value of minus y-hat wild type, exactly what a purely additive model with an intercept produces, with no correlation to the measured residuals

what is an epistatic residual?

for a double mutant, measured double minus (measured single i + measured single j). zero means additive, nonzero is the epistasis signal

blow three: the hyperparameter grid

they re-ran MULTI-evolve's own grid search but added a zero hidden layer option, which is tantamount to a linear model, plus a ridge baseline. on held out data the nonlinear networks do not significantly outperform either

what was the DMS benchmark in the original paper supposed to show?

that networks trained on nested sets (singles, then singles plus doubles, then plus triples) got better at predicting held out multimutants as training grew, which was read as evidence of learning interactions

the benchmark demolition, in one formula

that improvement is what an additive null predicts from ordinary variance reduction. prediction error variance for a k-mutant falls from (k+1) sigma squared with singles only to (k/p + 1) sigma squared once you add all pairwise doubles, to first order in 1/p

why does adding doubles help an additive model at all?

each double mutant is a redundant measurement of the same single mutation coefficients, so it tightens the estimates without teaching the model anything about interactions

what happened when they fit a ridge additive model to the same nested DMS data?

it reproduced the same improvement pattern, and its raw Pearson r values were not systematically lower, in fact higher in the singles only setting

what linear algebra tool is behind the variance result?

the Sherman-Morrison formula, a rank one update identity for matrix inversion, used in appendix B to get a closed form for prediction error variance under the singles plus doubles design

the scale mismatch they flag

the DMS benchmark uses datasets of tens to hundreds of thousands of variants, while the engineering campaigns train on a small, highly biased corner: doubles formed only from the top 15 singles. benchmark behavior says little about engineering behavior

what does the robustness check show?

alternative additive predictors agree too, including a naive predictor that just adds measured single mutant effects with no regression at all

what does the critique explicitly concede?

the engineering success is real. verbatim, the source of performance "is that mutational effects are sufficiently additive for the proteins and mutations considered, not that the neural network has learned epistatic synergies"

what is the scope disclaimer?

"this is a claim about MULTI-evolve, not about epistasis in general: epistasis is well-documented in protein genotype-phenotype maps"

the strongest defense of MULTI-evolve, which you should raise yourself

the doubles were built only from top ranked singles, a near orthogonal and low power design for estimating interactions, so "this data cannot support an epistasis claim" may be fairer than "this network cannot learn one". appendix C partly formalizes this

what richer baselines did they NOT fit?

global epistasis (additive on a latent phenotype seen through a nonlinear readout) and stability buffering models. they argue the strict additive baseline already matches, so there is no gap left to close, which is reasonable but untested

what is global epistasis?

apparent interaction that arises only because an additive underlying quantity is read out through a nonlinear or saturating assay. a more forgiving null than strict additivity

what should you NOT quote from this paper?

precise p values or r values from figures 2-4. those claims are stated qualitatively in the text and the numbers live inside figure images. the one hard number in the text is r > 0.999

what is the part of this paper that is argument rather than evidence?

the sociological half: it invokes Lipton and Steinhardt on troubling trends in ML scholarship and criticizes the schematic showing MULTI-evolve navigating a rugged epistatic landscape. fair criticism of hype, but not data

the transferable lesson for a design lab

before crediting a designed nanoparticle or immunogen result to a clever model, check whether a dumb baseline gets there too. when stacking stabilizing mutations into a trimer or an interface, ask whether the gain is from a learned interaction model or from combining the best singles

explain this figure: critique fig 1 (the r > 0.999 result)

four joint scatterplots, MULTI-evolve prediction on the y axis against the additive model on the x axis, for APEX, dCasRx, HuABC2 expression and HuABC2 binding. every mutational load is a colour (5 through 9 mutations for APEX, n in the thousands each) and every legend entry reads r = 1.00. the large circles are the variants Tran et al. actually built, sitting at the top of both models alike. the inset at the right shows the HuABC2 expression versus binding Pareto frontier drawn twice, additive and MULTI-evolve, and the two curves are the same shape

explain this figure: critique fig 2 (no epistasis even in training)

four panels, one per task. x axis is the MEASURED epistatic residual for each training double, y axis is the PREDICTED residual. if the network had learned interactions the points would follow a diagonal. instead every panel is a flat horizontal line of points sitting at a constant negative value, with the dashed orange line marking minus y-hat wild type. that flatness is the whole argument

explain this figure: critique fig 3 (extended hyperparameter search)

held out test performance after adding a zero hidden layer option and a ridge additive baseline to MULTI-evolve's own grid search, for all four tasks. the nonlinear networks do not separate from the linear and additive baselines. note the text reports this qualitatively, so do not quote exact values off the plot