Structure portion part 1

Quaternary

Quaternary = many folded chains

Domain folds semi independtly, active site created when catalytic residues align.

Mos1 transposase works as a dimer (TIR ends bind together)

Folding Problem

Cant predict fold based off AA sequence alone.

Shape = mechanism.

Structure

set of 3D coordinates per backbone + sidechain stored in PDB/ mmCIF file

Back bone, contact map, and distance map all are unchanged by translation or rotation

use dihedral angles. w is planar.

Distance map L x L matrix

Dij = ∥xi − xj ∥, distance between residues i and j.

captures the fold. Zero diagonal

Contact Map

Is a distnce map with a threshold.

Cij = 1[ Dij < θ ] , θ ≈ 8 ˚A(Cβ–Cβ),

a binary “in contact / not.” Symmetric and diagonal-dominant.

α-helix → thick band hugging the diagonal.

• β-sheet → off-diagonal stripe:

parallel = aligned with diagonal, antiparallel = transverse to it.

• Long-range contact → a far isolated dot

from the diagonal: two residues distant in

sequence, touching in space

Do contact determine fold?

Long range contacts predict fold. Predicting contacts is best way to predict structure.

Conserved vs covariation

conserved column (low entropy) is single site & done by

Covariation = columns move in pairs for structure.

spatial proximity over evolutionary time

What happens if mutation

If in 3D contact, must stay chemically compatiable. If one mutates, the other co mutates.

columns that covary across MSA are likely to be in 3D contact.

Miyazawa–Jernigan (MJ96). Sub matrix. → Potts

log odss for spatial contacts. MI(i, j) = log [p(a, b)]/[p(a) p(b)]

MI ≥ 0 (Gibbs), = 0 iff columns are independent;

large MI flags a covarying pair.

Mutual Info = how much they share = naive coevolution score

What Inflates MI (mutual info)

Redundancy. A family alignment is full of near-identical relatives. so weight sequences before counting.

Transitivity. A correlation that is real but indirect. = Potts model (cant do redundancy)

Direct vs Indirect

A real contact i–k plus k–j induces a spurious apparent correlation i–j (transitivity).

MI / covariance cannot tell a direct contact from an indirect chain.

MI does well on clean 1:1 pairing like RNA

Position-specific couplings

Extend the profile with pairwise terms:

Pr(s) ∝ exp Sum( Hi(si)) + Sum ( Jij (si, sj ))

Fields Hi = the conservation

PSWM part; couplings Jij = pairwise compatibility.

A PSWM is exactly the special case all Jij = 0

Most entropy

pick the one with the most entropy= the flattest, least-assuming one that still fits the constraints. Assume nothing beyond

the evidence.

Random variable, What you constrain , Max-entropy answer

discrete nothing uniform

real-valued mean and variance Gaussian

integer-valued mean geometric

sequence-valued position-specific freq PSWM

sequence-valued + pairwise correlations Potts model

maximize entropy, with one Lagrange multiplier per constraint.

= 1/Z exp (Sum[hi(si) ] + Sum[Jij (si, sj )])

Z = e ^ (1+λ)

The fields and couplings are the Lagrange multipliers. Jij = 0 recovers the PSWM.

DCA: solve for the direct couplings

fits the couplings Jij directly: a large ∥Jij ∥ ⇒ a direct contact, with indirect chains filtered out. That is what MI could not do.

On CheY of the top 99 pairs, 41 really touch under MI and 55

under the fitted couplings. The transitive pair S56--A103 is the kind of thing that moves — MI ranks it 115 of

4479, the Potts model 1477.

i<j Jij ) is a sum over all 20L sequences, which is intractable. So DCA maximizes a regularized, tractable surrogate whose optimum

stays near the true couplings (never computing Z).

For transposase

Many transposon paralogs across a host clade ⇒ a deep MSA ⇒ a strong coevolution

only works if selection is acting on the sequences — selection is what couples two positions, so a copy nothing contributes noise to every pair. So, for transposases in particular, you must always

take care to exclude nonfunctional copies, e.g. pseudogenes.

Dca

DCA / EVcouplings: fit a Potts model → predicted contact map → “fold up” into 3D.

• Strong on deep families; limited on shallow ones (too few Neff)

2 way to map 3d coordinates

• AlphaFold2: query sequence + an MSA (where B2’s coevolution enters); more accurate, slower.

• ESMfold: a single sequence, with a language model supplying the signal; faster, and it works on orphans.

A model is a function fθ(x) with parameters θ — the millions of weights. Learning = choosing θ so the outputs match examples.

Neurons

neuron takes a weighted sum of its inputs and passes it through a nonlinear activation function: y = σ Sum( wi*xi + b). The weights wi (and bias b) are the parameters;

the activation is what makes it more than a straight line.

Loss

Training minimizes a loss L(θ). The canonical loss is the negative log-likelihood of the data:

L(θ) = − log Pr(data | θ).

Minimizing it is maximizing the likelihood, the same MLE

loss is a composition of differentiable units

Autodiff computes ∂L/∂θ exactly, automatically, by the chain rule (no hand calculus).

• SGD (stochastic gradient descent): step θ ← θ − η ∇L downhill, on small mini-batches.

Regularizers

Add a penalty on the parameters, e.g. L(θ) + λ∥θ∥2:

− log Pr(θ | data) = − log Pr(data | θ) − log Pr(θ) +const.

loss regularizer

A common regularizer is the L2 penalty, effectively a Gaussian prior on the parameters.

[Bayes] regularizer = − log prior; it pulls toward simpler models.

Hyperparameters: the knobs you don’t fit

not fit by the training loss: that would just pick the most flexible model every time. You choose them on a held-out validation set.

The regularization strength λ — how hard the prior pulls;

• the learning rate η — how big an SGD step is;

• the architecture: how many layers, how wide, which nonlinearity;

• the mini-batch size, and how long to train before stopping;

• the pseudocount α of a PSWM or profile HMM (PROFILES B1) — same idea, long before any neural network

Splitting data

before any fitting happens, split the data

• Train — fit the parameters θ.

• Validation — tune the hyperparameters (and stop early).

• Test — touched once, for an honest estimate of generalization.

Overfitting = fitting noise; the train−test gap is the tell. Disjoint is not enough for sequences: homologues

either side of the split leak the answer across it, so the test set must be separated by similarity, not at random.

Ensambles

Average several models (different seeds / data) ⇒ lower variance and better calibrated predictions.

[Bayes] an ensemble approximates Bayesian model averaging, integrating over models instead of trusting one

Zero units

Randomly zero units during training. It regularizes (stops units co-adapting) and behaves like averaging an ensemble of sub-networks.

Masking to learn from unlabeled data

Hide part of the input and predict it (self-supervision). It needs no labels and no solved structures, just raw sequence. This is exactly ESM’s training objective.

Then autodiff + SGD train the whole chain end to end

symmetries

symmetries, for example different rotations of the same

protein structure. Three terms to fix:

• Invariance — anything that does not transform under the symmetry operation. For example, distances between residues.

• Equivariance — things that transform in a predictable way under the symmetry. For example, if you rotate a protein, its active site rotates along with it.

• Data augmentation — applying the symmetry operation to your training data, to teach the network to see past it. For example, randomly rotating protein structures during training.

KEY IDEA

You can build invariance and equivariance into your model mathematically, but data augmentation is much simpler and almost always enough

Recap Alpha vs ESM

AlphaFold2 — a structure prediction model. Inputs are protein sequences, outputs are protein structures; the loss function measures structure prediction error.

• ESM — a language model. Outputs are protein sequences; trained by the masked-language objective