AlphaFold2 from scratch (or not)

This article is my attempt to reconstruct AlphaFold2 from the ground up - or at least from the lowest level I currently understand. I want to know what information enters the system, what each learned representation can express, how those representations communicate, and how the network finally commits to three-dimensional coordinates.

A short story about AlphaFold2

AlphaFold2 was entered in the CASP14 blind assessment in 2020 Show sidenoteCASP (Critical Assessment of Structure Prediction) is a community wide experiment to determine and advance the state of the art in modeling protein structure from amino acid sequence. Every two years, participants are invited to submit models for a set of proteins for which the experimental structures are not yet public. Independent assessors then compare the models with experiment. CASP14 means this assessment was held for 14th time.. Its authors described it as the first computational method, to their knowledge, capable of predicting protein structures to near-experimental accuracy in a majority of cases. Across the CASP14 domains, the median root-mean-square deviation of $\ce{C}\alpha$ positions over the best-aligned 95% of residues was $0.96\,\text{\AA}$, compared with $2.8\,\text{\AA}$ for the next-best method [Jumper et al., 2021]. Those numbers do not mean that every AlphaFold2 prediction has atomic accuracy. They locate the claim in a particular assessment, on particular targets, under a particular metric.

In 2024, Demis Hassabis and John Jumper jointly received half of the Nobel Prize in Chemistry “for protein structure prediction”; David Baker received the other half for computational protein design [Nobel Prize, 2024]. AlphaFold2 was central to that recognition, but an AI system did not literally receive the prize—the people did. I still find it remarkable that understanding a neural architecture leads us into work recognized by the Nobel Committee in chemistry.

That success does not mean AlphaFold2 simulates the physical trajectory by which a protein folds, determines protein function from structure alone, or removes the need for experiments. It predicts a structural model from sequence-derived evidence. In this article, AlphaFold2 means the original monomer system described in the 2021 paper and its supplementary methods; later additions such as the pTM variants will be identified when they matter.

The biological prerequisite develops the protein chemistry and geometry needed here: amino-acid residues, peptide bonds, torsion angles, long-range contacts, chirality, and residue-local coordinate frames. I will use those ideas rather than derive them again. Our question is now specific to AlphaFold2:

How does AlphaFold2 turn sequence-derived evidence into one geometrically explicit structural hypothesis?

Write the target sequence as $A=(a_1,\ldots,a_N)$, where $a_i$ is the residue type at position $i$. The end product is a set of predicted heavy-atom coordinates together with estimates of confidence. If residue type $a_i$ has heavy-atom set $\mathcal{H}(a_i)$, we can denote the coordinate output by

This notation suppresses an implementation detail: the network packs atoms into fixed-size arrays and uses existence masks for atoms absent from a residue. It tells us what eventually comes out, but not how the system gets there. The difficult part is constructing useful intermediate states between a one-dimensional sequence and a three-dimensional structure.

The model sees more than one sequence

At the user boundary, inference starts with a target amino-acid sequence, usually supplied in FASTA format. Before the neural network runs, an external data pipeline searches sequence and structure databases. It produces three sources of evidence:

• the target sequence whose structure we want;
• a multiple sequence alignment (MSA) of evolutionarily related sequences;
• zero or more structural templates aligned to the target.

These database searches are part of the AlphaFold2 prediction system, but they are not learned layers of the neural network. This boundary matters. “AlphaFold2 takes a sequence” is correct from the user’s point of view; “the AlphaFold2 network consumes only a sequence” is not.

Training begins from different sources. Experimentally determined structures and their sequences, principally obtained from PDB mmCIF files, provide supervised examples; the final model also trains on a larger self-distillation set of structures predicted by an earlier model. I will keep inference preprocessing, network computation, and training supervision separate throughout the article.

Multiple sequence alignment: evolutionary evidence

A multiple sequence alignment places evolutionarily related protein sequences into a common coordinate system. Each row is one sequence; each column is intended to collect residues that correspond across those sequences.

Target:    M G S A K E
Homolog 1: M G T A K E
Homolog 2: M - S V K E

Without alignment, an insertion or deletion can shift every later array index. Comparing raw position 4 in two sequences would then compare residues that do not play corresponding roles. The alignment makes the intended correspondence explicit.

After the alignment has been projected onto the target’s columns, if the selected MSA contains $S$ sequences and the target contains $N$ residues, its model-facing residue identities can be pictured as

where $\mathcal{A}_{\mathrm{gap}}$ contains amino-acid symbols and a gap symbol. The network-facing residue axis remains tied to the $N$ target positions. Gap state and deletion counts associated with those positions are stored in feature channels; insertions do not create a longer residue axis for the model.

Why should other sequences help predict the target’s structure? Evolution supplies repeated natural perturbations of a related protein family. Two simple patterns illustrate the evidence available in an alignment.

Conservation

The highlighted column barely changes across the sampled sequences. This suggests that substitutions at this position were often unfavorable in the evolutionary contexts represented by the alignment.

Conservation does not tell us why the position is constrained. The residue may contribute to core packing, catalysis, binding, a disulfide bond, or another structural or functional requirement. It is evidence that the position matters, not a complete explanation.

Correlated variation

Let the two highlighted columns be target positions $i$ and $j$. The important observation is not one particular substitution. It is that the residue appearing at $i$ is statistically associated with the residue appearing at $j$ across the alignment.

One possible explanation is a structural constraint: if the two residues interact in the folded protein, a change at one position may favor a compensating change at the other. But correlated columns do not prove a direct contact. Correlations can also arise through indirect couplings, shared ancestry, and sampling bias.

AlphaFold2 does not receive a hand-computed table that declares which correlations are contacts. During training, its masked-MSA objective encourages the network to learn dependencies within the alignment while the structural objectives decide which learned patterns are useful for prediction [Jumper et al., 2021].

Templates: structural evidence from related proteins

The data pipeline may also find experimentally determined structures whose sequences can be aligned to the target. These templates provide direct but incomplete geometric evidence: aligned residue identities, backbone torsions, pairwise distances, and masks indicating which coordinates are available.

A template is not simply copied into the output. It may cover only part of the target, differ at important residues, or represent a relative whose structure does not transfer exactly. AlphaFold2 embeds template features into the same learned states that will also receive sequence and MSA evidence.

From biological records to numerical features

The neural network does not operate directly on FASTA text, alignment files, or PDB coordinate records. The original supplementary methods aggregate them into six main numerical inputs. Here, $N_{\text{res}}$ is the number of target residues, $N_{\text{clust}}$ the number of MSA cluster centres, $N_{\text{extra\_seq}}$ the number of additional unclustered sequences, and $N_{\text{templ}}$ the number of templates:

These arrays are input features, not the internal representations processed by the main Evoformer stack. Their dimensions also carry semantics. The MSA features need a sequence axis and a target-residue axis. Template pair features need two target-residue axes because they describe a relationship between positions $i$ and $j$. This table is a view of the six aggregated inputs in the supplementary description; masks and other bookkeeping features also accompany them wherever observations are missing or padded [Jumper et al., 2021].

Two learned views of the same protein

AlphaFold2 begins its main computation with two learned representations [Jumper et al., 2021]:

The word representation matters. An element $m_{si}$ is no longer an amino-acid symbol, and $z_{ij}$ is not a predicted distance. Each is a learned feature vector whose meaning is distributed across its channels and refined by later layers.

The two arrays organize different questions:

• $m_{si}$ asks what the network currently needs to retain about sequence $s$ at target-aligned position $i$.
• $z_{ij}$ asks what the network currently needs to retain about the relationship from target residue $i$ to target residue $j$.

The pair entries are indexed by ordered pairs. We should therefore not assume that $z_{ij}=z_{ji}$, even when a physical quantity such as distance is symmetric. Directional pair features can carry orientation-dependent information.

Initializing the MSA representation

Let $f^{\text{msa}}_{si}\in\mathbb{R}^{49}$ be the clustered-MSA feature vector and $f^{\text{target}}_i\in\mathbb{R}^{21}$ the target amino-acid feature. AlphaFold2 projects both into the common MSA channel dimension and adds them:

The target projection is shared across the sequence rows. Every MSA cell therefore begins with both row-specific alignment information and the identity of the target residue to which that column corresponds.

Initializing the pair representation

The target features are also projected in two different roles and combined across each ordered pair:

Here $r_i$ is the supplied residue_index for target position $i$. The first two terms identify the residues at the two ends of the pair. The final term records their signed residue-index separation, clipped to the range $[-32,32]$. In a contiguous single chain, this is simply $i-j$. Positions separated by more than 32 residues therefore share the same initial long-range bin. The model knows that they are far apart in sequence without assigning a distinct embedding to every possible separation.

At this point, $z^{(0)}_{ij}$ has not yet extracted a coevolutionary relationship between MSA columns $i$ and $j$. That communication happens later: the extra-MSA stack and the main Evoformer use outer-product updates to move information from MSA states into pair states.

Adding templates and the extra MSA

The two initial arrays are then enriched through specialized paths [Jumper et al., 2021]:

• Template angle features are embedded and concatenated to the MSA representation as additional rows.
• Template pair features pass through a template pair stack and pointwise attention before their update is added to $z_{ij}$.
• The much larger set of unclustered MSA sequences is processed by a smaller four-block extra-MSA stack. Its final MSA activations are discarded, but the information it transfers into $z_{ij}$ is retained.

After template-angle rows are concatenated, the MSA sequence axis entering the main Evoformer has size $N_{\text{seq}}=N_{\text{clust}}+N_{\text{templ}}$ when templates are present. This division is computational as well as conceptual. The model preserves a manageable set of rich MSA rows for the 48-block main Evoformer while still allowing many additional homologous sequences to inform residue-pair relationships.

We now have the state that enters the main Evoformer: an MSA representation $m_{si}$ and a pair representation $z_{ij}$. The next question is no longer “what are the inputs?” It is what must one Evoformer block do so that evolutionary evidence becomes geometrically useful?

How Evoformer updates the representations

From learned representations to three-dimensional structure

After the final Evoformer block, AlphaFold2 projects the first, target-sequence row of the MSA representation into a per-residue single representation $s_i\in\mathbb{R}^{384}$. The structure module receives this single representation together with the final pair representation $z_{ij}$. It must turn these abstract learned states into explicit geometry.

The structure module’s first representation choice is how to describe the pose of an individual residue.

A local backbone frame

A coordinate frame gives the backbone atoms $\ce{N}$, $\ce{C\alpha}$, and $\ce{C}$ a local coordinate system. For residue $i$, AlphaFold2 represents the backbone frame as a rigid transformation

where $R_i\in\text{SO}(3)$ is a rotation and $t_i\in\mathbb{R}^3$ is a translation. The transform maps a local point into the global coordinate system:

The notation $\text{SE}(3)$ records that $T_i$ preserves distances and orientation.

AlphaFold2 uses the $\ce{C\alpha}$ position as $t_i$. Its ground-truth frame orientation is constructed from the positions of $\ce{N}$, $\ce{C\alpha}$, and $\ce{C}$ by Gram–Schmidt orthonormalization [Jumper et al., 2021]. If

then the basis is

The construction fixes a reproducible, right-handed local orientation for every non-degenerate backbone geometry. A global reflection of the finished structure cannot be obtained from the original by one proper rigid motion in $\text{SE}(3)$; this distinction later matters to frame-aligned supervision.

Why AlphaFold2 predicts frames and torsions together

AlphaFold2 does not simulate the physical process by which a chain folds. Its structure module iteratively refines a geometric hypothesis. The concrete part of that hypothesis is one independently movable backbone frame per residue—the paper calls this collection a residue gas [Jumper et al., 2021].

At the beginning of the structure module, every frame is initialized to the identity transform, so the computational state places all residues at the same origin with the same orientation. This is an initialization of the model’s geometric hypothesis, not a claim that a physical protein passes through such a configuration. Each of the module’s eight weight-shared layers then performs four relevant operations:

1. Invariant point attention updates the per-residue single representation using the single, pair, and current geometric representations.
2. A transition network further updates the single representation.
3. A learned local rigid update is composed with each backbone frame.
4. A small residual torsion head predicts seven two-component torsion vectors for $(\operatorname{pre}\text{-}\omega,\phi,\psi,\chi_1,\chi_2,\chi_3,\chi_4)$ from the current and initial single representations.

In compact notation, one layer looks roughly like

Here $s_i$ is the current single representation for residue $i$, $z_{ij}$ is the pair representation from residue $i$ to residue $j$, and $\Delta T(s_i)$ is a rotation-and-translation update expressed in residue $i$‘s local frame. The equations suppress normalization, residual details, dropout, masking, and intermediate losses so that the information flow remains visible.

The frame update predicts a translation and three components of a quaternion whose scalar component is fixed to one before normalization. This produces a valid rotation while biasing each composed update toward a small rotation. Each torsion is predicted as a two-dimensional vector that is normalized onto the unit circle, rather than as a scalar in $[0,2\pi)$. Interpreting that vector as $(\cos\theta,\sin\theta)$ avoids an artificial discontinuity where $0$ and $2\pi$ denote the same angle. Valid torsion slots are masked as needed; $\chi$ angles have residue-specific masks, and periodic or symmetric cases require alternative targets. Residue-specific rigid-group tables ensure that nonexistent side-chain groups contribute no atoms, while backbone geometry is also supervised through frame-aligned losses [Jumper et al., 2021].

After the eighth layer, AlphaFold2 constructs atom coordinates from the final backbone frames, predicted torsions, residue identities, and tables of idealized bond lengths and angles. Residue-type-specific ideal transforms generate the backbone rigid group containing $\ce{N}$, $\ce{C\alpha}$, $\ce{C}$, and—except for glycine—$\ce{C\beta}$. The $\psi$ rigid group places the carbonyl oxygen, while the $\chi$ groups successively place side-chain atoms. In AlphaFold2’s particular construction, no predicted heavy-atom position depends on its $\operatorname{pre}\text{-}\omega$ or $\phi$ rigid groups, although those frames still participate in supervision through the frame-aligned loss [Jumper et al., 2021].

The backbone frames and torsions are predicted states, not independently learned model parameters. Each residue frame has six geometric degrees of freedom, and the torsion head predicts additional angle vectors, but the same structure-module weights produce those values for every residue. Frames give the network a local geometric language in which it can reason about relative positions while deterministically reconstructing chemically standardized groups of atoms.

What happens under a global rotation or translation?

Suppose we apply the same rigid motion $G\in\text{SE}(3)$ to the entire predicted structure. Every residue frame becomes

The absolute coordinates change, but the relationship between two frames does not:

AlphaFold2 uses this structure deliberately. Distances between IPA query and key points are unchanged by a shared rigid motion, so the attention update is invariant to the arbitrary global coordinate frame. The predicted frame updates are applied locally, making the resulting structure update equivariant: rotate and translate the input frames, and the updated frames rotate and translate in the same way. The frame-aligned point error follows the same principle when comparing predicted and experimental structures [Jumper et al., 2021].

We now know what geometric state the structure module updates: one frame per residue, followed by torsion-controlled rigid groups. To understand how the learned single and pair representations move those frames, we next need to derive invariant point attention.

References