AF2Rank

AF2Rank webserver overview

A set of models for the same protein can look equally plausible, and a set of point mutants can be hard to rank without a lab. AF2Rank asks AlphaFold which one it believes.

It hands each candidate structure to AlphaFold as a template, with the sequence and side chains hidden by default, and records how confident AlphaFold is and how far its output moves from the template. A template that AlphaFold reproduces with high confidence is one it considers a real fold.

This page covers the AF2Rank notebook of the ColabDesign project (release v1.1.3), a re-implementation of the method by Roney and Ovchinnikov. It has two modes. Rank structures scores a reference structure and any number of decoys of the same protein.

Rank sequences keeps the reference as the template and scores each sequence you provide by setting it onto that template, for example a library of point mutants. To predict a structure from a sequence alone, use AlphaFold 2.

Pricing

AF2Rank is billed by the minute of runtime: 15 credits per minute on an NVIDIA T4 GPU. Completed runs are charged in proportion to elapsed time, rounded up to a whole credit, with a minimum charge of 20 credits, so a run of up to 80 seconds costs 20. A run that fails is not charged, and unused reserved credits are returned. Set a spending limit before submission.

The quote before submission is an estimate from the residues of the reference and the number of decoys or sequences, and the final charge follows the measured runtime. Most runs with a handful of structures are short enough to cost the minimum. The table is for Rank structures with the default Recycles and Iterations of 1.

Reference residues	No decoys	10 decoys	50 decoys	100 decoys
20	20	20	20	20
76	20	20	38	69
150	20	22	76	145
250	20	33	128	247
400	20	49	206	401

Rank sequences is quoted the same way, with each sequence counting like a decoy. A reference of 76 residues with 500 sequences quotes 316 credits.

Recycles and Iterations multiply the scoring time. For a reference of 150 residues with 50 decoys, the quote of 76 credits becomes 146 with 3 recycles, 216 with 3 iterations and 426 with both. Chain changes the quote through the number of residues it selects. Model family, Model number, the masking options and Return predicted structures do not change it.

Inputs

Input	Mode	Accepted	Description
Reference structure	Both	.pdb, .ent, or a PDB ID fetched from RCSB, up to 50 MiB	The native structure. Its chains define the template, and the TM-score columns compare every input against it.
Decoy structures	Rank structures	.pdb, .ent, up to 100 files of up to 50 MiB each	Optional. Each decoy must have the same number of residues, in the same order, as the reference.
Sequences	Rank sequences	.fasta, .fa, .txt or pasted FASTA, up to 10 MiB	One FASTA file. Each record is one candidate, named by its header, and must be as long as the chains selected from the reference. Names must be unique.

File names label the result rows, so every decoy needs its own name, different from the reference.

Settings

Ranking

Parameter	Type	Default	Description
Mode	enum	Rank structures	Rank structures or Rank sequences. Chosen with the Task cards above the inputs.
Chain	string	blank	Chain or chains of the reference to use, such as A or A,B for a complex. Blank uses every chain. The notebook examples use A and A,B.
Model family	enum	AlphaFold	AlphaFold uses the pTM models, AlphaFold-Multimer the multimer v3 models.
Model number	integer	1	Which of the five parameter sets to use, 1 to 5.
Recycles	integer	1	Recycles of the AlphaFold model, 0 to 4.
Iterations	integer	1	How many times AlphaFold's output is fed back as the template.

Advanced

Parameter	Type	Default	Description
Mask template sequence	boolean	on	Hides the template's sequence from AlphaFold.
Mask template side chains	boolean	on	Hides the template's side chains.
Mask inter-chain template information	boolean	off	Hides the template's inter-chain geometry, for complexes.
Return predicted structures	boolean	off	Returns the structure AlphaFold predicts for each decoy or sequence as a PDB file.

Outputs

File	Description
scores.csv, scores.json	One row per scored input, in submitted order, with the native scores below. scores.json also holds the correlation printed beside each plot.
output_pdbs/<name>.pdb	Predicted structures, when requested. The reference is not written.
tm_i_vs_composite.png, tm_o_vs_ptm.png, tm_i_vs_tm_o.png	Rank structures only. The three plots of the notebook, coloured by pLDDT.
provenance.json, run.log	Settings, source release, library versions and the run log.

Understanding results

Reading the scores

Each row of scores.csv has these columns, named as in the notebook.

Column	Meaning
plddt	Mean pLDDT of AlphaFold's output, from 0 to 1.
ptm, i_ptm	Predicted TM-score and predicted interface TM-score. See ipTM and pTM.
pae	Mean predicted aligned error of the output in angstroms. See PAE.
tm_i	TM-score of the input structure against the reference.
tm_o	TM-score of AlphaFold's output against the reference.
tm_io	TM-score between the input structure and AlphaFold's output.
rmsd_io	RMSD between the input and the output.
composite	ptm times plddt times tm_io. The score the method ranks by.
id	The file name of the structure or the FASTA header of the sequence.

The remaining columns are the model's own log for the run, such as loss, dgram_cce and seqid.

Rank candidates by composite, high to low. A candidate scores high when AlphaFold is confident in its output and the output stays close to the input. The method's check is the correlation between a score and a measure of quality that you trust, which is why the plots put composite or ptm against tm_i; the Spearman value in scores.json summarises each plot.

What the paper reports

The paper gives no cutoff for composite, so there is no pass mark. It measures how well the ranking agrees with decoy quality. Across 133 sets of Rosetta decoys, composite had an average Spearman correlation of 0.925 with the TM-score of each decoy, using AlphaFold model 1, one recycle and a masked template sequence and side chains, which are the defaults here. The paper also notes that AlphaFold predicts poorly without an alignment or templates, so a ranking is only as good as the candidates it is given.

Worked example

Rank structures with chain A of PDB 1AKE as the reference and chain A of PDB 4AKE as the only decoy, with default settings, gives these rows.

id	plddt	ptm	tm_i	tm_io	rmsd_io	composite
1AKE.pdb	0.917	0.817	1.000	0.761	4.9	0.570
4AKE.pdb	0.926	0.827	0.684	0.963	1.1	0.738

Both are structures of the same 214-residue enzyme. The decoy differs from the reference (tm_i 0.68), yet AlphaFold's output stays close to it (tm_io 0.96, 1.1 angstroms), while its output for the reference moves 4.9 angstroms away from the input. The decoy therefore ranks above the reference by composite. The method ranks the inputs AlphaFold believes, and a different conformation of the same protein can outscore the reference. The correlations in scores.json are exactly 1 or -1 here because only two structures were scored, so they carry no information.

Common misreadings

• The reference scores itself with tm_i equal to 1, which does not make it the top row by composite.
• ptm depends on the length of the chain, so composite values are only comparable between inputs of the same protein in one run. A 20-residue peptide can score near 0.06 with a confident plddt, where a 214-residue protein scores 0.57 or more.
• With few decoys the correlation says little.
• For Rank sequences the scores order candidates only relative to one template: a mutation that needs the backbone to move cannot score well on a fixed template.

Sources

• Roney J. P. and Ovchinnikov S. State-of-the-art estimation of protein model accuracy using AlphaFold. Physical Review Letters 129, 238101 (2022). doi:10.1103/PhysRevLett.129.238101
• ColabDesign, release v1.1.3, notebook AF2Rank.ipynb. Code under the Beer-Ware licence. TM-score is computed with the TMscore program of the Zhang group.
• Jumper J. et al. Highly accurate protein structure prediction with AlphaFold. Nature 596, 583 (2021). The AlphaFold parameters are released under CC BY 4.0.