AntiBERTy
v1.0.0Antibody language model for scoring, classification, embeddings and masked-residue filling
Antibody language model for scoring, classification, embeddings and masked-residue filling
Give AntiBERTy the variable domains of three therapeutic antibodies and it sorts them the way a repertoire biologist would. Adalimumab, a fully human antibody, comes back as human and Natural. Muromonab-CD3, a mouse antibody, comes back as mouse. Trastuzumab, a humanized antibody built by grafting mouse CDRs onto a human framework, comes back as Grafted with a graft probability of 1.0 for its heavy chain.
AntiBERTy is an antibody language model from Jeffrey Ruffolo, Jeffrey Gray and Jeremias Sulam (Ruffolo et al., 2021). It is an 8-layer BERT encoder with 512 hidden units and 26 million parameters, trained on 558 million natural antibody sequences, and it provides the sequence features used by IgFold. ProteinIQ runs AntiBERTy 1.0.0 with its released weights and offers its four commands: pseudo log-likelihood scoring, species and chain classification with graft detection, embeddings with optional attention, and masked-residue filling.
For a model trained on paired chains with a focus on somatic mutations, see AbLang-2.
An AntiBERTy job costs 7 credits for one or two sequences, plus 0.08 credits for each sequence, rounded to a whole credit. The price is the same for every task, and the exact quote is calculated before submission.
| Sequences in the file | Credits |
|---|---|
| 1 or 2 | 7 |
| 10 | 8 |
| 50 | 11 |
| 100 | 15 |
| 500 | 47 |
| 1,000 | 87 |
Only the number of FASTA records changes the quote. The task, sequence length and embedding settings do not.
| Input | Accepted formats | Notes |
|---|---|---|
| Antibody sequences | .fasta, .fa, .faa, .txt, or pasted FASTA | Required. One record per chain, each with a > header. Up to 510 residues per record. |
Free accounts can submit up to 10 sequences per job and paid plans up to 1,000. Each record is one heavy or light chain variable domain, so a heavy and a light chain go in as two records. The record ID is the first word of the header, and a repeated ID gets _2, _3 and so on in the results. A FASTA file with one record per chain is all the input needs.
AntiBERTy reads its own alphabet. Lower case is upper-cased, _ marks a masked residue, and any other character outside the 20 standard amino acids, such as X, is read as an unknown token rather than rejected. The model was trained on variable domains of roughly 100 to 130 residues, so scores for full-length chains or other proteins have no reference.
| Parameter | Type | Default | Description |
|---|---|---|---|
| Task | enum | Pseudo log-likelihood | Runs antiberty pll, classify, embed or fill on every record in the file. |
| Parameter | Type | Default | Description |
|---|---|---|---|
| Report CDR grafting | boolean | off | Adds graft (Natural or Grafted) and p_graft, the graft probability. |
| Parameter | Type | Default | Description |
|---|---|---|---|
| Hidden layer | enum | Last layer (8) | 0 is the input embedding layer and 8 the last encoder layer; All layers saves a stack of 9. |
| Pooling | enum | None | None keeps the [CLS] and [SEP] positions, Per residue drops them, Mean gives one 512-dimensional vector. |
| Save attention matrices | boolean | off | Adds <id>_attention with shape 8 x 8 x (L+2) x (L+2): layers, heads, then positions. |
| File format | enum | NumPy (.npz) | .npz is written with numpy.savez, .pt is a dictionary of tensors written with torch.save. |
Pseudo log-likelihood and masked-residue filling have no settings of their own.
Every task returns a spreadsheet, the native file it was read from, a run log with the exact antiberty command, and provenance.json with the settings, source version and weight checksum.
| Task | Native file | Spreadsheet columns |
|---|---|---|
| Pseudo log-likelihood | pseudo_log_likelihood.tsv | ID, Pseudo log-likelihood |
| Species and chain | classification.tsv | ID, Species, Chain, and with grafting on Graft, Graft probability |
| Embeddings | embeddings.npz or embeddings.pt | Array, Shape |
| Fill masked residues | filled.fasta | ID, Completed sequence |
Rows follow the order of the input file. The embeddings file holds one array per record, keyed by record ID, and the spreadsheet lists each array with its shape so a download can be checked before it is loaded with numpy.load or torch.load.
AntiBERTy masks each position in turn, predicts it from the rest of the chain, and averages the log probability it gives the residue that is actually there. The score is a natural log, so it is negative, and a value closer to zero means the chain looks more like the natural antibodies the model was trained on. Positions holding _ or an unknown character are skipped.
On ProteinIQ, trastuzumab's heavy and light chains scored -0.645 and -0.582, and adalimumab's scored -0.491 and -0.252. There is no published threshold for a good or bad score, so use it to compare chains of the same type, such as variants of one antibody, rather than as a pass mark. Scores from AbLang-2 and AntiBERTy come from different models and cannot be compared with each other.
Species is one of Camel, Human, Mouse, Rabbit, Rat or Rhesus, and Chain is Heavy or Light. These are the model's most likely labels, so a humanized or engineered chain can sit between species. Trastuzumab's light chain, for example, is labelled Rhesus. For humanization work, BioPhi scores humanness directly.
Graft probability is the model's probability that the CDRs come from a different sequence than the framework, and Graft is Grafted above 0.5. In the example above, trastuzumab's heavy chain is 1.000 and its light chain 0.844, while adalimumab's chains are 0.000 and 0.002. A graft call describes sequence composition, not whether an antibody is clinically humanized.
With the defaults, each record becomes an array of (L+2) x 512 values, where the two extra positions are [CLS] and [SEP]. Trastuzumab's 120-residue heavy chain gives 122 x 512. Per-residue pooling gives L x 512, mean pooling gives 512 values per sequence, and All layers adds a leading dimension of 9. Mean-pooled vectors from the last layer are the usual starting point for clustering or a downstream classifier.
Attention arrays grow with the square of the chain length and with the number of records. A file of around 1,000 chains with attention matrices can run out of GPU memory, so split large files into batches of about 100 when saving attention.
Each _ in a record is replaced by the amino acid the model considers most likely given the rest of the chain, and every other position is returned unchanged. The completed sequence is the model's prediction of what a natural antibody would carry there; it is not a prediction of binding.
AntiBERTy takes one chain at a time and does not model heavy and light chain pairing. It expects antibody variable domains of at most 510 residues. Its outputs describe how a sequence relates to natural antibody repertoires and carry no information about affinity, stability or developability.
Predict non-germline residues and generate embeddings for paired or unpaired antibody sequences.
Generate protein embeddings with ESM-C.
Analyze protein sequences and generate embeddings.
Screen paired antibody variable domains for low or high polyreactivity.
Bidirectional translation between protein sequences and 3Di structural tokens
Restore residues and generate antibody-specific representations or likelihood scores.
Characterize nanobody developability with sequence, structure, and surface-property metrics.
Language-model numbering for antibodies, T cell receptors, and VNAR/VHH domains
Score antibody and nanobody complexes with predicted DockQ.
Score amino acid substitutions with masked protein language models.