ESM-2

What is ESM-2?

ESM-2 is a family of protein language models developed by Meta AI. ProteinIQ runs the 650 million parameter model, esm2_t33_650M_UR50D, which has 33 transformer layers and produces 1280 features per amino acid. It learns sequence representations through masked amino acid prediction rather than requiring an experimental structure as input.

An embedding is a numerical representation of a residue in its sequence context. The downloaded arrays can become inputs to a classifier, clustering analysis, or other downstream model. Individual embedding dimensions do not correspond to named physical properties or probabilities.

This tool returns embeddings, not predicted coordinates, function annotations, or mutation scores. For a predicted protein structure, use ESMFold.

How to use ESM-2 online

Paste protein sequences in FASTA format, upload a sequence file, or fetch sequences by RCSB PDB ID. Select a representation layer and whether to include mean pooling. ProteinIQ returns downloadable NumPy arrays for every sequence, with a heatmap preview for inspecting the embeddings without a local model installation.

Inputs and settings

Keep the checkpoint and representation layer consistent when combining embeddings into a dataset. Equal array dimensions do not make features from different layers interchangeable. Layer 33 is the default starting point; compare layers on held-out data when choosing features for a predictive model.

How does ESM-2 work?

ESM-2 uses a transformer architecture similar to large language models, but trained on protein sequences instead of text. The model processes sequences using self-attention, where each amino acid attends to all other positions to build context-aware representations.

Masked language modeling

During training, amino acids are masked and the model predicts their identities from surrounding sequence context. The hidden representations learned for this task can then be extracted without training a new model for each input sequence.

Embedding extraction

The per-residue output has shape (L, 1280), where L is the sequence length. ProteinIQ removes the beginning and end special tokens before saving the array. With mean pooling enabled, it averages those residue rows into a single vector of shape (1280,).

Understanding the results

ESM-2 outputs embeddings in NPY format. File names use the sequence label so that arrays can be matched back to their FASTA records.

Per-residue embeddings

Array row zero corresponds to the first input residue. For a 76-residue sequence, the expected shape is (76, 1280). A residue numbered 42 in that input is at array row 41; PDB residue numbers are not stored as the array index.

Mean embeddings

Mean pooling gives proteins of different lengths the same feature-vector size. It discards the separate representation of each position, so use the per-residue array when a downstream task needs to locate a motif or label individual residues.

Heatmap preview

The preview samples embedding dimensions and, for longer sequences, residue positions. Its colors are normalized within each sequence for display. Download the NPY files for numerical comparisons; heatmap colors are not confidence scores, and similar-looking plots do not establish functional similarity.

Use cases

ESM-2 embeddings can be reused as fixed input features for a classifier or regressor. Preserve sequence identifiers, layer selection, and the train/test split alongside the arrays so results remain reproducible.

Clustering or cosine similarity can help prioritize sequences for further investigation. An embedding neighbor is a candidate relationship, not proof of shared function. HMMER can test sequence homology against a supplied protein database and report statistical evidence for matches.

For variant studies, these arrays can support a model trained on measured effects. A large distance between wild-type and mutant embeddings alone does not establish whether a mutation is beneficial, harmful, or neutral.

Limitations

Sequence embeddings do not explicitly describe ligands, solvent, experimental conditions, or the chemistry of post-translational modifications. Use protein sequence input rather than structure coordinates or chemical modification notation.

Memory use increases with sequence length. ProteinIQ processes each submitted sequence as a whole; this tool does not automatically split long proteins into chunks. Splitting a sequence externally changes the context available to the model, so chunk-derived features should not be treated as equivalent to full-sequence embeddings.

The usefulness of embeddings depends on the downstream task and its evaluation data. Test performance on the kinds of proteins being studied, especially when applying a model trained on natural sequences to engineered designs.