Accurately predicting protein function remains one of the central challenges in biological machine learning (ML). Despite recent successes of protein language models (pLMs) and structure predictors, one question remains unclear: which aspects of a protein’s function do current models capture and how well?

Protein function is intrinsically multifaceted; a single protein can perform multiple functions, including catalysis, ligand binding, and allosteric signaling, which rely on environmental contexts such as expression, pH, macromolecular crowding, and other perturbations. Descriptors of protein function range from coarse-grained labels like Gene Ontology (GO) terms and Enzyme Commission identifiers to specific properties such as catalytic activity, binding affinity, and specificity. Experimental assays typically measure only one of these properties at a time, meaning no single measurement can capture the full functional landscape.

A well-rounded protein function model should predict general function (e.g. GO terms), but also more specific properties such as allostery, binding affinity, and catalytic activity. However, most models are developed and evaluated against a single downstream task, and strong performance on one benchmark dataset does not always translate to success on another. Protein foundation models aim to learn broad representations of proteins, but often struggle to outperform narrower models on functional prediction, making their generalization difficult to assess. Towards better generalized modeling of protein function, it is necessary to build valid tests that evaluate model performance as a function of task specificity and training similarity.

The problem of data leakage

Defining data leakage for protein function prediction is itself a challenging problem. Leakage is often treated as a simple sequence similarity problem, where proteins above a chosen sequence identity threshold are removed from the training set. In practice, however, leakage can occur in nearly countless ways. Proteins with low overall sequence identity may still share highly conserved domains, catalytic sites, or other motifs that are hard to detect. Conversely, proteins with divergent sequences can adopt very similar structures, allowing a structural model to associate a particular topology with a functional label, thereby missing the underlying mechanism. In mutational datasets, holding out individual variants while training on neighboring mutations or different substitutions at the same residue often measures interpolation within a mutational landscape rather than true generalization. Even seemingly straightforward notions of sequence leakage become complicated by multidomain proteins, proteins with large and highly conserved non-active site regions, and experimentally dictated expressions of the same mechanism (i.e. an scFv vs. a Fab), making it difficult to define what constitutes an independent training example.

Benchmark quality also degrades over time. As new experimental datasets are released, proteins that once only appeared in test sets are gradually added to new training corpora. Without careful filtering, models may inadvertently train on proteins that are highly similar or even identical to benchmark examples, inflating reported performance. Maintaining meaningful evaluation therefore requires continually rebuilding training sets as new data become available rather than relying on static benchmark splits.

PLUG: Protein Leakage-free evaluation for Unbiased Generators of function

To better discern a model’s functional knowledge, we developed PLUG (Protein Leakage-free evaluation for Unbiased Generators of function), an intentionally modular framework for constructing leakage-controlled training sets across continuously evolving collections of functional benchmarks. Rather than introducing a single new benchmark, PLUG provides a reproducible framework for compiling many.

PLUG has two primary components. First, it aggregates multifaceted validation datasets spanning biologically relevant functional axes into a common space with standard ML operational pipelines. Second, it constructs leakage-free training subsets from either a sequence reservoir (UniRef90), a structure reservoir (PDB), or both, while explicitly checking for overlap with one, multiple, or all functional test sets.

This produces a set of sequences and structures for model training that excludes proteins similar to the evaluation sets, ensuring safe benchmarking across different protein functions. PLUG easily accepts new benchmarks, requiring only an adapter for new data, allowing training sets to be rebuilt as the benchmark suite expands. This enables the community to maintain continually evolving evaluation targets that resist gradual contamination.

Functional benchmark suite

The initial suite of benchmarks spans multiple views of protein function across scales and domains. Examples currently supported include thermostability through the megascale dataset, deep mutational scan fitness from ProteinGym, GO term annotation from CAFA, allosteric site annotation from Allobench and PASSerRank, protein–protein interactions (PPI) from STRING and others, binding affinity from PDBBind and BindingDB, and RMSF from ATLAS.

Together, these tasks evaluate different aspects of protein function. They include interactions between residues within the same protein domain, such as in Allobench, interactions between proteins, as in STRING, and protein-level properties such as expression and enzyme classification. These test sets also vary in label noise, experimental conditions, and downstream applications, providing a broader basis for evaluating protein representation learning than optimizing for a single benchmark metric.

Leakage-controlled training set construction

Evaluating data leakage is inherently difficult because fully unbiased sampling is nearly impossible. For example, the PDB itself reflects decades of structural biology focused on experimentally tractable proteins. Nevertheless, stringent screening criteria can produce high-fidelity training sets for assessing generalization.

The core technical contribution of PLUG is a leakage filtering pipeline operating directly on a reservoir of sequences, structures, or both (e.g. UniRef90 and PDB). PLUG is applied to this reservoir and any observation sufficiently similar to a protein in the evaluation set is removed (with default thresholds of MMSeqs2 ID > 0.2 or Foldseek TM-score > 0.5, both of which are user configurable), resulting in a leakage-controlled training set and rigorous evaluation set. Traditional methods rely solely on a unidirectional sequence similarity score or temporal holdouts to remove homologs from the training set, whereas PLUG checks for bidirectional leakage with tunable coverage across both sequence and structure.

Rather than exhaustively searching every entry in the reservoir, PLUG provides functionality to iteratively sample candidate training proteins that satisfy the desired training distribution. For example, training proteins can be sampled by sequence length, protein family, deposition date, or other user-defined criteria. Candidate proteins are repeatedly sampled and screened until the desired training set quota is reached.

Training sets are evaluated for contamination at the level of full proteins, domains, and subdomains using multiple coverage modes. In the case of a model trained on both sequence and structure, a union graph is built over sequence and structure, and entire clusters are removed until the training set is adequately safe. Thresholds and parameters are deliberately user-defined to characterize generalizability as a function of data leakage.

Looking ahead

PLUG is an open-source modular framework intended for continual development. Beyond sequence and structural homology filtering, future extensions include additional sampling strategies, improved ligand encoding, and expanded benchmark coverage aimed at conformational dynamics. In a forthcoming publication, we will describe the complete methodology, present the full framework, and evaluate current models across the benchmark suite.

We hope PLUG provides the community with a reproducible foundation for constructing leakage-controlled protein function benchmarks and contributes to more rigorous evaluation of protein function models.

Code:

Code is available on GitHub.

We chat about the conformational ensembles conference, NIH-proposed policy changes, 4! preprints, & running.

NIH News:

• OMB Proposal
• Blog Post Summary on OMB Proposal
• RFI to Cap the Number of Simultaneous Research Project Grants (NOT-OD-26-086)
• NIH Director’s Early Independence Award

Preprints:

• Detection of residual native state entropy changes upon mutation in Fyn SH3
• Protein-ligand binding kinetics are primarily controlled by the protein, not the ligand
• ESMFold2

• sampleworks: A Modular Platform for Experimentally Guided Biomolecular Ensemble Generation

• Spotify
• Apple Podcasts

Guidance: Default to open: “When in doubt, put it out”

Written output: FAIR-compliant i.e., The Stacks, BioRxiv, Zenodo, Radial is developing The Stacks and is actively working with groups within the Radial ecosystem to identify the best way to communicate the findings.

Publish as you go: Do not wait for a ‘completed story’

Code: Repositories should be developed in the open.

Data release: Before or at publication release. Use a reasonable balance of releasing data that does not over-burden researchers, but does not hold back data Use existing FAIR repositories whenever available (PDB, EMPIAR, AddGene, Zenodo, protocols.io, etc).

License permissively: (CC0 for content, CC0 or CC-BY for writen publications, MIT or Apache 2.0 for code)

Acknowledge Radial: As a collaborator, you can directly acknowledge Radial in the acknowledgments or support/materials/funding as “Radial (https://ror.org/050rbg919)”

sampleworks is an open modular software framework for generating and evaluating biomolecular conformational ensembles from structure predictors guided by experimental data (https://github.com/diff-use/sampleworks/). You can swap in different structure predictors, steering/guidance methods, and loss functions. We welcome community contributions to help strengthen the codebase and expand its applications. Development is already underway by the DiffUSE team and external collaborators to integrate new structure prediction models and guidance from reciprocal-space data.

In this first publication, we use sampleworks to examine how well current models recover experimentally determined alternative conformations from the Protein Data Bank. Success varied widely across proteins, suggesting that models may be memorizing rather than learning the true distribution of structural states.

True to Astera’s open-science ethos, we have developed sampleworks in the open. The code has lived in a public Git repository throughout. Alongside it, we documented progress in real time through blog posts such as “To ensembles and beyond”. These provided informal dispatches that shared early thinking and invited community engagement. This is highlighted by the forks the GitHub repository already has. While the blog posts offer speed and accessibility, the scholarly publication is a freestanding, reproducible, citable unit that links all underlying research artifacts to the relevant context and is disseminated with a persistent digital object identifier (DOI). Together, these modes of communication embody open science in practice. Blog posts, open development, and scholarly publication each address a different dimension of scientific communication. Their combination makes the whole greater than the sum of its parts.

The DiffUSE Project is sharing this first publication on The Stacks, an experimental publishing platform currently open to select users, to spark broader public discussion than typically done during closed pre-publication review in traditional scientific journals. The work remains both citable and discoverable, so it will receive a persistent identifier and be indexed in Google Scholar and Open Alex. At this moment, The Stacks is not yet open to external submissions and is hosting work produced internally by Radial and affiliated organizations.

DiffUSE’s vision, a world where we can observe, predict, and act on how proteins move, is not a challenge any single team can tackle alone. By openly developing and sharing data and results, we aim to accelerate the field’s progress toward establishing conformational ensembles as its foundational layer. With that foundation, we will greatly improve our understanding of life at the molecular level, accelerating our ability to design therapeutics, engineer enzymes, and predict the functional consequences of mutations.

We chat about NIH strategic plan comments, AI lab guidelines, preprints on cryo-EM vitrification and automated image processing, and training scientists for the future.

NIH Strategic Plan Comments (Due 5/26):

• NIH seeks input on framework for next NIH-wide strategic plan

Preprints:

• Cooling fast and slow: Characterising the effects of vitrification in cryo-EM and the subsequent recovery of equilibrium populations
• cryoAgent: An agentic workflow for robust and adaptive end-to-end cryo-EM image processing

Other Links:

• You need to make AI guidelines for your lab

• We’re Training Scientists for a World That No Longer Exists

• Spotify
• Apple Podcasts

PDBx/mmCIF models do not adequately encode conformational and compositional heterogeneity intrinsic in structural biology data. This problem becomes more pressing when examining new classes of experiments that frequently or expressly induce such heterogeneities, including fragment screening and time-resolved techniques. Leveraging mmCIF’s expressive nature, we propose a new category that enables users and software to explicitly describe heterogeneity by explicitly encoding the relationships between different pieces of heterogeneity.

The Problem

Most PDB depositions result from X-ray crystallography or single-particle cryo-electron microscopy (cryo-EM), which collect information from thousands to trillions of copies of the macromolecular complex. Each copy of the macromolecular complex can vary in the relative positions of atoms within it (conformational heterogeneity) or in the presence/absence of part of the complex (compositional heterogeneity). Conformational heterogeneity can occur across length scales from a single atom (e.g. a terminal hydroxyl in serine partially occupying two locations) to entire domain movements (e.g. the ribosome stalk partially occupying two locations)1–6. Compositional heterogeneity can also occur across length scales from an ion to a protein subunit7–11. Models intended to describe this experimental data must be able to effectively capture this inherent conformational and compositional heterogeneity; however, the current encoding falls short.

Currently, in PDBx/mmCIFs, harmonic conformational heterogeneity is captured by B-factors or translation-libration-screw [TLS] parameters. In contrast, non-harmonic conformational or compositional heterogeneity is captured through the alternative location indicator (altloc). Altlocs indicate the mutually exclusive positions any given atom can occupy, providing information on coordinates, occupancy, and B-factors for each position. Multiple atoms are often grouped into the same altloc, indicating that, on a local level, they are part of the same state.

This convention falls well short when it becomes necessary to describe how altlocs are related to each other12. It has long been recognized that additional information on the relationships between altlocs is necessary to adequately fit this data13,14. However, this information is only provided in refinement programs (via grouped occupancy) and is not explicitly stated in deposited models.

For example, conditional on the presence or absence of a ligand, two ethylene glycol (EDO) molecules may be present, one of which can occupy two mutually exclusive states (Figure 1a). While there are ways to apply restraints and constraints in refinement, this conditionality cannot be expressed in PDBx/mmCIF altlocs. This limitation hurts our ability to read a model and understand the downstream

Biological Applications

While the interplay between conformational and compositional heterogeneity is present in all structural experiments, accurate encoding has particularly large implications for several classes of experiments that frequently or explicitly induce such heterogeneities, including fragment screening and time-resolved techniques.

Prominent use cases include: X-ray crystallography fragment screening: X-ray crystallography fragment screening is an increasingly popular drug discovery technique that involves collecting hundreds to thousands of crystal structures of small-molecule fragments bound to proteins. Collectively, these structures provide building blocks for ligands by observing and modeling diverse small molecule starting points15. The challenge arises because the fragment or ligand is bound to only a subset of macromolecules, inducing partial occupancy changes across states, resulting in experimental data with high compositional and conformational heterogeneity 10,16,17. Such screens make up an increasing fraction of depositions in the PDB and are of significant importance to the pharmaceutical industry. A clear understanding of the hierarchical heterogeneity is essential for future chemical modifications and for using these data in automated methods or machine learning.

Time-resolved experiments: Time-resolved X-ray crystallography or cryo-EM involves collecting multiple datasets over time as the sample undergoes a dynamic process. To achieve this, the sample (typically protein crystals) is rapidly subjected to a specific stimulus (e.g. with temperature, light, or electric field). The changes induce structural rearrangements in the protein, and diffraction data are collected at multiple time points, enabling real-time tracking of these changes. These studies are critical to understanding protein function in terms of time-domain structural heterogeneity, such as catalysis, ligand binding, or macromolecular machines 18–22. While experiments often yield multiple depositions linked to different maps, within a single deposition, the ground state contains a mixture of conformers, and the excited state also contains a mixture of conformers, although we expect different occupancies. The inability to express the dependencies among occupancy groups in the mmCIF/PDBx means that the structure can be understood only with the accompanying literature, which is problematic for automated methods and machine learning.

Other experiments purposefully seek to enrich compositional heterogeneity. In these cases, a single collection reflects data from many biochemically distinct samples. For example, a collection of small molecules in a “cocktail” can be soaked into the same protein crystal, resulting in a single protein bound to many ligands - potentially in the same binding site 23. In another example, native complexes with multiple binding partners can be purified for Cryo-EM 24 or imaged by cryo-electron tomography (cryo-ET) 6,25.

Proposed encoding of heterogeneity in PDBx/mmCIF

Thanks to the more expressive data model of PDBx/mmCIF, it is now possible to explicitly encode heterogeneity-related relationships that are implicit or ambiguous in current representations. To enable this, we propose a new category that directly encodes (i) the hierarchical relationships among compositional and conformational states and (ii) the coexistence or mutual exclusivity of those states. Together, these two tables provide a formal mechanism for representing both the dependency structure and logical compatibility among heterogeneous states in multiconformer or ensemble models. The core table, termed the heterogeneity hierarchy, is a mandatory loop that links heterogeneity identifiers associated with atoms in the _atom_site category (and, when applicable, model IDs) to one another through explicit parent–child relationships. This hierarchy encodes how individual heterogeneous states are related—such as alternative ligand-binding events, conformational substates, or compositional variants—allowing users and software to reconstruct a structured view of heterogeneity rather than relying on proximity-based altloc conventions alone. The hierarchy table is complemented by a second table, the state coexistence table, which explicitly specifies which heterogeneity states may or may not co-occur. Within the _atom_site category, we introduce a heterogeneity ID. This identifier is optional for PDB entries that do not require explicit heterogeneity encoding, but becomes mandatory for atoms that participate in the heterogeneity hierarchy. In some cases, heterogeneity IDs may correspond directly to existing altloc identifiers; however, unlike altlocs, each heterogeneity ID must represent a distinct biochemical or structural state. For example, two spatially distant atoms may both be labeled with altloc “A” under current conventions, even though there is no evidence that they represent the same underlying state. In such cases, distinct heterogeneity IDs would be assigned, allowing their relationships (or lack thereof) to be represented unambiguously within the hierarchy. The heterogeneity hierarchy category uses four fields: name, id, parent, and details. The name field is mandatory and provides a short, human-readable label for the state. The id field is the category key and serves as the formal link between this table and the _atom_site category. The optional details field allows descriptive text (quoted if it contains spaces). The parent field references another heterogeneity ID and encodes the hierarchical structure: each heterogeneity ID has at most one parent, whereas any parent may have multiple children. This constraint enforces a tree-like hierarchy that captures how states refine or branch from one another. By traversing parent–child relationships, software can explicitly reconstruct the hierarchy of heterogeneity encoded across the atomic model. Unlike traditional altloc usage—where identical altloc identifiers implicitly define a single mutually exclusive state within a local clash radius—the heterogeneity hierarchy provides a global, explicit description of state relationships. When conformational or compositional states are known to be linked biochemically, physically, or mechanistically, they may share the same altloc and heterogeneity ID. Conversely, when no such linkage is known, distinct heterogeneity IDs are assigned, even if the altloc labels coincide. This ensures that altloc usage remains compatible with existing practice while heterogeneity IDs provide the formal semantic layer needed to interpret relationships correctly. Mutual compatibility among states is encoded separately in the state coexistence table. This table operates on heterogeneity IDs defined in the hierarchy and specifies logical rules(e.g., OR or NOT) that determine whether states may co-occur. Hierarchical relationships inherently encode AND relationships (parent and child states are coexistent by definition), whereas the coexistence table captures non-hierarchical logical constraints. Importantly, if no coexistence rule is specified between two heterogeneity IDs, they are assumed not to be mutually exclusive. As with altlocs, individual heterogeneity IDs are not expected to represent complete models in isolation; rather, the full hierarchy and coexistence definitions together should account for all atoms and fully explain the experimental data.

Examples

A few examples are visually laid out here. Example mmCIF models and other updated information can be found in the Encoding Github Repository.

Consider a common situation in fragment screen data with extensive compositional heterogeneity. This structure has four ligands: three EDO molecules and a small fragment bound (Figure 1A). Based on occupancy, biochemical information, and overlap, it is known that when the fragment or EDO1 is bound, either EDO2 or EDO3 can then be bound. When EDO2 or EDO3 is bound, either EDO1 or the fragment is bound (Figure 1B). This example is shown in the proposed heterogeneity table in Figure 1C, and Figure 1D displays the atom table (which shows only one atom per ligand for simplicity).

Situations not addressed by the heterogeneity category

We deliberately chose not to encode the following scenarios in our current model. We excluded a many-to-many relationship, specifying that a parent could have multiple children, but a child could not be associated with multiple parents. This does not strike us as a scientifically sensible thing to do: it would hardcode the commonality between conformations, whereas rationalising such correlations is decidedly a downstream task, namely the analysis of what the models mean, and highly dependent on the questions asked. We did not include the association of multiple maps, such as those used in time-resolved or classification methods in cryoEM. It is conceivable that a single model, supported by multiple maps, could be created within this hierarchical framework in the future. The current encoding of hierarchical compositional and conformational heterogeneity is based solely on thermodynamic ensembles without accounting for timescales. However, the framework is flexible enough that timescale information could be incorporated into the table, such as with time-resolved techniques.

Working with existing software

As the proposed loop does not change the atom_site category, it should not break any existing software. However, to fully realize the potential of this encoding, it should be used by existing structural biology software, including model building10,27–30, refinement programs31–33, and visualization software30,34. Overall, these programs need to account for the heterogeneity loop when present. Additionally, software should account for the possibility that models using the new category may interpret altlocs slightly differently, with each altloc representing a distinct state.

Model Building

For modeling building software, the software should, to the best of its ability, output the loop and/or a grouped occupancy file describing how altlocs are related. While this loop/grouped occupancy may need to be manually adjusted for local heterogeneity hierarchy, it can be automated when clashes determine labeling, as is currently done in qFit and Coot28,30. Determining distally hierarchical relationships cannot be done using information from atom_site and will require external methods and individual input from modelers. Additionally, model-building software should err on the side of assigning new altlocs (_atom_site.label_alt_id), unless there is evidence two pieces of heterogeneity are linked. In Coot, we imagine a user-defined flag (i.e. use hierarchy category) for assigning altlocs. Coot would also need to re-assign altlocs as altlocs are deleted from the model during manual building. If the hierarchy category exists in the input model, Coot must amend the table as needed. Currently, this would be based solely on user input and/or conflicting information regarding the conformations.

We also encourage the development of tools that enable modelers to specify the hierarchical relationships between altlocs through a grouped occupancy file or by manually creating the hierarchy loop, with the software automatically correcting the labeling of altlocs in the atom table.

Refinement

We propose that refinement software include a flag to enable the use of information from the hierarchy category. If the flag is enabled, the software should expect a grouped-occupancy file and/or a hierarchy category within the PDBx/mmCIF file. Refinement software should then generate constraints and/or restraints based on the hierarchy or grouped occupancy file or hierarchy category. If a grouped occupancy file is present, it will overwrite the existing hierarchy category in the PDBx/mmCIF file. Refinement should read the hierarchy loop from children to parents and/or use a grouped occupancy file to determine which hierarchy levels should be equal to or less than one occupancy. Similar to Refmac’s current “occupancy group alts complete” parameter, we envision a modified version of this command that would create a restraint ensuring that the occupancy of specific IDs sums to that of a specified parent, reflecting their hierarchical relationship. This would be in addition to the restraint of having atoms with the same ID (pdbx_heterogeneity_hierarchy.id) having the same occupancy. In Phenix and SHELX31,35, while you can create constraints of mutually exclusive groups using grouped occupancy or PART parameters, it is currently impossible to encode the hierarchy between these groups. When flagged, refinement software must output the category in the PDBx/mmCIF file, specifying how heterogeneity hierarchy is encoded and refined within the model. Visualization Visualization software should read the hierarchical heterogeneity loop to display hierarchical heterogeneity. Currently, Pymol supports the use of altlocs in selection criteria and visualization. We aim for Pymol to support the new heterogeneity loop directive in selection and visualization, allowing users to select a state and display all its children and/or parents.

Future Indications

This proposal is primarily motivated by X-ray crystallography, which enables the precise detection of individual states owing to its high resolution and recent technological advances. While cryo-EM currently captures most of its compositional and conformational heterogeneity through 3D classification of maps36, we are beginning to routinely obtain structures at sufficiently high resolution to capture heterogeneity through both map classification and the encoding of multiplicity within structural models.

Furthermore, the machine-readability and human interpretability of the inherent heterogeneity within models are critical for advancing the use and prediction of conformational ensembles. By implementing this model, we anticipate that it will yield new tools and analyses to examine heterogeneity within and across structures, significantly expanding our understanding of the inherent heterogeneity in experimental structural biology data.

References

1. Baxevanis, A. D. & Francis Ouellette, B. F. Bioinformatics: A Practical Guide to the Analysis of Genes and Proteins. (John Wiley & Sons, 2004).
2. DePristo, M. A., de Bakker, P. I. W. & Blundell, T. L. Heterogeneity and inaccuracy in protein structures solved by X-ray crystallography. Structure 12, 831–838 (2004).
3. Leschziner, A. E. & Nogales, E. Visualizing flexibility at molecular resolution: analysis of heterogeneity in single-particle electron microscopy reconstructions. Annu Rev Biophys Biomol Struct 36, 43–62 (2007).
4. Smith, J. L., Hendrickson, W. A., Honzatko, R. B. & Sheriff, S. Structural heterogeneity in protein crystals. Biochemistry 25, 5018–5027 (1986).
5. Zhong, E. D., Bepler, T., Berger, B. & Davis, J. H. CryoDRGN: reconstruction of heterogeneous cryo-EM structures using neural networks. Nat Methods 18, 176–185 (2021).
6. Powell, B. M. & Davis, J. H. Learning structural heterogeneity from cryo-electron sub-tomograms with tomoDRGN. Nat Methods 21, 1525–1536 (2024).
7. Forsberg, B. O., Shah, P. N. M. & Burt, A. A robust normalized local filter to estimate compositional heterogeneity directly from cryo-EM maps. Nat Commun 14, 5802 (2023).
8. Rabuck-Gibbons, J. N., Lyumkis, D. & Williamson, J. R. Quantitative mining of compositional heterogeneity in cryo-EM datasets of ribosome assembly intermediates. Structure 30, 498–509.e4 (2022).
9. Du, S. et al. Refinement of multiconformer ensemble models from multi-temperature X-ray diffraction data. Methods Enzymol 688, 223–254 (2023).
10. Pearce, N. M. et al. A multi-crystal method for extracting obscured crystallographic states from conventionally uninterpretable electron density. Nat Commun 8, 15123 (2017).
11. Pearce, N. M., Krojer, T. & von Delft, F. Proper modelling of ligand binding requires an ensemble of bound and unbound states. Acta Crystallogr D Struct Biol 73, 256–266 (2017).
12. Wankowicz, S. A. & Fraser, J. S. Comprehensive encoding of conformational and compositional protein structural ensembles through the mmCIF data structure. IUCrJ 11, 494–501 (2024).
13. Hendrickson, W. A. Stereochemically restrained refinement of macromolecular structures. Methods Enzymol 115, 252–270 (1985).
14. Afonine, P. V. et al. Towards automated crystallographic structure refinement with phenix.refine. Acta Crystallogr D Biol Crystallogr 68, 352–367 (2012).
15. Correy, G. J. et al. Extensive exploration of structure activity relationships for the SARS-CoV-2 macrodomain from shape-based fragment merging and active learning. bioRxiv (2024) doi:10.1101/2024.08.25.609621.
16. Douangamath, A. et al. Achieving Efficient Fragment Screening at XChem Facility at Diamond Light Source. J Vis Exp (2021) doi:10.3791/62414.
17. Erlanson, D. et al. Where to house big data on small fragments? ChemRxiv (2025) doi:10.26434/chemrxiv-2025-hjjnj.
18. Šrajer, V. & Schmidt, M. Watching Proteins Function with Time-resolved X-ray Crystallography. J Phys D Appl Phys 50, (2017).
19. De Zitter, E., Coquelle, N., Oeser, P., Barends, T. R. M. & Colletier, J.-P. Xtrapol8 enables automatic elucidation of low-occupancy intermediate-states in crystallographic studies. Commun Biol 5, 640 (2022).
20. Greisman, J. B. et al. Resolving conformational changes that mediate a two-step catalytic mechanism in a model enzyme. bioRxiv (2023) doi:10.1101/2023.06.02.543507.
21. Wolff, A. M. et al. Mapping protein dynamics at high spatial resolution with temperature-jump X-ray crystallography. Nat Chem 15, 1549–1558 (2023).
22. Thompson, M. C. et al. Temperature-jump solution X-ray scattering reveals distinct motions in a dynamic enzyme. Nat Chem 11, 1058–1066 (2019).
23. Verlinde, C. L. M. J. et al. Fragment-based cocktail crystallography by the medical structural genomics of pathogenic protozoa consortium. Curr Top Med Chem 9, 1678–1687 (2009).
24. Chen, Z. et al. EMC chaperone-Ca structure reveals an ion channel assembly intermediate. Nature 619, 410–419 (2023).
25. Baker, L. A., Grange, M. & Grünewald, K. Electron cryo-tomography captures macromolecular complexes in native environments. Curr Opin Struct Biol 46, 149–156 (2017).
26. Westbrook, J. D. et al. PDBx/mmCIF Ecosystem: Foundational Semantic Tools for Structural Biology. J Mol Biol 434, 167599 (2022).
27. Flowers, J. et al. Expanding Automated Multiconformer Ligand Modeling to Macrocycles and Fragments. bioRxiv (2024) doi:10.1101/2024.09.20.613996.
28. Wankowicz, S. A. et al. Automated multiconformer model building for X-ray crystallography and cryo-EM. Elife 12, (2024).
29. Stachowski, T. R. & Fischer, M. FLEXR: automated multi-conformer model building using electron-density map sampling. Acta Crystallogr D Struct Biol 79, 354–367 (2023).
30. Emsley, P., Lohkamp, B., Scott, W. G. & Cowtan, K. Features and development of Coot. Acta Crystallogr D Biol Crystallogr 66, 486–501 (2010).
31. Adams, P. D. et al. PHENIX: a comprehensive Python-based system for macromolecular structure solution. Acta Crystallogr D Biol Crystallogr 66, 213–221 (2010).
32. Smart, O. S. et al. Exploiting structure similarity in refinement: automated NCS and target-structure restraints in BUSTER. Acta Crystallogr D Biol Crystallogr 68, 368–380 (2012).
33. Murshudov, G. N. et al. REFMAC5 for the refinement of macromolecular crystal structures. Acta Crystallogr D Biol Crystallogr 67, 355–367 (2011).
34. Lill, M. A. & Danielson, M. L. Computer-aided drug design platform using PyMOL. J Comput Aided Mol Des 25, 13–19 (2011).
35. Schneider, T. R. & Sheldrick, G. M. Substructure solution with SHELXD. Acta Crystallogr D Biol Crystallogr 58, 1772–1779 (2002).
36. Kimanius, D., Dong, L., Sharov, G., Nakane, T. & Scheres, S. H. W. New tools for automated cryo-EM single-particle analysis in RELION-4.0. Biochem J 478, 4169–4185 (2021).

On the DiffUSE project, we have been thinking about solvent modeling and major hurdles that need to be overcome as we model protein ensembles (see our previous all hands post for more context). In this post, I hope to initiate discussions on why water prediction is hard with some preliminary analysis. In particular, is there something about the deposited data that makes them difficult to learn by data-driven approaches?

How is water modeled?

In crystallography, solvent modeling is simplified as either ordered or bulk. Bulk solvent represents disordered water molecules with a flat density. Ordered solvent molecules form hydrogen bonds with the protein and/or other solvent and are represented with atomic coordinates. They have attracted much research interest as some ordered water molecules can be conserved across protein families and participate in or influence chemical reactions. Several machine learning approaches, such as SuperWater, HydroProt, and GalaxyWater-CNN attempted to learn directly from the PDB on placing ordered water around proteins. While protein structure predictors trained on the PDB have achieved impressive results, existing water predictors are still not at the same level of out-of-the box performance.

Unlike for proteins, validation procedures are not enforced for the deposition of ordered water in the PDB. Wlodawer et al. analyzed the expected ratio of the number of ordered waters to the number of residues in PDB as a function of the resolution (check out their interactive figure), and found many structures with an unusual number of water molecules modeled, including some completely waterless structures on which the authors did in-depth examination. Realistically, especially in high-resolution structures, only a small subset of ordered waters might be near an active site and are interesting or important enough for the depositor to carefully inspect.

For a scalable approach to model ordered water, established refinement programs have built automated protocols. For example, PDB-REDO re-refines structures at scale with the removal of water molecules that are unsupported by the density. Density-based automated protocols, however, are not perfect, as they are subjected to phase errors. Other heuristics are often used as an auxiliary to model as well as evaluate the individual ordered water molecules. The distance and the number of hydrogen bonds suggest how physically plausible the ordered water is, and the B-factor and density-based metrics such as EDIA show how well the water molecules fit to experimental data.

What do the water data look like?

It is unclear how effective the auto-refinement and popular quality metrics are in providing “clean” data from the perspective of statistical learning. Consider each individual PDB file as containing some water molecules that are true positives, some that are false positives, and some water molecules are missing, how good is a PDB file as an instance of prediction? One way to explore this analysis is by examining conserved waters as a surrogate of the ground truth that can be reliably evaluated against. Our motivation is that the water molecules that consistently show up are the ones we want to predict and are probably more learnable.

For the consensus analysis, I gather isomorphous structures from PDB-REDO for 3 protein cases – 66 structures of hen egg white lysozyme (HEWL), 18 structures of beta-lactamase (TEM1), and 60 structures of protein tyrosine phosphatase 1B (PTP1B) – and cluster the ordered waters with a 1Å distance cutoff after alignment. Water molecules that show up in more than half of the structures (i.e. cluster occupancy >= 0.5) are considered as conserved.

Precision-coverage tradeoff as a function of the number of water

The figures above show that there is a spectrum of how well each PDB file models the set of conserved water when considering a PDB file as an instance of prediction. In all 3 cases, there is a precision-coverage tradeoff with the number of water molecules modeled, which is highly correlated to the resolution of the structure.

High-resolution structures resolve more ordered water molecules and have a good coverage of the conserved water. Low-resolution structures, however, also contain useful information, as they can be quite precise on capturing the conserved waters. In other words, even though some structures have fewer ordered waters resolved, presumably, the ones that can still be modeled have high confidence. The frontier of the precision-coverage tradeoff, however, varies case by case.

Consensus water is a subset of the high quality water

Given the high coverage of conserved water in high resolution structures, one might wonder if we can deploy some filtering to improve the precision of water placement. To this end, I made a scatter plot of the water conservedness against quality metrics. Conservedness is defined as cluster occupancy, which is the fraction of PDB files that has a water molecule near the cluster center.

While there is a positive relationship between high EDIA (or low B-factor) and cluster occupancy, clusters in the upper left corner suggest that there are some water molecules that are high-quality but not conserved. They might be unique waters specific to certain experimental contexts (crystallization condition, presence of ligands, protonation states, etc.). Alternatively, waters at those cluster positions might be missing in some structures due to limited resolution, experimental errors, and modeling errors of water as well as protein (e.g. alternative conformations). This makes interpreting the precision evaluation metric especially challenging.

What should we do about water?

The consensus water analysis here gives a glimpse into the complexity of the data challenges we have on water prediction. Between the true positives and overfitted noises, there are ordered waters in PDB that are hard to validate, let alone to learn. While it is common practice to align and compare related protein structures, similar inspection for water is much rarer and limited to a few case studies. We think it is worth expanding the consensus analysis at scale for more generalizable insights, including identifying potential sources of biases such as the initial model used and depositor preferences.

We understand that consensus water is not equal to the ground truth water, and physical factors on an apparently small or local scale like an alternative conformation of a protein side chain can change the structure of water networks. Predicting water positions from just the protein structure is strictly speaking an ill-posed task. That said, one could still try to learn a statistical prior for experimental guidance as Bayesian refinement, in similar spirit of Sampleworks. We are building a water predictor for this purpose, and stay tuned for Vratin’s post about this!

Echoing discussion points in Wlodawer et al., we hope the structural community can work together on the water modeling challenge – whether it’s increasing awareness at the individual depositor level, exploration of new computational modeling methods and evaluation metrics, or infrastructure-level support in the PDB.

We chat about preprints, AI grad students, the diffUSE project, an April Fools blog, NIH funding news, and the Fraser Lab “broken monitor” April Fools’ Day prank — caught live during recording!

Preprints:

• Intrinsic dataset features drive mutational effect prediction by protein language models
• Ribosome Molecular Aging Shapes Translation Dynamics

AI Grad Student Articles:

• Why I may ‘hire’ AI instead of a graduate student
• Vibe physics: The AI grad student

Other Links:

• diffUSE Project
• April Fools Blog

NIH News:

• Funding Curve

• How the 2025 NIH grant terminations varied by researchers’ demographic groups

• Spotify
• Apple Podcasts

What could we discover about biology if we had protein ensembles at the scale of the Protein Data Bank?

Could we understand how drug efficacy and resistance arise? Could we better explain why mutations cause disease, or what makes a designed enzyme functional? Could we deliberately engineer dynamics to bias cell signaling?

Today, we are excited to announce that the DiffUSE Project is expanding. We are building the infrastructure to collect and analyze protein ensembles at the scale of the Protein Data Bank. Because many of the most important biological questions can only be answered through ensembles.

The DiffUSE Project, by Radial, and funded by the Astera Institute, began by asking how we could rethink the pipeline for extracting the faint yet information-rich diffuse scattering signal in the “background” of X-ray crystallography experiments. Diffuse scattering may provide key, but often untapped, information on the heterogeneity of macromolecules in a crystal. With funding and operational support from the Astera Institute, we assembled a purpose-built team spanning expertise across Astera and multiple institutions and began to re-engineer the components needed to fully optimize this, revealing structural ensembles.

But diffuse scattering is just one window into protein conformational ensembles. Different techniques uncover different facets of the same underlying ensemble reality. Cryo-EM captures heterogeneity across length scales. NMR resolves timescale-specific motions. SAXS reports on global shape fluctuations. HDX-MS maps solvent accessibility changes. No single method can reveal the full dynamic landscape. But we do not yet know which techniques, or which combination, will best uncover these ensembles. So we start with a not-so-simple question: which data are most fruitful for understanding biology?

But collecting data is only part of the challenge. Every stage of structural biology today reinforces a static worldview. Algorithms model single conformations. Representations emphasize static structures. Metrics evaluate fit to a single model. Biological interpretations are based on static structures. Addressing any piece in isolation will fall short. The entire pipeline must be redesigned.

The DiffUSE Project is completely rethinking how we collect, process, model, encode, disseminate, and interpret dynamic structural data. We are designing tools to enable downstream applications that meaningfully impact biology.

At every step, we are guided by the following questions:

• What structural ensemble data are we missing, and how do we build the technologies to capture it?
• Where will structural ensemble data best help reveal biological function?
• How do we create tools that empower everyone, not just specialists, to work with conformational ensembles?
• How do we openly disseminate what we build to maximize scientific impact?

We are taking this approach because we cannot get there alone. AlphaFold succeeded because decades of experimental structure determination built the data foundation it needed. No equivalent foundation exists for ensembles. Building that foundation and building the prediction methods must happen together — at a scale no single group can achieve. That is why we are not just generating data ourselves. We are building the tools, methods, and infrastructure to enable the entire community to collect, curate, and benchmark dynamic structural data.

This entire project is also a metascience experiment that asks what it truly takes to tackle a large, systematic, and deeply integrated scientific problem. Some of what lies ahead is engineering. Some of it is science. Some of it is community building. We are pursuing this work both internally at Radial and through a growing network of partnerships and collaborations. Experimenting not only with the science itself but with how best to do it.

What are we building?

The DiffUSE project will focus on four areas, each advancing a different part of the dynamic structural pipeline simultaneously. Technique focused spokes will transform specialized dynamics techniques into routine, accessible methods. Algorithms will uncover hidden conformational states and integrate signals across techniques to help reveal the full dynamic landscape and provide key integration points for ensemble prediction. Infrastructure will evolve the PDB from a static archive into a living database capable of storing, validating, and querying dynamic structural data. And Biological Impact will ask questions that help ground our technical advance by asking: how does understanding dynamics actually change what we can understand, predict, design, and discover biology?

Technique focused spokes: Every technique for capturing protein ensembles faces its own barriers to making the data truly usable. We are aiming to experiment with how to overcome these barriers and make these techniques routine and accessible for the broader community. Current methods often discard the dynamic signal, and none were designed to scale to the thousands of ensembles needed to uncover general principles. Each spoke tackles these challenges for a specific technique, producing open algorithms, protocols, models, and metrics, all designed from the start to be adopted and built upon by others. We do not yet know which techniques will generate truly additive data, which approaches will scale, or which will result in the best biological insights. We plan to run parallel experiments, and we will follow the evidence, expanding what works, rethinking what doesn’t, and sharing what we learn along the way.

Algorithms: No single technique captures the full picture of how a protein moves. Each method sees a different slice of the conformational landscape. To tackle this, we are developing algorithms along three fronts: guidance frameworks that steer ensemble models using experimental data, machine learning approaches that learn directly from experimental measurements rather than structures, and agentic modeling pipelines that autonomously build, evaluate, and refine ensemble models. Everything we build is open, designed for the community to use, extend, and improve. Our first major algorithm is Sampleworks, a platform for testing new guidance frameworks, stress-testing ensemble predictions, understanding where they fail, and establishing the datasets needed to train the next generation of machine learning models on conformational ensembles rather than static snapshots.

Infrastructure: The PDB transformed structural biology by providing the community with a shared, standardized, and trusted platform for depositing and retrieving structures. But the PDB was designed for static structures. Without accessible ensemble data, researchers default to inconsistent practices: ignoring the inherent heterogeneity in the data and training models on static structures that don’t reflect how proteins actually behave. These limitations don’t stay contained; they propagate through the entire discovery workflow, leading to biased biological conclusions.

We are working on evolving the PDB into a living database that supports dynamic structural data through three advances. First, ensemble-aware validation, because today, when an ensemble model is deposited, only part of it is validated against experimental data. Validation metrics are essential for model deposition and for benchmarking new machine learning models. Second, heterogeneity-aware encoding enables queries that let users do things currently impossible: retrieve only ligand-bound states across the database, find entries where a specific loop adopts multiple conformations, or compare ensemble representations across related proteins. Third, a unified, continually updated repository where experimental data serve as ground truth and the models explaining them improve over time as algorithms advance. The endgame is to enable a virtuous cycle where labs deposit dynamic data, the community builds better ensemble models, those models generate new biological insights, and the standardized data fuels structural ensemble prediction.

Biological Impact: We want our efforts to be grounded in a simple, but demanding question: how does understanding dynamics change what we can predict, design, and discover? To answer that, we are building ensemble-aware representations to help quantify metrics for what has long been hard to quantify, conformational and solvent entropy, flexibility, and long-range dynamic coupling, and validating them against real biological problems in binding, enzyme design, and allosteric regulation. But metrics alone aren’t enough. We aim to help understand how dynamics propagate and change protein function. How does a single mutation redistribute an entire conformational ensemble and alter functional output? How do changes in hydration networks reshape binding? Can we trace these effects well enough to use dynamics not just as a descriptor, but as a design principle, such as designing allosteric switches by tuning the dynamic landscape rather than the static structure?

What’s next?

I am thrilled to be joining the DiffUSE team full-time as Scientific Program Director. In this role, I will design and direct the roadmap to unlock protein ensembles. I will be directly leading the efforts in algorithms, infrastructure, and biology, and guiding technical deep dives. My passion has always been building tools and working with open data. I see this role and this project as an incredible opportunity to deliver on this vision of unlocking protein ensembles at scale. The biology questions I care most about, the role that ensembles and entropy play in protein function, and how we leverage that understanding for design, can only be answered once we commit to an ensemble point of view. That is exactly what the DiffUSE Project intends to do. We’re also growing the team on the algorithms, infrastructure, and biology team, engaging with those looking to build open tools to make this shift happen. And we are doing it all in the open.

Our point is this - break’s over.

Generative biomolecular structure predictors like AlphaFold 3 (Abramson et al., 2024), Boltz-1/2 (Passaro et al., 2025, Wohlwend et al., 2025), RoseTTAFold 3 (Corley et al., 2025), and Protenix (Protenix Team, 2026) produce remarkably accurate single-state structures. These structure predictors sample from a distribution of conformations learned from data–but largely from the single-state structures they were trained on, implicitly assuming a single protein sequence maps to just one conformation. We have only begun to understand the extent to which these models learn the actual conformational heterogeneity of each protein, or simply memorize the individual protein structures (Chakravarty et al., 2024; del Alamo et al., 2022).

“Diffusion”-based generative structure predictors work by predicting a series of updates which map from noise to a final protein structure–creating a “trajectory” that can be steered by modifying the updates to better match some criteria–such as how well the model fits experimental data. This “guidance” approach was recently used (Maddipatla et al., 2026) to steer Protenix to identify previously unmodeled states in existing PDB entries. CryoBoltz (Raghu et al. 2025) similarly guides Boltz-1 to better model cryoEM density maps when multiple conformations are found during classification of the particle stack. But to date, there have been no systematic explorations of how well different structure predictors respond to guidance, or of the trade-offs involved in applying this guidance to sample ensembles consistent with the experimental data. To simplify and generalize experiments that generate ensemble predictions and use ensemble data, we introduce Sampleworks.

What is Sampleworks?

Sampleworks is a framework for integrating structural biology observables, structure predictors, and guidance to improve the modeling of conformational ensembles. Sampleworks does this by defining a common interface for structure predictors and linking that interface to methods for computing experimental observables, like an X-ray electron density map. Sampleworks then applies existing methods to guide generative models during inference, steering sample generation towards an ensemble that reflects the data.

To date, we’ve implemented:

• Two guidance methods: diffusion posterior sampling (Chung et al., 2024; Levy et al., 2024) and Feynman-Kaç steering (Singhal et al., 2025)
• Wrappers for Boltz-1 and -2; Protenix; and RosettaFold3.
• Guidance based on real-space electron density maps.

We will continue to expand the guidance methods, structure predictors, and the kinds of experimental data used for guidance, and we welcome contributions from this community. You can use Sampleworks, follow our progress, and contribute here!

Where we’re at

Sampleworks’s first goal is to benchmark the extent to which diffusion-based structure predictors can be guided to generate accurate structural ensembles. Specifically, we ask whether these models can produce out-of-distribution conformations and, when they do, how well those conformations balance between fit to experimental data and geometric quality.

Since these structure predictors’ training data were on single state structures, this is a natural test of their ability to generalize. In this first test, we guide the predictors using synthetic electron density maps that mix two conformations. We evaluate how well each predictor recovers the synthetic ensemble while still satisfying protein-geometry constraints.

(Left) Without guidance, Boltz-2 does not predict the two conformations (grey) in 6B8X when run with multiple random seeds. (Right) Guidance using synthetic density representing the two conformations enables Boltz-2 to predict both.

We’ve now tested our initial structure predictors with guidance for 40 different structures from the PDB that have known alternate conformations in the original depositions. In the example above, for the protein phosphatase PTP1B (PDB code 6B8X), the predictor, without guidance, falls back on its training data and can only generate structures in one of the two known conformations. Encouragingly, when incorporating guidance with electron density representing both conformations, we observe the structure predictors now predicting two states. However, this is not universally true across different proteins and models. Because Sampleworks is modular and easily accommodates different structure predictors, we can quickly test them all at different guidance levels across a wide range of proteins.

Geometry penalty vs. fit to data for each model’s ensemble prediction on 40 examples of multiple conformations when guided with synthetic density. (Left) Weak guidance, zoomed close to show differences between models. (Right) Strong guidance, resulting in much larger geometry penalties.

We also assessed the trade-offs between fit to electron-density data and geometry. To determine fit to electron density, we use the Real Space Correlation Coefficient (RSCC). We also compute a conglomerate geometry score, which becomes larger with increased clashes, bad backbone dihedrals, or unrealistic bond lengths. With relatively weak guidance, we can see that the models are all able to improve RSCC without incurring too much cost to protein geometry. Paradoxically, when we use stronger guidance, most structure predictors fall apart completely, and the RSCC gets worse! This probably means that strong guidance pushes the predictors’ trajectories outside what they’ve been trained on, far enough that they cannot recover. Despite their very similar architectures and training data, these structure predictors differ in how they respond to guidance.

We are just beginning to learn how best to guide each model to better model biologically relevant ensembles. We are already seeing signs that the models are limited in their current forms, and we will be working to explore and overcome those limitations. In the coming months, we’ll have more to share about the first results shown above and about the experiments in progress now. We’ll be looking to expand the range of predictors and experimental data we can work with, and to engage with the community to make protein conformational ensembles more accessible for everyone.