Apple Researchers Unveil SimpleDesign AI Model for Protein Design

The model follows SimpleFold, a study Apple researchers published last September under the title 'SimpleFold: Folding Proteins is Simpler than You Think.' SimpleFold used a flow-matching model to predict a protein's 3D structure directly from its amino acid sequence. Flow matching starts from a noisy, random basis and learns a relatively direct path to the final result, in contrast with diffusion models, which typically remove noise iteratively. Apple paired flow matching with general-purpose Transformer blocks, avoiding some computationally expensive techniques used by protein-folding models such as DeepMind's AlphaFold.

SimpleDesign applies the same approach to protein design rather than only structure prediction. In a new study titled 'SimpleDesign: A Joint Model for Protein Sequence and Structure Codesign,' Apple researchers write that existing models often rely on multi-stage training, in which autoencoders first tokenize data into latent representations and a generative model is then trained in that latent space. The researchers hypothesize that this multi-stage training is not necessary for a performant co-design model. SimpleDesign is trained directly in data space and learns to generate amino acid sequences and continuous 3D structures in a single end-to-end process.

Many current protein co-design models first train a separate model to convert protein structures into discrete tokens, then train a generative model to produce new sequences and structures. SimpleDesign skips the intermediate step and learns from paired amino acid sequences and 3D coordinates. Apple researchers trained it on more than 2 million protein sequence-and-structure pairs, taken primarily from the AFESM dataset, which combines predicted structures from the AlphaFold Database with additional samples.

During training, the researchers corrupted both parts of each pair. Amino acids in the sequence were randomly hidden behind masked tokens, and noise was added to the corresponding 3D structure. The degree of scrambling varied. When the sequence was mostly intact but the structure heavily corrupted, the task resembled protein folding: the model had to recover a structure from a known sequence. When the structure was mostly intact but the sequence heavily masked, the task resembled inverse folding: the model had to generate a sequence capable of producing a given structure. When both were partially scrambled, the model learned to work on the two problems simultaneously, training it for protein co-design.

According to the study, SimpleDesign delivered competitive results across protein co-design, structure generation, and sequence generation benchmarks despite using a much simpler training pipeline. The researchers also found that it could generate plausible protein structures, and that the amino acid sequences it produced were generally as good as or better than those generated by most competing multimodal models.

The researchers noted that SimpleDesign's results remain limited to computer-based evaluations. The generated proteins were not experimentally tested to confirm that they would fold, function, or behave safely in actual biological systems. The full study goes deeper into the model's architecture, training process, benchmarks, and results.