Work overview

Section 17 of 17

References

Highly Accurate Protein Structure Prediction with AlphaFold

John Jumper, Richard Evans, Alexander Pritzel, Tim Green, Michael Figurnov, Olaf Ronneberger, Kathryn Tunyasuvunakool, Russ Bates, Augustin Žídek, Anna Potapenko, Alex Bridgland, Clemens Meyer, Simon A. A. Kohl, Andrew J. Ballard, Andrew Cowie, Bernardino Romera-Paredes, Stanislav Nikolov, Rishub Jain, Jonas Adler, Trevor Back, Stig Petersen, David Reiman, Ellen Clancy, Michal Zielinski, Martin Steinegger, Michalina Pacholska, Tamas Berghammer, Sebastian Bodenstein, David Silver, Oriol Vinyals, Andrew W. Senior, Koray Kavukcuoglu, Pushmeet Kohli, and Demis Hassabis · 2021

Contents

Section 17 of 17

  1. 01Abstract
  2. 02Main Results
  3. 03AlphaFold Network
  4. 04Evoformer
  5. 05End-to-End Structure Prediction
  6. 06Training with Labelled and Unlabelled Data
  7. 07Interpreting the Network
  8. 08MSA Depth
  9. 09Related Work
  10. 10Discussion
  11. 11Methods
  12. 12Acknowledgements
  13. 13Author Contributions
  14. 14Data Availability
  15. 15Code Availability
  16. 16Competing Interests
  17. 17References
Text size
Work overview

Section 17 of 17

References

John Jumper, Richard Evans, Alexander Pritzel, Tim Green, Michael Figurnov, Olaf Ronneberger, Kathryn Tunyasuvunakool, Russ Bates, Augustin Žídek, Anna Potapenko, Alex Bridgland, Clemens Meyer, Simon A. A. Kohl, Andrew J. Ballard, Andrew Cowie, Bernardino Romera-Paredes, Stanislav Nikolov, Rishub Jain, Jonas Adler, Trevor Back, Stig Petersen, David Reiman, Ellen Clancy, Michal Zielinski, Martin Steinegger, Michalina Pacholska, Tamas Berghammer, Sebastian Bodenstein, David Silver, Oriol Vinyals, Andrew W. Senior, Koray Kavukcuoglu, Pushmeet Kohli, and Demis Hassabis · about 14 minutes

Original source ↗

References

  • 1.Thompson, M. C., Yeates, T. O. & Rodriguez, J. A. Advances in methods for atomic resolution macromolecular structure determination. F1000Res. 9, 667 (2020). [DOI] [PMC free article] [PubMed] [Google Scholar]

  • 2.Bai, X.-C., McMullan, G. & Scheres, S. H. W. How cryo-EM is revolutionizing structural biology. Trends Biochem. Sci. 40, 49–57 (2015). [DOI] [PubMed] [Google Scholar]

  • 3.Jaskolski, M., Dauter, Z. & Wlodawer, A. A brief history of macromolecular crystallography, illustrated by a family tree and its Nobel fruits. FEBS J. 281, 3985–4009 (2014). [DOI] [PMC free article] [PubMed] [Google Scholar]

  • 4.Wüthrich, K. The way to NMR structures of proteins. Nat. Struct. Biol. 8, 923–925 (2001). [DOI] [PubMed] [Google Scholar]

  • 5.wwPDB Consortium. Protein Data Bank: the single global archive for 3D macromolecular structure data. Nucleic Acids Res. 47, D520–D528 (2018). [DOI] [PMC free article] [PubMed] [Google Scholar]

  • 6.Mitchell, A. L. et al. MGnify: the microbiome analysis resource in 2020. Nucleic Acids Res. 48, D570–D578 (2020). [DOI] [PMC free article] [PubMed] [Google Scholar]

  • 7.Steinegger, M., Mirdita, M. & Söding, J. Protein-level assembly increases protein sequence recovery from metagenomic samples manyfold. Nat. Methods16, 603–606 (2019). [DOI] [PubMed] [Google Scholar]

  • 8.Dill, K. A., Ozkan, S. B., Shell, M. S. & Weikl, T. R. The protein folding problem. Annu. Rev. Biophys. 37, 289–316 (2008). [DOI] [PMC free article] [PubMed] [Google Scholar]

  • 9.Anfinsen, C. B. Principles that govern the folding of protein chains. Science181, 223–230 (1973). [DOI] [PubMed] [Google Scholar]

  • 10.Senior, A. W. et al. Improved protein structure prediction using potentials from deep learning. Nature577, 706–710 (2020). [DOI] [PubMed] [Google Scholar]

  • 11.Wang, S., Sun, S., Li, Z., Zhang, R. & Xu, J. Accurate de novo prediction of protein contact map by ultra-deep learning model. PLOS Comput. Biol. 13, e1005324 (2017). [DOI] [PMC free article] [PubMed] [Google Scholar]

  • 12.Zheng, W. et al. Deep-learning contact-map guided protein structure prediction in CASP13. Proteins87, 1149–1164 (2019). [DOI] [PMC free article] [PubMed] [Google Scholar]

  • 13.Abriata, L. A., Tamò, G. E. & Dal Peraro, M. A further leap of improvement in tertiary structure prediction in CASP13 prompts new routes for future assessments. Proteins87, 1100–1112 (2019). [DOI] [PubMed] [Google Scholar]

  • 14.Pearce, R. & Zhang, Y. Deep learning techniques have significantly impacted protein structure prediction and protein design. Curr. Opin. Struct. Biol. 68, 194–207 (2021). [DOI] [PMC free article] [PubMed] [Google Scholar]

  • 15.Moult, J., Fidelis, K., Kryshtafovych, A., Schwede, T. & Topf, M. Critical assessment of techniques for protein structure prediction, fourteenth round. CASP 14 Abstract Bookhttps://www.predictioncenter.org/casp14/doc/CASP14_Abstracts.pdf (2020).

  • 16.Brini, E., Simmerling, C. & Dill, K. Protein storytelling through physics. Science370, eaaz3041 (2020). [DOI] [PMC free article] [PubMed] [Google Scholar]

  • 17.Sippl, M. J. Calculation of conformational ensembles from potentials of mean force. An approach to the knowledge-based prediction of local structures in globular proteins. J. Mol. Biol. 213, 859–883 (1990). [DOI] [PubMed] [Google Scholar]

  • 18.Šali, A. & Blundell, T. L. Comparative protein modelling by satisfaction of spatial restraints. J. Mol. Biol. 234, 779–815 (1993). [DOI] [PubMed] [Google Scholar]

  • 19.Roy, A., Kucukural, A. & Zhang, Y. I-TASSER: a unified platform for automated protein structure and function prediction. Nat. Protocols5, 725–738 (2010). [DOI] [PMC free article] [PubMed] [Google Scholar]

  • 20.Altschuh, D., Lesk, A. M., Bloomer, A. C. & Klug, A. Correlation of co-ordinated amino acid substitutions with function in viruses related to tobacco mosaic virus. J. Mol. Biol. 193, 693–707 (1987). [DOI] [PubMed] [Google Scholar]

  • 21.Shindyalov, I. N., Kolchanov, N. A. & Sander, C. Can three-dimensional contacts in protein structures be predicted by analysis of correlated mutations? Protein Eng. 7, 349–358 (1994). [DOI] [PubMed] [Google Scholar]

  • 22.Weigt, M., White, R. A., Szurmant, H., Hoch, J. A. & Hwa, T. Identification of direct residue contacts in protein–protein interaction by message passing. Proc. Natl Acad. Sci. USA106, 67–72 (2009). [DOI] [PMC free article] [PubMed] [Google Scholar]

  • 23.Marks, D. S. et al. Protein 3D structure computed from evolutionary sequence variation. PLoS ONE6, e28766 (2011). [DOI] [PMC free article] [PubMed] [Google Scholar]

  • 24.Jones, D. T., Buchan, D. W. A., Cozzetto, D. & Pontil, M. PSICOV: precise structural contact prediction using sparse inverse covariance estimation on large multiple sequence alignments. Bioinformatics28, 184–190 (2012). [DOI] [PubMed] [Google Scholar]

  • 25.Moult, J., Pedersen, J. T., Judson, R. & Fidelis, K. A large-scale experiment to assess protein structure prediction methods. Proteins23, ii–iv (1995). [DOI] [PubMed] [Google Scholar]

  • 26.Kryshtafovych, A., Schwede, T., Topf, M., Fidelis, K. & Moult, J. Critical assessment of methods of protein structure prediction (CASP)-round XIII. Proteins87, 1011–1020 (2019). [DOI] [PMC free article] [PubMed] [Google Scholar]

  • 27.Zhang, Y. & Skolnick, J. Scoring function for automated assessment of protein structure template quality. Proteins57, 702–710 (2004). [DOI] [PubMed] [Google Scholar]

  • 28.Tu, Z. & Bai, X. Auto-context and its application to high-level vision tasks and 3D brain image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 32, 1744–1757 (2010). [DOI] [PubMed] [Google Scholar]

  • 29.Carreira, J., Agrawal, P., Fragkiadaki, K. & Malik, J. Human pose estimation with iterative error feedback. In Proc. IEEE Conference on Computer Vision and Pattern Recognition 4733–4742 (2016).

  • 30.Mirabello, C. & Wallner, B. rawMSA: end-to-end deep learning using raw multiple sequence alignments. PLoS ONE14, e0220182 (2019). [DOI] [PMC free article] [PubMed] [Google Scholar]

  • 31.Huang, Z. et al. CCNet: criss-cross attention for semantic segmentation. In Proc. IEEE/CVF International Conference on Computer Vision 603–612 (2019).

  • 32.Hornak, V. et al. Comparison of multiple Amber force fields and development of improved protein backbone parameters. Proteins65, 712–725 (2006). [DOI] [PMC free article] [PubMed] [Google Scholar]

  • 33.Zemla, A. LGA: a method for finding 3D similarities in protein structures. Nucleic Acids Res. 31, 3370–3374 (2003). [DOI] [PMC free article] [PubMed] [Google Scholar]

  • 34.Mariani, V., Biasini, M., Barbato, A. & Schwede, T. lDDT: a local superposition-free score for comparing protein structures and models using distance difference tests. Bioinformatics29, 2722–2728 (2013). [DOI] [PMC free article] [PubMed] [Google Scholar]

  • 35.Xie, Q., Luong, M.-T., Hovy, E. & Le, Q. V. Self-training with noisy student improves imagenet classification. In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition 10687–10698 (2020).

  • 36.Mirdita, M. et al. Uniclust databases of clustered and deeply annotated protein sequences and alignments. Nucleic Acids Res. 45, D170–D176 (2017). [DOI] [PMC free article] [PubMed] [Google Scholar]

  • 37.Devlin, J., Chang, M.-W., Lee, K. & Toutanova, K. BERT: pre-training of deep bidirectional transformers for language understanding. In Proc. 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies 1, 4171–4186 (2019).

  • 38.Rao, R. et al. MSA transformer. In Proc. 38th International Conference on Machine Learning PMLR 139, 8844–8856 (2021).

  • 39.Tunyasuvunakool, K. et al. Highly accurate protein structure prediction for the human proteome. Nature10.1038/s41586-021-03828-1 (2021). [DOI] [PMC free article] [PubMed]

  • 40.Kuhlman, B. & Bradley, P. Advances in protein structure prediction and design. Nat. Rev. Mol. Cell Biol. 20, 681–697 (2019). [DOI] [PMC free article] [PubMed] [Google Scholar]

  • 41.Marks, D. S., Hopf, T. A. & Sander, C. Protein structure prediction from sequence variation. Nat. Biotechnol. 30, 1072–1080 (2012). [DOI] [PMC free article] [PubMed] [Google Scholar]

  • 42.Qian, N. & Sejnowski, T. J. Predicting the secondary structure of globular proteins using neural network models. J. Mol. Biol. 202, 865–884 (1988). [DOI] [PubMed] [Google Scholar]

  • 43.Fariselli, P., Olmea, O., Valencia, A. & Casadio, R. Prediction of contact maps with neural networks and correlated mutations. Protein Eng. 14, 835–843 (2001). [DOI] [PubMed] [Google Scholar]

  • 44.Yang, J. et al. Improved protein structure prediction using predicted interresidue orientations. Proc. Natl Acad. Sci. USA117, 1496–1503 (2020). [DOI] [PMC free article] [PubMed] [Google Scholar]

  • 45.Li, Y. et al. Deducing high-accuracy protein contact-maps from a triplet of coevolutionary matrices through deep residual convolutional networks. PLOS Comput. Biol. 17, e1008865 (2021). [DOI] [PMC free article] [PubMed] [Google Scholar]

  • 46.He, K., Zhang, X., Ren, S. & Sun, J. Deep residual learning for image recognition. In Proc. IEEE Conference on Computer Vision and Pattern Recognition 770–778 (2016).

  • 47.AlQuraishi, M. End-to-end differentiable learning of protein structure. Cell Syst. 8, 292–301 (2019). [DOI] [PMC free article] [PubMed] [Google Scholar]

  • 48.Senior, A. W. et al. Protein structure prediction using multiple deep neural networks in the 13th Critical Assessment of Protein Structure Prediction (CASP13). Proteins87, 1141–1148 (2019). [DOI] [PMC free article] [PubMed] [Google Scholar]

  • 49.Ingraham, J., Riesselman, A. J., Sander, C. & Marks, D. S. Learning protein structure with a differentiable simulator. in Proc. International Conference on Learning Representations (2019).

  • 50.Li, J. Universal transforming geometric network. Preprint at https://arxiv.org/abs/1908.00723 (2019).

  • 51.Xu, J., McPartlon, M. & Li, J. Improved protein structure prediction by deep learning irrespective of co-evolution information. Nat. Mach. Intell. 3, 601–609 (2021). [DOI] [PMC free article] [PubMed] [Google Scholar]

  • 52.Vaswani, A. et al. Attention is all you need. In Advances in Neural Information Processing Systems 5998–6008 (2017).

  • 53.Wang, H. et al. Axial-deeplab: stand-alone axial-attention for panoptic segmentation. in European Conference on Computer Vision 108–126 (Springer, 2020).

  • 54.Alley, E. C., Khimulya, G., Biswas, S., AlQuraishi, M. & Church, G. M. Unified rational protein engineering with sequence-based deep representation learning. Nat. Methods16, 1315–1322 (2019). [DOI] [PMC free article] [PubMed] [Google Scholar]

  • 55.Heinzinger, M. et al. Modeling aspects of the language of life through transfer-learning protein sequences. BMC Bioinformatics20, 723 (2019). [DOI] [PMC free article] [PubMed] [Google Scholar]

  • 56.Rives, A. et al. Biological structure and function emerge from scaling unsupervised learning to 250 million protein sequences. Proc. Natl Acad. Sci. USA118, e2016239118 (2021). [DOI] [PMC free article] [PubMed] [Google Scholar]

  • 57.Pereira, J. et al. High-accuracy protein structure prediction in CASP14. Proteins10.1002/prot.26171 (2021). [DOI] [PubMed] [Google Scholar]

  • 58.Gupta, M. et al. CryoEM and AI reveal a structure of SARS-CoV-2 Nsp2, a multifunctional protein involved in key host processes. Preprint at 10.1101/2021.05.10.443524 (2021).

  • 59.Ingraham, J., Garg, V. K., Barzilay, R. & Jaakkola, T. Generative models for graph-based protein design. in Proc. 33rd Conference on Neural Information Processing Systems (2019).

  • 60.Johnson, L. S., Eddy, S. R. & Portugaly, E. Hidden Markov model speed heuristic and iterative HMM search procedure. BMC Bioinformatics11, 431 (2010). [DOI] [PMC free article] [PubMed] [Google Scholar]

  • 61.Remmert, M., Biegert, A., Hauser, A. & Söding, J. HHblits: lightning-fast iterative protein sequence searching by HMM-HMM alignment. Nat. Methods9, 173–175 (2012). [DOI] [PubMed] [Google Scholar]

  • 62.The UniProt Consortium. UniProt: the universal protein knowledgebase in 2021. Nucleic Acids Res. 49, D480–D489 (2020). [DOI] [PMC free article] [PubMed] [Google Scholar]

  • 63.Steinegger, M. & Söding, J. Clustering huge protein sequence sets in linear time. Nat. Commun. 9, 2542 (2018). [DOI] [PMC free article] [PubMed] [Google Scholar]

  • 64.Steinegger, M. & Söding, J. MMseqs2 enables sensitive protein sequence searching for the analysis of massive data sets. Nat. Biotechnol. 35, 1026–1028 (2017). [DOI] [PubMed] [Google Scholar]

  • 65.Deorowicz, S., Debudaj-Grabysz, A. & Gudyś, A. FAMSA: fast and accurate multiple sequence alignment of huge protein families. Sci. Rep. 6, 33964 (2016). [DOI] [PMC free article] [PubMed] [Google Scholar]

  • 66.Steinegger, M. et al. HH-suite3 for fast remote homology detection and deep protein annotation. BMC Bioinformatics20, 473 (2019). [DOI] [PMC free article] [PubMed] [Google Scholar]

  • 67.Suzek, B. E., Wang, Y., Huang, H., McGarvey, P. B. & Wu, C. H. UniRef clusters: a comprehensive and scalable alternative for improving sequence similarity searches. Bioinformatics31, 926–932 (2015). [DOI] [PMC free article] [PubMed] [Google Scholar]

  • 68.Eddy, S. R. Accelerated profile HMM searches. PLOS Comput. Biol. 7, e1002195 (2011). [DOI] [PMC free article] [PubMed] [Google Scholar]

  • 69.Eastman, P. et al. OpenMM 7: rapid development of high performance algorithms for molecular dynamics. PLOS Comput. Biol. 13, e1005659 (2017). [DOI] [PMC free article] [PubMed] [Google Scholar]

  • 70.Ashish, A. M. A. et al. TensorFlow: large-scale machine learning on heterogeneous systems. Preprint at https://arxiv.org/abs/1603.04467 (2015).

  • 71.Reynolds, M. et al. Open sourcing Sonnet – a new library for constructing neural networks. DeepMindhttps://deepmind.com/blog/open-sourcing-sonnet/ (7 April 2017).

  • 72.Harris, C. R. et al. Array programming with NumPy. Nature585, 357–362 (2020). [DOI] [PMC free article] [PubMed] [Google Scholar]

  • 73.Van Rossum, G. & Drake, F. L. Python 3 Reference Manual (CreateSpace, 2009).

  • 74.Bisong, E. in Building Machine Learning and Deep Learning Models on Google Cloud Platform: A Comprehensive Guide for Beginners 59–64 (Apress, 2019).

  • 75.TensorFlow. XLA: Optimizing Compiler for TensorFlow. https://www.tensorflow.org/xla (2018).

  • 76.Wu, T., Hou, J., Adhikari, B. & Cheng, J. Analysis of several key factors influencing deep learning-based inter-residue contact prediction. Bioinformatics36, 1091–1098 (2020). [DOI] [PMC free article] [PubMed] [Google Scholar]

  • 77.Jiang, W. et al. MrpH, a new class of metal-binding adhesin, requires zinc to mediate biofilm formation. PLoS Pathog. 16, e1008707 (2020). [DOI] [PMC free article] [PubMed] [Google Scholar]

  • 78.Dunne, M., Ernst, P., Sobieraj, A., Pluckthun, A. & Loessner, M. J. The M23 peptidase domain of the Staphylococcal phage 2638A endolysin. PDB10.2210/pdb6YJ1/pdb (2020).

  • 79.Drobysheva, A. V. et al. Structure and function of virion RNA polymerase of a crAss-like phage. Nature589, 306–309 (2021). [DOI] [PMC free article] [PubMed] [Google Scholar]

  • 80.Flaugnatti, N. et al. Structural basis for loading and inhibition of a bacterial T6SS phospholipase effector by the VgrG spike. EMBO J. 39, e104129 (2020). [DOI] [PMC free article] [PubMed] [Google Scholar]

  • 81.ElGamacy, M. et al. An interface-driven design strategy yields a novel, corrugated protein architecture. ACS Synth. Biol. 7, 2226–2235 (2018). [DOI] [PubMed] [Google Scholar]

  • 82.Lim, C. J. et al. The structure of human CST reveals a decameric assembly bound to telomeric DNA. Science368, 1081–1085 (2020). [DOI] [PMC free article] [PubMed] [Google Scholar]

  • 83.Debruycker, V. et al. An embedded lipid in the multidrug transporter LmrP suggests a mechanism for polyspecificity. Nat. Struct. Mol. Biol. 27, 829–835 (2020). [DOI] [PMC free article] [PubMed] [Google Scholar]

  • 84.Flower, T. G. et al. Structure of SARS-CoV-2 ORF8, a rapidly evolving immune evasion protein. Proc. Natl Acad. Sci. USA118, e2021785118 (2021). [DOI] [PMC free article] [PubMed] [Google Scholar]