ACS Publications. Most Trusted. Most Cited. Most Read
Harnessing the Power of Multi-GPU Acceleration into the Quantum Interaction Computational Kernel Program

OR SEARCH CITATIONS

My Activity
Publications

Figure 1Loading Img
Download Hi-Res ImageDownload to MS-PowerPointCite This:J. Chem. Theory Comput. 2021, 17, 7, 3955-3966
    Quantum Electronic Structure

    Harnessing the Power of Multi-GPU Acceleration into the Quantum Interaction Computational Kernel Program
    Click to copy article linkArticle link copied!

    • Madushanka Manathunga
      Madushanka Manathunga
      Department of Chemistry and Department of Biochemistry and Molecular Biology, Michigan State University, 578 S. Shaw Lane, East Lansing, Michigan 48824-1322, United States
      More by Madushanka Manathunga
      Orcidhttps://orcid.org/0000-0002-3594-8112
    • Chi Jin
      Chi Jin
      Department of Chemistry and Department of Biochemistry and Molecular Biology, Michigan State University, 578 S. Shaw Lane, East Lansing, Michigan 48824-1322, United States
      More by Chi Jin
    • Vinícius Wilian D. Cruzeiro
      Vinícius Wilian D. Cruzeiro
      San Diego Supercomputer Center, University of California San Diego, 9500 Gilman Drive, La Jolla, California 92093-0505, United States
      Department of Chemistry and Biochemistry, University of California San Diego, La Jolla, California 92093, United States
      More by Vinícius Wilian D. Cruzeiro
      Orcidhttps://orcid.org/0000-0002-4739-5447
    • Yipu Miao
      Yipu Miao
      Facebook, 1 Hacker Way, Menlo Park, California 94025, United States
      More by Yipu Miao
    • Dawei Mu
      Dawei Mu
      National Center for Supercomputing Applications, University of Illinois at Urbana−Champaign, 1205 W Clark Street, Urbana, Illinois 61801, United States
      More by Dawei Mu
    • Kamesh Arumugam
      Kamesh Arumugam
      NVIDIA Corporation, Santa Clara, California 95051, United States,
      More by Kamesh Arumugam
    • Kristopher Keipert
      Kristopher Keipert
      NVIDIA Corporation, Santa Clara, California 95051, United States,
      More by Kristopher Keipert
    • Hasan Metin Aktulga
      Hasan Metin Aktulga
      Department of Computer Science and Engineering, Michigan State University, 428 S. Shaw Lane, East Lansing, Michigan 48824-1322, United States
      More by Hasan Metin Aktulga
    • Kenneth M. Merz Jr.*
      Kenneth M. Merz, Jr.
      Department of Chemistry and Department of Biochemistry and Molecular Biology, Michigan State University, 578 S. Shaw Lane, East Lansing, Michigan 48824-1322, United States
      *Email: [email protected]
      More by Kenneth M. Merz, Jr.
      Orcidhttps://orcid.org/0000-0001-9139-5893
    • Andreas W. Götz*
      Andreas W. Götz
      San Diego Supercomputer Center, University of California San Diego, 9500 Gilman Drive, La Jolla, California 92093-0505, United States
      *Email: [email protected]
      More by Andreas W. Götz
      Orcidhttps://orcid.org/0000-0002-8048-6906
    Other Access OptionsSupporting Information (2)

    Journal of Chemical Theory and Computation

    Cite this: J. Chem. Theory Comput. 2021, 17, 7, 3955–3966
    Click to copy citationCitation copied!
    https://doi.org/10.1021/acs.jctc.1c00145
    Published June 1, 2021
    Copyright © 2021 American Chemical Society
    Request reuse permissions

    Abstract

    Click to copy section linkSection link copied!
    Abstract Image

    We report a new multi-GPU capable ab initio Hartree–Fock/density functional theory implementation integrated into the open source QUantum Interaction Computational Kernel (QUICK) program. Details on the load balancing algorithms for electron repulsion integrals and exchange correlation quadrature across multiple GPUs are described. Benchmarking studies carried out on up to four GPU nodes, each containing four NVIDIA V100-SXM2 type GPUs demonstrate that our implementation is capable of achieving excellent load balancing and high parallel efficiency. For representative medium to large size protein/organic molecular systems, the observed parallel efficiencies remained above 82% for the Kohn–Sham matrix formation and above 90% for nuclear gradient calculations. The accelerations on NVIDIA A100, P100, and K80 platforms also have realized parallel efficiencies higher than 68% in all tested cases, paving the way for large-scale ab initio electronic structure calculations with QUICK.

    Copyright © 2021 American Chemical Society

    Subjects

    what are subjects

    Read this article

    To access this article, please review the available access options below.

    Get instant access

    Purchase Access

    Read this article for 48 hours. Check out below using your ACS ID or as a guest.

    Recommended

    Access through Your Institution

    You may have access to this article through your institution.

    Your institution does not have access to this content. Add or change your institution or let them know you’d like them to include access.

    Supporting Information

    Click to copy section linkSection link copied!

    The Supporting Information is available free of charge at https://pubs.acs.org/doi/10.1021/acs.jctc.1c00145.

    • Figures and tables with benchmark data for ERI thread walking strategies, XC load balancing strategies, load balancing and MPI communication times, parallel efficiencies, load balancing, and performance on different GPU types (PDF)

    • QUICK input files with Cartesian coordinates of the benchmark molecules (ZIP)

    Terms & Conditions

    Most electronic Supporting Information files are available without a subscription to ACS Web Editions. Such files may be downloaded by article for research use (if there is a public use license linked to the relevant article, that license may permit other uses). Permission may be obtained from ACS for other uses through requests via the RightsLink permission system: http://pubs.acs.org/page/copyright/permissions.html.

    Cited By

    Click to copy section linkSection link copied!
    Citation Statements
    Explore this article's citation statements on scite.ai
    powered by  Scite

    This article is cited by 26 publications.

    1. Saadiallakh Normatov, Pavel V. Nesterov, Timur A. Aliev, Alexandra A. Timralieva, Alexander S. Novikov, Ekaterina V. Skorb. Search for Correlations Between the Results of the Density Functional Theory and Hartree–Fock Calculations Using Neural Networks and Classical Machine Learning Algorithms. ACS Omega 2025, 10 (6) , 5919-5933. https://doi.org/10.1021/acsomega.4c09861
    2. Fusong Ju, Xinran Wei, Lin Huang, Andrew J. Jenkins, Leo Xia, Jia Zhang, Jianwei Zhu, Han Yang, Bin Shao, Peggy Dai, David B. Williams-Young, Ashwin Mayya, Zahra Hooshmand, Alexandra Efimovskaya, Nathan A. Baker, Matthias Troyer, Hongbin Liu. Acceleration without Disruption: DFT Software as a Service. Journal of Chemical Theory and Computation 2024, 20 (24) , 10838-10851. https://doi.org/10.1021/acs.jctc.4c00940
    3. Elise Palethorpe, Ryan Stocks, Giuseppe M. J. Barca. Advanced Techniques for High-Performance Fock Matrix Construction on GPU Clusters. Journal of Chemical Theory and Computation 2024, 20 (23) , 10424-10442. https://doi.org/10.1021/acs.jctc.4c00994
    4. Inkoo Kim, Daun Jeong, Leah P. Weisburn, Alexandra Alexiu, Troy Van Voorhis, Young Min Rhee, Won-Joon Son, Hyung-Jin Kim, Jinkyu Yim, Sungmin Kim, Yeonchoo Cho, Inkook Jang, Seungmin Lee, Dae Sin Kim. Very-Large-Scale GPU-Accelerated Nuclear Gradient of Time-Dependent Density Functional Theory with Tamm–Dancoff Approximation and Range-Separated Hybrid Functionals. Journal of Chemical Theory and Computation 2024, 20 (20) , 9018-9031. https://doi.org/10.1021/acs.jctc.4c01003
    5. Ryan Stocks, Elise Palethorpe, Giuseppe M. J. Barca. Multi-GPU RI-HF Energies and Analytic Gradients─Toward High-Throughput Ab Initio Molecular Dynamics. Journal of Chemical Theory and Computation 2024, 20 (17) , 7503-7515. https://doi.org/10.1021/acs.jctc.4c00877
    6. Akhil Shajan, Madushanka Manathunga, Andreas W. Götz, Kenneth M. Merz, Jr.. Geometry Optimization: A Comparison of Different Open-Source Geometry Optimizers. Journal of Chemical Theory and Computation 2023, 19 (21) , 7533-7541. https://doi.org/10.1021/acs.jctc.3c00188
    7. Federico Zahariev, Peng Xu, Bryce M. Westheimer, Simon Webb, Jorge Galvez Vallejo, Ananta Tiwari, Vaibhav Sundriyal, Masha Sosonkina, Jun Shen, George Schoendorff, Megan Schlinsog, Tosaporn Sattasathuchana, Klaus Ruedenberg, Luke B. Roskop, Alistair P. Rendell, David Poole, Piotr Piecuch, Buu Q. Pham, Vladimir Mironov, Joani Mato, Sam Leonard, Sarom S. Leang, Joe Ivanic, Jackson Hayes, Taylor Harville, Karthik Gururangan, Emilie Guidez, Igor S. Gerasimov, Christian Friedl, Katherine N. Ferreras, George Elliott, Dipayan Datta, Daniel Del Angel Cruz, Laura Carrington, Colleen Bertoni, Giuseppe M. J. Barca, Melisa Alkan, Mark S. Gordon. The General Atomic and Molecular Electronic Structure System (GAMESS): Novel Methods on Novel Architectures. Journal of Chemical Theory and Computation 2023, 19 (20) , 7031-7055. https://doi.org/10.1021/acs.jctc.3c00379
    8. David A. Case, Hasan Metin Aktulga, Kellon Belfon, David S. Cerutti, G. Andrés Cisneros, Vinícius Wilian D. Cruzeiro, Negin Forouzesh, Timothy J. Giese, Andreas W. Götz, Holger Gohlke, Saeed Izadi, Koushik Kasavajhala, Mehmet C. Kaymak, Edward King, Tom Kurtzman, Tai-Sung Lee, Pengfei Li, Jian Liu, Tyler Luchko, Ray Luo, Madushanka Manathunga, Matias R. Machado, Hai Minh Nguyen, Kurt A. O’Hearn, Alexey V. Onufriev, Feng Pan, Sergio Pantano, Ruxi Qi, Ali Rahnamoun, Ali Risheh, Stephan Schott-Verdugo, Akhil Shajan, Jason Swails, Junmei Wang, Haixin Wei, Xiongwu Wu, Yongxian Wu, Shi Zhang, Shiji Zhao, Qiang Zhu, Thomas E. Cheatham, III, Daniel R. Roe, Adrian Roitberg, Carlos Simmerling, Darrin M. York, Maria C. Nagan, Kenneth M. Merz, Jr.. AmberTools. Journal of Chemical Information and Modeling 2023, 63 (20) , 6183-6191. https://doi.org/10.1021/acs.jcim.3c01153
    9. Jeheon Woo, Seonghwan Kim, Woo Youn Kim. Dynamic Precision Approach for Accelerating Large-Scale Eigenvalue Solvers in Electronic Structure Calculations on Graphics Processing Units. Journal of Chemical Theory and Computation 2023, 19 (5) , 1457-1465. https://doi.org/10.1021/acs.jctc.2c00983
    10. Madushanka Manathunga, Hasan Metin Aktulga, Andreas W. Götz, Kenneth M. Merz, Jr.. Quantum Mechanics/Molecular Mechanics Simulations on NVIDIA and AMD Graphics Processing Units. Journal of Chemical Information and Modeling 2023, 63 (3) , 711-717. https://doi.org/10.1021/acs.jcim.2c01505
    11. K. Grace Johnson, Seema Mirchandaney, Ellis Hoag, Alan Heirich, Alex Aiken, Todd J. Martínez. Multinode Multi-GPU Two-Electron Integrals: Code Generation Using the Regent Language. Journal of Chemical Theory and Computation 2022, 18 (11) , 6522-6536. https://doi.org/10.1021/acs.jctc.2c00414
    12. Christopher Seidl, Giuseppe M. J. Barca. Q-Next: A Fast, Parallel, and Diagonalization-Free Alternative to Direct Inversion of the Iterative Subspace. Journal of Chemical Theory and Computation 2022, 18 (7) , 4164-4176. https://doi.org/10.1021/acs.jctc.2c00073
    13. Giuseppe M. J. Barca, Melisa Alkan, Jorge L. Galvez-Vallejo, David L. Poole, Alistair P. Rendell, Mark S. Gordon. Faster Self-Consistent Field (SCF) Calculations on GPU Clusters. Journal of Chemical Theory and Computation 2021, 17 (12) , 7486-7503. https://doi.org/10.1021/acs.jctc.1c00720
    14. Satoki Tsuji, Yasuaki Ito, Koji Nakano, Akihiko Kasagi. GPU Acceleration of the Boys Function Evaluation in Computational Quantum Chemistry. Concurrency and Computation: Practice and Experience 2025, 37 (2) https://doi.org/10.1002/cpe.8328
    15. Satoki Tsuji, Yasuaki Ito, Haruto Fujii, Nobuya Yokogawa, Kanta Suzuki, Koji Nakano, Akihiko Kasagi. Dynamic Screening of Two-Electron Repulsion Integrals in GPU Parallelization. 2024, 211-217. https://doi.org/10.1109/CANDARW64572.2024.00041
    16. Runfeng Jin, Wenhao Liang, Haoyuan Zhang, Yinxuan Song, Zhen Luo, Haibo Ma, Yingjin Ma, Zhong Jin. PASCI : A Scalable Framework for Heterogeneous Parallel Calculation of Dynamical Electron Correlation. 2024, 1103-1113. https://doi.org/10.1145/3673038.3673098
    17. Andrey Asadchev, Edward F. Valeev. 3-center and 4-center 2-particle Gaussian AO integrals on modern accelerated processors. The Journal of Chemical Physics 2024, 160 (24) https://doi.org/10.1063/5.0217001
    18. Yujun Tao, Timothy J. Giese, Şölen Ekesan, Jinzhe Zeng, Bálint Aradi, Ben Hourahine, Hasan Metin Aktulga, Andreas W. Götz, Kenneth M. Merz, Darrin M. York. Amber free energy tools: Interoperable software for free energy simulations using generalized quantum mechanical/molecular mechanical and machine learning potentials. The Journal of Chemical Physics 2024, 160 (22) https://doi.org/10.1063/5.0211276
    19. Colton B. Hicks, Todd J. Martinez. Massively scalable workflows for quantum chemistry: BigChem and ChemCloud. The Journal of Chemical Physics 2024, 160 (14) https://doi.org/10.1063/5.0190834
    20. Inkoo Kim, Daun Jeong, Won-Joon Son, Hyung-Jin Kim, Young Min Rhee, Yongsik Jung, Hyeonho Choi, Jinkyu Yim, Inkook Jang, Dae Sin Kim. Kohn–Sham time-dependent density functional theory with Tamm–Dancoff approximation on massively parallel GPUs. npj Computational Materials 2023, 9 (1) https://doi.org/10.1038/s41524-023-01041-4
    21. Ji Qi, Yingfeng Zhang, Minghui Yang. A hybrid CPU/GPU method for Hartree–Fock self-consistent-field calculation. The Journal of Chemical Physics 2023, 159 (10) https://doi.org/10.1063/5.0156934
    22. Jorge L. Galvez Vallejo, Calum Snowdon, Ryan Stocks, Fazeleh Kazemian, Fiona Chuo Yan Yu, Christopher Seidl, Zoe Seeger, Melisa Alkan, David Poole, Bryce M. Westheimer, Mehaboob Basha, Marco De La Pierre, Alistair Rendell, Ekaterina I. Izgorodina, Mark S. Gordon, Giuseppe M. J. Barca. Toward an extreme-scale electronic structure system. The Journal of Chemical Physics 2023, 159 (4) https://doi.org/10.1063/5.0156399
    23. David B. Williams-Young, Andrey Asadchev, Doru Thom Popovici, David Clark, Jonathan Waldrop, Theresa L. Windus, Edward F. Valeev, Wibe A. de Jong. Distributed memory, GPU accelerated Fock construction for hybrid, Gaussian basis density functional theory. The Journal of Chemical Physics 2023, 158 (23) https://doi.org/10.1063/5.0151070
    24. Jorge Luis Galvez Vallejo, Giuseppe M.J. Barca, Mark S. Gordon. High-performance GPU-accelerated evaluation of electron repulsion integrals. Molecular Physics 2023, 121 (9-10) https://doi.org/10.1080/00268976.2022.2112987
    25. Madushanka Manathunga, Andreas W. Götz, Kenneth M. Merz. Computer-aided drug design, quantum-mechanical methods for biological problems. Current Opinion in Structural Biology 2022, 75 , 102417. https://doi.org/10.1016/j.sbi.2022.102417
    26. David B. Williams-Young, Abhishek Bagusetty, Wibe A. de Jong, Douglas Doerfler, Hubertus J.J. van Dam, Álvaro Vázquez-Mayagoitia, Theresa L. Windus, Chao Yang. Achieving performance portability in Gaussian basis set density functional theory on accelerator based architectures in NWChemEx. Parallel Computing 2021, 108 , 102829. https://doi.org/10.1016/j.parco.2021.102829
    Download PDF
    Go to Journal of Chemical Theory and Computation
    Get e-Alerts
    Get e-Alerts

    Journal of Chemical Theory and Computation

    Cite this: J. Chem. Theory Comput. 2021, 17, 7, 3955–3966
    Click to copy citationCitation copied!
    https://doi.org/10.1021/acs.jctc.1c00145
    Published June 1, 2021
    Copyright © 2021 American Chemical Society
    Request reuse permissions

    Article Views

    1138

    Altmetric

    -

    Citations

    26
    Learn about these metrics

    Article Views are the COUNTER-compliant sum of full text article downloads since November 2008 (both PDF and HTML) across all institutions and individuals. These metrics are regularly updated to reflect usage leading up to the last few days.

    Citations are the number of other articles citing this article, calculated by Crossref and updated daily. Find more information about Crossref citation counts.

    The Altmetric Attention Score is a quantitative measure of the attention that a research article has received online. Clicking on the donut icon will load a page at altmetric.com with additional details about the score and the social media presence for the given article. Find more information on the Altmetric Attention Score and how the score is calculated.