DEKOIS: Demanding Evaluation Kits for Objective in Silico Screening — A Versatile Tool for Benchmarking Docking Programs and Scoring FunctionsClick to copy article linkArticle link copied!
Abstract

For widely applied in silico screening techniques success depends on the rational selection of an appropriate method. We herein present a fast, versatile, and robust method to construct demanding evaluation kits for objective in silico screening (DEKOIS). This automated process enables creating tailor-made decoy sets for any given sets of bioactives. It facilitates a target-dependent validation of docking algorithms and scoring functions helping to save time and resources. We have developed metrics for assessing and improving decoy set quality and employ them to investigate how decoy embedding affects docking. We demonstrate that screening performance is target-dependent and can be impaired by latent actives in the decoy set (LADS) or enhanced by poor decoy embedding. The presented method allows extending and complementing the collection of publicly available high quality decoy sets toward new target space. All present and future DEKOIS data sets will be made accessible at www.dekois.com.
Cited By
Smart citations by scite.ai include citation statements extracted from the full text of the citing article. The number of the statements may be higher than the number of citations provided by ACS Publications if one paper cites another multiple times or lower if scite has not yet processed some of the citing articles.
This article is cited by 69 publications.
- Christian Kramer, John Chodera, Kelly L. Damm-Ganamet, Michael K. Gilson, Judith Günther, Uta Lessel, Richard A. Lewis, David Mobley, Eva Nittinger, Adam Pecina, Matthieu Schapira, W. Patrick Walters. The Need for Continuing Blinded Pose- and Activity Prediction Benchmarks. Journal of Chemical Information and Modeling 2025, 65
(5)
, 2180-2190. https://doi.org/10.1021/acs.jcim.4c02296
- Monica A. Kamal, Hedy A. Badary, Dalia Omran, Hend I. Shousha, Ashraf O. Abdelaziz, Hend M. El Tayebi, Yasmine M. Mandour. Virtual Screening and Biological Evaluation of Potential PD-1/PD-L1 Immune Checkpoint Inhibitors as Anti-Hepatocellular Carcinoma Agents. ACS Omega 2023, 8
(37)
, 33242-33254. https://doi.org/10.1021/acsomega.3c00279
- Chao Shen, Xujun Zhang, Yafeng Deng, Junbo Gao, Dong Wang, Lei Xu, Peichen Pan, Tingjun Hou, Yu Kang. Boosting Protein–Ligand Binding Pose Prediction and Virtual Screening Based on Residue–Atom Distance Likelihood Potential and Graph Transformer. Journal of Medicinal Chemistry 2022, 65
(15)
, 10691-10706. https://doi.org/10.1021/acs.jmedchem.2c00991
- Xujun Zhang, Chao Shen, Ben Liao, Dejun Jiang, Jike Wang, Zhenxing Wu, Hongyan Du, Tianyue Wang, Wenbo Huo, Lei Xu, Dongsheng Cao, Chang-Yu Hsieh, Tingjun Hou. TocoDecoy: A New Approach to Design Unbiased Datasets for Training and Benchmarking Machine-Learning Scoring Functions. Journal of Medicinal Chemistry 2022, 65
(11)
, 7918-7932. https://doi.org/10.1021/acs.jmedchem.2c00460
- Viet-Khoa Tran-Nguyen, Célien Jacquemard, Didier Rognan. LIT-PCBA: An Unbiased Data Set for Machine Learning and Virtual Screening. Journal of Chemical Information and Modeling 2020, 60
(9)
, 4263-4273. https://doi.org/10.1021/acs.jcim.0c00155
- Adrian Stecula, Muhammad S. Hussain, Ronald E. Viola. Discovery of Novel Inhibitors of a Critical Brain Enzyme Using a Homology Model and a Deep Convolutional Neural Network. Journal of Medicinal Chemistry 2020, 63
(16)
, 8867-8875. https://doi.org/10.1021/acs.jmedchem.0c00473
- Jochen Sieg, Florian Flachsenberg, Matthias Rarey. In Need of Bias Control: Evaluating Chemical Data for Machine Learning in Structure-Based Virtual Screening. Journal of Chemical Information and Modeling 2019, 59
(3)
, 947-961. https://doi.org/10.1021/acs.jcim.8b00712
- Jie Xia, Terry-Elinor Reid, Song Wu, Liangren Zhang, Xiang Simon Wang. Maximal Unbiased Benchmarking Data Sets for Human Chemokine Receptors and Comparative Analysis. Journal of Chemical Information and Modeling 2018, 58
(5)
, 1104-1120. https://doi.org/10.1021/acs.jcim.8b00004
- David Xu and Samy O. Meroueh . Effect of Binding Pose and Modeled Structures on SVMGen and GlideScore Enrichment of Chemical Libraries. Journal of Chemical Information and Modeling 2016, 56
(6)
, 1139-1151. https://doi.org/10.1021/acs.jcim.5b00709
- Heather A. Carlson . Lessons Learned over Four Benchmark Exercises from the Community Structure–Activity Resource. Journal of Chemical Information and Modeling 2016, 56
(6)
, 951-954. https://doi.org/10.1021/acs.jcim.6b00182
- Alexander Sebastian Hauser and Björn Windshügel . LEADS-PEP: A Benchmark Data Set for Assessment of Peptide Docking Performance. Journal of Chemical Information and Modeling 2016, 56
(1)
, 188-200. https://doi.org/10.1021/acs.jcim.5b00234
- Tamer M. Ibrahim, Matthias R. Bauer, Alexander Dörr, Erdem Veyisoglu, and Frank M. Boeckler . pROC-Chemotype Plots Enhance the Interpretability of Benchmarking Results in Structure-Based Virtual Screening. Journal of Chemical Information and Modeling 2015, 55
(11)
, 2297-2307. https://doi.org/10.1021/acs.jcim.5b00475
- Nathalie Lagarde, Jean-François Zagury, and Matthieu Montes . Benchmarking Data Sets for the Evaluation of Virtual Ligand Screening Methods: Review and Perspectives. Journal of Chemical Information and Modeling 2015, 55
(7)
, 1297-1307. https://doi.org/10.1021/acs.jcim.5b00090
- Sabine Schultes, Albert J. Kooistra, Henry F. Vischer, Saskia Nijmeijer, Eric E. J. Haaksma, Rob Leurs, Iwan J. P. de Esch, and Chris de Graaf . Combinatorial Consensus Scoring for Ligand-Based Virtual Fragment Screening: A Comparative Case Study for Serotonin 5-HT3A, Histamine H1, and Histamine H4 Receptors. Journal of Chemical Information and Modeling 2015, 55
(5)
, 1030-1044. https://doi.org/10.1021/ci500694c
- Martin Lindh, Fredrik Svensson, Wesley Schaal, Jin Zhang, Christian Sköld, Peter Brandt, and Anders Karlén . Toward a Benchmarking Data Set Able to Evaluate Ligand- and Structure-based Virtual Screening Using Public HTS Data. Journal of Chemical Information and Modeling 2015, 55
(2)
, 343-353. https://doi.org/10.1021/ci5005465
- Jie Xia, Ermias Lemma Tilahun, Eyob Hailu Kebede, Terry-Elinor Reid, Liangren Zhang, and Xiang Simon Wang . Comparative Modeling and Benchmarking Data Sets for Human Histone Deacetylases and Sirtuin Families. Journal of Chemical Information and Modeling 2015, 55
(2)
, 374-388. https://doi.org/10.1021/ci5005515
- Jie Xia, Hongwei Jin, Zhenming Liu, Liangren Zhang, and Xiang Simon Wang . An Unbiased Method To Build Benchmarking Sets for Ligand-Based Virtual Screening and its Application To GPCRs. Journal of Chemical Information and Modeling 2014, 54
(5)
, 1433-1450. https://doi.org/10.1021/ci500062f
- Matthias R. Bauer, Tamer M. Ibrahim, Simon M. Vogel, and Frank M. Boeckler . Evaluation and Optimization of Virtual Screening Workflows with DEKOIS 2.0 – A Public Library of Challenging Docking Benchmark Sets. Journal of Chemical Information and Modeling 2013, 53
(6)
, 1447-1462. https://doi.org/10.1021/ci400115b
- Michael M. Mysinger, Michael Carchia, John. J. Irwin, and Brian K. Shoichet . Directory of Useful Decoys, Enhanced (DUD-E): Better Ligands and Decoys for Better Benchmarking. Journal of Medicinal Chemistry 2012, 55
(14)
, 6582-6594. https://doi.org/10.1021/jm300687e
- Rainer Wilcken, Xiangrui Liu, Markus O. Zimmermann, Trevor J. Rutherford, Alan R. Fersht, Andreas C. Joerger, and Frank M. Boeckler . Halogen-Enriched Fragment Libraries as Leads for Drug Rescue of Mutant p53. Journal of the American Chemical Society 2012, 134
(15)
, 6810-6818. https://doi.org/10.1021/ja301056a
- Sophia M. N. Hönig, Torben Gutermuth, Christiane Ehrt, Christian Lemmen, Matthias Rarey. Combining crystallographic and binding affinity data towards a novel dataset of small molecule overlays. Journal of Computer-Aided Molecular Design 2025, 39
(1)
https://doi.org/10.1007/s10822-024-00581-1
- Lucas N. Alberca, Denis N. Prada Gori, Maximiliano J. Fallico, Alexandre V. Fassio, Alan Talevi, Carolina L. Bellera. LIDEB's Useful Decoys (LUDe): A freely available decoy-generation tool. Benchmarking and scope. Artificial Intelligence in the Life Sciences 2025, 7 , 100129. https://doi.org/10.1016/j.ailsci.2025.100129
- Noha Galal, Botros Y. Beshay, Omar Soliman, Muhammad I. Ismail, Mohamed Abdelfadil, Mohamed El-Hadidi, Reem K. Arafa, Tamer M. Ibrahim, . Evaluating the structure-based virtual screening performance of SARS-CoV-2 main protease: A benchmarking approach and a multistage screening example against the wild-type and Omicron variants. PLOS ONE 2025, 20
(2)
, e0318712. https://doi.org/10.1371/journal.pone.0318712
- Samir Bondock, Nada Alabbad, Aisha Hossan, Ibrahim A. Shaaban, Ali A. Shati, Mohammad Y. Alfaifi, SeragE.I. Elbehairi, Rehab H. Abd El-Aleam, Moaz M. Abdou. Novel nano-sized N-Thiazolylpyridylamines targeting CDK2: Design, divergent synthesis, conformational studies, and multifaceted In silico analysis. Chemico-Biological Interactions 2025, 407 , 111366. https://doi.org/10.1016/j.cbi.2024.111366
- Yara A. Zaky, Mai W. Rashad, Marwa A. Zaater, Ahmed M. El Kerdawy. Discovery of dual rho-associated protein kinase 1 (ROCK1)/apoptosis signal–regulating kinase 1 (ASK1) inhibitors as a novel approach for non-alcoholic steatohepatitis (NASH) treatment. BMC Chemistry 2024, 18
(1)
https://doi.org/10.1186/s13065-023-01081-3
- Lin Wang, Shihang Wang, Hao Yang, Shiwei Li, Xinyu Wang, Yongqi Zhou, Siyuan Tian, Lu Liu, Fang Bai. Conformational Space Profiling Enhances Generic Molecular Representation for AI‐Powered Ligand‐Based Drug Discovery. Advanced Science 2024, 11
(40)
https://doi.org/10.1002/advs.202403998
- Neeraj Kumar, Vishal Acharya. Advances in machine intelligence‐driven virtual screening approaches for big‐data. Medicinal Research Reviews 2024, 44
(3)
, 939-974. https://doi.org/10.1002/med.21995
- Tobias Harren, Torben Gutermuth, Christoph Grebner, Gerhard Hessler, Matthias Rarey. Modern
machine‐learning
for binding affinity estimation of
protein–ligand
complexes: Progress, opportunities, and challenges. WIREs Computational Molecular Science 2024, 14
(3)
https://doi.org/10.1002/wcms.1716
- Fernando D. Prieto-Martínez, Jennifer Mendoza-Cañas, Karina Martínez-Mayorga. To Bind or Not to Bind? A Comprehensive Characterization of TIR1 and Auxins Using Consensus In Silico Approaches. Computation 2024, 12
(5)
, 94. https://doi.org/10.3390/computation12050094
- Nikolai Schapin, Maciej Majewski, Alejandro Varela-Rial, Carlos Arroniz, Gianni De Fabritiis. Machine learning small molecule properties in drug discovery. Artificial Intelligence Chemistry 2023, 1
(2)
, 100020. https://doi.org/10.1016/j.aichem.2023.100020
- Viet-Khoa Tran-Nguyen, Muhammad Junaid, Saw Simeon, Pedro J. Ballester. A practical guide to machine-learning scoring for structure-based virtual screening. Nature Protocols 2023, 18
(11)
, 3460-3511. https://doi.org/10.1038/s41596-023-00885-w
- Xujun Zhang, Chao Shen, Tianyue Wang, Yafeng Deng, Yu Kang, Dan Li, Tingjun Hou, Peichen Pan. ML-PLIC: a web platform for characterizing protein–ligand interactions and developing machine learning-based scoring functions. Briefings in Bioinformatics 2023, 24
(5)
https://doi.org/10.1093/bib/bbad295
- Philippe Pinel, Gwenn Guichaoua, Matthieu Najm, Stéphanie Labouille, Nicolas Drizard, Yann Gaston‐Mathé, Brice Hoffmann, Véronique Stoven. Exploring isofunctional molecules: Design of a benchmark and evaluation of prediction performance. Molecular Informatics 2023, 42
(4)
https://doi.org/10.1002/minf.202200216
- Heba H. A. Hassan, Muhammad I. Ismail, Mohammed A. S. Abourehab, Frank M. Boeckler, Tamer M. Ibrahim, Reem K. Arafa. In Silico Targeting of Fascin Protein for Cancer Therapy: Benchmarking, Virtual Screening and Molecular Dynamics Approaches. Molecules 2023, 28
(3)
, 1296. https://doi.org/10.3390/molecules28031296
- Clara Blanes-Mira, Pilar Fernández-Aguado, Jorge de Andrés-López, Asia Fernández-Carvajal, Antonio Ferrer-Montiel, Gregorio Fernández-Ballester. Comprehensive Survey of Consensus Docking for High-Throughput Virtual Screening. Molecules 2023, 28
(1)
, 175. https://doi.org/10.3390/molecules28010175
- Denis N. Prada Gori, Lucas N. Alberca, Santiago Rodriguez, Juan I. Alice, Manuel A. Llanos, Carolina L. Bellera, Alan Talevi. LIDeB Tools: A Latin American resource of freely available, open-source cheminformatics apps. Artificial Intelligence in the Life Sciences 2022, 2 , 100049. https://doi.org/10.1016/j.ailsci.2022.100049
- Mingna Li, Jianxing Hu, Yanxing Wang, Yibo Li, Liangren Zhang, Zhenming Liu. Challenging Reverse Screening: A Benchmark Study for Comprehensive Evaluation. Molecular Informatics 2022, 41
(4)
https://doi.org/10.1002/minf.202100063
- Juan I. Di Filippo, Claudio N. Cavasotto. Guided structure-based ligand identification and design via artificial intelligence modeling. Expert Opinion on Drug Discovery 2022, 17
(1)
, 71-78. https://doi.org/10.1080/17460441.2021.1979514
- Andrea Basciu, Lara Callea, Stefano Motta, Alexandre M.J.J. Bonvin, Laura Bonati, Attilio V. Vargiu. No dance, no partner! A tale of receptor flexibility in docking and virtual screening. 2022, 43-97. https://doi.org/10.1016/bs.armc.2022.08.006
- Fergus Imrie, Anthony R Bradley, Charlotte M Deane, . Generating property-matched decoy molecules using deep learning. Bioinformatics 2021, 37
(15)
, 2134-2141. https://doi.org/10.1093/bioinformatics/btab080
- Laila K. Elghoneimy, Muhammad I. Ismail, Frank M. Boeckler, Hassan M.E. Azzazy, Tamer M. Ibrahim. Facilitating SARS CoV-2 RNA-Dependent RNA polymerase (RdRp) drug discovery by the aid of HCV NS5B palm subdomain binders: In silico approaches and benchmarking. Computers in Biology and Medicine 2021, 134 , 104468. https://doi.org/10.1016/j.compbiomed.2021.104468
- Claudio N. Cavasotto, Juan I. Di Filippo. Artificial intelligence in the early stages of drug discovery. Archives of Biochemistry and Biophysics 2021, 698 , 108730. https://doi.org/10.1016/j.abb.2020.108730
- Vivek Srivastava, Chandrabose Selvaraj, Sanjeev Kumar Singh. Chemoinformatics and QSAR. 2021, 183-212. https://doi.org/10.1007/978-981-33-6191-1_10
- Bruno Junior Neves, Melina Mottin, José Teofilo Moreira-Filho, Bruna Katiele de Paula Sousa, Sabrina Silva Mendonca, Carolina Horta Andrade. Best Practices for Docking-Based Virtual Screening. 2021, 75-98. https://doi.org/10.1016/B978-0-12-822312-3.00001-1
- Tamer M. Ibrahim, Muhammad I. Ismail, Matthias R. Bauer, Adnan A. Bekhit, Frank M. Boeckler. Supporting SARS-CoV-2 Papain-Like Protease Drug Discovery: In silico Methods and Benchmarking. Frontiers in Chemistry 2020, 8 https://doi.org/10.3389/fchem.2020.592289
- Yusuf O. Adeshina, Eric J. Deeds, John Karanicolas. Machine learning classification can reduce false positives in structure-based virtual screening. Proceedings of the National Academy of Sciences 2020, 117
(31)
, 18477-18488. https://doi.org/10.1073/pnas.2000585117
- Divya Bafna, Fuqiang Ban, Paul S. Rennie, Kriti Singh, Artem Cherkasov. Computer-Aided Ligand Discovery for Estrogen Receptor Alpha. International Journal of Molecular Sciences 2020, 21
(12)
, 4193. https://doi.org/10.3390/ijms21124193
- Viet-Khoa Tran-Nguyen, Didier Rognan. Benchmarking Data Sets from PubChem BioAssay Data: Current Scenario and Room for Improvement. International Journal of Molecular Sciences 2020, 21
(12)
, 4380. https://doi.org/10.3390/ijms21124380
- Jie Xia, Shan Li, Yu Ding, Song Wu, Xiang Simon Wang. MUBD‐DecoyMaker 2.0: A Python GUI Application to Generate Maximal Unbiased Benchmarking Data Sets for Virtual Drug Screening. Molecular Informatics 2020, 39
(4)
https://doi.org/10.1002/minf.201900151
- Saltuk M. Eyrilmez, Cemal Köprülüoğlu, Jan Řezáč, Pavel Hobza. Impressive Enrichment of Semiempirical Quantum Mechanics‐Based Scoring Function: HSP90 Protein with 4541 Inhibitors and Decoys. ChemPhysChem 2019, 20
(21)
, 2759-2766. https://doi.org/10.1002/cphc.201900628
- Pedro H. M. Torres, Ana C. R. Sodero, Paula Jofily, Floriano P. Silva-Jr. Key Topics in Molecular Docking for Drug Design. International Journal of Molecular Sciences 2019, 20
(18)
, 4574. https://doi.org/10.3390/ijms20184574
- Chang Wen, Xin Yan, Qiong Gu, Jiewen Du, Di Wu, Yutong Lu, Huihao Zhou, Jun Xu. Systematic Studies on the Protocol and Criteria for Selecting a Covalent Docking Tool. Molecules 2019, 24
(11)
, 2183. https://doi.org/10.3390/molecules24112183
- Isabella A. Guedes, Felipe S. S. Pereira, Laurent E. Dardenne. Empirical Scoring Functions for Structure-Based Virtual Screening: Applications, Critical Aspects, and Challenges. Frontiers in Pharmacology 2018, 9 https://doi.org/10.3389/fphar.2018.01089
- Manon Réau, Florent Langenfeld, Jean-François Zagury, Nathalie Lagarde, Matthieu Montes. Decoys Selection in Benchmarking Datasets: Overview and Perspectives. Frontiers in Pharmacology 2018, 9 https://doi.org/10.3389/fphar.2018.00011
- Luminita Crisan, Sorin Avram, Liliana Pacureanu. Pharmacophore-based screening and drug repurposing exemplified on glycogen synthase kinase-3 inhibitors. Molecular Diversity 2017, 21
(2)
, 385-405. https://doi.org/10.1007/s11030-016-9724-5
- Sakari Lätti, Sanna Niinivehmas, Olli T. Pentikäinen. Rocker: Open source, easy-to-use tool for AUC and enrichment calculations and ROC visualization. Journal of Cheminformatics 2016, 8
(1)
https://doi.org/10.1186/s13321-016-0158-y
- Sunghwan Kim. Getting the most out of PubChem for virtual screening. Expert Opinion on Drug Discovery 2016, 11
(9)
, 843-855. https://doi.org/10.1080/17460441.2016.1216967
- Zhe Wang, Huiyong Sun, Xiaojun Yao, Dan Li, Lei Xu, Youyong Li, Sheng Tian, Tingjun Hou. Comprehensive evaluation of ten docking programs on a diverse set of protein–ligand complexes: the prediction accuracy of sampling power and scoring power. Physical Chemistry Chemical Physics 2016, 18
(18)
, 12964-12975. https://doi.org/10.1039/C6CP01555G
- Tamer M Ibrahim, Matthias R Bauer, Frank M Boeckler. Applying DEKOIS 2.0 in structure-based virtual screening to probe the impact of preparation procedures and score normalization. Journal of Cheminformatics 2015, 7
(1)
https://doi.org/10.1186/s13321-015-0074-6
- Fen Pei, Hongwei Jin, Xin Zhou, Jie Xia, Lidan Sun, Zhenming Liu, Liangren Zhang. Enrichment Assessment of Multiple Virtual Screening Strategies for Toll‐Like Receptor 8 Agonists Based on a Maximal Unbiased Benchmarking Data Set. Chemical Biology & Drug Design 2015, 86
(5)
, 1226-1241. https://doi.org/10.1111/cbdd.12590
- Jie Xia, Ermias Lemma Tilahun, Terry-Elinor Reid, Liangren Zhang, Xiang Simon Wang. Benchmarking methods and data sets for ligand enrichment assessment in virtual screening. Methods 2015, 71 , 146-157. https://doi.org/10.1016/j.ymeth.2014.11.015
- Francois Berenger, Arnout Voet, Xiao Yin Lee, Kam YJ Zhang. A rotation-translation invariant molecular descriptor of partial charges and its use in ligand-based virtual screening. Journal of Cheminformatics 2014, 6
(1)
https://doi.org/10.1186/1758-2946-6-23
- Frank M Boeckler, Matthias R Bauer, Tamer M Ibrahim, Simon M Vogel. Use of DEKOIS 2.0 to gain insights for virtual screening. Journal of Cheminformatics 2014, 6
(S1)
https://doi.org/10.1186/1758-2946-6-S1-O24
- Tamer M Ibrahim, Matthias R Bauer, Frank M Boeckler. Probing the impact of protein and ligand preparation procedures on chemotype enrichment in structure-based virtual screening using DEKOIS 2.0 benchmark sets. Journal of Cheminformatics 2014, 6
(S1)
https://doi.org/10.1186/1758-2946-6-S1-P19
- Elizabeth Yuriev. Challenges and Advances in Structure-Based Virtual Screening. Future Medicinal Chemistry 2014, 6
(1)
, 5-7. https://doi.org/10.4155/fmc.13.186
- Elizabeth Yuriev, Paul A. Ramsland. Latest developments in molecular docking: 2010–2011 in review. Journal of Molecular Recognition 2013, 26
(5)
, 215-239. https://doi.org/10.1002/jmr.2266
- Marius Mihăşan. What in silico molecular docking can do for the ‘bench-working biologists’. Journal of Biosciences 2012, 37
(S1)
, 1089-1095. https://doi.org/10.1007/s12038-012-9273-8
- Simon M. Vogel, Matthias R. Bauer, Andreas C. Joerger, Rainer Wilcken, Tobias Brandt, Dmitry B. Veprintsev, Trevor J. Rutherford, Alan R. Fersht, Frank M. Boeckler. Lithocholic acid is an endogenous inhibitor of MDM4 and MDM2. Proceedings of the National Academy of Sciences 2012, 109
(42)
, 16906-16910. https://doi.org/10.1073/pnas.1215060109
- Preethi Badrinarayan, G. Narahari Sastry. Virtual screening filters for the design of type II p38 MAP kinase inhibitors: A fragment based library generation approach. Journal of Molecular Graphics and Modelling 2012, 34 , 89-100. https://doi.org/10.1016/j.jmgm.2011.12.009
Article Views are the COUNTER-compliant sum of full text article downloads since November 2008 (both PDF and HTML) across all institutions and individuals. These metrics are regularly updated to reflect usage leading up to the last few days.
Citations are the number of other articles citing this article, calculated by Crossref and updated daily. Find more information about Crossref citation counts.
The Altmetric Attention Score is a quantitative measure of the attention that a research article has received online. Clicking on the donut icon will load a page at altmetric.com with additional details about the score and the social media presence for the given article. Find more information on the Altmetric Attention Score and how the score is calculated.