ACS Publications. Most Trusted. Most Cited. Most Read
Knowledge Extraction from Atomically Resolved Images

OR SEARCH CITATIONS

My Activity
Publications

Figure 1Loading Img
Download Hi-Res ImageDownload to MS-PowerPointCite This:ACS Nano 2017, 11, 10, 10313-10320
    Article

    Knowledge Extraction from Atomically Resolved Images
    Click to copy article linkArticle link copied!

    View Author Information
    † Chemical Sciences Division, Oak Ridge National Laboratory, Oak Ridge Tennessee 37831, United States
    ‡ Joint Institute for Computational Sciences, University of Tennessee, Oak Ridge National Laboratory, Oak Ridge Tennessee 37831, United States
    § ∥ §The Institute for Functional Imaging of Materials and ∥The Center for Nanophase Materials Sciences, Oak Ridge National Laboratory, Oak Ridge, Tennessee 37831, United States
    ⊥ Bredesen Center for Interdisciplinary Research, University of Tennessee, Knoxville, Tennessee 37996, United States
    # School of Physics, Huazhong University of Science & Technology, Wuhan 430074, China
    *E-mail: [email protected]
    Other Access OptionsSupporting Information (1)

    ACS Nano

    Cite this: ACS Nano 2017, 11, 10, 10313–10320
    Click to copy citationCitation copied!
    https://doi.org/10.1021/acsnano.7b05036
    Published September 27, 2017
    Copyright © 2017 American Chemical Society
    Request reuse permissions

    Abstract

    Click to copy section linkSection link copied!
    Abstract Image

    Tremendous strides in experimental capabilities of scanning transmission electron microscopy and scanning tunneling microscopy (STM) over the past 30 years made atomically resolved imaging routine. However, consistent integration and use of atomically resolved data with generative models is unavailable, so information on local thermodynamics and other microscopic driving forces encoded in the observed atomic configurations remains hidden. Here, we present a framework based on statistical distance minimization to consistently utilize the information available from atomic configurations obtained from an atomically resolved image and extract meaningful physical interaction parameters. We illustrate the applicability of the framework on an STM image of a FeSexTe1–x superconductor, with the segregation of the chalcogen atoms investigated using a nonideal interacting solid solution model. This universal method makes full use of the microscopic degrees of freedom sampled in an atomically resolved image and can be extended via Bayesian inference toward unbiased model selection with uncertainty quantification.

    Copyright © 2017 American Chemical Society

    Subjects

    what are subjects

    Keywords

    what are keywords

    Read this article

    To access this article, please review the available access options below.

    Get instant access

    Purchase Access

    Read this article for 48 hours. Check out below using your ACS ID or as a guest.

    Recommended

    Access through Your Institution

    You may have access to this article through your institution.

    Your institution does not have access to this content. Add or change your institution or let them know you’d like them to include access.

    Supporting Information

    Click to copy section linkSection link copied!

    The Supporting Information is available free of charge on the ACS Publications website at DOI: 10.1021/acsnano.7b05036.

    • More details and results from the simulations, and details of image analysis (PDF)

    Terms & Conditions

    Most electronic Supporting Information files are available without a subscription to ACS Web Editions. Such files may be downloaded by article for research use (if there is a public use license linked to the relevant article, that license may permit other uses). Permission may be obtained from ACS for other uses through requests via the RightsLink permission system: http://pubs.acs.org/page/copyright/permissions.html.

    Cited By

    Click to copy section linkSection link copied!

    This article is cited by 33 publications.

    1. Li Shao, Jinrong Ma, Jesse L. Prelesnik, Yicheng Zhou, Mary Nguyen, Mingfei Zhao, Samson A. Jenekhe, Sergei V. Kalinin, Andrew L. Ferguson, Jim Pfaendtner, Christopher J. Mundy, James J. De Yoreo, François Baneyx, Chun-Long Chen. Hierarchical Materials from High Information Content Macromolecular Building Blocks: Construction, Dynamic Interventions, and Prediction. Chemical Reviews 2022, 122 (24) , 17397-17478. https://doi.org/10.1021/acs.chemrev.2c00220
    2. Sai Mani Prudhvi Valleti, Qiang Zou, Rui Xue, Lukas Vlcek, Maxim Ziatdinov, Rama Vasudevan, Mingming Fu, Jiaqiang Yan, David Mandrus, Zheng Gai, Sergei V. Kalinin. Bayesian Learning of Adatom Interactions from Atomically Resolved Imaging Data. ACS Nano 2021, 15 (6) , 9649-9657. https://doi.org/10.1021/acsnano.0c10851
    3. Karim T. Mukaddem, Edward J. Beard, Batuhan Yildirim, Jacqueline M. Cole. ImageDataExtractor: A Tool To Extract and Quantify Data from Microscopy Images. Journal of Chemical Information and Modeling 2020, 60 (5) , 2492-2509. https://doi.org/10.1021/acs.jcim.9b00734
    4. Sergei V. Kalinin, Ondrej Dyck, Nina Balke, Sabine Neumayer, Wan-Yu Tsai, Rama Vasudevan, David Lingerfelt, Mahshid Ahmadi, Maxim Ziatdinov, Matthew T. McDowell, Evgheni Strelcov. Toward Electrochemical Studies on the Nanometer and Atomic Scales: Progress, Challenges, and Opportunities. ACS Nano 2019, 13 (9) , 9735-9780. https://doi.org/10.1021/acsnano.9b02687
    5. Lukas Vlcek, Maxim Ziatdinov, Artem Maksov, Alexander Tselev, Arthur P. Baddorf, Sergei V. Kalinin, Rama K. Vasudevan. Learning from Imperfections: Predicting Structure and Thermodynamics from Atomic Imaging of Fluctuations. ACS Nano 2019, 13 (1) , 718-727. https://doi.org/10.1021/acsnano.8b07980
    6. Andrew J. Medford, M. Ross Kunz, Sarah M. Ewing, Tammie Borders, Rebecca Fushimi. Extracting Knowledge from Data through Catalysis Informatics. ACS Catalysis 2018, 8 (8) , 7403-7429. https://doi.org/10.1021/acscatal.8b01708
    7. Maxim A. Ziatdinov, Muammer Yusuf Yaman, Yongtao Liu, David Ginger, Sergei V. Kalinin. Semi-supervised learning of images with strong rotational disorder: assembling nanoparticle libraries. Digital Discovery 2024, 3 (6) , 1213-1220. https://doi.org/10.1039/D3DD00196B
    8. Ganesh Narasimha, Saban Hus, Arpan Biswas, Rama Vasudevan, Maxim Ziatdinov. Autonomous convergence of STM control parameters using Bayesian optimization. APL Machine Learning 2024, 2 (1) https://doi.org/10.1063/5.0185362
    9. K. Barakati, Hui Yuan, Amit Goyal, S. V. Kalinin. Physics-based reward driven image analysis in microscopy. Digital Discovery 2024, 20 https://doi.org/10.1039/D4DD00132J
    10. Yunchao Xie, Kianoosh Sattari, Chi Zhang, Jian Lin. Toward autonomous laboratories: Convergence of artificial intelligence and experimental automation. Progress in Materials Science 2023, 132 , 101043. https://doi.org/10.1016/j.pmatsci.2022.101043
    11. Jiang-Yang Shao, Dongmei Li, Jiangjian Shi, Chuang Ma, Yousheng Wang, Xiaomin Liu, Xianyuan Jiang, Mengmeng Hao, Luozheng Zhang, Chang Liu, Yiting Jiang, Zhenhan Wang, Yu-Wu Zhong, Shengzhong Frank Liu, Yaohua Mai, Yongsheng Liu, Yixin Zhao, Zhijun Ning, Lianzhou Wang, Baomin Xu, Lei Meng, Zuqiang Bian, Ziyi Ge, Xiaowei Zhan, Jingbi You, Yongfang Li, Qingbo Meng. Recent progress in perovskite solar cells: material science. Science China Chemistry 2023, 66 (1) , 10-64. https://doi.org/10.1007/s11426-022-1445-2
    12. Maxim Ziatdinov, Ayana Ghosh, Chun Yin Wong, Sergei V. Kalinin. AtomAI framework for deep learning analysis of image and spectroscopy data in electron and scanning probe microscopy. Nature Machine Intelligence 2022, 4 (12) , 1101-1112. https://doi.org/10.1038/s42256-022-00555-8
    13. Sergei V. Kalinin, Colin Ophus, Paul M. Voyles, Rolf Erni, Demie Kepaptsoglou, Vincenzo Grillo, Andrew R. Lupini, Mark P. Oxley, Eric Schwenker, Maria K. Y. Chan, Joanne Etheridge, Xiang Li, Grace G. D. Han, Maxim Ziatdinov, Naoya Shibata, Stephen J. Pennycook. Machine learning in scanning transmission electron microscopy. Nature Reviews Methods Primers 2022, 2 (1) https://doi.org/10.1038/s43586-022-00095-w
    14. Sergei V. Kalinin, Ayana Ghosh, Rama Vasudevan, Maxim Ziatdinov. From atomically resolved imaging to generative and causal models. Nature Physics 2022, 18 (10) , 1152-1160. https://doi.org/10.1038/s41567-022-01666-0
    15. Petro Maksymovych, Jiaqiang Yan, Brian Sales, Jun Wang. Atomic-scale analysis of disorder by similarity learning from tunneling spectroscopy. Physical Review Research 2022, 4 (3) https://doi.org/10.1103/PhysRevResearch.4.033058
    16. Maxim A. Ziatdinov, Yongtao Liu, Anna N. Morozovska, Eugene A. Eliseev, Xiaohang Zhang, Ichiro Takeuchi, Sergei V. Kalinin. Hypothesis Learning in Automated Experiment: Application to Combinatorial Materials Libraries. Advanced Materials 2022, 34 (20) https://doi.org/10.1002/adma.202201345
    17. Christina Doty, Shaun Gallagher, Wenqi Cui, Wenya Chen, Shweta Bhushan, Marjolein Oostrom, Sarah Akers, Steven R. Spurgeon. Design of a graphical user interface for few-shot machine learning classification of electron microscopy data. Computational Materials Science 2022, 203 , 111121. https://doi.org/10.1016/j.commatsci.2021.111121
    18. Suhas Somnath, Rama K. Vasudevan, Stephen Jesse, Sergei Kalinin, Nageswara Rao, Christopher Brumgard, Feiyi Wang, Olga Kuchar, Arjun Shankar, Ben Mintz, Elke Arenholz, J. Robert Michael, Sarp Oral. Building an Integrated Ecosystem of Computational and Observational Facilities to Accelerate Scientific Discovery. 2022, 58-75. https://doi.org/10.1007/978-3-030-96498-6_4
    19. Rama K. Vasudevan, Maxim Ziatdinov, Lukas Vlcek, Sergei V. Kalinin. Off-the-shelf deep learning is not enough, and requires parsimony, Bayesianity, and causality. npj Computational Materials 2021, 7 (1) https://doi.org/10.1038/s41524-020-00487-0
    20. Lukas Vlcek, Shize Yang, Yongji Gong, Pulickel Ajayan, Wu Zhou, Matthew F. Chisholm, Maxim Ziatdinov, Rama K. Vasudevan, Sergei V. Kalinin. Thermodynamics of order and randomness in dopant distributions inferred from atomically resolved imaging. npj Computational Materials 2021, 7 (1) https://doi.org/10.1038/s41524-021-00507-7
    21. Sergei V. Kalinin, Mark P. Oxley, Mani Valleti, Junjie Zhang, Raphael P. Hermann, Hong Zheng, Wenrui Zhang, Gyula Eres, Rama K. Vasudevan, Maxim Ziatdinov. Deep Bayesian local crystallography. npj Computational Materials 2021, 7 (1) https://doi.org/10.1038/s41524-021-00621-6
    22. Sarah Akers, Elizabeth Kautz, Andrea Trevino-Gavito, Matthew Olszta, Bethany E. Matthews, Le Wang, Yingge Du, Steven R. Spurgeon. Rapid and flexible segmentation of electron microscopy data using few-shot machine learning. npj Computational Materials 2021, 7 (1) https://doi.org/10.1038/s41524-021-00652-z
    23. Mingren Shen, Guanzhao Li, Dongxia Wu, Yuhan Liu, Jacob R.C. Greaves, Wei Hao, Nathaniel J. Krakauer, Leah Krudy, Jacob Perez, Varun Sreenivasan, Bryan Sanchez, Oigimer Torres-Velázquez, Wei Li, Kevin G. Field, Dane Morgan. Multi defect detection and analysis of electron microscopy images with deep learning. Computational Materials Science 2021, 199 , 110576. https://doi.org/10.1016/j.commatsci.2021.110576
    24. Maxim Ziatdinov, Nicole Creange, Xiaohang Zhang, Anna Morozovska, Eugene Eliseev, Rama K. Vasudevan, Ichiro Takeuchi, Chris Nelson, Sergei V. Kalinin. Predictability as a probe of manifest and latent physics: The case of atomic scale structural, chemical, and polarization behaviors in multiferroic Sm-doped BiFeO3. Applied Physics Reviews 2021, 8 (1) https://doi.org/10.1063/5.0016792
    25. Mani Valleti, L. Vlcek, Maxim Ziatdinov, Rama K. Vasudevan, Sergei V. Kalinin. Reconstruction and uncertainty quantification of lattice Hamiltonian model parameters from observations of microscopic degrees of freedom. Journal of Applied Physics 2020, 128 (21) https://doi.org/10.1063/5.0024387
    26. Maxim Ziatdinov, Christopher T. Nelson, Xiaohang Zhang, Rama K. Vasudevan, Eugene Eliseev, Anna N. Morozovska, Ichiro Takeuchi, Sergei V. Kalinin. Causal analysis of competing atomistic mechanisms in ferroelectric materials from high-resolution scanning transmission electron microscopy data. npj Computational Materials 2020, 6 (1) https://doi.org/10.1038/s41524-020-00396-2
    27. Elsa A. Olivetti, Jacqueline M. Cole, Edward Kim, Olga Kononova, Gerbrand Ceder, Thomas Yong-Jin Han, Anna M. Hiszpanski. Data-driven materials research enabled by natural language processing and information extraction. Applied Physics Reviews 2020, 7 (4) https://doi.org/10.1063/5.0021106
    28. M. Chakraborty, M. Ziatdinov, O. Dyck, S. Jesse, A. D. White, Sergei V. Kalinin. Reconstruction of the interatomic forces from dynamic scanning transmission electron microscopy data. Journal of Applied Physics 2020, 127 (22) https://doi.org/10.1063/5.0009413
    29. Rama K. Vasudevan, Kamal Choudhary, Apurva Mehta, Ryan Smith, Gilad Kusne, Francesca Tavazza, Lukas Vlcek, Maxim Ziatdinov, Sergei V. Kalinin, Jason Hattrick-Simpers. Materials science in the artificial intelligence age: high-throughput library generation, machine learning, and a pathway from correlations to the underpinning physics. MRS Communications 2019, 9 (3) , 821-838. https://doi.org/10.1557/mrc.2019.95
    30. Lukas Vlcek, Shize Yang, Maxim Ziatdinov, Sergei Kalinin, Rama Vasudevan. Statistical Physics-based Framework and Bayesian Inference for Model Selection and Uncertainty Quantification. Microscopy and Microanalysis 2019, 25 (S2) , 130-131. https://doi.org/10.1017/S1431927619001387
    31. Sergei V. Kalinin, Andrew R. Lupini, Ondrej Dyck, Stephen Jesse, Maxim Ziatdinov, Rama K. Vasudevan. Lab on a beam—Big data and artificial intelligence in scanning transmission electron microscopy. MRS Bulletin 2019, 44 (7) , 565-575. https://doi.org/10.1557/mrs.2019.159
    32. J.M. Rickman, T. Lookman, S.V. Kalinin. Materials informatics: From the atomic-level to the continuum. Acta Materialia 2019, 168 , 473-510. https://doi.org/10.1016/j.actamat.2019.01.051
    33. Jason R. Hattrick-Simpers, Kamal Choudhary, Claudio Corgnale. A simple constrained machine learning model for predicting high-pressure-hydrogen-compressor materials. Molecular Systems Design & Engineering 2018, 3 (3) , 509-517. https://doi.org/10.1039/C8ME00005K
    Download PDF
    Go to ACS Nano
    Get e-Alerts
    Get e-Alerts

    ACS Nano

    Cite this: ACS Nano 2017, 11, 10, 10313–10320
    Click to copy citationCitation copied!
    https://doi.org/10.1021/acsnano.7b05036
    Published September 27, 2017
    Copyright © 2017 American Chemical Society
    Request reuse permissions

    Article Views

    1239

    Altmetric

    -

    Citations

    33
    Learn about these metrics

    Article Views are the COUNTER-compliant sum of full text article downloads since November 2008 (both PDF and HTML) across all institutions and individuals. These metrics are regularly updated to reflect usage leading up to the last few days.

    Citations are the number of other articles citing this article, calculated by Crossref and updated daily. Find more information about Crossref citation counts.

    The Altmetric Attention Score is a quantitative measure of the attention that a research article has received online. Clicking on the donut icon will load a page at altmetric.com with additional details about the score and the social media presence for the given article. Find more information on the Altmetric Attention Score and how the score is calculated.