ACS Publications. Most Trusted. Most Cited. Most Read
My Activity

Knowledge Extraction from Atomically Resolved Images

View Author Information
Chemical Sciences Division, Oak Ridge National Laboratory, Oak Ridge Tennessee 37831, United States
Joint Institute for Computational Sciences, University of Tennessee, Oak Ridge National Laboratory, Oak Ridge Tennessee 37831, United States
§ ∥ §The Institute for Functional Imaging of Materials and The Center for Nanophase Materials Sciences, Oak Ridge National Laboratory, Oak Ridge, Tennessee 37831, United States
Bredesen Center for Interdisciplinary Research, University of Tennessee, Knoxville, Tennessee 37996, United States
# School of Physics, Huazhong University of Science & Technology, Wuhan 430074, China
Cite this: ACS Nano 2017, 11, 10, 10313–10320
Publication Date (Web):September 27, 2017
Copyright © 2017 American Chemical Society

    Article Views





    Other access options
    Supporting Info (1)»


    Abstract Image

    Tremendous strides in experimental capabilities of scanning transmission electron microscopy and scanning tunneling microscopy (STM) over the past 30 years made atomically resolved imaging routine. However, consistent integration and use of atomically resolved data with generative models is unavailable, so information on local thermodynamics and other microscopic driving forces encoded in the observed atomic configurations remains hidden. Here, we present a framework based on statistical distance minimization to consistently utilize the information available from atomic configurations obtained from an atomically resolved image and extract meaningful physical interaction parameters. We illustrate the applicability of the framework on an STM image of a FeSexTe1–x superconductor, with the segregation of the chalcogen atoms investigated using a nonideal interacting solid solution model. This universal method makes full use of the microscopic degrees of freedom sampled in an atomically resolved image and can be extended via Bayesian inference toward unbiased model selection with uncertainty quantification.

    Read this article

    To access this article, please review the available access options below.

    Get instant access

    Purchase Access

    Read this article for 48 hours. Check out below using your ACS ID or as a guest.


    Access through Your Institution

    You may have access to this article through your institution.

    Your institution does not have access to this content. You can change your affiliated institution below.

    Supporting Information

    Jump To

    The Supporting Information is available free of charge on the ACS Publications website at DOI: 10.1021/acsnano.7b05036.

    • More details and results from the simulations, and details of image analysis (PDF)

    Terms & Conditions

    Most electronic Supporting Information files are available without a subscription to ACS Web Editions. Such files may be downloaded by article for research use (if there is a public use license linked to the relevant article, that license may permit other uses). Permission may be obtained from ACS for other uses through requests via the RightsLink permission system:

    Cited By

    This article is cited by 31 publications.

    1. Li Shao, Jinrong Ma, Jesse L. Prelesnik, Yicheng Zhou, Mary Nguyen, Mingfei Zhao, Samson A. Jenekhe, Sergei V. Kalinin, Andrew L. Ferguson, Jim Pfaendtner, Christopher J. Mundy, James J. De Yoreo, François Baneyx, Chun-Long Chen. Hierarchical Materials from High Information Content Macromolecular Building Blocks: Construction, Dynamic Interventions, and Prediction. Chemical Reviews 2022, 122 (24) , 17397-17478.
    2. Sai Mani Prudhvi Valleti, Qiang Zou, Rui Xue, Lukas Vlcek, Maxim Ziatdinov, Rama Vasudevan, Mingming Fu, Jiaqiang Yan, David Mandrus, Zheng Gai, Sergei V. Kalinin. Bayesian Learning of Adatom Interactions from Atomically Resolved Imaging Data. ACS Nano 2021, 15 (6) , 9649-9657.
    3. Karim T. Mukaddem, Edward J. Beard, Batuhan Yildirim, Jacqueline M. Cole. ImageDataExtractor: A Tool To Extract and Quantify Data from Microscopy Images. Journal of Chemical Information and Modeling 2020, 60 (5) , 2492-2509.
    4. Sergei V. Kalinin, Ondrej Dyck, Nina Balke, Sabine Neumayer, Wan-Yu Tsai, Rama Vasudevan, David Lingerfelt, Mahshid Ahmadi, Maxim Ziatdinov, Matthew T. McDowell, Evgheni Strelcov. Toward Electrochemical Studies on the Nanometer and Atomic Scales: Progress, Challenges, and Opportunities. ACS Nano 2019, 13 (9) , 9735-9780.
    5. Lukas Vlcek, Maxim Ziatdinov, Artem Maksov, Alexander Tselev, Arthur P. Baddorf, Sergei V. Kalinin, Rama K. Vasudevan. Learning from Imperfections: Predicting Structure and Thermodynamics from Atomic Imaging of Fluctuations. ACS Nano 2019, 13 (1) , 718-727.
    6. Andrew J. Medford, M. Ross Kunz, Sarah M. Ewing, Tammie Borders, Rebecca Fushimi. Extracting Knowledge from Data through Catalysis Informatics. ACS Catalysis 2018, 8 (8) , 7403-7429.
    7. Ganesh Narasimha, Saban Hus, Arpan Biswas, Rama Vasudevan, Maxim Ziatdinov. Autonomous convergence of STM control parameters using Bayesian optimization. APL Machine Learning 2024, 2 (1)
    8. Yunchao Xie, Kianoosh Sattari, Chi Zhang, Jian Lin. Toward autonomous laboratories: Convergence of artificial intelligence and experimental automation. Progress in Materials Science 2023, 132 , 101043.
    9. Jiang-Yang Shao, Dongmei Li, Jiangjian Shi, Chuang Ma, Yousheng Wang, Xiaomin Liu, Xianyuan Jiang, Mengmeng Hao, Luozheng Zhang, Chang Liu, Yiting Jiang, Zhenhan Wang, Yu-Wu Zhong, Shengzhong Frank Liu, Yaohua Mai, Yongsheng Liu, Yixin Zhao, Zhijun Ning, Lianzhou Wang, Baomin Xu, Lei Meng, Zuqiang Bian, Ziyi Ge, Xiaowei Zhan, Jingbi You, Yongfang Li, Qingbo Meng. Recent progress in perovskite solar cells: material science. Science China Chemistry 2023, 66 (1) , 10-64.
    10. Maxim Ziatdinov, Ayana Ghosh, Chun Yin Wong, Sergei V. Kalinin. AtomAI framework for deep learning analysis of image and spectroscopy data in electron and scanning probe microscopy. Nature Machine Intelligence 2022, 4 (12) , 1101-1112.
    11. Sergei V. Kalinin, Colin Ophus, Paul M. Voyles, Rolf Erni, Demie Kepaptsoglou, Vincenzo Grillo, Andrew R. Lupini, Mark P. Oxley, Eric Schwenker, Maria K. Y. Chan, Joanne Etheridge, Xiang Li, Grace G. D. Han, Maxim Ziatdinov, Naoya Shibata, Stephen J. Pennycook. Machine learning in scanning transmission electron microscopy. Nature Reviews Methods Primers 2022, 2 (1)
    12. Sergei V. Kalinin, Ayana Ghosh, Rama Vasudevan, Maxim Ziatdinov. From atomically resolved imaging to generative and causal models. Nature Physics 2022, 18 (10) , 1152-1160.
    13. Petro Maksymovych, Jiaqiang Yan, Brian Sales, Jun Wang. Atomic-scale analysis of disorder by similarity learning from tunneling spectroscopy. Physical Review Research 2022, 4 (3)
    14. Maxim A. Ziatdinov, Yongtao Liu, Anna N. Morozovska, Eugene A. Eliseev, Xiaohang Zhang, Ichiro Takeuchi, Sergei V. Kalinin. Hypothesis Learning in Automated Experiment: Application to Combinatorial Materials Libraries. Advanced Materials 2022, 34 (20)
    15. Christina Doty, Shaun Gallagher, Wenqi Cui, Wenya Chen, Shweta Bhushan, Marjolein Oostrom, Sarah Akers, Steven R. Spurgeon. Design of a graphical user interface for few-shot machine learning classification of electron microscopy data. Computational Materials Science 2022, 203 , 111121.
    16. Suhas Somnath, Rama K. Vasudevan, Stephen Jesse, Sergei Kalinin, Nageswara Rao, Christopher Brumgard, Feiyi Wang, Olga Kuchar, Arjun Shankar, Ben Mintz, Elke Arenholz, J. Robert Michael, Sarp Oral. Building an Integrated Ecosystem of Computational and Observational Facilities to Accelerate Scientific Discovery. 2022, 58-75.
    17. Rama K. Vasudevan, Maxim Ziatdinov, Lukas Vlcek, Sergei V. Kalinin. Off-the-shelf deep learning is not enough, and requires parsimony, Bayesianity, and causality. npj Computational Materials 2021, 7 (1)
    18. Lukas Vlcek, Shize Yang, Yongji Gong, Pulickel Ajayan, Wu Zhou, Matthew F. Chisholm, Maxim Ziatdinov, Rama K. Vasudevan, Sergei V. Kalinin. Thermodynamics of order and randomness in dopant distributions inferred from atomically resolved imaging. npj Computational Materials 2021, 7 (1)
    19. Sergei V. Kalinin, Mark P. Oxley, Mani Valleti, Junjie Zhang, Raphael P. Hermann, Hong Zheng, Wenrui Zhang, Gyula Eres, Rama K. Vasudevan, Maxim Ziatdinov. Deep Bayesian local crystallography. npj Computational Materials 2021, 7 (1)
    20. Sarah Akers, Elizabeth Kautz, Andrea Trevino-Gavito, Matthew Olszta, Bethany E. Matthews, Le Wang, Yingge Du, Steven R. Spurgeon. Rapid and flexible segmentation of electron microscopy data using few-shot machine learning. npj Computational Materials 2021, 7 (1)
    21. Mingren Shen, Guanzhao Li, Dongxia Wu, Yuhan Liu, Jacob R.C. Greaves, Wei Hao, Nathaniel J. Krakauer, Leah Krudy, Jacob Perez, Varun Sreenivasan, Bryan Sanchez, Oigimer Torres-Velázquez, Wei Li, Kevin G. Field, Dane Morgan. Multi defect detection and analysis of electron microscopy images with deep learning. Computational Materials Science 2021, 199 , 110576.
    22. Maxim Ziatdinov, Nicole Creange, Xiaohang Zhang, Anna Morozovska, Eugene Eliseev, Rama K. Vasudevan, Ichiro Takeuchi, Chris Nelson, Sergei V. Kalinin. Predictability as a probe of manifest and latent physics: The case of atomic scale structural, chemical, and polarization behaviors in multiferroic Sm-doped BiFeO3. Applied Physics Reviews 2021, 8 (1)
    23. Mani Valleti, L. Vlcek, Maxim Ziatdinov, Rama K. Vasudevan, Sergei V. Kalinin. Reconstruction and uncertainty quantification of lattice Hamiltonian model parameters from observations of microscopic degrees of freedom. Journal of Applied Physics 2020, 128 (21)
    24. Maxim Ziatdinov, Christopher T. Nelson, Xiaohang Zhang, Rama K. Vasudevan, Eugene Eliseev, Anna N. Morozovska, Ichiro Takeuchi, Sergei V. Kalinin. Causal analysis of competing atomistic mechanisms in ferroelectric materials from high-resolution scanning transmission electron microscopy data. npj Computational Materials 2020, 6 (1)
    25. Elsa A. Olivetti, Jacqueline M. Cole, Edward Kim, Olga Kononova, Gerbrand Ceder, Thomas Yong-Jin Han, Anna M. Hiszpanski. Data-driven materials research enabled by natural language processing and information extraction. Applied Physics Reviews 2020, 7 (4)
    26. M. Chakraborty, M. Ziatdinov, O. Dyck, S. Jesse, A. D. White, Sergei V. Kalinin. Reconstruction of the interatomic forces from dynamic scanning transmission electron microscopy data. Journal of Applied Physics 2020, 127 (22)
    27. Rama K. Vasudevan, Kamal Choudhary, Apurva Mehta, Ryan Smith, Gilad Kusne, Francesca Tavazza, Lukas Vlcek, Maxim Ziatdinov, Sergei V. Kalinin, Jason Hattrick-Simpers. Materials science in the artificial intelligence age: high-throughput library generation, machine learning, and a pathway from correlations to the underpinning physics. MRS Communications 2019, 9 (3) , 821-838.
    28. Lukas Vlcek, Shize Yang, Maxim Ziatdinov, Sergei Kalinin, Rama Vasudevan. Statistical Physics-based Framework and Bayesian Inference for Model Selection and Uncertainty Quantification. Microscopy and Microanalysis 2019, 25 (S2) , 130-131.
    29. Sergei V. Kalinin, Andrew R. Lupini, Ondrej Dyck, Stephen Jesse, Maxim Ziatdinov, Rama K. Vasudevan. Lab on a beam—Big data and artificial intelligence in scanning transmission electron microscopy. MRS Bulletin 2019, 44 (7) , 565-575.
    30. J.M. Rickman, T. Lookman, S.V. Kalinin. Materials informatics: From the atomic-level to the continuum. Acta Materialia 2019, 168 , 473-510.
    31. Jason R. Hattrick-Simpers, Kamal Choudhary, Claudio Corgnale. A simple constrained machine learning model for predicting high-pressure-hydrogen-compressor materials. Molecular Systems Design & Engineering 2018, 3 (3) , 509-517.

    Pair your accounts.

    Export articles to Mendeley

    Get article recommendations from ACS based on references in your Mendeley library.

    Pair your accounts.

    Export articles to Mendeley

    Get article recommendations from ACS based on references in your Mendeley library.

    You’ve supercharged your research process with ACS and Mendeley!

    STEP 1:
    Click to create an ACS ID

    Please note: If you switch to a different device, you may be asked to login again with only your ACS ID.

    Please note: If you switch to a different device, you may be asked to login again with only your ACS ID.

    Please note: If you switch to a different device, you may be asked to login again with only your ACS ID.

    Your Mendeley pairing has expired. Please reconnect