Commit 1a811433 authored by Unknown's avatar Unknown
Browse files

Notebook update

parent 5b20bd77
%% Cell type:markdown id: tags:
# Analyzing Ptychography data using pycroscopy
### Stephen Jesse, Suhas Somnath, and Chris R. Smith,
The Center for Nanophase Materials Science and The Institute for Functional Imaging for Materials <br>
Oak Ridge National Laboratory<br>
1/19/2017
Here, we will be working with ptychography datasets acquired using a scanning transmission electron microscope (STEM). These ptychography datsets have four dimensions - two (x, y) dimensions from the position of the electron beam and each spatial pixel contains a two dimensional (u, v) image, called a ronchigram, recorded by the detector. Though the ronchigrams are typically averaged to two values (bright field, dark field), retaining the raw ronchigrams enables deeper investigation of data to reveal the existence of different phases in the material and other patterns that would not be visible in the averaged data
%% Cell type:markdown id: tags:
## Configure the notebook first
%% Cell type:code id: tags:
``` python
!pip install -U pycroscopy scipy numpy matplotlib h5py
# Ensure python 3 compatibility
from __future__ import division, print_function, absolute_import
# Import necessary libraries:
import h5py
import numpy as np
import matplotlib.pyplot as plt
import matplotlib.patches as patches
from IPython.display import display
from IPython.display import display, HTML
import ipywidgets as widgets
import pycroscopy as px
# set up notebook to show plots within the notebook
% matplotlib inline
% matplotlib notebook
# Make Notebook take up most of page width
display(HTML(data="""
<style>
div#notebook-container { width: 95%; }
div#menubar-container { width: 65%; }
div#maintoolbar-container { width: 99%; }
</style>
"""))
```
%% Cell type:markdown id: tags:
## Load pycroscopy compatible ptychography dataset
For simplicity we will use a dataset that has already been transalated form its original data format into a pycroscopy compatible hierarchical data format (HDF5 or H5) file
#### H5 files:
* are like smart containers that can store matrices with data, folders to organize these datasets, images, metadata like experimental parameters, links or shortcuts to datasets, etc.
* are readily compatible with high-performance computing facilities
* scale very efficiently from few kilobytes to several terabytes
* can be read and modified using any language including Python, Matlab, C/C++, Java, Fortran, Igor Pro, etc.
%% Cell type:code id: tags:
``` python
# Select a file to work on:
# h5_path = px.io_utils.uiGetFile('*.h5', 'pycroscopy formatted Ptychography dataset')
h5_path = r"\\nanophase\IFgroup\SPM software development\Raw_Data\Ptychography\20120212_21_GB.h5"
h5_path = "/home/challtdow/workspace/pycroscopy_data/Raw_Data/20120212_21_GB/20120212_21_GB.h5"
print('Working on:\n' + h5_path)
# Open the file
h5_file = h5py.File(h5_path, mode='r+')
```
%% Output
Working on:
\\nanophase\IFgroup\SPM software development\Raw_Data\Ptychography\20120212_21_GB.h5
%% Cell type:markdown id: tags:
## Inspect the contents of this h5 data file
The file contents are stored in a tree structure, just like files on a contemporary computer.
The data is stored as a 2D matrix (position, spectroscopic value) regardless of the dimensionality of the data.
In the case of these 4D ptychography datasets, the data is stored as a N x P dataset where N is the number of spatial positions of the electron beam and P is the number of pixels in the detector.
The main dataset is always accompanied by four ancillary datasets that explain the position and spectroscopic value of any given element in the dataset.
In the case of the 2d images, the positions will be arranged as row0-col0, row0-col1.... row0-colN, row1-col0....
The spectroscopic information is trivial since the data at any given pixel is just a scalar value
%% Cell type:code id: tags:
``` python
print('Datasets and datagroups within the file:\n------------------------------------')
px.hdf_utils.print_tree(h5_file)
print('\nThe main dataset:\n------------------------------------')
print(h5_file['/Measurement_000/Channel_000/Raw_Data'])
print('\nThe ancillary datasets:\n------------------------------------')
print(h5_file['/Measurement_000/Channel_000/Position_Indices'])
print(h5_file['/Measurement_000/Channel_000/Position_Values'])
print(h5_file['/Measurement_000/Channel_000/Spectroscopic_Indices'])
print(h5_file['/Measurement_000/Channel_000/Spectroscopic_Values'])
print('\nMetadata or attributes in a datagroup\n------------------------------------')
for key in h5_file['/Measurement_000'].attrs:
print('{} : {}'.format(key, h5_file['/Measurement_000'].attrs[key]))
```
%% Output
Datasets and datagroups within the file:
------------------------------------
/
Measurement_000
Measurement_000/Channel_000
Measurement_000/Channel_000/Mean_Ronchigram
Measurement_000/Channel_000/Position_Indices
Measurement_000/Channel_000/Position_Values
Measurement_000/Channel_000/Raw_Data
Measurement_000/Channel_000/Raw_Data-Cluster_000
Measurement_000/Channel_000/Raw_Data-Cluster_000/Cluster_Indices
Measurement_000/Channel_000/Raw_Data-Cluster_000/Cluster_Values
Measurement_000/Channel_000/Raw_Data-Cluster_000/Label_Spectroscopic_Indices
Measurement_000/Channel_000/Raw_Data-Cluster_000/Label_Spectroscopic_Values
Measurement_000/Channel_000/Raw_Data-Cluster_000/Labels
Measurement_000/Channel_000/Raw_Data-Cluster_000/Mean_Response
Measurement_000/Channel_000/Raw_Data-SVD_000
Measurement_000/Channel_000/Raw_Data-SVD_000/Component_Indices
Measurement_000/Channel_000/Raw_Data-SVD_000/S
Measurement_000/Channel_000/Raw_Data-SVD_000/U
Measurement_000/Channel_000/Raw_Data-SVD_000/U-Cluster_000
Measurement_000/Channel_000/Raw_Data-SVD_000/U-Cluster_000/Cluster_Indices
Measurement_000/Channel_000/Raw_Data-SVD_000/U-Cluster_000/Cluster_Values
Measurement_000/Channel_000/Raw_Data-SVD_000/U-Cluster_000/Label_Spectroscopic_Indices
Measurement_000/Channel_000/Raw_Data-SVD_000/U-Cluster_000/Label_Spectroscopic_Values
Measurement_000/Channel_000/Raw_Data-SVD_000/U-Cluster_000/Labels
Measurement_000/Channel_000/Raw_Data-SVD_000/U-Cluster_000/Mean_Response
Measurement_000/Channel_000/Raw_Data-SVD_000/V
Measurement_000/Channel_000/Spectroscopic_Indices
Measurement_000/Channel_000/Spectroscopic_Mean
Measurement_000/Channel_000/Spectroscopic_Values
The main dataset:
------------------------------------
<HDF5 dataset "Raw_Data": shape (25921, 9216), type "<f4">
The ancillary datasets:
------------------------------------
<HDF5 dataset "Position_Indices": shape (25921, 2), type "<u4">
<HDF5 dataset "Position_Values": shape (25921, 2), type "<f8">
<HDF5 dataset "Spectroscopic_Indices": shape (2, 9216), type "<u4">
<HDF5 dataset "Spectroscopic_Values": shape (2, 9216), type "<f8">
Metadata or attributes in a datagroup
------------------------------------
image_size_u : 96
image_size_v : 96
num_images : 25921
scan_size_y : 161
scan_size_x : 161
translator : Ptychography
num_pixels : 9216
%% Cell type:markdown id: tags:
## Read some basic parameters for visualization
%% Cell type:code id: tags:
``` python
# Select the dataset containing the raw data to start working with:
h5_main = px.hdf_utils.getDataSet(h5_file, 'Raw_Data')[-1]
# Read some necessary parameters:
h5_pos_inds = px.hdf_utils.getAuxData(h5_main, auxDataName=['Position_Indices'])[0]
num_rows = len(np.unique(h5_pos_inds[:, 0]))
num_cols = len(np.unique(h5_pos_inds[:, 1]))
h5_spec_inds = px.hdf_utils.getAuxData(h5_main, auxDataName=['Spectroscopic_Indices'])[0]
num_sensor_rows = len(np.unique(h5_spec_inds[0, :]))
num_sensor_cols = len(np.unique(h5_spec_inds[1, :]))
```
%% Output
[<HDF5 dataset "Position_Indices": shape (25921, 2), type "<u4">]
[<HDF5 dataset "Spectroscopic_Indices": shape (2, 9216), type "<u4">]
%% Cell type:markdown id: tags:
## Visualize the Raw Ronchigrams
%% Cell type:code id: tags:
``` python
coarse_row = int(0.5*num_rows)
coarse_col = int(0.5*num_cols)
coarse_pos = coarse_row * num_rows + coarse_col
current_ronch = np.reshape(h5_main[coarse_pos], (num_sensor_rows, num_sensor_cols))
fig, axes = plt.subplots(ncols=2, figsize=(14,7))
axes[0].hold(True)
axes[0].set_title('Mean Response')
axes[0].imshow(np.reshape(h5_main.parent['Spectroscopic_Mean'], (num_rows, num_cols)),
cmap=px.plot_utils.cmap_jet_white_center(), origin='lower')
main_map = axes[0].imshow(np.reshape(h5_main.parent['Spectroscopic_Mean'], (num_rows, num_cols)),
cmap=px.plot_utils.cmap_jet_white_center(), origin='lower')
main_vert_line = axes[0].axvline(x=coarse_col, color='k')
main_hor_line = axes[0].axhline(y=coarse_row, color='k')
axes[1].set_title('Ronchigram at current pixel')
img_zoom = axes[1].imshow(current_ronch,cmap=px.plot_utils.cmap_jet_white_center(), origin='lower')
def move_zoom_box(coarse_row, coarse_col):
def move_zoom_box(event):
if not main_map.axes.in_axes(event):
return
coarse_col = int(round(event.xdata))
coarse_row = int(round(event.ydata))
main_vert_line.set_xdata(coarse_col)
main_hor_line.set_ydata(coarse_row)
coarse_pos = coarse_row * num_rows + coarse_col
current_ronch = np.reshape(h5_main[coarse_pos], (num_sensor_rows, num_sensor_cols))
img_zoom.set_data(current_ronch)
#img_zoom.set_clim(vmax=ronch_max, vmin=ronch_min)
display(fig)
fig.canvas.draw()
widgets.interact(move_zoom_box, coarse_row=(0, num_rows, 1),
coarse_col=(0, num_cols, 1));
```
%% Output