{
  "cells": [
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "%matplotlib inline"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "\n# Writing to hdf5 using the Microdata objects\n\n\n\n\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "# Code source: Chris Smith -- cq6@ornl.gov\n# Liscense: MIT\n\nimport os\nimport numpy as np\nimport pycroscopy as px"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Create some MicroDatasets and MicroDataGroups that will be written to the file.\nWith h5py, groups and datasets must be created from the top down,\nbut the Microdata objects allow us to build them in any order and link them later.\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "# First create some data\ndata1 = np.random.rand(5, 7)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Now use the array to build the dataset.  This dataset will live\ndirectly under the root of the file.  The MicroDataset class also implements the\ncompression and chunking parameters from h5py.Dataset.\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "ds_main = px.MicroDataset('Main_Data', data=data1, parent='/')"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "We can also create an empty dataset and write the values in later\nWith this method, it is neccessary to specify the dtype and maxshape kwarg parameters.\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "ds_empty = px.MicroDataset('Empty_Data', data=[], dtype=np.float32, maxshape=[7, 5, 3])"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "We can also create groups and add other MicroData objects as children.\nIf the group's parent is not given, it will be set to root.\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "data_group = px.MicroDataGroup('Data_Group', parent='/')\n\nroot_group = px.MicroDataGroup('/')\n\n# After creating the group, we then add an existing object as its child.\ndata_group.addChildren([ds_empty])\nroot_group.addChildren([ds_main, data_group])"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "The showTree method allows us to view the data structure before the hdf5 file is\ncreated.\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "root_group.showTree()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Now that we have created the objects, we can write them to an hdf5 file\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "# First we specify the path to the file\nh5_path = 'microdata_test.h5'\n\n# Then we use the ioHDF5 class to build the file from our objects.\nhdf = px.ioHDF5(h5_path)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "The writeData method builds the hdf5 file using the structure defined by the\nMicroData objects.  It returns a list of references to all h5py objects in the\nnew file.\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "h5_refs = hdf.writeData(root_group, print_log=True)\n\n# We can use these references to get the h5py dataset and group objects\nh5_main = px.io.hdf_utils.getH5DsetRefs(['Main_Data'], h5_refs)[0]\nh5_empty = px.io.hdf_utils.getH5DsetRefs(['Empty_Data'], h5_refs)[0]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Compare the data in our dataset to the original\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "print(np.allclose(h5_main[()], data1))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "As mentioned above, we can now write to the Empty_Data object\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "data2 = np.random.rand(*h5_empty.shape)\nh5_empty[:] = data2[:]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Now that we are using h5py objects, we must use flush to write the data to file\nafter it has been altered.\nWe need the file object to do this.  It can be accessed as an attribute of the\nhdf object.\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "h5_file = hdf.file\nh5_file.flush()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Now that we are done, we should close the file so that it can be accessed elsewhere.\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "h5_file.close()\nos.remove(h5_path)"
      ]
    }
  ],
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.6.3"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}