Commit ebfef828 authored by Unknown's avatar Unknown
Browse files

Merge remote-tracking branch 'origin/master'

parents 88083e7c 5445ef23
==========
pycroscopy
==========
|statusimage|
.. |statusimage| image:: https://travis-ci.org/pycroscopy/pycroscopy.svg?branch=master
......
......@@ -3,17 +3,17 @@
v 1.0 goals
-----------
1. test utils - 2+ weeks
2. good utilities for interrogating data - pycro data - done
3. good documentation for both users and developers
2. DONE - good utilities for interrogating data - pycro data
3. partially done - good documentation for both users and developers
* Need more on dealing with data and (for developers) explaining what is where and why
4. generic visualizer - mostly complete -
* Need more on dealing with data + plot_utils tour
* (for developers) explaining what is where and why + io utils + hdf utils tour etc.
4. mostly done - generic visualizer
5. settle on a structure for process and analysis - moderate ~ 1 day
* Process should implement some checks.
* Model needs to catch up with Process
6. good utils for generating publishable plots - easy ~ 1 day
7. Promote / demote lesser used utilites to processes / analyses.
6. mostly done - good utils for generating publishable plots - easy ~ 1 day
7. Reorganize package - promote / demote lesser used utilites to processes / analyses.
Short-term goals
--------------------
......@@ -24,13 +24,12 @@ Short-term goals
Documentation
-------------
* Upload clean exports of paper notebooks - Stephen and Chris
* Organize papers by instrument / technique
* Include examples in documentation
* Links to references for all functions and methods used in our workflows.
Fundamental tutorials on how to use pycroscopy
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
* Writing a custom Translator
* A tour of what is where and why
* A tour of the hdf_utils functions used for writing h5 files since these functions need data to show / explain them.
* chunking the main dataset
......@@ -73,11 +72,10 @@ Core development
* All these MUST implement the check for previous computations at the very least
* Absorb functionality from Process into Model
* Bayesian GIV should actually be an analysis <-- depends on above
* Reogranize processing and analysis - promote / demote classes etc.
* multi-node computing capability in parallel_compute
* Image cleaning should be (something like at the very least) a Process
* Bayesian GIV should actually be an analysis
* Demystify analyis / optimize. Use parallel_compute instead of optimize and guess_methods and fit_methods
* Data Generators
* Consistency in the naming of and placement of attributes (chan or meas group) in all translators - Some put attributes in the measurement level, some in the channel level! hyperspy appears to create datagroups solely for the purpose of organizing metadata in a tree structure!
* Consider developing a generic curve fitting class a la `hyperspy <http://nbviewer.jupyter.org/github/hyperspy/hyperspy-demos/blob/master/Fitting_tutorial.ipynb>`_
* Improve visualization of file contents in print_tree() like hyperspy's `metadata <http://hyperspy.org/hyperspy-doc/current/user_guide/metadata_structure.html>`_
......
:orphan:
===================
Pycroscopy Examples
===================
========================
Data Analytics Tutorials
========================
.. raw:: html
......@@ -156,14 +156,6 @@ Developer Tutorials
<div style='clear:both'></div>
=======================
Pycroscopy Publications
=======================
.. raw:: html
<div style='clear:both'></div>
==============
User Tutorials
==============
......
=======================
Pycroscopy Publications
=======================
====================
Papers / Conferences
====================
Journal Papers using pycroscopy
-------------------------------
......
What's New
----------
==========
v 0.59.2
--------
`Dec 7 2017: <https://github.com/pycroscopy/pycroscopy/pull/127>`_
~~~~~~~~
* `Interactive (Jupyter) visualization <https://github.com/pycroscopy/pycroscopy/blob/master/pycroscopy/viz/jupyter_utils.py>`_ for any dataset (< 4 dimensions)
* Handy shortcut to this function added to `Pycrodataset.visualize() <https://github.com/pycroscopy/pycroscopy/blob/master/pycroscopy/io/pycro_data.py>`_
* Significantly more customizable plotting functions - `plot_utils <https://github.com/pycroscopy/pycroscopy/blob/master/pycroscopy/viz/plot_utils.py>`_
* Improved `Process class <https://github.com/pycroscopy/pycroscopy/blob/master/pycroscopy/processing/process.py>`_ that provides the framework for:
* checking for prior instances of a process run on the same dataset with the same parameters
* resuming an aborted process / computation
* Reorganized doSVD() into a new Process called `SVD <https://github.com/pycroscopy/pycroscopy/blob/master/pycroscopy/processing/svd_utils.py>`_ to take advantage of above advancements.
* The same changes will be rolled out to `Cluster <https://github.com/pycroscopy/pycroscopy/blob/master/pycroscopy/processing/cluster.py>`_ and `Decomposition <https://github.com/pycroscopy/pycroscopy/blob/master/pycroscopy/processing/decomposition.py>`_ soon
`Nov 17 2017: <https://github.com/pycroscopy/pycroscopy/pull/126>`_
~~~~~~~~
* Significant improvements and bug fixes to `Bayesian Inference <https://github.com/pycroscopy/pycroscopy/blob/master/pycroscopy/processing/giv_bayesian.py>`_ for `G-mode IV <https://github.com/pycroscopy/pycroscopy/blob/master/pycroscopy/processing/giv_utils.py>`_.
`Nov 11 2017: <https://github.com/pycroscopy/pycroscopy/pull/125>`_
~~~~~~~~
* New robust class for `Bayesian Inference <https://github.com/pycroscopy/pycroscopy/blob/master/pycroscopy/processing/giv_bayesian.py>`_ on G-mode IV data
* Significant bug fixes for `Bayesian Inference <https://github.com/pycroscopy/pycroscopy/blob/master/pycroscopy/processing/giv_utils.py>`_.
* Utilies for reading files from `Nanois controllers <https://github.com/pycroscopy/pycroscopy/tree/master/pycroscopy/io/translators/df_utils/nanonispy>`_
* New robust class for `FFT Signal Filtering <https://github.com/pycroscopy/pycroscopy/blob/master/pycroscopy/processing/signal_filter.py>`_ on any data including G-mode
* FFT filtering rewritten as easy to use `objects <https://github.com/pycroscopy/pycroscopy/blob/master/pycroscopy/processing/fft.py>`_.
`Oct 9 2017: <https://github.com/pycroscopy/pycroscopy/pull/124>`_
~~~~~~~~
* New `PycroDataset class <https://github.com/pycroscopy/pycroscopy/blob/master/pycroscopy/io/pycro_data.py>`_ that simplifies handling, reshaping, and interpretation of **Main** datasets.
`Sep 6 2017: <https://github.com/pycroscopy/pycroscopy/pull/123>`_
~~~~~~~~
* New `Process class <https://github.com/pycroscopy/pycroscopy/blob/master/pycroscopy/processing/process.py>`_ that provides a framework for data processing in Pycroscopy.
`Sep 5 2017: <https://github.com/pycroscopy/pycroscopy/pull/122>`_
~~~~~~~~
* Improved the example on `parallel computing <https://pycroscopy.github.io/pycroscopy/auto_examples/dev_tutorials/plot_tutorial_04_parallel_computing.html#sphx-glr-auto-examples-dev-tutorials-plot-tutorial-04-parallel-computing-py>`_
`Sep 5 2017: <https://github.com/pycroscopy/pycroscopy/pull/121>`_
~~~~~~~~
* Added a new tutorial on `parallel computing <https://pycroscopy.github.io/pycroscopy/auto_examples/dev_tutorials/plot_tutorial_04_parallel_computing.html#sphx-glr-auto-examples-dev-tutorials-plot-tutorial-04-parallel-computing-py>`_
`Aug 31 2017: <https://github.com/pycroscopy/pycroscopy/pull/118>`_
~~~~~~~~
* New plot function - single_img_cbar_plot (now merged into plot_map) for nicr `2D image plots <https://github.com/pycroscopy/pycroscopy/blob/master/pycroscopy/viz/plot_utils.py>`_ with colobars.
`Aug 29 2017: <https://github.com/pycroscopy/pycroscopy/pull/117>`_
~~~~~~~~
* Improvements to `Bayesian Inference <https://github.com/pycroscopy/pycroscopy/blob/master/pycroscopy/processing/giv_utils.py>`_ on G-mode IV data including resistance compensation.
v 59.2
~~~~~~
* Interactive (Jupyter) visualization for any pycrodataset < 4 dimensions.
......@@ -230,8 +230,23 @@ h5_path = tran.translate(h5_path, raw_data_2d, num_rows, num_cols,
# ===============================
# * Steps 1-3 would be performed anyway in order to begin data analysis
# * The actual pycroscopy translation step are reduced to just 3-4 lines in step 4.
# * A modular / formal version of this translator has been implemented as a class in pycroscopy as the AscTranslator
# found at: https://github.com/pycroscopy/pycroscopy/blob/master/pycroscopy/io/translators/omicron_asc.py .
# This custom translator packages the same code used above into functions that focus on the individual tasks such
# as extracting parameters, reading data, and writing to h5. Once the necessary pieces (parameters, data) are gathered,
# the h5 file can be written very easily using pycroscopy.io.Translator 's .simple_write() function.
# The NumpyTranslator used above uses the very same .simple_write() function underneath to write its data as well.
# * There are many benefits to writing such a formal Translator class instead of standalone scripts like this including:
# * Unlike such a stand-alone script, a Translator class in the package can be used by everyone repeatedly
# * The custom Translator class can ensure consistency when translating multiple files.
# * A single, robust Translator class can handle the finer variations / modes in the data. See the IgorIBWTranslator
# as an example - https://github.com/pycroscopy/pycroscopy/blob/master/pycroscopy/io/translators/igor_ibw.py.
# * While this approach is feasible and encouraged for simple and small data, it may be necessary to use lower level
# calls to write efficient translators
# calls to write efficient translators. As an example, please see the BEPSndfTranslator at:
# https://github.com/pycroscopy/pycroscopy/blob/master/pycroscopy/io/translators/beps_ndf.py
# * We have found python packages online to open a few proprietary file formats and have written translators using these
# packages. If you are having trouble reading the data in your files and cannot find any packages online, consider
# contacting the manufacturer of the instrument which generated the data in the proprietary format for help.
#
# Verifying the newly written H5 file:
# ====================================
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment