Documentation changes for version 0.9.0

HISTORY.rst

@@ -6,10 +6,21 @@ History
 .. to_doc
 
 ---------------------
-0.9.0.dev0
+0.9.0 (2019-04-12)
 ---------------------
-
     
+* Add configuration parameter to limit stream size read from disk. `Pull
+  Request 157`_
+* Pass full job status for failed and lost jobs. `Pull Request 159`_
+* Improve message handling if problems occur during job setup/staging. `Pull
+  Request 160`_
+* Rework preprocessing job state to improve restartability and reduce job loss.
+  **This change should be applied while no jobs are running.**
+  `Pull Request 164`_
+* Add support for overriding config through environment variables (thanks to
+  @nuwang). `Pull Request 165`_
+* Minor docs updates (thanks to @afgane). `Pull Request 170`_
+* Python 3 fixes in Pulsar client (thanks to @mvdbeek). `Pull Request 172`_
 
 ---------------------
 0.8.3 (2018-02-08)
@@ -186,6 +197,13 @@ History
 
 
 .. github_links
+.. _Pull Request 172: https://github.com/galaxyproject/pulsar/pull/172
+.. _Pull Request 170: https://github.com/galaxyproject/pulsar/pull/170
+.. _Pull Request 165: https://github.com/galaxyproject/pulsar/pull/165
+.. _Pull Request 164: https://github.com/galaxyproject/pulsar/pull/164
+.. _Pull Request 160: https://github.com/galaxyproject/pulsar/pull/160
+.. _Pull Request 159: https://github.com/galaxyproject/pulsar/pull/159
+.. _Pull Request 157: https://github.com/galaxyproject/pulsar/pull/157
 .. _Pull Request 156: https://github.com/galaxyproject/pulsar/pull/156
 .. _Pull Request 145: https://github.com/galaxyproject/pulsar/pull/145
 .. _Pull Request 146: https://github.com/galaxyproject/pulsar/pull/146

docs/pulsar.cache.rst

-pulsar\.cache package
-=====================
+pulsar.cache package
+====================
 
 Submodules
 ----------
 
-pulsar\.cache\.persistence module
----------------------------------
+pulsar.cache.persistence module
+-------------------------------
 
 .. automodule:: pulsar.cache.persistence
     :members:
     :undoc-members:
     :show-inheritance:
 
-pulsar\.cache\.util module
---------------------------
+pulsar.cache.util module
+------------------------
 
 .. automodule:: pulsar.cache.util
     :members:

docs/pulsar.client.rst

-pulsar\.client package
-======================
+pulsar.client package
+=====================
 
 Subpackages
 -----------
@@ -13,120 +13,120 @@ Subpackages
 Submodules
 ----------
 
-pulsar\.client\.action\_mapper module
--------------------------------------
+pulsar.client.action\_mapper module
+-----------------------------------
 
 .. automodule:: pulsar.client.action_mapper
     :members:
     :undoc-members:
     :show-inheritance:
 
-pulsar\.client\.amqp\_exchange module
--------------------------------------
+pulsar.client.amqp\_exchange module
+-----------------------------------
 
 .. automodule:: pulsar.client.amqp_exchange
     :members:
     :undoc-members:
     :show-inheritance:
 
-pulsar\.client\.amqp\_exchange\_factory module
-----------------------------------------------
+pulsar.client.amqp\_exchange\_factory module
+--------------------------------------------
 
 .. automodule:: pulsar.client.amqp_exchange_factory
     :members:
     :undoc-members:
     :show-inheritance:
 
-pulsar\.client\.client module
------------------------------
+pulsar.client.client module
+---------------------------
 
 .. automodule:: pulsar.client.client
     :members:
     :undoc-members:
     :show-inheritance:
 
-pulsar\.client\.config\_util module
------------------------------------
+pulsar.client.config\_util module
+---------------------------------
 
 .. automodule:: pulsar.client.config_util
     :members:
     :undoc-members:
     :show-inheritance:
 
-pulsar\.client\.decorators module
----------------------------------
+pulsar.client.decorators module
+-------------------------------
 
 .. automodule:: pulsar.client.decorators
     :members:
     :undoc-members:
     :show-inheritance:
 
-pulsar\.client\.destination module
-----------------------------------
+pulsar.client.destination module
+--------------------------------
 
 .. automodule:: pulsar.client.destination
     :members:
     :undoc-members:
     :show-inheritance:
 
-pulsar\.client\.exceptions module
----------------------------------
+pulsar.client.exceptions module
+-------------------------------
 
 .. automodule:: pulsar.client.exceptions
     :members:
     :undoc-members:
     :show-inheritance:
 
-pulsar\.client\.interface module
---------------------------------
+pulsar.client.interface module
+------------------------------
 
 .. automodule:: pulsar.client.interface
     :members:
     :undoc-members:
     :show-inheritance:
 
-pulsar\.client\.job\_directory module
--------------------------------------
+pulsar.client.job\_directory module
+-----------------------------------
 
 .. automodule:: pulsar.client.job_directory
     :members:
     :undoc-members:
     :show-inheritance:
 
-pulsar\.client\.manager module
-------------------------------
+pulsar.client.manager module
+----------------------------
 
 .. automodule:: pulsar.client.manager
     :members:
     :undoc-members:
     :show-inheritance:
 
-pulsar\.client\.object\_client module
--------------------------------------
+pulsar.client.object\_client module
+-----------------------------------
 
 .. automodule:: pulsar.client.object_client
     :members:
     :undoc-members:
     :show-inheritance:
 
-pulsar\.client\.path\_mapper module
------------------------------------
+pulsar.client.path\_mapper module
+---------------------------------
 
 .. automodule:: pulsar.client.path_mapper
     :members:
     :undoc-members:
     :show-inheritance:
 
-pulsar\.client\.setup\_handler module
--------------------------------------
+pulsar.client.setup\_handler module
+-----------------------------------
 
 .. automodule:: pulsar.client.setup_handler
     :members:
     :undoc-members:
     :show-inheritance:
 
-pulsar\.client\.util module
----------------------------
+pulsar.client.util module
+-------------------------
 
 .. automodule:: pulsar.client.util
     :members:

docs/pulsar.client.staging.rst

-pulsar\.client\.staging package
-===============================
+pulsar.client.staging package
+=============================
 
 Submodules
 ----------
 
-pulsar\.client\.staging\.down module
-------------------------------------
+pulsar.client.staging.down module
+---------------------------------
 
 .. automodule:: pulsar.client.staging.down
     :members:
     :undoc-members:
     :show-inheritance:
 
-pulsar\.client\.staging\.up module
-----------------------------------
+pulsar.client.staging.up module
+-------------------------------
 
 .. automodule:: pulsar.client.staging.up
     :members:

docs/pulsar.client.test.rst

-pulsar\.client\.test package
-============================
+pulsar.client.test package
+==========================
 
 Submodules
 ----------
 
-pulsar\.client\.test\.check module
-----------------------------------
+pulsar.client.test.check module
+-------------------------------
 
 .. automodule:: pulsar.client.test.check
     :members:
     :undoc-members:
     :show-inheritance:
 
-pulsar\.client\.test\.test\_common module
------------------------------------------
+pulsar.client.test.test\_common module
+--------------------------------------
 
 .. automodule:: pulsar.client.test.test_common
     :members:

docs/pulsar.client.transport.rst

-pulsar\.client\.transport package
-=================================
+pulsar.client.transport package
+===============================
 
 Submodules
 ----------
 
-pulsar\.client\.transport\.curl module
---------------------------------------
+pulsar.client.transport.curl module
+-----------------------------------
 
 .. automodule:: pulsar.client.transport.curl
     :members:
     :undoc-members:
     :show-inheritance:
 
-pulsar\.client\.transport\.poster module
-----------------------------------------
+pulsar.client.transport.poster module
+-------------------------------------
 
 .. automodule:: pulsar.client.transport.poster
     :members:
     :undoc-members:
     :show-inheritance:
 
-pulsar\.client\.transport\.requests module
-------------------------------------------
+pulsar.client.transport.requests module
+---------------------------------------
 
 .. automodule:: pulsar.client.transport.requests
     :members:
     :undoc-members:
     :show-inheritance:
 
-pulsar\.client\.transport\.ssh module
--------------------------------------
+pulsar.client.transport.ssh module
+----------------------------------
 
 .. automodule:: pulsar.client.transport.ssh
     :members:
     :undoc-members:
     :show-inheritance:
 
-pulsar\.client\.transport\.standard module
-------------------------------------------
+pulsar.client.transport.standard module
+---------------------------------------
 
 .. automodule:: pulsar.client.transport.standard
     :members:

docs/pulsar.managers.base.rst

-pulsar\.managers\.base package
-==============================
+pulsar.managers.base package
+============================
 
 Submodules
 ----------
 
-pulsar\.managers\.base\.base\_drmaa module
-------------------------------------------
+pulsar.managers.base.base\_drmaa module
+---------------------------------------
 
 .. automodule:: pulsar.managers.base.base_drmaa
     :members:
     :undoc-members:
     :show-inheritance:
 
-pulsar\.managers\.base\.directory module
-----------------------------------------
+pulsar.managers.base.directory module
+-------------------------------------
 
 .. automodule:: pulsar.managers.base.directory
     :members:
     :undoc-members:
     :show-inheritance:
 
-pulsar\.managers\.base\.external module
----------------------------------------
+pulsar.managers.base.external module
+------------------------------------
 
 .. automodule:: pulsar.managers.base.external
     :members:

docs/pulsar.managers.rst

-pulsar\.managers package
-========================
+pulsar.managers package
+=======================
 
 Subpackages
 -----------
@@ -13,80 +13,80 @@ Subpackages
 Submodules
 ----------
 
-pulsar\.managers\.queued module
--------------------------------
+pulsar.managers.queued module
+-----------------------------
 
 .. automodule:: pulsar.managers.queued
     :members:
     :undoc-members:
     :show-inheritance:
 
-pulsar\.managers\.queued\_cli module
-------------------------------------
+pulsar.managers.queued\_cli module
+----------------------------------
 
 .. automodule:: pulsar.managers.queued_cli
     :members:
     :undoc-members:
     :show-inheritance:
 
-pulsar\.managers\.queued\_condor module
----------------------------------------
+pulsar.managers.queued\_condor module
+-------------------------------------
 
 .. automodule:: pulsar.managers.queued_condor
     :members:
     :undoc-members:
     :show-inheritance:
 
-pulsar\.managers\.queued\_drmaa module
---------------------------------------
+pulsar.managers.queued\_drmaa module
+------------------------------------
 
 .. automodule:: pulsar.managers.queued_drmaa
     :members:
     :undoc-members:
     :show-inheritance:
 
-pulsar\.managers\.queued\_drmaa\_xsede module
----------------------------------------------
+pulsar.managers.queued\_drmaa\_xsede module
+-------------------------------------------
 
 .. automodule:: pulsar.managers.queued_drmaa_xsede
     :members:
     :undoc-members:
     :show-inheritance:
 
-pulsar\.managers\.queued\_external\_drmaa module
-------------------------------------------------
+pulsar.managers.queued\_external\_drmaa module
+----------------------------------------------
 
 .. automodule:: pulsar.managers.queued_external_drmaa
     :members:
     :undoc-members:
     :show-inheritance:
 
-pulsar\.managers\.queued\_pbs module
-------------------------------------
+pulsar.managers.queued\_pbs module
+----------------------------------
 
 .. automodule:: pulsar.managers.queued_pbs
     :members:
     :undoc-members:
     :show-inheritance:
 
-pulsar\.managers\.stateful module
----------------------------------
+pulsar.managers.stateful module
+-------------------------------
 
 .. automodule:: pulsar.managers.stateful
     :members:
     :undoc-members:
     :show-inheritance:
 
-pulsar\.managers\.status module
--------------------------------
+pulsar.managers.status module
+-----------------------------
 
 .. automodule:: pulsar.managers.status
     :members:
     :undoc-members:
     :show-inheritance:
 
-pulsar\.managers\.unqueued module
----------------------------------
+pulsar.managers.unqueued module
+-------------------------------
 
 .. automodule:: pulsar.managers.unqueued
     :members:

docs/pulsar.managers.staging.rst

-pulsar\.managers\.staging package
-=================================
+pulsar.managers.staging package
+===============================
 
 Submodules
 ----------
 
-pulsar\.managers\.staging\.post module
---------------------------------------
+pulsar.managers.staging.post module
+-----------------------------------
 
 .. automodule:: pulsar.managers.staging.post
     :members:
     :undoc-members:
     :show-inheritance:
 
-pulsar\.managers\.staging\.pre module
--------------------------------------
+pulsar.managers.staging.pre module
+----------------------------------
 
 .. automodule:: pulsar.managers.staging.pre
     :members:

docs/pulsar.managers.util.cli.job.rst

-pulsar\.managers\.util\.cli\.job package
-========================================
+pulsar.managers.util.cli.job package
+====================================
 
 Submodules
 ----------
 
-pulsar\.managers\.util\.cli\.job\.slurm module
-----------------------------------------------
+pulsar.managers.util.cli.job.slurm module
+-----------------------------------------
 
 .. automodule:: pulsar.managers.util.cli.job.slurm
     :members:
     :undoc-members:
     :show-inheritance:
 
-pulsar\.managers\.util\.cli\.job\.slurm\_torque module
-------------------------------------------------------
+pulsar.managers.util.cli.job.slurm\_torque module
+-------------------------------------------------
 
 .. automodule:: pulsar.managers.util.cli.job.slurm_torque
     :members:
     :undoc-members:
     :show-inheritance:
 
-pulsar\.managers\.util\.cli\.job\.torque module
------------------------------------------------
+pulsar.managers.util.cli.job.torque module
+------------------------------------------
 
 .. automodule:: pulsar.managers.util.cli.job.torque
     :members:

docs/pulsar.managers.util.cli.rst

-pulsar\.managers\.util\.cli package
-===================================
+pulsar.managers.util.cli package
+================================
 
 Subpackages
 -----------
@@ -12,8 +12,8 @@ Subpackages
 Submodules
 ----------
 
-pulsar\.managers\.util\.cli\.factory module
--------------------------------------------
+pulsar.managers.util.cli.factory module
+---------------------------------------
 
 .. automodule:: pulsar.managers.util.cli.factory
     :members:

docs/pulsar.managers.util.cli.shell.rst

-pulsar\.managers\.util\.cli\.shell package
-==========================================
+pulsar.managers.util.cli.shell package
+======================================
 
 Submodules
 ----------
 
-pulsar\.managers\.util\.cli\.shell\.local module
-------------------------------------------------
+pulsar.managers.util.cli.shell.local module
+-------------------------------------------
 
 .. automodule:: pulsar.managers.util.cli.shell.local
     :members:
     :undoc-members:
     :show-inheritance:
 
-pulsar\.managers\.util\.cli\.shell\.rsh module
-----------------------------------------------
+pulsar.managers.util.cli.shell.rsh module
+-----------------------------------------
 
 .. automodule:: pulsar.managers.util.cli.shell.rsh
     :members:

docs/pulsar.managers.util.condor.rst

-pulsar\.managers\.util\.condor package
-======================================
+pulsar.managers.util.condor package
+===================================
 
 Module contents
 ---------------

docs/pulsar.managers.util.drmaa.rst

-pulsar\.managers\.util\.drmaa package
-=====================================
+pulsar.managers.util.drmaa package
+==================================
 
 Module contents
 ---------------

docs/pulsar.managers.util.job_script.rst

-pulsar\.managers\.util\.job\_script package
-===========================================
+pulsar.managers.util.job\_script package
+========================================
 
 Module contents
 ---------------

docs/pulsar.managers.util.rst

-pulsar\.managers\.util package
-==============================
+pulsar.managers.util package
+============================
 
 Subpackages
 -----------
@@ -14,40 +14,40 @@ Subpackages
 Submodules
 ----------
 
-pulsar\.managers\.util\.env module
-----------------------------------
+pulsar.managers.util.env module
+-------------------------------
 
 .. automodule:: pulsar.managers.util.env
     :members:
     :undoc-members:
     :show-inheritance:
 
-pulsar\.managers\.util\.external module
----------------------------------------
+pulsar.managers.util.external module
+------------------------------------
 
 .. automodule:: pulsar.managers.util.external
     :members:
     :undoc-members:
     :show-inheritance:
 
-pulsar\.managers\.util\.kill module
------------------------------------
+pulsar.managers.util.kill module
+--------------------------------
 
 .. automodule:: pulsar.managers.util.kill
     :members:
     :undoc-members:
     :show-inheritance:
 
-pulsar\.managers\.util\.retry module
-------------------------------------
+pulsar.managers.util.retry module
+---------------------------------
 
 .. automodule:: pulsar.managers.util.retry
     :members:
     :undoc-members:
     :show-inheritance:
 
-pulsar\.managers\.util\.sudo module
------------------------------------
+pulsar.managers.util.sudo module
+--------------------------------
 
 .. automodule:: pulsar.managers.util.sudo
     :members:

docs/pulsar.mesos.rst

-pulsar\.mesos package
-=====================
+pulsar.mesos package
+====================
 
 Submodules
 ----------
 
-pulsar\.mesos\.framework module
--------------------------------
+pulsar.mesos.framework module
+-----------------------------
 
 .. automodule:: pulsar.mesos.framework
     :members:

docs/pulsar.messaging.rst

-pulsar\.messaging package
-=========================
+pulsar.messaging package
+========================
 
 Submodules
 ----------
 
-pulsar\.messaging\.bind\_amqp module
-------------------------------------
+pulsar.messaging.bind\_amqp module
+----------------------------------
 
 .. automodule:: pulsar.messaging.bind_amqp
     :members:

docs/pulsar.rst

@@ -19,40 +19,40 @@ Subpackages
 Submodules
 ----------
 
-pulsar\.core module
--------------------
+pulsar.core module
+------------------
 
 .. automodule:: pulsar.core
     :members:
     :undoc-members:
     :show-inheritance:
 
-pulsar\.locks module
---------------------
+pulsar.locks module
+-------------------
 
 .. automodule:: pulsar.locks
     :members:
     :undoc-members:
     :show-inheritance:
 
-pulsar\.main module
--------------------
+pulsar.main module
+------------------
 
 .. automodule:: pulsar.main
     :members:
     :undoc-members:
     :show-inheritance:
 
-pulsar\.manager\_endpoint\_util module
---------------------------------------
+pulsar.manager\_endpoint\_util module
+-------------------------------------
 
 .. automodule:: pulsar.manager_endpoint_util
     :members:
     :undoc-members:
     :show-inheritance:
 
-pulsar\.manager\_factory module
--------------------------------
+pulsar.manager\_factory module
+------------------------------
 
 .. automodule:: pulsar.manager_factory
     :members:

docs/pulsar.scripts.rst

-pulsar\.scripts package
-=======================
+pulsar.scripts package
+======================
 
 Submodules
 ----------
 
-pulsar\.scripts\.chown\_working\_directory module
--------------------------------------------------
+pulsar.scripts.bootstrap module
+-------------------------------
+
+.. automodule:: pulsar.scripts.bootstrap
+    :members:
+    :undoc-members:
+    :show-inheritance:
+
+pulsar.scripts.chown\_working\_directory module
+-----------------------------------------------
 
 .. automodule:: pulsar.scripts.chown_working_directory
     :members:
     :undoc-members:
     :show-inheritance:
 
-pulsar\.scripts\.config module
-------------------------------
+pulsar.scripts.config module
+----------------------------
 
 .. automodule:: pulsar.scripts.config
     :members:
     :undoc-members:
     :show-inheritance:
 
-pulsar\.scripts\.drmaa\_kill module
------------------------------------
+pulsar.scripts.drmaa\_kill module
+---------------------------------
 
 .. automodule:: pulsar.scripts.drmaa_kill
     :members:
     :undoc-members:
     :show-inheritance:
 
-pulsar\.scripts\.drmaa\_launch module
--------------------------------------
+pulsar.scripts.drmaa\_launch module
+-----------------------------------
 
 .. automodule:: pulsar.scripts.drmaa_launch
     :members:
     :undoc-members:
     :show-inheritance:
 
-pulsar\.scripts\.mesos\_executor module
----------------------------------------
+pulsar.scripts.mesos\_executor module
+-------------------------------------
 
 .. automodule:: pulsar.scripts.mesos_executor
     :members:
     :undoc-members:
     :show-inheritance:
 
-pulsar\.scripts\.mesos\_framework module
-----------------------------------------
+pulsar.scripts.mesos\_framework module
+--------------------------------------
 
 .. automodule:: pulsar.scripts.mesos_framework
     :members:
     :undoc-members:
     :show-inheritance:
 
-pulsar\.scripts\.run module
----------------------------
+pulsar.scripts.run module
+-------------------------
 
 .. automodule:: pulsar.scripts.run
     :members:
     :undoc-members:
     :show-inheritance:
 
-pulsar\.scripts\.submit module
-------------------------------
+pulsar.scripts.submit module
+----------------------------
 
 .. automodule:: pulsar.scripts.submit
     :members:
     :undoc-members:
     :show-inheritance:
 
-pulsar\.scripts\.submit\_util module
-------------------------------------
+pulsar.scripts.submit\_util module
+----------------------------------
 
 .. automodule:: pulsar.scripts.submit_util
     :members: