Largely stolen from planemo's which in turn was prepolated using cookiecutter (https://github.com/audreyr/cookiecutter).

SSH Transport Implemented

Galaxy file_actions.yaml:

    paths:
        - path: /home/vagrant/
          action: remote_scp_transfer

Behaves identically to rsync

Provides the file action "remote_rsync_transfer". This currently
*requires* you to use a `file_action_config` on the Galaxy side, as
extra configuration is required.

Galaxy job_conf.xml:

    <destination ...>
        <param id="file_action_config">file_actions.yaml</param>

Galaxy file_actions.yaml:

    paths:
        - path: /home/vagrant/
          action: remote_rsync_transfer
          ssh_user: vagrant
          ssh_host: my.fqdn
          ssh_port: 2222
          ssh_key: |
            -----BEGIN RSA PRIVATE KEY-----
            MIIEowIBAAKCAQEAwWYGEOhiJgbWv8eBV2LJp0MCSgrAPeX9FbsGn1I+UC110PPW

This passes configuration information in every message to Pulsar. While
suboptimal in terms of message size (ssh key is ~3x the size of a normal
message), it opens the door to multiple galaxies connecting to a single pulsar,
still using rsync. Additionally it lets the Galaxy admin reconfigure the SSH
parameters without any interaction on the Pulsar admin's part (with the
exception of `ssh_key`, as of now). This will be very beneficial if the two
groups do not have a close working relationship.

Added MQ tutorial/intro and linked docs

Updated the example configuration

Updated the example configuration to use the same parameter name for concurrent jobs that is being read in pulsar/managers/queued.py

Auto jobs directory

... instead of Pulsar root. This allows Galaxy to assume even less about how Pulsar is reflected.

`min_polling_interval` must be treated as a number

Otherwise it gets passed to a timedelta somewhere which is unhappy with having a str passed to it.

Galaxy docker fixes thanks to @kellrott.

https://github.com/jmchilton/planemo

Overhaul of job cancelling.

The LWR use to not make a distinction between cancelled jobs and complete jobs but with server side postprocessing it is more nessecary. Bugfixes including actually recording and use cancelled status in external job runners and persisting the the recorded cancel status in queued and unqueued Python job managers (in some conditions it would revert the cancelled and mark the job as complete).

Catch and suppress exceptions related to cancelling jobs and fetching that status. Failure to ping the metadata store (disk) and determine if a job has been cancelled shouldn't be interpreted as a failure to cancel.

Lots of refactoring and tests to support this.

Testing improvements.

Extend CLI scripts to support only loading application configuration from a YAML file (no .ini needed) - useful in cases such as this where there is no webserver involved.

Test improvements

Can now build a Docker image and run test suite including MQ, run as real user, DRMAA, CLI torque stuff, etc....

Allows run as real user stuff to work in an unmodified Pulsar root if these values are specified.

Hack around failing integration tests?

Will let travis ci see if this works.

Allow configuring job managers in app.yml.

Various fixes, doc updates, etc... as well as allowing dependency_resolvers_config_file and job_metrics_config_file to point at YAML files.

https://bitbucket.org/galaxy/galaxy-central/commits/5fac4d4b7d7d6b056056bf2a6094c7777dc28a71.

Can specify a dictionary of managers of the form:

managers:
  name:
    type: queued_drmaa  # defaults to queued_python
    option1: value1
    option2: value2

or just a single default manager with

manager:
  type: queued_drmaa
  option1: value1
  option2: value2

Breaking configuration and building stages out adds some consistency for how the default manager is constructed and will allow configuring managers via other mechanisms then the job_managers.ini file more easily (namely app.yml).

Configuration Overhaul (Part 1)

This is for values such as staging_directory, persistence_directory, etc.... Anything from app:main except for paste.app_factory.

Can specify the location in server.ini as app_config, otherwise is there is a file called app.yml deployed next to server.ini (or whatever .ini) this will be used.

The documentation state that this could be unset to disable persistence - this is not the case. Added a new way to do this (by setting it to __none__) and updated documentation in sample file. Also commented it out in the sample file.

...regardless is being loaded from the command line or from a WSGI server.