Fix Pulsar client staging up for $__tool_directory__ references.

curl transport improvements (timeout handling)

transport.

general framework for setting any other transport options that might be
necessary), and add an exception that can be raised up to the pulsar
runner in Galaxy to handle communication problems while checking state
or submitting jobs.

Fixes a problem encountered on usegalaxy.org where the monitor thread
would encounter some network problem and block forever (until handler
restart).

Partially fixes #114 in that restarts will not kill running jobs, but if
Galaxy attempts to submit a job while Pulsar is down it will still fail.

... per my discretion as BDFN (see Project Goverance for more information https://github.com/galaxyproject/pulsar/blob/master/docs/organization.rst).

Thanks for the continued awesome work on all things Galaxy @mvdbeek, your contributions rock.

Use managers defined in job destination parameters

Fix for locating cli shell/job plugins

The path to the CLI job/shell plugins was build based on the current working
directory, but this would not work if pulsar had been installed into a
virtualenv. Instead we infer the package path with inspect.getsourcefile().

and log a warning.

See https://pypi.python.org/pypi/pulsar-galaxy-lib.

Allow per-manager job directories.

Update: Rebased on to the latest master and documentation for this feature in docs/job_managers.rst and linked to this general manager configuration information from app.yml.sample (per excellent request of @mvdbeek).

If manager name was specified in the destination parameters, include the
managers/<manager_name> route in the url.

Ensure destination directories exist when copying files

Fail jobs that error during job submission

Previously jobs that failed to submit (bad resource requests, malformed
scripts, no qsub available, staging problems) would remain in waiting status
until pulsar was restarted.

for the queued_cli manager.

Fix PBS status reporting

Without this commit only very short jobs completed successfully, because the
status was always set to complete. This was because the status returned by
qstat contains leading spaces before the `job_state = Q` part, so I'm adding a
`.strip()`.