- Depend on galaxy-lib.
 - Move things into pulsar.util and/or pulsar.client.util as makes sense.
 - Use more Galaxy idioms.

Fixes #62.

(In case I need it in the future again.)

Go further than runtime and ensure threads join so that they cannot interfer with other tests.

 - Rearrange things so ssh_key can be alternatively defined in file_actions.yml or the other properties ssh_user, ssh_host, ssh_port can be defined in destination_params. This allows these actions to be usable as the default file action and made it much easier to test.
 - Fixes when constructing scp command-line (my version of Python didn't like a tuple in middle of subprocess.check_call).
 - Increase integration test timeout (these are slower than localhost RESTful transfer).
 - Add integration test for each of these commands.

 - Imports must come before everything else now.
 - Cannot assign lambda to vars - boo!

The LWR use to not make a distinction between cancelled jobs and complete jobs but with server side postprocessing it is more nessecary. Bugfixes including actually recording and use cancelled status in external job runners and persisting the the recorded cancel status in queued and unqueued Python job managers (in some conditions it would revert the cancelled and mark the job as complete).

Catch and suppress exceptions related to cancelling jobs and fetching that status. Failure to ping the metadata store (disk) and determine if a job has been cancelled shouldn't be interpreted as a failure to cancel.

Lots of refactoring and tests to support this.

Will let travis ci see if this works.

Poster is not the best candidate because it is kind of old and likely uncompatible with Python 3 - but is available in Galaxy. Should be targetting requests + requests_toolbelt I guess.

... with pulsar counterparts.

... with pulsar counterparts.

... also improve error log location.

Deprecating old wsgi app_factory module in lieu of lwr.web.wsgi.

Outputs weren't getting rewritten when using remote_copy and remote_transfer, this made a bunch of tests pass that should have been failing and things seem more correct than they actually were.

The exact outlines of how this works will probably change as Galaxy is plugged in, but I am trying to get client as independent of Galaxy as reasonably possible.

This also allows a full integration test using RemoteTransferActions.

This action will transfer files via URL get/post (using pycurl currently). Added test case (including dummy app to mock a potential Galaxy files API) to test this functionality.

Still more work to do - there is no way to populate such urls during setup process and Galaxy still doesn't have a job files API in order to access these files.

Will want to create a different kind of test server to mimic a potential Galaxy job files API.

Create new manager proxy that attempts to deal with complex state changes on its own (this will be a single place to add stateful post-processing shortly). Use new stateful manager proxy to simplfiy other state-y things managers - (e.g. persistence of external ids, job locking when not external DRM is used, recovery).

Eliminate the use of shelf for persistence everything is file based now - but it in a way that I think it would actually be even easier to add a different persistent backend.

Use pylockfile if available for improved lockfile handling.

Simplify the manager interface by exposing job_directory (was being used anyway :)) and eliminating all methods that were just simple proxy calls to it.

Normally, the client obtains a bunch of information about the job at the outset from the LWR server. This work is aimed at computing that on the client side so that this synchronous communication doesn't need to occur - thereby enabling message queue based communication between the LWR and Galaxy.

A lot of work left to do though - this just eliminates that setup step without doing anything about staging files - so it is effectively useless and somewhat untestable in this state - but it is a first step.

In addition to introducing the concept of a setup handler to abstract out this difference, there is various refactoring of stuff into the lwr_client so it can be used inside of Galaxy while computing the setup information on the client side.

Mirrors change made by @dannon to Galaxy.

Added following comment to galaxy/util/__init__.py

This file is a mess, it is a merge of random stuff that is in galaxy.util and
stuff that was in lwr.util. This should be reworked to only contain stuff in
galaxy.util and the rest should be moved into galaxy.util.lwr_io or something
like that.

File mappers by default won't match to these but if path_types includes "unstructure" or is set to "*any*" then such matches will be used.

The unstructured file mappers can include depth attribute on these matches - the default of "0" just stages the file itself, setting depth equal to "1" will stage the whole directory containing the file (though only the matching path will be rewritten in the job inputs), a depth of "2" will cause the whole parent directory of the file's directory to be staged, etc.... This depth feature should be useful when staging implicit index files used by various bioinformatics tools.

Updated the PathMapper and ClientJobDescription so that Galaxy should be able to be instrumented build and pass along these files during tool evaluation when write_parameters is false - though only this default path rewriting variant has been implemented and tested so far.

Additionally, LWR server side changes have been made to support this feature and unit and integration tests are included.

Add option to stager to not attempt to rewrite input, output, and config file paths. Add path mapper to allow an LWR client to build expected paths before transferring files. These options together enable the LWR Galaxy client to be tweaked to build job inputs (command-line and config files) with the correct remote paths instead of building the inputs and then rewritting them.

Fix failing unit tests caused by mock/test apps not defining dependency_manager. Rework logic in stager:finish_job to reduce cyclomatic complexity. Exclude Galaxy test file from PEP-8 checks... since PEP-8 is not standard for Galaxy.

This could hopefully allow the LWR to be nested right inside of Galaxy someday, in the interim it does boost reported test coverage from 75% to 84% because of the way the test web server is started, none of the Python code was being instrumented properly. There is now a 'direct' communication variant of each of these tests resulting in the increased test coverage.

Remove .travis.yml installation of condor - it doesn't work on Ubuntu LTS. Make condor and DRMAA based tests skipable if external environment not configured properly.