Create new manager proxy that attempts to deal with complex state changes on its own (this will be a single place to add stateful post-processing shortly). Use new stateful manager proxy to simplfiy other state-y things managers - (e.g. persistence of external ids, job locking when not external DRM is used, recovery).

Eliminate the use of shelf for persistence everything is file based now - but it in a way that I think it would actually be even easier to add a different persistent backend.

Use pylockfile if available for improved lockfile handling.

Simplify the manager interface by exposing job_directory (was being used anyway :)) and eliminating all methods that were just simple proxy calls to it.