This will write to metadata/memory_statement.log, instead of the working
directory. Fixes a bunch of tests that use metadata collection.

and always set up job i/o redirection on the pulsar side.
Since pulsar stdio is broken prior to this on 23.0 this is maybe
a good time to make the switch. Seems like this detail should be
controlled on pulsar, not Galaxy.

Load JobFilesApp from new external dependency.

[WIP] Allow overriding amqp prefix that doesn't match the manager name.

Taking advice from Nicola in https://github.com/galaxyproject/pulsar/pull/312.

See https://tox.wiki/en/latest/changelog.html#deprecations-and-removals-4-0-0rc4

Cluster memory statement changes from Galaxy

[WIP] Refactor base co-execution client out of Kubernetes specific stuff.

Try to setup TES tests for CI.

Currently the only container-based scheduled (where we schedule containers and not schedule jobs that launch containers) is the Kubernetes message queue based coexecution approach.

On one hand, we have some intriguing applications TES and AWS Batch where we would like to schedule containers directly and on the other the MQ approach with Kuberentes can easily be generalized to not require the MQ (falling back to polling the way the Kubernetes job runner in Galaxy does).

The goal of this work is to generalize the MQ Kubernetes approach into six approaches:

- MQ + Kubernetes (the current recommendation)
- Kubernetes w/polling API (a simpler Kubernetes approach that retains all the advantages of the Pulsar approach over the Kubernetes runner in Galaxy without requiring a MQ).
- MQ + TES.
- TES w/polling.
- MQ + AWS Batch
- AWS Batch w/polling.

TES
----------

I've developed a client library for TES called pydantic-tes (https://github.com/jmchilton/pydantic-tes) - that should use validated models to communicate with a TES server and is tested against Funnel. It also distributes a pytest fixture that can build and launch funnel for writing automated tests and that works with tox and Github actions as demonstrated by the pydantic-tes CI.

AWS Batch
----------

TODO:

Sequential vs Parallel Container Execution
-------------------------------------------

This work contains a generalization of the approach used in Kubernetes of co-execution of Pulsar and Biocontainers, but the model for TES and AWS Batch are more serial container executions - this runs, then that, then that, etc... In TES this is given as a list of "Executors" and AWS Batch has the idea of the job dependencies that I believe can capture this - but this will require a slightly alternative approach (probably simpler) than the K8S co-execution approach in which the containers wait on each other to write files in order to coordinate.

Fix https://github.com/galaxyproject/pulsar/pull/299.

Last Galaxy error I think.