Merge branch '31-get-status' into 'main' (586fe193) · Commits · NDIP / Nova Packages / nova-galaxy

CHANGELOG.md

+19 −1

Original line number	Diff line number	Diff line
		## Nova Galaxy 0.9.1
		- Added `get_full_status` method to tool in order to get detailed messages mostly for error states (thanks to Gregory Cage). [Merge Request 23](https://code.ornl.gov/ndip/public-packages/nova-galaxy/-/merge_requests/23)

		### Nova Galaxy 0.9.0
		- When uploading datasets with manually set content, the upstream name will mirror the local name property of the dataset (thanks to Gregory Cage). [Merge Request 22](https://code.ornl.gov/ndip/public-packages/nova-galaxy/-/merge_requests/22)
		- New WorkStates for the actual process of stopping and canceling jobs (separate from the terminal states already present) (thanks to Sergey Yakubov and Gregory Cage). [Merge Request 22](https://code.ornl.gov/ndip/public-packages/nova-galaxy/-/merge_requests/22)
		- Fixed major bug where tools were not being stopped and fetching results properly (canceling worked fine) (thanks to Gregory Cage). [Merge Request 22](https://code.ornl.gov/ndip/public-packages/nova-galaxy/-/merge_requests/22)
		- Made tool status thread safe (thanks to Sergey Yakubov and Gregory Cage). [Merge Request 22](https://code.ornl.gov/ndip/public-packages/nova-galaxy/-/merge_requests/22)
		- If canceling or stopping jobs in the uploading data state, will stop the uploading when able (thanks to Sergey Yakubov and Gregory Cage). [Merge Request 22](https://code.ornl.gov/ndip/public-packages/nova-galaxy/-/merge_requests/22)
		- Misc backend code cleanup (thanks to Sergey Yakubov and Gregory Cage). [Merge Request 22](https://code.ornl.gov/ndip/public-packages/nova-galaxy/-/merge_requests/22)

		### Nova Galaxy, 0.8.2
		- Now returns file type automatically if available (thanks to Gregory Cage). [Merge Request 21](https://code.ornl.gov/ndip/public-packages/nova-galaxy/-/merge_requests/21)
		- Returns file content as bytes instead of string (thanks to Gregory Cage). [Merge Request 21](https://code.ornl.gov/ndip/public-packages/nova-galaxy/-/merge_requests/21)


		### Nova Galaxy, 0.8.1
		- Can now fetch specific stdout and stderr positions and length (thanks to Gregory Cage). [Merge Request 19](https://code.ornl.gov/ndip/public-packages/nova-galaxy/-/merge_requests/19)

		### Nova Galaxy, 0.8.0
		- `get_data_store()` has been added to the ConnectionHelper class. This functionally does the same thing as create_data_store, but users can choose whether to only use existing upstream data stores. `create_data_store` creates data stores by default and connects to existing ones as well automatically (thanks to Gregory Cage). [Merge Request 18](https://code.ornl.gov/ndip/public-packages/nova-galaxy/-/merge_requests/18)
		- `Connections.connect()` can now be used with or without the `with` keyword. Consequently, stores can also be created outside a `with` block. `Connection.close()` performs the clean up that exiting the `with` block provides (thanks to Gregory Cage). [Merge Request 18](https://code.ornl.gov/ndip/public-packages/nova-galaxy/-/merge_requests/18)
		- Data stores can be cleaned up manually (thanks to Gregory Cage). [Merge Request 18](https://code.ornl.gov/ndip/public-packages/nova-galaxy/-/merge_requests/18)
		- Can now wait for the result of a running tool (thanks to Gregory Cage). [Merge Request 18](https://code.ornl.gov/ndip/public-packages/nova-galaxy/-/merge_requests/18)


		### Nova Galaxy, 0.7.4
		- Allow users to choose to check URL when calling get_url() from a Tool (thanks to Gregory Cage). [Merge Request 17](https://code.ornl.gov/ndip/public-packages/nova-galaxy/-/merge_requests/17)
		- Return more detailed information when getting the content of a DatasetCollection (thanks to Gregory Cage). [Merge Request 17](https://code.ornl.gov/ndip/public-packages/nova-galaxy/-/merge_requests/17)

pyproject.toml

+1 −1

Original line number	Diff line number	Diff line
		[tool.poetry]
		name = "nova-galaxy"
		version = "0.9.0"
		version = "0.9.1"
		description = "Utilties for accessing the ORNL Galaxy instance"
		authors = ["Greg Watson <watsongr@ornl.gov>", "Gregory Cage <cagege@ornl.gov>"]
		readme = "README.md"

src/nova/galaxy/job.py

+17 −6

Original line number	Diff line number	Diff line
		@@ -19,9 +19,8 @@ class JobStatus:
		"""Internal structure to hold job status info."""

		def __init__(self) -> None:
		self.details = ""
		self.error_msg = ""
		self.lock = Lock()
		self._details = ""
		self._state = WorkState.NOT_STARTED

		@property
		@@ -34,6 +33,16 @@ class JobStatus:
		with self.lock:
		self._state = value

		@property
		def details(self) -> str:
		with self.lock:
		return self._details

		@details.setter
		def details(self, value: str) -> None:
		with self.lock:
		self._details = value


		class Job:
		"""Internal class managing Galaxy job execution. Should not be used by end users."""
		@@ -51,8 +60,8 @@ class Job:

		def _run_and_wait(self, params: Optional[Parameters]) -> None:
		"""Runs tools and waits for result."""
		self.submit(params)
		try:
		self.submit(params)
		self.wait_for_results()
		except Exception as e:
		self.url = None
		@@ -60,7 +69,7 @@ class Job:
		self.status.state = WorkState.CANCELED
		return
		self.status.state = WorkState.ERROR
		self.status.error_msg = str(e)
		self.status.details = str(e)
		return

		self.status.state = WorkState.FINISHED
		@@ -171,7 +180,7 @@ class Job:
		if response:
		return True
		else:
		self.status.error_msg = "could not stop job"
		self.status.details = "could not stop job"
		return False

		def cancel(self) -> bool:
		@@ -206,7 +215,7 @@ class Job:
		pass
		return self.status

		def get_results(self) -> Outputs:
		def get_results(self) -> Optional[Outputs]:
		"""Return results from finished job."""
		if self.status.state == WorkState.FINISHED:
		outputs = Outputs()
		@@ -225,6 +234,8 @@ class Job:
		outputs.add_output(dc)

		return outputs
		elif self.status.state == WorkState.ERROR:
		return None
		else:
		raise Exception(f"Job {self.id} has not finished running.")

src/nova/galaxy/tool.py

+20 −3

Original line number	Diff line number	Diff line
		@@ -7,7 +7,7 @@ if TYPE_CHECKING:
		from .data_store import Datastore # Only imports for type checking

		from .dataset import AbstractData
		from .job import Job
		from .job import Job, JobStatus
		from .outputs import Outputs
		from .parameters import Parameters
		from .util import WorkState
		@@ -108,14 +108,31 @@ class Tool(AbstractWork):
		Returns
		-------
		WorkState
		Returns the status of the tool which will be one of the following values: not_started, uploading, queued,
		running, finished, error
		Returns the status of the tool which will be a WorkState value.
		"""
		if self._job:
		return self._job.get_state().state
		else:
		return WorkState.NOT_STARTED

		def get_full_status(self) -> JobStatus:
		"""Returns the full status and state of the tool. Includes any error or warning messages.

		Raises
		------
		Exception
		If the job has not been started.

		Returns
		-------
		JobStatus
		Returns the full status of the tool including the WorkState and any error messages set by the tool.
		"""
		if self._job:
		return self._job.get_state()
		else:
		raise Exception("Job has not started.")

		def get_results(self) -> Optional[Outputs]:
		"""Returns the results from running this tool.

tests/test_run_tool.py

+13 −0

Original line number	Diff line number	Diff line
		@@ -81,6 +81,19 @@ def test_status(nova_instance: Connection) -> None:
		assert state == WorkState.FINISHED


		def test_full_status(nova_instance: Connection) -> None:
		with nova_instance.connect() as connection:
		store = connection.get_data_store(name="nova_galaxy_testing")
		store.mark_for_cleanup()
		test_tool = Tool(TEST_TOOL_ID)
		params = Parameters()
		params.add_input("command_mode\|command", "fail")
		test_tool.run(data_store=store, params=params)
		assert test_tool.get_status() == WorkState.ERROR
		assert test_tool.get_full_status().details != ""
		assert test_tool.get_stderr() != ""


		def test_cancel_tool(nova_instance: Connection) -> None:
		with nova_instance.connect() as connection:
		store = connection.get_data_store(name="nova_galaxy_testing")