This creates a Docker image named `nova-tutorial` with the tag `latest` that contains our application and all its dependencies.
::::::::::::::::::::::::::::::::::::::::: callout
Normally, we would push this image to a container registry so that NDIP can access it. However, for this tutorial, we'll skip this step. In a real deployment, you would push the image using a command like `docker push <registry-url>/nova-tutorial:latest`. For the tutorial, we'll use a pre-pushed container.
When releasing new versions of your tool, you'll want to use versioned tags for your container images, such as `nova-tutorial:1.0.0`, to maintain backward compatibility while allowing for updates.
::::::::::::::::::::::::::::::::::::::::::::::::
### Cloning the Galaxy Tools Repository
To deploy our tool to the NDIP platform, we need to add its XML definition to the galaxy-tools repository. Let's clone the repository and switch to the prototype branch:
Now we'll create an XML file that defines our tool for the NDIP platform. We'll place it in the appropriate directory within the galaxy-tools repository:
git commit -m"Add <username>'s NOVA tutorial tool"
# Push the changes to the prototype branch
git push origin prototype
```
Once your changes are pushed to the prototype branch, an automated CI job will deploy your tool to the calvera-test instance. You can then access your tool through the NDIP web interface at https://calvera-test.ornl.gov.
::::::::::::::::::::::::::::::::::::::::: callout
In a production environment, when your tool is ready for users, you would create a merge request from the prototype branch to the dev branch. The NDIP team reviews these changes, merges them, and your tool will be deployed to the production instance during the next deployment.
In this section, we will look at other resources you may want to integrate with your application and outline the process for taking an application like the one we\'ve created in this tutorial and deploying it to the NOVA/NDIP platform. While we won\'t actually perform the deployment in this tutorial, we will cover the key steps and resources involved.
In this section, we will look at other resources you may want to integrate with your application.
## ONCat Integration
If needed, you can integrate your application with ONCat via [pyoncat](https://pypi.org/project/pyoncat/2.1/). If you need to access non-public information with the API then you will need to use an authenticated client in `pyoncat`. We strongly recommend you email oncat-support@ornl.gov explaining the use case for your ONCat integration as they can advise you on the most appropriate form of authentication for your application and how to set it up.
## Containerizing Your Application
## Advanced Container Configurations
The first step in deploying your application to the NOVA/NDIP platform is to package it as a Docker container. Docker containers provide a lightweight and portable way to package your application and all of its dependencies. This ensures that your application will run consistently across different environments.
Fortunately, the template application we used in this tutorial already includes a `Dockerfile` that you can use as a starting point. The Dockerfile is a set of instructions that Docker uses to build your container image.
Here\'s what the Dockerfile typically includes:
***Base Image:** Specifies the base operating system and environment for your application.
***Dependencies:** Describes how to install any required libraries or packages.
***Application Files:** Defines how to copy your application code into the container.
***Entrypoint:** Sets the command that is run when the container starts.
To containerize your application, you would:
1. Navigate to the top level of your project (where the `dockerfiles` folder is).
2. Run the docker build command in the following format `docker build -t <your_image_name>:<your_image_tag> -f dockerfiles/Dockerfile .`
3. Test your docker container using the command `docker run <your_image_name>:<your_image_tag>`.
4. Push your docker container to a container registry.
After the docker container is deployed to a registry, it can then be used by the platform.
The basic container configuration we created in Episode 2 works well for most applications, but there are some advanced configurations that may be useful for more complex applications.
::::::::::::::::::::::::::::::::::::::::: callout
GPU acceleration in a container is possible but beyond the scope of this tutorial. Typically, a base container is chosen which already has all of the gpu dependencies installed such as ```regproxy.ornl.gov/hub_proxy/kitware/trame:py3.10-glvnd-2024-12```. The team has built similar containers already which can be used as a reference for development, such as,```https://code.ornl.gov/ndip/trame-apps/ct-scan-visualizer/```
GPU acceleration in a container is possible but beyond the scope of this tutorial. Typically, a base container is chosen which already has all of the GPU dependencies installed such as ```regproxy.ornl.gov/hub_proxy/kitware/trame:py3.10-glvnd-2024-12```. The team has built similar containers already which can be used as a reference for development, such as ```https://code.ornl.gov/ndip/trame-apps/ct-scan-visualizer/```
Once your application is containerized, you will also need to define your tool using an XML file. This XML file describes your tool to the NDIP platform, including its inputs, outputs, parameters, and the Docker container image that should be used to run the tool. The NDIP platform makes use of the Galaxy tool framework.
The XML file includes:
## Development Lifecycle
***Tool ID:** A unique identifier for your tool.
***Name and Description:** User-friendly name and description of the tool.
***Inputs:** Defines the input parameters, including their types, labels, and optional constraints.
***Outputs:** Describes the output files or datasets produced by the tool.
***Container Image:** Specifies the Docker image that should be used to run the tool.
***Command:** Specifies the command line that is executed inside the docker container.
As we saw in Episode 2, we can deploy our tools to the NDIP platform by adding XML files to the galaxy-tools repository's prototype branch. Let's discuss what happens after that initial deployment.
You can find numerous examples of Galaxy tool XML files in the NDIP GitLab repository:
Once your tool is deployed to the platform, you may want to continue development to fix bugs or add new features. The process for continuing development on an existing tool is similar to getting a new tool on the platform.
## Development Lifecycle
You will continue to develop on the *prototype* branch, where you can push and test changes. Once you are satisfied with the new version of your tool, submit a merge request to update the tool in the *dev* branch. The NDIP team will review these changes, perform the merge, and the new version of the tool will be updated on the NDIP production instance, Calvera, during the next deployment.
### How to get a new tool on NDIP
### Versioning Your Tools
After creating your tool\'s XML file, it needs to be added to the NDIP platform. For testing your tool on the platform, it should be added to the *prototype* branch of the GitLab repository linked above in a repository folder `tools/neutrons`, or it's subfolder. With a git commit, an automated CI job will push your tool to the calvera-test instance. Test your application via the web browser interface. After you\'ve verified that your tool is performing as expected, submit a merge request to the repository\'s *dev* branch and engage with our team.
As you continue to develop your tool, it's important to keep track of versions. The XML file we created in Episode 2 includes a version attribute that you should update whenever you make significant changes to your tool.
The *dev* branch is used as a staging branch for tools that are ready to be put in front of users. Tools here will be added to the NDIP production instance, Calvera, during the next deployment.
### Continued Development
The process for continuing development on an existing tool is similar to getting a new tool on the platform. You will continue to develop on the *prototype* branch, where you can push and test changes. Once you are satisified with the new version of your tool, submit a merge request to update the tool in the *dev* branch. Our team will review these changes, perform the merge, and the new version of the tool will be updated on the NDIP production instance, Calvera, during the next deployment.
## Putting It All Together
## Future Tool Enhancements
Once you have your Docker container and tool XML file, you would:
As you become more familiar with NDIP and the Galaxy platform, you might want to explore more advanced features:
1. Upload the Docker image to a container registry.
2. Upload the XML file to the NDIP platform.
3. Make sure that the API_KEY and GALAXY_URL are passed to the application as environmental variables.
4. Test your application via the web browser interface.
- Creating complex workflows that combine multiple tools
- Integrating with high-performance computing resources
- Developing specialized data analysis pipelines
After performing these steps, your application will be available to NDIP users.
These topics are beyond the scope of this introductory tutorial, but the NDIP team is available to help you explore these possibilities as your tools mature.
## Additional Resources
@@ -90,10 +57,11 @@ After performing these steps, your application will be available to NDIP users.
By following the steps outlined in this section, you can deploy your own applications to the NDIP platform and make them available to the wider scientific community.
By following this tutorial, you've learned how to create and deploy a NOVA application to the NDIP platform. You can now build on this foundation to create more complex scientific applications that can be easily shared with the wider scientific community.
