From b0ef01a315c82f122110bea79905de62a34760df Mon Sep 17 00:00:00 2001 From: "Ayres, Andrew" Date: Fri, 28 Feb 2025 10:39:20 -0600 Subject: [PATCH 01/19] Add pushing tool to ndip --- episodes/02-Copy-Template.md | 136 ++++++++++++++++++++++++++++++++++- episodes/08-Next-Steps.md | 72 ++++++------------- 2 files changed, 155 insertions(+), 53 deletions(-) diff --git a/episodes/02-Copy-Template.md b/episodes/02-Copy-Template.md index 4db170b5..40a7f088 100755 --- a/episodes/02-Copy-Template.md +++ b/episodes/02-Copy-Template.md @@ -170,6 +170,137 @@ The template includes a basic GitLab CI configuration file (`.gitlab-ci.yml`). ::::::::::::::::::::::::: :::::::::::::::::::::::::::::::::::::::::::::::::: +## Deploying Your Tool to NDIP + +Now that we have our template application set up, we need to integrate it with the NDIP platform. This involves three main steps: + +1. Building a Docker container for our application +2. Cloning the galaxy-tools repository +3. Creating an XML file that defines our tool and adding it to the repository + +### Building a Docker Container + +The template comes with a Dockerfile that we can use to build a container for our application. Let's do that: + +```bash +cd nova_tutorial +docker build -t nova-tutorial:latest -f dockerfiles/Dockerfile . +``` + +This creates a Docker image named `nova-tutorial` with the tag `latest` that contains our application and all its dependencies. + +::::::::::::::::::::::::::::::::::::::::: callout +Normally, we would push this image to a container registry so that NDIP can access it. However, for this tutorial, we'll skip this step. In a real deployment, you would push the image using a command like `docker push /nova-tutorial:latest`. For the tutorial, we'll use a pre-pushed container. + +When releasing new versions of your tool, you'll want to use versioned tags for your container images, such as `nova-tutorial:1.0.0`, to maintain backward compatibility while allowing for updates. +:::::::::::::::::::::::::::::::::::::::::::::::: + +### Cloning the Galaxy Tools Repository + +To deploy our tool to the NDIP platform, we need to add its XML definition to the galaxy-tools repository. Let's clone the repository and switch to the prototype branch: + +```bash +# Clone the galaxy-tools repository +git clone https://code.ornl.gov/ndip/galaxy-tools.git + +# Navigate to the repository +cd galaxy-tools + +# Check out the prototype branch +git checkout prototype +``` + +### Creating and Adding the Tool XML File + +Now we'll create an XML file that defines our tool for the NDIP platform. We'll place it in the appropriate directory within the galaxy-tools repository: + +```bash +# Create the directory if needed +mkdir -p tools/neutrons/tutorial + +# Create the XML file +touch tools/neutrons/tutorial/_nova_tutorial.xml +``` + +Replace `` with your username to ensure the tool ID is unique. + +Now, edit the file and add the following XML content: + +```xml + + A simple NOVA template application + + ghcr.io/ornl/ndip/nova-tutorial:latest + + + + 8080 + app + + + + $__history_id__ + $__galaxy_url__ + + + + + +``` + +Make sure to replace `` with your actual username in both the `id` and `name` attributes to ensure your tool has a unique identifier. + +Let's break down the key elements of this XML file: + +- `` element with attributes: + - `id`: A unique identifier for your tool (includes your username for uniqueness) + - `tool_type="interactive"`: Specifies that this is an interactive tool + - `name`: A user-friendly name for the tool + - `version`: The tool version + +- ``: Specifies the Docker container to use + +- ``: Defines how users can access the tool + - ``: The port the application is running on + - ``: The URL path to access the application + +- ``: Critical variables passed to the tool + - `HISTORY_ID`: The current Galaxy history ID + - `GALAXY_URL`: The Galaxy server URL + - `API_KEY`: The user's API key for Galaxy + +- ``: The command to run inside the container + +- ``: Documentation for the tool + +### Committing and Pushing Your Changes + +Now that we've created our tool XML file, we need to commit the changes and push them to the prototype branch: + +```bash +# Add the new file to git +git add tools/neutrons/tutorial/_nova_tutorial.xml + +# Commit the changes +git commit -m "Add 's NOVA tutorial tool" + +# Push the changes to the prototype branch +git push origin prototype +``` + +Once your changes are pushed to the prototype branch, an automated CI job will deploy your tool to the calvera-test instance. You can then access your tool through the NDIP web interface at https://calvera-test.ornl.gov. + +::::::::::::::::::::::::::::::::::::::::: callout +In a production environment, when your tool is ready for users, you would create a merge request from the prototype branch to the dev branch. The NDIP team reviews these changes, merges them, and your tool will be deployed to the production instance during the next deployment. +:::::::::::::::::::::::::::::::::::::::::::::::: + ## References * **Nova Documentation**: https://nova-application-development.readthedocs.io/en/latest/ @@ -180,7 +311,10 @@ The template includes a basic GitLab CI configuration file (`.gitlab-ci.yml`). :::::::::::::::::::::::::::::::::::::::: keypoints - Nova provides a template application to help get started developing your application. -- Use the copier tool to set clone the template application. +- Use the copier tool to clone the template application. - Poetry is a project management tool used to install dependencies and manage virtual environments. - The template application includes everything you need to get started such as basic CI, dockerfile, and tests. +- Docker containers package your application and all its dependencies for deployment. +- Galaxy tool XML files define how your tool appears and functions in NDIP. +- Tools are deployed by adding their XML files to the galaxy-tools repository's prototype branch. :::::::::::::::::::::::::::::::::::::::::::::::::: diff --git a/episodes/08-Next-Steps.md b/episodes/08-Next-Steps.md index 420968fa..7999ba8c 100755 --- a/episodes/08-Next-Steps.md +++ b/episodes/08-Next-Steps.md @@ -5,79 +5,46 @@ exercises: 0 --- # 8. Development Cycle and Next Steps -In this section, we will look at other resources you may want to integrate with your application and outline the process for taking an application like the one we\'ve created in this tutorial and deploying it to the NOVA/NDIP platform. While we won\'t actually perform the deployment in this tutorial, we will cover the key steps and resources involved. +In this section, we will look at other resources you may want to integrate with your application. ## ONCat Integration If needed, you can integrate your application with ONCat via [pyoncat](https://pypi.org/project/pyoncat/2.1/). If you need to access non-public information with the API then you will need to use an authenticated client in `pyoncat`. We strongly recommend you email oncat-support@ornl.gov explaining the use case for your ONCat integration as they can advise you on the most appropriate form of authentication for your application and how to set it up. -## Containerizing Your Application +## Advanced Container Configurations -The first step in deploying your application to the NOVA/NDIP platform is to package it as a Docker container. Docker containers provide a lightweight and portable way to package your application and all of its dependencies. This ensures that your application will run consistently across different environments. - -Fortunately, the template application we used in this tutorial already includes a `Dockerfile` that you can use as a starting point. The Dockerfile is a set of instructions that Docker uses to build your container image. - -Here\'s what the Dockerfile typically includes: - -* **Base Image:** Specifies the base operating system and environment for your application. -* **Dependencies:** Describes how to install any required libraries or packages. -* **Application Files:** Defines how to copy your application code into the container. -* **Entrypoint:** Sets the command that is run when the container starts. - -To containerize your application, you would: - -1. Navigate to the top level of your project (where the `dockerfiles` folder is). -2. Run the docker build command in the following format `docker build -t : -f dockerfiles/Dockerfile .` -3. Test your docker container using the command `docker run :`. -4. Push your docker container to a container registry. - -After the docker container is deployed to a registry, it can then be used by the platform. +The basic container configuration we created in Episode 2 works well for most applications, but there are some advanced configurations that may be useful for more complex applications. ::::::::::::::::::::::::::::::::::::::::: callout -GPU acceleration in a container is possible but beyond the scope of this tutorial. Typically, a base container is chosen which already has all of the gpu dependencies installed such as ```regproxy.ornl.gov/hub_proxy/kitware/trame:py3.10-glvnd-2024-12```. The team has built similar containers already which can be used as a reference for development, such as, ```https://code.ornl.gov/ndip/trame-apps/ct-scan-visualizer/``` +GPU acceleration in a container is possible but beyond the scope of this tutorial. Typically, a base container is chosen which already has all of the GPU dependencies installed such as ```regproxy.ornl.gov/hub_proxy/kitware/trame:py3.10-glvnd-2024-12```. The team has built similar containers already which can be used as a reference for development, such as ```https://code.ornl.gov/ndip/trame-apps/ct-scan-visualizer/``` :::::::::::::::::::::::::::::::::::::::::::::::::: -## Defining Your Tool with XML - -Once your application is containerized, you will also need to define your tool using an XML file. This XML file describes your tool to the NDIP platform, including its inputs, outputs, parameters, and the Docker container image that should be used to run the tool. The NDIP platform makes use of the Galaxy tool framework. -The XML file includes: +## Development Lifecycle -* **Tool ID:** A unique identifier for your tool. -* **Name and Description:** User-friendly name and description of the tool. -* **Inputs:** Defines the input parameters, including their types, labels, and optional constraints. -* **Outputs:** Describes the output files or datasets produced by the tool. -* **Container Image:** Specifies the Docker image that should be used to run the tool. -* **Command:** Specifies the command line that is executed inside the docker container. +As we saw in Episode 2, we can deploy our tools to the NDIP platform by adding XML files to the galaxy-tools repository's prototype branch. Let's discuss what happens after that initial deployment. -You can find numerous examples of Galaxy tool XML files in the NDIP GitLab repository: -[https://code.ornl.gov/ndip/galaxy-tools](https://code.ornl.gov/ndip/galaxy-tools) +### Continued Development -Detailed documentation on creating tool XML files is available on the Calvera documentation site: -[https://calvera.ornl.gov/docs/dev](https://calvera.ornl.gov/docs/dev) +Once your tool is deployed to the platform, you may want to continue development to fix bugs or add new features. The process for continuing development on an existing tool is similar to getting a new tool on the platform. -## Development Lifecycle +You will continue to develop on the *prototype* branch, where you can push and test changes. Once you are satisfied with the new version of your tool, submit a merge request to update the tool in the *dev* branch. The NDIP team will review these changes, perform the merge, and the new version of the tool will be updated on the NDIP production instance, Calvera, during the next deployment. -### How to get a new tool on NDIP +### Versioning Your Tools -After creating your tool\'s XML file, it needs to be added to the NDIP platform. For testing your tool on the platform, it should be added to the *prototype* branch of the GitLab repository linked above in a repository folder `tools/neutrons`, or it's subfolder. With a git commit, an automated CI job will push your tool to the calvera-test instance. Test your application via the web browser interface. After you\'ve verified that your tool is performing as expected, submit a merge request to the repository\'s *dev* branch and engage with our team. +As you continue to develop your tool, it's important to keep track of versions. The XML file we created in Episode 2 includes a version attribute that you should update whenever you make significant changes to your tool. The *dev* branch is used as a staging branch for tools that are ready to be put in front of users. Tools here will be added to the NDIP production instance, Calvera, during the next deployment. -### Continued Development - -The process for continuing development on an existing tool is similar to getting a new tool on the platform. You will continue to develop on the *prototype* branch, where you can push and test changes. Once you are satisified with the new version of your tool, submit a merge request to update the tool in the *dev* branch. Our team will review these changes, perform the merge, and the new version of the tool will be updated on the NDIP production instance, Calvera, during the next deployment. - -## Putting It All Together +## Future Tool Enhancements -Once you have your Docker container and tool XML file, you would: +As you become more familiar with NDIP and the Galaxy platform, you might want to explore more advanced features: -1. Upload the Docker image to a container registry. -2. Upload the XML file to the NDIP platform. -3. Make sure that the API_KEY and GALAXY_URL are passed to the application as environmental variables. -4. Test your application via the web browser interface. +- Creating complex workflows that combine multiple tools +- Integrating with high-performance computing resources +- Developing specialized data analysis pipelines -After performing these steps, your application will be available to NDIP users. +These topics are beyond the scope of this introductory tutorial, but the NDIP team is available to help you explore these possibilities as your tools mature. ## Additional Resources @@ -90,10 +57,11 @@ After performing these steps, your application will be available to NDIP users. * **nova-mvvm documentation**: https://nova-application-development.readthedocs.io/projects/mvvm-lib/en/latest/ * **Calvera documentation**: https://calvera-test.ornl.gov/docs/ -By following the steps outlined in this section, you can deploy your own applications to the NDIP platform and make them available to the wider scientific community. +By following this tutorial, you've learned how to create and deploy a NOVA application to the NDIP platform. You can now build on this foundation to create more complex scientific applications that can be easily shared with the wider scientific community. :::::::::::::::::::::::::::::::::::::::: keypoints - Tools must be containerized to run on NDIP. -- NDIP requires tools to have an XML file which defines input, outputs, tool id, and the container location. +- NDIP requires tools to have an XML file which defines inputs, outputs, tool ID, and the container location. - Tool XML files must be added to the Galaxy Tools Repository. +- The development lifecycle involves continuous testing on the prototype branch before promoting to dev. :::::::::::::::::::::::::::::::::::::::::::::::::: -- GitLab From 7d981487c47b1745eed9cb320bd6a7b79e262191 Mon Sep 17 00:00:00 2001 From: "Ayres, Andrew" Date: Fri, 28 Feb 2025 10:44:23 -0600 Subject: [PATCH 02/19] Update ci to push preview --- .gitlab-ci.yml | 22 +++++++++++++++++++++- 1 file changed, 21 insertions(+), 1 deletion(-) mode change 100644 => 100755 .gitlab-ci.yml diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml old mode 100644 new mode 100755 index f59a678b..5dbe858c --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -46,6 +46,26 @@ pages: - if: $CI_COMMIT_REF_NAME != $CI_DEFAULT_BRANCH when: manual - when: on_success - tags: - rse-multi-builder + +preview: + stage: deploy + script: + - docker build -t tutorial -f dockerfiles/Dockerfile . + - mkdir public + - docker run -v `pwd`/public:/app/site tutorial Rscript -e "sandpaper::build_lesson()" + - cp $CALVERA_DOCS_SSH_KEY key && chmod 600 key + - cat key + - rsync -avz -e "ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i key" public/docs/ cloud@10.64.195.250:/home/cloud/nova-tutorial/tutorial/Preview + artifacts: + paths: + - public + rules: + - if: $CI_PIPELINE_SOURCE == "merge_request_event" + when: never + - if: $CI_COMMIT_BRANCH == "Preview" + when: manual + - when: on_success + tags: + - rse-multi-builder \ No newline at end of file -- GitLab From 5cea110cda3b5122da53808376f0ef42deef23b3 Mon Sep 17 00:00:00 2001 From: "Ayres, Andrew" Date: Fri, 28 Feb 2025 12:00:32 -0600 Subject: [PATCH 03/19] Small fixes --- code/episode_4/src/nova_tutorial/app/main.py | 1 - episodes/05-Working-with-Trame.md | 16 ++++++++++++++++ episodes/07-Advanced-Visualizations.md | 4 ++-- learners/setup.md | 3 ++- 4 files changed, 20 insertions(+), 4 deletions(-) diff --git a/code/episode_4/src/nova_tutorial/app/main.py b/code/episode_4/src/nova_tutorial/app/main.py index add49c11..2977dbe1 100755 --- a/code/episode_4/src/nova_tutorial/app/main.py +++ b/code/episode_4/src/nova_tutorial/app/main.py @@ -1,7 +1,6 @@ """Main Application.""" import sys -from .models.fractal import Fractal def main() -> None: diff --git a/episodes/05-Working-with-Trame.md b/episodes/05-Working-with-Trame.md index fb210aac..5c5bcfce 100755 --- a/episodes/05-Working-with-Trame.md +++ b/episodes/05-Working-with-Trame.md @@ -78,6 +78,22 @@ Layouts are responsible for arraging your content in a consistent manner. In Tra **1. `src/nova_tutorial/app/views/main.py` (Modify):** ```python +import logging + +from nova.mvvm.trame_binding import TrameBinding +from nova.trame import ThemedApp +from nova.trame.view import layouts +from trame.app import get_server +from trame.widgets import vuetify3 as vuetify + +from ..mvvm_factory import create_viewmodels +from ..view_models.main import MainViewModel +from .tab_content_panel import TabContentPanel +from .tabs_panel import TabsPanel + +logger = logging.getLogger(__name__) +logger.setLevel(logging.INFO) + class MainApp(ThemedApp): """Main application view class. Calls rendering of nested UI elements.""" diff --git a/episodes/07-Advanced-Visualizations.md b/episodes/07-Advanced-Visualizations.md index 58ff0036..c9bb217b 100755 --- a/episodes/07-Advanced-Visualizations.md +++ b/episodes/07-Advanced-Visualizations.md @@ -151,7 +151,7 @@ class PlotlyView: As with our previous examples, there is a corresponding model. -**2. `PlotlyConfig` Model Class (src/viz_examples/models/plotly.py) (Create):** +**2. `PlotlyConfig` Model Class (src/viz_examples/app/models/plotly.py) (Create):** * **Imports**: The graph_objects module is how we will define the content for our chart. The iris module defines an example dataset. @@ -218,7 +218,7 @@ class PlotlyConfig(BaseModel): First, let's add replace the sample tabs from the template with the following: -**3. `src/viz_examples/views/tab_content_panel.py` (Modify):** +**3. `src/app/viz_examples/app/views/tab_content_panel.py` (Modify):** * **Import `PlotlyView`** diff --git a/learners/setup.md b/learners/setup.md index c988ad64..3bf0b707 100755 --- a/learners/setup.md +++ b/learners/setup.md @@ -10,12 +10,13 @@ This section guides you through setting up your development environment to follo Before proceeding, ensure you have met the following prerequisites: * **Basic Python Knowledge:** A basic understanding of Python programming concepts is required. -* **Python Installation:** You must have Python 3.8 or higher installed on your system. Verify your Python version by running `python --version` or `python3 --version` in your terminal. +* **Python Installation:** You must have Python 3.10 or higher installed on your system. Verify your Python version by running `python --version` or `python3 --version` in your terminal. * **Python's `copier` Library:** We will be using this library to generate a starting application from a template. Install it using `pip install copier`. * **Poetry:** The code samples provided in this tutorial leverage Poetry for dependency management. If you don't have Poetry installed, follow the instructions on the official Poetry website: [https://python-poetry.org/docs/#installation](https://python-poetry.org/docs/#installation). * **NDIP Access:** You should have access to a working NDIP system. (Specific details about NDIP access will need to be provided by the instructor.) Ensure you have the necessary credentials (API key and NDIP URL) to connect to NDIP. These will be set as environment variables, as described later. * **NOVA Libraries:** While Poetry will manage these dependencies for you, it's helpful to be aware of the core NOVA libraries: `nova-galaxy`, `nova-trame`, and `nova-mvvm`. You can find documentation for these libraries on ReadTheDocs (links provided in the "References" section). * **A Text Editor or IDE:** You will need a text editor or IDE (such as VS Code, Sublime Text, or Atom) for writing code. +* **Git:** You must have git installed on your system. * **Familiarity with the Command Line:** You will need to be comfortable using the command line or terminal. ## 2. Getting your Galaxy API Key -- GitLab From c54422af0f0a72de564884bdacc85e8c554f368a Mon Sep 17 00:00:00 2001 From: "Ayres, Andrew" Date: Fri, 7 Mar 2025 02:29:25 -0600 Subject: [PATCH 04/19] Reorder 4 and fix remotefile in 5 --- .../nova_tutorial/app/views/sample_tab_1.py | 2 +- episodes/04-MVVM-Design-Pattern.md | 202 +++++++++++------- episodes/05-Working-with-Trame.md | 27 ++- 3 files changed, 152 insertions(+), 79 deletions(-) diff --git a/code/episode_5/src/nova_tutorial/app/views/sample_tab_1.py b/code/episode_5/src/nova_tutorial/app/views/sample_tab_1.py index ccdfc4e2..5f5fd62c 100755 --- a/code/episode_5/src/nova_tutorial/app/views/sample_tab_1.py +++ b/code/episode_5/src/nova_tutorial/app/views/sample_tab_1.py @@ -10,5 +10,5 @@ class SampleTab1: self.create_ui() def create_ui(self) -> None: - RemoteFileInput(v_model="file", base_paths=["/HFIR", "/SNS"]) + RemoteFileInput(v_model="config.file", base_paths=["/HFIR", "/SNS"]) InputField(v_model="config.username") diff --git a/episodes/04-MVVM-Design-Pattern.md b/episodes/04-MVVM-Design-Pattern.md index bc7714a6..16d34c69 100755 --- a/episodes/04-MVVM-Design-Pattern.md +++ b/episodes/04-MVVM-Design-Pattern.md @@ -142,78 +142,77 @@ Finally, we connect a UI component to the connector object. The template applica InputField(v_model="config.username") ``` -## Implementing MVVM with `nova-mvvm` and Pydantic +## Project Structure -Let\'s see how to implement the MVVM pattern using `nova-mvvm` and incorporate Pydantic for data validation. +The template creates a well-organized project structure following best practices, including the Model-View-ViewModel (MVVM) design pattern. This structure promotes code maintainability, testability, and separation of concerns. Here's a breakdown of the key directories and files: -**1. Adding Fractal to the ViewModel (`src/nova_tutorial/app/view_models/main.py`) (Modify):** +* `nova_tutorial/`: The root directory of your project. This is the top-level directory containing all project files and subdirectories. -* **Running our Model**: We start by adding a method to bottom of our ViewModel which will run the Fractal tool. +* `nova_tutorial/src/`: This directory contains all the source code for your application. The separation into `src` helps distinguish your application code from configuration files, tests, and other project-related files that reside in the root. -```python - def run_fractal(self) -> None: - self.model.fractal.run_fractal_tool() - self.update_view() -``` +* `nova_tutorial/src/nova_tutorial/`: This is the main Python package for your application. Its name (`nova_tutorial` in this case) is used when importing modules within your project. Inside this directory, you'll find the core application logic, organized according to the MVVM pattern: -**2. Updating our Fractal Class for pydantaic and MVVM (`src/nova_tutorial/app/models/fractal.py`) (Modify)** + * `nova_tutorial/src/nova_tutorial/app/`: This directory contains the main application logic, further subdivided to reflect the MVVM structure. -* **Adding new imports**: We need to add some imports for pydantic and working with base64 encodings to deal with the image. Modify your import block to match below. + * `nova_tutorial/src/nova_tutorial/app/models/`: **(Model)** This is where you define your data models and business logic. These classes represent the data your application works with and the rules for manipulating that data. -```python -import os -from base64 import b64encode -from typing import Literal + * `nova_tutorial/src/nova_tutorial/app/view_models/`: **(ViewModel)** This directory holds the ViewModels. These classes act as intermediaries between the Models and the Views. They prepare data for display and handle user interactions from the View. -from pydantic import BaseModel, Field -from nova.galaxy import Connection, Parameters, Tool -``` + * `nova_tutorial/src/nova_tutorial/app/views/`: **(View)** This directory contains the user interface (UI) components. These are built using Trame and Vuetify (via `nova-trame`). They are responsible for displaying data and capturing user input. -* **Update class variables:** Now we will update fractal_type and other class variables to support pydantic. We will also add an image variable to store the image. Modify the variable declarations to the following: + * `nova_tutorial/src/nova_tutorial/app/main.py`: The entry point for your NOVA application. This file initializes and starts the Trame server and the `MainApp` view. -```python -class Fractal(BaseModel): - fractal_type: Literal["mandelbrot", "julia", "random", "markus"] = Field(default="mandelbrot") - galaxy_url: str = Field(default_factory=lambda: os.getenv("GALAXY_URL"), description="NDIP Galaxy URL") - galaxy_key: str = Field(default_factory=lambda: os.getenv("GALAXY_API_KEY"), description="NDIP Galaxy API Key") - image_data: str = Field(default="", description="Base64 encoded PNG") +* `nova_tutorial/tests/`: Contains unit tests for your application. A well-structured project should include tests to ensure code quality and prevent regressions. The tests are typically organized to mirror the structure of your application code (e.g., tests for models, view models, and potentially UI components). - def set_fractal_type(self, fractal_type: str): - self.fractal_type = fractal_type -``` +* `nova_tutorial/README.md`: A Markdown file providing a description of your project, instructions for setup and usage, and any other relevant information. -* **Decode the image data:** Finally, we need to decode the image that we receive as the output from the tool execution. Modify the section where we execute the tool to the following: +* `pyproject.toml`: A configuration file for Poetry, the dependency management and packaging tool used by NOVA. It specifies project dependencies, build settings, and other metadata. -```python - output.get_dataset("output").download("tmp.png") +## Implementing MVVM with `nova-mvvm` and Pydantic - with open("tmp.png", "rb") as image_file: - self.image_data = f"data:image/png;base64,{b64encode(image_file.read()).decode()}" -``` +Let's implement the MVVM pattern, starting with the UI and basic ViewModel connections, and then building up the Model functionality. -**3. Updating our MainModel Class to add the new Fractal Class (`src/nova_tutorial/app/models/main_model.py`) (Modify):** +**1. Initial Setup and ViewModel Basics** -* **Add Fractal to imports**: Add an import for the Fractal class into our MainModel. +First, let's simplify our `main.py` and set up the basic structure of our UI and ViewModel interaction. We'll create a button in the UI that, when clicked, will eventually run our Fractal tool. For now, it will just trigger a placeholder method in the ViewModel. -```python -from .fractal import Fractal # Import Fractal +* **`main.py` - Simplifying the Application Entry Point (`src/nova_tutorial/app/main.py`) (Modify):** + +We're removing the direct Fractal tool execution from `main()`. The application will now solely focus on launching the NOVA app. + + ```python +import sys + +def main() -> None: + kwargs = {} + from .views.main import MainApp + + app = MainApp() + for arg in sys.argv[2:]: + try: + key, value = arg.split("=") + kwargs[key] = int(value) + except Exception: + pass + app.server.start(**kwargs) ``` -* **Add the Fractal Model to the MainModel**: Modify the end of the MainModel class so that it matches the code below. +* **Adding a Placeholder Method to the ViewModel (`src/nova_tutorial/app/view_models/main.py`) (Modify):** + +Add a `run_fractal` method to the `MainViewModel`. For now, it just prints a message to the console. This confirms that the button click is connected to the ViewModel. ```python - password: str = Field(default="test_password", title="User Password") - fractal: Fractal = Field(default_factory=Fractal) #Add Fractal Model + def run_fractal(self) -> None: + print("run_fractal method called!") ``` -**4. Creating a FractalTab (`src/nova_tutorial/app/views/fractal_tab.py`) (Create):** +* **Creating a FractalTab (`src/nova_tutorial/app/views/fractal_tab.py`) (Create):** -* **Create a fractal tab**: Create a new file and add the following code: +This is the UI for our Fractal interaction. It includes a button that calls the `run_fractal` method in the ViewModel. We don't have image display yet. ```python from trame.widgets import vuetify3 as vuetify -from nova.trame.view.components import InputField from nova_tutorial.app.view_models.main import MainViewModel class FractalTab: @@ -225,14 +224,13 @@ class FractalTab: InputField(v_model="config.fractal.fractal_type") vuetify.VBtn( "Run Fractal", - click=self.view_model.run_fractal # calls the run_fractal_tool method - ) - vuetify.VImg(src=("config.fractal.image_data",), height="400", width="400") + click=self.view_model.run_fractal + ) ``` -**5. Modify the tab panel (`src/nova_tutorial/app/views/tabs_panel.py`) (Modify):** +* **Modify the tab panel (`src/nova_tutorial/app/views/tabs_panel.py`) (Modify):** -* **Add Fractal Tab to the tab panel**: Modify the tab panel to add our new Fractal tab +Add the "Fractal" tab to the tab bar. ```python with vuetify.VTabs(v_model=("active_tab", 0), classes="pl-5"): @@ -241,17 +239,14 @@ class FractalTab: vuetify.VTab("Sample Tab 2", value=3) ``` -**6. Modify the tab panel content (`src/nova_tutorial/app/views/tab_content_panel.py`) (Modify):** +* **Modify the tab panel content (`src/nova_tutorial/app/views/tab_content_panel.py`) (Modify):** -* **Add FractalTab to imports**: Import the newly created FractalTab class into our tab_content_panel. +Display the `FractalTab` content when the "Fractal" tab is selected. ```python from .fractal_tab import FractalTab # Import the FractalTab -``` - -* **Add the Fractal Tab to our existing tabs**: Add the Fractal Tab lines to the vuetify.VWindow section and modify the values. -```python + # ... (rest of the file) ... with vuetify.VWindow(v_model="active_tab"): with vuetify.VWindowItem(value=1): FractalTab(self.view_model) # Add FractalTab @@ -261,40 +256,95 @@ from .fractal_tab import FractalTab # Import the FractalTab SampleTab2() ``` -**7. `main.py` - Calling the Model (`src/nova_tutorial/app/main.py`) (Modify):** +**Demonstration (Initial UI and ViewModel Connection):** + +Run the application: `poetry run app` + +You should see a new "Fractal" tab in the application. Click the "Run Fractal" button. You should see "run_fractal method called!" printed in your terminal. This demonstrates that the button click in the View is successfully triggering the `run_fractal` method in the ViewModel, even though the method doesn't do anything substantial yet. This establishes the basic MVVM wiring. + +**2. Fractal Model and Pydantic Integration** + +Now, let's build out the `Fractal` model using Pydantic and integrate it into our `MainModel`. -We are now going to modify the existing `main.py` file. Change the main method to match the code below. +* **Updating our Fractal Class for pydantaic and MVVM (`src/nova_tutorial/app/models/fractal.py`) (Modify)** -* **Instantiate and Run**: In the `main()` function, we no longer need to setup the Fractal tool as it's managed via our MVVM application now. +* **Adding new imports**: Add imports for Pydantic and base64 handling. ```python -import sys -from .models.fractal import Fractal +import os +from base64 import b64encode +from typing import Literal +from pydantic import BaseModel, Field +from nova.galaxy import Connection, Parameters, Tool + ``` -def main() -> None: - kwargs = {} - from .views.main import MainApp +* **Update class variables:** Use Pydantic's `Field` for type hinting and validation. Add the `image_data` field. - app = MainApp() - for arg in sys.argv[2:]: - try: - key, value = arg.split("=") - kwargs[key] = int(value) - except Exception: - pass - app.server.start(**kwargs) +```python +class Fractal(BaseModel): + fractal_type: Literal["mandelbrot", "julia", "random", "markus"] = Field(default="mandelbrot") + galaxy_url: str = Field(default_factory=lambda: os.getenv("GALAXY_URL"), description="NDIP Galaxy URL") + galaxy_key: str = Field(default_factory=lambda: os.getenv("GALAXY_API_KEY"), description="NDIP Galaxy API Key") + image_data: str = Field(default="", description="Base64 encoded PNG") + + def set_fractal_type(self, fractal_type: str): + self.fractal_type = fractal_type ``` -## Running the application +* **Decode the data:** Update how the image is decoded. +```python + output.get_dataset("output").download("tmp.png") -To run the code, use the following command in the top level of your `nova_tutorial` project: + with open("tmp.png", "rb") as image_file: + self.image_data = f"data:image/png;base64,{b64encode(image_file.read()).decode()}" +``` + +* **Updating our MainModel Class to add the new Fractal Class (`src/nova_tutorial/app/models/main_model.py`) (Modify):** + +Import and include the `Fractal` model as a field in the `MainModel`. + +```python + from .fractal import Fractal # Import Fractal -```bash -poetry run app +class MainModel(BaseModel): + # ... (other fields) ... + password: str = Field(default="test_password", title="User Password") + fractal: Fractal = Field(default_factory=Fractal) #Add Fractal Model ``` -The application should launch a tab in your web browser. The GUI will have a `FRACTAL` tab and a few sample tabs which were created by the template application. The run button on the `Fractal` tab can be used to launch the `Fractal` NDIP tool. The tool will take a few minutes to complete but when it does, the resulting `Fractal` image will be displayed. +* **Connect the UI elements in FractalTab (`src/nova_tutorial/app/views/fractal_tab.py`) (Modify):** + +Update the create UI section to use InputField and the image. +```python +from nova.trame.view.components import InputField + + # ...(rest of file)... + def create_ui(self) -> None: + InputField(v_model="config.fractal.fractal_type") + vuetify.VBtn( + "Run Fractal", + click=self.view_model.run_fractal + ) + vuetify.VImg(src=("config.fractal.image_data",), height="400", width="400") +``` + +* **Add Full Functionality to the View Model (`src/nova_tutorial/app/view_models/main.py`) (Modify)** +Update the code in the run_fractal method. + +```python + def run_fractal(self) -> None: + self.model.fractal.run_fractal_tool() + self.update_view() +``` + +**Final Demonstration (Full Application):** + +Run the application: `poetry run app` + +Now, when you click "Run Fractal," the Fractal tool will execute in Galaxy, and the resulting image will be displayed in the UI. You can also change the `fractal_type` using the input field. This demonstrates the complete MVVM flow, with data binding, Pydantic validation, and the interaction between the View, ViewModel, and Model. + +This revised structure breaks down the implementation into smaller, more manageable steps, with demonstrations after each stage to show the progress and confirm that each part is working as expected. This addresses the feedback about making too many code changes at once and improves the learning experience. ::::::::::::::::::::::::::::::::::::::::: callout If you don't want Trame to launch a tab by default, you can instead run ```poetry run app --server```. diff --git a/episodes/05-Working-with-Trame.md b/episodes/05-Working-with-Trame.md index 5c5bcfce..e4cb56d9 100755 --- a/episodes/05-Working-with-Trame.md +++ b/episodes/05-Working-with-Trame.md @@ -200,10 +200,33 @@ class SampleTab1: self.create_ui() def create_ui(self) -> None: - RemoteFileInput(v_model="file", base_paths=["/HFIR", "/SNS"]) + RemoteFileInput(v_model="config.file", base_paths=["/HFIR", "/SNS"]) InputField(v_model="config.username") ``` +**6. `src/nova_tutorial/app/models/main_model.py` (Modify):** + +Add a `file` field to the `MainModel` to store the selected file path. We use `Optional[str]` because initially, no file will be selected. + +```python +from pydantic import BaseModel, Field +from .fractal import Fractal + + +class MainModel(BaseModel): + username: str = Field( + default="test_name", + min_length=1, + title="User Name", + description="Please provide the name of the user", + examples=["user"], + ) + password: str = Field(default="test_password", title="User Password") + file: str = Field(default="", title="Select a File") + fractal: Fractal = Field(default_factory=Fractal) +``` + + :::::::::::::::::::::::::::::::: callout If you want to connect your application to the analysis cluster, then it will need to be run on a computer where the filesystem is mounted. If your application is deployed through our platform, then we can ensure that your application runs in the correct environment to support your needs. @@ -268,7 +291,7 @@ By combining these layout components, you can create complex and responsive UI l As an example, we can use the layout classes to center the "Run Fractal" button. -**6. `src/nova_tutorial/app/views/main.py` (Modify):** +**7. `src/nova_tutorial/app/views/main.py` (Modify):** ```python from nova.trame.view import layouts -- GitLab From de8f0583992882bb9bf6468bb5121fe8c075925e Mon Sep 17 00:00:00 2001 From: "Ayres, Andrew" Date: Fri, 7 Mar 2025 02:42:35 -0600 Subject: [PATCH 05/19] Minor styling --- episodes/04-MVVM-Design-Pattern.md | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/episodes/04-MVVM-Design-Pattern.md b/episodes/04-MVVM-Design-Pattern.md index 16d34c69..06ba11d5 100755 --- a/episodes/04-MVVM-Design-Pattern.md +++ b/episodes/04-MVVM-Design-Pattern.md @@ -305,7 +305,7 @@ class Fractal(BaseModel): Import and include the `Fractal` model as a field in the `MainModel`. ```python - from .fractal import Fractal # Import Fractal +from .fractal import Fractal # Import Fractal class MainModel(BaseModel): # ... (other fields) ... @@ -344,8 +344,6 @@ Run the application: `poetry run app` Now, when you click "Run Fractal," the Fractal tool will execute in Galaxy, and the resulting image will be displayed in the UI. You can also change the `fractal_type` using the input field. This demonstrates the complete MVVM flow, with data binding, Pydantic validation, and the interaction between the View, ViewModel, and Model. -This revised structure breaks down the implementation into smaller, more manageable steps, with demonstrations after each stage to show the progress and confirm that each part is working as expected. This addresses the feedback about making too many code changes at once and improves the learning experience. - ::::::::::::::::::::::::::::::::::::::::: callout If you don't want Trame to launch a tab by default, you can instead run ```poetry run app --server```. -- GitLab From 46d709de82c3c7009a5653bed72dc25954b6093a Mon Sep 17 00:00:00 2001 From: "Ayres, Andrew" Date: Fri, 7 Mar 2025 02:52:07 -0600 Subject: [PATCH 06/19] Git command fixes --- episodes/02-Copy-Template.md | 9 +++------ 1 file changed, 3 insertions(+), 6 deletions(-) diff --git a/episodes/02-Copy-Template.md b/episodes/02-Copy-Template.md index 40a7f088..17ddbd6c 100755 --- a/episodes/02-Copy-Template.md +++ b/episodes/02-Copy-Template.md @@ -190,7 +190,7 @@ docker build -t nova-tutorial:latest -f dockerfiles/Dockerfile . This creates a Docker image named `nova-tutorial` with the tag `latest` that contains our application and all its dependencies. ::::::::::::::::::::::::::::::::::::::::: callout -Normally, we would push this image to a container registry so that NDIP can access it. However, for this tutorial, we'll skip this step. In a real deployment, you would push the image using a command like `docker push /nova-tutorial:latest`. For the tutorial, we'll use a pre-pushed container. +Normally, we would push this image to a container registry so that NDIP can access it. However, for this tutorial, we\'ll skip this step. In a real deployment, you would push the image using a command like `docker push /nova-tutorial:latest`. For the tutorial, we\'ll use a pre-pushed container. When releasing new versions of your tool, you'll want to use versioned tags for your container images, such as `nova-tutorial:1.0.0`, to maintain backward compatibility while allowing for updates. :::::::::::::::::::::::::::::::::::::::::::::::: @@ -215,9 +215,6 @@ git checkout prototype Now we'll create an XML file that defines our tool for the NDIP platform. We'll place it in the appropriate directory within the galaxy-tools repository: ```bash -# Create the directory if needed -mkdir -p tools/neutrons/tutorial - # Create the XML file touch tools/neutrons/tutorial/_nova_tutorial.xml ``` @@ -230,7 +227,7 @@ Now, edit the file and add the following XML content: A simple NOVA template application - ghcr.io/ornl/ndip/nova-tutorial:latest + savannah.ornl.gov/ndip/nova-tutorial:latest @@ -292,7 +289,7 @@ git add tools/neutrons/tutorial/_nova_tutorial.xml git commit -m "Add 's NOVA tutorial tool" # Push the changes to the prototype branch -git push origin prototype +git push ``` Once your changes are pushed to the prototype branch, an automated CI job will deploy your tool to the calvera-test instance. You can then access your tool through the NDIP web interface at https://calvera-test.ornl.gov. -- GitLab From 0211b5000368fd1133b248b959f8d543de2d7cff Mon Sep 17 00:00:00 2001 From: John Duggan Date: Fri, 7 Mar 2025 15:38:28 -0500 Subject: [PATCH 07/19] Add link to Vuetify docs in Trame episode after first use of classes --- episodes/05-Working-with-Trame.md | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/episodes/05-Working-with-Trame.md b/episodes/05-Working-with-Trame.md index fb210aac..58449cb6 100755 --- a/episodes/05-Working-with-Trame.md +++ b/episodes/05-Working-with-Trame.md @@ -267,6 +267,14 @@ from nova.trame.view import layouts ) ``` +:::::::::::::::::::::::::::::::::::::::: callout + +In the above example, we use the `classes` parameter to `HBoxLayout` to add the `my-2` CSS class to the element. This parameter can be used on any Trame component to customize your interface's appearance without having to write CSS. + +The `my-2` class is provided by [Vuetify](https://vuetifyjs.com/) and gives the element vertical margin (space above and below the element). https://vuetifyjs.com/en/styles/spacing documents this class and other classes related to spacing. There are also many other pages on the Vuetify docs describing classes that together give you a wide range of options for customizing your interface. + +:::::::::::::::::::::::::::::::::::::::::::::::: + For a more detailed explanation of how to work with our layout and theme, please refer to the [`nova-trame documentation`](https://nova-application-development.readthedocs.io/projects/nova-trame/en/stable/working_with_trame.html). ## Running the application -- GitLab From f8fa635b779b35604f4048d8f0734d5651eda60a Mon Sep 17 00:00:00 2001 From: "Ayres, Andrew" Date: Fri, 14 Mar 2025 09:59:09 -0500 Subject: [PATCH 08/19] Update episode 2 git procedure --- episodes/02-Copy-Template.md | 144 +++++++++++++---------------------- 1 file changed, 55 insertions(+), 89 deletions(-) diff --git a/episodes/02-Copy-Template.md b/episodes/02-Copy-Template.md index 17ddbd6c..c75cef81 100755 --- a/episodes/02-Copy-Template.md +++ b/episodes/02-Copy-Template.md @@ -172,127 +172,93 @@ The template includes a basic GitLab CI configuration file (`.gitlab-ci.yml`). ## Deploying Your Tool to NDIP -Now that we have our template application set up, we need to integrate it with the NDIP platform. This involves three main steps: +Now that we have our template application set up, we need to integrate it with the NDIP platform. The template now includes built-in utilities to streamline this process, handling the GitLab repository setup and Galaxy tool XML management automatically. -1. Building a Docker container for our application -2. Cloning the galaxy-tools repository -3. Creating an XML file that defines our tool and adding it to the repository +### Initialize Your Project Repository -### Building a Docker Container - -The template comes with a Dockerfile that we can use to build a container for our application. Let's do that: +The template comes with a utility script to initialize your Git repository and push it to the correct location in the NDIP GitLab. Run: ```bash cd nova_tutorial -docker build -t nova-tutorial:latest -f dockerfiles/Dockerfile . +poetry run init-repo ``` -This creates a Docker image named `nova-tutorial` with the tag `latest` that contains our application and all its dependencies. +This script will: +1. Initialize a Git repository (if not already done) +2. Set up the remote to point to `https://code.ornl.gov/ndip/tool-sources/tutorial/-nova-tutorial` +3. Add all project files to the repository +4. Create an initial commit (if needed) +5. Push the code to the GitLab repository ::::::::::::::::::::::::::::::::::::::::: callout -Normally, we would push this image to a container registry so that NDIP can access it. However, for this tutorial, we\'ll skip this step. In a real deployment, you would push the image using a command like `docker push /nova-tutorial:latest`. For the tutorial, we\'ll use a pre-pushed container. - -When releasing new versions of your tool, you'll want to use versioned tags for your container images, such as `nova-tutorial:1.0.0`, to maintain backward compatibility while allowing for updates. +The repository path is automatically determined based on your answers during the template setup. For tutorial projects, it will use `tutorial/-project-name`, while for non-tutorial projects it will use the project group you specified. :::::::::::::::::::::::::::::::::::::::::::::::: -### Cloning the Galaxy Tools Repository +### Continuous Integration and Container Building -To deploy our tool to the NDIP platform, we need to add its XML definition to the galaxy-tools repository. Let's clone the repository and switch to the prototype branch: +Once your code is pushed to GitLab, the included CI/CD pipeline will automatically build a Docker container for your application. The CI configuration is already set up in the `.gitlab-ci.yml` file and includes: -```bash -# Clone the galaxy-tools repository -git clone https://code.ornl.gov/ndip/galaxy-tools.git +1. Running tests to verify your code works correctly +2. Building a Docker image containing your application +3. Pushing the image to the NDIP container registry (at `code.ornl.gov:4567/ndip/tool-sources/tutorial/-nova-tutorial`) -# Navigate to the repository -cd galaxy-tools +The Docker image tag is derived from your project's version in `pyproject.toml`. Each time you update the version and push, a new container will be built automatically. -# Check out the prototype branch -git checkout prototype -``` +### Tool XML File -### Creating and Adding the Tool XML File +The template has already generated a Galaxy tool XML file for your project. You can find this file at: -Now we'll create an XML file that defines our tool for the NDIP platform. We'll place it in the appropriate directory within the galaxy-tools repository: - -```bash -# Create the XML file -touch tools/neutrons/tutorial/_nova_tutorial.xml ``` - -Replace `` with your username to ensure the tool ID is unique. - -Now, edit the file and add the following XML content: - -```xml - - A simple NOVA template application - - savannah.ornl.gov/ndip/nova-tutorial:latest - - - - 8080 - app - - - - $__history_id__ - $__galaxy_url__ - - - - - +xml/tool.xml ``` -Make sure to replace `` with your actual username in both the `id` and `name` attributes to ensure your tool has a unique identifier. +This file defines how your tool appears and functions within the NDIP platform. It includes: -Let's break down the key elements of this XML file: +- A unique tool ID based on your username (for tutorial projects) +- The correct container reference pointing to your GitLab repository +- Port and URL configurations for accessing your application +- Environment variables needed by the NDIP platform +- Command to run your application inside the container +- Help and description text for users -- `` element with attributes: - - `id`: A unique identifier for your tool (includes your username for uniqueness) - - `tool_type="interactive"`: Specifies that this is an interactive tool - - `name`: A user-friendly name for the tool - - `version`: The tool version +### Pushing the Tool XML to Galaxy Tools Repository -- ``: Specifies the Docker container to use +To deploy your tool to the NDIP platform, you need to add the XML file to the galaxy-tools repository. The template includes a utility for this: -- ``: Defines how users can access the tool - - ``: The port the application is running on - - ``: The URL path to access the application - -- ``: Critical variables passed to the tool - - `HISTORY_ID`: The current Galaxy history ID - - `GALAXY_URL`: The Galaxy server URL - - `API_KEY`: The user's API key for Galaxy +```bash +poetry run push-xml +``` -- ``: The command to run inside the container +This script will: +1. Clone the Galaxy tools repository +2. Copy your tool XML file to the correct location (`tools/neutrons/tutorials/-nova-tutorial.xml` for tutorial projects) +3. Commit the changes +4. Push to the `prototype` branch of the galaxy-tools repository -- ``: Documentation for the tool +Once your XML file is pushed to the prototype branch, an automated CI job will deploy your tool to the calvera-test instance. You can then access your tool through the NDIP web interface at https://calvera-test.ornl.gov. -### Committing and Pushing Your Changes +::::::::::::::::::::::::::::::::::::::::: callout +The tool XML is automatically pushed to the correct location. For tutorial projects, it goes to `tools/neutrons/tutorials/-project-name.xml`, and for regular projects, it goes to `tools/neutrons/project-name.xml`. +:::::::::::::::::::::::::::::::::::::::::::::::: -Now that we've created our tool XML file, we need to commit the changes and push them to the prototype branch: +### Understanding Your Tool's Integration -```bash -# Add the new file to git -git add tools/neutrons/tutorial/_nova_tutorial.xml +Let's understand the key components that make your tool work in NDIP: -# Commit the changes -git commit -m "Add 's NOVA tutorial tool" +1. **Repository Structure**: + - Your code is hosted at `https://code.ornl.gov/ndip/tool-sources/tutorial/-nova-tutorial` + - The Docker container is built automatically by CI and stored at `code.ornl.gov:4567/ndip/tool-sources/tutorial/-nova-tutorial` -# Push the changes to the prototype branch -git push -``` +2. **Tool XML File**: + - Defines your tool for Galaxy/NDIP + - References your container so NDIP knows which image to run + - Configures ports, environment variables, and other runtime settings + - Is stored in the galaxy-tools repository at `tools/neutrons/tutorials/-nova-tutorial.xml` -Once your changes are pushed to the prototype branch, an automated CI job will deploy your tool to the calvera-test instance. You can then access your tool through the NDIP web interface at https://calvera-test.ornl.gov. +3. **Deployment Process**: + - When you push code to your repository → CI builds a new container + - When you run `push-xml` → Your tool XML is updated in the galaxy-tools prototype branch + - After XML is merged → Your tool appears in the NDIP interface ::::::::::::::::::::::::::::::::::::::::: callout In a production environment, when your tool is ready for users, you would create a merge request from the prototype branch to the dev branch. The NDIP team reviews these changes, merges them, and your tool will be deployed to the production instance during the next deployment. -- GitLab From 26c6358ad58dc47dfb08e8eb9bdf849df4ae20db Mon Sep 17 00:00:00 2001 From: "Ayres, Andrew" Date: Mon, 17 Mar 2025 10:00:31 -0500 Subject: [PATCH 09/19] Update deploy section --- episodes/02-Copy-Template.md | 88 ++++++++++++++++++++++++++++-------- 1 file changed, 70 insertions(+), 18 deletions(-) diff --git a/episodes/02-Copy-Template.md b/episodes/02-Copy-Template.md index c75cef81..a9d79dcc 100755 --- a/episodes/02-Copy-Template.md +++ b/episodes/02-Copy-Template.md @@ -172,11 +172,67 @@ The template includes a basic GitLab CI configuration file (`.gitlab-ci.yml`). ## Deploying Your Tool to NDIP -Now that we have our template application set up, we need to integrate it with the NDIP platform. The template now includes built-in utilities to streamline this process, handling the GitLab repository setup and Galaxy tool XML management automatically. +Now that we have our template application set up, we need to integrate it with the NDIP platform. The template includes built-in utilities to streamline this process, handling the GitLab repository setup and Galaxy tool XML management. + +### Manual Configuration for Tutorial + +Since this is a tutorial, we need to modify a few files to properly configure our repository path. In a real project, you would use your own team's project name. + +#### Modify Repository Path + +1. Open `src/nova_tutorial/scripts/git_utils.py` in your editor and modify the line setting the repository path: + + ```python + # Find this line: + remote_url += "{{ project_name.lower().replace(' ', '-').replace('_', '-') }}" + + # Change it to: + remote_url += "tutorial/YOUR_USERNAME-nova-tutorial" + ``` + + Replace `YOUR_USERNAME` with your actual username (all lowercase). + +2. Open `src/nova_tutorial/scripts/xml_utils.py` and update the following: + + ```python + # Find these lines: + dest_dir = os.path.join(temp_dir, "tools", "neutrons") + xml_filename = "{{ project_name.lower().replace(' ', '-').replace('_', '-') }}.xml" + + # Change them to: + dest_dir = os.path.join(temp_dir, "tools", "neutrons", "tutorials") + xml_filename = "YOUR_USERNAME-nova-tutorial.xml" + + # Then find this line: + container_path = "ndip/tool-sources/{{ project_name.lower().replace(' ', '-').replace('_', '-') }}" + + # Change it to: + container_path = "ndip/tool-sources/tutorial/YOUR_USERNAME-nova-tutorial" + ``` + +3. Open `xml/tool.xml` and modify the tool ID: + + ```xml + + + + + + ``` + + And update the repository link in the help section: + + ```xml + + For more information, see the repository: https://code.ornl.gov/ndip/tool-sources/{{ project_name.lower().replace(' ', '-').replace('_', '-') }} + + + For more information, see the repository: https://code.ornl.gov/ndip/tool-sources/tutorial/YOUR_USERNAME-nova-tutorial + ``` ### Initialize Your Project Repository -The template comes with a utility script to initialize your Git repository and push it to the correct location in the NDIP GitLab. Run: +After making these changes, you can initialize your Git repository and push it to the correct location in the NDIP GitLab: ```bash cd nova_tutorial @@ -185,22 +241,18 @@ poetry run init-repo This script will: 1. Initialize a Git repository (if not already done) -2. Set up the remote to point to `https://code.ornl.gov/ndip/tool-sources/tutorial/-nova-tutorial` +2. Set up the remote to point to the configured repository URL 3. Add all project files to the repository 4. Create an initial commit (if needed) 5. Push the code to the GitLab repository -::::::::::::::::::::::::::::::::::::::::: callout -The repository path is automatically determined based on your answers during the template setup. For tutorial projects, it will use `tutorial/-project-name`, while for non-tutorial projects it will use the project group you specified. -:::::::::::::::::::::::::::::::::::::::::::::::: - ### Continuous Integration and Container Building Once your code is pushed to GitLab, the included CI/CD pipeline will automatically build a Docker container for your application. The CI configuration is already set up in the `.gitlab-ci.yml` file and includes: 1. Running tests to verify your code works correctly 2. Building a Docker image containing your application -3. Pushing the image to the NDIP container registry (at `code.ornl.gov:4567/ndip/tool-sources/tutorial/-nova-tutorial`) +3. Pushing the image to the NDIP container registry (at `code.ornl.gov:4567/ndip/tool-sources/tutorial/YOUR_USERNAME-nova-tutorial`) The Docker image tag is derived from your project's version in `pyproject.toml`. Each time you update the version and push, a new container will be built automatically. @@ -214,13 +266,13 @@ xml/tool.xml This file defines how your tool appears and functions within the NDIP platform. It includes: -- A unique tool ID based on your username (for tutorial projects) +- A unique tool ID (now manually configured for the tutorial) - The correct container reference pointing to your GitLab repository -- Port and URL configurations for accessing your application -- Environment variables needed by the NDIP platform - Command to run your application inside the container - Help and description text for users +After the manual changes we made in the previous step, your tool XML will be correctly configured for the tutorial environment. + ### Pushing the Tool XML to Galaxy Tools Repository To deploy your tool to the NDIP platform, you need to add the XML file to the galaxy-tools repository. The template includes a utility for this: @@ -231,14 +283,14 @@ poetry run push-xml This script will: 1. Clone the Galaxy tools repository -2. Copy your tool XML file to the correct location (`tools/neutrons/tutorials/-nova-tutorial.xml` for tutorial projects) +2. Copy your tool XML file to the correct location (now configured as `tools/neutrons/tutorials/YOUR_USERNAME-nova-tutorial.xml`) 3. Commit the changes 4. Push to the `prototype` branch of the galaxy-tools repository Once your XML file is pushed to the prototype branch, an automated CI job will deploy your tool to the calvera-test instance. You can then access your tool through the NDIP web interface at https://calvera-test.ornl.gov. ::::::::::::::::::::::::::::::::::::::::: callout -The tool XML is automatically pushed to the correct location. For tutorial projects, it goes to `tools/neutrons/tutorials/-project-name.xml`, and for regular projects, it goes to `tools/neutrons/project-name.xml`. +The tool XML utility has been enhanced to check for the existence of your Docker image before proceeding with the push. This helps prevent deployment errors by ensuring your container has been built first. :::::::::::::::::::::::::::::::::::::::::::::::: ### Understanding Your Tool's Integration @@ -246,18 +298,18 @@ The tool XML is automatically pushed to the correct location. For tutorial proje Let's understand the key components that make your tool work in NDIP: 1. **Repository Structure**: - - Your code is hosted at `https://code.ornl.gov/ndip/tool-sources/tutorial/-nova-tutorial` - - The Docker container is built automatically by CI and stored at `code.ornl.gov:4567/ndip/tool-sources/tutorial/-nova-tutorial` + - Your code is hosted at `https://code.ornl.gov/ndip/tool-sources/tutorial/YOUR_USERNAME-nova-tutorial` + - The Docker container is built automatically by CI and stored at `code.ornl.gov:4567/ndip/tool-sources/tutorial/YOUR_USERNAME-nova-tutorial` 2. **Tool XML File**: - Defines your tool for Galaxy/NDIP - References your container so NDIP knows which image to run - - Configures ports, environment variables, and other runtime settings - - Is stored in the galaxy-tools repository at `tools/neutrons/tutorials/-nova-tutorial.xml` + - Configures the command to run your application + - Is stored in the galaxy-tools repository at `tools/neutrons/tutorials/YOUR_USERNAME-nova-tutorial.xml` 3. **Deployment Process**: - When you push code to your repository → CI builds a new container - - When you run `push-xml` → Your tool XML is updated in the galaxy-tools prototype branch + - When you run `push-xml` → The utility checks if your container exists and pushes your tool XML to the galaxy-tools prototype branch - After XML is merged → Your tool appears in the NDIP interface ::::::::::::::::::::::::::::::::::::::::: callout -- GitLab From d90ef5f21a86bd73ee76b6900be17ec1949abcd1 Mon Sep 17 00:00:00 2001 From: "Ayres, Andrew" Date: Mon, 17 Mar 2025 16:15:01 +0000 Subject: [PATCH 10/19] Update .gitlab-ci.yml file --- .gitlab-ci.yml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 5dbe858c..aa8ab651 100755 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -36,7 +36,7 @@ pages: - docker run -v `pwd`/public:/app/site tutorial Rscript -e "sandpaper::build_lesson()" - cp $CALVERA_DOCS_SSH_KEY key && chmod 600 key - cat key - - rsync -avz -e "ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i key" public/docs/ cloud@10.64.195.250:/home/cloud/nova-tutorial/tutorial + - rsync -avz --mkpath -e "ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i key" public/docs/ cloud@10.64.196.237:/home/cloud/nova-tutorial/tutorial artifacts: paths: - public @@ -57,7 +57,7 @@ preview: - docker run -v `pwd`/public:/app/site tutorial Rscript -e "sandpaper::build_lesson()" - cp $CALVERA_DOCS_SSH_KEY key && chmod 600 key - cat key - - rsync -avz -e "ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i key" public/docs/ cloud@10.64.195.250:/home/cloud/nova-tutorial/tutorial/Preview + - rsync -avz --mkpath -e "ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i key" public/docs/ cloud@10.64.196.237:/home/cloud/nova-tutorial/tutorial/preview artifacts: paths: - public -- GitLab From d56fd3217dc0ad87279f0276154243d47fd2aa1e Mon Sep 17 00:00:00 2001 From: "Ayres, Andrew" Date: Tue, 18 Mar 2025 09:53:07 -0500 Subject: [PATCH 11/19] Fixes to tutorial code modifications --- episodes/02-Copy-Template.md | 17 +++++++++-------- 1 file changed, 9 insertions(+), 8 deletions(-) diff --git a/episodes/02-Copy-Template.md b/episodes/02-Copy-Template.md index a9d79dcc..a924f3ae 100755 --- a/episodes/02-Copy-Template.md +++ b/episodes/02-Copy-Template.md @@ -184,8 +184,8 @@ Since this is a tutorial, we need to modify a few files to properly configure ou ```python # Find this line: - remote_url += "{{ project_name.lower().replace(' ', '-').replace('_', '-') }}" - + remote_url += "nova-tutorial" + # Change it to: remote_url += "tutorial/YOUR_USERNAME-nova-tutorial" ``` @@ -197,14 +197,15 @@ Since this is a tutorial, we need to modify a few files to properly configure ou ```python # Find these lines: dest_dir = os.path.join(temp_dir, "tools", "neutrons") - xml_filename = "{{ project_name.lower().replace(' ', '-').replace('_', '-') }}.xml" + xml_filename = "nova-tutorial.xml" + # Change them to: dest_dir = os.path.join(temp_dir, "tools", "neutrons", "tutorials") xml_filename = "YOUR_USERNAME-nova-tutorial.xml" # Then find this line: - container_path = "ndip/tool-sources/{{ project_name.lower().replace(' ', '-').replace('_', '-') }}" + container_path = "ndip/tool-sources/nova-tutorial" # Change it to: container_path = "ndip/tool-sources/tutorial/YOUR_USERNAME-nova-tutorial" @@ -214,17 +215,17 @@ Since this is a tutorial, we need to modify a few files to properly configure ou ```xml - + - + ``` And update the repository link in the help section: ```xml - For more information, see the repository: https://code.ornl.gov/ndip/tool-sources/{{ project_name.lower().replace(' ', '-').replace('_', '-') }} + For more information, see the repository: https://code.ornl.gov/ndip/tool-sources/nova-tutorial For more information, see the repository: https://code.ornl.gov/ndip/tool-sources/tutorial/YOUR_USERNAME-nova-tutorial @@ -235,7 +236,6 @@ Since this is a tutorial, we need to modify a few files to properly configure ou After making these changes, you can initialize your Git repository and push it to the correct location in the NDIP GitLab: ```bash -cd nova_tutorial poetry run init-repo ``` @@ -282,6 +282,7 @@ poetry run push-xml ``` This script will: + 1. Clone the Galaxy tools repository 2. Copy your tool XML file to the correct location (now configured as `tools/neutrons/tutorials/YOUR_USERNAME-nova-tutorial.xml`) 3. Commit the changes -- GitLab From 3993912c4b3e1c147c9f7c8b6102c275e4a5d475 Mon Sep 17 00:00:00 2001 From: "Ayres, Andrew" Date: Tue, 18 Mar 2025 15:36:34 -0500 Subject: [PATCH 12/19] Formatting fix --- episodes/02-Copy-Template.md | 1 + 1 file changed, 1 insertion(+) diff --git a/episodes/02-Copy-Template.md b/episodes/02-Copy-Template.md index a924f3ae..618a2446 100755 --- a/episodes/02-Copy-Template.md +++ b/episodes/02-Copy-Template.md @@ -240,6 +240,7 @@ poetry run init-repo ``` This script will: + 1. Initialize a Git repository (if not already done) 2. Set up the remote to point to the configured repository URL 3. Add all project files to the repository -- GitLab From 0ced072ca69ffb970b16cb731801d627eea8a240 Mon Sep 17 00:00:00 2001 From: "Ayres, Andrew" Date: Wed, 19 Mar 2025 00:14:21 -0500 Subject: [PATCH 13/19] Moved template scripts --- episodes/02-Copy-Template.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/episodes/02-Copy-Template.md b/episodes/02-Copy-Template.md index 618a2446..6a6a245c 100755 --- a/episodes/02-Copy-Template.md +++ b/episodes/02-Copy-Template.md @@ -180,7 +180,7 @@ Since this is a tutorial, we need to modify a few files to properly configure ou #### Modify Repository Path -1. Open `src/nova_tutorial/scripts/git_utils.py` in your editor and modify the line setting the repository path: +1. Open `scripts/git_utils.py` in your editor and modify the line setting the repository path: ```python # Find this line: @@ -192,7 +192,7 @@ Since this is a tutorial, we need to modify a few files to properly configure ou Replace `YOUR_USERNAME` with your actual username (all lowercase). -2. Open `src/nova_tutorial/scripts/xml_utils.py` and update the following: +2. Open `scripts/xml_utils.py` and update the following: ```python # Find these lines: -- GitLab From 034168a1ac0a821a9dc8fd55b1132a4f2a2cb98f Mon Sep 17 00:00:00 2001 From: "Ayres, Andrew" Date: Fri, 21 Mar 2025 11:23:24 -0500 Subject: [PATCH 14/19] add gitlab ssh to setup --- learners/setup.md | 27 ++++++++++++++++++++++++++- 1 file changed, 26 insertions(+), 1 deletion(-) diff --git a/learners/setup.md b/learners/setup.md index 3bf0b707..e2ec07c7 100755 --- a/learners/setup.md +++ b/learners/setup.md @@ -60,7 +60,32 @@ The NOVA framework requires you to set environment variables for your NDIP URL a **Important:** For security reasons, it is recommended to avoid hardcoding your API key directly in your code. Using environment variables is a more secure and flexible approach. -## 4. Verify Your Setup +## 4. Create and Add SSH Key to GitLab + +For detailed instructions, refer to the [official GitLab SSH documentation](https://docs.gitlab.com/user/ssh/#generate-an-ssh-key-pair). + +To contribute code to GitLab repositories, you'll need to set up SSH authentication: + +1. **Generate an SSH key pair** if you don't already have one: + ```bash + ssh-keygen -t ed25519 -C "your_email@example.com" + ``` + +2. **Add your SSH key to the ssh-agent**: + ```bash + eval "$(ssh-agent -s)" + ssh-add ~/.ssh/id_ed25519 + ``` + +3. **Add your public key to your GitLab account**: + - Copy your public key to clipboard: + ```bash + cat ~/.ssh/id_ed25519.pub + ``` + - Go to GitLab > Preferences > SSH Keys + - Paste your key and add a descriptive title + +## 5. Verify Your Setup To ensure your setup is correct, run the following command in your terminal within the `nova_tutorial` directory: -- GitLab From 7b069b21cf584b698ecfe403fb655714a247b234 Mon Sep 17 00:00:00 2001 From: "Ayres, Andrew" Date: Mon, 24 Mar 2025 10:58:16 -0500 Subject: [PATCH 15/19] Move from ssh to https gitlab instructions. Remove tutorial template manual stuff --- episodes/02-Copy-Template.md | 63 ++---------------------------------- learners/setup.md | 39 +++++++++------------- 2 files changed, 19 insertions(+), 83 deletions(-) diff --git a/episodes/02-Copy-Template.md b/episodes/02-Copy-Template.md index 6a6a245c..0c6e57fe 100755 --- a/episodes/02-Copy-Template.md +++ b/episodes/02-Copy-Template.md @@ -36,7 +36,7 @@ The setup section detailed the prerequisites required for the tutorial. One of t To clone the template application, run the following command: ```bash -copier copy https://code.ornl.gov/ndip/project-templates/nova-application-template.git nova_tutorial +copier copy https://code.ornl.gov/ndip/project-templates/nova-application-template-tutorial.git nova_tutorial ``` This command will download the template to a directory called `nova_tutorial`. Copier will prompt you with a series of questions. Please answer the questions as follows: @@ -174,66 +174,9 @@ The template includes a basic GitLab CI configuration file (`.gitlab-ci.yml`). Now that we have our template application set up, we need to integrate it with the NDIP platform. The template includes built-in utilities to streamline this process, handling the GitLab repository setup and Galaxy tool XML management. -### Manual Configuration for Tutorial - -Since this is a tutorial, we need to modify a few files to properly configure our repository path. In a real project, you would use your own team's project name. - -#### Modify Repository Path - -1. Open `scripts/git_utils.py` in your editor and modify the line setting the repository path: - - ```python - # Find this line: - remote_url += "nova-tutorial" - - # Change it to: - remote_url += "tutorial/YOUR_USERNAME-nova-tutorial" - ``` - - Replace `YOUR_USERNAME` with your actual username (all lowercase). - -2. Open `scripts/xml_utils.py` and update the following: - - ```python - # Find these lines: - dest_dir = os.path.join(temp_dir, "tools", "neutrons") - xml_filename = "nova-tutorial.xml" - - - # Change them to: - dest_dir = os.path.join(temp_dir, "tools", "neutrons", "tutorials") - xml_filename = "YOUR_USERNAME-nova-tutorial.xml" - - # Then find this line: - container_path = "ndip/tool-sources/nova-tutorial" - - # Change it to: - container_path = "ndip/tool-sources/tutorial/YOUR_USERNAME-nova-tutorial" - ``` - -3. Open `xml/tool.xml` and modify the tool ID: - - ```xml - - - - - - ``` - - And update the repository link in the help section: - - ```xml - - For more information, see the repository: https://code.ornl.gov/ndip/tool-sources/nova-tutorial - - - For more information, see the repository: https://code.ornl.gov/ndip/tool-sources/tutorial/YOUR_USERNAME-nova-tutorial - ``` - ### Initialize Your Project Repository -After making these changes, you can initialize your Git repository and push it to the correct location in the NDIP GitLab: +You can initialize your Git repository and push it to the correct location in the NDIP GitLab: ```bash poetry run init-repo @@ -279,7 +222,7 @@ After the manual changes we made in the previous step, your tool XML will be cor To deploy your tool to the NDIP platform, you need to add the XML file to the galaxy-tools repository. The template includes a utility for this: ```bash -poetry run push-xml +poetry run deploy-tool ``` This script will: diff --git a/learners/setup.md b/learners/setup.md index e2ec07c7..aaa9a297 100755 --- a/learners/setup.md +++ b/learners/setup.md @@ -60,32 +60,25 @@ The NOVA framework requires you to set environment variables for your NDIP URL a **Important:** For security reasons, it is recommended to avoid hardcoding your API key directly in your code. Using environment variables is a more secure and flexible approach. -## 4. Create and Add SSH Key to GitLab +## 5. Create a GitLab Personal Access Token -For detailed instructions, refer to the [official GitLab SSH documentation](https://docs.gitlab.com/user/ssh/#generate-an-ssh-key-pair). - -To contribute code to GitLab repositories, you'll need to set up SSH authentication: - -1. **Generate an SSH key pair** if you don't already have one: - ```bash - ssh-keygen -t ed25519 -C "your_email@example.com" - ``` - -2. **Add your SSH key to the ssh-agent**: - ```bash - eval "$(ssh-agent -s)" - ssh-add ~/.ssh/id_ed25519 - ``` +::::::::::::::::::::::::::::::::::::::::: callout -3. **Add your public key to your GitLab account**: - - Copy your public key to clipboard: - ```bash - cat ~/.ssh/id_ed25519.pub - ``` - - Go to GitLab > Preferences > SSH Keys - - Paste your key and add a descriptive title +If you currently use an ssh key for gitlab, you will still need to create a personal access token for the tutorial. -## 5. Verify Your Setup +:::::::::::::::::::::::::::::::::::::::::::::::::: +To interact with repositories on code.ornl.gov, you'll need to create a personal access token: + +1. Navigate to [https://code.ornl.gov](https://code.ornl.gov) and log in with your credentials +2. In the left sidebar, select your avatar. +3. Select Edit Profile +4. On the left sidebar, select Access tokens +5. In Token name, enter a name for the token (such as Nova Tutorial) +6. Provide the desired scopes (at minimum, select "read repository", "write repository", and api) +7. Click Create personal access token. +8. **Important** Copy and save your token to your computer immediately. You will no longer have access to it after leaving the page. + +## 6. Verify Your Setup To ensure your setup is correct, run the following command in your terminal within the `nova_tutorial` directory: -- GitLab From 0913b91b808ea86ebfbf394a19aecc0c531f91dc Mon Sep 17 00:00:00 2001 From: John Duggan Date: Tue, 25 Mar 2025 09:21:38 -0400 Subject: [PATCH 16/19] Add analysis cluster instructions --- episodes/01-Introduction.md | 6 ++++++ learners/setup.md | 14 ++++++++++++++ 2 files changed, 20 insertions(+) diff --git a/episodes/01-Introduction.md b/episodes/01-Introduction.md index 9452e586..b3220fe6 100755 --- a/episodes/01-Introduction.md +++ b/episodes/01-Introduction.md @@ -93,6 +93,12 @@ poetry install # Install dependencies for this episode poetry run app # Run the application for this episode ``` +::::::::::::::::::::::::::::::::::::::::: callout + +If you are using the analysis cluster for the tutorial, then please note that `poetry run app` will by default attempt to bind to port 8080 and will fail if the port is already in use. This can happen if others are on the same node as you running the same commands. If this happens, you can change the port the application binds to with `poetry run app --port {myport}`. + +:::::::::::::::::::::::::::::::::::::::::::::::: + This structure ensures that each code example is isolated and runnable, making it easier for you to follow along with the tutorial and experiment with the code. ## References diff --git a/learners/setup.md b/learners/setup.md index aaa9a297..8ab55b95 100755 --- a/learners/setup.md +++ b/learners/setup.md @@ -19,6 +19,20 @@ Before proceeding, ensure you have met the following prerequisites: * **Git:** You must have git installed on your system. * **Familiarity with the Command Line:** You will need to be comfortable using the command line or terminal. +::::::::::::::::::::::::::::::::::::::::: callout + +You can use the analysis cluster for this tutorial. This is recommended if you use Windows or otherwise can't meet the above prerequisites on your laptop. By default, the `python` command on the cluster will use 3.9, so please explicitly reference `python3.11` where needed. + +You can create a virtual environment suitable for the tutorial on the cluster with: + +```bash +python3.11 -m venv .venv +source .venv/bin/activate +pip install copier poetry +``` + +:::::::::::::::::::::::::::::::::::::::::::::::: + ## 2. Getting your Galaxy API Key In order to run the code examples in this tutorial, an API Key is required. An API key is obtained from the NDIP instance directly. -- GitLab From 477b3464c112e0c2ea85c3dfedfe52dda04a3623 Mon Sep 17 00:00:00 2001 From: John Duggan Date: Tue, 25 Mar 2025 09:21:54 -0400 Subject: [PATCH 17/19] Fix setup instructions numbering --- learners/setup.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/learners/setup.md b/learners/setup.md index 8ab55b95..401bcadc 100755 --- a/learners/setup.md +++ b/learners/setup.md @@ -35,7 +35,7 @@ pip install copier poetry ## 2. Getting your Galaxy API Key -In order to run the code examples in this tutorial, an API Key is required. An API key is obtained from the NDIP instance directly. +In order to run the code examples in this tutorial, an API Key is required. An API key is obtained from the NDIP instance directly. ::::::::::::::::::::::::::::::::::::::::: callout @@ -74,7 +74,7 @@ The NOVA framework requires you to set environment variables for your NDIP URL a **Important:** For security reasons, it is recommended to avoid hardcoding your API key directly in your code. Using environment variables is a more secure and flexible approach. -## 5. Create a GitLab Personal Access Token +## 4. Create a GitLab Personal Access Token ::::::::::::::::::::::::::::::::::::::::: callout @@ -92,7 +92,7 @@ To interact with repositories on code.ornl.gov, you'll need to create a personal 7. Click Create personal access token. 8. **Important** Copy and save your token to your computer immediately. You will no longer have access to it after leaving the page. -## 6. Verify Your Setup +## 5. Verify Your Setup To ensure your setup is correct, run the following command in your terminal within the `nova_tutorial` directory: -- GitLab From 9841e25f4b9fc2888382b7ec0577e19946b917eb Mon Sep 17 00:00:00 2001 From: John Duggan Date: Tue, 25 Mar 2025 09:22:25 -0400 Subject: [PATCH 18/19] Add note on GitLab auth and point to Harbor instead of NDIP container registry --- episodes/02-Copy-Template.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/episodes/02-Copy-Template.md b/episodes/02-Copy-Template.md index 0c6e57fe..6267abff 100755 --- a/episodes/02-Copy-Template.md +++ b/episodes/02-Copy-Template.md @@ -182,6 +182,10 @@ You can initialize your Git repository and push it to the correct location in th poetry run init-repo ``` +::::::::::::::::::::::::::::::::::::::::: callout +If prompted for a username and password by GitLab, then please use your three-character ID as the username and the Personal Access Token you set up earlier as the password. +:::::::::::::::::::::::::::::::::::::::::::::::: + This script will: 1. Initialize a Git repository (if not already done) @@ -196,7 +200,7 @@ Once your code is pushed to GitLab, the included CI/CD pipeline will automatical 1. Running tests to verify your code works correctly 2. Building a Docker image containing your application -3. Pushing the image to the NDIP container registry (at `code.ornl.gov:4567/ndip/tool-sources/tutorial/YOUR_USERNAME-nova-tutorial`) +3. Pushing the image to the Harbor container registry (at `savannah.ornl.gov/ndip/tool-sources/tutorial/YOUR_USERNAME-nova-tutorial`) The Docker image tag is derived from your project's version in `pyproject.toml`. Each time you update the version and push, a new container will be built automatically. -- GitLab From ad54d5b2e96476526184590bd65aac7a28cdb292 Mon Sep 17 00:00:00 2001 From: John Duggan Date: Tue, 25 Mar 2025 09:27:30 -0400 Subject: [PATCH 19/19] Escape quote --- learners/setup.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/learners/setup.md b/learners/setup.md index 401bcadc..edd17a99 100755 --- a/learners/setup.md +++ b/learners/setup.md @@ -21,7 +21,7 @@ Before proceeding, ensure you have met the following prerequisites: ::::::::::::::::::::::::::::::::::::::::: callout -You can use the analysis cluster for this tutorial. This is recommended if you use Windows or otherwise can't meet the above prerequisites on your laptop. By default, the `python` command on the cluster will use 3.9, so please explicitly reference `python3.11` where needed. +You can use the analysis cluster for this tutorial. This is recommended if you use Windows or otherwise can\'t meet the above prerequisites on your laptop. By default, the `python` command on the cluster will use 3.9, so please explicitly reference `python3.11` where needed. You can create a virtual environment suitable for the tutorial on the cluster with: -- GitLab