Commit 31323fbe authored by Parete-Koon, Suzanne's avatar Parete-Koon, Suzanne
Browse files

Update README.md

parent a5f0e5da
# Globus Sign in # Globus Sign in
You may have alredy tested your globus sign in with the instuctions you were sent in your training registration confirmation.
1. Make sure you can sign in to globus.org
During the training Globus will be the only way to get some of the examples and challenge problems that we will be doing.
**Globus Sign In for CADES users with a UCAMS ID (ORNL employee)** **Globus Sign In for CADES users with a UCAMS ID (ORNL employee)**
...@@ -28,39 +26,23 @@ Globus Endpoints are storage locations to which you have access. Endpoints may b ...@@ -28,39 +26,23 @@ Globus Endpoints are storage locations to which you have access. Endpoints may b
Users may also create their own personal endpoints, and manage access and sharing. That is beyond the scope of this training, for instructions see: https://support.cades.ornl.gov/user-documentation/_book/data-transfer-storage/globus-connect-personal.html Users may also create their own personal endpoints, and manage access and sharing. That is beyond the scope of this training, for instructions see: https://support.cades.ornl.gov/user-documentation/_book/data-transfer-storage/globus-connect-personal.html
Activating an endpoint generally requires one or more of the following:
1. Credentials at the institutes hosting the endpoints.
* For example, activating the OLCF or NERSC endpoints requires accounts at those facilities.
* Authentication is commonly required for institutional Data Transfer Nodes (DTNs)
2. Someone having created and shared a [Globus Connect Personal](https://www.globus.org/globus-connect-personal) endpoint with you.
* Globus provides the authentication layer, and access to data is permitted by the owner explicitly to other Globus users.
* Credentials at the institute hosting the data are not needed; delegated access has been extended to you.
# CADES and OLCF Endpoint Names # CADES and OLCF Endpoint Names
Once logged into Globus, you can select the Endpoints icon and search for public endpoints by name. Once logged into Globus, you can select the Endpoints icon in the left blue menu, and search for public endpoints by name.
Searching for ```CADES OR``` or ```OLCF DTN``` will present these endpoint for you to activate. Searching for ```CADES OR``` or ```OLCF DTN``` will present these endpoint for you to activate.
Once activated the following paths are allowed.
| Endpoint Search Term(s) | Storage System | Path | Description |
| ------------------------- | --------------- | ------ | ------------ |
| CADES OR | NFS | /~/ | CADES open research, user home directory |
| CADES OR | NFS | /data/ | CADES open research, NFS project directories |
| CADES OR | Lustre | /lustre/or-hydra/ | CADES open research, project directories. High-performance, temporary storage. |
| OLCF ALPINE | OLCF DTN | /path/to/project/file/data | OLCF-managed NFS and GPFS storage system. OLCF RSA required. |
<!--| NCCS Open DTN| Lustre | See [here](https://www.olcf.ornl.gov/for-users/system-user-guides/titan/file-systems/#data-management-policy-summary) | Requires NCCS Open (XCAMS) account | -->
## Activating Endpoints ## Activating Endpoints
> Note: Credentials for each institutions endpoint are usually required. Activating an endpoint generally requires credentials at the institutes hosting the endpoints.
* For example, activating the OLCF or NERSC endpoints requires accounts at those facilities.
* Authentication is commonly required for institutional Data Transfer Nodes (DTNs)
### Activiating the CADES-OR Endpoint
1. Click in the Endpoint box on the left side and search for `CADES-OR` (CADES Open Research). 1. Click in the "Endpoints" box on the left blue menu on globus.org side and search for `CADES-OR` (CADES Open Research).
* You will be redirected to enter your UCAMS credentials. * You will be redirected to enter your UCAMS credentials or globusID.
* Authenticating the Endpoint with your credentials is known as *Endpoint Activation* and can be done when adding and using an Endpoint for the first time, or can be completed by navigating to the "Manage Endpoints" screen as shown in the following image (`Endpoints`&rarr;`Endpoint List`&rarr;`activate`). * Authenticating the Endpoint with your credentials is known as *Endpoint Activation* and can be done when adding and using an Endpoint for the first time, or can be completed by navigating to the "Manage Endpoints" screen as shown in the following image (`Endpoints`&rarr;`Endpoint List`&rarr;`activate`).
...@@ -68,36 +50,39 @@ Once activated the following paths are allowed. ...@@ -68,36 +50,39 @@ Once activated the following paths are allowed.
## Moving Files ## Moving Files
The "File Manager" allows you to move files between diffrent endpoints or different directories at the same endpoint. Once you have signed in to globus and activated the CADES-OR endpoint you can move files with the "File Manager" between diffrent directories.
*Example:* *Example:*
This examples set up shows you how to move files from Lustre to your NFS /home area to aviod losing in the Lustre purge.
Next got to “filemanager” on the left blue column menu: This example setup shows you how to move files from Lustre to your NFS /home/userID area to aviod losing them in the Lustre purge.
* In the right Collection window type Cades-OR. Next go to “filemanager” on the left blue column menu:
* Notice that you see your home directory contents listed, this is the default landing point. * In the left "Collection" window type CADES-OR.
* In the left collection window type CADES-OR * Notice that you see your NFS home/userID directory contents listed, this is the default landing point.
* Enter the path to birthright Lustre scratch space: * In the right "Collection" window type "CADES-OR"
* /lustre/or-hydra/cades-birthright * In the "Path" window enter the path to birthright Lustre scratch space world-shared area:
* Click on a file in Lustre and use the option at the bottom of the page to move it to NFS. * /lustre/or-hydra/cades-birthright/world-shared
* Click on *my_test_file.txt* in world-shared and use the " < Start" option at the bottom of right colum to move it to NFS.
* Notice that you can move whole folders. * Notice that you can move whole folders.
A note about data managment: A note about data managment:
This makes it easy to move folders and files out of Lustre, but you really need to curate and keep only the data that you need. This makes it easy to move folders and files out of Lustre, but you really should save only the data that you need to keep.
When you start a simulation campaign, plan first for what files you will keep and what files you will discard.
## Globus Challange ## Globus Challange
Sign in to gloubs.org. 1. Sign in to gloubs.org.
Activate the CADES-OR endpoint.
2. Activate the CADES-OR endpoint.
3. Use Globus to:
* Get /lustre/or-hydra/cades-birthright/world-shared/GPU_basics and
* Put it in your /home/userID NFS directory
Use Globus to:
* Get /lustre/or-hydra/cades-birthright/world-share/GPU_basics and
* Put it in your Lustre scratch space.
* /lustre/or-hydra/cades-birthright/userID/
* What was your transfer speed? * What was your transfer speed?
Can you find the OLCF HPSS endpoint? 4. Can you find the OLCF HPSS endpoint?
Can you find the NERSC endpoints? 5. Can you find the NERSC endpoints?
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment