Merge branch 'fs-integration' into 'master'

# Standalone MinIO example application

This Chart provides an example that deploys a standalone MinIO application. It is a single MinIO server running in a pod, as a "deployment", with a small persistent volume claim to persist data.

This Chart provides an example that deploys a standalone [MinIO](https://min.io/) application. It is a single MinIO server running in a pod, as a "deployment", with a small persistent volume claim to persist data.

This example is a derivative of the [MinIO provided example](https://github.com/minio/minio/blob/master/docs/orchestration/kubernetes/k8s-yaml.md) to target Slate specifically.

Minio needs persistent storage to store objects. The idea of persistent data applies to many data applictions, so this fundemental piece is applicable in many scenarios. Without persistent storage, the data stored in an application instance will be stored in the container file system, which will be destroyed as soon as the container restarts. This is true for any application needing to persist data on Slate. So, to persist data in your application use a persistent volume claim, or "PVC".

The file we use to do this is [minio-standalone-pvc.yaml](https://code.ornl.gov/ryu/slate_helm_examples/-/blob/prout-dev/charts/minio-standalone/templates/minio-standalone-pvc.yaml): 

The file we use to do this is [minio-standalone-pvc.yaml](templates/minio-standalone-pvc.yaml).

```

apiVersion: v1

kind: PersistentVolumeClaim

metadata:

  # This name uniquely identifies the PVC. This is used in deployment.

  name: minio-standalone-pv-claim

spec:

  # Read more about access modes here: http://kubernetes.io/docs/user-guide/persistent-volumes/#access-modes

  accessModes:

    # The volume is mounted as read-write by a single node

    - ReadWriteOnce

  resources:

    # This is the request for storage. Should be available in the cluster.

    requests:

      storage: 10Gi

```

### The Deployment

A deployment encapsulates [ReplicaSets](https://kubernetes.io/docs/concepts/workloads/controllers/replicaset/) and [Pods](https://kubernetes.io/docs/concepts/workloads/pods/pod/). If a pod goes down, the replication controller makes sure another pod comes up automatically. This creates the ability to handle pod failures automatically, without you having to worry about them, enabliing stable applications and services.

The file we use to do this is [minio-standaline-deployment.yaml](https://code.ornl.gov/ryu/slate_helm_examples/-/blob/prout-dev/charts/minio-standalone/templates/minio-standalone-deployment.yaml):

```

apiVersion: apps/v1

kind: Deployment

metadata:

  # This name uniquely identifies the Deployment

  name: minio-standalone

spec:

  selector:

    matchLabels:

      app: minio-standalone # has to match .spec.template.metadata.labels

  strategy:

    # Specifies the strategy used to replace old Pods by new ones

    # Refer: https://kubernetes.io/docs/concepts/workloads/controllers/deployment/#strategy

    type: Recreate

  template:

    metadata:

      labels:

        # This label is used as a selector in Service definition

        app: minio-standalone

    spec:

      # Volumes used by this deployment

      volumes:

      - name: data

        # This volume is based on PVC

        persistentVolumeClaim:

          # Name of the PVC created earlier

          claimName: minio-standalone-pv-claim

      containers:

      - name: minio

        # Volume mounts for this container

        volumeMounts:

        # Volume 'data' is mounted to path '/data'

        - name: data 

          mountPath: "/data"

        # Pulls the lastest Minio image from Docker Hub

        image: minio/minio:RELEASE.2020-05-08T02-40-49Z

        args:

        - server

        - /data

        env:

        # MinIO access key and secret key

        - name: MINIO_ACCESS_KEY

          valueFrom:

            secretKeyRef:

              key: SECRET_TOKEN

              name: minio-standalone-access-key

        - name: MINIO_SECRET_KEY

          valueFrom:

            secretKeyRef:

              key: SECRET_TOKEN

              name: minio-standalone-secret-key 

        ports:

        - containerPort: 9000

        # Readiness probe detects situations when MinIO server instance

        # is not ready to accept traffic. Kubernetes doesn't forward

        # traffic to the pod while readiness checks fail.

        readinessProbe:

          httpGet:

            path: /minio/health/ready

            port: 9000

          initialDelaySeconds: 120

          periodSeconds: 20

        # Liveness probe detects situations where MinIO server instance

        # is not working properly and needs restart. Kubernetes automatically

        # restarts the pods if liveness checks fail.

        livenessProbe:

          httpGet:

            path: /minio/health/live

            port: 9000

          initialDelaySeconds: 120

          periodSeconds: 20

```

The file we use to do this is [minio-standaline-deployment.yaml](templates/minio-standalone-deployment.yaml).

### The Service

A service allows us to expose our deployment externally. There are three major service types - the default is ClusterIP, which exposes a service to connection from inside the cluster. The NodePort and LoadBalancer types enable the ability to expose services to external traffic directly.

In this example we use the ClusterIP type in our [minio-standalone-service.yaml](https://code.ornl.gov/ryu/slate_helm_examples/-/blob/prout-dev/charts/minio-standalone/templates/minio-standalone-service.yaml):

```

apiVersion: v1

kind: Service

metadata:

  # This name uniquely identifies the service

  name: minio-standalone-service

  labels:

    app: minio-standalone

spec:

  type: ClusterIP

  ports:

    - name: 9000-tcp

      port: 9000

      targetPort: 9000

      protocol: TCP

  selector:

    # Looks for labels `app:minio-standalone` in the namespace and applies the spec

    app: minio-standalone

```

In this example we use the ClusterIP type in our [minio-standalone-service.yaml](templates/minio-standalone-service.yaml).

### The Route

Since we are using ClusterIP, in the service type above, and our MinIO application is accessible via HTTP/HTTPS, we expose the application via a route. If our service used another protocol, besides HTTP/HTTPS, we would use a NodePort service type.

Here is our [route.yaml](https://code.ornl.gov/ryu/slate_helm_examples/-/blob/prout-dev/charts/minio-standalone/templates/route.yaml) file:

```

apiVersion: route.openshift.io/v1

kind: Route

metadata:

  name: minio-standalone

spec:

  host: minio-standalone.apps.marble.ccs.ornl.gov

  to:

    # Associate with the service we created

    kind: Service

    name: minio-standalone-service

  port:

    # Needs to match name of port in the service file

    targetPort: 9000-tcp

  tls:

    # Terminate TLS at the router, before sending to service. Preferred method of securing a route.

    termination: edge

 ```

Here is our [route.yaml](templates/route.yaml) file.

Simple diagram of networking relationship and the deployment wrapping the pod(s):

Network Policies are specifications of how groups of pods are allowed to communicate with each other and other network endpoints. A pod is selected in Network Policy based on a label, defining rules for network traffic, specific to that pod.

In this example, we create a Network Policy for our minio-standalone applicaiton. Here is the [network-policy.yaml](https://code.ornl.gov/ryu/slate_helm_examples/-/blob/prout-dev/charts/minio-standalone/templates/network-policy.yaml) file:

In this example, we create a Network Policy for our minio-standalone applicaiton. Here is the [network-policy.yaml](templates/network-policy.yaml).

```

kind: NetworkPolicy

apiVersion: networking.k8s.io/v1

metadata:

  name: web-allow-external

  namespace: stf007

spec:

  podSelector:

    matchLabels:

      # how we match our application

      app: minio-standalone

  ingress:

    # allow all

    - {}

  policyTypes:

    - Ingress

```

### The Values File

Helm templates provide built-in objects. One of these objects is "Values". This object provides access to values passed into the chart. The contents, in this case, come from [values.yaml](https://code.ornl.gov/ryu/slate_helm_examples/-/blob/prout-dev/charts/minio-standalone/values.yaml):

Helm templates provide built-in objects. One of these objects is "Values". This object provides access to values passed into the chart. The contents, in this case, come from [values.yaml](./values.yaml).

```

# This can be used to provide variables to your chart. 

# A simple example would be resources in the minio-standalone-deployment.yaml:

minio:

  resources:

    requests:

      cpu: 2

      memory: 1Gi

    limits:

      cpu: 2

      memory: 1Gi

```

**NOTE**: Please make sure you rename your ```minio.host``` value. This needs to be unique.

 No newline at end of file

## Installing the Chart

### Log into Marble

Log into Slate's Marble cluster with the OC CLI Tool.

```

oc login https://api.marble.ccs.ornl.gov

```

You should see a response similar to this, seeing your available project namespace(s):

```

Login successful.

You have one project on this server: "stf007"

Using project "stf007".

```

### Create secret token (do not keep this in a repo, only locally)

One thing we need to do is create the access key and secret key for the MinIO deployment. To avoid keeping the secret-tokens.yaml file in the public repository, we will quickly create those now.

Example secret-token.yaml file:

```

apiVersion: v1

kind: List

metadata: {}

items:

- apiVersion: v1

  kind: Secret

  metadata:

    name: minio-standalone-access-key

  stringData:

    SECRET_TOKEN: <your_choice>

- apiVersion: v1

  kind: Secret

  metadata:

    name: minio-standalone-secret-key

  stringData:

    SECRET_TOKEN: <your_choice>

```

Replace ```<your_choice>``` with your secret access tokens of choice. These are needed to log into the MinIO GUI.

Once you have your secret-tokens.yaml file established, you can apply it like this:

```

oc apply -f secret-token.yaml

```

The [minio-standalone-deployment.yaml](https://code.ornl.gov/ryu/slate_helm_examples/-/blob/master/charts/minio-standalone/templates/minio-standalone-deployment.yaml) file picks these up as environment variables.

### Clone this repo

Clone the repo to your local system. Then, 'cd' into 'slate_helm_examples/charts'.

```

git clone https://code.ornl.gov/ryu/slate_helm_examples.git && cd slate_helm_examples/charts

```

### Check Helm works

Run simple Helm command to make sure it works properly (assuming the use of helm3):

```

helm ls

```

Output will be similar, depending on if you have any existing applications (may be empty):

```

NAME                              NAMESPACE REVISION  UPDATED                               STATUS    CHART                   APP VERSION

gitlab-runner-for-slate-examples  stf007    9         2020-05-18 11:31:08.3245 -0400 EDT    deployed  gitlab-runner-0.16.1    12.10.2    

minio-standalone                  stf007    4         2020-05-19 13:00:38.165911 -0400 EDT  deployed  minio-standalone-1.0.0

```

### Use Helm to install MinIO Standalone

Use helm to install the application into your namespace/project with a unique name:

```

helm install <your-uid>-minio-standalone minio-standalone/ --namespace <your_namespace>

```

**NOTE:** ```<your_namespace>``` is available above, from the login output, or you can run ```oc get projects``` to list your projects/namespaces.

Check your deployment:

```

helm ls

```

Log into the Marble GUI to see all the pieces there too: https://console-openshift-console.apps.marble.ccs.ornl.gov/

MinIO will take several minutes to spin up, but after a few minutes you will be able to access your MinIO applicaiton at: 

```https://<your_UID>-minio-standalone.apps.marble.ccs.ornl.gov/minio/```

### Use Helm to Delete the MinIO Standalone Deployment

```helm delete minio-standalone```

apiVersion: apps/v1

kind: Deployment

metadata:

  {{- if eq .Values.minio.use_olcf_fs "enabled" }}

  annotations:

    ccs.ornl.gov/fs: olcf

  {{- end }}

  # This name uniquely identifies the Deployment

  name: {{ .Values.minio.name }}

spec:

        # This label is used as a selector in Service definition

        app: {{ .Values.minio.name }}

    spec:

      {{- if eq .Values.minio.use_olcf_fs "disabled" }}

      # Volumes used by this deployment

      volumes:

      - name: data

        persistentVolumeClaim:

          # Name of the PVC created earlier

          claimName: {{ .Values.minio.name }}-pv-claim

      {{- end }}

      containers:

      - name: minio

      {{- if eq .Values.minio.use_olcf_fs "disabled" }}

        # Volume mounts for this container

        volumeMounts:

        # Volume 'data' is mounted to path '/data'

        - name: data 

          mountPath: "/data"

      {{- end }}

        # Pulls the lastest Minio image from Docker Hub

        image: minio/minio:RELEASE.2020-05-08T02-40-49Z

        args:

        - server

        {{- if eq .Values.minio.use_olcf_fs "disabled" }}

        - /data

        {{- end }}

        {{- if eq .Values.minio.use_olcf_fs "enabled" }}

        - {{ .Values.minio.olcfMount }}

        {{- end }}

        env:

        # MinIO access key and secret key

        - name: MINIO_ACCESS_KEY

  resources:

    # This is the request for storage. Should be available in the cluster.

    requests:

      storage: 10Gi

      storage: {{ .Values.minio.pvc_storage }}

# This can be used to provide variables to your chart. 

# A simple example would be resources in the minio-standalone-deployment.yaml:

# Below are the current configurable variables.

minio:

  resources:

    requests:

  host: rprout-minio-standalone.apps.marble.ccs.ornl.gov

  # Change this to create unique app name

  name: rprout-minio-standalone

  # Set this to "disbled" to not use OLCF fileystem. If "disabled" it will use a volume isolated to the MinIO Pod.

  use_olcf_fs: enabled

  # This is the OLCF file system path MinIO will server out of, if "enabled" above.

  olcfMount: /ccs/proj/stf007/minio-test

  # Amount of storage to use, if use_olcf_fs is "disabled"

  pvc_storage: 10Gi