前提条件：
k8s集群（已搭建好）
nfs服务器（已搭建好）
nfs客户端（pod可调度节点作为客户端，已搭建好）
Kubernetes 推荐使用 PersistentVolume (PV) 和 PersistentVolumeClaim (PVC) 来管理存储。
静态PV配置
该方法需要管理员手动配置PV
#测试pod使用nfs存储
---
apiVersion: v1
kind: PersistentVolume
metadata:
name: nfs-pv
namespace: test-ng
spec:
capacity:
storage: 1Gi # 你想要的存储大小
accessModes:
- ReadWriteMany # 允许多个节点读写
nfs:
path: /data/k8s_nfs # NFS 共享的路径
server: x.x.x.x # NFS 服务器地址
persistentVolumeReclaimPolicy: Retain
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: nfs-pvc
namespace: test-ng
spec:
accessModes:
- ReadWriteMany
resources:
requests:
storage: 1Gi
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: nfsdemo-v1
namespace: test-ng
labels:
app: nfsdemo
version: v1
spec:
replicas: 1
selector:
matchLabels:
app: nfsdemo
version: v1
template:
metadata:
labels:
app: nfsdemo
version: v1
spec:
containers:
- name: nfsdemo
image: swr.cn-north-4.myhuaweicloud.com/ddn-k8s/docker.io/nginx:stable-alpine
ports:
- containerPort: 80
volumeMounts:
- name: nfs-storage
mountPath: "/usr/share/nginx/html"# 容器内的挂载路径
volumes:
- name: nfs-storage
persistentVolumeClaim:
claimName: nfs-pvc
---
apiVersion: v1
kind: Service
metadata:
name: nfsdemo-service
namespace: test-ng
spec:
type: NodePort # 设置为 NodePort 类型
selector:
app: nfsdemo # 与 Deployment 中的 Pod 匹配的标签
ports:
- port: 80 # 服务的端口
targetPort: 80 # Pod 中容器的端口动态PV配置
该方法无需手动创建 PV，当用户创建 PVC 时，自动创建对应的 PV。
这需要部署一个 NFS Subdir External Provisioner（例如 nfs-subdir-external-provisioner Helm Chart）。这个 Provisioner 会监视 PVC 的创建，并自动在指定的 NFS 共享目录下创建以命名空间和 PVC 名称命名的子目录作为新的 PV。
使用helm配置
使用helm添加nfs-subdir-external-provisioner
$ helm repo add nfs-subdir-external-provisioner https://kubernetes-sigs.github.io/nfs-subdir-external-provisioner/
$ helm install nfs-subdir-external-provisioner nfs-subdir-external-provisioner/nfs-subdir-external-provisioner \
--set nfs.server=x.x.x.x \
--set nfs.path=/data/k8s_nfs查看安装的chart的详细配置
[root@k8smaster ~]# helm show all nfs-subdir-external-provisioner/nfs-subdir-external-provisioner
apiVersion: v1
appVersion: 4.0.2
description: nfs-subdir-external-provisioner is an automatic provisioner that used
your *already configured* NFS server, automatically creating Persistent Volumes.
home: https://github.com/kubernetes-sigs/nfs-subdir-external-provisioner
keywords:
- nfs
- storage
- provisioner
kubeVersion: '>=1.9.0-0'
name: nfs-subdir-external-provisioner
sources:
- https://github.com/kubernetes-sigs/nfs-subdir-external-provisioner
version: 4.0.18
---
replicaCount: 1
strategyType: Recreate
image:
repository: registry.k8s.io/sig-storage/nfs-subdir-external-provisioner
tag: v4.0.2
pullPolicy: IfNotPresent
imagePullSecrets: []
nfs:
server:
path: /nfs-storage
mountOptions:
volumeName: nfs-subdir-external-provisioner-root
# Reclaim policy for the main nfs volume
reclaimPolicy: Retain
# For creating the StorageClass automatically:
storageClass:
create: true
# Set a provisioner name. If unset, a name will be generated.
# provisionerName:
# Set StorageClass as the default StorageClass
# Ignored if storageClass.create is false
defaultClass: false
# Set a StorageClass name
# Ignored if storageClass.create is false
name: nfs-client
# Allow volume to be expanded dynamically
allowVolumeExpansion: true
# Method used to reclaim an obsoleted volume
reclaimPolicy: Delete
# When set to false your PVs will not be archived by the provisioner upon deletion of the PVC.
archiveOnDelete: true
# If it exists and has 'delete' value, delete the directory. If it exists and has 'retain' value, save the directory.
# Overrides archiveOnDelete.
# Ignored if value not set.
onDelete:
# Specifies a template for creating a directory path via PVC metadata's such as labels, annotations, name or namespace.
# Ignored if value not set.
pathPattern:
# Set access mode - ReadWriteOnce, ReadOnlyMany or ReadWriteMany
accessModes: ReadWriteOnce
# Set volume bindinng mode - Immediate or WaitForFirstConsumer
volumeBindingMode: Immediate
# Storage class annotations
annotations: {}
leaderElection:
# When set to false leader election will be disabled
enabled: true
## For RBAC support:
rbac:
# Specifies whether RBAC resources should be created
create: true
# If true, create & use Pod Security Policy resources
# https://kubernetes.io/docs/concepts/policy/pod-security-policy/
podSecurityPolicy:
enabled: false
# Deployment pod annotations
podAnnotations: {}
## Set pod priorityClassName
# priorityClassName: ""
podSecurityContext: {}
securityContext: {}
serviceAccount:
# Specifies whether a ServiceAccount should be created
create: true
# Annotations to add to the service account
annotations: {}
# The name of the ServiceAccount to use.
# If not set and create is true, a name is generated using the fullname template
name:
resources: {}
# limits:
# cpu: 100m
# memory: 128Mi
# requests:
# cpu: 100m
# memory: 128Mi
nodeSelector: {}
tolerations: []
affinity: {}
# Additional labels for any resource created
labels: {}
podDisruptionBudget:
enabled: false
maxUnavailable: 1
---
# NFS Subdirectory External Provisioner Helm Chart
The [NFS subdir external provisioner](https://github.com/kubernetes-sigs/nfs-subdir-external-provisioner) is an automatic provisioner for Kubernetes that uses your *already configured* NFS server, automatically creating Persistent Volumes.
## TL;DR;
```console
$ helm repo add nfs-subdir-external-provisioner https://kubernetes-sigs.github.io/nfs-subdir-external-provisioner/
$ helm install nfs-subdir-external-provisioner nfs-subdir-external-provisioner/nfs-subdir-external-provisioner \
--set nfs.server=x.x.x.x \
--set nfs.path=/exported/path
```
## Introduction
This charts installs custom [storage class](https://kubernetes.io/docs/concepts/storage/storage-classes/) into a [Kubernetes](http://kubernetes.io) cluster using the [Helm](https://helm.sh) package manager. It also installs a [NFS client provisioner](https://github.com/kubernetes-sigs/nfs-subdir-external-provisioner) into the cluster which dynamically creates persistent volumes from single NFS share.
## Prerequisites
- Kubernetes >=1.9
- Existing NFS Share
## Installing the Chart
To install the chart with the release name `my-release`:
```console
$ helm install my-release nfs-subdir-external-provisioner/nfs-subdir-external-provisioner \
--set nfs.server=x.x.x.x \
--set nfs.path=/exported/path
```
The command deploys the given storage class in the default configuration. It can be used afterwards to provision persistent volumes. The [configuration](#configuration) section lists the parameters that can be configured during installation.
> **Tip**: List all releases using `helm list`
## Uninstalling the Chart
To uninstall/delete the `my-release` deployment:
```console
$ helm delete my-release
```
The command removes all the Kubernetes components associated with the chart and deletes the release.
## Configuration
The following tables lists the configurable parameters of this chart and their default values.
| Parameter | Description | Default |
| ------------------------------------ | ----------------------------------------------------------------------------------------------------- | ------------------------------------------------------------- |
| `replicaCount` | Number of provisioner instances to deployed | `1` |
| `strategyType` | Specifies the strategy used to replace old Pods by new ones | `Recreate` |
| `image.repository` | Provisioner image | `registry.k8s.io/sig-storage/nfs-subdir-external-provisioner` |
| `image.tag` | Version of provisioner image | `v4.0.2` |
| `image.pullPolicy` | Image pull policy | `IfNotPresent` |
| `imagePullSecrets` | Image pull secrets | `[]` |
| `storageClass.name` | Name of the storageClass | `nfs-client` |
| `storageClass.defaultClass` | Set as the default StorageClass | `false` |
| `storageClass.allowVolumeExpansion` | Allow expanding the volume | `true` |
| `storageClass.reclaimPolicy` | Method used to reclaim an obsoleted volume | `Delete` |
| `storageClass.provisionerName` | Name of the provisionerName | null |
| `storageClass.archiveOnDelete` | Archive PVC when deleting | `true` |
| `storageClass.onDelete` | Strategy on PVC deletion. Overrides archiveOnDelete when set to lowercase values 'delete' or 'retain' | null |
| `storageClass.pathPattern` | Specifies a template for the directory name | null |
| `storageClass.accessModes` | Set access mode for PV | `ReadWriteOnce` |
| `storageClass.volumeBindingMode` | Set volume binding mode for Storage Class | `Immediate` |
| `storageClass.annotations` | Set additional annotations for the StorageClass | `{}` |
| `leaderElection.enabled` | Enables or disables leader election | `true` |
| `nfs.server` | Hostname of the NFS server (required) | null (ip or hostname) |
| `nfs.path` | Basepath of the mount point to be used | `/nfs-storage` |
| `nfs.mountOptions` | Mount options (e.g. 'nfsvers=3') | null |
| `nfs.volumeName` | Volume name used inside the pods | `nfs-subdir-external-provisioner-root` |
| `nfs.reclaimPolicy` | Reclaim policy for the main nfs volume used for subdir provisioning | `Retain` |
| `resources` | Resources required (e.g. CPU, memory) | `{}` |
| `rbac.create` | Use Role-based Access Control | `true` |
| `podSecurityPolicy.enabled` | Create & use Pod Security Policy resources | `false` |
| `podAnnotations` | Additional annotations for the Pods | `{}` |
| `priorityClassName` | Set pod priorityClassName | null |
| `serviceAccount.create` | Should we create a ServiceAccount | `true` |
| `serviceAccount.name` | Name of the ServiceAccount to use | null |
| `serviceAccount.annotations` | Additional annotations for the ServiceAccount | `{}` |
| `nodeSelector` | Node labels for pod assignment | `{}` |
| `affinity` | Affinity settings | `{}` |
| `tolerations` | List of node taints to tolerate | `[]` |
| `labels` | Additional labels for any resource created | `{}` |
| `podDisruptionBudget.enabled` | Create and use Pod Disruption Budget | `false` |
| `podDisruptionBudget.maxUnavailable` | Set maximum unavailable pods in the Pod Disruption Budget | `1` |
## Install Multiple Provisioners
It is possible to install more than one provisioner in your cluster to have access to multiple nfs servers and/or multiple exports from a single nfs server. Each provisioner must have a different `storageClass.provisionerName` and a different `storageClass.name`. For example:
```console
helm install second-nfs-subdir-external-provisioner nfs-subdir-external-provisioner/nfs-subdir-external-provisioner \
--set nfs.server=y.y.y.y \
--set nfs.path=/other/exported/path \
--set storageClass.name=second-nfs-client \
--set storageClass.provisionerName=k8s-sigs.io/second-nfs-subdir-external-provisioner
```
卸载 nfs-subdir-external-provisioner
helm uninstall nfs-subdir-external-provisioner动态PV测试
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: test-claim
spec:
storageClassName: "nfs-sc"
accessModes:
- ReadWriteMany
resources:
requests:
storage: 1Gi
---
apiVersion: v1
kind: Pod
metadata:
name: test-pod
spec:
containers:
- name: test-pod
image: swr.cn-north-4.myhuaweicloud.com/ddn-k8s/docker.io/nginx:stable-alpine
volumeMounts:
- name: nfs-pvc
mountPath: "/usr/share/nginx/html"
volumes:
- name: nfs-pvc
persistentVolumeClaim:
claimName: test-claim
