如何在kubernetes上安装kerberised NFS？

有没有办法将Kerberos认证的NFS服务器作为创建pod的用户装载到Kubernetes pod中

我们使用FreeIPA进行用户管理，我们有一个Kubernetes群集设置，用于培训我们的深度学习模型。我们的数据位于NFS上，该NFS使用Kerberos进行身份验证。以下是我们正在努力实现的目标：

在吊舱中安装kerberized NFS

NFS的装载权限应与部署pod的用户的权限相同

用户不能执行其他用户部署的POD并访问其数据

我们将GKE用于kubernetes，我们的NFS位于同一VPC中。

我就是这样做的

对于我的方法，您需要：

• 工人们加入了克尔贝索王国。（NFS由工作进程和主机TGT装载）
• 密钥表存储在安全的地方，我使用vault和vault代理
• 将管理凭据的Sidecar容器
• 服务器和共享KCM套接字的POD之间共享KCM，避免其他POD访问存储的TGT
• Krb5存储为命名空间中的configmap
• NFS导出数据中的适当权限，由IPA中存在的用户拥有，并且哪些UID/GID将用于运行pod中的容器

采取这种做法的原因是：

• 保险库中的安全信息
• 从存储器中删除并仅存储在存储器中的秘密
• 每个容器管理一个进程或任务

考虑到所有这些，您首先要写下krb5侧车的Dockerfile。 命令：[“/bin/sh”] 参数：[“-c”，“/usr/bin/sleep 3600000”]

这是管理的入口点脚本

键表加载到KCM内存中

从密钥表的共享

/vault/secrets

文件中删除

根据您的Krb5策略续订kerberos票证

当然，您需要为部署创建PersistentVolumes和PersistentVolumeClaims

持续音量

apiVersion: v1
kind: PersistentVolume
metadata:
  name: NFS-vol
spec:
  volumeMode: Filesystem
  accessModes:
    - ReadWriteMany
  persistentVolumeReclaimPolicy: Recycle
  storageClassName: slow
  mountOptions:
    - sec=krb5
  nfs:
    path: /exports
    server: nfs.server.test

PersistentVolumeClaim：

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: nfsvol
spec:
  storageClassName: manual
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 3Gi

最后是部署：

apiVersion: apps/v1
kind: Deployment
metadata:
  name: deployment-user
spec:
  selector:
    matchLabels:
      test: test
  template:
    metadata:
      labels:
        test: test
      annotations:
        vault.hashicorp.com/agent-inject: 'true'
        vault.hashicorp.com/agent-inject-secret-userKeytab: 'user/keytabs/user'
        vault.hashicorp.com/role: 'nfs'
        vault.hashicorp.com/ca-cert: 'certs/ca.crt'
        vault.hashicorp.com/tls-secret: 'tls-ca'
        vault.hashicorp.com/agent-pre-populate-only: "true"
    spec:
      securityContext:
        # Here we defined the user uid, this user must be present in the NFS server
        runAsUser: 2500
        runAsGroup: 2500
      # This may be needed or not depending on your DNS setup
      hostAliases:
        - ip: "192.168.111.130"
          hostnames:
            - "IPA"
            - "IPA.server"
        - ip: "192.168.111.131"
          hostnames:
            - "nfs"
            - "nfs.serer"
      restartPolicy: Always
      volumes:
      - name: nfs-user
        persistentVolumeClaim:
          claimName: nfs-vol
      - name: krb5
        configMap:
          name: keos-kerberos-config
      - name: kcmsocket
        hostPath:
          path: /var/run/.heim_org.h5l.kcm-socket
          type: File
      containers:
      - name: krb5-sidecar
        image: krb5-sidecar:0.1.0
        env:
        - name: KRB5CCNAME
          value: "KCM:"
        - name: USERNAME
          value: user
        - name: REALM
          value: server
        volumeMounts:
        - name: krb5
          mountPath: "/etc/krb5.conf"
          subPath: "krb5.conf"
        - name: kcmsocket
          mountPath: "/var/run/.heim_org.h5l.kcm-socket"
        lifecycle:
          preStop:
            exec:
              command: ["/usr/bin/kdestroy"]
      - name: mount-nfs-container
        image: nfs-centos:0.2.0
        env:
        - name: KRB5CCNAME
          value: "KCM:"
        volumeMounts:
        - name: nfs-user
          mountPath: "/nfs"
        - name: krb5
          mountPath: "/etc/krb5.conf"
          subPath: "krb5.conf"
        - name: kcmsocket
          mountPath: "/var/run/.heim_org.h5l.kcm-socket"

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: nfsvol
spec:
  storageClassName: manual
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 3Gi

apiVersion: apps/v1
kind: Deployment
metadata:
  name: deployment-user
spec:
  selector:
    matchLabels:
      test: test
  template:
    metadata:
      labels:
        test: test
      annotations:
        vault.hashicorp.com/agent-inject: 'true'
        vault.hashicorp.com/agent-inject-secret-userKeytab: 'user/keytabs/user'
        vault.hashicorp.com/role: 'nfs'
        vault.hashicorp.com/ca-cert: 'certs/ca.crt'
        vault.hashicorp.com/tls-secret: 'tls-ca'
        vault.hashicorp.com/agent-pre-populate-only: "true"
    spec:
      securityContext:
        # Here we defined the user uid, this user must be present in the NFS server
        runAsUser: 2500
        runAsGroup: 2500
      # This may be needed or not depending on your DNS setup
      hostAliases:
        - ip: "192.168.111.130"
          hostnames:
            - "IPA"
            - "IPA.server"
        - ip: "192.168.111.131"
          hostnames:
            - "nfs"
            - "nfs.serer"
      restartPolicy: Always
      volumes:
      - name: nfs-user
        persistentVolumeClaim:
          claimName: nfs-vol
      - name: krb5
        configMap:
          name: keos-kerberos-config
      - name: kcmsocket
        hostPath:
          path: /var/run/.heim_org.h5l.kcm-socket
          type: File
      containers:
      - name: krb5-sidecar
        image: krb5-sidecar:0.1.0
        env:
        - name: KRB5CCNAME
          value: "KCM:"
        - name: USERNAME
          value: user
        - name: REALM
          value: server
        volumeMounts:
        - name: krb5
          mountPath: "/etc/krb5.conf"
          subPath: "krb5.conf"
        - name: kcmsocket
          mountPath: "/var/run/.heim_org.h5l.kcm-socket"
        lifecycle:
          preStop:
            exec:
              command: ["/usr/bin/kdestroy"]
      - name: mount-nfs-container
        image: nfs-centos:0.2.0
        env:
        - name: KRB5CCNAME
          value: "KCM:"
        volumeMounts:
        - name: nfs-user
          mountPath: "/nfs"
        - name: krb5
          mountPath: "/etc/krb5.conf"
          subPath: "krb5.conf"
        - name: kcmsocket
          mountPath: "/var/run/.heim_org.h5l.kcm-socket"