PangYeon`s IT Log

k8s cluster를 kubernetes admin을 사용하는게 아닌 인증서를 통한 사용자를 생성하여 k8s cluster를 사용하는 방법이다.

1. 개인키 생성

# key 생성
openssl genrsa -out pangyeon.key 2048

# csr 생성
openssl req -new -key pangyeon.key -out pangyeon.csr -subj "/CN=pangyeon"

# csr 값 조회
cat pangyeon.csr | base64 | tr -d "\n"
LS0tLS1CRUdJTiBDRVJUSUZJQ0FURSBSRVFVRVNULS0tLS0KTUlJQ1dEQ0NBVUFDQVFBd0V6RVJNQThHQTFVRUF3d0ljR0Z1WjNsbGIyNHdnZ0VpTUEwR0NTcUdTSWIzRFFFQgpBUVVBQTRJQkR3QXdnZ0VLQW9JQkFRRGpFa0FaSmhBdElhcTEzSjczREFsa1R4cUExeGdQRWx6dkxXbEVzdzdLCjBRQ29YMGdBR3JFVTlQcDNubWl2U051T0dKd0F4REN3dFQ1TThteDd2a21ZNHRPRC80ZWhVVGhQd3FhOXMwUjMKUUQ5SHlWQ0hqcm1PV1ZDempmNWJ5WkJXc3RZSjZxcGQ2ZUc2NVRlbDF0RmdNRHM4QUdsNmNiazJ4SE1WK2s2dQpNTkVsV3FnRGUwTmt1S3hLWW5OZnY5QWVRanlSek1NUm5Ia1c0cTNucFRqaXhnWTY4eFhzdHpQUlpTZ0YwVU8wCjdUdjlSRE5VdlVKQ3NBdzlrSFlObXdGWkw3SHNtcHdHV3lNdWRCVVVBQ0w3dFUyL0MwZFQ2VkpuSzBmV1VnVlMKNWtRK0NkdW5BdHVDTm8xOTdLS2hQalR5OXE3c1NnajVlcmltNEtGNkdIN2hBZ01CQUFHZ0FEQU5CZ2txaGtpRwo5dzBCQVFzRkFBT0NBUUVBMnJxbHFGTHJjVUhkRU5qclQvVkxOUE5IVEFNMElyTEROVlVsWFRBSDA5S1JQZCsvCkNyUkF3TVEzOXJqRjE5R213Y1czZHNnSkRXR2NacXcxV1JTRlplQy9UaklJZXUzM0EvRm55TC8vYXVUZWg5REEKaEROcURTdnZpNEQzbmwyOTVGNFVMdzJVZG9QbTRyaWxzUHh0eEU3U1dUSFlpYm90QmZnSjJ2NUExb085UU5JeAo4NUE0eHJBVEEvR1EycWZhSlRhZ2pSUGQrNUVHK0tkTm85cnN0Vjl2Y3ZyTkdzS2F0YXpYL2dCTEhoWW9EMzVXCjJ0UFBCSUhmSUU4UUtmakJtZGJKQzlMY2o2UU00V2FDOVJRd293KzFEbTMwZnhmRjWVMVUszckVac3MKU21xVFphZFpWek9XWDdEQnVzM1A5V3lWOExU

2. CertificateSigningRequest 생성

# CertificateSigningReques 생성
cat <<EOF | kubectl apply -f -
apiVersion: certificates.k8s.io/v1
kind: CertificateSigningRequest
metadata:
  name: pangyeon
spec:
  request: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURSBSRVFVRVNULS0tLS0KTUlJQTUEwR0NTcUdTSWIzUVVBQTRJQkR3QXdnZ0VLQW9JQkFRRGpFa0FaSmhBdElhcTEzSjczREFsa1R4cUExeGdQRWx6dkxXbEVzdzdLCjBRQ29YMGdBR3JFVTlQcDNubWl2U051T0dKd0F4REN3dFQ1TThteDd2a21ZNHRPRC80ZWhVVGhQd3FhOXMwUjMKUUQ5SHlWQ0hqcm1PV1ZDempmNWJ5WkJXc3RZSjZxcGQ2ZUc2NVRlbDF0RmdNRHM4QUdsNmNiazJ4SE1WK2s2dQpNTkVsV3FnRGUwTmt1S3hLWW5OZnY5QWVRanlSek1NUm5Ia1c0cTNucFRqaXhnWTY4eFhzdHpQUlpTZ0YwVU8wCjdUdjlSRE5VdlVKQ3NBdzlrSFlObXdGWkw3SHNtcHdHV3lNdWRCVVVBQ0w3dFUyL0MwZFQ2VkpuSzBmV1VnVlMKNWtRK0NkdW5BdHVDTm8xOTdLS2hQalR5OXE3c1NnajVlcmltNEtGNkdIN2hBZ01CQUFHZ0FEQU5CZ2txaGtpRwo5dzBCQVFzRkFBT0NBUUVBMnJxbHFGTHJjVUhkRU5qclQvVkxOUE5IVEFNMElyTEROVlVsWFRBSDA5S1JQZCsvCkNyUkF3TVEzOXJqRjE5R213Y1czZHNnSkRXR2NacXcxV1JTRlplQy9UaklJZXUzM0EvRm55TC8vYXVUZWg5REEKaEROcURTdnZpNEQzbmwyOTVGNFVMdzJVZG9QbTRyaWxzUHh0eEU3U1dUSFlpYm90QmZnSjJ2NUExb085UU5JeAo4NUE0eHJBVEEvR1EycWZhSlRhZ2pSUGQrNUVHK0tkTm85cnN0Vjl2Y3ZyTkdzS2F0YXpYL2dCTEhoWW9EMzVXCjJ0UFBCSUhmSUU4UUtmakJtZGJKQzlMY2o2UU00V2FDOVJRd293KzFEbTMwZnhmRjdUN05XdWVMVUszckVac3MKU21xVFphZFpWek9XWDdEQnVzM1A5V3lWOExUOFZKVG1H
  signerName: kubernetes.io/kube-apiserver-client
  expirationSeconds: 86400  # one day
  usages:
  - client auth
EOF

# 생성한 CertificateSigningRequest 조회
kubectl get csr
NAME       AGE   SIGNERNAME                            REQUESTOR          REQUESTEDDURATION   CONDITION
pangyeon   48s   kubernetes.io/kube-apiserver-client   kubernetes-admin   24h                 Pending

# 승인 요청
kubectl certificate approve pangyeon

# 승인확인
kubectl get csr
NAME       AGE     SIGNERNAME                            REQUESTOR          REQUESTEDDURATION   CONDITION
pangyeon   5m17s   kubernetes.io/kube-apiserver-client   kubernetes-admin   24h                 Approved,Issued

# 승인한 인증서 추출
kubectl get csr pangyeon -o jsonpath='{.status.certificate}'| base64 -d > pangyeon.crt

3. Cluster Role 생성

해당 예시는 모든 자원에 대해 View 권한 부여

# CluserRole, CluserRolebinding 생성
vi role.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: read-only
rules:
- apiGroups: [""]
  resources: ["*"]
  verbs: ["get", "list"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: read-only-binding
subjects:
- kind: User
  name: pangyeon
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: ClusterRole
  name: read-only
  apiGroup: rbac.authorization.k8s.io
  
# 적용
kubectl apply -f role.yaml

4. Contexts 생성

# Credentials 생성
kubectl config set-credentials pangyeon --client-key=pangyeon.key --client-certificate=pangyeon.crt --embed-certs=true

# Context 추가
kubectl config set-context pangyeon --cluster=kubernetes --user=pangyeon

# 조회
kubectl config get-contexts
CURRENT   NAME                          CLUSTER      AUTHINFO           NAMESPACE
*         kubernetes-admin@kubernetes   kubernetes   kubernetes-admin   
          pangyeon                      kubernetes   pangyeon
          
# 사용
kubectl config use-context pangyeon
Switched to context "pangyeon".

5. 테스트

조회시 인증서로 추가된 부분 확인 및 권한으로 인해 조회만 가능하고 삭제 불가

# 조회시 인증서로 계정 추가된 것 확인
cat ~/.kube/config
apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURCVENDQWUyZ0F3SUJBZ0lJSFl4OUR6MmlDUHN3RFFZSktvWklodmNOQVFFTEJRQXdGVEVUTUJFR0ExVUUKQXhNS2EzVmlaWEp1WlhSbGN6QkpoeWJCdElXSE1KRHY4WUtlQ2o0SEJ3MzlKT0E5MXJwcld0UURlUE4KYzRwWEZVMTQ5L1VZTSs2Q3hLZUtVTXphK0EyQU9vRzFMbUhzbzZYMktEYm5TRFlidFk2dDVLNFBlUmI1NXhMbAplRnZ5eDlzKzgxc0RZZGZVSlFVSlR4TEo1em9VOFlPQldhWjRrdUJObkF1TVJkbE40LzNCNnNwNkUrOXY2TDB2ClVGelpqYmhocFJyZjJ0TmZJV2hiU3Q4am9xcCtWc2hRMDZIN09WUU5nR1BwNGxxZGR5ZGJ4aGhKYWJvbnJMZ2oKTTF0elU0UFVadjFDVjh3dWRHclV4dDIxNkgwSC83SGN5UHRMWnE4MmpWOGl1Zk9FQzV5OFd5VHBBL25mcDJaNAp6RmR2ZjdpdkRaSXFhUjJQSTVScUxxeGNDOEJoQWdNQkFBR2pXVEJYTUE0R0ExVWREd0VCL3dRRUF3SUNwREFQCkJnTlZIUk1CQWY4RUJUQURBUUgvTUIwR0ExVWREZ1FXQkJRd0hCQ0NJRHJZQ29OckRIancyNXVVOSs3RTFqQVYKQmdOVkhSRUVEakFNZ2dwcmRXSmxjbTVsZEdWek1BMEdDU3FHU0liM0RRRUJDd1VBQTRJQkFRQThvZ1lQVjQrZwo5cHEwN0FZUStzbittdmNaVi93ZE9wbml3bkt1NXBBY1plWG1aQU1wOExEaUcwRWJ5RkJFdlNqWTZ6RmxIa2xmCjZNRE1QM1NPVXl0cmZ2Qnd5ZEpVdjNtOXNKOXdpYU1OY0RCekJXdlFNZ0NPQktWWXB2UTZHY05UOGEvdWV5UDYKVC9rQnNyeXhLNDV4OWMvSXRCVHVoOTdzMFgvTkthNHd2eTVJelg0YmZzVkJrVHpQdTVlRVRPZmtoWWY2aUxLNgpBV0ZiQWJWTUN0NWxsWFdxb2VtRm4wRm9ERDROdjV3ejQ0M3FmSTVRTktaQlBMUHZFTXMwMEo2RHNJY1NjOUoxCnpET3hWTVRJUk5ieUpWMk5GV2hYdXlLTUNUS05iUzgraGgvazVTVk41TUc3ZjBSVUdmSm1XV2N2SE0zS3Q5RXIKNitEREJ6aHgyZVppCi0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0K
    server: https://127.0.0.1:6443
  name: kubernetes
contexts:
- context:
    cluster: kubernetes
    user: kubernetes-admin
  name: kubernetes-admin@kubernetes
- context:
    cluster: kubernetes
    user: pangyeon
  name: pangyeon
current-context: pangyeon
kind: Config
preferences: {}
users:
- name: kubernetes-admin
  user:
    client-certificate-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURLVENDQWhHZ0F3SUJBZ0lJWUxSTnZQOFl4Qnd3RFFZSktvWklodmNOQVFFTEJRQXdGVEVUTUJFR0ExVUUKQXhNS2EzVmlaWEp1WlhSbGN6QWVGdzB5TkRFd016QXhORFF4TWpkYUZ3MHlOVEV3TXpBeE5EUTJNamRhTUR3eApIekFkQmdOVkJBb1RGbXQxWW1WaFpHMDZZMngxYzNSbGNpMWhaRzFwYm5NeEdUQVhCZ05WQkFNVEVHdDFZbVZ5CmJtVjBaWE10WVdSdGFXNHdnZ0VpTUEwR0NTcUdTSWIzRFFFQkFRVUFBNElCRHdBd2dnRUtBb0lCQVFDZ2xVWmkKYWFweHBaYWdTT2wvcnpTckkzaWJFbUFpa2cyMVNqUGVMMXQ1U21jeEVINnNjUVZTcjV5REd4WUI1ZElLa0lrNwpSN0oyUGVlQi92RFFmbXdrUEQrUlE2bnFkS1QyM0c3R09QeDJla2pwcnJ2ejd4T0xjNGJDVURPTWVGcXl4TjVqClIrZTVSM2w1S3NQU2NxUUs4cTlDSUNVUXJ2NVplYVV1TW96NnFWQXZ3ZWtwZzNZanM2STlwWjJrWTEwTzRwYVAKZHRJTHcyU1NYSUR6YWNCa2txVy9SM1ZUR3ZNdHlqU1Nwak9YMVBwNkEwbHJOaUl1RzA5cDhOanI2U1VsMlptSwpvWGZweXBRV2p2UUJ4aUVXTVB1M2lqcTY3cmhQUVlhTWFmMzJjWmRhRHpFNHBqaTMxS2habWxUTCtlZS9YanhoClZwcWY2MmZIUFVyb01HMkxBZ01CQUFHalZqQlVNQTRHQTFVZER3RUIvd1FFQXdJRm9EQVRCZ05WSFNVRUREQUsKQmdnckJnRUZCUWNEQWpBTUJnTlZIUk1CQWY4RUFqQUFNQjhHQTFVZEl3UVlNQmFBRkRBY0VJSWdPdGdLZzJzTQplUERibTVUMzdzVFdNQTBHQ1NxR1NJYjNEUUVCQ3dVQUE0SUJBUUIrSys4MGVkNWJzNDhNM09acXBtbVdhNmdiCnd5bXl2c3BCZ1JNSGNLQXNMNTNuejNOaytYcDJZRUp1R3h2TDR4RlR0RWUxalhxRzRidkowUWJQcGhtdi9JRkEKWWJHUGNQU3YzMDR2WWhLcnBPMlc3NkE2ZUtFUUFxcS9YVzFaK0VTNEhMTUFYQWtLNzZ0dXNXV3lmY3pxb2NJMwpoYTc2bklBSjA5dkR6VlJQb2VWVGV4TmdWdEYvSFR5b1B1cGFaREhBV1FtOTBPUlduQmM4dHR5Z1pQdU4rVVI0CnFRYTJqRDJySmQ2UXREMUFTT2hkOFNtTHpJTkNBYlVtdXhTZXVZVkNHSy9oUWlWZS9EYVRPL1V4VXRUZStDb1oKYmFNbHBKNk96dVFQZzRZQVBsUXFvYisvSW1hSlVablFPYzdWQWxzN3QwWFEvTDF6Mmpxb0N4OWovcW56Ci0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0K
    client-key-data: LS0tLS1CRUdJTiBSU0EgUFJJVkFURSBLRVktLS0tLQpNSUlFb3dJQkFBS0NBUUVBb0pWR1ltbX
- name: pangyeon
  user:
    client-certificate-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUMrRENDQWVDZ0F3SUJBZ0lRTno0OVVBM2h5QUlVZFY5NUVDbVdQakFOQmdrcWhraUc5dzBCQVFzRkFEQVYKTVJNd0VRWURWUVFERXdwcmRXSmxjbTVsZEdWek1CNFhEVEkwTVRFeU9URTBNVGN5TWxvWERUSTBNVEV6TURFMApNVGN5TWxvd0V6RVJNQThHQTFVRUF4TUljR0Z1WjNsbGIyNHdnZ0VpTUEwR0NTcUdTSWIzRFFFQkFRVUFBNElCCkR3QXdnZ0VLQW9JQkFRRGpFa0FaSmhBdElhcTEzSjczREFsa1R4cUExeGdQRWx6dkxXbEVzdzdLMFFDb1gwZ0EKR3JFVTlQcDNubWl2U051T0dKd0F4REN3dFQ1TThteDd2a21ZNHRPRC80ZWhVVGhQd3FhOXMwUjNRRDlIeVZDSApqcm1PV1ZDempmNWJ5WkJXc3RZSjZxcGQ2ZUc2NVRlbDF0RmdNRHM4QUdsNmNiazJ4SE1WK2s2dU1ORWxXcWdECmUwTmt1S3hLWW5OZnY5QWVRanlSek1NUm5Ia1c0cTNucFRqaXhnWTY4eFhzdHpQUlpTZ0YwVU8wN1R2OVJETlUKdlVKQ3NBdzlrSFlObXdGWkw3SHNtcHdHV3lNdWRCVVVBQ0w3dFUyL0MwZFQ2VkpuSzBmV1VnVlM1a1ErQ2R1bgpBdHVDTm8xOTdLS2hQalR5OXE3c1NnajVlcmltNEtGNkdIN2hBZ01CQUFHalJqQkVNQk1HQTFVZEpRUU1NQW9HCkNDc0dBUVVGQndNQ01Bd0dBMVVkRXdFQi93UUNNQUF3SHdZRFZSMGpCQmd3Rm9BVU1Cd1FnaUE2MkFxRGF3eDQKOE51YmxQZnV4Tll3RFFZSktvWklodmNOQVFFTEJRQURnZ0VCQUZVRXlwNU1HS29YbmUzWjBRSG56N2ZZYVd3OAoyRE14SUdhWGZnSFR5dFlMM01SUGhqYlNXQXl1d0Z5alFSRVhWNTdIMm1iYTY2YjF6eDRvcHhTUFRXRFBValFjCkJ5bkg0b1c4TUs0b1BWSXJzWU9YcUVjVlZqZUhndEcwN1VCb0lTZlhvRnc4dWEwNS9EczJmaVhQbDZXSVpwSTYKTGovY2hhd25rSG8vbnk4QTVaWTRIbVI1UjdXL09uSnlmbW9BN244c3hydzhCN2NURXdpajgxUTBLbWtVT2crNgp2MVdwQlE0MHZMRWZUSnJuczE4TitxOW94Sk5SOGNRY2ZhW
    client-key-data: LS0tLS1CRUdJTiBQUklWQVRFIEtFWS0tLS0tCk1JSUV2Z0lCQURBTkJna3Foa2lHOXcwQkFRRUZBQVNDQktnd2dnU2tBZ0VBQW9JQkFRRGpFa0FaSmhBdElhcTEKM0o3M0RBbGtUeHFBMXhnUEVsenZMV2xFc3c3SzBRQ29YMGdBR3JFVTlQcDNubWl2U051T0dKd0F4REN3dFQ1TQo4bXg3dmttWTR0T0QvNGVoVVRoUHdxYTlzMFIzUUQ5SHlWQ0hqcm1PV1ZDempmNWJ5WkJXc3RZSjZxcGQ2ZUc2CjVUZWwxdEZnTURzOEFHbDZjYmsyeEhNVitrNnVNTkVsV3FnRGUwTmt1S3hLWW5OZnY5QWVRanlSek1NUm5Ia1cKNHEzbnBUaml4Z1k2OHhYc3R6UFJaU2dGMFVPMDdUdjlSRE5VdlVKQ3NBdzlrSFlObXdGWkw3SHNtcHdHV3lNdQpkQlVVQUNMN3RVMi9DMGRUNlZKbkswZldVZ1ZTNWtRK0NkdW5BdHVDTm8xOTdLS2hQalR5OXE3c1NnajVlcmltCjRLRjZHSDdoQWdNQkFBRUNnZ0VBTzBQYktQVXZTYWc4MXdTREZQVzJTZEQvbU5zSzgzd1dkM0tCeENWNzJlR2MKNjFVYkJMUHl2Z2FHak12eWhMVmVZSUw1ekpWb0krYmFJTmt4Q1VjTURIUS9RbmRpSGUrRjVBTm80NkF6WVhDSwpVNkV1ZklMNjJUVmtnOGl1dDZRdklRSENMWXBxOXVJQlFYZHNBOFBDbC9sZXJIVnJFa00yVlI0Rzc1aUtDcHBECk9hSzB0VWJPWnNkQnlHOWg2eWF0Wk90cVhOMktKSzVJK0J4dVpnVHMxUGhENHNqSnBBRUxhb3M4dHFicG56cDEKNDZYZUtjTlU1Ky9FSHBwaEdTdEExRzdLRDBaSWVHRmZ6SEdFUmk3aXVhZzEvV1hjdk9Cc3I2NXd1U3NyYU1QVApIRTNkcXNoUXdNK1dDYlQ1djBtWFMrRWdOWFN5SXRUQXpydVk4MWMxQlFLQmdRRDZjNlZSV1gyQzBISEpQWHBsCkdiaTJ2cmQwY1NseTY2VEJnc0xkYzdZYWdmQ0VTY3BBQTFuWEQrRWR4V29MS084UGl3S0ZVbUJqZFI4NVk2NXIKakdObGN2NytwRmJSZ3dXZjk5TzZsdjE0Ty80RXdUWkF3ZjRnSHlGKzRodUJScjBJL0loa0cyRmV2WmZ6RWdOWApLSExLRVJwVEJCU0h6SVZ0UEVVUk5YbWxId0tCZ1FEb0dnS1hBVGo2bzlqWWZSODNjQ1l3Njk2cUFzdmpCbTFVCkJDdnQ3ZzRPY00wN3UrWW5hKytuQm5pYUZJY0JZTzd2b3VMUGhrVGlXSlFTREhKcFIxWkNtWGFaMXFaZERjNHIKT283TldFZlFKRXAvOEdFck9nRDJ6YXZSNk9LUGl1dyt6UTE2V1hMamxkQ0lpRVc4UVN5cEM2dW9Oa0tYWUxxRApGWWhkeldWYi93S0JnUURWeGM0dkVLYWIrTldXd3Izcys3WjViWEpqbG8rZGd0dGZQUUNkU3ozOWhEbktnTDE4ClJCL3ovSjdXN1lGbFF5eENaUkhpd0h4N2lDWDlzMExXazc3bmdlOTdaTVNpRWliRDh5SXJHdVFCTTV2UGJTZWsKd0xEcnRBYkFLYmoyY0cyNzlPbHFJU0RNWUNJSm5LOXpQcGcwTjhMelp3RXJKSHdpMEJYWDZZQUtXd0tCZ0g3TApSc0xyZmc4ZVZ5WGRKS0tLZDdLZUNDUGtKekc4bnhrWXRrN2lqM2RBRkQ0ZnBkbS9VMHB4ZEl6bnplRG83VjZvCkl6T3ZiQTRpeWJFYWI1NG54RzNabkRycVVqUGZpTk9BeCtaUjVkbEZHaFhPWWFiVnB4VXN3a0tIOE16dDNhVnAKSzRXOU85QXNWYUZnb0lmNUtzYW1nMzMvTmwyd0QvUHdYWEN3OWtCTkFvR0JBSldScDk0OW1CcU1za2lwTzNhWAp5NnNuYlJ2Y3JhdmJFTUx2UHdISmRJdVg2Kzg0WGdUcHRwREJVZWpyRkhZM1FmVXA2VytnSngzaHFnOTQwV2I1CnhEM2lNblVzamhoZW4vSmtwN3Fma25SWmVGRUdTVVZ
    
# 조회
kubectl get pods
NAME                           READY   STATUS 
stdout-node-6f945d7989-6xjp2   1/1     Running 

# Read권한만 있어 삭제 불가
kubectl delete pods stdout-node-6f945d7989-6xjp2 
Error from server (Forbidden): pods "stdout-node-6f945d7989-6xjp2" is forbidden: User "pangyeon" cannot delete resource "pods" in API group "" in the namespace "default

참고 : https://kubernetes.io/docs/reference/access-authn-authz/certificate-signing-requests/#normal-user

OpenSearch Operator 설치시 인증서 설정하는 방법이다.

(가이드가 진짜 너무 너무 심하게 불친절하게 되어있음.... )

1. 먼저 자체 서명 인증서를 생성한다.

가이드에는  딱히 어떻게 뭘 생성하라는 얘기는 없으며, 인증서가 하나만 있으면 된다고 나와있으나

사실상 root-ca 와 Admin 인증서, Node 인증서가 필요하다.

또한, adminSecret 역시  tls.crt와 tls.key만 있으면 된다고 나와있지만 ca.crt 가 필요하며,

각종 필드에 대한 설명들이 많이 부족하다.

Admin인증서의 경우 OpenSearch Operator가 OpenSearch Cluster를 관리하는관리자 인증서이며,

Node 인증서의 경우 OpenSearch Node 인증서이다. Node인증서는 각 Node별로 만들어서 관리할 수 있으나,

해당 가이드는 하나만 만들어서 공유하는 방법이다.

subjectAltName에 들어가는 내용들은 OpenSearch cluster 설치시 포함되는 metadata의 name기준으로 만들어야하며,

OpenSearch Cluster가 생성되면서 뒤에 masters가 붙기 때문에 masters는 반드시 포함한다.

# root ca 생성
openssl genrsa -out root-ca-key.pem 2048
openssl req -new -x509 -sha256 -key root-ca-key.pem -subj "/CN=opensearch" -out root-ca.pem -days 3650

# OpenSearch Operator가 OpenSearch Cluster를 관리하는 관리자 인증서
openssl genrsa -out admin-key-temp.pem 2048
openssl pkcs8 -inform PEM -outform PEM -in admin-key-temp.pem -topk8 -nocrypt -v1 PBE-SHA1-3DES -out admin-key.pem
openssl req -new -key admin-key.pem -subj "/CN=opensearch-admin" -out admin.csr
openssl x509 -req -in admin.csr -CA root-ca.pem -CAkey root-ca-key.pem -CAcreateserial -sha256 -out admin.pem -days 3650

# Node간 DNS 통신을 위해 subjectAltName 생성
echo 'subjectAltName=DNS:opensearch-cluster-masters,DNS:opensearch-cluster-masters.opensearch,DNS:opensearch-cluster-masters.opensearch.svc,DNS:opensearch-cluster-masters.opensearch.svc.cluster.local' > san.ext

# OpenSearch Node간 인증서
openssl genrsa -out node-key-temp.pem 2048
openssl pkcs8 -inform PEM -outform PEM -in node-key-temp.pem -topk8 -nocrypt -v1 PBE-SHA1-3DES -out node-key.pem
openssl req -new -key node-key.pem -subj "/CN=opensearch-cluster-masters" -out node.csr
openssl x509 -req -in node.csr -CA root-ca.pem -CAkey root-ca-key.pem -CAcreateserial -sha256 -out node.pem -days 3650 -extfile san.ext

# OpenSearch yaml에 등록할 인증서 파일들 secret으로 생성
kubectl create secret generic opensearch-cert-admin --from-file=ca.crt=root-ca.pem --from-file=tls.crt=admin.pem --from-file=tls.key=admin-key.pem -n opensearch
kubectl create secret generic opensearch-cert-node --from-file=ca.crt=root-ca.pem --from-file=tls.crt=node.pem --from-file=tls.key=node-key.pem -n opensearch

2. OpenSearch Cluster를 생성할 yaml파일 작성

securityConfig 에는 OpenSearch Operator가 OpenSearch Cluster를 관리하는관리자 인증서를 추가해야하며,

transport secret과 http secret dashboard의 tls secret에는 Node 인증서를 추가하면된다.

그리고 metadata name과 위에 생성한 subjectAltName에 포함된 DNS의 이름과 맞춰주어야 한다.

nodesDn은 root-ca에 생성한 subj 를 입력하고 wildcard를 추가해도된다.

adminDn은 관리자 인증서 생성할 때 입력하였던 subj를 추가한다.

apiVersion: opensearch.opster.io/v1
kind: OpenSearchCluster
metadata:
  name: opensearch-cluster
  namespace: opensearch
spec:
  security:
    config:
      adminSecret:
        name: opensearch-cert-admin
    tls:
       http:
         generate: false
         secret:
           name: opensearch-cert-node
       transport:
         generate: false
         perNode: false
         secret:
           name: opensearch-cert-node
         nodesDn: ["CN=opensearch-cluster-masters*"]
         adminDn: ["CN=opensearch-admin"]
  general:
    httpPort: 9200
    serviceName: my-first-cluster
    version: 2.14.0
    pluginsList: ["repository-s3"]
    drainDataNodes: true
  dashboards:
    tls:
      enable: true
      generate: false
      secret:
        name: opensearch-cert-node
    version: 2.14.0
    enable: true
    replicas: 1
    resources:
      requests:
         memory: "512Mi"
         cpu: "200m"
      limits:
         memory: "512Mi"
         cpu: "200m"
  nodePools:
    - component: masters
      replicas: 3
      resources:
         requests:
            memory: "4Gi"
            cpu: "1000m"
         limits:
            memory: "4Gi"
            cpu: "1000m"
      roles:
        - "data"
        - "cluster_manager"
      persistence:
         emptyDir: {}

이후 해당 yaml파일을 적용하면 문제없이 생성된다.

참고로 tls transport 는 반드시 인증서를 적용해주어야 한다.(필수임)

Containerd 에서 인증서 에러 발생시 인증서가 있는 경우와 없는 경우 설정하는 방법이다.

참고로 containerd 를 설정했다고 ctr 명령어 까지 같이 적용되는건 아니다. crictl 만 적용된다.

ctr 의  경우 인증서가 있으면 --tlscacert, --tlscrt, --tlskey 3개옵션을 적용하여 사용하면 되며

인증서가 없을 경우 --skip-verify 옵션을 사용하면된다.

1. containerd 설정파일을 수정한다.

$ vi /etc/containerd/config.toml

[plugins."io.containerd.grpc.v1.cri".registry]
   config_path = "/etc/containerd/certs.d"

2. certs.d 폴더를 생성한다.

$ cd /etc/containerd/

$ mkdir certs.d

3. 내부에 인증서 연결할 서버 폴더생성

$ cd certs.d

$ mkdir registry.wky.kr

$ cd registry.wky.kr

$ pwd
/etc/containerd/certs.d/registry.wky.kr

4. hosts.toml 파일 생성 

인증서가 있는 경우.

$ vi hosts.toml

server = "https://registry.wky.kr"

[host."https://registry.wky.kr"]
  capabilities = ["pull", "resolve"]
  # pemfile 이 있는 경로 및 파일 입력
  ca = pemfile.pem
  # crt, key, pem 파일이 있는 경로 및 파일 입력
  client = [[crtfile.crt, keyfile.key], [pemfile.pem]]

인증서가 없는 경우

$ vi hosts.toml

server = "https://registry.wky.kr"

[host."https://registry.wky.kr"]
  capabilities = ["pull", "resolve"]
  skip_verify = true

참고 : https://github.com/containerd/containerd/blob/main/docs/hosts.md