[ELK] k8s에 ElasticSearch 설치하기 (오류 해결과정 포함)

단일 노드로 구축

출처: https://ssnotebook.tistory.com/entry/ELK-Kubernetes-환경에서-Elasticsearch-Kibana-설치 [S_Notebook:티스토리]

1. ElasticSearch 세팅

아래의 yaml 파일을 생성한다. elastic-search-deployment.yaml

참고로 엘라스틱서치를 devops-tools 네임스페이스에 배포하려면 YAML 파일의 각 리소스(예: StatefulSet 및 Service)에 namespace 필드를 추가해야 합니다. namespace 필드를 추가하여 해당 리소스가 특정 네임스페이스에 배포되도록 지정할 수 있습니다.
여기서 문제는 ssl인증을 적용한것이다.

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: elasticsearch # StatefulSet의 이름
  namespace: devops-tools  # 네임스페이스 지정
  labels:
    app: elasticsearch # 라벨 설정
spec:
  replicas: 1 # 복제본 수
  selector:
    matchLabels:
      app: elasticsearch # Pod를 선택할 라벨
  template:
    metadata:
      labels:
        app: elasticsearch # Pod 템플릿의 라벨
    spec:
      containers:
      - name: elasticsearch # 컨테이너 이름
        image: elastic/elasticsearch:8.13.3 # 사용할 이미지 및 버전
        env:
        - name: discovery.type
          value: single-node # 노드 발견 유형 설정
        - name: ELASTIC_PASSWORD
          value: "elk" # 엘라스틱서치 비밀번호 설정
        - name: xpack.security.enabled
          value: "true" # xpack 보안 활성화
        - name: xpack.security.transport.ssl.enabled
          value: "true" # xpack 보안 전송 SSL 활성화
        - name: xpack.security.enrollment.enabled
          value: "true" # 자동 등록 활성화
        - name: ES_JAVA_OPTS
          value: "-Xms500m -Xmx1000m" # JVM 옵션 설정
        ports:
        - containerPort: 9200 # 컨테이너에서 노출할 포트
        - containerPort: 9300 # 컨테이너에서 노출할 포트
        volumeMounts:
        - name: elastic-data
          mountPath: /usr/share/elasticsearch/data # 데이터 디렉토리를 마운트할 경로
      volumes:
      - name: elastic-data
        emptyDir:
          sizeLimit: 5Gi # 볼륨의 최대 크기
---
# Kibana 또는 타 플랫폼에서 사용할 서비스

apiVersion: v1
kind: Service
metadata:
  labels:
    app: elasticsearch
  name: elasticsearch-master # 서비스의 이름
  namespace: devops-tools  # 네임스페이스 지정
spec:
  ports:
  - name: elasticsearch
    port: 9200 # http 서비스가 노출하는 포트
    protocol: TCP
    targetPort: 9200 # 서비스가 대상으로 하는 컨테이너 포트
  - name: elasticsearch-ssl
    port: 9300 # ssl 서비스가 노출하는 포트
    protocol: TCP
    targetPort: 9300 # 서비스가 대상으로 하는 컨테이너 포트
  selector:
    app: elasticsearch # 서비스가 선택하는 Pod를 식별하는 라벨
  type: ClusterIP # 서비스 유형

yaml 파일 생성 후 배포까지 진행했다.

Untitled

get pod로 정보를 확인한다.

Untitled

???? 오류가 발생한다.

{"@timestamp":"2024-05-03T15:09:08.247Z", "log.level":"ERROR", "message":"fatal exception while booting Elasticsearch", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.bootstrap.Elasticsearch","elasticsearch.node.name":"elasticsearch-0","elasticsearch.cluster.name":"docker-cluster","error.type":"org.elasticsearch.ElasticsearchSecurityException","error.message":"invalid SSL configuration for xpack.security.transport.ssl - server ssl configuration requires a key and certificate, but these have not been configured; you must set either [xpack.security.transport.ssl.keystore.path], or both [xpack.security.transport.ssl.key] and [xpack.security.transport.ssl.certificate]","error.stack_trace":"org.elasticsearch.ElasticsearchSecurityException: invalid SSL configuration for xpack.security.transport.ssl - server ssl configuration requires a key and certificate, but these have not been configured; you must set either [xpack.security.transport.ssl.keystore.path], or both [xpack.security.transport.ssl.key] and [xpack.security.transport.ssl.certificate]\\n\\tat [email protected]/org.elasticsearch.xpack.core.ssl.SSLService.throwExceptionForMissingKeyMaterial(SSLService.java:686)\\n\\tat [email protected]/org.elasticsearch.xpack.core.ssl.SSLService.validateServerConfiguration(SSLService.java:642)\\n\\tat [email protected]/org.elasticsearch.xpack.core.ssl.SSLService.loadSslConfigurations(SSLService.java:627)\\n\\tat [email protected]/org.elasticsearch.xpack.core.ssl.SSLService.<init>(SSLService.java:160)\\n\\tat [email protected]/org.elasticsearch.xpack.core.XPackPlugin.createSSLService(XPackPlugin.java:493)\\n\\tat [email protected]/org.elasticsearch.xpack.core.XPackPlugin.createComponents(XPackPlugin.java:323)\\n\\tat [email protected]/org.elasticsearch.node.NodeConstruction.lambda$construct$12(NodeConstruction.java:804)\\n\\tat [email protected]/org.elasticsearch.plugins.PluginsService.lambda$flatMap$1(PluginsService.java:253)\\n\\tat java.base/java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:273)\\n\\tat java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)\\n\\tat java.base/java.util.AbstractList$RandomAccessSpliterator.forEachRemaining(AbstractList.java:722)\\n\\tat java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)\\n\\tat java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)\\n\\tat java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:575)\\n\\tat java.base/java.util.stream.AbstractPipeline.evaluateToArrayNode(AbstractPipeline.java:260)\\n\\tat java.base/java.util.stream.ReferencePipeline.toArray(ReferencePipeline.java:616)\\n\\tat java.base/java.util.stream.ReferencePipeline.toArray(ReferencePipeline.java:622)\\n\\tat java.base/java.util.stream.ReferencePipeline.toList(ReferencePipeline.java:627)\\n\\tat [email protected]/org.elasticsearch.node.NodeConstruction.construct(NodeConstruction.java:804)\\n\\tat [email protected]/org.elasticsearch.node.NodeConstruction.prepareConstruction(NodeConstruction.java:262)\\n\\tat [email protected]/org.elasticsearch.node.Node.<init>(Node.java:192)\\n\\tat [email protected]/org.elasticsearch.bootstrap.Elasticsearch$2.<init>(Elasticsearch.java:237)\\n\\tat [email protected]/org.elasticsearch.bootstrap.Elasticsearch.initPhase3(Elasticsearch.java:237)\\n\\tat [email protected]/org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:74)\\n"}

로그 json을 정리해보자 (https://jsonformatter.curiousconcept.com/#)

{
   "@timestamp":"2024-05-03T15:09:08.247Z",
   "log.level":"ERROR",
   "message":"fatal exception while booting Elasticsearch",
   "ecs.version":"1.2.0",
   "service.name":"ES_ECS",
   "event.dataset":"elasticsearch.server",
   "process.thread.name":"main",
   "log.logger":"org.elasticsearch.bootstrap.Elasticsearch",
   "elasticsearch.node.name":"elasticsearch-0",
   "elasticsearch.cluster.name":"docker-cluster",
   "error.type":"org.elasticsearch.ElasticsearchSecurityException",
   "error.message":"invalid SSL configuration for xpack.security.transport.ssl - server ssl configuration requires a key and certificate, but these have not been configured; you must set either [xpack.security.transport.ssl.keystore.path], or both [xpack.security.transport.ssl.key] and [xpack.security.transport.ssl.certificate]",
   "error.stack_trace":"org.elasticsearch.ElasticsearchSecurityException: invalid SSL configuration for xpack.security.transport.ssl - server ssl configuration requires a key and certificate, but these have not been configured; you must set either [xpack.security.transport.ssl.keystore.path], or both [xpack.security.transport.ssl.key] and [xpack.security.transport.ssl.certificate]\\n\\tat [email protected]/org.elasticsearch.xpack.core.ssl.SSLService.throwExceptionForMissingKeyMaterial(SSLService.java:686)\\n\\tat [email protected]/org.elasticsearch.xpack.core.ssl.SSLService.validateServerConfiguration(SSLService.java:642)\\n\\tat [email protected]/org.elasticsearch.xpack.core.ssl.SSLService.loadSslConfigurations(SSLService.java:627)\\n\\tat [email protected]/org.elasticsearch.xpack.core.ssl.SSLService.<init>(SSLService.java:160)\\n\\tat [email protected]/org.elasticsearch.xpack.core.XPackPlugin.createSSLService(XPackPlugin.java:493)\\n\\tat [email protected]/org.elasticsearch.xpack.core.XPackPlugin.createComponents(XPackPlugin.java:323)\\n\\tat [email protected]/org.elasticsearch.node.NodeConstruction.lambda$construct$12(NodeConstruction.java:804)\\n\\tat [email protected]/org.elasticsearch.plugins.PluginsService.lambda$flatMap$1(PluginsService.java:253)\\n\\tat java.base/java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:273)\\n\\tat java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)\\n\\tat java.base/java.util.AbstractList$RandomAccessSpliterator.forEachRemaining(AbstractList.java:722)\\n\\tat java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)\\n\\tat java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)\\n\\tat java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:575)\\n\\tat java.base/java.util.stream.AbstractPipeline.evaluateToArrayNode(AbstractPipeline.java:260)\\n\\tat java.base/java.util.stream.ReferencePipeline.toArray(ReferencePipeline.java:616)\\n\\tat java.base/java.util.stream.ReferencePipeline.toArray(ReferencePipeline.java:622)\\n\\tat java.base/java.util.stream.ReferencePipeline.toList(ReferencePipeline.java:627)\\n\\tat [email protected]/org.elasticsearch.node.NodeConstruction.construct(NodeConstruction.java:804)\\n\\tat [email protected]/org.elasticsearch.node.NodeConstruction.prepareConstruction(NodeConstruction.java:262)\\n\\tat [email protected]/org.elasticsearch.node.Node.<init>(Node.java:192)\\n\\tat [email protected]/org.elasticsearch.bootstrap.Elasticsearch$2.<init>(Elasticsearch.java:237)\\n\\tat [email protected]/org.elasticsearch.bootstrap.Elasticsearch.initPhase3(Elasticsearch.java:237)\\n\\tat [email protected]/org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:74)\\n"
}

해당 메시지에서 SSL 구성에 관한 오류가 다음과 같이 기술되어 있다.

"error.message":"invalid SSL configuration for xpack.security.transport.ssl - server ssl configuration requires a key and certificate, but these have not been configured; you must set either [xpack.security.transport.ssl.keystore.path], or both [xpack.security.transport.ssl.key] and [xpack.security.transport.ssl.certificate]"