将在 Docker 上启动的 Elasticsearch 快照保存到 S3 的方法

3 年 ago
逸, 科
2 minutes

Contents

    • dockerfile

 

    • docker-composefile

 

    • aws認証設定

 

    • snapshotrepositoryの登録

 

    snapshotの作成

Dockerfile -> Docker 文件

FROM docker.elastic.co/elasticsearch/elasticsearch:7.6.0
RUN bin/elasticsearch-plugin install --batch repository-s3

Docker Compose 文件

version: '3.8'

services:
  localstack:
    container_name: localstack
    image: localstack/localstack:0.13.3
    environment:
      SERVICES: s3
      EDGE_PORT: 4566
    ports:
      - '4566:4566'

  es01:
    build: ./elasticsearch
    volumes:
      - /usr/share/elasticsearch/data
    environment:
      - discovery.type=single-node
      - cluster.name=docker-cluster
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
    ulimits:
      memlock:
        soft: -1
        hard: -1
    ports:
      - '9200:9200'

AWS认证设置

进入容器内部

docker container exec -it container_name bash

注册并确认访问密钥与秘密密钥

elasticsearch-keystore add s3.client.default.access_key

elasticsearch-keystore add s3.client.default.secret_key

bin/elasticsearch-keystore list

重新加载节点中的keystore。

curl -X POST -v "localhost:9200/_nodes/reload_secure_settings"

注册快照仓库

首先在S3上创建一个存储桶，然后发送以下请求。

import requests

url = 'http://localhost:9200/_snapshot/snapshotrepository_name'
headers ={'Content-Type': 'application/json'}
payload = {
  "type": "s3",
  "settings": {
    "bucket": "bucket_name",
    "endpoint" : "endpoint_name",
    "protocol": "http",
    "path_style_access": "true"
  }
}
response = requests.put(url,headers=headers,json=payload)
print(response.text)

拍摄快照

import requests

url = 'http://localhost:9200/_snapshot/repository_name/snapshot_name'
response = requests.put(url)

print(response.text)

备考

如果不重新加载keystore，将会出现下面的错误。

{"error":{"root_cause":[{"type":"repository_verification_exception","reason":"[repository_name] path  is not accessible on master node"}],"type":"repository_verification_exception","reason":"[repository_name] path  is not accessible on master node","caused_by":{"type":"i_o_exception","reason":"Unable to upload object [tests-rQf2zvkRSFmXhH4i52mbcg/master.dat] using a single upload","caused_by":{"type":"sdk_client_exception","reason":"Unable to load credentials from service endpoint","caused_by":{"type":"connect_exception","reason":"Connection refused"}}}},"status":500}
bannerAds