參考自文檔:
https://www.cnblogs.com/balloon72/p/13177872.html elasticsearch和kibana安裝
https://www.cnblogs.com/fuguang/p/13745336.html monstache同步數(shù)據(jù)

1.ElasticSearch和kibana 安裝

準備配置文件

mkdir -p /mydata/elasticsearch/config
mkdir -p /mydata/elasticsearch/data
mkdir -p /mydata/kibana/config
mkdir -p /mydata/monstache-conf
echo "http.host: 0.0.0.0" >> /mydata/elasticsearch/config/elasticsearch.yml
echo "http.cors.enabled: true" >> /mydata/elasticsearch/config/elasticsearch.yml
echo "http.cors.allow-origin: \"*\"" >> /mydata/elasticsearch/config/elasticsearch.yml
#2022-5-7新增以下兩行 開啟安全驗證 
echo "xpack.security.enabled: true" >> /mydata/elasticsearch/config/elasticsearch.yml
echo "xpack.security.transport.ssl.enabled: true" >> /mydata/elasticsearch/config/elasticsearch.yml

chmod 777 /mydata/elasticsearch/config
chmod 777 /mydata/kibana/config
chmod 777 /mydata/elasticsearch/data
chmod 777 /mydata/monstache-conf

編輯：/mydata/kibana/config/kibana.yml

elasticsearch.hosts: http://elasticsearch:9200
server.host: "0.0.0.0"
server.name: kibana
xpack.monitoring.ui.container.elasticsearch.enabled: true
#2022-5-7新增以下兩行 開啟安全驗證 
elasticsearch.username: "elastic"  # es賬號
elasticsearch.password: "******"   # es密碼
i18n.locale: zh-CN

編輯：/mydata/monstache-conf/monstache.config.toml文件內(nèi)容：

# connectionn settings

# connect to MongoDB using the following URL
2022-5-12修改以下一行密碼內(nèi)容 
mongo-url = "mongodb://root:******@192.168.3.208:27017,192.168.3.208:27018/nfy-csia?slaveOk=true&write=1&readPreference=secondaryPreferred&connectTimeoutMS=300000&authSource=admin&authMechanism=SCRAM-SHA-1"
# connect to the Elasticsearch REST API at the following node URLs
elasticsearch-urls = ["http://192.168.3.208:9200"]
direct-read-namespaces = ["nfy-csia.capMessage","nfy-csia.vehicleMessage"]
change-stream-namespaces = ["nfy-csia.capMessage","nfy-csia.vehicleMessage"]

# use the following user name for Elasticsearch basic auth
elasticsearch-user = "elastic"
# use the following password for Elasticsearch basic auth
#2022-5-7修改以下一行密碼內(nèi)容 
elasticsearch-password = "******"
# use 4 go routines concurrently pushing documents to Elasticsearch
elasticsearch-max-conns = 4
# propogate dropped collections in MongoDB as index deletes in Elasticsearch
dropped-collections = true
# propogate dropped databases in MongoDB as index deletes in Elasticsearch
dropped-databases = true
# in the log if you had synced previously. This just means that you are replaying old docs which are already
# in Elasticsearch with a newer version. Elasticsearch is preventing the old docs from overwriting new ones.
replay = false
# resume processing from a timestamp saved in a previous run
resume = true
index-as-update = true
# use a custom resume strategy (tokens) instead of the default strategy (timestamps)
# tokens work with MongoDB API 3.6+ while timestamps work only with MongoDB API 4.0+
resume-strategy = 0
# print detailed information including request traces
verbose = true

準備容器配置
docker-compose編排腳本新增內(nèi)容（實際追加在192.168.3.249服務器原docker-compose）：

elasticsearch:
  image: elasticsearch:7.14.2
  restart: always
  container_name: elasticsearch
  deploy:
    resources:
      limits:
        cpus: "4"
        memory: 6G
      reservations:
        memory: 2G
	environment:
    - discovery.type=single-node
	  - "ES_JAVA_OPTS=-Xms1024m -Xmx4096m"
  volumes:
    - /mydata/elasticsearch/config/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml
	  - /mydata/elasticsearch/data:/usr/share/elasticsearch/data
	  - /mydata/elasticsearch/plugins:/usr/share/elasticsearch/plugins
	ports:
    - 9200:9200
	  - 9300:9300
	networks: 
	  - csia
kibana:
  image: kibana:7.14.2
  restart: always
  container_name: es-kibana
  deploy:
    resources:
      limits:
        cpus: "1"
        memory: 500M
      reservations:
        memory: 100M
	environment:
    - ELASTICSEARCH_HOSTS=http://elasticsearch:9200
  volumes:
    - /mydata/kibana/config/kibana.yml:/usr/share/kibana/config/kibana.yml
  ports:
    - 5601:5601
	depends_on: 
	  - elasticsearch
	networks: 
	  - csia

#2022-5-7新增密碼設置步驟：
es啟動后進入容器設置密碼：

# 進入容器
docker exec -it elasticsearch /bin/bash
# 設置密碼-手動設置密碼 會有多個用戶需要設置，如下圖
elasticsearch-setup-passwords interactive
# 完成后測試訪問
curl 127.0.0.1:9200 -u elastic:******

如圖為以上多個用戶配置密碼，最主要是第一個elastic用戶設為 ****** ，后面用戶密碼相同即可，不重要。

驗證：http://IP:5601/app/kibana

2.mongoDB同步配置和 工具monstache安裝：

在已有的mongoDB容器追加一行，并重啟：

command: mongod --replSet repset

docker-compose添加mongoDB副本集容器：

mongo-replSet:
  image: mongo:4.1.13
  restart: always
  deploy:
    resources:
      limits:
        cpus: "2"
        memory: 4G
      reservations:
        memory: 200M
  logging:
    driver: "json-file"
    options:
      max-size: "500m"
  privileged: true
  ports:
    - 27018:27017
  networks:
    - csia
  command: mongod --replSet repset

mongo兩個容器都運行后，進入其中一個執(zhí)行關聯(lián)副本集命令：

docker exec -it mongo容器名 bash
mongo
rs.initiate({_id:"repset",members:[{_id:0,host:"192.168.3.249:27017"},{_id:1,host:"192.168.3.249:27018"}]})

返回ok結束
docker-compose新增同步工具monstache容器：

monstache:
  image: rwynn/monstache:rel6
  restart: always
  container_name: monstache
  volumes:
    - /mydata/monstache-conf/monstache.config.toml:/app/monstache.config.toml
  restart: always
  deploy:
    resources:
      limits:
        cpus: "1"
        memory: 500M
      reservations:
        memory: 100M
  command: -f /app/monstache.config.toml
	depends_on: 
	  - mongo

完成后啟動docker-compose

查看es內(nèi)索引和數(shù)據(jù)量：curl -s -XGET --user elastic:密碼 'http://127.0.0.1:9200/_cat/indices/?v’
正常情況如圖，能看到mongo的索引并數(shù)據(jù)不斷增加

2022-5-7更新： 以下功能實測效率低已放棄使用
附：由于技戰(zhàn)法功能需要用到聚合查詢，初始化es后需要將cameraId和archivesInfo.archivesId兩個字段執(zhí)行如下配置：
kibana->開發(fā)工具->
執(zhí)行如下
1、
PUT nfy-csia.capmessage/_mapping?pretty
{
~~ “properties”: {
“cameraId”: {
“type”: “text”,
“fielddata”: true~~
~~ }
}
}
2、
PUT nfy-csia.capmessage/_mapping?pretty
{
“properties”: {
“archivesInfo.archivesId”: {
“type”: “text”,
“fielddata”: true~~
~~ }
}
}~~
文章來源地址http://www.zghlxwxcb.cn/news/detail-412613.html

到了這里，關于ElasticSearch單節(jié)點部署并通過monstache同步MongoDB數(shù)據(jù)的文章就介紹完了。如果您還想了解更多內(nèi)容，請在右上角搜索TOY模板網(wǎng)以前的文章或繼續(xù)瀏覽下面的相關文章，希望大家以后多多支持TOY模板網(wǎng)！