【ElasticSearch】docker部署ElasticSearch、常用Restful API的使用(一)
前言
Elaticsearch,簡稱為es,es是一個開源的高擴展的分布式全文檢索引擎,它可以近乎實時的存儲、檢索數(shù)據(jù);本身擴展性很好,可以擴展到上百臺服務(wù)器,處理PB級別(大數(shù)據(jù)時代)的數(shù)據(jù)。es也使用java開發(fā)并使用Lucene作為其核心來實現(xiàn)所有索引和搜索的功能,但是它的目的是通過簡單的RESTful API來隱藏Lucene的復(fù)雜性,從而讓全文搜索變得簡單。
一、docker部署ElasticSearch
1.1 配置ElasticSearch容器
docker pull elasticsearch:7.4.0
mkdir -p /home/elasticsearch/plugins
mkdir -p /home/elasticsearch/data
docker run -d \
--name es \
--restart=always \
-p 9200:9200 \
-p 9300:9300 \
-e "discovery.type=single-node" \
-e "ES_JAVA_OPTS=-Xms512m -Xmx512m" \
-v /home/elasticsearch/data:/usr/share/elasticsearch/data \
-v /home/elasticsearch/plugins:/usr/share/elasticsearch/plugins \
--privileged \
elasticsearch:7.4.0
命令參數(shù)解釋:
-e “cluster.name=es-docker-cluster”:設(shè)置集群名稱
-e “http.host=0.0.0.0”:監(jiān)聽的地址,可以外網(wǎng)訪問
-e “ES_JAVA_OPTS=-Xms512m -Xmx512m”:內(nèi)存大小
-e “discovery.type=single-node”:非集群模式
-v /home/elasticsearch/data:/usr/share/elasticsearch/data:掛載邏輯卷,綁定es的數(shù)據(jù)目錄
-v /home/elasticsearch/es-logs:/usr/share/elasticsearch/logs:掛載邏輯卷,綁定es的日志目錄
-v /home/elasticsearch/plugins:/usr/share/elasticsearch/plugins:掛載邏輯卷,綁定es的插件目錄
–privileged:授予邏輯卷訪問權(quán)
–network es-net :加入一個名為es-net的網(wǎng)絡(luò)中
-p 9200:9200:端口映射配置
在瀏覽器中輸入:http://ip:9200 即可看到elasticsearch的響應(yīng)結(jié)果
bug:docker logs es發(fā)現(xiàn)報錯:
原因:數(shù)據(jù)卷掛載的時候宿主機上創(chuàng)建的目錄權(quán)限不夠
chmod 777 /home/elasticsearch/plugins
chmod 777 /home/elasticsearch/data
1.2 配置ik中文分詞器
在創(chuàng)建elasticsearch容器的時候,映射了插件目錄·(-v /home/elasticsearch/plugins:/usr/share/elasticsearch/plugins
),所以將ik分詞器插件直接復(fù)制到映射目錄中;將elasticsearch-analysis-ik-7.4.0.zip
上傳到服務(wù)器上,將解壓好的文件放在/home/elasticsearch/plugins/analysis-ik下
注意:ik分詞器版本需要和elasticsearch的版本匹配
bug:ik分詞器出現(xiàn) “reason“: “failed to find global analyzer [ik_max_word]“ 報錯
解決:上傳插件后重啟一下容器
二、常用Restful API的使用
2.1 基本Rest命令說明
method | url地址 | 描述 |
---|---|---|
PUT(創(chuàng)建,修改) | localhost:9200/索引名稱/類型名稱/文檔id | 創(chuàng)建文檔(指定文檔id) |
POST(創(chuàng)建) | localhost:9200/索引名稱/類型名稱 | 創(chuàng)建文檔(隨機文檔id) |
POST(修改) | localhost:9200/索引名稱/類型名稱/文檔id/_update | 修改文檔 |
DELETE(刪除) | localhost:9200/索引名稱/類型名稱/文檔id | 刪除文檔 |
GET(查詢) | localhost:9200/索引名稱/類型名稱/文檔id | 查詢文檔通過文檔ID |
POST(查詢) | localhost:9200/索引名稱/類型名稱/文檔id/_search | 查詢所有數(shù)據(jù) |
2.2 索引操作
創(chuàng)建索引
創(chuàng)建索引類似于關(guān)系型數(shù)據(jù)庫中的創(chuàng)建數(shù)據(jù)庫
在 Postman 中,向 ES 服務(wù)器發(fā) PUT 請求 :http://127.0.0.1:9200/friend
PUT friend
響應(yīng):
{
"acknowledged": true, # 響應(yīng)結(jié)果: true 操作成功
"shards_acknowledged": true, # 分片結(jié)果:分片操作成功
"index": "friend" # 索引名稱
}
# 注意:創(chuàng)建索引庫的分片數(shù)默認 1 片,在 7.0.0 之前的 Elasticsearch 版本中,默認 5 片
如果重復(fù)添加索引,會返回錯誤信息
{
"error": {
"root_cause": [
{
"type": "resource_already_exists_exception",
"reason": "index [friend/OftUkabOT3eWGQ6QhXcliQ] already exists",
"index_uuid": "OftUkabOT3eWGQ6QhXcliQ",
"index": "friend"
}
],
"type": "resource_already_exists_exception",
"reason": "index [friend/OftUkabOT3eWGQ6QhXcliQ] already exists",
"index_uuid": "OftUkabOT3eWGQ6QhXcliQ",
"index": "friend"
},
"status": 400
}
查看所有索引
在 Postman 中,向 ES 服務(wù)器發(fā) GET 請求 :http://127.0.0.1:9200/_cat/indices?v
GET /_cat/indices?v
這里請求路徑中的_cat
表示查看的意思,indices
表示索引,所以整體含義就是查看當前 ES服務(wù)器中的所有索引,就好像 MySQL 中的 show tables 的感覺,服務(wù)器響應(yīng)結(jié)果如下:
查看單個索引
在 Postman 中,向 ES 服務(wù)器發(fā) GET 請求 :http://127.0.0.1:9200/friend
服務(wù)器響應(yīng)結(jié)果如下:
{
"friend": { # 索引名
"aliases": {}, # 別名
"mappings": {}, # 映射
"settings": { # 設(shè)置
"index": { # 設(shè)置——索引
"creation_date": "1704785323704", # 設(shè)置 - 索引 - 創(chuàng)建時間
"number_of_shards": "1", # 設(shè)置 - 索引 - 主分片數(shù)量
"number_of_replicas": "1", # 設(shè)置 - 索引 - 副分片數(shù)量
"uuid": "OftUkabOT3eWGQ6QhXcliQ", # 設(shè)置 - 索引 - 唯一標識
"version": { # 設(shè)置 - 索引 - 版本
"created": "7040099"
},
"provided_name": "friend" # 設(shè)置 - 索引 - 名稱
}
}
}
}
刪除索引
在 Postman 中,向 ES 服務(wù)器發(fā) DELETE 請求 :http://127.0.0.1:9200/friend
服務(wù)器響應(yīng)結(jié)果如下:
{
"acknowledged": true
}
重新訪問索引時,服務(wù)器返回響應(yīng):索引不存在
{
"error": {
"root_cause": [
{
"type": "index_not_found_exception",
"reason": "no such index [friend]",
"resource.type": "index_or_alias",
"resource.id": "friend",
"index_uuid": "_na_",
"index": "friend"
}
],
"type": "index_not_found_exception",
"reason": "no such index [friend]",
"resource.type": "index_or_alias",
"resource.id": "friend",
"index_uuid": "_na_",
"index": "friend"
},
"status": 404
}
2.3 文檔操作
創(chuàng)建文檔
索引已經(jīng)創(chuàng)建好了,接下來我們來創(chuàng)建文檔,并添加數(shù)據(jù)。這里的文檔可以類比為關(guān)系型數(shù)據(jù)庫中的表數(shù)據(jù),添加的數(shù)據(jù)格式為 JSON 格式
在 Postman 中,向 ES 服務(wù)器發(fā) POST 請求 :http://127.0.0.1:9200/friend/_doc
(數(shù)據(jù)創(chuàng)建后,由于沒有指定數(shù)據(jù)唯一性標識(ID),默認情況下,ES 服務(wù)器會隨機生成一個)
注意:此處發(fā)送請求的方式必須為 POST,不能是 PUT,否則會發(fā)生錯誤(PUT具有冪等性,而id是隨機生成的),此處使用post方式同樣的請求可多次發(fā)送,生成的id不一樣
請求體內(nèi)容為:
{
"name": "chandler",
"age": 27
}
服務(wù)器返回:
{
"_index": "friend", # 索引
"_type": "_doc", # 類型-文檔
"_id": "M9Ex7YwBc3wEkvsQvLgp", # 唯一標識,可以類比為 MySQL 中的主鍵,隨機生成
"_version": 1, # 版本
"result": "created", # 結(jié)果
"_shards": { # 分片
"total": 2, # 分片-總數(shù)
"successful": 1, # 分片-成果
"failed": 0 # 分片-失敗
},
"_seq_no": 0,
"_primary_term": 1
}
如果想要自定義唯一性標識,需要在創(chuàng)建時指定:http://127.0.0.1:9200/friend/_doc/1
此處需要注意:如果增加數(shù)據(jù)時明確數(shù)據(jù)主鍵,那么請求方式也可以為 PUT
{
"_index": "friend",
"_type": "_doc",
"_id": "1",
"_version": 1,
"result": "created",
"_shards": {
"total": 2,
"successful": 1,
"failed": 0
},
"_seq_no": 1,
"_primary_term": 1
}
查看文檔
查看文檔時,需要指明文檔的唯一性標識,類似于 MySQL 中數(shù)據(jù)的主鍵查詢
在 Postman 中,向 ES 服務(wù)器發(fā) GET 請求 :http://127.0.0.1:9200/friend/_doc/1
_doc表示文檔數(shù)據(jù)
{
"_index": "friend", # 索引
"_type": "_doc", # 文檔類型
"_id": "1",
"_version": 1,
"_seq_no": 1,
"_primary_term": 1,
"found": true, # 查詢結(jié)果: true 表示查找到,false 表示未查找到
"_source": {
"name": "monica",
"age": 27
}
}
查看索引下的所有文檔
在 Postman 中,向 ES 服務(wù)器發(fā) GET 請求 :http://127.0.0.1:9200/friend/_search
{
"took": 2,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 0,
"relation": "eq"
},
"max_score": null,
"hits": []
}
}
修改文檔(全覆蓋)
和新增文檔一樣,輸入相同的 URL 地址請求,如果請求體變化,會將原有的數(shù)據(jù)內(nèi)容覆蓋
在 Postman 中,向 ES 服務(wù)器發(fā) POST 請求 :http://127.0.0.1:9200/friend/_doc/1
請求體內(nèi)容為:
{
"name": "monica" #注意:此時請求體中相比于原來的數(shù)據(jù)少了age字段
}
響應(yīng):
{
"_index": "friend",
"_type": "_doc",
"_id": "1",
"_version": 2, # 注意版本變?yōu)榱?
"result": "updated", # 結(jié)果: updated 表示數(shù)據(jù)被更新
"_shards": {
"total": 2,
"successful": 1,
"failed": 0
},
"_seq_no": 2,
"_primary_term": 1
}
再次查詢,發(fā)現(xiàn)文檔只有name字段
{
"_index": "friend",
"_type": "_doc",
"_id": "1",
"_version": 2,
"_seq_no": 2,
"_primary_term": 1,
"found": true,
"_source": {
"name": "monica"
}
}
修改字段(更新局部信息)
修改數(shù)據(jù)時,也可以只修改某一給條數(shù)據(jù)的局部信息,在 Postman 中,向 ES 服務(wù)器發(fā) POST 請求 :http://127.0.0.1:9200/friend/_doc/1/_update
請求體:
{
"doc": {
"hobby": "cooking"
}
}
響應(yīng)結(jié)果:
{
"_index": "friend",
"_type": "_doc",
"_id": "1",
"_version": 3,
"result": "updated",
"_shards": {
"total": 2,
"successful": 1,
"failed": 0
},
"_seq_no": 3,
"_primary_term": 1
}
再次查詢該文檔:
{
"_index": "friend",
"_type": "_doc",
"_id": "1",
"_version": 5,
"_seq_no": 5,
"_primary_term": 1,
"found": true,
"_source": {
"name": "monica",
"hobby": "cooking" # 新增的hobby
}
}
刪除文檔
刪除一個文檔不會立即從磁盤上移除,它只是被標記成已刪除(邏輯刪除)。
在 Postman 中,向 ES 服務(wù)器發(fā) DELETE 請求 :http://127.0.0.1:9200/friend/_doc/1
響應(yīng):
{
"_index": "friend",
"_type": "_doc",
"_id": "1",
"_version": 6, #對數(shù)據(jù)的操作,都會更新版本
"result": "deleted", # deleted 表示數(shù)據(jù)被標記為刪除
"_shards": {
"total": 2,
"successful": 1,
"failed": 0
},
"_seq_no": 6,
"_primary_term": 1
}
刪除后再查詢當前文檔信息
{
"_index": "friend",
"_type": "_doc",
"_id": "1",
"found": false
}
果刪除一個并不存在的文檔 DELETE http://127.0.0.1:9200/friend/_doc/2
,結(jié)果返回not found
{
"_index": "friend",
"_type": "_doc",
"_id": "2",
"_version": 1,
"result": "not_found", # not_found 表示未查找到
"_shards": {
"total": 2,
"successful": 1,
"failed": 0
},
"_seq_no": 7,
"_primary_term": 1
}
條件刪除文檔
一般刪除數(shù)據(jù)都是根據(jù)文檔的唯一性標識進行刪除,實際操作時,也可以根據(jù)條件對多條數(shù)據(jù)進行刪除
首先分別增加多條數(shù)據(jù):
http://111.229.80.188:9200/friend/_doc/1
{
"name": "chandler",
"age": 27
}
http://111.229.80.188:9200/friend/_doc/2
{
"name": "monica",
"age": 27
}
http://111.229.80.188:9200/friend/_doc/3
{
"name": "joey",
"age": 27
}
向 ES 服務(wù)器發(fā) POST 請求 :http://127.0.0.1:9200/friend/_delete_by_query
請求體:
{
"query":{
"match":{
"age": 27
}
}
}
響應(yīng)
{
"took": 419, # 耗時
"timed_out": false, # 是否超時
"total": 3, # 數(shù)據(jù)總數(shù)
"deleted": 3, # 刪除數(shù)量
"batches": 1,
"version_conflicts": 0,
"noops": 0,
"retries": {
"bulk": 0,
"search": 0
},
"throttled_millis": 0,
"requests_per_second": -1.0,
"throttled_until_millis": 0,
"failures": []
}
2.4 映射操作
有了索引庫,等于有了數(shù)據(jù)庫中的 database。接下來就需要建索引庫(index)中的映射了,類似于數(shù)據(jù)庫(database)中的表結(jié)構(gòu)(table)。創(chuàng)建數(shù)據(jù)庫表需要設(shè)置字段名稱,類型,長度,約束等;索引庫也一樣,需要知道這個類型下有哪些字段,每個字段有哪些約束信息,這就叫做映射(mapping)。
創(chuàng)建映射
在 Postman 中,向 ES 服務(wù)器發(fā) PUT 請求 :http://127.0.0.1:9200/article/_mapping
注意:先創(chuàng)建索引—— PUT 請求 :http://127.0.0.1:9200/article
請求體:
{
"properties": {
"name":{ # 字段名
"type": "keyword", # 字段類型
"index": true
},
"author":{
"type": "text",
"index": true
},
"prices":{
"type": "long",
"index": false
}
}
}
返回:
{
"acknowledged": true
}
映射數(shù)據(jù)說明:
-
字段名:任意填寫,下面指定許多屬性,例如:title、subtitle、images、price
-
type:類型,Elasticsearch 中支持的數(shù)據(jù)類型非常豐富:
- String 類型,又分兩種:
- text:可分詞
- keyword:不可分詞,數(shù)據(jù)會作為完整字段進行匹配
- Numerical:數(shù)值類型,分兩類
- 基本數(shù)據(jù)類型:long、integer、short、byte、double、float、half_float
- 浮點數(shù)的高精度類型:scaled_float
- Date:日期類型
- Array:數(shù)組類型
- Object:對象
- String 類型,又分兩種:
-
index:是否索引,默認為 true,也就是說你不進行任何配置,所有字段都會被索引。
- true:字段會被索引,則可以用來進行搜索
- false:字段不會被索引,不能用來搜索
-
store:是否將數(shù)據(jù)進行獨立存儲,默認為 false
原始的文本會存儲在_source 里面,默認情況下其他提取出來的字段都不是獨立存儲的,是從_source 里面提取出來的。當然你也可以獨立的存儲某個字段,只要設(shè)置"store": true 即可,獲取獨立存儲的字段要比從_source 中解析快得多,但是也會占用更多的空間,所以要根據(jù)實際業(yè)務(wù)需求來設(shè)置。
-
analyzer:分詞器
"fields name": {
"type": "text", //文本類型
"index": "analyzed"http:// 決定倒排索引的構(gòu)建,控制在get/search請求時可不可用該字段作為請求體條件。分詞器使用的是指定的字段或者索引的分詞器。
analyzed:字段被索引,會做分詞,可搜索。反過來,如果需要根據(jù)某個字段進搜索,index屬性就應(yīng)該設(shè)置為analyzed。
not_analyzed:字段值不分詞,會被原樣寫入索引。反過來,如果某些字段需要完全匹配,比如人名、地名,index屬性設(shè)置為not_analyzed為佳。
no:字段不寫入索引,當然也就不能搜索。以該字段作為請求體時會報錯。
"enabled" : true, #是否會被索引,但都會存儲;可以針對一整個_doc
分詞器是對text類型配置的,keyword不支持分詞器
"analyzer" : "ik_max_word", #指定寫入分詞器,比如默認的空格分詞器等
"search_analyzer" : "ik_max_word" , #查詢時分詞器;一般情況和analyzer對應(yīng)
copy_to:將多個字段連接到一個字段中存儲方便查詢,他是復(fù)制字段中的值。查詢時可以指定該字段查詢
"copy_to" : "field_name", #自定義_all字段,
index_option:存儲倒排索引的哪些信息
4個可選參數(shù):
docs:索引文檔號
freqs:文檔號+詞頻
positions:文檔號+詞頻+位置,通常用來距離查詢
offsets:文檔號+詞頻+位置+偏移量,通常被使用在高亮字段
分詞字段默認是positions,其他默認時docs
"index_options": "docs"
"store":false//是否單獨設(shè)置此字段的是否存儲,于_source字段之外單獨再存儲一份,設(shè)置true時查詢該字段從store獲取值而不是source屬性,查詢更快
"boost":1/2/3 //字段級別的分數(shù)加權(quán)
"doc_values":false//對not_analyzed字段,默認都是開啟,analyzed字段不能使用,排序聚合必須開啟
對排序和聚合能提升較大性能,數(shù)據(jù)存儲在磁盤中,不占用內(nèi)存空間,不會oom。如果不需要對字段進行排序或聚合,或者從script訪問字段值,
則可以禁用doc值以節(jié)省磁盤空間:如果需要排序,比如range查詢則必須開啟
"fielddata":{"loading" : "eager" }//es加載內(nèi)存 fielddata 的默認行為是延遲加載 ,查詢一次既被存儲。占用內(nèi)存的緩存空間,不存在磁盤中。
當 Elasticsearch 第一次查詢某個字段時,它將會完整加載這個字段所有 Segment 中的倒排索引到內(nèi)存中,
以便于以后的查詢能夠獲取更好的性能。
"fields":{"keyword": {"type": "keyword","ignore_above": 256}} //可以對一個字段提供多種索引模式,同一個字段的值,一個分詞,一個不分詞
"ignore_above":100 //超過100個字符的文本,將會被忽略,不被索引
"include_in_all":ture//設(shè)置是否此字段包含在_all字段中,默認是true,除非index設(shè)置成no選項
"norms":{"enable":true,"loading":"lazy"}//分詞字段默認配置,不分詞字段:默認{"enable":false},
存儲長度因子和索引時boost,建議對需要參與評分字段使用 ,會額外增加內(nèi)存消耗量
"null_value":"NULL"http://設(shè)置一些缺失字段的初始化值,只有string可以使用,分詞字段的null值也會被分詞
"position_increament_gap":0//影響距離查詢或近似查詢,可以設(shè)置在多值字段的數(shù)據(jù)上火分詞字段上,查詢時可指定slop間隔,默認值是100
"search_analyzer":"ik"http://設(shè)置搜索時的分詞器,默認跟ananlyzer是一致的,比如index時用standard+ngram,搜索時用standard用來完成自動提示功能
"similarity":"BM25"http://默認是TF/IDF算法,指定一個字段評分策略,僅僅對字符串型和分詞類型有效
"term_vector":"no"http://默認不存儲向量信息,支持參數(shù)yes(term存儲),with_positions(term+位置),with_offsets(term+偏移量),
with_positions_offsets(term+位置+偏移量) 對快速高亮fast vector highlighter能提升性能,
但開啟又會加大索引體積,不適合大數(shù)據(jù)量用
}
詳細介紹博客地址:(六:2)elasticsearch索引的mapping詳解
查看映射
在 Postman 中,向 ES 服務(wù)器發(fā) GET 請求 :http://127.0.0.1:9200/article/_mapping
響應(yīng):
{
"article": {
"mappings": {
"properties": {
"author": {
"type": "text"
},
"name": {
"type": "text"
},
"prices": {
"type": "long",
"index": false
}
}
}
}
}
測試text、keyword、index類型
插入一條數(shù)據(jù): PUT /article/_create/1
{
"name": "時間簡史",
"author": "路人甲",
"prices": "29.9"
}
根據(jù)name(keyword, 不可分)查詢:
GET /article/_search
{
"query": {
"match": {
"name": "時間" # name 為keyword, 不可拆分
}
}
}
響應(yīng):查詢不到
{
"took": 1,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 0,
"relation": "eq"
},
"max_score": null,
"hits": []
}
}
根據(jù)author(text, 可分)查詢
GET /article/_search
{
"query": {
"match": {
"author": "甲" #author為text, 可拆分
}
}
}
{
"took": 50,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 1,
"relation": "eq"
},
"max_score": 0.2876821,
"hits": [
{
"_index": "article",
"_type": "_doc",
"_id": "1",
"_score": 0.2876821,
"_source": {
"name": "時間簡史",
"author": "路人甲",
"prices": "29.9"
}
}
]
}
}
根據(jù)prices(index=false)進行查詢
GET /article/_search
{
"query": {
"match": {
"prices": 29.9 # index=false,不是索引,不支持查詢
}
}
}
#報錯
{
"error": {
"root_cause": [
{
"type": "query_shard_exception",
"reason": "failed to create query: {\n \"match\" : {\n \"prices\" : {\n \"query\" : 29.9,\n \"operator\" : \"OR\",\n \"prefix_length\" : 0,\n \"max_expansions\" : 50,\n \"fuzzy_transpositions\" : true,\n \"lenient\" : false,\n \"zero_terms_query\" : \"NONE\",\n \"auto_generate_synonyms_phrase_query\" : true,\n \"boost\" : 1.0\n }\n }\n}",
"index_uuid": "JxyObp0uTAiijCRocOD2vA",
"index": "article"
}
],
"type": "search_phase_execution_exception",
"reason": "all shards failed",
"phase": "query",
"grouped": true,
"failed_shards": [
{
"shard": 0,
"index": "article",
"node": "l07VX9xeTzicai0eswz7ag",
"reason": {
"type": "query_shard_exception",
"reason": "failed to create query: {\n \"match\" : {\n \"prices\" : {\n \"query\" : 29.9,\n \"operator\" : \"OR\",\n \"prefix_length\" : 0,\n \"max_expansions\" : 50,\n \"fuzzy_transpositions\" : true,\n \"lenient\" : false,\n \"zero_terms_query\" : \"NONE\",\n \"auto_generate_synonyms_phrase_query\" : true,\n \"boost\" : 1.0\n }\n }\n}",
"index_uuid": "JxyObp0uTAiijCRocOD2vA",
"index": "article",
"caused_by": {
"type": "illegal_argument_exception",
"reason": "Cannot search on field [prices] since it is not indexed."
}
}
}
]
},
"status": 400
}
2.5 查詢操作
Elasticsearch 提供了基于 JSON 提供完整的查詢 DSL 來定義查詢
初始化數(shù)據(jù):
# POST /student/_doc/1001
{
"name":"zhangsan",
"nickname":"zhangsan",
"sex":"男",
"age":30
}
# POST /student/_doc/1002
{
"name":"lisi",
"nickname":"lisi",
"sex":"男",
"age":20
}
# POST /student/_doc/1003
{
"name":"wangwu",
"nickname":"wangwu",
"sex":"女",
"age":40
}
# POST /student/_doc/1004
{
"name":"zhangsan1",
"nickname":"zhangsan1",
"sex":"女",
"age":50
}
# POST /student/_doc/1005
{
"name":"zhangsan2",
"nickname":"zhangsan2",
"sex":"女",
"age":30
}
查詢所有文檔
在 Postman 中,向 ES 服務(wù)器發(fā) GET 請求 :http://127.0.0.1:9200/student/_search
{
"query": {
"match_all": {}
}
}
# "query":這里的 query 代表一個查詢對象,里面可以有不同的查詢屬性
# "match_all":查詢類型,例如:match_all(代表查詢所有), match,term , range 等等
# {查詢條件}:查詢條件會根據(jù)類型的不同,寫法也有差異
{
"took": 786, # 查詢花費時間,單位毫秒
"timed_out": false, # 是否超時
"_shards": { # 分片信息
"total": 1, # 總數(shù)
"successful": 1, # 成功
"skipped": 0, # 忽略
"failed": 0 # 失敗
},
"hits": { # 搜索命中結(jié)果
"total": { # 搜索條件匹配的文檔總數(shù)
"value": 5, # 總命中計數(shù)的值
"relation": "eq" # 計數(shù)規(guī)則: eq 表示計數(shù)準確, gte 表示計數(shù)不準確
},
"max_score": 1.0, # 匹配度分值
"hits": [ # 命中結(jié)果集合
{
"_index": "student",
"_type": "_doc",
"_id": "1001",
"_score": 1.0,
"_source": {
"name": "zhangsan",
"nickname": "zhangsan",
"sex": "男",
"age": 30
}
},
{
"_index": "student",
"_type": "_doc",
"_id": "1002",
"_score": 1.0,
"_source": {
"name": "lisi",
"nickname": "lisi",
"sex": "男",
"age": 20
}
},
...
]
}
}
匹配查詢
match 匹配類型查詢,會把查詢條件進行分詞,然后進行查詢,多個詞條之間是 or 的關(guān)系
在 Postman 中,向 ES 服務(wù)器發(fā) GET 請求 :http://127.0.0.1:9200/student/_search
文章來源:http://www.zghlxwxcb.cn/news/detail-809602.html
{
"query": {
"match": {
"name":"zhangsan"
}
}
}
響應(yīng):
{
"took": 2,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 1,
"relation": "eq"
},
"max_score": 1.3862944,
"hits": [
{
"_index": "student",
"_type": "_doc",
"_id": "1001",
"_score": 1.3862944,
"_source": {
"name": "zhangsan",
"nickname": "zhangsan",
"sex": "男",
"age": 30
}
}
]
}
}
字段匹配查詢
multi_match 與 match 類似,不同的是它可以在多個字段中查詢。
在 Postman 中,向 ES 服務(wù)器發(fā) GET 請求 :http://127.0.0.1:9200/student/_search
{
"query": {
"multi_match": {
"query": "zhangsan",
"fields": ["name","nickname"]
}
}
}
響應(yīng):
{
"took": 258,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 1,
"relation": "eq"
},
"max_score": 1.3862944,
"hits": [
{
"_index": "student",
"_type": "_doc",
"_id": "1001",
"_score": 1.3862944,
"_source": {
"name": "zhangsan",
"nickname": "zhangsan",
"sex": "男",
"age": 30
}
}
]
}
}
關(guān)鍵字精確查詢
term 查詢,精確的關(guān)鍵詞匹配查詢,不對查詢條件進行分詞。
在 Postman 中,向 ES 服務(wù)器發(fā) GET 請求 :http://127.0.0.1:9200/student/_search
{
"query": {
"term": {
"name": {
"value": "zhangsan"
}
}
}
}
響應(yīng):命中了name為zhangsan的文檔
{
"took": 1,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 1,
"relation": "eq"
},
"max_score": 1.3862944,
"hits": [
{
"_index": "student",
"_type": "_doc",
"_id": "1001",
"_score": 1.3862944,
"_source": {
"name": "zhangsan",
"nickname": "zhangsan",
"sex": "男",
"age": 30
}
}
]
}
}
多關(guān)鍵字精確查詢
terms 查詢和 term 查詢一樣,但它允許你指定多值進行匹配。
如果這個字段包含了指定值中的任何一個值,那么這個文檔滿足條件,類似于 mysql 的 in
在 Postman 中,向 ES 服務(wù)器發(fā) GET 請求 :http://127.0.0.1:9200/student/_search
{
"query": {
"terms": {
"name": ["zhangsan","lisi"]
}
}
}
響應(yīng):命中了name為zhangsan或lisi的文檔
{
"took": 185,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 2,
"relation": "eq"
},
"max_score": 1.0,
"hits": [
{
"_index": "student",
"_type": "_doc",
"_id": "1001",
"_score": 1.0,
"_source": {
"name": "zhangsan",
"nickname": "zhangsan",
"sex": "男",
"age": 30
}
},
{
"_index": "student",
"_type": "_doc",
"_id": "1002",
"_score": 1.0,
"_source": {
"name": "lisi",
"nickname": "lisi",
"sex": "男",
"age": 20
}
}
]
}
}
指定查詢字段
默認情況下,Elasticsearch 在搜索的結(jié)果中,會把文檔中保存在_source 的所有字段都返回。
如果我們只想獲取其中的部分字段,我們可以添加_source 的過濾
在 Postman 中,向 ES 服務(wù)器發(fā) GET 請求 :http://127.0.0.1:9200/student/_search
{
"_source": ["name","nickname"],
"query": {
"terms": {
"nickname": ["zhangsan"]
}
}
}
響應(yīng):
{
"took": 26,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 1,
"relation": "eq"
},
"max_score": 1.0,
"hits": [
{
"_index": "student",
"_type": "_doc",
"_id": "1001",
"_score": 1.0,
"_source": {
"name": "zhangsan",
"nickname": "zhangsan"
}
}
]
}
}
過濾字段
我們也可以通過:
- includes:來指定想要顯示的字段
- excludes:來指定不想要顯示的字段
在 Postman 中,向 ES 服務(wù)器發(fā) GET 請求 :http://127.0.0.1:9200/student/_search
測試includes
{
"_source": {
"includes": ["name","nickname"]
},
"query": {
"terms": {
"nickname": ["zhangsan"]
}
}
}
響應(yīng)
{
"took": 1,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 1,
"relation": "eq"
},
"max_score": 1.0,
"hits": [
{
"_index": "student",
"_type": "_doc",
"_id": "1001",
"_score": 1.0,
"_source": {
"name": "zhangsan",
"nickname": "zhangsan"
}
}
]
}
}
測試excludes
{
"_source": {
"excludes": ["sex","age"]
},
"query": {
"terms": {
"nickname": ["zhangsan"]
}
}
}
{
"took": 2,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 1,
"relation": "eq"
},
"max_score": 1.0,
"hits": [
{
"_index": "student",
"_type": "_doc",
"_id": "1001",
"_score": 1.0,
"_source": {
"name": "zhangsan",
"nickname": "zhangsan"
}
}
]
}
}
組合查詢
bool
把各種其它查詢通過must
(必須 )、must_not
(必須不)、should
(應(yīng)該)的方式進行組合
bool 子元素區(qū)別:
1、must (must字段對應(yīng)的是個列表,也就是說可以有多個并列的查詢條件,一個文檔滿足各個子條件后才最終返回)
2、should (只要符合其中一個條件就返回)
3、must_not (與must相反,也就是說可以有多個并列的查詢條件,一個文檔各個子條件后才最終的結(jié)果都不滿足)
4、filter(條件過濾查詢,過濾條件的范圍用range表示gt表示大于、lt表示小于、gte表示大于等于、lte表示小于等于)
在 Postman 中,向 ES 服務(wù)器發(fā) GET 請求 :http://127.0.0.1:9200/student/_search
{
"query": {
"bool": {
"must": [
{
"match": {
"name": "zhangsan"
}
}
],
"must_not": [
{
"match": {
"age": "40"
}
}
],
"should": [
{
"match": {
"sex": "女"
}
}
]
}
}
}
響應(yīng):找到name為張三 && age不為40 或者 sex為女
{"took":109,"timed_out":false,"_shards":{"total":1,"successful":1,"skipped":0,"failed":0},"hits":{"total":{"value":1,"relation":"eq"},"max_score":2.261763,"hits":[{"_index":"student","_type":"_doc","_id":"1001","_score":2.261763,"_source":{
"name":"zhangsan",
"nickname":"zhangsan",
"sex":"男",
"age":30
}}]}}
范圍查詢
range 查詢找出那些落在指定區(qū)間內(nèi)的數(shù)字或者時間。range 查詢允許以下字符
- gte ?大于等于
- gt ?大于
- lte ?小于等于
- lt ?小于
在 Postman 中,向 ES 服務(wù)器發(fā) GET 請求 :http://127.0.0.1:9200/student/_search
{
"query": {
"range": {
"age": {
"gte": 30,
"lte": 35
}
}
}
}
響應(yīng):
{"took":96,"timed_out":false,"_shards":{"total":1,"successful":1,"skipped":0,"failed":0},"hits":{"total":{"value":2,"relation":"eq"},"max_score":1.0,"hits":[{"_index":"student","_type":"_doc","_id":"1001","_score":1.0,"_source":{
"name":"zhangsan",
"nickname":"zhangsan",
"sex":"男",
"age":30
}},{"_index":"student","_type":"_doc","_id":"1005","_score":1.0,"_source":{
"name":"zhangsan2",
"nickname":"zhangsan2",
"sex":"女",
"age":30
}}]}}
模糊查詢
返回包含與搜索字詞相似的字詞的文檔。
編輯距離是將一個術(shù)語轉(zhuǎn)換為另一個術(shù)語所需的一個字符更改的次數(shù)。這些更改可以包括:
- 更改字符(box → fox)
- 刪除字符(black → lack)
- 插入字符(sic → sick)
- 轉(zhuǎn)置兩個相鄰字符(act → cat)
為了找到相似的術(shù)語,fuzzy 查詢會在指定的編輯距離內(nèi)創(chuàng)建一組搜索詞的所有可能的變體或擴展。然后查詢返回每個擴展的完全匹配。
通過 fuzziness 修改編輯距離。一般使用默認值 AUTO,根據(jù)術(shù)語的長度生成編輯距離。
在 Postman 中,向 ES 服務(wù)器發(fā) GET 請求 :http://127.0.0.1:9200/student/_search
{
"query": {
"fuzzy": {
"title": {
"value": "zhangsan"
}
}
}
}
{
"query": {
"fuzzy": {
"title": {
"value": "zhangsan",
"fuzziness": 2
}
}
}
}
單字段排序
sort 可以讓我們按照不同的字段進行排序,并且通過 order 指定排序的方式。desc 降序,asc升序。
在 Postman 中,向 ES 服務(wù)器發(fā) GET 請求 :http://127.0.0.1:9200/student/_search
{
"sort": [{
"age": {
"order":"desc"
}
}]
}
{"took":1,"timed_out":false,"_shards":{"total":1,"successful":1,"skipped":0,"failed":0},"hits":{"total":{"value":5,"relation":"eq"},"max_score":null,"hits":[{"_index":"student","_type":"_doc","_id":"1004","_score":null,"_source":{
"name":"zhangsan1",
"nickname":"zhangsan1",
"sex":"女",
"age":50
},"sort":[50]},{"_index":"student","_type":"_doc","_id":"1003","_score":null,"_source":{
"name":"wangwu",
"nickname":"wangwu",
"sex":"女",
"age":40
},"sort":[40]},{"_index":"student","_type":"_doc","_id":"1001","_score":null,"_source":{
"name":"zhangsan",
"nickname":"zhangsan",
"sex":"男",
"age":30
},"sort":[30]},{"_index":"student","_type":"_doc","_id":"1005","_score":null,"_source":{
"name":"zhangsan2",
"nickname":"zhangsan2",
"sex":"女",
"age":30
},"sort":[30]},{"_index":"student","_type":"_doc","_id":"1002","_score":null,"_source":{
"name":"lisi",
"nickname":"lisi",
"sex":"男",
"age":20
},"sort":[20]}]}}
多字段排序
假定我們想要結(jié)合使用 age 和 _score 進行查詢,并且匹配的結(jié)果首先按照年齡排序,然后按照相關(guān)性得分排序
在 Postman 中,向 ES 服務(wù)器發(fā) GET 請求 :http://127.0.0.1:9200/student/_search
{
"query": {
"match_all": {}
},
"sort": [{
"age": {
"order": "desc"
}
},
{
"_score":{
"order": "desc"
}
}]
}
{"took":7,"timed_out":false,"_shards":{"total":1,"successful":1,"skipped":0,"failed":0},"hits":{"total":{"value":5,"relation":"eq"},"max_score":null,"hits":[{"_index":"student","_type":"_doc","_id":"1004","_score":1.0,"_source":{
"name":"zhangsan1",
"nickname":"zhangsan1",
"sex":"女",
"age":50
},"sort":[50,1.0]},{"_index":"student","_type":"_doc","_id":"1003","_score":1.0,"_source":{
"name":"wangwu",
"nickname":"wangwu",
"sex":"女",
"age":40
},"sort":[40,1.0]},{"_index":"student","_type":"_doc","_id":"1001","_score":1.0,"_source":{
"name":"zhangsan",
"nickname":"zhangsan",
"sex":"男",
"age":30
},"sort":[30,1.0]},{"_index":"student","_type":"_doc","_id":"1005","_score":1.0,"_source":{
"name":"zhangsan2",
"nickname":"zhangsan2",
"sex":"女",
"age":30
},"sort":[30,1.0]},{"_index":"student","_type":"_doc","_id":"1002","_score":1.0,"_source":{
"name":"lisi",
"nickname":"lisi",
"sex":"男",
"age":20
},"sort":[20,1.0]}]}}
高亮查詢
在進行關(guān)鍵字搜索時,搜索出的內(nèi)容中的關(guān)鍵字會顯示不同的顏色,稱之為高亮。
Elasticsearch 可以對查詢內(nèi)容中的關(guān)鍵字部分,進行標簽和樣式(高亮)的設(shè)置。
在使用 match 查詢的同時,加上一個 highlight 屬性:
- pre_tags:前置標簽
- post_tags:后置標簽
- fields:需要高亮的字段
- title:這里聲明 title 字段需要高亮,后面可以為這個字段設(shè)置特有配置,也可以空
在 Postman 中,向 ES 服務(wù)器發(fā) GET 請求 :http://127.0.0.1:9200/student/_search
{
"query": {
"match": {
"name": "zhangsan"
}
},
"highlight": {
"pre_tags": "<font color='red'>",
"post_tags": "</font>",
"fields": {
"name": {}
}
}
}
響應(yīng):
{"took":411,"timed_out":false,"_shards":{"total":1,"successful":1,"skipped":0,"failed":0},"hits":{"total":{"value":1,"relation":"eq"},"max_score":1.3862944,"hits":[{"_index":"student","_type":"_doc","_id":"1001","_score":1.3862944,"_source":{
"name":"zhangsan",
"nickname":"zhangsan",
"sex":"男",
"age":30
},"highlight":{"name":["<font color='red'>zhangsan</font>"]}}]}}
分頁查詢
from:當前頁的起始索引,默認從 0 開始。 from = (pageNum - 1) * size
size:每頁顯示多少條
在 Postman 中,向 ES 服務(wù)器發(fā) GET 請求 :http://127.0.0.1:9200/student/_search
{
"query": {
"match_all": {}
},
"sort": [{
"age": {
"order": "desc"
}
}],
"from": 0,
"size": 2
}
響應(yīng):按照年齡降序排列后只展示了兩條數(shù)據(jù)
{"took":2,"timed_out":false,"_shards":{"total":1,"successful":1,"skipped":0,"failed":0},"hits":{"total":{"value":5,"relation":"eq"},"max_score":null,"hits":[{"_index":"student","_type":"_doc","_id":"1004","_score":null,"_source":{
"name":"zhangsan1",
"nickname":"zhangsan1",
"sex":"女",
"age":50
},"sort":[50]},{"_index":"student","_type":"_doc","_id":"1003","_score":null,"_source":{
"name":"wangwu",
"nickname":"wangwu",
"sex":"女",
"age":40
},"sort":[40]}]}}
聚合查詢
聚合允許使用者對 es 文檔進行統(tǒng)計分析,類似與關(guān)系型數(shù)據(jù)庫中的 group by,當然還有很多其他的聚合,例如取最大值、平均值等等。
- 對某個字段取最大值 max
在 Postman 中,向 ES 服務(wù)器發(fā) GET 請求 :http://127.0.0.1:9200/student/_search
{
"aggs":{
"max_age":{
"max":{"field":"age"}
}
},
"size":0
}
{"took":65,"timed_out":false,"_shards":{"total":1,"successful":1,"skipped":0,"failed":0},"hits":{"total":{"value":5,"relation":"eq"},"max_score":null,"hits":[]},"aggregations":{"max_age":{"value":50.0}}}
- 對某個字段取最小值 min
在 Postman 中,向 ES 服務(wù)器發(fā) GET 請求 :http://127.0.0.1:9200/student/_search
{
"aggs":{
"min_age":{
"min":{"field":"age"}
}
},
"size":0
}
{"took":5,"timed_out":false,"_shards":{"total":1,"successful":1,"skipped":0,"failed":0},"hits":{"total":{"value":5,"relation":"eq"},"max_score":null,"hits":[]},"aggregations":{"min_age":{"value":20.0}}}
- 對某個字段求和 sum
在 Postman 中,向 ES 服務(wù)器發(fā) GET 請求 :http://127.0.0.1:9200/student/_search
{
"aggs":{
"sum_age":{
"sum":{"field":"age"}
}
},
"size":0
}
{"took":21,"timed_out":false,"_shards":{"total":1,"successful":1,"skipped":0,"failed":0},"hits":{"total":{"value":5,"relation":"eq"},"max_score":null,"hits":[]},"aggregations":{"sum_age":{"value":170.0}}}
- 對某個字段取平均值 avg
在 Postman 中,向 ES 服務(wù)器發(fā) GET 請求 :http://127.0.0.1:9200/student/_search
{
"aggs":{
"avg_age":{
"avg":{"field":"age"}
}
},
"size":0
}
{"took":5,"timed_out":false,"_shards":{"total":1,"successful":1,"skipped":0,"failed":0},"hits":{"total":{"value":5,"relation":"eq"},"max_score":null,"hits":[]},"aggregations":{"avg_age":{"value":34.0}}}
- 對某個字段的值進行去重之后再取總數(shù)
在 Postman 中,向 ES 服務(wù)器發(fā) GET 請求 :http://127.0.0.1:9200/student/_search
{
"aggs":{
"distinct_age":{
"cardinality":{"field":"age"}
}
},
"size":0
}
{"took":534,"timed_out":false,"_shards":{"total":1,"successful":1,"skipped":0,"failed":0},"hits":{"total":{"value":5,"relation":"eq"},"max_score":null,"hits":[]},"aggregations":{"distinct_age":{"value":4}}}
- State 聚合
stats 聚合,對某個字段一次性返回 count,max,min,avg 和 sum 五個指標
在 Postman 中,向 ES 服務(wù)器發(fā) GET 請求 :http://127.0.0.1:9200/student/_search
{
"aggs":{
"stats_age":{
"stats":{"field":"age"}
}
},
"size":0
}
{"took":6,"timed_out":false,"_shards":{"total":1,"successful":1,"skipped":0,"failed":0},"hits":{"total":{"value":5,"relation":"eq"},"max_score":null,"hits":[]},"aggregations":{"stats_age":{"count":5,"min":20.0,"max":50.0,"avg":34.0,"sum":170.0}}}
桶聚合查詢
桶聚和相當于 sql 中的 group by 語句
- terms 聚合,分組統(tǒng)計
在 Postman 中,向 ES 服務(wù)器發(fā) GET 請求 :http://127.0.0.1:9200/student/_search
{
"aggs":{
"age_groupby":{
"terms":{"field":"age"}
}
},
"size":0
}
{"took":36,"timed_out":false,"_shards":{"total":1,"successful":1,"skipped":0,"failed":0},"hits":{"total":{"value":5,"relation":"eq"},"max_score":null,"hits":[]},"aggregations":{"age_groupby":{"doc_count_error_upper_bound":0,"sum_other_doc_count":0,"buckets":[{"key":30,"doc_count":2},{"key":20,"doc_count":1},{"key":40,"doc_count":1},{"key":50,"doc_count":1}]}}}
- 在 terms 分組下再進行聚合
{
"aggs":{
"age_groupby":{
"terms":{"field":"age"},
"aggs":{
"sum_age":{
"sum":{"field":"age"}
}
}
}
},
"size":0
}
{"took":12,"timed_out":false,"_shards":{"total":1,"successful":1,"skipped":0,"failed":0},"hits":{"total":{"value":5,"relation":"eq"},"max_score":null,"hits":[]},"aggregations":{"age_groupby":{"doc_count_error_upper_bound":0,"sum_other_doc_count":0,"buckets":[{"key":30,"doc_count":2,"sum_age":{"value":60.0}},{"key":20,"doc_count":1,"sum_age":{"value":20.0}},{"key":40,"doc_count":1,"sum_age":{"value":40.0}},{"key":50,"doc_count":1,"sum_age":{"value":50.0}}]}}}
總結(jié)
本文總結(jié)了使用docker部署ElasticSearch、配置ik分詞器,以及常用ElasticSearch的Restful API的使用,包含常用的索引、文檔、映射以及查詢操作。常用索引操作包含創(chuàng)建索引、查看所有索引、查看單個索引、刪除索引;常用文檔操作包含創(chuàng)建文檔、查看文檔、查看索引下的所有文檔、修改文檔(全覆蓋)、修改字段(更新局部信息)、刪除文檔、條件刪除文檔;常用映射操作包含創(chuàng)建映射、查看映射、測試text、keyword、index類型;常用查詢操作包含查詢所有文檔、匹配查詢、字段匹配查詢、關(guān)鍵字精確查詢、多關(guān)鍵字精確查詢、指定查詢字段、過濾字段、組合查詢、范圍查詢、模糊查詢、單字段排序、多字段排序、高亮查詢、分頁查詢、聚合查詢、桶聚合查詢。文章來源地址http://www.zghlxwxcb.cn/news/detail-809602.html
到了這里,關(guān)于【ElasticSearch】docker部署ElasticSearch、常用Restful API的使用(一)的文章就介紹完了。如果您還想了解更多內(nèi)容,請在右上角搜索TOY模板網(wǎng)以前的文章或繼續(xù)瀏覽下面的相關(guān)文章,希望大家以后多多支持TOY模板網(wǎng)!