ES刪除大批量數(shù)據(jù)方法

1.Delete By Query API

POST twitter/_delete_by_query
{
  "query": { 
"match": {
  "message": "some message"
}
  }
}

一般回用如下：

http://172.16.96.*:9200/index/_delete_by_query?slices=3&wait_for_completion=false&scroll_size=5000&conflicts=proceed

slices：線程數(shù)（根據(jù)CPU的數(shù)量設(shè)置）

wait_for_completion:如果設(shè)置為true，則導致 API 阻塞，直到索引器狀態(tài)完全停止。如果設(shè)置為false，API 立即返回，并且索引器在后臺異步停止。默認為 false。如果請求包含wait_for_completion=false，則 Elasticsearch 將執(zhí)行一些預檢檢查，啟動請求，然后返回一個task 可用于任務 API 以取消或獲取任務狀態(tài)的內(nèi)容。Elasticsearch 還將創(chuàng)建此任務的記錄作為文檔，位于.tasks/task/${taskId}. 這是您認為合適的保留或刪除。完成后，將其刪除，以便 Elasticsearch 可以回收它使用的空間。

scroll_size：游標查詢，根據(jù)index.max_result_window值設(shè)置，scroll_size應當小于index.max_result_window值，默認是10000

conflicts：在_delete_by_query執(zhí)行過程中，依次執(zhí)行多個搜索請求，以便找到所有匹配的文檔進行刪除。每找到一批文檔，就會執(zhí)行相應的批量請求，刪除所有這些文檔。如果搜索或批量請求被拒絕，_delete_by_query 則依靠默認策略重試被拒絕的請求（最多 10 次，指數(shù)回退）。達到最大重試限制會導致_delete_by_query 中止，并且所有失敗都在failures響應中返回。已執(zhí)行的刪除仍然存在。換句話說，該過程沒有回滾，只是中止。當?shù)谝淮问е轮兄箷r，失敗的批量請求返回的所有失敗都在failures 元素; 因此，可能會有相當多的失敗實體。如果您想計算版本沖突而不是導致它們中止，請conflicts=proceed在 url 或"conflicts": "proceed"請求正文中設(shè)置。

2.根據(jù)task查看正在刪除任務的狀態(tài)細節(jié)

GET http://172.16.96.*:9200/_tasks?detailed=true&actions=*/delete/byquery

{
"nodes": {}
}

示例：

"w4axaLWqQiq19k0wYvHQIw:4550655": {
"node": "w4axaLWqQiq19k0wYvHQIw",
"id": 4550655,
"type": "transport",
"action": "indices:data/write/delete/byquery",
"status": {
"slice_id": 5,
"total": 43088333,
"updated": 0,
"created": 0,
"deleted": 0,
"batches": 1,
"version_conflicts": 0,
"noops": 0,
"retries": {
"bulk": 0,
"search": 0
},
"throttled_millis": 0,
"requests_per_second": -1.0,
"throttled_until_millis": 0
},
"description": "delete-by-query [history-alarm-data]",
"start_time_in_millis": 1644559029887,
"running_time_in_nanos": 5024318204892,
"cancellable": true,
"cancelled": false,
"parent_task_id": "w4axaLWqQiq19k0wYvHQIw:4550644",
"headers": {}
}

如果刪除任務完成了，返回如下：

{
"nodes": {}
}

數(shù)據(jù)查詢?nèi)蝿眨?span toymoban-style="hidden">文章來源地址http://www.zghlxwxcb.cn/news/detail-529061.html

GET http://172.16.96.*:9200/_tasks?detailed=true&actions=*/read/search

{
"nodes": {
"YM7smnJOQGWJ4VJf_THNNg": {
"name": "YM7smnJ",
"transport_address": "172.16.96.*:9300",
"host": "172.16.96.*",
"ip": "172.16.96.*:9300",
"roles": [
"master",
"data",
"ingest"
],
"tasks": {
"YM7smnJOQGWJ4VJf_THNNg:172443363": {
"node": "YM7smnJOQGWJ4VJf_THNNg",
"id": 172443363,
"type": "netty",
"action": "indices:data/read/search",
"description": "indices[tracker_travel], types[travel], search_type[DFS_QUERY_THEN_FETCH], source[{\"from\":0,\"size\":5000,\"query\":{\"bool\":{\"must\":[{\"match\":{\"imei\":{\"query\":\"863637044306814\",\"operator\":\"OR\",\"prefix_length\":0,\"max_expansions\":50,\"fuzzy_transpositions\":true,\"lenient\":false,\"zero_terms_query\":\"NONE\",\"boost\":1.0}}},{\"range\":{\"begin_time\":{\"from\":1644795551000,\"to\":null,\"include_lower\":true,\"include_upper\":true,\"boost\":1.0}}}],\"disable_coord\":false,\"adjust_pure_negative\":true,\"boost\":1.0}},\"version\":true,\"sort\":[{\"begin_time\":{\"order\":\"asc\"}}]}]",
"start_time_in_millis": 1644819308136,
"running_time_in_nanos": 21187617,
"cancellable": true
},
"YM7smnJOQGWJ4VJf_THNNg:172443360": {
"node": "YM7smnJOQGWJ4VJf_THNNg",
"id": 172443360,
"type": "netty",
"action": "indices:data/read/search",
"description": "indices[tracker_travel], types[travel], search_type[DFS_QUERY_THEN_FETCH], source[{\"from\":0,\"size\":5000,\"query\":{\"bool\":{\"must\":[{\"match\":{\"imei\":{\"query\":\"863637044306806\",\"operator\":\"OR\",\"prefix_length\":0,\"max_expansions\":50,\"fuzzy_transpositions\":true,\"lenient\":false,\"zero_terms_query\":\"NONE\",\"boost\":1.0}}},{\"range\":{\"begin_time\":{\"from\":1644806393000,\"to\":null,\"include_lower\":true,\"include_upper\":true,\"boost\":1.0}}}],\"disable_coord\":false,\"adjust_pure_negative\":true,\"boost\":1.0}},\"version\":true,\"sort\":[{\"begin_time\":{\"order\":\"asc\"}}]}]",
"start_time_in_millis": 1644819308131,
"running_time_in_nanos": 26798586,
"cancellable": true
},
"YM7smnJOQGWJ4VJf_THNNg:172443354": {
"node": "YM7smnJOQGWJ4VJf_THNNg",
"id": 172443354,
"type": "netty",
"action": "indices:data/read/search",
"description": "indices[tracker_travel], types[travel], search_type[DFS_QUERY_THEN_FETCH], source[{\"from\":0,\"size\":5000,\"query\":{\"bool\":{\"must\":[{\"match\":{\"imei\":{\"query\":\"863637044306749\",\"operator\":\"OR\",\"prefix_length\":0,\"max_expansions\":50,\"fuzzy_transpositions\":true,\"lenient\":false,\"zero_terms_query\":\"NONE\",\"boost\":1.0}}},{\"range\":{\"begin_time\":{\"from\":1644792687000,\"to\":null,\"include_lower\":true,\"include_upper\":true,\"boost\":1.0}}}],\"disable_coord\":false,\"adjust_pure_negative\":true,\"boost\":1.0}},\"version\":true,\"sort\":[{\"begin_time\":{\"order\":\"asc\"}}]}]",
"start_time_in_millis": 1644819308126,
"running_time_in_nanos": 31671549,
"cancellable": true
}
}
}
}
}

到了這里，關(guān)于elasticsearch刪除大批量數(shù)據(jù)方法的文章就介紹完了。如果您還想了解更多內(nèi)容，請在右上角搜索TOY模板網(wǎng)以前的文章或繼續(xù)瀏覽下面的相關(guān)文章，希望大家以后多多支持TOY模板網(wǎng)！