??概述

在使用graylog時，默認(rèn)分頁查詢存在限制，真實使用不能滿足，需要我們手動處理。當(dāng)查詢超過執(zhí)行長度時，會出現(xiàn)一下錯誤提示

While retrieving data for this widget, the following error(s) occurred:
Unable to perform search query: Elasticsearch exception [type=illegal_argument_exception, reason=Result window is too large, from + size must be less than or equal to: [10000] but was [3382050]. See the scroll api for a more efficient way to request large data sets. This limit can be set by changing the [index.max_result_window] index level setting.].

??Elasticsearch檢索問題

??Elasticsearch的max_result_window限制

問題描述

查詢超過10000頁，Elasticsearch出現(xiàn)異常

Elasticsearch exception [type=illegal_argument_exception, reason=Result window is too large, from + size must be less than or equal to: [10000] but was [7135950]. See the scroll api for a more efficient way to request large data sets. This limit can be set by changing the [index.max_result_window] index level setting.].

解決方案

方案一：修改配置文件，重啟Elasticsearch服務(wù)【Elasticsearch5.x版本以后不支持】

修改Elasticsearch 集群中的 配置文件 config/elasticsearch.yml
在配置文件最后增加一行，如下：

index.max_result_window: 1000000

??注意：
日志文件路徑：/var/log/elasticsearch/graylog.log

方案二：通過接口修改具體的index

具體操作命令，如下（比如，設(shè)置可查詢 1000000 條數(shù)據(jù)，其中 alarm 是index名稱）：
推薦使用全局修改方式。

# 修改個別索引
PUT alarm/_settings
{ 
  "max_result_window" : 1000000
} 
# 修改全局 100W
PUT _settings
{
    "index": {
        "max_result_window": "1000000"
    }
}  

CURL方式

curl -H "Content-Type: application/json" -XPUT http://127.0.0.1:9200/_all/_settings -d '{ "index" : { "max_result_window" : 1000000}}'

注意：文章來源地址http://www.zghlxwxcb.cn/news/detail-756898.html

• 上述修改方式，對于新建的索引不會生效。如果需要讓新建的索引也生效，必須重新覆蓋_template

方案三：修改template【推薦】

curl -H "Content-Type: application/json" -XPUT http://127.0.0.1:9200/_template/graylog-gdmp-mapping -d '{
  "order": 1,
  "index_patterns": [
    "gdmp_*"
  ],
  "settings": {
    "index": {
      "analysis": {
        "analyzer": {
          "analyzer_keyword": {
            "filter": "lowercase",
            "tokenizer": "keyword"
          }
        }
      },
      "max_result_window": 1000000
    }
  },
  "mappings": {
    "_source": {
      "enabled": true
    },
    "dynamic_templates": [
      {
        "internal_fields": {
          "mapping": {
            "type": "keyword"
          },
          "match_mapping_type": "string",
          "match": "gl2_*"
        }
      },
      {
        "store_generic": {
          "mapping": {
            "type": "keyword"
          },
          "match_mapping_type": "string"
        }
      }
    ],
    "properties": {
      "streams": {
        "type": "keyword"
      },
      "message": {
        "fielddata": false,
        "analyzer": "standard",
        "type": "text"
      },
      "timestamp": {
        "format": "uuuu-MM-dd HH:mm:ss.SSS",
        "type": "date"
      }
    }
  }
}'

# 查看索引映射 /索引名/_mapping
GET /gdmp_f08985deb3064a02ab46eeaff55fe001_0/_mapping
# 查看索引配置  /索引名/_settings
GET /gdmp_da7eb85c302f4224b10eeed5314c2cae_1/_settings

??參考資料

1. 使用elasticsearch分頁時報max_result_window is too large的錯誤解決方案 | 寶貝云計算知識分享
2. 京東面試題：ElasticSearch深度分頁解決方案_Java_小小怪下士_InfoQ寫作社區(qū)
3. https://www.cnblogs.com/rongfengliang/p/16845628.html
4. https://blog.csdn.net/weixin_44692700/article/details/122160837

到了這里，關(guān)于Graylog日志查詢超過10000限制問題的文章就介紹完了。如果您還想了解更多內(nèi)容，請在右上角搜索TOY模板網(wǎng)以前的文章或繼續(xù)瀏覽下面的相關(guān)文章，希望大家以后多多支持TOY模板網(wǎng)！