springboot實(shí)現(xiàn)ES多種分頁(yè)方式

這篇具有很好參考價(jià)值的文章主要介紹了springboot實(shí)現(xiàn)ES多種分頁(yè)方式。希望對(duì)大家有所幫助。如果存在錯(cuò)誤或未考慮完全的地方，請(qǐng)大家不吝賜教，您也可以點(diǎn)擊"舉報(bào)違法"按鈕提交疑問(wèn)。

es有兩種分頁(yè)方式

from+size 淺分頁(yè)
scroll 深分頁(yè)

在這里就不展開(kāi)介紹了，網(wǎng)絡(luò)上有很多相關(guān)介紹。
我們還可以通過(guò)維護(hù)自增主鍵來(lái)實(shí)現(xiàn)分頁(yè)
下面結(jié)合Springboot，通過(guò)代碼展現(xiàn)實(shí)現(xiàn)es分頁(yè)的三種實(shí)現(xiàn)方式

需要引入依賴：

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-data-elasticsearch</artifactId>
            <version>2.2.12.RELEASE</version>
        </dependency>

代碼：

ESBase.java

@Service
@Slf4j
public class ESBase<T, U extends UserIdxESReq> {

//    @Autowired
//    ElasticsearchRepository<T, String> elasticsearchRepository;

    @Autowired
    ElasticsearchTemplate elasticsearchTemplate;

    @Value("${scroll.timeout.ms:60000}")
    private long scrollTimeoutMs;

    public NativeSearchQueryBuilder getNativeSearchQueryBuilder(U esReq) {
        /*NativeSearchQueryBuilder searchQueryBuilder = new NativeSearchQueryBuilder();
        BoolQueryBuilder boolQueryBuilder = QueryBuilders.boolQuery();
        if (StrUtil.isNotBlank(taskItemAddReq.getSettleNo())) {
            TermsQueryBuilder deptNoTermsQb = QueryBuilders.termsQuery(CommonConstant.SETTLE_NO, portBillDetailFactoringFreightReq.getSettleNo());
            boolQueryBuilder.must(deptNoTermsQb);
        }
        searchQueryBuilder.withQuery(boolQueryBuilder);*/
        return new NativeSearchQueryBuilder();
    }

    protected NativeSearchQuery getNativeSearchQuery(U esReq) {
        String reqStr = JSONUtil.toJsonStr(esReq);
        String reqMd5 = SecureUtil.md5(reqStr);
        NativeSearchQuery searchQuery = getNativeSearchQueryBuilder(esReq)
//                .withSort(SortBuilders.fieldSort("pin").order(SortOrder.ASC))
                .withPageable(PageRequest.of(esReq.getPageNo(),
                        esReq.getPageSize())).build();
        searchQuery.setPreference(reqMd5);
        return searchQuery;
    }

    /**
     * 通過(guò)自增主鍵來(lái)實(shí)現(xiàn)分頁(yè)
     * @param start
     * @param end
     * @param clazz
     * @return
     */
    public List<T> autoIncrementKeyPage(long start, long end, Class<T> clazz) {
        NativeSearchQueryBuilder searchQueryBuilder = new NativeSearchQueryBuilder();
        BoolQueryBuilder boolQueryBuilder = QueryBuilders.boolQuery();
        RangeQueryBuilder rangeQueryBuilder = QueryBuilders.rangeQuery("key").gte(start).lte(end);
        boolQueryBuilder.must(rangeQueryBuilder);

        searchQueryBuilder.withQuery(boolQueryBuilder);
        NativeSearchQuery searchQuery = searchQueryBuilder
                .withSort(SortBuilders.fieldSort("key").order(SortOrder.ASC))
                .build();
        List<T> list = elasticsearchTemplate.queryForList(searchQuery, clazz);
        return list;
    }

    /**
     * from+size 淺分頁(yè)
     * @param esReq
     * @return
     */
    public EsQueryPojo<List<T>> fromSizePage(U esReq, Class<T> clazz) {
        Page<T> page = elasticsearchTemplate.queryForPage(getNativeSearchQuery(esReq), clazz);
//        Page<T> page = elasticsearchRepository.search(getNativeSearchQuery(esReq));
        EsQueryPojo<List<T>> listEsQueryPojo = new EsQueryPojo<>();
        listEsQueryPojo.setRows(page.getContent());//當(dāng)前分頁(yè)返回的結(jié)果
        listEsQueryPojo.setPage(page.getNumber());//當(dāng)前頁(yè)碼
        listEsQueryPojo.setTotal(page.getTotalPages());//分頁(yè)總數(shù)
        listEsQueryPojo.setRecords(page.getTotalElements());//元素總數(shù)
        listEsQueryPojo.setSize(page.getSize());//當(dāng)前頁(yè)返回的數(shù)量
        return listEsQueryPojo;
    }

    /**
     * scroll 深分頁(yè),另一種方式是通過(guò)維護(hù)自增主鍵
     * @param esReq
     * @return
     */
    public EsScrollQueryPojo<List<T>> scrollPage(U esReq, Class<T> clazz) {
        EsScrollQueryPojo<List<T>> listEsQueryPojo = new EsScrollQueryPojo<>();
        ScrolledPage<T> scrolledPage;
        try {
            if (StringUtils.isEmpty(esReq.getScrollId())) {
                //第一次查詢
                scrolledPage = elasticsearchTemplate.startScroll(scrollTimeoutMs, getNativeSearchQuery(esReq), clazz);//游標(biāo)的過(guò)期時(shí)間是指兩個(gè)查詢之間的間隔，而不是整個(gè)查詢過(guò)程
            } else {
                //帶scrollId查詢
                scrolledPage = elasticsearchTemplate.continueScroll(esReq.getScrollId(), scrollTimeoutMs, clazz);
            }

            listEsQueryPojo.setRows(scrolledPage.getContent());//當(dāng)前分頁(yè)返回的結(jié)果
            listEsQueryPojo.setPage(scrolledPage.getNumber());//當(dāng)前頁(yè)碼，在深度分頁(yè)不生效
            listEsQueryPojo.setTotal(scrolledPage.getTotalPages());//分頁(yè)總數(shù)
            listEsQueryPojo.setRecords(scrolledPage.getTotalElements());//元素總數(shù)
            listEsQueryPojo.setSize(scrolledPage.getSize());//當(dāng)前返回的數(shù)量
            listEsQueryPojo.setScrollId(scrolledPage.getScrollId());
        } catch (Exception e) {
            log.error(ExceptionEnum.ES_SCROLL_PAGE_EXCEPTION.getMsg() + e.getMessage());
            throw e;
        }
        return listEsQueryPojo;
    }

    public void clearScroll(String scrollId) {
        if (scrollId != null) {
            elasticsearchTemplate.clearScroll(scrollId);
        }
    }

    public void deleteIndexData(Class<T> clazz) {
        if (elasticsearchTemplate.indexExists(clazz)) {
            DeleteQuery deleteQuery = new DeleteQuery();
            deleteQuery.setQuery(QueryBuilders.matchAllQuery());
            elasticsearchTemplate.delete(deleteQuery, clazz);//刪除語(yǔ)句
            log.info("clearing index data");
        } else {
            log.warn("index not exsit");
        }
    }

    public long countIndex(Class<T> clazz) {
        if (elasticsearchTemplate.indexExists(clazz)) {
            NativeSearchQueryBuilder searchQueryBuilder = new NativeSearchQueryBuilder();
            searchQueryBuilder.withQuery(QueryBuilders.matchAllQuery());
            return elasticsearchTemplate.count(searchQueryBuilder.build(), clazz);
        } else {
            log.warn("index not exsit");
            return 0L;
        }
    }
}

UserIdxES.java

@Service
@Slf4j
public class UserIdxES extends ESBase<UserIdx, UserIdxESReq> {

    @Override
    protected NativeSearchQuery getNativeSearchQuery(UserIdxESReq esReq) {
        String reqStr = JSONUtil.toJsonStr(esReq);
        String reqMd5 = SecureUtil.md5(reqStr);
        NativeSearchQuery searchQuery = getNativeSearchQueryBuilder(esReq)
//                .withSort(SortBuilders.fieldSort("pin").order(SortOrder.ASC))
                .withPageable(PageRequest.of(esReq.getPageNo(),
                        esReq.getPageSize())).build();
        searchQuery.setPreference(reqMd5);
        return searchQuery;
    }
}

UserIdx.java

@AllArgsConstructor
@NoArgsConstructor
@Data
@Document(indexName = "user_idx", type = "user_type")
public class UserIdx {

    @Id
    private String id;

    @Field(name = "pin", type = FieldType.Keyword)
    private String pin;

    @Field(name = "name", type = FieldType.Keyword)
    private String name;

    @Field(name = "sex", type = FieldType.Keyword)
    private String sex;

    @Field(name = "age", type = FieldType.Keyword)
    private String age;

    @Field(name = "score", type = FieldType.Keyword)
    private String score;
}

UserIdxESReq.java

@NoArgsConstructor
@AllArgsConstructor
@Data
@ApiModel(value = "UserIdxESReq", description = "UserIdxESReq request bean")
public class UserIdxESReq {

    @ApiModelProperty(example = "0")
    private int pageNo;

    @ApiModelProperty(example = "10")
    private int pageSize;

    @ApiModelProperty(value = "分頁(yè)ID", name = "scrollId")
    private String scrollId;
}

測(cè)試
ESBaseTest.java

@SpringBootTest
@RunWith(SpringRunner.class)
//TODO
public class ESBaseTest {
    @Autowired
    ESBase<UserIdx, UserIdxESReq> esBase;

    @Autowired
    ESBase<UserWithKeyIdx, UserIdxESReq> esBase1;
    @Test
    public void fromSizePageTest() {
        UserIdxESReq UserIdxEsReq = new UserIdxESReq();
        UserIdxEsReq.setPageNo(0);
        UserIdxEsReq.setPageSize(10);
        esBase.fromSizePage(UserIdxEsReq, UserIdx.class);
    }

    @Test
    public void scrollPageTest() {
        UserIdxESReq UserIdxEsReq = new UserIdxESReq();
        UserIdxEsReq.setPageNo(0);
        UserIdxEsReq.setPageSize(10);
        esBase.scrollPage(UserIdxEsReq, UserIdx.class);
    }

    @Test
    public void autoIncrementIdPageTest() {
        List<UserIdx> list = esBase.autoIncrementKeyPage(0, 1000, UserIdx.class);
        Assert.assertNotNull(list);
    }

    @Test
    public void deleteIndexDataTest() {
        esBase.deleteIndexData(UserIdx.class);
    }

    @Test
    public void countIndexTest() {
        long cnt = esBase1.countIndex(UserWithKeyIdx.class);
        Assert.assertEquals(cnt,100);
    }
}

UserIdxESBaseTest.java

@SpringBootTest
@RunWith(SpringRunner.class)
public class UserIdxESBaseTest {

    @Autowired
    UserIdxES esService;

    @Test
    public void fromSizePageTest() {
        UserIdxESReq esReq = new UserIdxESReq();
//        esReq.setAge(123);
//        esReq.setPageNo(0);
        esReq.setPageSize(40);
        int pages = 0;
        int currentPage = 0;
        do {
            esReq.setPageNo(currentPage);
            EsQueryPojo<List<UserIdx>> esScrollQueryPojo = esService.fromSizePage(esReq,UserIdx.class);
            pages = esScrollQueryPojo.getTotal();
            currentPage = esScrollQueryPojo.getPage();
            List<UserIdx> list = esScrollQueryPojo.getRows();
//            currentPage++;
        } while (currentPage++ < pages);
    }

    @Test
    public void scrollPageTest() {
        UserIdxESReq esReq = new UserIdxESReq();
        esReq.setPageNo(0);
        esReq.setPageSize(10);
//        esService.scrollPage(esReq, TaskItemAddIndex.class);
        EsScrollQueryPojo<List<UserIdx>> esScrollQueryPojo = esService.scrollPage(esReq, UserIdx.class);
        List<UserIdx> taskItemAddDtoList = (List<UserIdx>) esScrollQueryPojo.getRows();
        Assert.assertNotNull(taskItemAddDtoList);
    }
}

附錄：
2. http 測(cè)試筆記

測(cè)試es:
spring.data.elasticsearch.cluster-name=docker-cluster
spring.data.elasticsearch.cluster-nodes=x.x.x.x:9300

查看：
    curl -X GET http://x.x.x.x:9200/_cat
創(chuàng)建索引：
    curl -X PUT http://x.x.x.x:9200/user_with_key_idx
刪除索引：
    curl -XDELETE http://x.x.x.x:9200/user_with_key_idx
查看mapping
    curl -XGET http://x.x.x.x:9200/user_with_key_idx?pretty
    curl -XGET http://x.x.x.x:9200/user_with_key_idx?pretty
查詢數(shù)據(jù)：
    curl -XGET http://x.x.x.x:9200/user_with_key_idx/user_with_key_type/_search?pretty


深度分頁(yè)scroll游標(biāo)使用
    curl -XGET http://x.x.x.x:9200/user_with_key_idx/_search?scroll=1m \
    -H "Content-Type: application/json" \
    -d '
    {
        "query": { "match_all": {}},
        "sort" : ["_doc"],
        "size":  1
    }'

curl -XGET http://x.x.x.x:9200/_search/scroll?pretty \
-H "Content-Type: application/json" \
-d '{
    "scroll": "1m",
    "scroll_id" : "DnF1ZXJ5VGhlbkZldGNoBQAAAAAAAAJFFlNsSXFUcS1qUzFlYWFFWGpWUkE5RncAAAAAAAACRxZTbElxVHEtalMxZWFhRVhqVlJBOUZ3AAAAAAAAAkYWU2xJcVRxLWpTMWVhYUVYalZSQTlGdwAAAAAAAAJIFlNsSXFUcS1qUzFlYWFFWGpWUkE5RncAAAAAAAACSRZTbElxVHEtalMxZWFhRVhqVlJBOUZ3"
}'

創(chuàng)建mapping
curl -XPOST 'http://x.x.x.x:9200/user_with_key_idx/user_with_key_type/_mapping?pretty' -H "Content-Type: application/json"  \
-d '
{
    "user_with_key_type": {
        "properties": {
            "pin": {
                "type": "keyword"
            },
            "name": {
                "type": "keyword"
            },
            "sex": {
                "type": "keyword"
            },
            "age": {
                "type": "keyword"
            },
            "score": {
                "type": "keyword"
            },
            "key": {
                "type": "long"
            }
        }
    }
}'

user_with_key_idx/user_with_key_type/_mapping

PUT user_with_key_idx
{
    "mappings": {
        "user_with_key_type": {
                "properties": {
                    "pin": {
                        "type": "keyword"
                    },
                    "name": {
                        "type": "keyword"
                    },
                    "sex": {
                        "type": "keyword"
                    },
                    "age": {
                        "type": "keyword"
                    },
                    "score": {
                        "type": "keyword"
                    },
                     "key": {
                         "type": "long"
                     }
                }
        }
    }
}

插入數(shù)據(jù)：
    curl -XPOST http://x.x.x.x:9200/user_with_key_idx/user_with_key_type?pretty -H "Content-Type: application/json" \
    -d '{
        "pin": "pin_001",
        "phone": "1234567890",
        "name": "lisi",
        "bizData": "",
        "sex": 1,
        "age": 18
        }'

    curl -XPOST http://x.x.x.x:9200/user_with_key_idx/user_with_key_type?pretty -H "Content-Type: application/json" \
    -d '{
        "pin": "pin_004",
        "phone": "1234567890",
        "name": "lisi02",
        "bizData": "",
        "sex": 1,
        "age": 18
        }'


查詢數(shù)據(jù)：
    curl -XGET http://x.x.x.x:9200/user_with_key_idx/user_with_key_type/_search?pretty

深分頁(yè)和淺分頁(yè)的使用區(qū)別

scroll 深分頁(yè)

curl -XGET http://11.51.197.4:9200/user_idx/user_type/_search?pretty&scroll=3m \
-d '{
  "query":{
    "match_all": {}
  },
  "from": 0,
  "size": 3,
  "sort": [
    {
      "id": {
        "order": "asc"
      }
    }
  ]
}'

curl -XGET http://11.51.197.4:9200/_search/scroll \
-d '{
  "scroll_id":"DnF1ZXJ5VGhlbkZldGNoBQAAAAAAAABvFlNsSXFUcS1qUzFlYWFFWGpWUkE5RncAAAAAAAAAcRZTbElxVHEtalMxZWFhRVhqVlJBOUZ3AAAAAAAAAHAWU2xJcVRxLWpTMWVhYUVYalZSQTlGdwAAAAAAAAByFlNsSXFUcS1qUzFlYWFFWGpWUkE5RncAAAAAAAAAcxZTbElxVHEtalMxZWFhRVhqVlJBOUZ3",
  "scroll": "3m"
}'

-淺分頁(yè)，調(diào)整索引的配置項(xiàng)index.max_result_window文章來(lái)源地址http://www.zghlxwxcb.cn/news/detail-498065.html

PUT index_name/_settings
{
"index.max_result_window":10000
}

到了這里，關(guān)于springboot實(shí)現(xiàn)ES多種分頁(yè)方式的文章就介紹完了。如果您還想了解更多內(nèi)容，請(qǐng)?jiān)谟疑辖撬阉鱐OY模板網(wǎng)以前的文章或繼續(xù)瀏覽下面的相關(guān)文章，希望大家以后多多支持TOY模板網(wǎng)！