南昌网站建设公司如何排名前十的大学

java查询es超过10000条数据

背景:需要每天零点导出es中日志数据到数据库中给数据分析人员做清洗，然后展示给业务人员。但在es中默认一次最多只能查询10000条数据。
在这里我就只贴一下关键代码

SearchRequest searchRequest = new SearchRequest("索引名");
SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();//时间字段
RangeQueryBuilder rangeQuery = QueryBuilders.rangeQuery("logdate").from("2025-03-20 00:00") // 设置开始时间.to("2025-03-20 23:00")         // 设置结束时间.format("yyyy-MM-dd HH:mm")    //设置时间格式.includeLower(true)            // 包含起始时间点.includeUpper(false);          // 不包含结束时间点
searchSourceBuilder.query(rangeQuery);
searchSourceBuilder.size(1000); //每批次大小
SortBuilder sortBuilder = SortBuilders.fieldSort("logDate").order(SortOrder.ASC);
searchSourceBuilder.sort(sortBuilder);        searchSourceBuilder.sort(SortBuilders.fieldSort("_id").order(SortOrder.ASC));
Object[] searchAfter = null;
do {if (searchAfter != null){searchSourceBuilder.searchAfter(searchAfter);}searchRequest.source(searchSourceBuilder);SearchResponse search = null;try {search = client.search(searchRequest, RequestOptions.DEFAULT);} catch (IOException e) {throw new RuntimeException(e);}SearchHit[] hits = search.getHits().getHits();if (hits.length==0){break;}for (SearchHit hit : hits) {System.out.println(hit.getSourceAsString());}//获取最后一个文档的排序值searchAfter = hits[hits.length-1].getSortValues();}while (searchAfter != null);System.out.println("总共查询到 "+i+" 条数据");}

上面这段代码的本质就是用的分页查询
所用es相关依赖为

<dependency><groupId>org.springframework.boot</groupId><artifactId>spring-boot-starter-data-elasticsearch</artifactId><version>2.5.14</version></dependency>

es客户端为RestHighLevelClient