所有: 现在,我想使用“扫描”和“滚动”按日期删除数据。起初,它运作良好。有一天,它不起作用。响应返回空结果列表。 然后,我试图删除代码 - '设置SearchType = Scan',它可以工作。但是,滚动操作变得特别慢。现在,我在另一个ES Cluster上运行代码,效果很好。
ES版本:2.1.2 这是代码:
public static void main(String[] args) {
String date = args[0];
logger.info("delete date:" + date);
QueryBuilder qb = QueryBuilders.termQuery("date", date);
Client client = EsConfigUtil.getClient();
new DeleteDataByDate().scanEs(client, qb, 6000, "ad_user_info", "impression_user_data");
}
private void scanEs(Client client, QueryBuilder qb, int size, String indices, String indexType) {
SearchResponse scrollResp = client.prepareSearch(indices)
.**setSearchType(SearchType.SCAN)** // THIE IS THE KEY.
.setScroll(new TimeValue(60000))
.setQuery(qb)
.setSize(size).execute().actionGet();
long start = System.currentTimeMillis();
//Scroll until no hits are returned
BulkRequestBuilder bulkRequest = null;
long totalNum = 0;
while (true) {
long scanStart = System.currentTimeMillis();
if (scrollResp.getHits().hits().length > 0) {
bulkRequest = client.prepareBulk();
for (SearchHit hit : scrollResp.getHits().getHits()) {
bulkRequest.add(client.prepareDelete(indices, indexType, hit.getId()));
totalNum += 1;
}
BulkResponse response = bulkRequest.execute().actionGet();
logger.info("bulk time:" + response.getTookInMillis());
}
scrollResp = client.prepareSearchScroll(scrollResp.getScrollId())
.setScroll(new TimeValue(60000))
.execute().actionGet();
if (scrollResp.getHits().getHits().length == 0) {
logger.error("hits length is 0, QB:" + qb.toString());
break;
}
logger.info("scan time :" + (System.currentTimeMillis() - scanStart));
}
long end = System.currentTimeMillis();
logger.info("total time:" + (end - start));
logger.info("total handle num:" + totalNum);
}
我的贫穷英语,帮助我!!!