ElasticSearch Java扫描并滚动返回空列表

时间:2016-12-13 06:05:56

标签: elasticsearch

所有:   现在,我想使用“扫描”和“滚动”按日期删除数据。起初,它运作良好。有一天,它不起作用。响应返回空结果列表。   然后,我试图删除代码 - '设置SearchType = Scan',它可以工作。但是,滚动操作变得特别慢。现在,我在另一个ES Cluster上运行代码,效果很好。

ES版本:2.1.2   这是代码:

public static void main(String[] args) {
    String date = args[0];
    logger.info("delete date:" + date);
    QueryBuilder qb = QueryBuilders.termQuery("date", date);

    Client client = EsConfigUtil.getClient();
    new DeleteDataByDate().scanEs(client, qb, 6000, "ad_user_info", "impression_user_data");
}

private void scanEs(Client client, QueryBuilder qb, int size, String indices, String indexType) {
    SearchResponse scrollResp = client.prepareSearch(indices)
            .**setSearchType(SearchType.SCAN)**  // THIE IS THE KEY.
            .setScroll(new TimeValue(60000))
            .setQuery(qb)
            .setSize(size).execute().actionGet();
    long start = System.currentTimeMillis();
    //Scroll until no hits are returned
    BulkRequestBuilder bulkRequest = null;
    long totalNum = 0;
    while (true) {
        long scanStart = System.currentTimeMillis();
        if (scrollResp.getHits().hits().length > 0) {
            bulkRequest = client.prepareBulk();
            for (SearchHit hit : scrollResp.getHits().getHits()) {
                bulkRequest.add(client.prepareDelete(indices, indexType, hit.getId()));
                totalNum += 1;
            }
            BulkResponse response = bulkRequest.execute().actionGet();
            logger.info("bulk time:" + response.getTookInMillis());
        }
        scrollResp = client.prepareSearchScroll(scrollResp.getScrollId())
                .setScroll(new TimeValue(60000))
                .execute().actionGet();
        if (scrollResp.getHits().getHits().length == 0) {
            logger.error("hits length is 0, QB:" + qb.toString());
            break;
        }
        logger.info("scan time :" + (System.currentTimeMillis() - scanStart));
    }
    long end = System.currentTimeMillis();
    logger.info("total time:" + (end - start));
    logger.info("total handle num:" + totalNum);
}

我的贫穷英语,帮助我!!!

0 个答案:

没有答案
相关问题