Question

我正在使用SolrJ API 4.8将富文档索引到solr。但我想异步索引这些文档。我做的功能同步发送文件，但我不知道如何更改它以使其异步。有什么想法吗？

功能：

public Boolean indexDocument(HttpSolrServer server, String PathFile, InputReader external)
{  

        ContentStreamUpdateRequest up = new ContentStreamUpdateRequest("/update/extract");

        try {
                up.addFile(new File(PathFile), "text");
        } catch (IOException e) {
                Logger.getLogger(ANOIndexer.class.getName()).log(Level.SEVERE, null, e);
                return false;
        }

        up.setAction(AbstractUpdateRequest.ACTION.COMMIT, true, true);

        try {
                server.request(up);
        } catch (SolrServerException e) {
                Logger.getLogger(ANOIndexer.class.getName()).log(Level.SEVERE, null, e);
                return false;

        } catch (IOException e) {
                Logger.getLogger(ANOIndexer.class.getName()).log(Level.SEVERE, null, e);
                return false;   
        }
        return true;
}

Solr服务器：版本4.8

Answer 1

听起来您可能希望使用ExecutorService和FutureTask来执行此操作：

private static HttpSolrServer server;
private static int threadPoolSize = 4;  //Set this to something appropiate for your environment

public static void main(String[] args) {
    ExecutorService executor = Executors.newFixedThreadPool(threadPoolSize);
    ArrayList<FutureTask<Boolean>> taskList = new ArrayList<FutureTask<Boolean>>();
    ArrayList<String> paths = new ArrayList<String>();
    //Initialize your list of paths here

    for (String path : paths) {
        FutureTask<Boolean> futureTask = new FutureTask<Boolean>(new IndexDocumentTask(path));
        taskList.add(futureTask);
        executor.execute(futureTask);
    }

    for (int i = 0; i < taskList.size(); i++) {
        FutureTask<Boolean> futureTask = taskList.get(i);

        try {
            System.out.println("Index Task " + i + (futureTask.get() ? " finished successfully." : " encountered an error."));
        } catch (ExecutionException e) {
            System.out.println("An Execution Exception occurred with Index Task " + i);
        } catch (InterruptedException e) {
            System.out.println("An Interrupted Exception occurred with Index Task " + i);
        }
    }

    executor.shutdown();
}

static class IndexDocumentTask implements Callable<Boolean> {

    private String pathFile;

    public IndexDocumentTask(String pathFile) {
        this.pathFile = pathFile;
    }

    @Override
    public Boolean call() {
        return indexDocument(pathFile);
    }

    public Boolean indexDocument(String pathFile) {
        ContentStreamUpdateRequest up = new ContentStreamUpdateRequest("/update/extract");

        try {
            up.addFile(new File(pathFile), "text");
        } catch (IOException e) {
            Logger.getLogger(ANOIndexer.class.getName()).log(Level.SEVERE, null, e);
            return false;
        }

        up.setAction(AbstractUpdateRequest.ACTION.COMMIT, true, true);

        try {
            server.request(up);
        } catch (SolrServerException e) {
            Logger.getLogger(ANOIndexer.class.getName()).log(Level.SEVERE, null, e);
            return false;

        } catch (IOException e) {
            Logger.getLogger(ANOIndexer.class.getName()).log(Level.SEVERE, null, e);
            return false;
        }
        return true;
    }
}

这是未经测试的代码，因此我不确定调用server.request(up)是否是线程安全的。我认为只使用一个HttpSolrServer实例更简洁，但你也可以在每个任务中创建新的HttpSolrServer实例。

如果您愿意，可以扩充IndexDocumentTask以实现Callable<Tuple<String, Boolean>>，这样您就可以检索要编制索引的文档的文件名，以及索引是否成功。

即使我不认为一次向Solr服务器发送多个请求应该是一个问题，您可能想要限制您的请求，以免过载Solr服务器。

SolrJ - 使用ContentStreamUpdateRequest异步索引文档

1 个答案: