并行化实验时堆堆空间错误

时间:2016-01-25 18:44:40

标签: java parallel-processing heap weka

我有一个程序,旨在并行运行大量的weka实验。 为此,我构建了一个程序,从文件中加载protobuf实验描述。然后将它们包装在Runnable中,由ExecutorService执行。

public static Runnable runExperiment(
      ExperimentDescription experimentDescription) throws Exception {
    class RunnableExperiment implements Runnable {
      ExperimentDescription experimentDescription;
      public RunnableExperiment(
          ExperimentDescription experimentDescription) {
        this.experimentDescription = experimentDescription;
      }
      @Override
      public void run() {
        try {
          Experiment experiment = Experiment.build(experimentDescription);
          experiment.trainClassifier();
          experiment.evaluate();
          experiment=null;
        } catch (Exception e) {
            e.printStackTrace();
        }
      }
    }
    RunnableExperiment experiment = new RunnableExperiment(
          experimentDescription);
    return experiment;  
  }
public static void main(String[] args) throws Exception {
    int cores = Runtime.getRuntime().availableProcessors();
    ExecutorService executor = Executors.newFixedThreadPool(cores);
    for (ExperimentDescription experimentDescription : experimentsInFile) {
        executor.execute(runExperiment(experimentDescription));
     }
}

实验他们自己基本上只包含他们的数据集,最大100mb,他们从实验说明中指定的文件中读取。
我使用15芯和80G柱塞的强力机器,我用50-xmx50000m。 然而,我得到了典型的堆空间错误,我找不到我的错误 谢谢你的帮助。

0 个答案:

没有答案