http://hadoop.apache.org/docs/r2.1.0-beta/hadoop-yarn/hadoop-yarn-site/WritingYarnApplications.html
我试图通过上面的链接使示例正常工作。但我无法编译下面的代码
Resource capability = Records.newRecord(Resource.class);
capability.setMemory(512);
amContainer.setResource(capability);
// Set the container launch content into the
// ApplicationSubmissionContext
appContext.setAMContainerSpec(amContainer);
amContainer是ContainerLaunchContext,我的hadoop版本是2.1.0-beta。 我做了一些调查。我发现ContainerLaunchContext中没有方法“setResource”
我对此有3个问题
1)方法已被删除或什么?
2)如果方法已被删除,我现在该怎么办?
3)有没有关于纱线的文档,因为我发现网站上的文档很容易,我希望我能得到一本手册或者其他东西。例如,
capability.setMemory(512);
根据代码中的评论,我不知道它是512k或512M。
答案 0 :(得分:1)
这实际上是问题的正确解决方案。以前的答案可能导致执行错误!!!
@Dyin我不能在评论中使用它;)验证为2.2.0和2.3.0
为AppMaster设置资源的驱动程序:
ApplicationSubmissionContext appContext = app.getApplicationSubmissionContext();
ApplicationId appId = appContext.getApplicationId();
appContext.setApplicationName(this.appName);
// Set up the container launch context for the application master
ContainerLaunchContext amContainer = Records.newRecord(ContainerLaunchContext.class);
Resource capability = Records.newRecord(Resource.class);
capability.setMemory(amMemory);
appContext.setResource(capability);
appContext.setAMContainerSpec(amContainer);
Priority pri = Records.newRecord(Priority.class);
pri.setPriority(amPriority);
appContext.setPriority(pri);
appContext.setQueue(amQueue);
// Submit the application to the applications manager
yarnClient.submitApplication(appContext); // this.yarnClient = YarnClient.createYarnClient();
在ApplicationMaster中,您应该如何为容器(工作人员)指定资源。
private AMRMClient.ContainerRequest setupContainerAskForRM() {
// setup requirements for hosts
// using * as any host will do for the distributed shell app
// set the priority for the request
Priority pri = Records.newRecord(Priority.class);
pri.setPriority(requestPriority);
// Set up resource type requirements
// For now, only memory is supported so we set memory requirements
Resource capability = Records.newRecord(Resource.class);
capability.setMemory(containerMemory);
AMRMClient.ContainerRequest request = new AMRMClient.ContainerRequest(capability, null, null,
pri);
return request;
}
AppMaster中的某些run()或main()方法
AMRMClientAsync.CallbackHandler allocListener = new RMCallbackHandler();
resourceManager = AMRMClientAsync.createAMRMClientAsync(1000, allocListener);
resourceManager.init(conf);
resourceManager.start();
for (int i = 0; i < numTotalContainers; ++i) {
AMRMClient.ContainerRequest containerAsk = setupContainerAskForRM();
resourceManager.addContainerRequest(containerAsk); //
}
启动容器 您可以使用原始答案解决方案(java cmd),但它只是一个顶级的樱桃。无论如何它应该有用。
答案 1 :(得分:0)
您可以通过推荐设置ApplicationMaster可用的内存。就这样:
// Set the necessary command to execute the application master
Vector<CharSequence> vargs = new Vector<CharSequence>(30);
...
vargs.add("-Xmx" + amMemory + "m"); // notice "m" indicating megabytes, you can use also -Xms combined with -Xmx
... // transform vargs to String commands
amContainer.setCommands(commands);
这应该可以解决您的问题。至于3个问题。纱线是快速发展的软件。我的建议忘了文档,获取源代码并阅读它。这将回答你的很多问题。