Question

我试图使用java来获取特定作业的所有计数器... 我已经编写了Hadoop 0.23.1的工作代码：

    JobClient client = new JobClient(new JobConf(createConfiguration()));
    RunningJob job;
    system.out.print("Looking for job with title containing the string '" + jobName + "'");
    List<JobStatus> jobStatusList = Arrays.asList(client.getAllJobs());

    Collections.sort(jobStatusList, new Comparator<JobStatus>() {

        @Override
        public int compare(JobStatus o1, JobStatus o2) {
            return (o1.getStartTime() < o2.getStartTime()) ? 1 : (o1.getStartTime() > o2.getStartTime() ? -1 : 0);
        }
    });

    for (JobStatus jobStatus : jobStatusList) {
        job = client.getJob(jobStatus.getJobID());
        if (job.getJobName().contains(jobName)) {
            system.out.print(String.format("Job Name '%s' was found in job ID: %s...", jobName, jobStatus.getJobID().toString()));
            return job.getCounters();
        }
    }
    system.out.print("Did not find job that contains the string '" + jobName + "'. Counters are set to null.");

知道为什么它不能在2.7.1中工作吗？

Answer 1

成功使用REST API：

http://<history server http address:port>/ws/v1/history/mapreduce/jobs/{jobid}/counters

如何在Hadoop 2.7.1中获得作业计数器？

1 个答案: