Question

我有关于运行hadoop mapreduce作业的问题。我有一个表工作人员，按加入日期分区。创建这样的语句：

创建由（join_date字符串）分隔的表格谱（id int，age int）以'\;'结尾的行格式分隔字段;

我把一些数据放到'20130921'，然后当我执行下面的语句时，结果还可以：

select count(*) from staff where join_date='20130921';**

但是当我执行分区'20130922'（没有数据的分区）时，地图缩减作业等待时间太长，看起来像是永远运行：

hive> select count(*) from staff where join_date='20130922';**

Total MapReduce jobs = 1**

Launching Job 1 out of 1**

**Number of reduce tasks determined at compile time: 1**

**In order to change the average load for a reducer (in bytes):**

    set hive.exec.reducers.bytes.per.reducer=<number>**

**In order to limit the maximum number of reducers:**

    set hive.exec.reducers.max=<number>**

**In order to set a constant number of reducers:**

    set mapred.reduce.tasks=<number>**

**Starting Job** = `job_201309231116_0131, Tracking URL = ....jobid=job_201309231116_0131`

**Kill Command** = `/u01/hadoop-0.20.203.0/bin/../bin/hadoop job  -kill job_201309231116_0131`

Hadoop job information for Stage-1: number of mappers: 0; number of reducers: 1
2013-09-23 17:19:07,182 Stage-1 map = 0%,  reduce = 0%
2013-09-23 17:19:07,182 Stage-1 map = 0%,  reduce = 0%
2013-09-23 17:19:07,182 Stage-1 map = 0%,  reduce = 0%

jobtracker show减少任务待定，这项工作似乎无法完成。

我使用hadoop-0.20.203.0和hive-0.10.0。我一整天用Google搜索，但没有发现任何话题都有同样的问题，请帮助我。

最好的问候。

Answer 1

这似乎是您的Hive安装的问题。我遇到了类似的问题。您可以尝试重新启动Hive Server和Hive Metastore。这解决了我的问题。

hadoop map减少了等待太久的工作

1 个答案: