检查Spark日志以获取在AWS集群中运行的Spark Streaming作业

时间:2018-10-18 10:56:29

标签: apache-spark spark-streaming

火花流作业失败,并且当我尝试使用

查找日志时
yarn logs -applicationId application_1539671310842_0001

除了“ Container”(容器)日志之外,我什么都没有看到

18/10/16 11:57:32 ERROR CoarseGrainedExecutorBackend: RECEIVED SIGNAL TERM

并在标准输出中

2018-10-16T11:56:29.937+0000: [CMS-concurrent-sweep-start]
2018-10-16T11:56:29.948+0000: [CMS-concurrent-sweep: 0.011/0.011 secs] [Times: user=0.06 sys=0.00, real=0.02 secs] 
2018-10-16T11:56:29.948+0000: [CMS-concurrent-reset-start]
2018-10-16T11:56:30.011+0000: [CMS-concurrent-reset: 0.063/0.063 secs] [Times: user=0.32 sys=0.01, real=0.06 secs] 
Heap
 par new generation   total 629120K, used 507490K [0x0000000080000000, 0x00000000aaaa0000, 0x00000000f7990000)
  eden space 559232K,  80% used [0x0000000080000000, 0x000000009b6effa8, 0x00000000a2220000)
  from space 69888K,  83% used [0x00000000a2220000, 0x00000000a5ac8a20, 0x00000000a6660000)
  to   space 69888K,   0% used [0x00000000a6660000, 0x00000000a6660000, 0x00000000aaaa0000)
 concurrent mark-sweep generation total 1398144K, used 62567K [0x00000000f7990000, 0x000000014cef0000, 0x0000000800000000)
 Metaspace       used 63612K, capacity 64486K, committed 64668K, reserved 1105920K
  class space    used 8067K, capacity 8286K, committed 8380K, reserved 1048576K
End of LogType:stdout

任何有关如何对失败的作业进行故障排除(该作业运行10个小时然后失败)的信息,将不胜感激。

0 个答案:

没有答案