容器以非零退出代码134退出

时间:2016-10-05 08:52:40

标签: matlab ubuntu hadoop

我试图在Ubunt 2.7.0中使用MCR从Ubuntu 16.04中的Matlab R2016a(9.0.1)运行一些matlab部署的代码。

尝试执行Matlab示例的部署代码时,使用以下命令:

sudo ./run_airlinesmall.sh /usr/local/MATLAB/MATLAB_Runtime/v901 -D mw.mcrroot=/usr/local/MATLAB/MATLAB_Runtime/v901 "/pruebas/datasets/airline.csv" /pruebas/resultados/myresults

我收到以下错误输出:

    Using MATLAB_HADOOP_INSTALL: /usr/hadoop/hadoop-2.7.3/bin/hadoop
    Using HADOOP_PREFIX: /usr/hadoop/hadoop-2.7.3/bin/hadoop
    Using HADOOP_HOME: /usr/hadoop/hadoop-2.7.3/bin/hadoop
    Find out Hadoop version: /usr/hadoop/hadoop-2.7.3/bin/hadoop version
    Use Hadoop V2 JAR files
    ------------------------------------------
    Launch Hadoop job airlinesmall.ctf
    hola
    /usr/local/MATLAB/MATLAB_Runtime/v901
    Executing: /usr/hadoop/hadoop-2.7.3/bin/hadoop jar "/usr/local/MATLAB/MATLAB_Runtime/v901/toolbox/mlhadoop/jar/a2.2.0/mwmapreduce.jar" com.mathworks.hadoop.MWMapReduceDriver   "-D" "mw.mcrroot=/usr/local/MATLAB/MATLAB_Runtime/v901" "airlinesmall.ctf" "/pruebas/datasets/airline.csv" "/pruebas/resultados/myresults"
    java.library.path: /usr/hadoop/hadoop-2.7.3/lib/native
    HDFSCTFPath=hdfs://localhost:9000/user/root/airlinesmall/airlinesmall.ctf
    Uploading CTF into distributed cache completed.
    mapred.child.env: MCR_CACHE_ROOT=/tmp,LD_LIBRARY_PATH=/usr/local/MATLAB/MATLAB_Runtime/v901/runtime/glnxa64:/usr/local/MATLAB/MATLAB_Runtime/v901/bin/glnxa64
    mapred.child.java.opts: -Xmx200m -Djava.library.path=/usr/local/MATLAB/MATLAB_Runtime/v901/runtime/glnxa64:/usr/local/MATLAB/MATLAB_Runtime/v901/bin/glnxa64
    New java.library.path: /usr/hadoop/hadoop-2.7.3/lib/native:/usr/local/MATLAB/MATLAB_Runtime/v901/runtime/glnxa64:/usr/local/MATLAB/MATLAB_Runtime/v901/bin/glnxa64
    Using MATLAB mapper.
    Set input format class to: ChunkFileRecordReader.
    Using MATLAB reducer.
    Set outputformat class to: class org.apache.hadoop.mapreduce.lib.output.SequenceFileOutputFormat
    Set map output key class to: class com.mathworks.hadoop.MxArrayWritable2
    Set map output value class to: class com.mathworks.hadoop.MxArrayWritable2
    Set reduce output key class to: class com.mathworks.hadoop.MxArrayWritable2
    Set reduce output value class to: class com.mathworks.hadoop.MxArrayWritable2
    *************** run ******************
    16/10/05 10:27:59 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
    16/10/05 10:28:00 INFO input.FileInputFormat: Total input paths to process : 1
    16/10/05 10:28:00 INFO mapreduce.JobSubmitter: number of splits:1
    16/10/05 10:28:00 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1475655722794_0002
    16/10/05 10:28:00 INFO impl.YarnClientImpl: Submitted application application_1475655722794_0002
    16/10/05 10:28:00 INFO mapreduce.Job: The url to track the job: http://pcitbu:8088/proxy/application_1475655722794_0002/
    16/10/05 10:28:00 INFO mapreduce.Job: Running job: job_1475655722794_0002
    16/10/05 10:28:04 INFO mapreduce.Job: Job job_1475655722794_0002 running in uber mode : false
    16/10/05 10:28:04 INFO mapreduce.Job:  map 0% reduce 0%
    16/10/05 10:28:07 INFO mapreduce.Job: Task Id : attempt_1475655722794_0002_m_000000_0, Status : FAILED
    Exception from container-launch.
    Container id: container_1475655722794_0002_01_000002
    Exit code: 134
    Exception message: /bin/bash: línea 1:  6811 Abortado                (`core' generado) /usr/java/jdk1.8.0_101/bin/java -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Xmx200m -Djava.library.path=/usr/local/MATLAB/MATLAB_Runtime/v901/runtime/glnxa64:/usr/local/MATLAB/MATLAB_Runtime/v901/bin/glnxa64 -Djava.io.tmpdir=/tmp/hadoop-pcitbu/nm-local-dir/usercache/root/appcache/application_1475655722794_0002/container_1475655722794_0002_01_000002/tmp -Dlog4j.configuration=container-log4j.properties -Dyarn.app.container.log.dir=/usr/hadoop/hadoop-2.7.3/logs/userlogs/application_1475655722794_0002/container_1475655722794_0002_01_000002 -Dyarn.app.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA -Dhadoop.root.logfile=syslog org.apache.hadoop.mapred.YarnChild 127.0.1.1 35025 attempt_1475655722794_0002_m_000000_0 2 > /usr/hadoop/hadoop-2.7.3/logs/userlogs/application_1475655722794_0002/container_1475655722794_0002_01_000002/stdout 2> /usr/hadoop/hadoop-2.7.3/logs/userlogs/application_1475655722794_0002/container_1475655722794_0002_01_000002/stderr
...


Stack trace: ExitCodeException exitCode=134: /bin/bash: línea 1:  6944 Abortado                (`core' generado) /usr/java/jdk1.8.0_101/bin/java -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Xmx200m -Djava.library.path=/usr/local/MATLAB/MATLAB_Runtime/v901/runtime/glnxa64:/usr/local/MATLAB/MATLAB_Runtime/v901/bin/glnxa64 -Djava.io.tmpdir=/tmp/hadoop-pcitbu/nm-local-dir/usercache/root/appcache/application_1475655722794_0002/container_1475655722794_0002_01_000004/tmp -Dlog4j.configuration=container-log4j.properties -Dyarn.app.container.log.dir=/usr/hadoop/hadoop-2.7.3/logs/userlogs/application_1475655722794_0002/container_1475655722794_0002_01_000004 -Dyarn.app.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA -Dhadoop.root.logfile=syslog org.apache.hadoop.mapred.YarnChild 127.0.1.1 35025 attempt_1475655722794_0002_m_000000_2 4 > /usr/hadoop/hadoop-2.7.3/logs/userlogs/application_1475655722794_0002/container_1475655722794_0002_01_000004/stdout 2> /usr/hadoop/hadoop-2.7.3/logs/userlogs/application_1475655722794_0002/container_1475655722794_0002_01_000004/stderr

    at org.apache.hadoop.util.Shell.runCommand(Shell.java:582)
    at org.apache.hadoop.util.Shell.run(Shell.java:479)
    at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:773)
    at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:212)
    at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302)
    at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)


Container exited with a non-zero exit code 134

16/10/05 10:28:21 INFO mapreduce.Job:  map 100% reduce 100%
16/10/05 10:28:21 INFO mapreduce.Job: Job job_1475655722794_0002 failed with state FAILED due to: Task failed task_1475655722794_0002_m_000000
Job failed as tasks failed. failedMaps:1 failedReduces:0

16/10/05 10:28:21 INFO mapreduce.Job: Counters: 16
    Job Counters 
        Failed map tasks=4
        Killed reduce tasks=1
        Launched map tasks=4
        Other local map tasks=3
        Data-local map tasks=1
        Total time spent by all maps in occupied slots (ms)=8006
        Total time spent by all reduces in occupied slots (ms)=0
        Total time spent by all map tasks (ms)=8006
        Total time spent by all reduce tasks (ms)=0
        Total vcore-milliseconds taken by all map tasks=8006
        Total vcore-milliseconds taken by all reduce tasks=0
        Total megabyte-milliseconds taken by all map tasks=8198144
        Total megabyte-milliseconds taken by all reduce tasks=0
    Map-Reduce Framework
        CPU time spent (ms)=0
        Physical memory (bytes) snapshot=0
        Virtual memory (bytes) snapshot=0

我已经尝试了一段时间,但我无法解决它。

提前致谢。

编辑: 日志:

/usr/hadoop/hadoop-2.7.3/logs/userlogs/application_1475737682094_0001/container_1475737682094_0001_01_000001/stderr

oct 06, 2016 9:10:36 AM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register
INFORMACIÓN: Registering org.apache.hadoop.mapreduce.v2.app.webapp.JAXBContextResolver as a provider class
oct 06, 2016 9:10:36 AM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register
INFORMACIÓN: Registering org.apache.hadoop.yarn.webapp.GenericExceptionHandler as a provider class
oct 06, 2016 9:10:36 AM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register
INFORMACIÓN: Registering org.apache.hadoop.mapreduce.v2.app.webapp.AMWebServices as a root resource class
oct 06, 2016 9:10:36 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFORMACIÓN: Initiating Jersey application, version 'Jersey: 1.9 09/02/2011 11:17 AM'
oct 06, 2016 9:10:36 AM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider
INFORMACIÓN: Binding org.apache.hadoop.mapreduce.v2.app.webapp.JAXBContextResolver to GuiceManagedComponentProvider with the scope "Singleton"
oct 06, 2016 9:10:36 AM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider
INFORMACIÓN: Binding org.apache.hadoop.yarn.webapp.GenericExceptionHandler to GuiceManagedComponentProvider with the scope "Singleton"
oct 06, 2016 9:10:36 AM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider
INFORMACIÓN: Binding org.apache.hadoop.mapreduce.v2.app.webapp.AMWebServices to GuiceManagedComponentProvider with the scope "PerRequest"
log4j:WARN No appenders could be found for logger (org.apache.hadoop.ipc.Server).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.

/usr/hadoop/hadoop-2.7.3/logs/userlogs/application_1475737682094_0001/container_1475737682094_0001_01_000002/stderr(对于容器2,3和5也是如此)

terminate called after throwing an instance of 'boost::exception_detail::clone_impl<fl::i18n::MessageCatalog::MessageCatalogNotInitialized>'
  what():  Message Catalog has not been initialized by a successful call to MessageCatalog::MessageCatalogInit

nodemanager log:

2016-10-06 10:40:16,391 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor: Stack trace: ExitCodeException exitCode=134: /bin/bash: línea 1:  9908 Abortado                (`core' generado) /usr/java/jdk1.8.0_101/bin/java -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Xmx200m -Djava.library.path=/usr/local/MATLAB/MATLAB_Runtime/v901/runtime/glnxa64:/usr/local/MATLAB/MATLAB_Runtime/v901/bin/glnxa64 -Djava.io.tmpdir=/tmp/hadoop-pcitbu/nm-local-dir/usercache/root/appcache/application_1475742936553_0001/container_1475742936553_0001_01_000003/tmp -Dlog4j.configuration=container-log4j.properties -Dyarn.app.container.log.dir=/usr/hadoop/hadoop-2.7.3/logs/userlogs/application_1475742936553_0001/container_1475742936553_0001_01_000003 -Dyarn.app.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA -Dhadoop.root.logfile=syslog org.apache.hadoop.mapred.YarnChild 127.0.1.1 33516 attempt_1475742936553_0001_m_000000_1 3 > /usr/hadoop/hadoop-2.7.3/logs/userlogs/application_1475742936553_0001/container_1475742936553_0001_01_000003/stdout 2> /usr/hadoop/hadoop-2.7.3/logs/userlogs/application_1475742936553_0001/container_1475742936553_0001_01_000003/stderr
2016-10-06 10:40:16,392 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor: 
2016-10-06 10:40:16,392 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor:   at org.apache.hadoop.util.Shell.runCommand(Shell.java:582)
2016-10-06 10:40:16,392 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor:   at org.apache.hadoop.util.Shell.run(Shell.java:479)
2016-10-06 10:40:16,392 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor:   at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:773)
2016-10-06 10:40:16,392 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor:   at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:212)
2016-10-06 10:40:16,392 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor:   at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302)
2016-10-06 10:40:16,392 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor:   at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82)
2016-10-06 10:40:16,392 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor:   at java.util.concurrent.FutureTask.run(FutureTask.java:266)
2016-10-06 10:40:16,392 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor:   at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
2016-10-06 10:40:16,392 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor:   at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
2016-10-06 10:40:16,392 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor:   at java.lang.Thread.run(Thread.java:745)
    2016-10-06 10:40:16,392 WARN org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch: Container exited with a non-zero exit code 134
    2016-10-06 10:40:16,392 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerImpl: Container container_1475742936553_0001_01_000003 transitioned from RUNNING to EXITED_WITH_FAILURE
    2016-10-06 10:40:16,392 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch: Cleaning up container container_1475742936553_0001_01_000003
    2016-10-06 10:40:16,404 INFO org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor: Deleting absolute path : /tmp/hadoop-pcitbu/nm-local-dir/usercache/root/appcache/application_1475742936553_0001/container_1475742936553_0001_01_000003
    2016-10-06 10:40:16,404 WARN org.apache.hadoop.yarn.server.nodemanager.NMAuditLogger: USER=root OPERATION=Container Finished - Failed   TARGET=ContainerImpl    RESULT=FAILURE  DESCRIPTION=Container failed with state: EXITED_WITH_FAILURE    APPID=application_1475742936553_0001    CONTAINERID=container_1475742936553_0001_01_000003
    2016-10-06 10:40:16,404 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerImpl: Container container_1475742936553_0001_01_000003 transitioned from EXITED_WITH_FAILURE to DONE
    2016-10-06 10:40:16,404 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationImpl: Removing container_1475742936553_0001_01_000003 from application application_1475742936553_0001
    2016-10-06 10:40:16,404 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices: Got event CONTAINER_STOP for appId application_1475742936553_0001
    2016-10-06 10:40:16,479 INFO SecurityLogger.org.apache.hadoop.ipc.Server: Auth successful for appattempt_1475742936553_0001_000001 (auth:SIMPLE)
    2016-10-06 10:40:16,486 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl: Stopping container with container Id: container_1475742936553_0001_01_000003
    2016-10-06 10:40:16,487 INFO org.apache.hadoop.yarn.server.nodemanager.NMAuditLogger: USER=root IP=127.0.0.1    OPERATION=Stop Container Request    TARGET=ContainerManageImpl  RESULT=SUCCESS  APPID=application_1475742936553_0001    CONTAINERID=container_1475742936553_0001_01_000003
    2016-10-06 10:40:17,126 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Starting resource-monitoring for container_1475742936553_0001_01_000003
    2016-10-06 10:40:17,127 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Stopping resource-monitoring for container_1475742936553_0001_01_000003
    2016-10-06 10:40:17,169 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 9727 for container-id container_1475742936553_0001_01_000001: 397.7 MB of 2 GB physical memory used; 2.8 GB of 4.2 GB virtual memory used
    2016-10-06 10:40:18,410 INFO org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Removed completed containers from NM context: [container_1475742936553_0001_01_000003]
    2016-10-06 10:40:18,486 INFO SecurityLogger.org.apache.hadoop.ipc.Server: Auth successful for appattempt_1475742936553_0001_000001 (auth:SIMPLE)
    2016-10-06 10:40:18,490 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl: Start request for container_1475742936553_0001_01_000004 by user root
    2016-10-06 10:40:18,490 INFO org.apache.hadoop.yarn.server.nodemanager.NMAuditLogger: USER=root IP=127.0.0.1    OPERATION=Start Container Request   TARGET=ContainerManageImpl  RESULT=SUCCESS  APPID=application_1475742936553_0001    CONTAINERID=container_1475742936553_0001_01_000004
    2016-10-06 10:40:18,490 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationImpl: Adding container_1475742936553_0001_01_000004 to application application_1475742936553_0001
    2016-10-06 10:40:18,491 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerImpl: Container container_1475742936553_0001_01_000004 transitioned from NEW to LOCALIZING
    2016-10-06 10:40:18,491 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices: Got event CONTAINER_INIT for appId application_1475742936553_0001
    2016-10-06 10:40:18,491 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices: Got event APPLICATION_INIT for appId application_1475742936553_0001
    2016-10-06 10:40:18,491 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices: Got APPLICATION_INIT for service mapreduce_shuffle
    2016-10-06 10:40:18,491 INFO org.apache.hadoop.mapred.ShuffleHandler: Added token for job_1475742936553_0001
    2016-10-06 10:40:18,491 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerImpl: Container container_1475742936553_0001_01_000004 transitioned from LOCALIZING to LOCALIZED
    2016-10-06 10:40:18,504 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerImpl: Container container_1475742936553_0001_01_000004 transitioned from LOCALIZED to RUNNING
    2016-10-06 10:40:18,505 INFO org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor: 
...
org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor: Exit code: 134
    2016-10-06 10:40:25,458 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor: Exception message: /bin/bash: línea 1: 10040 Abortado                (`core' generado) /usr/java/jdk1.8.0_101/bin/java -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Xmx200m -Djava.library.path=/usr/local/MATLAB/MATLAB_Runtime/v901/runtime/glnxa64:/usr/local/MATLAB/MATLAB_Runtime/v901/bin/glnxa64 -Djava.io.tmpdir=/tmp/hadoop-pcitbu/nm-local-dir/usercache/root/appcache/application_1475742936553_0001/container_1475742936553_0001_01_000005/tmp -Dlog4j.configuration=container-log4j.properties -Dyarn.app.container.log.dir=/usr/hadoop/hadoop-2.7.3/logs/userlogs/application_1475742936553_0001/container_1475742936553_0001_01_000005 -Dyarn.app.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA -Dhadoop.root.logfile=syslog org.apache.hadoop.mapred.YarnChild 127.0.1.1 33516 attempt_1475742936553_0001_m_000000_3 5 > /usr/hadoop/hadoop-2.7.3/logs/userlogs/application_1475742936553_0001/container_1475742936553_0001_01_000005/stdout 2> /usr/hadoop/hadoop-2.7.3/logs/userlogs/application_1475742936553_0001/container_1475742936553_0001_01_000005/stderr
    2016-10-06 10:40:25,458 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor: 
    2016-10-06 10:40:25,458 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor: Stack trace: ExitCodeException exitCode=134: /bin/bash: línea 1: 10040 Abortado                (`core' generado) /usr/java/jdk1.8.0_101/bin/java -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Xmx200m -Djava.library.path=/usr/local/MATLAB/MATLAB_Runtime/v901/runtime/glnxa64:/usr/local/MATLAB/MATLAB_Runtime/v901/bin/glnxa64 -Djava.io.tmpdir=/tmp/hadoop-pcitbu/nm-local-dir/usercache/root/appcache/application_1475742936553_0001/container_1475742936553_0001_01_000005/tmp -Dlog4j.configuration=container-log4j.properties -Dyarn.app.container.log.dir=/usr/hadoop/hadoop-2.7.3/logs/userlogs/application_1475742936553_0001/container_1475742936553_0001_01_000005 -Dyarn.app.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA -Dhadoop.root.logfile=syslog org.apache.hadoop.mapred.YarnChild 127.0.1.1 33516 attempt_1475742936553_0001_m_000000_3 5 > /usr/hadoop/hadoop-2.7.3/logs/userlogs/application_1475742936553_0001/container_1475742936553_0001_01_000005/stdout 2> /usr/hadoop/hadoop-2.7.3/logs/userlogs/application_1475742936553_0001/container_1475742936553_0001_01_000005/stderr
    2016-10-06 10:40:25,458 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor: 
    2016-10-06 10:40:25,458 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor:   at org.apache.hadoop.util.Shell.runCommand(Shell.java:582)
    2016-10-06 10:40:25,458 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor:   at org.apache.hadoop.util.Shell.run(Shell.java:479)
    2016-10-06 10:40:25,458 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor:   at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:773)
    2016-10-06 10:40:25,458 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor:   at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:212)
    2016-10-06 10:40:25,458 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor:   at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302)
    2016-10-06 10:40:25,458 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor:   at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82)
    2016-10-06 10:40:25,458 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor:   at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    2016-10-06 10:40:25,458 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor:   at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    2016-10-06 10:40:25,458 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor:   at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    2016-10-06 10:40:25,458 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor:   at java.lang.Thread.run(Thread.java:745)
    2016-10-06 10:40:25,459 WARN org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch: Container exited with a non-zero exit code 134
    2016-10-06 10:40:25,459 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerImpl: Container container_1475742936553_0001_01_000005 transitioned from RUNNING to EXITED_WITH_FAILURE
    2016-10-06 10:40:25,459 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch: Cleaning up container container_1475742936553_0001_01_000005
    2016-10-06 10:40:25,468 ERROR org.apache.hadoop.yarn.server.nodemanager.NodeManager: RECEIVED SIGNAL 15: SIGTERM
    2016-10-06 10:40:25,497 INFO org.mortbay.log: Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@0.0.0.0:8042
    2016-10-06 10:40:25,497 WARN org.apache.hadoop.http.HttpServer2: HttpServer Acceptor: isRunning is false. Rechecking.
    2016-10-06 10:40:25,497 WARN org.apache.hadoop.http.HttpServer2: HttpServer Acceptor: isRunning is false
    2016-10-06 10:40:25,498 WARN org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor: Exit code from container container_1475742936553_0001_01_000001 is : 143
    2016-10-06 10:40:25,505 INFO org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor: Deleting absolute path : /tmp/hadoop-pcitbu/nm-local-dir/usercache/root/appcache/application_1475742936553_0001/container_1475742936553_0001_01_000005
    2016-10-06 10:40:25,505 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerImpl: Container container_1475742936553_0001_01_000001 transitioned from RUNNING to EXITED_WITH_FAILURE
    2016-10-06 10:40:25,505 WARN org.apache.hadoop.yarn.server.nodemanager.NMAuditLogger: USER=root OPERATION=Container Finished - Failed   TARGET=ContainerImpl    RESULT=FAILURE  DESCRIPTION=Container failed with state: EXITED_WITH_FAILURE    APPID=application_1475742936553_0001    CONTAINERID=container_1475742936553_0001_01_000005
    2016-10-06 10:40:25,505 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerImpl: Container container_1475742936553_0001_01_000005 transitioned from EXITED_WITH_FAILURE to DONE
    2016-10-06 10:40:25,505 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch: Cleaning up container container_1475742936553_0001_01_000001
    2016-10-06 10:40:25,518 INFO org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor: Deleting absolute path : /tmp/hadoop-pcitbu/nm-local-dir/usercache/root/appcache/application_1475742936553_0001/container_1475742936553_0001_01_000001
    2016-10-06 10:40:25,518 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationImpl: Removing container_1475742936553_0001_01_000005 from application application_1475742936553_0001
    2016-10-06 10:40:25,518 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices: Got event CONTAINER_STOP for appId application_1475742936553_0001
    2016-10-06 10:40:25,518 WARN org.apache.hadoop.yarn.server.nodemanager.NMAuditLogger: USER=root OPERATION=Container Finished - Failed   TARGET=ContainerImpl    RESULT=FAILURE  DESCRIPTION=Container failed with state: EXITED_WITH_FAILURE    APPID=application_1475742936553_0001    CONTAINERID=container_1475742936553_0001_01_000001
    2016-10-06 10:40:25,518 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerImpl: Container container_1475742936553_0001_01_000001 transitioned from EXITED_WITH_FAILURE to DONE
    2016-10-06 10:40:25,518 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationImpl: Removing container_1475742936553_0001_01_000001 from application application_1475742936553_0001
    2016-10-06 10:40:25,518 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices: Got event CONTAINER_STOP for appId application_1475742936553_0001
    2016-10-06 10:40:25,597 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl: Applications still running : [application_1475742936553_0001]
    2016-10-06 10:40:25,598 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl: Waiting for Applications to be Finished
    2016-10-06 10:40:25,598 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationImpl: Application application_1475742936553_0001 transitioned from RUNNING to APPLICATION_RESOURCES_CLEANINGUP
    2016-10-06 10:40:25,598 INFO org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor: Deleting absolute path : /tmp/hadoop-pcitbu/nm-local-dir/usercache/root/appcache/application_1475742936553_0001
    2016-10-06 10:40:25,598 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices: Got event APPLICATION_STOP for appId application_1475742936553_0001
    2016-10-06 10:40:25,599 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationImpl: Application application_1475742936553_0001 transitioned from APPLICATION_RESOURCES_CLEANINGUP to FINISHED
    2016-10-06 10:40:25,599 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.loghandler.NonAggregatingLogHandler: Scheduling Log Deletion for application: application_1475742936553_0001, with delay of 10800 seconds

完整日志:http://pastebin.com/j76TjcHF

现在有时打印该消息,然后系统重新启动。 如果您可能需要更多信息,请告诉我。

0 个答案:

没有答案