Java AmazonS3Client getObject挂起,在socketRead0期间线程状态卡在IN_NATIVE中

时间:2015-10-16 12:54:01

标签: java sockets amazon-s3

我有一个Java应用程序,它运行一系列并行线程,从AmazonS3下载大量对象。我注意到,在没有任何问题的情况下运行了几个小时之后,最终得到一个特定的块会挂起并导致程序冻结。

Java进程仍在运行,但CPU使用率和网络IO都很小。执行线程转储时,有一个线程永远不会离开IN_NATIVE状态,而socketRead0()似乎永远不会在SSL握手期间返回。这是释义的堆栈跟踪:

Thread 17260: (state = IN_NATIVE)
 - java.net.SocketInputStream.socketRead0(java.io.FileDescriptor, byte[], int, int, int) @bci=0 (Compiled frame; information may be imprecise)
 - java.net.SocketInputStream.socketRead(java.io.FileDescriptor, byte[], int, int, int) @bci=8, line=116 (Compiled frame)
 - java.net.SocketInputStream.read(byte[], int, int, int) @bci=79, line=170 (Compiled frame)
 - java.net.SocketInputStream.read(byte[], int, int) @bci=11, line=141 (Compiled frame)
 - sun.security.ssl.InputRecord.readFully(java.io.InputStream, byte[], int, int) @bci=21, line=465 (Compiled frame)
 - sun.security.ssl.InputRecord.read(java.io.InputStream, java.io.OutputStream) @bci=32, line=503 (Compiled frame)
 - sun.security.ssl.SSLSocketImpl.readRecord(sun.security.ssl.InputRecord, boolean) @bci=44, line=973 (Compiled frame)
 - sun.security.ssl.SSLSocketImpl.performInitialHandshake() @bci=84, line=1375 (Compiled frame)
 - sun.security.ssl.SSLSocketImpl.startHandshake(boolean) @bci=13, line=1403 (Compiled frame)
 - sun.security.ssl.SSLSocketImpl.startHandshake() @bci=2, line=1387 (Compiled frame)
 - org.apache.http.conn.ssl.SSLSocketFactory.connectSocket(int, java.net.Socket, org.apache.http.HttpHost, java.net.InetSocketAddress, java.net.InetSocketAddress, org.apache.http.protocol.HttpContext) @bci=87, line=533 (Compiled frame)
 - org.apache.http.conn.ssl.SSLSocketFactory.connectSocket(java.net.Socket, java.net.InetSocketAddress, java.net.InetSocketAddress, org.apache.http.params.HttpParams) @bci=69, line=401 (Compiled frame)
 - com.amazonaws.http.conn.ssl.SdkTLSSocketFactory.connectSocket(java.net.Socket, java.net.InetSocketAddress, java.net.InetSocketAddress, org.apache.http.params.HttpParams) @bci=60, line=128 (Compiled frame)
 - org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(org.apache.http.conn.OperatedClientConnection, org.apache.http.HttpHost, java.net.InetAddress, org.apache.http.protocol.HttpContext, org.apache.http.params.HttpParams) @bci=226, line=177 (Compiled frame)
 - org.apache.http.impl.conn.ManagedClientConnectionImpl.open(org.apache.http.conn.routing.HttpRoute, org.apache.http.protocol.HttpContext, org.apache.http.params.HttpParams) @bci=126, line=304 (Compiled frame)
 - org.apache.http.impl.client.DefaultRequestDirector.tryConnect(org.apache.http.impl.client.RoutedRequest, org.apache.http.protocol.HttpContext) @bci=50, line=610 (Compiled frame)
 - org.apache.http.impl.client.DefaultRequestDirector.execute(org.apache.http.HttpHost, org.apache.http.HttpRequest, org.apache.http.protocol.HttpContext) @bci=389, line=445 (Compiled frame)
 - org.apache.http.impl.client.AbstractHttpClient.doExecute(org.apache.http.HttpHost, org.apache.http.HttpRequest, org.apache.http.protocol.HttpContext) @bci=365, line=863 (Compiled frame)
 - org.apache.http.impl.client.CloseableHttpClient.execute(org.apache.http.client.methods.HttpUriRequest, org.apache.http.protocol.HttpContext) @bci=14, line=82 (Compiled frame)
 - org.apache.http.impl.client.CloseableHttpClient.execute(org.apache.http.client.methods.HttpUriRequest, org.apache.http.protocol.HttpContext) @bci=3, line=57 (Compiled frame)
 - com.amazonaws.http.AmazonHttpClient.executeOneRequest(com.amazonaws.Request, com.amazonaws.http.HttpResponseHandler, com.amazonaws.http.HttpResponseHandler, com.amazonaws.http.ExecutionContext, com.amazonaws.util.AWSRequestMetrics, com.amazonaws.http.AmazonHttpClient$ExecOneRequestParams) @bci=369, line=728 (Compiled frame)
 - com.amazonaws.http.AmazonHttpClient.executeHelper(com.amazonaws.Request, com.amazonaws.http.HttpResponseHandler, com.amazonaws.http.HttpResponseHandler, com.amazonaws.http.ExecutionContext) @bci=312, line=489 (Compiled frame)
 - com.amazonaws.http.AmazonHttpClient.execute(com.amazonaws.Request, com.amazonaws.http.HttpResponseHandler, com.amazonaws.http.HttpResponseHandler, com.amazonaws.http.ExecutionContext) @bci=150, line=310 (Compiled frame)
 - com.amazonaws.services.s3.AmazonS3Client.invoke(com.amazonaws.Request, com.amazonaws.http.HttpResponseHandler, java.lang.String, java.lang.String) @bci=168, line=3796 (Compiled frame)
 - com.amazonaws.services.s3.AmazonS3Client.getObject(com.amazonaws.services.s3.model.GetObjectRequest) @bci=235, line=1201 (Compiled frame)
...

以下是有关如何执行此操作的代码段:

protected static byte[] getChunk(AmazonS3Client client, long start, int offset, String bucket, String key) {
    if(offset < 1){
        return null;
    }
    S3Object obj = null;
    S3ObjectInputStream is = null;
    byte[] buffer = new byte[offset];

    try {
        GetObjectRequest request = new GetObjectRequest(bucket, key);
        request.setRange(start, start + offset);

        obj = client.getObject(request);
        is = obj.getObjectContent();
        int copied = 0;
        int length;
        while ((length = is.read(buffer,copied,offset-copied)) > 0){
            copied += length;
        }

        buffer = (copied < 1) ? null : Arrays.copyOf(buffer, copied);
    } catch (Exception e) {
        buffer = null;
    } finally {
        Util.close(is);
    }
    return buffer;
}

作为旁注,如果可以使用不同的AmazonS3Client配置解决此问题,则以下是客户端的实例化方式:

private static final String S3_ENDPOINT = "https://s3.amazonaws.com/";

public static AmazonS3Client getAmazonClient(String accessKey, String secretKey){
    AmazonS3Client client = null;
    try {
        ClientConfiguration clientConfig = new ClientConfiguration();
        clientConfig.setProtocol(Protocol.HTTP);

        AWSCredentials credentials = new BasicAWSCredentials(accessKey, secretKey);

        client = new AmazonS3Client(credentials, clientConfig);
        client.setEndpoint(S3_ENDPOINT);
    } catch (Exception e){
        e.printStackTrace();
        client = null;
    }
    return client;
}

(这也是aws-java-sdk版本1.10.9)。以下是JVM规范:

java version "1.8.0_51"
Java(TM) SE Runtime Environment (build 1.8.0_51-b16)
Java HotSpot(TM) 64-Bit Server VM (build 25.51-b03, mixed mode)

如何解决这个问题的任何想法都会挂起套接字初始化?

1 个答案:

答案 0 :(得分:0)

从你的代码中, 我可以看到这个

ClientConfiguration clientConfig = new ClientConfiguration();

ClientConfiguration附带了很多选项。

也许你可以用这个:

setSocketTimeout

并将其设置为5000。 根据文件:

  

设置在连接超时和关闭之前,通过已建立的打开连接传输数据的等待时间(以毫秒为单位)。值0表示无穷大,不建议使用。

DEFAULT_REQUEST_TIMEOUT的默认值为0。

REF:

DEFAULT_REQUEST_TIMEOUT

setSocketTimeout

相关问题