我有一个Java应用程序,它运行一系列并行线程,从AmazonS3下载大量对象。我注意到,在没有任何问题的情况下运行了几个小时之后,最终得到一个特定的块会挂起并导致程序冻结。
Java进程仍在运行,但CPU使用率和网络IO都很小。执行线程转储时,有一个线程永远不会离开IN_NATIVE状态,而socketRead0()似乎永远不会在SSL握手期间返回。这是释义的堆栈跟踪:
Thread 17260: (state = IN_NATIVE) - java.net.SocketInputStream.socketRead0(java.io.FileDescriptor, byte[], int, int, int) @bci=0 (Compiled frame; information may be imprecise) - java.net.SocketInputStream.socketRead(java.io.FileDescriptor, byte[], int, int, int) @bci=8, line=116 (Compiled frame) - java.net.SocketInputStream.read(byte[], int, int, int) @bci=79, line=170 (Compiled frame) - java.net.SocketInputStream.read(byte[], int, int) @bci=11, line=141 (Compiled frame) - sun.security.ssl.InputRecord.readFully(java.io.InputStream, byte[], int, int) @bci=21, line=465 (Compiled frame) - sun.security.ssl.InputRecord.read(java.io.InputStream, java.io.OutputStream) @bci=32, line=503 (Compiled frame) - sun.security.ssl.SSLSocketImpl.readRecord(sun.security.ssl.InputRecord, boolean) @bci=44, line=973 (Compiled frame) - sun.security.ssl.SSLSocketImpl.performInitialHandshake() @bci=84, line=1375 (Compiled frame) - sun.security.ssl.SSLSocketImpl.startHandshake(boolean) @bci=13, line=1403 (Compiled frame) - sun.security.ssl.SSLSocketImpl.startHandshake() @bci=2, line=1387 (Compiled frame) - org.apache.http.conn.ssl.SSLSocketFactory.connectSocket(int, java.net.Socket, org.apache.http.HttpHost, java.net.InetSocketAddress, java.net.InetSocketAddress, org.apache.http.protocol.HttpContext) @bci=87, line=533 (Compiled frame) - org.apache.http.conn.ssl.SSLSocketFactory.connectSocket(java.net.Socket, java.net.InetSocketAddress, java.net.InetSocketAddress, org.apache.http.params.HttpParams) @bci=69, line=401 (Compiled frame) - com.amazonaws.http.conn.ssl.SdkTLSSocketFactory.connectSocket(java.net.Socket, java.net.InetSocketAddress, java.net.InetSocketAddress, org.apache.http.params.HttpParams) @bci=60, line=128 (Compiled frame) - org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(org.apache.http.conn.OperatedClientConnection, org.apache.http.HttpHost, java.net.InetAddress, org.apache.http.protocol.HttpContext, org.apache.http.params.HttpParams) @bci=226, line=177 (Compiled frame) - org.apache.http.impl.conn.ManagedClientConnectionImpl.open(org.apache.http.conn.routing.HttpRoute, org.apache.http.protocol.HttpContext, org.apache.http.params.HttpParams) @bci=126, line=304 (Compiled frame) - org.apache.http.impl.client.DefaultRequestDirector.tryConnect(org.apache.http.impl.client.RoutedRequest, org.apache.http.protocol.HttpContext) @bci=50, line=610 (Compiled frame) - org.apache.http.impl.client.DefaultRequestDirector.execute(org.apache.http.HttpHost, org.apache.http.HttpRequest, org.apache.http.protocol.HttpContext) @bci=389, line=445 (Compiled frame) - org.apache.http.impl.client.AbstractHttpClient.doExecute(org.apache.http.HttpHost, org.apache.http.HttpRequest, org.apache.http.protocol.HttpContext) @bci=365, line=863 (Compiled frame) - org.apache.http.impl.client.CloseableHttpClient.execute(org.apache.http.client.methods.HttpUriRequest, org.apache.http.protocol.HttpContext) @bci=14, line=82 (Compiled frame) - org.apache.http.impl.client.CloseableHttpClient.execute(org.apache.http.client.methods.HttpUriRequest, org.apache.http.protocol.HttpContext) @bci=3, line=57 (Compiled frame) - com.amazonaws.http.AmazonHttpClient.executeOneRequest(com.amazonaws.Request, com.amazonaws.http.HttpResponseHandler, com.amazonaws.http.HttpResponseHandler, com.amazonaws.http.ExecutionContext, com.amazonaws.util.AWSRequestMetrics, com.amazonaws.http.AmazonHttpClient$ExecOneRequestParams) @bci=369, line=728 (Compiled frame) - com.amazonaws.http.AmazonHttpClient.executeHelper(com.amazonaws.Request, com.amazonaws.http.HttpResponseHandler, com.amazonaws.http.HttpResponseHandler, com.amazonaws.http.ExecutionContext) @bci=312, line=489 (Compiled frame) - com.amazonaws.http.AmazonHttpClient.execute(com.amazonaws.Request, com.amazonaws.http.HttpResponseHandler, com.amazonaws.http.HttpResponseHandler, com.amazonaws.http.ExecutionContext) @bci=150, line=310 (Compiled frame) - com.amazonaws.services.s3.AmazonS3Client.invoke(com.amazonaws.Request, com.amazonaws.http.HttpResponseHandler, java.lang.String, java.lang.String) @bci=168, line=3796 (Compiled frame) - com.amazonaws.services.s3.AmazonS3Client.getObject(com.amazonaws.services.s3.model.GetObjectRequest) @bci=235, line=1201 (Compiled frame) ...
以下是有关如何执行此操作的代码段:
protected static byte[] getChunk(AmazonS3Client client, long start, int offset, String bucket, String key) { if(offset < 1){ return null; } S3Object obj = null; S3ObjectInputStream is = null; byte[] buffer = new byte[offset]; try { GetObjectRequest request = new GetObjectRequest(bucket, key); request.setRange(start, start + offset); obj = client.getObject(request); is = obj.getObjectContent(); int copied = 0; int length; while ((length = is.read(buffer,copied,offset-copied)) > 0){ copied += length; } buffer = (copied < 1) ? null : Arrays.copyOf(buffer, copied); } catch (Exception e) { buffer = null; } finally { Util.close(is); } return buffer; }
作为旁注,如果可以使用不同的AmazonS3Client配置解决此问题,则以下是客户端的实例化方式:
private static final String S3_ENDPOINT = "https://s3.amazonaws.com/"; public static AmazonS3Client getAmazonClient(String accessKey, String secretKey){ AmazonS3Client client = null; try { ClientConfiguration clientConfig = new ClientConfiguration(); clientConfig.setProtocol(Protocol.HTTP); AWSCredentials credentials = new BasicAWSCredentials(accessKey, secretKey); client = new AmazonS3Client(credentials, clientConfig); client.setEndpoint(S3_ENDPOINT); } catch (Exception e){ e.printStackTrace(); client = null; } return client; }
(这也是aws-java-sdk版本1.10.9)。以下是JVM规范:
java version "1.8.0_51" Java(TM) SE Runtime Environment (build 1.8.0_51-b16) Java HotSpot(TM) 64-Bit Server VM (build 25.51-b03, mixed mode)
如何解决这个问题的任何想法都会挂起套接字初始化?
答案 0 :(得分:0)
从你的代码中, 我可以看到这个
ClientConfiguration clientConfig = new ClientConfiguration();
ClientConfiguration附带了很多选项。
也许你可以用这个:
setSocketTimeout
并将其设置为5000。 根据文件:
设置在连接超时和关闭之前,通过已建立的打开连接传输数据的等待时间(以毫秒为单位)。值0表示无穷大,不建议使用。
DEFAULT_REQUEST_TIMEOUT
的默认值为0。
REF: