I am trying to upload file using Drive.Files.Create api. It ran well but suddenly getting SocketTimeOut Exception after 10-15 request.
drive = new Drive.Builder(httpTransport, JSON_FACTORY, credential).setApplicationName(
APPLICATION_NAME).build();
Drive.Files.Create insert = drive.files().create(fileMetadata, mediaContent);
insert.setUseContentAsIndexableText(true);
File f=insert.execute();
Exception StackTrace:
Exception in thread "main" java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:170)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at sun.security.ssl.InputRecord.readFully(InputRecord.java:465)
at sun.security.ssl.InputRecord.read(InputRecord.java:503)
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:973)
at sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:930)
at sun.security.ssl.AppInputStream.read(AppInputStream.java:105)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
at java.io.BufferedInputStream.read1(BufferedInputStream.java:286)
at java.io.BufferedInputStream.read(BufferedInputStream.java:345)
at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:704)
at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:647)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1536)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1441)
at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480)
at sun.net.www.protocol.https.HttpsURLConnectionImpl.getResponseCode(HttpsURLConnectionImpl.java:338)
at com.google.api.client.http.javanet.NetHttpResponse.<init>(NetHttpResponse.java:37)
at com.google.api.client.http.javanet.NetHttpRequest.execute(NetHttpRequest.java:94)
at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:981)
at com.google.api.client.googleapis.media.MediaHttpUploader.executeCurrentRequestWithoutGZip(MediaHttpUploader.java:545)
at com.google.api.client.googleapis.media.MediaHttpUploader.resumableUpload(MediaHttpUploader.java:417)
at com.google.api.client.googleapis.media.MediaHttpUploader.upload(MediaHttpUploader.java:336)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:427)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:352)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:469)
at com.ob.pdfparser.MyClass.uploadFile(MyClass.java:42)
at com.ob.pdfparser.MyClass.main(MyClass.java:91)
Usage Limit:
Queries
requests/day 20 of 1,000,000,000
requests/100seconds/user 1,000
Any Idea??
Below thing worked
Drive drive = new Drive.Builder(this.httpTransport, this.jsonFactory, this.credential).setHttpRequestInitializer(new HttpRequestInitializer() {#Override
public void initialize(HttpRequest httpRequest) throws IOException {
credential.initialize(httpRequest);
httpRequest.setConnectTimeout(300 * 60000); // 300 minutes connect timeout
httpRequest.setReadTimeout(300 * 60000); // 300 minutes read timeout
}
}).setApplicationName("My Application").build();
SocketTimeoutException: Read timed out can be solved by defining a connection timeout or adjusting the timeout barrier so that it's more flexible on network delays. Other than that, you can also check what causes delays in your network because, as far as I know, this kind of error is a network connectivity issue and not a programming issue.
Complete explanation about this kind of error can be found in java.net.SocketTimeoutException – How to Solve SocketTimeoutException and this SO post - Getting java.net.SocketTimeoutException: Connection timed out in android might also help.
Related
Need help with this socket timeout exception that I keep getting intermittently in my application while trying to read messages from AWS SQS queues.
Process:
I keep calling SQS queue receive message request with default wait message time of 20 seconds. (AmazonSQSClient.receiveMessage)
If no message was received in above request another call is made to SQS to again poll the queue (same receive message request).
Issue:
Intermittently in my application, I get the following exception in SQS receiveMessage requests:
com.amazonaws.http.AmazonHttpClient:executeHelper:492 - main - INFO :
Unable to execute HTTP request: Read timed out
java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method) ~[?:1.8.0_121]
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116) ~[?:1.8.0_121]
at java.net.SocketInputStream.read(SocketInputStream.java:171) ~[?:1.8.0_121]
at java.net.SocketInputStream.read(SocketInputStream.java:141) ~[?:1.8.0_121]
at sun.security.ssl.InputRecord.readFully(InputRecord.java:465) ~[?:1.8.0_121]
at sun.security.ssl.InputRecord.read(InputRecord.java:503) ~[?:1.8.0_121]
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:973) ~[?:1.8.0_121]
at sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:930) ~[?:1.8.0_121]
By checking SQS queue from the console I see that during the failures a message is read from the queue (message goes in flight) but is not received at the application end. This leads to message getting locked for the visibility timeout period (1 hour) and hence a delayed processing of the message.
Additional details:
waitTimeSeconds for ReceiveMessageRequest is SQS default of 20 seconds.
Socket timeout for the request is AmazonHttpClient's default 50 seconds. (Exception is also seen after 50 seconds from previous read)
Max number of messages to be received in a request is set to 10 messages.
AWS SDK version used : 1.10.24 (aws-java-sdk-sqs-1.10.24.jar)
Any help around how to resolve or debug this further is greatly appreciated. Let me know if any further details are needed.
I need to communicate with some web service. I've downloaded Apache CXF from https://cxf.apache.org/download.html and generated Java classes with command
wsdl2java.bat -encoding UTF-8 <url>
But when I'm trying to run a long request after about 30 seconds I get exception:
Caused by: javax.xml.ws.soap.SOAPFaultException: java.net.SocketTimeoutException: Read timed out
at org.apache.axis2.jaxws.marshaller.impl.alt.MethodMarshallerUtils.createSystemException(MethodMarshallerUtils.java:1326)
at org.apache.axis2.jaxws.marshaller.impl.alt.MethodMarshallerUtils.demarshalFaultResponse(MethodMarshallerUtils.java:1052)
at org.apache.axis2.jaxws.marshaller.impl.alt.DocLitBareMethodMarshaller.demarshalFaultResponse(DocLitBareMethodMarshaller.java:415)
at org.apache.axis2.jaxws.client.proxy.JAXWSProxyHandler.getFaultResponse(JAXWSProxyHandler.java:580)
at org.apache.axis2.jaxws.client.proxy.JAXWSProxyHandler.createResponse(JAXWSProxyHandler.java:523)
at org.apache.axis2.jaxws.client.proxy.JAXWSProxyHandler.invokeSEIMethod(JAXWSProxyHandler.java:389)
at org.apache.axis2.jaxws.client.proxy.JAXWSProxyHandler.invoke(JAXWSProxyHandler.java:174)
at com.sun.proxy.$Proxy718.loadLoanHistorySynch(Unknown Source)
... 55 more
Caused by: java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:170)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
at java.io.BufferedInputStream.read(BufferedInputStream.java:265)
at org.apache.commons.httpclient.HttpParser.readRawLine(HttpParser.java:88)
at org.apache.commons.httpclient.HttpParser.readLine(HttpParser.java:124)
at org.apache.commons.httpclient.HttpParser.readLine(HttpParser.java:105)
at org.apache.commons.httpclient.HttpConnection.readLine(HttpConnection.java:1116)
at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.readLine(MultiThreadedHttpConnectionManager.java:1413)
at org.apache.commons.httpclient.HttpMethodBase.readStatusLine(HttpMethodBase.java:1973)
at org.apache.commons.httpclient.HttpMethodBase.readResponse(HttpMethodBase.java:1735)
at org.apache.commons.httpclient.HttpMethodBase.execute(HttpMethodBase.java:1098)
at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:398)
at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171)
at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397)
at org.apache.axis2.transport.http.AbstractHTTPSender.executeMethod(AbstractHTTPSender.java:621)
at org.apache.axis2.transport.http.HTTPSender.sendViaPost(HTTPSender.java:193)
at org.apache.axis2.transport.http.HTTPSender.send(HTTPSender.java:75)
at org.apache.axis2.transport.http.CommonsHTTPTransportSender.writeMessageWithCommons(CommonsHTTPTransportSender.java:404)
at org.apache.axis2.transport.http.CommonsHTTPTransportSender.invoke(CommonsHTTPTransportSender.java:231)
at org.apache.axis2.engine.AxisEngine.send(AxisEngine.java:443)
at org.apache.axis2.description.OutInAxisOperationClient.send(OutInAxisOperation.java:406)
at org.apache.axis2.description.OutInAxisOperationClient.executeImpl(OutInAxisOperation.java:229)
at org.apache.axis2.client.OperationClient.execute(OperationClient.java:165)
at org.apache.axis2.jaxws.core.controller.impl.AxisInvocationController.execute(AxisInvocationController.java:578)
at org.apache.axis2.jaxws.core.controller.impl.AxisInvocationController.doInvoke(AxisInvocationController.java:127)
at org.apache.axis2.jaxws.core.controller.impl.InvocationControllerImpl.invoke(InvocationControllerImpl.java:93)
at org.apache.axis2.jaxws.client.proxy.JAXWSProxyHandler.invokeSEIMethod(JAXWSProxyHandler.java:376)
... 67 more
I tried to increase timeout with different parameters but nothing helped:
int timeout = 60 * 1000;
((BindingProvider)client).getRequestContext().put("javax.xml.ws.client.connectionTimeout", timeout);
((BindingProvider)client).getRequestContext().put("javax.xml.ws.client.receiveTimeout", timeout);
((BindingProvider)client).getRequestContext().put("com.sun.xml.internal.ws.request.timeout", timeout);
((BindingProvider)client).getRequestContext().put("com.sun.xml.internal.ws.connect.timeout", timeout);
((BindingProvider)client).getRequestContext().put("com.sun.xml.ws.request.timeout", timeout);
((BindingProvider)client).getRequestContext().put("com.sun.xml.ws.connect.timeout", timeout);
((BindingProvider)client).getRequestContext().put("timeout", timeout);
How can I increase request timeout?
The timeout is in milliseconds so:
60 * 1000 = 60000;
which is basically 60 seconds, I tried like this below and it worked for me.
HashMap<String, java.lang.Object> properties = new HashMap<>();
properties.put(HTTPConstants.HTTP_PROTOCOL_VERSION,
HTTPConstants.HEADER_PROTOCOL_10);
properties.put(HTTPConstants.SO_TIMEOUT, new java.lang.Integer(
360000));
ServiceStub._getServiceClient().getOptions()
.setProperties(properties);
I'm having the same issue as described here but unlike the author the issue is not gone away even if I connect from European or US-based servers.
I'm using pretty straightforward code to query the storage:
Storage storage = new Storage.Builder(
new NetHttpTransport(),
new JacksonFactory(),
new GoogleCredential().setAccessToken(accessToken))
.setApplicationName("FooBar")
.build();
Storage.Objects.Get getObject = storage.objects().get(bucket, fn);
getObject.getMediaHttpDownloader().setDirectDownloadEnabled(true);
getObject.executeMediaAndDownloadTo(outputStream);
wrapped with map-reduce job.
When job is ran it fetches files (~15GB) in parallel using few nodes.
Some runs succeeds but some from time to time up to 50% of download attempts fail with the exception as below:
java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:129)
at com.sun.net.ssl.internal.ssl.InputRecord.readFully(InputRecord.java:293)
at com.sun.net.ssl.internal.ssl.InputRecord.read(InputRecord.java:331)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:798)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:755)
at com.sun.net.ssl.internal.ssl.AppInputStream.read(AppInputStream.java:75)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
at java.io.BufferedInputStream.read1(BufferedInputStream.java:258)
at java.io.BufferedInputStream.read(BufferedInputStream.java:317)
at sun.net.www.MeteredStream.read(MeteredStream.java:116)
at java.io.FilterInputStream.read(FilterInputStream.java:116)
at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2676)
at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:2671)
at com.google.api.client.util.ByteStreams.copy(ByteStreams.java:51)
at com.google.api.client.util.IOUtils.copy(IOUtils.java:94)
at com.google.api.client.util.IOUtils.copy(IOUtils.java:63)
at com.google.api.client.googleapis.media.MediaHttpDownloader.executeCurrentRequest(MediaHttpDownloader.java:261)
at com.google.api.client.googleapis.media.MediaHttpDownloader.download(MediaHttpDownloader.java:209)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeMediaAndDownloadTo(AbstractGoogleClientRequest.java:553)
at com.google.api.services.storage.Storage$Objects$Get.executeMediaAndDownloadTo(Storage.java:4494)
at ***
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:771)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:375)
at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1132)
at org.apache.hadoop.mapred.Child.main(Child.java:249)
I've tried using Amazon (region US_EAST_1) and own servers in Europe, but nothing helped. The download fails randomly.
Is there any way to configure the timeout in google storage client?
The OAuth2 access token is shared across all nodes of the cluster. Is there a quota for simultaneous connections to the google storage?
When I insert documents into Mongo-DB using morphia, it always occur com.mongodb.MongoException$Network: Write operation to server exceptions, maybe interval of one minute
The follow is the stack info:
com.mongodb.MongoException$Network: Write operation to server
at com.mongodb.DBTCPConnector.say(DBTCPConnector.java:153)
at com.mongodb.DBTCPConnector.say(DBTCPConnector.java:115)
at com.mongodb.DBApiLayer$MyCollection.update(DBApiLayer.java:327)
at com.mongodb.DBCollection.update(DBCollection.java:178)
at com.mongodb.DBCollection.save(DBCollection.java:818)
at com.google.code.morphia.DatastoreImpl.save(DatastoreImpl.java:882)
at com.google.code.morphia.DatastoreImpl.save(DatastoreImpl.java:949)
at com.google.code.morphia.DatastoreImpl.save(DatastoreImpl.java:934)
at com.yeahmobi.datasystem.conversion.datarepository.mongodb.MongoTransmappingRepository.insert(MongoTransmappingRepository.java:36)
at com.yeahmobi.datasystem.conversion.datarepository.merge.MergeTransmappingRepository.insert(MergeTransmappingRepository.java:24)
at com.yeahmobi.datasystem.conversion.threads.JumpInserter.saveJumpLog(JumpInserter.java:134)
at com.yeahmobi.datasystem.conversion.threads.JumpInserter.run(JumpInserter.java:163)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:129)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
at java.io.BufferedInputStream.read1(BufferedInputStream.java:258)
at java.io.BufferedInputStream.read(BufferedInputStream.java:317)
at org.bson.io.Bits.readFully(Bits.java:46)
at org.bson.io.Bits.readFully(Bits.java:33)
at org.bson.io.Bits.readFully(Bits.java:28)
at com.mongodb.Response.<init>(Response.java:40)
at com.mongodb.DBPort.go(DBPort.java:142)
at com.mongodb.DBPort.go(DBPort.java:106)
at com.mongodb.DBPort.findOne(DBPort.java:162)
at com.mongodb.DBPort.runCommand(DBPort.java:170)
at com.mongodb.DBTCPConnector._checkWriteError(DBTCPConnector.java:100)
at com.mongodb.DBTCPConnector.say(DBTCPConnector.java:142)
Is there anyone meet the same issues? Any sugguestion is appreciated.
Thanks
This is unusual and should not happen often during normal operation.
Try to debug from networking/OS perspective, check the following:
Is the connectivity between application and Mongo reliable? What's the packet drop rate and latency?
Is there enough network bandwidth between application and Mongo?
Has there been any software/hardware trouble on the application/Mongo server?
Was the server on high load when it happens?
If I set a socket SoTimeout, and read from it. when read time exceed the timeout limit, I'll get an "SocketTimeoutException: Read timed out".
and here is the stack in my case:
java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:150)
at java.net.SocketInputStream.read(SocketInputStream.java:121)
at java.io.FilterInputStream.read(FilterInputStream.java:133)
at org.apache.hadoop.ipc.Client$Connection$PingInputStream.read(Client.java:277)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
at java.io.BufferedInputStream.read(BufferedInputStream.java:254)
at java.io.DataInputStream.readInt(DataInputStream.java:387)
at org.apache.hadoop.ipc.Client$Connection.receiveResponse(Client.java:527)
at org.apache.hadoop.ipc.Client$Connection.run(Client.java:462)
but here I encountered "IOExcetion: Connection timed out", i don't know how it happened.
Stacks:
java.io.IOException: Connection timed out
at sun.nio.ch.FileDispatcher.read0(Native Method)
at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:21)
at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:198)
at sun.nio.ch.IOUtil.read(IOUtil.java:171)
at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:245)
at org.apache.hadoop.net.SocketInputStream$Reader.performIO(SocketInputStream.java:55)
at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:142)
at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:155)
at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:128)
at java.io.FilterInputStream.read(FilterInputStream.java:116)
at org.apache.hadoop.ipc.Client$Connection$PingInputStream.read(Client.java:277)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
at java.io.BufferedInputStream.read(BufferedInputStream.java:237)
at java.io.DataInputStream.readInt(DataInputStream.java:370)
at org.apache.hadoop.ipc.Client$Connection.receiveResponse(Client.java:527)
at org.apache.hadoop.ipc.Client$Connection.run(Client.java:462)
Can someone tell me what's the differences between the two exceptions, Thanks.
A connection timeout means you attempted to connect to the remote IP/port pair and failed to do so: it did not answer at all. Another possible error at that stage would be connection refused, in which this pair is available but rejected your connection attempt. Both of these errors appear on the initial setup of a socket. Note that these errors only occur with TCP, since a TCP connection requires the establishment of a session.
When you have a socket read timeout, it means you are connected, but failed to read data in time. Timeouts on sockets are configurable. You may also get a connection reset error, which means you did connect successfully, but the other end decided that after all you're not worth it :p
Simple answer:
In one case (Connection timed out) your application cannot connect to the server in a timely manner. In the other case (Read timed out) the connection can be established but during read the connection times out.
'Connection timed out' after the connect phase means that something has gone seriously wrong with the connection and it must be closed. 'Read timeout' just means that no data arrived within the specified receive timeout period: it isn't fatal.