I have a weird problem in my OS X (Yosemite) machine: it seems that the network communication is slower than in other Windows machines of the same network, and the timeout seems shorter.
When I try to run our Java application, each transaction that is longer than 15s returns:
java.sql.SQLRecoverableException: IO Error: Operation timed out
at oracle.jdbc.driver.T4CPreparedStatement.executeForDescribe(T4CPreparedStatement.java:876)
at oracle.jdbc.driver.OracleStatement.executeMaybeDescribe(OracleStatement.java:1153)
at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1275)
at oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3576)
at oracle.jdbc.driver.OraclePreparedStatement.executeQuery(OraclePreparedStatement.java:3620)
at oracle.jdbc.driver.OraclePreparedStatementWrapper.executeQuery(OraclePreparedStatementWrapper.java:1491)
at dome.dbComponents.dataModels.rowSet.StatementStatus.executeQuery(StatementStatus.java:71)
at com.dome.dCell.common.services.enterprise.types.Infusr.loadSQLFieldList(Infusr.java:976)
at dome.util.xml.converter.typeSQLManager.ThreadLoadSqlField.run(ThreadLoadSqlField.java:103)
Caused by: java.net.SocketException: Operation timed out
at java.net.SocketInputStream.socketRead0(Native Method)
INFO [28 Aug 2015 14:35:32,427] DomeEnterpriseService.loadAll(true) executed in 19724 ms
The timeout occurs when our Enterprise Service takes more than 15s to load all the enterprises from the database (it's not a bad query, there's simply a lot of information to store in memory).
This is not likely related directly to our software configuration being incompatible with OS X. I tried to run the same application in a Windows 7 VM (with VirtualBox) on the same iMac, and the problem remains.
I checked the connection with the database machine (a Linux server), and everything seems fine.
I've also tried different configurations in /etc/sysctl.conf (net.inet.tcp.msl) but it doesn't seem to make a big difference. Also, I don't find anything like the net.ipv4.tcp_syn_retries configuration of Linux, and maybe I'm googling the wrong keywords, but I don't find much about tweaking the timeout of OS X.
I don't recall having this problem before on this machine. I updated the JDK 1.7 to the last version (1.7.0_79), all the OS updates, nothing improves.
We tried with different machines (Linux & Windows, it's the only Mac), and no one is able to reproduce this.
Any ideas?
Related
I have a remote Linux server and I want to connect to an Oracle database which is in another server, using ojdbc7 lib
When I try to connect directly to the database from my Windows PC, using the same client and ojdbc7 lib, I have reasonable connection time.
Now, when I want to connect through my linux server, I get extreme slowness, just in the connection time. . Once connected, the execution is OK.
I have read about adding -Djava.security.egd=file:/dev/urandom like in this post, but nothing happened.
What could I do to fix this delay in setting up a connection from linux?
Close, but no cigar: it's "file:///dev/urandom", or one of the variations, see eg. https://anirban-m.blogspot.com/2014/03/jdbc-connection-reset-error-java.html
I noticed you are using version 12.1.0.1.
There was an Oracle bug where JDBC connections could take excessive times because the data being sent required the listener to perform a DNS lookup for each connection and that could apparently be very slow for some reason.
The bug was fixed in 12.2 and there is a back-ported fix (patch) for 12.1.0.2.
In the meantime, try getting your Linux admin to go through the process of tuning DNS lookups on that server. E.g., tune /etc/resolv.conf or enable the name service cache daemon. I'm not really expert in Linux administration so I can't help you. But based on the problem and the version you are using, that's where I'd look.
“Communications link failure” error on MySQL using JDBC after the program has run for 38 hrs. Another program on the same machine hitting the same server (also java) continues to run. Both are using the exact same java code to access the server, as do the java programs running on 4 other machines at the same time. I get this error randomly on different machines, after a program has been running successfully (reads/inserts/updates to the database) for quite a while. The error includes the first line above the stack trace:
The last packet successfully received from the server was n milliseconds ago. where n has varied between 0 and 25.
I might suspect that the server has run out connections to hand out, but I get a totally different error when that happens (I encountered it and updated my server to increase max connections, which eliminated that error). I get the error whether I use c3p0 to pool connections, or not.
Please read before flagging as a duplicate. The connect string and database are fine, as it worked for 38 hours (and continues to work in other processes on the same machine and different machines accessing the same MySQL server) using exactly the same java code. This is a transient/random error. If I simply restart the program, it continues successfully. I would say it's OS related, except that it occurs on both Windows and Mac OS X. Which leads me to believe it's likely something going on in the java VM.
Considering the point that you mention, that it is running fine until 38 hours and then you get the following issue - It only leads to one logical point:
If your connection is IDLE for a considerably longer duration. This idle connection would evaluate connection.isClosed(); to TRUE, and when we try to execute the above statement it will then fire the following exception.
com.mysql.jdbc.exceptions.jdbc4.CommunicationsException
How to remediate this issue?
Switch to DB Connection Pooling. Hope this helps!
Problem Description
Hi everyone,
I was having problems remote debugging to a new server today and it initially appeared as if Eclipse was taking a long time to connect to a remote JVM. However, after some investigation I realized jdb was having a similar problem. Some digging turned up this...
Connecting to the remote JVM is not a problem. Both debuggers establish the socket connection properly within a few seconds. jdb even processes commands. However, after a remote debugger has connected to it successfully, the Sun Java 1.7.0_60-b19 JVM appears to be hanging (or transmitting extremely slowly) when sending its thread debugging information (JVMTI/JDWP) across the network via TCPIP.
Listing the remote JVM's threads appears to be the problem. JDB's threads command either hangs or executes extremely slowly. The load on the remote JVM is reasonable, and there aren't any breakpoints set. There are many threads executing on this JVM concurrently (~2005 threads), and there may be some WAN latency, but there still needs to be a way to successfully use a remote debugger with it!
Observation. I'm guessing this is related to inefficient transmission of thread information via TCPIP by the JVM's Java Debug Wire Protocol (JDWP) implementation, as the machine is on the other side of the planet on a WAN. HOWEVER -- Windows Remote Desktop Connection to the same machine is acceptably fast and performant. Given that, it seems that having to wait 45 minutes for Eclipse or JDB to list thread information on a JVM is unacceptable and likely a bug (or a very poorly implemented feature).
Possibly related to?
http://bugs.java.com/bugdatabase/view_bug.do?bug_id=6401245 - This
looks like it was fixed and shouldn't be the issue anymore. Plus the JVM is running on Windows Datacenter not Linux.
http://www.eclipsecon.org/2013/comment/reply/1153.html - Possible
solution but requires the SAP JVM?
https://github.com/vpotapev/jbreakpoint - Open source jdb interface
but doesn't fix issue with Eclipse.
Question. Does anyone have any ideas how to make the thread data transmission more efficient so the JVM can be remote debugged effectively? Is this a bug in the JVM's Java Debugging Wire Protocol (JDWP)?
Problem Details
Java Version:
C:\Users\Administrator>"C:\Program Files\Java\jdk1.7.0_60\bin\java" -version
java version "1.7.0_60"
Java(TM) SE Runtime Environment (build 1.7.0_60-b19)
Java HotSpot(TM) 64-Bit Server VM (build 24.60-b09, mixed mode)
Application Server. Happens with Weblogic 10g. Also happens with IBM WebSphere 8.5
Eclipse. Eclipse remote debugging hangs indefinitely when trying to populate the Debug view with thread information.
JDB - Remote. Java's jdb remote debugger hangs for upwards of 10 minutes when listing the threads with the thread command. Then it lists them VERY slowly (1 thread per second, would take 33 minutes to list them all).
"C:\Program Files\Java\jdk1.7.0_60\bin\jdb.exe -connect com.sun.jdi.SocketAttach:hostname=xxx.yyy.com,port=7777
Set uncaught java.lang.Throwable
Set deferred uncaught java.lang.Throwable
Initializing jdb ...
> threads
Group system: <<jdb hangs here trying to get thread information>>
JDB - Local. Java's jdb remote debugger executes the threads command in 3 seconds when run on the JVM's local machine.
"C:\Program Files\Java\jdk1.7.0_60\bin\jdb.exe -connect com.sun.jdi.SocketAttach:hostname=xxx.yyy.com,port=7777
Set uncaught java.lang.Throwable
Set deferred uncaught java.lang.Throwable
Initializing jdb ...
> threads
Group system:
> threads
Group system:
(java.lang.ref.Reference$ReferenceHandler)0x7484 Reference Handler
(java.lang.ref.Finalizer$FinalizerThread)0x7485 Finalizer
(java.lang.Thread)0x7486 ...
GC Daemon
(java.lang.Thread)0x748b RMI RenewClean-
...
>
After some investigation and packet sniffing, it appears that having a large number of threads in the remote process causes debugging packet congestion as well as a feedback loop of processing delays between the remote process debug agent and the remote debugger. Basically, the JVM debugging agent isn't engineered for high latency connections or debugging a large number of threads remotely. It also doesn't make an attempt to compress thread/stack/variable update information to lower transfer time to the debugger.
Our solution was eventually to add a thread debug mode to our product that reduced the various server thread pool sizes, then debugging was suddenly acceptably responsive again across overseas networks.
A future suggestion to the Java community would be to add a thread filter feature to both Eclipse and the Java Debug Agent so that only information for threads of interest is transmitted to the debugger after each code step is signalled.
I need to create 1000 server sockets using Java. Somewhere between creating 600 and 700 sockets I'm getting this error:
05.04.2012 10:23:57 java.net.SocketException: No buffer space available (maximum connections reached?): listen at sun.nio.ch.ServerSocketChannelImpl.listen(Native Method) at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:127) at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:59) at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:52) at channelserver.NIOAppServer.initSelector(NIOAppServer.java:40) at channelserver.NIOAppServer.(NIOAppServer.java:27) at channelserver.NIOServer.main(NIOServer.java:433) at channelserver.NIOServer.main(NIOServer.java:438)
After that my computer reboots.
I've tried to edit the registry setting MaxUserPort = 65534 and MaxFreeTcbs = 2000, but it didn't help.
What else can I do?
On http://serverfault.com someone told me that it may be related to JDBC, but I connect to DB only once. I use SQLite.
Any suggestions?
on Windows XP have a limit on half-open connection to broke this rule you need to patch your tcpip.sys driver (but it's illigable) or using modern OS like Windows Vista/7 they haven't those limits.
Just in case someone would face this problem too.
Finally it seemed to me that it's just WinXP's problem. I've tried the same code with more RAM - results were the same. But when I've tried it on Win7, everything worked fine even with 1200 sockets.
So, WinXP is just not suitable for such tasks.
I actual have a big problem! I open a connection to a oracle database with "DriverManager.getConnection(url, properties)".
On UNIX machines (currently on a VM), the problem occurs that 99% of the time it needs minutes till the function return a connection. I increased the connection timout of oracle so that I don't get a SQLException, but it needs up to 3 Minutes to get a connection. On my windows machine the connection is returned in under 1 second.
telnet to server + port works, ping is sucessfully, traceroute looks good. I also tried from several VMs or on different databases on different physical machines.
I run the actual JDBC Driver "ojdbc6-11.2.0.2.0.jar".
Does anyone have a good idea?
After a long time we figured out the problem. The Oracle JDBC driver blocked at the point where a unique id was read. After setting the VM Argument
-Djava.security.egd=file:/dev/urandom
we could gurantee to always receive a uniqueid in an adequate time. The default /dev/random unfortunately just generates a uniqueid if the machine has enough entropy, which is often missing on virtual machines.
Maybe this helps some of you folks one day.
It's a little bit strange but it could be a REVERSE DNS problem.
If you Oracle server is on unix, try the following:
$ host IP_ADDRESS_OF_WIN_MACHINE
$ host IP_ADDRESS_OF_LINUX_MACHINE
See if there is something different on the two name resolutions. If there is, then it might be the case that trying to do a reverse DNS lookup on the LINUX IP is taking too long.
It's happened to me.