Just moved my app to a new CentOS server. After fiddling for a long time, I can't play to compile or get any errors. The shell output will show me the last file compiling and then hang. I'm running the Scala 0.9.1 module.
I did a strace on the process, this is all I got. Not sure if it helps.
futex(0x410489d0, FUTEX_WAIT, 5403, NULL
Any ideas to the cause? Or where I can begin troubleshooting?
Thanks.
Googling on FUTEX_WAIT I found this mailing list thread for OpenJDK describing your problem. They advised the user to go to the distro-specific forum and ask the same question, which led me to this forum thread which is specific for Fedora 10. This seems to be a "common" problem related to some Linux distros, some JVMs and gvfs-fuse-daemon or some other process keeping locks.
No idea towards a specific solution, but I hope one of the following helps:
Read this question which deals with the same issue, only related to Citrix
This problem seems to have been around a while, so make sure you update your JVM to the latest version
Update your distro to the latest version if at all possible (especially any kernel updates, which according to this Sun bug fixed the issue)
Contact the CentOS community as this issue seems to be related to futex and Linux; they should be able to help you further
I didn't immediately come across a clear solution for this issue, so if you find one, please update this question so we can save it for future generations to come ;-)
As it turns out tmbrggmn was right and this is indeed related to the particular kernel on the machine I was using.
The particular kernel was an open source Zen kernel running CentOS and was using software virtualization. I migrated to a Citrix Kernel and now Play! compiles just fine.
A uname -a on the buggy kernel was:
Linux examplehost.com 2.6.18-274.17.1.el5xen #1 SMP Tue Jan 10 18:06:37 EST 2012 x86_64 x86_64 x86_64 GNU/Linux
Related
So I am getting the most random JVM crashes while my Web Application is in production. My environment is Linux Redhat enterprise edition 7, Jboss AS 7, Java 8, JSF 2.2
This crash seems to happen when my user persists some data to the database. Here is the JVM log:
Here is a link to my full log it exceeded the maximum character amount to post on this site.
https://drive.google.com/file/d/0B_JrHFckYWh5d2ktRGphUTJlRFk/view?usp=sharing
I have no idea what is relevant, this has been happening a lot recently I have a hunch it's because of too little memory allocated somewhere, I appreciate the help,
Sam
Crash seems to be Kernel related issue, Please find the complete blog details here https://blogs.oracle.com/poonam/entry/analysis_of_strange_hotspot_crashes
Form your logs it is not clear which kernel version you are using, logs are missing System information. Please upgrade your kernel version. FInd the similar issue here - https://bugs.openjdk.java.net/browse/JDK-8161645
Crash occurs somewhere in code cache while trying to run this:
J 12713 C2 com.sun.crypto.provider.GCTR.update([BII[BI)I (158 bytes).
I found similar bug in openjdk's jira and it says that -XX:-UseAESIntrinsics can help you.
Also you can exclude this method from JIT compilation -XX:CompileCommand=exclude,com/sun/crypto/provider/GCTR.update
I'm experiencing this weird problem that the JVM hangs forever very frequently.
I first observed the problem when my Java IDEs frequently hang the entire system GUI. IntelliJ IDEA hangs on indexing almost every single time upon start. Sometimes it proceeds to resolving dependency but always hangs in the end. When this happens, I can type in gnome-terminal, but the commands can't seem to be executed. I can't launch new applications with Alt-F2 or anything alike.
I had to switch to a text console and "killall -9 java" to kill the IDEA process and get control back. "kill -3 java" won't work. The log file contains nothing related, the thread dump is empty. Once the IDE hung, jstack cannot be attached to the process. "jstack -l pid" also hangs. "jstack -F pid" can't attach to the process. Visualvm hangs as well.
The CPU usage by the Java process is 0% and there is no I/O going on.
I've observed the same behavior when using Eclipse. Sometimes it hangs on start up, sometimes upon saving and sometimes upon running a Java application.
Maven / sbt builds executed within text-only ttys cause the same kind of hang, so I guess it's not a window manager / desktop environment / display driver problem.
I highly suspect it's a file system or I/O issue but I have no clue how to debug that. I've tried fsck with no luck, and my system works perfectly fine when not running java programs.
Things I've ruled out:
Permission issues: running IntelliJ with sudo doesn't help, hangs 100% of the time.
Display driver: I've tried both the Nvidia proprietary driver and nouveau, the open source one. Doesn't help.
Window manager / desktop environment: I use Cinnamon, but I've tried running IntelliJ under Unity. Doesn't help.
Java version: I've tried both Oracle Java 7 and Oracle Java 8. I'll probably try OpenJDK but I doubt it would work.
IntelliJ version: I've tried IntelliJ 13 through 14.1. All exhibited the same behavior.
Limited memory: I have 16G RAM with 16G swap space, so memory should not be a limiting factor.
Kernel log doesn't look suspicious. I can't get any kind of log remotely indicating what went wrong.
Any idea?
UPDATE (2015/04/29): The problem seems to have fixed itself after I accidentally kicked the power cable and cold restarted the computer... Still a mystery but IntelliJ is usable as of now.
Some things to check
- The Java IDEs run best with a lot of ram. I usually ask for at least
8G of memory for my dev workstation.
- Make sure you have a stable version of everything, look for known working versions/configurations on Ubuntu
- You have to manually allocate memory in IntelliJ IDEA versions < 14. For example: How to increase IDE memory limit in IntelliJ IDEA on Mac?
- Besides system logs, run tools like top and see what's happening in terms for cpu and ram when running the IDE
I had similar problems a while ago but with Eclipse. The problem was that there was no swap place at all ;) - obviously it should not be a problem with 16GB of RAM.
Could You post JVM arguments for Intellij? And also I have an idea to create another Intellij installation (eg. go back to 14 version) and see if there is similar problem (also compare JVMs settings between these two).
Edit
Ok so try:
use different JRE/JDK. If the problem disappears it will tell us more.
You are on linux so it makes it easy to monitor several things - you said that there is no CPU utilization or hard I/O. But how do you know that? Maybe it will be informative to have some statistics gathered - eg.jstat for JVM itself or for system information (You think that is I/O problem) try:
iostat -hm -p sda 1
Which will print I/O statistics for sda (if you have different discs change device parameter) in 1sec loops (can be also changed). Start this with system and dump output to the file - maybe there is some kind of 'disaster' happening before JVM hangs. Note: iostat sometimes is not available on system itself (on my Linux Mint is not) - install then package sysstat and the command will be available.
Seem to have fixed after a cold restart from an accidental power loss.. Weirdest problem I have ever seen.
At my wits end here. We've recently grabbed a Mac Pro to test out as a development environment, moving from Ubuntu - we were really hoping to see some serious decrease of our build times, but in fact the opposite is true. I'm wondering if we're missing something config related with Mavericks, we've already disabled spotlight on the appropriate directories.. messed around with ANT options giving up to 8G of memory for the builds.
Is HFS+ that much slower that EXT3? The build is writing 55k+ files to an exploded ear.
I've run disk tests on the mac, and we're seeing close to 1GB/s read time & 750MB/s write time so it doesn't seem to be hardware related. The linux build completes in just under 8 minutes, whereas the Mac Pro is looking at 9.5 minutes.
I'd like to avoid any kind of religious style OS discussions, I also know we could try installing Linux on the mac and see what happens, but at that point I'm not sure we're seeing a reason to spend $1600 more than our current high end non-Apple build. Looking more for any 'obvious' reasons we might be seeing this issue.
Thanks.
Answering my own question - unfortunately we sent the machine back, but later when moving to macbook pros, we discovered that we were still using a pretty old version of ANT (1.7.x), and the weblogic deps also had ant 1.7.x in the class path. We moved to 1.9.x & saw a big increase in speed on the macbook pros, so I suspect that a newer version of any may have seen some speed increase on the Mac Pro as well.
You've run some disk tests on the mac - how about on the Linux box? It's possible it's faster than the mac in that regard. Something else to check out - is the mac using filevault? I've noticed that will really eat into compile times vs raw disk, but it doesn't usually show up on a disk read / write test that's using a single large block of data.
I have a problem with Eclipse for some time. When I move to Windows 7 x64 on my notebook, Eclipse starts getting "Freeze", for example, when using Content Assist (Code Helper), or using any other option in Eclipse. I am using quite bunch of plugins, so, I tried to delete them all, and check clean IDE. But this didn't help. I downloaded fresh Eclipse Helios for Windows x64, didn't help. I even formated the disk, reinstall Windows, install only JDK and Eclipse but it always occur. What can I do ?
Edit:
Memory: I did not change memory, and IDE freeze, change memory to 512,1024,2048 MB, keeps freezing. (via vm parameters).
Anti-Virus: I am using ESET Smart Security, but with our without it, Eclipse keeps freezing.
After much frustration, I disabled AVG and it worked fine.
Several leads.
Check whether this freeze the freeze is linked to a huge consumption of CPU or disk usage. Unlikely.
If not then this is probably a network issue. Then disable the firewall for a while and try again. Eclipse now reports your plugin usage at the beginning of a session and it might be busy looking for a connection.
Close all editors from previous session. In the past, eclipse tried to access xml DTD with from the network instead of the local catalog and that would fail if you were offline of course.
Finally, let me tell you that if this is for running eclipse you've selected the worst OS. OSX and Linux are much better options. I used to do so as well. But for the last two years, I've run Windows only inside VirtualBox when I couldn't avoid it (TOAD, Macromedia Fireworks) and I wished I had migrated before.
The crucial point is how much memory you have for Eclipse and if you have any anti-virus software installed that needs to preparse all the class files Eclipse wants to look in.
Does it settle after some usage?
I develop using Java 5 and 6 on Intellij IDEA 7 and 9M1. From time to time (during run/compile) my Mac (10.5.7) will die with a Bluescreen of death.
Has anybody else had this issue and somehow managed to solve it? If so, how? Any suggestions other than running the IDE in a VM?
I suggest to file a bug against Apple's Java VM. This really should not happen.
I've heard people complaining about the latest MacOS Java updates in combination with IDEA. Maybe check out this thread
It could be specific to your hardware and/or software environment. I'd suggest performing a hardware test (faulty memory is the most likely cause). Java is a stress to the system, I've seen such situations before (other applications work fine, but Java is causing system crashes, in most cases such problems were related to memory, replacing the modules fixed them). For testing the memory you can use the Memtest OS X.
Another thing to try is Java for Mac OS X 10.5 Update 5 Developer Preview which is available via Apple Developer Connection.
Have you had a look at the system console (/Applications/Utilities/Console)? Java or the OS may have recorded some dying gasp there before the BSOD happened.