I use Selenium with HtmlUnitDriver with javascript enabled and I get Out Of Memory errors (I use Java). I just browse the same page. I am only using a single GET command. Which is the solution to overcome the situation?
I've had a similar issue. It ended up being an issue with auto-loading of frames... a feature that can't be disabled.
Take a look at this: Extremely simple code not working in HtmlUnit
It might be of help.
Update
Current version of HtmlUnit is 2.10. I started using HtmlUnit from version 2.8 and each new version ended up eating more memory. I got to a point in which fetching 5 pages with javascript enabled resulted in a process of 2GB.
There are many ways to improve this situation from a javascript point of view. However, when you can't modify the javascript (eg: if you are crawling a site) your hands are tied. Disabling javascript is, of course, the best way to go. However, this might result in fetched pages being different from the expected ones.
I did manage to overcome this situation, though. After many tests, I noticed that it might not be an issue with HtmlUnit (which I thought was the guilty one from the beginning). It seemed to be the JVM. Changing from Sun's JVM to OpenJDK did the trick and now the process instead of eating 2GB of memory only requires 200MB. I'm adding version information.
Sun's (Oracle) 32-bit JVM:
$java -version
java version "1.6.0.26"
Java(TM) SE Runtime Environment (build 1.6.0_26-b03)
Java HotSpot(TM) Server VM (build 20.1-b02, mixed mode)
OpenJDK 32-bit JVM:
$java -version
java version "1.6.0_18"
OpenJDK Runtime Environment (IcedTea6 1.8.13) (6b18-1.8.13-0+squeeze2)
OpenJDK Server VM (build 14.0-b16, mixed mode)
Operative system:
$ uname -a
Linux vostro1015 2.6.32-5-686-bigmem #1 SMP Sun May 6 04:39:05 UTC 2012 i686 GNU/Linux
Please, share your experience with this.
Give more memory to the JVM by adding this to the java command line that starts the JVM in which Selenium is running:
-Xmx512m
This example give a maximum of 512 Mb to the JVM.
It depends on where you're running Selenium from. If maven, you can add it to the MAVEN_OPTS environment variable, if Eclipse, you'll need to edit the run configuration for the test class, etc.
Related to HtmlUnit:
Do not forget to call webClient.closeAllWindows();. I always put it in a finally-block around the area I use the webclient. This way it is sure that all javascript is stopped and all resources are released.
Aslo useful is setting for the webClient:
webClient.setJavaScriptTimeout(JAVASCRIPT_TIMOUT);
webClient.setTimeout(WEB_TIMEOUT);
webClient.setCssEnabled(false); // for most pages you do not need css to be enabled
webClient.setThrowExceptionOnScriptError(false); // I never want Exceptions because of javascript
JAVASCRIPT_TIMOUT should be not too high long running javascript may be a reason for memory problems.
WEB_TIMEOUT think about how long you want to wait maximal.
Related
I work on Rstudio running on a dedicated linux server. I get an error initialising h2o:
> h2o.init()
H2O is not running yet, starting it now...
<simpleError in system2(command, "-version", stdout = TRUE, stderr = TRUE): error in running command>
Error in value[[3L]](cond) :
You have a 32-bit version of Java. H2O works best with 64-bit Java.
Please download the latest Java SE JDK 8 from the following URL:
http://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html
I checked the java config and got:
system("java -version")
openjdk version "1.8.0_77"
OpenJDK Runtime Environment (build 1.8.0_77-b03)
OpenJDK 64-Bit Server VM (build 25.77-b03, mixed mode)
Apparently I have the good version of java. I have found people with similar problems on Windows. They were able to solve the problem by finding and indicating the good java path. However in the environnement I am working in, i am not sure how to do that.
Any idea on how to solve the problem ? to bypass the error ?
Check if the JAVA_HOME environment variable is set. It may be pointing to the wrong spot, tricking H2O to find the wrong one.
You want JAVA_HOME/bin/java to be a good 64-bit java.
(The other thing you can optionally do is, when you do find a 32-bit java, uninstall it. There is no real reason to have it unintentionally anymore these days; memory sizes are much bigger than 15 years ago.)
I am using a computer withe following specification:
OS-Windows7 professional
Installed Memory(RAM) :8Gb
System Type: 64 bit Operating System
JVM: Java version 8 update 91(jre1.8.0_91) 64 bit version
java version "1.8.0_91"
Java(TM) SE Runtime Environment (build 1.8.0_91-b15)
Java HotSpot(TM) 64-Bit Server VM (build 25.91-b15, mixed mode)
For using one desktop application I need large JVM memory allocation. With the above specification I can set the JVM at maximum 1.5gb(1536mb) using -Xmx command.
If I am increasing the value above 1.5 GB getting the following error:
"The JVM could not be started.The main method may have thrown out an exception."
Please let me know how can I allocate more memory to th e JVM.
The 1.5 GB limit means you are most likely using the Windows XP/32-bit version. When this question has come up before the OP has been sure they were using the 64-bit version but on investigation found they were not.
I suggest making sure there is only one version of Java installed, the 64-bit version you want to use.
In my opinion the best way of detecting your problem is to use JProfiler, it can detects if you're having a problem inside your vm and can monitor the status, it's simple to use and give a lot of debug. I saw at some articles that sometimes increasing too much the memory cause an error like this but I don't have a clue, try to look this too, hope it helps.
Does anyone experienced and know how to fix this issue:
Android Studio sometimes (randomly) does not show any content in dialogs (find, commit, compare dialogs). It takes a part of screen which was visible in that area where dialogs appears. And content of dialog is not refreshed and not visible at all.
All functionality of that dialog works, just I can't see anything.
Here's how it looks like:
Anyone knows how to fix this?
I'm working on Ununtu 15.04 with Android Studio 1.2.1.1, which is based on IntelliJ IDEA.
Java version is 1.8:
vilius#vilnius:~$ java -version
Picked up JAVA_TOOL_OPTIONS: -javaagent:/usr/share/java/jayatanaag.jar
java version "1.8.0_45"
Java(TM) SE Runtime Environment (build 1.8.0_45-b14)
Java HotSpot(TM) 64-Bit Server VM (build 25.45-b02, mixed mode)
vilius#vilnius:~$
STUDIO_JDK=/usr/lib/jvm/jdk1.8.0_60 add to /etc/environment
create new Launcher over old: Android Studio - Tools -CreateDesktopEntry
So apparently it's because I've done "Lock to Launcher" to keep Android Studio visible in Launcher all the time. It creates shortcut which then uses OpenJDK instead of Oracle Java.
That's when I'm getting this message:
OpenJDK shows intermittent performance and UI issues. We recommend
using the Oracle JRE/JDK.
In android-studio/bin/studio.sh file it's written, that $STUDIO_JDK environment variable is used to determine which java to use when launching Android Studio IDE.
To find which java is running for your Android Studio, you can check System Monitor:
Or Android Studio-> about also tells you which JVM is used:
The answer
Install Oracle Java. Create $STUDIO_JDK environment variable which directs to Oracle Java. Don't use Lock to Launcher functionality.
It's Ubuntu's bug, and solution is to just kill compiz. You can do it from command line or System monitor.
All running applications will be brought to 1st desktop, and you'll be able to continue your work.
Tried both Oracle and OpenJDK Java by setting STUDIO_JDK environment variable:
#export STUDIO_JDK=/usr/lib/jvm/java-8-oracle
export STUDIO_JDK=/usr/lib/jvm/java-7-openjdk-amd64
Still experiencing this issue sometimes. :/
Changing Settings->Appearance & Behavior->Appearance->UI Options->Theme does not help either.
Apparently, it's Ubuntu's bug:
https://bugs.launchpad.net/ubuntu/+source/compiz/+bug/1072206
Bug Description
[ Impact ]
With nVidia hardware from time to time window content is rendered
black or transparent. The problem manifests more frequently as
session uptime increases.
Apparently the problem was caused by textures being cached and
not freed, resulting in memory exhaustion conditions. It turns out
the texture caching is unnecessary anyway.
[ Test Case ]
Verification requires an nVidia GPU with proprietary drivers.
The problem can not be reproduced at will, only after some
time (depending on resident GPU memory and usage) and even
then only randomly and unreliably.
[ Regression Potential ]
It is unlikely this change has the potential to introduce
new regressions.
[ Other Info ]
Fix was cherry-picked from the Ubuntu "Vivid Vervet" dev release
where it has been in use for some weeks without problem.
Sun's JVM comes in two flavors: -client and -server, where the Server VM is supposed to be optimized for long running processes, and is recommended for server applications.
When I run java with no parameters, it displays the usage options, which includes the following text:
The default VM is server,
because you are running on a server-class machine.
Having seen this, I didn't bother to add the -server to the process startup command.
However, on a recent JVM crash log, I noticed the following line near the end of the file:
vm_info: Java HotSpot(TM) Client VM (14.0-b16) for linux-x86 JRE (1.6.0_14-b08), built on May 21 2009 02:01:47 by "java_re" with gcc 3.2.1-7a (J2SE release)
It seems to me that Java is using the Client VM, despite what it says in the help message. I'm going to add the -server option to my startup command, but now I'm suspicious. So my question is: is there a way to make sure that the VM I'm running in is really the Server VM, without resorting to forcing a JVM crash?
The OS is ubuntu 8.04, but I'm using JDK 1.6.0_14 which I downloaded from Sun's website.
You can do
System.out.println(System.getProperty("java.vm.name"));
Which on my machine returns either:
Java HotSpot(TM) Client VM
or
Java HotSpot(TM) 64-Bit Server VM
Of course you shouldn't do anything critical based on this value, as it will probably change in the future, and will be completely different on another JVM.
I had a very similar question which I asked on ServerFault. I would say, if you care which version is run, always use -client or -server.
Well, if you explicitly start with the -server command line prompt, you are running in server mode. You can check that with this:
ManagementFactory.getRuntimeMXBean().getInputArguments();
You can look at RuntimeMXBean which may open up more information, but that would have to be tested on the specific JVM you are running.
Without writing any single line of code, if you use JConsole to connect to your JVM, under 'VM Summary' tab, it should say exactly which (server or client) Virtual Machine is being monitored.
For example,
Virtual Machine: OpenJDK Server VM version 1.6.0-b09
To remotely monitor your JVM using JConsole, simply enable the JMX (Java Management Extensions) agent by starting JVM with the following System Properties
> java.rmi.server.hostname=[your server IP/Name] // without this, on linux, jconsole will fail to connect to the remote server where JVM is running
> com.sun.management.jmxremote.authenticate=false
> com.sun.management.jmxremote.ssl=false
> com.sun.management.jmxremote.port=[portNum]
I am trying to get remote debugging working with Java on Solaris OS. Following is what I have tried-
I have a Java class called TestP which has the main method.
When I try
java -classpath . TestP
the program works fine. But when I try adding the debug parameters to the JVM-
java -Xdebug -Xrunjdwp:transport=dt_socket,address=8001,server=y,suspend=n -classpath . TestP
It fails giving this error-
UTF ERROR ["../../../src/solaris/npt/utf_md.c":49]: Failed to complete iconv_open() setup
Can anyone please help me on figuring out why this error is coming up?? The above works fine on my Linux box.
Java version on Solaris:
Java(TM) SE Runtime Environment (build 1.6.0_15-b03)
Java HotSpot(TM) Client VM (build 14.1-b02, mixed mode)
Java version on Linux:
java version "1.6.0_0"
OpenJDK Runtime Environment (IcedTea6 1.5) (fedora-20.b16.fc10-i386)
OpenJDK Server VM (build 14.0-b15, mixed mode)
Hmm... This seems to be a known Solaris (not specific to Solaris 10) issue with Java 6, not a Java issue (see this thread).
Someone has successfully applied a workaround (see this blog post) from a Sun guy, Jeff Moguillansky, but I wouldn't recommend it and rather consider searching sunsolve for a patch as indicated on Sun's forums.
Look at this one: http://sunsolve.sun.com/search/document.do?assetkey=1-1-6586755-1 (you'll need a Sun Online Account with a valid Support Contract or Software Subscription).
Using truss I found out that the process was looking for /usr/lib/iconv/geniconvtbl/binarytables/UTF-8%646.bt and 646%UTF-8.bt, so I just copied ISO8859-1%ISO646.bt to UTF-8%646.bt and ISO646%ISO8859-1.bt to 646%UTF-8.bt (yes, it is "646", not "ISO646"!)
This is of course a very ugly workaround and I have no idea if it has any negative effects on the JVM, but at least it starts the JVM without aborting. (I did this on OpenSolaris 2009.06, btw)
If Pascal Thivent is right, then you may want to try running OpenSolaris (either on a blank machine or in a vm) and see if the problem is also there. If not, then consider using that version for now if possible.