For the below given code , I see lot of GC activity. As per my understanding this is a suitable scenario for EA. Why EA is not effective. DummyObject has nothing allocated inside it. JVM options used : -server , -verbosegc.
static void anayzeEA()
{
for(int i = 0 ; i < 100000000; i++) {
DummyObject obj = new DummyObject();
if(obj.hashCode() == 97787) { //to prevent the obj being optimized
System.out.println(obj.hashCode());
}
}
}
See related Q&A here which suggests you can download a debug JDK and use command line options:
-XX:+UnlockDiagnosticVMOptions -XX:+PrintEscapeAnalysis -XX:+PrintEliminateAllocations
to print out the escape analysis events as they happen.
Some observations
It seems like the obj.hashCode() is a native call and the object may escape . Changing obj.hashCode() to obj.getMyCode() (a method which returns the System.currentTimeMillis()% staticObjCount) made it work . No GC activity observed. However following method never got escape analysis in effect with all the suggestions mentioned
here
public static long test1()
{
long r = 0;
byte[] arr = new byte[(int)System.currentTimeMillis() % 1024];
if(arr.length == 998 ) {
++r;
}
return r;
}
JVM options used
-server
-verbosegc
-XX:CompileThreshold=1
Test1 is called over a number of times . Same old story. Java allocates memory in heap, GC comes and makes everything slow. was too excited about this feature.
Java API says:
As much as is reasonably practical, the hashCode method defined by class Object does return distinct integers for distinct objects. (This is typically implemented by converting the internal address of the object into an integer, but this implementation technique is not required by the JavaTM programming language.)
So you are generating objects, producing non-predictable hashCodes for each and compare them to some value. Additional it's a native method, so the JIT doesn't know what's happening inside.
Escape Analysis may be good, but there's currently no support for crystal balls. ;-)
Try overriding it with your own method, returning 12345.
Related
I found this example code for Joshua Bloch's book, Effective Java. It's meant to demonstrate why you should avoid unnecessarily creating objects:
public class Sum {
private static long sum() {
Long sum = 0L;
for (long i = 0; i <= Integer.MAX_VALUE; i++)
sum += i;
return sum;
}
public static void main(String[] args) {
int numSets = Integer.parseInt(args[0]);
long x = 0;
for (int i = 0; i < numSets; i++) {
long start = System.nanoTime();
x += sum();
long end = System.nanoTime();
System.out.println((end - start) / 1_000_000. + " ms.");
}
// Prevents VM from optimizing away everything.
if (x == 42)
System.out.println();
}
}
What do the last two lines of the main method accomplish here?
That final comparison is the only usage of that variable. Without it, who would care about the value in that variable?
Nobody!
So, the compiler might assume: that value is never used, and the code that writes to it has node side effects. Things that don't get used, why waste time writing to them.
Thus: that final first "read" usages prevents over eager compilerd from "optimizing out" the call to the method you intend to measure!
The last two lines force the compiler to run the whole loop, to find the value of x. Otherwise it might detect that x is not being used at all and ignore the loop, given that inside of it no "real" work is being made. Even though sum() is called repeatedly, the result of accumulating its returned value would be discarded in the end if we do nothing with x.
Of course, this assumes that the println() statement inside the loop can be safely ignored, I'm unsure if the compiler can make such a decision. That would be one aggressive compiler!
The two last lines will make sure that the JVM doesn't remove calls that it could potentially consider no-ops otherwise, one option is to use the result - so you might sum all the return values and then display the sum at the end.
Another way to prevent vm from optimizing is to disable JIT:
-Djava.compiler=NONE
JIT:
When JVM compiles the class file, it doesn’t complete the full class
file; it compiles only a part of it on need basis. This avoids heavy
parsing of complete source code. This type of compilation is termed as
JIT or Just-In-Time compilation. JVM is Platform(OS) dependent Code
generation JIT is Platform Oriented, generates the native byte code,
so it is faster one than JVM :)
I am learning Java JNI and trying to understand the GetStringUTFChars & ReleaseStringUTFChars. Still i can't able to understand the ReleaseStringUTFChars.
As per my understanding from some article, in most cases that the GetStringUTFChars return a reference to the original string data and not a copy. So actually the ReleaseStringUTFChars release the jstring or the const char* (if copied) or both.
I can get a better understanding if i get the answer to the below question.
In the below code do i need to call the ReleaseStringUTFChars in a for loop or only once (with any one of the const char*)?
#define array_size 10
const char* chr[array_size];
jboolean blnIsCopy;
for (int i = 0; i < array_size; i++) {
chr[i] = env->GetStringUTFChars(myjstring, &blnIsCopy);
printf((bool)blnIsCopy ? "true\n" : "false\n"); //displays always true
printf("Address = %p\n\n",chr[i]); //displays different address
}
//ReleaseStringUTFChars with in a for loop or single statement is enough
for (int i = 0; i < array_size; i++) {
env->ReleaseStringUTFChars(myjstring, chr[i]);
}
Thanks in advance.
Get/ReleaseStringUTFChars must always be called in pairs, regardless of whether a copy or not is returned.
In practice, you pretty much always get a copy (at least with the JVM implementations I checked: OpenJDK and Dalvik) so that the GC is free to move the original array. It obviously can't collect it because you've got a reference to the string but it'll still move objects around.
There is also a GetStringCritical/ReleaseStringCritical call pair, which will always attempt to return a pointer to the original array (though in theory it may still return a copy). This makes it faster but it comes at a cost: the GC must not move the array until you release it. Again, in practice this is usually implemented by establishing a mutex with the GC, and incrementing a lock count for Get and decrementing it for Release. This means these must be called in pairs too, otherwise the lock count will never get back to zero and GC will probably never run. Please note: Get/ReleaseStringCritical also comes with other limitations which are less relevant to this question but are no less important.
I was browsing through the Java code today and I noticed something.
int[] m = mag;
int len = m.length;
int[] xm = xInt.mag;
if (len != xm.length)
return false;
(This is in the BigInteger class, which can be found by unzipping src.zip. It's in the equals method.) Why is an entirely new variable m created when it is only used once? Why isn't the code just int len = mag.length? I saw this in another method also (bitLength), and again, m is only used once. Is there any advantage to doing this or is it just a mistake by the creators of this class?
Edit: as #usernametbd pointed out, it is used a bit later:
for (int i = 0; i < len; i++)
if (xm[i] != m[i])
return false;
But they still could have just used mag. Why would an entirely new variable be made?
In a different function (in the same class, bitLength), a new variable m is made and it's only used a single time.
Because mag is a field, m is local variable. Access to local variable may be faster, though modern JITs can create such a substitute local variable automatically.
BTW you should have tell what the method you had in mind (I found it to be equals()), and cite original source (it is available) rather than decompiled one.
A bit (few lines) futher down, they use
for (int i = 0; i < len; i++)
if (xm[i] != m[i])
return false;
So m isn't completely isolated. They certainly could've used mag instead, but it's just a design choice.
When you call length (public final member variable of Array) via reflection which is constant time operation. But it is not same in C++. You have to get first array size in bytes and after divide this result to size of int to get exact value(Maybe there is better way). I think developer has the same reflex from him C++ times and carried value into local variable to use several times.
Why is it important to you? The statement is not copying an array, just copying a reference -- a pointer. And "m" will likely be allocated into a register, whereas the JVM standard requires that "mag" must usually be refetched from the object -- the JITC can't freely optimize away field references.
I have encountered a somewhat baffling problem with the simple task of filling an Array dynamically in Java. The following is a snapshot from where the problem originates:
entries = new Object[ (n = _entries.length + 1) ] ;
for(i = 0 ; i < n ; i++) {
entry = ( i == (n - 1) ) ? addition : _entries[i] ;
entries[i] = entry ;
//...
}
Where _entries is a source Array (field of the class); entries is initialized as an Array of Objects
Object[] entries = null ;
and addition is the Object to be added (passed as an Argument to the method this code is in).
The code passes the compiler but results in a memory-leak when called. I was able to narrow down the cause to the line where the code attempts to fill the new Array
entries[i] = entry ;
however, I cannot think of any reason why this would cause a memory-leak. I'm guessing the root of the issue must be either an extremely stupid fault on my part or an extremely arcane problem with Java. :-)
If you need more background let me know.
Edit:
Tomcat's log tells me:
A web application appears to have started a thread named ... but has failed to stop it.
This is very likely to create a memory leak.
Other than that obviously the page loading the class does not finish loading or loads very slowly.
Edit:
The problem might be somewhere else (at a more expected location) after all. Apparently Tomcat wasn't loading the class files all the time when I tried to pin down the faulty code and this mislead me a bit. I now suspect a infinte for-each loop caused by a defective Iterator implementation up in the call stack to be at fault.
In any case, thanks for your input! Always much appreciated!
I will use a Collection (probably a Vector) instead of an Array as a work-around; still, I'd like to know what the problem here is.
TIA,
FK82
So, about your Tomcat log message:
A web application appears to have started a thread named ... but has failed to stop it. This is very likely to create a memory leak.
This says that your servlet or something similar started a new thread, and this thread is still running when your servlet finished its operation. It doesn't relate at all to your example code (if this code isn't the one starting the thread).
Superfluous threads, even more when each HTTP-request starts a new one (which does not finish soon) can create a memory leak, since each thread needs quite some space for its stack, and also may inhibit garbage-collection by referencing objects who are not needed anymore. Make sure that your thread is really needed, and think about using a threadpool instead (preferably container-managed, if this is possible).
I cannot see a memory leak, but your code is more complicated than it needs to be. How about this:
newLength = $entries.length + 1;
entries = new Object[ newLength ] ;
for(i = 0 ; i < newLength - 1 ; i++) {
entries[i] = $entries[i];
//...
}
entries[ newLength - 1 ] = addition;
No need to check if you are at the last entry all the time and you could use a array copy method as suggested by Alison.
Think of this post as a comment. I just posted it as an answer because I don't know how code is formatted in comments...
It is working for me,
please find the sample code. and change it to accordingly
class test {
public static void main(String[] args) {
String[] str = new String[]{"1","2","3","4","5","6"};
int n=0;
Object[] entries = new Object[ (n = 5 + 1) ] ;
for(int i = 0 ; i < n ; i++) {
Object entry = ( i == (n - 1) ) ? new Object() : str [i] ;
entries[i] = entry ;
}
System.out.println(entries[3]);
}
}
Perhaps by Memory Leak you are meaning an OutOfMemoryException? Sometime you get that in Java if you do not have the minimum heap size set high enough (and also a well defined max heap size too) when you start up. If there is not enough heap created at startup then you can sometimes use it up faster than the JVM has time to allocate more memory to the heap or to garbage collect. Unfortunately, there is no "right answer" here. You just have to play with different settings to get the right result (ie, known as "tuning the JVM"). In other words, this is more of an art than a science.
And in case you didn't know, you pass the arguments to the JVM on the command line when firing up your program -Xmin 250m -Xmax 1024m is an example. You must specify the values in megabytes. The first sets the minimum heap (at startup) to 250 megabytes. The second argument sets the max heap size at one gigabyte.
Just another thought to go by as I too am puzzled by how you could trace a memory leak to one line of code.
UPDATES: thanks a lot to Gabe and Glenn for the detailed explanation. The test is wrote not for language comparison benchmark, just for my studying on VM optimization technologies.
I did a simple test to understand the performance of string concatenation between Java and Python.
The test is target for the default immutable String object/type in both languages. So I don't use StringBuilder/StringBuffer in Java test.
The test simply adds strings for 100k times. Java consumes ~32 seconds to finish, while Python only uses ~13 seconds for Unicode string and 0.042 seconds for non Unicode string.
I'm a bit surprise about the results. I thought Java should be faster than Python. What optimization technology does Python leverage to achieve better performance? Or String object is designed too heavy in Java?
OS: Ubuntu 10.04 x64
JDK: Sun 1.6.0_21
Python: 2.6.5
Java test did use -Xms1024m to minimize GC activities.
Java code:
public class StringConcateTest {
public static void test(int n) {
long start = System.currentTimeMillis();
String a = "";
for (int i = 0; i < n; i++) {
a = a.concat(String.valueOf(i));
}
long end = System.currentTimeMillis();
System.out.println(a.length() + ", time:" + (end - start));
}
public static void main(String[] args) {
for (int i = 0; i < 10; i++) {
test(1000 * 100);
}
}
}
Python code:
import time
def f(n):
start = time.time()
a = u'' #remove u to use non Unicode string
for i in xrange(n):
a = a + str(i)
print len(a), 'time', (time.time() - start)*1000.0
for j in xrange(10):
f(1000 * 100)
#Gabe's answer is correct, but needs to be shown clearly rather than hypothesized.
CPython (and probably only CPython) does an in-place string append when it can. There are limitations on when it can do this.
First, it can't do it for interned strings. That's why you'll never see this if you test with a = "testing"; a = a + "testing", because assigning a string literal results in an interned string. You have to create the string dynamically, as this code does with str(12345). (This isn't much of a limitation; once you do an append this way once, the result is an uninterned string, so if you append string literals in a loop this will only happen the first time.)
Second, Python 2.x only does this for str, not unicode. Python 3.x does do this for Unicode strings. This is strange: it's a major performance difference--a difference in complexity. This discourages using Unicode strings in 2.x, when they should be encouraging it to help the transition to 3.x.
And finally, there can be no other references to the string.
>>> a = str(12345)
>>> id(a)
3082418720
>>> a += str(67890)
>>> id(a)
3082418720
This explains why the non-Unicode version is so much faster in your test than the Unicode version.
The actual code for this is string_concatenate in Python/ceval.c, and works for both s1 = s1 + s2 and s1 += s2. The function _PyString_Resize in Objects/stringobject.c also says explicitly: The following function breaks the notion that strings are immutable. See also http://bugs.python.org/issue980695.
My guess is that Python just does a realloc on the string rather than create a new one with a copy of the old one. Since realloc takes no time when there is enough empty space following the allocation, it is very fast.
So how come Python can call realloc and Java can't? Python's garbage collector uses reference counting so it can tell that nobody else is using the string and it won't matter if the string changes. Java's garbage collector doesn't maintain reference counts so it can't tell whether any other reference to the string is extant, meaning it has no choice but to create a whole new copy of the string on every concatenation.
EDIT: Although I don't know that Python actually does call realloc on a concat, here's the comment for _PyString_Resize in stringobject.c indicating why it can:
The following function breaks the notion that strings are immutable:
it changes the size of a string. We get away with this only if there
is only one module referencing the object. You can also think of it
as creating a new string object and destroying the old one, only
more efficiently. In any case, don't use this if the string may
already be known to some other part of the code...
I don't think your test means a lot, since Java and Python handle strings differently (I am no expert in Python but I do know my way in Java). StringBuilders/Buffers exists for a reason in Java. The language designers didn't do any kind of more efficient memory management/manipulation exactly for this reason: there are other tools than the "String" object to do this kind of manipulation and they expect you to use them when you code.
When you do things the way they are meant to be done in Java, you will be surprised how fast the platform is... But I have to admit that I have been pretty much impressed by the performance of some Python applications I have tried recently.
I do not know the answer for sure. But here are some thoughts. First, Java internally stores strings as char [] arrays containing the UTF-16 encoding of the string. This means that every character in the strings takes at least two bytes. So just in terms of raw storage, Java would have to copy around twice as much data as python strings. Python unicode strings are therefore the better test because they are similarly capable. Perhaps python stores unicode strings as UTF-8 encoded bytes. In that case, if all you are storing in these are ASCII characters, then again you'd have Java using twice as much space and therefore doing twice as much copying. To get a better comparison you should concatenate strings containing more interesting characters that require two or more bytes in their UTF-8 encoding.
I ran Java code with a StringBuilder in place of a String and saw an average finish time of 10ms (high 34ms, low 5ms).
As for the Python code, using "Method 6" here (found to be the fastest method), I was able to achieve an average of 84ms (high 91ms, low 81ms) using unicode strings. Using non-unicode strings reduced these numbers by ~25ms.
As such, it can be said based on these highly unscientific tests that using the fastest available method for string concatenation, Java is roughly an order of magnitude faster than Python.
But I still <3 Python ;)