I get this question asked many times. What is a good way to answer
Can there be memory leak in Java?
The answer is that it depends on what kind of memory leak you are talking about.
Classic C / C++ memory leaks occur when an application neglects to free or dispose an object when they are done with it, and it leaks. Cyclic references are a sub-case of this where the application has difficulty knowing when to free / dispose, and neglects to do it as a result. Related problems are where the application uses an object after it has been freed, or attempts to free it twice. (You could call the latter problems memory leaks, or just bugs. Either way ... )
Java and other (fully1) managed languages mostly don't suffer from these problems because the GC takes care of freeing objects that are no longer reachable. (Certainly, dangling pointer and double-free problems don't exist, and cycles are not problematic as they are for C / C++ "smart pointers" and other reference count schemes.)
But in some cases GC in Java will miss objects that (from the perspective of the programmer) should be garbage collected. This happens when the GC cannot figure out that an object cannot be reached:
The logic / state of the program might be such that the execution paths that would use some variable cannot occur. The developer can see this as obvious, but the GC cannot be sure, and errs on the side of caution (as it is required to).
The programmer could be wrong about it, and the GC is avoiding what might otherwise result in a dangling reference.
(Note that the causes of memory leaks in Java can be simple, or quite subtle; see #jonathan.cone's answer for some subtle ones. The last one potentially involves external resources that you shouldn't rely on the GC to deal with anyway.)
Either way, you can have a situation where unwanted objects cannot be garbage collected, and hang around tying up memory ... a memory leak.
Then there is the problem that a Java application or library can allocate off-heap objects via native code that need to be managed manually. If the application / library is buggy or is used incorrectly, you can get a native memory leak. (For example: Android Bitmap memory leak ... noting that this problem is fixed in later versions of Android.)
1 - I'm alluding to a couple of things. Some managed languages allow you to write unmanaged code where you can create classic storage leaks. Some other managed languages (or more precisely language implementations) use reference counting rather than proper garbage collecting. A reference count-based storage manager needs something (i.e. the application) to break cycles ... or else storage leaks will ensue.
Yes. Memory leaks can still occur even when you have a GC. For example, you might hold on to resources such as database result sets which you must close manually.
Well, considering that java uses a garbage collector to collect unused objects, you can't have a dangling pointer. However, you could keep an object in scope for longer than it needs to be, which could be considered a memory leak. More on this here: http://web.archive.org/web/20120722095536/http://www.ibm.com:80/developerworks/rational/library/05/0816_GuptaPalanki/
Are you taking a test on this or something? Because that's at least an A+ right there.
The answer is a resounding yes, but this is generally a result of the programming model rather than an indication of some defect in the JVM. This is common when frameworks have lifecycles different of that than a running JVM. Some examples are:
Reloading a context
Failing to dereference observers (listeners)
Forgetting to clean up resources after you're finished using them *
* - Billions of consulting dollars have been made resolving the last one
Yes, in the sense that your Java application can accumulate memory over time that the garbage collector is unable to free.
By maintaining references to uneeded/unwanted objects they will never fall out of scope and their memory will not be claimed back.
yes, if you don't de-reference objects they will never be garbage-collected and memory usage will increase. however because of how java is designed, this is difficult to achieve whereas in some other languages this is sometimes difficult not to achieve.
edit: read Amokrane's link. it's good.
Yes it is possible.
In Effective Java there is an example involving a stack implemented using arrays. If your pop operations simply decrement the index value it is possible to have a memory leak. Why? Because your array still has a reference to the popped value and you still have a reference to the stack object. So the correct thing to do for this stack implementation would be to clear the reference to the popped value using an explicit null assignment at the popped array index.
The short answer:
A competent JVM has no memory
leaks, but more memory can be used
than is needed, because not all unused
objects have been garbage collected,
yet. Also, Java apps themselves can hold references to objects they no longer need and this can result in a memory leak.
The book Effective Java gives two more reasons for "memory leaks":
Once you put object reference in Cache and forget that it's there. The reference remains in cache long before becoming irrelevant. Solution is to represent cache as a WeakHashMap
in an API where clients register callbacks and don't re-register them explicitly. Solution is to store only weak references to them.
Yes, it can be, in a context when a program mistakenly hold a reference to an object that would be never used again and therefore it's not cleaned by the GC.
An example to it would be forgetting to close an opened stream:
class MemoryLeak {
private void startLeaking() throws IOException {
StringBuilder input = new StringBuilder();
URLConnection conn = new URL("www.example.com/file.txt").openConnection();
BufferedReader br = new BufferedReader(new InputStreamReader(conn.getInputStream(), StandardCharsets.UTF_8));
while (br.readLine() != null) {
input.append(br.readLine());
}
}
public static void main(String[] args) throws IOException {
MemoryLeak ml = new MemoryLeak();
ml.startLeaking();
}
}
One simple answer is : JVM will take care of all your initialization of POJO's [plain old java objects] as long as you are not working with JNI. With JNI if you have made any memory allocation with the native code you have to take care of that memory by yourself.
Yes. A memory leak is unused memory not released to the memory manager by the app.
I've seen many times Java code wich stores items on a data structure but the items are never removed from there, filling the memory until an OutOfMemoryError:
void f() {
List<Integer> w = new ArrayList<Integer>();
while (true) {
w.add(new Integer(42));
}
}
While this example is too obvious, Java memory errors tend to be more subtle. For example, using Dependency Injection storing a huge object on a component with SESSION scope, without releasing it when the object is no longer used.
On a 64 bits VM this tends to get worse since the swap memory space starts to get filled until the system crawls on too many IO operations.
Related
I have a question regarding dynamic memory allocation.
When it comes to C, memory is allocated using the functions malloc(), calloc() and realloc() and de-allocated using free().
However in objected oriented languages like C++,C# and Java, memory is dynamically allocated using the new and deallocated using delete keywords (operators) in case of C++.
My question is, why are there operators instead of functions for these objected oriented languages for dynamic memory allocation? Even when using new, finally a pointer is returned to the class object reference during allocation, just like a function.
Is this done only to simplify the syntax? Or is there a more profound reason?
In C, the memory allocation functions are just that. They allocate memory. Nothing else. And you have to remember to release that memory when done.
In the OO languages (C++, C#, Java, ...), a new operator will allocate memory, but it will also call the object constructor, which is a special method for initializing the object.
As you can see, that is semantically a totally different thing. The new operator is not just simpler syntax, it's actually different from plain memory allocation.
In C++, you still have to remember to release that memory when done.
In C# and Java, that will be handled for you by the Garbage Collector.
I believe it's done solely to simplify the syntax as you've said.
Operators are simply another way to call methods (or functions).
using "12 + 13" is no different than using Add(12, 13).
A way to see this is via the operator overrides in C# for example:
// Sample from - https://msdn.microsoft.com/en-us/library/8edha89s.aspx
public static Complex operator +(Complex c1, Complex c2)
{
Return new Complex(c1.real + c2.real, c1.imaginary + c2.imaginary);
}
It's a regular method but allows the usage of operators over complex classes.
I'm using the Add operator as an example since I see it as no different than the memory allocation operators such as "new".
The whole point of Object Oriented design/programming is to provide meaningful abstractions.
When you are doing good OO design; you do not think (immediately) on areas in memory. One thinks about of objects; that carry state and provide behavior.
Even when writing code in C++, in most cases, I don't have to worry about subtleties like "why will my bits be aligned", "how much memory does one of my objects required at runtime" and so on. Of course, these questions are relevant in certain situations; but within OO design; the true value comes from creating useful abstractions that help to solve "whatever domain" problems as precise, easy, maintainable, ... as possible.
For the "keyword" versus "function" thing: just have a look at Java. The fathers of the language simply didn't want Java programmers start thinking about "memory pointers". You only deal with objects; and references to objects. Thus, the concept of "allocating" memory, and getting back a "pointer" simply does not exist at all here. So, how would you then provide this functionality as library method?! Well, if you would like to: you can't.
Finally, to a certain degree, this is a matter of "taste/style" by the people designing the language. Sometimes people prefer a small language core; and do everything in libraries; and other people prefer to have "more" things built-in.
The new keyword is ideed to simplify the syntax, which is pretty suggestive and also does more than memory allocation, it invokes the constructor(s) also.
One thing you have said:
C++,C# and Java, memory is dynamically allocated and de-allocated using the new and delete keywords (operators)
for Java and C# it is only the new keyword, there is no delete. I know that in C# you are able to use using blocks to ensure that the resource will be released when the object is not used anymore, but this does not involves memory deallocation in every case, such as it's calling the Dispose method.
One more thing which needs to be pointed is that the goal of an object oriented programming language, as GhostCat just said, is to release the programmer to think of how memory is allocated in most of the cases, and more important, how are the objects released, this is why garbage collector was introduced.
The main principle is that as the programming language is higher, it has to abstract such things as memory management, and provide easy ways to solve the actual business problems one is looking for. Of course this might been considered when a programming langage is chosed for a specific task.
C :malloc calloc are basically the only ways in C to allocate memory.
malloc : it allocate uninitialized memory according to requested size without initializing it to any value
calloc : almost same as malloc ,plus it also initialize it to zero(0).
In both cases , you required something :
The requested memory size for allocation should be given at the time of initialization and it can be increase with realloc.
The allocated memory need to be deleted with free ,sometimes it can be result in a OOM error if somebody don't have a good memory to free the allocated memory although free is quite handy when you are doing lot of memory extensive work.
NOTE : Casting and size(to allocate memory) is required with malloc and calloc
C++: C++ also has malloc and calloc (free and reallocate too) along new and delete ,new and delete can think of as a modern way to allocate and free memory but not all of the OOP's based language have both. e.g java don't have delete.
new uses constructors to initialize default value so it's pretty useful while working with objects when you have various scenarios to set initial value using parameterize ,default or copy constructors.
NOTE : With new you don't have to do the appropriate casing unlike with malloc and calloc and no need to give a memory size for allocation. one less thing , right.
delete is used to release the memory, the delete call on some object also calls destructor which is the last place of the life-cycle of that object where you can do some farewell tasks like saving current state etc and then memory will be released .
Note : In C# and java the deallocation of memory is handled by Garbage-Collector who does the memory management to release the memory.It used various algos like mark-sweep to release the memory if there is no reference variable pointing to that memory or the reference variable value is set as null.
This may also lead to memory leak if there is a reference variable pointing to that object in memory which is no longer required.
The downside of GC is, this makes things slow
I try to do my best to explain my question. Maybe it's a bit abstract.
I read some literature about not invoking GC explictly in Java code, finalize method, pointing to null, etc.
I have some large XMLs files (customer invoices). Using Jaxb, the file marshals in a complex Java object. Its attributes are basic types (Integer, BigDecimal, String, etc.) but also class of other complex classes, list of other classes, list of classes with list as attribute, etc.
When I do my stuff with the object, I need to remove it from the memory. Some XML are very large and I can avoid a memory leak or OutOfMemoryError situation.
So, my questions are:
Is it enough to assign big object to null? I read that, if there are soft references, GC will not free the object.
Should I do a in deep clearing of the object, clearing all list, assigning null to the attributes, etc.?
What about JaxB (I'm using Java6, so JaxB is built in) and soft references? JaxB is faster than old JibX marshaller, but I don't know if it's worse in memory usage.
Should I wrap the megacomplex JaxB class with WeakReference or something like this?
Excuse me for mixing Java memory usage concepts, JaxB, etc. I'm studying the stability of a large running process, and the .hprof files evidence that all customers data of all invoices remains in memory.
Excuse me if it's a simple, basic or rare question.
Thanks in advance
Unless something else points to parts of your big object (graph), assigning the big object reference null is enough.
Safest though, would be to use a profiler after your application has been running for a while, and look at the object references, and see if there's something that isn't properly GC'ed.
Is it enough to assign big object to null? I read that, if there are soft references, GC will not free the object.
The short answer is yes. It is enough to assign (all strong references to) a big object to null - if you do this, the object will no longer be considered "strongly reachable" by the Garbage Collector.
Soft references will not be a problem in your case, because it's guaranteed that softly reachable objects will be garbage collected before an OutOfMemoryError is thrown. They might well prevent the garbage collector from collecting the object immediately (if they didn't, they'd act exactly the same as weak references). But this memory use would be "temporary", in that it would be freed up if it were needed to fulfil an allocation request.
Should I do a in deep clearing of the object, clearing all list, assigning null to the attributes, etc.?
That would probably be a bad idea. If the field values are only referenced by the outer big object, then they will also be garbage collected when the big object is collected. And if they are not, then the other parts of the code that reference them will not be happy to see that you're removing members from a list they're using!
In the best case this does nothing, and in the worst case this will break your program. Don't let the lure of this distract you from addressing the sole actual issue of whether your object is strongly-reachable or not.
What about JaxB (I'm using Java6, so JaxB is built in) and soft references? JaxB is faster than old JibX marshaller, but I don't know if it's worse in memory usage.
I'm not especially familiar with the relative time and space performance of those libraries. But in general, it's safe to assume a very strong "innocent until proven guilty" attitude with core libraries. If there were a memory leak bug, it would probably have been found, reported and fixed by now (unless you're doing something very niche).
If there's a memory leak, I'm 99.9% sure that it's your own code that's at fault.
Should I wrap the megacomplex JaxB class with WeakReference or something like this?
This sounds like you may be throwing GC "fixes" at the problem without thinking through what is actually needed.
If the JaxB class ought to be weakly referenced, then by all means this is a good idea (and it should be there already). But if it shouldn't, then definitely don't do this. Weak referencing is more a question of the overall semantics, and shouldn't be something you introduce specifically to avoid memory issues.
If the outer code needs a reference to the object, then it needs a reference - there's no magic you can do to have the intance be garbage collected yet still available. If it doesn't need a reference (beyond a certain point), then it doesn't need one at all - better to just nullify a standard [strong] reference, or let it fall out of scope. Weak references are a specialist situation, and are generally used when you don't have full control over the point where an object ceases to be relevant. Which is probably not the case here.
the .hprof files evidence that all customers data of all invoices remains in memory.
This suggests that they are indeed being referenced longer than is necessary.
The good news is that the hprof file will contain details of exactly what is referencing them. Look at an invoice instance that you would expect to have been GCed, and see what is referencing it and preventing it from being GCed. Then look into the class in question to see how you expect that reference to be freed, and why it hasn't been in this case.
All good performance/memory tweaking is based on measurements. Taking heap dumps, and inspecting the instances and references to them, is your measurements. Do this and act on the results, rather than trying to wrap things in WeakReferences on the hope that it might help.
You wrote
hprof files evidence that all customers data of all invoices remains in memory.
You should analyse it using mat. Some good notes at http://memoryanalyzer.blogspot.in/
Once the String object is created , we can't modify it But if we do any operations on it JVM will create New Object. Here by creating new objects then JVM consumes more memory. Then i think it causes to memory issue right.?
You are correct. It is definitely worth being aware of this issue, even if it doesn't affect you every time.
As you say, Strings cannot change after creation - they're immutable and they don't expose many ways to change them.
However, operations such as a split() will be generating additional string objects in the background, and each of those strings have a memory overhead if you are holding onto references to them.
As the other posters note, the objects will be small and garbage collection will usually clean up the old ones after they have gone out of scope, so you generally won't have to worry about this.
However, if you're doing something specific and holding onto large amounts of string references then this could bite you.
Look at String interning depending on your use case, noting the warnings on the linked page.
Two things to note:
1) Hard coded String literals will be automatically interned by Java, reducing the impact of this.
2) The + operator is more efficient in this regard, it will use String Builders underneath giving performance & memory benefits.
No, that does not. If you do not hold strong links to String instances they eventually will be collected by a garbage collector.
For example:
while (true) {
new String("that is a string");
}
in this snippet you continuously create new object instances, however you will never get OutOfMemoryException as created instances become garbage (there are obviously no strong links).
It consumes more memory for new objects, that's right. But that fact in itself does not create an issue, because garbage collector promptly reclaims all inaccessible memory. Of course you can turn it into an issue by creating links to the newly created strings, but that would be an issue of your program, not of JVM.
The biggest memory issue you have to know about is taking a small substring of a huge string. That substring shares the original string's char array and even if the original string gets gc'd, the huge char array will still be referenced by the substring. The workaround is to use new String(hugeString.substring(i)).
The issue that is generated is the fact that garbage is generated. This issue is resolved by the virtual machine by calling the garbage collector which frees the memory used by that garbage.
As soon as the old object is not used anymore, it can be removed by the garbage collector. (Which will be done far before any memory issue arises).
If you want to prevent the copying of the data, use a StringBuilder.
Unused objects are collected by GC.
and Immutability got many benefits in java.
In Java achieving as much immutability as possible is a good practice.
They can be safely used in Collections frameworks also.
Check this
As far as I know StringBuilder (or StringBuffer for thread safe) is useful for managing String and make them mutable.
Manipulate some characters in a huge String do not 'eat' many bytes in memory.
It is also more powerful/speed for concate.
Since a string instance is immutable it can be reused by the jvm. The String class is implemented with Flyweight Design Pattern that is used to avoid memory issues.
This is just an academic question (I would never do this in real code):
If I were to use shared_ptr<> universally in my code, would the behavior be equivalent to a gc-collected language like Java?
If not, how would the behavior be different from a gc-embedded language? Which C++ construct would yield equivalent behavior compared to a gc-embedded language?
Note: In real coding, I strongly prefer the use of RAII and strict ownership over the use of any smart pointers. I also know that other less-generic pointers, unique_ptr<> would be more efficient. This question is just a query into smart-pointer equivalence.
No, there'd be a couple of important differences:
You would get a memory leak any time you have a cyclic reference. A garbage collector can handle cycles, ref-counting can't.
You would avoid any stalls or pauses because no garbage collection ever occurs. On the other hand, you'd likely spend more total CPU time cleaning up resources, because the amortized cost of an occasional garbage collection is pretty low, and ref-counting can be relatively expensive if you do it on everything.
Obviously the first point is the killer. If you did this, many of your resources wouldn't get freed, and you'd leak memory and your app just wouldn't behave very well.
Which C++ construct would yield equivalent behavior compared to a gc-embedded language?
None. C++ doesn't have a garbage collector because there's no way to implement a correct, reliable one. (Yes, I'm aware of Boehm's GC, and it's a good approximation, but it's conservative, and doesn't detect all references, only the ones it can be 100% sure of. There is no way, in a general C++ program, to implement a garbage collector that Just Works(tm))
#jalf says this in his answer:
You would avoid any stalls or pauses because no garbage collection ever occurs.
While smart pointers (or any reference counting scheme) have no pause while garbage collection occurs, you can get a pause if you null the last external pointer to a large data structure, and trigger a cascade of reference count adjustments and finalizations for each node in the data structure. While a smart smart-pointer implementation could ameliorate this, you'd be sacrificing immediate reclamation ... which some people claim is an advantage of smart pointers.
Also, there is an overhead of a few instructions each time you assign to a smart pointer-typed variable, and the overheads of allocating an object is greater.
Garbage collection happens whenever the GC decides that it should. shared_ptrs are not collected. An object managed by a shared_ptr will only ever be destroyed in the destructor of a shared_ptr. And therefore, you know exactly when memory can and can not be freed.
You still have control over when memory goes away with shared_ptr. You don't have that with a garbage collector (outside of coarse-grained commands like turning it on/off or modifying it's behavior a bit).
The main difference is that reference counting alone can't free circular data structures.
Many cases of such structures can nevertheless be handled by using weak_ptr appropriately, and some cases can be handled by delegating cleanup responsibility to a collection object.
However, the most frivolous spaghetti structures, if you want them (e.g. for math), can't have automated cleanup implemented by reference counting alone, because there will be circular sub-structures.
Cheers & hth.,
Its worth noting that a shared ptr is much larger that a Java reference. Generally this won't matter but some situations it might.
In Java 6, 64-bit JVMs still use 32-bit references access up to 32 GB of heap (it can do this because objects are on 8 byte boundaries) However a shared ptr uses two pointers (each 8 bytes in a 64-bit applications), the second pointer references an object which contains the counter. On libgcc it allocates 32-byte minimum to any malloc/new object. In total the shared pointer could be using 48 bytes which is relatively larger than 4 bytes. 44 bytes is not going to make a difference, but it could if you have lots of these.
I've never seen such statements though,does it exist in java world at all?
Java's version of malloc is new -- it creates a new object of a specified type.
In Java, memory is managed for you, so you cannot explicitly delete or free an object.
Java has a garbage collector. That's why you never see such statements in your code(which is nice if you ask me)
In computer science, garbage
collection (GC) is a form of automatic
memory management. It is a special
case of resource management, in which
the limited resource being managed is
memory. The garbage collector, or just
collector, attempts to reclaim
garbage, or memory occupied by objects
that are no longer in use by the
program. Garbage collection was
invented by John McCarthy around 1959
to solve problems in Lisp.
new instead of malloc, garbage collector instead of free.
No direct equivalents exist in Java:
C malloc creates an untyped heap node and returns you a pointer to it that allows you to access the memory however you want.
Java does not have the concept of an untyped object, and does not allow you to access memory directly. The closest that you can get in Java to malloc would be new byte[size], but that returns you a strongly typed object that you can ONLY use as a byte array.
C free releases a heap node.
Java does not allow you to explicitly release objects. Object deallocation in Java is totally in the hands of the garbage collector. In some cases you can influence the behaviour of the GC; e.g. by assigning null to a reference variable and calling System.gc(). However, this does not force the object to be deallocated ... and is a very expensive way to proceed.
If you are up to no good (tm) I suppose you can get access to raw memory though the JNI interface. This is where you can call C programs from Java Programs. Of course you have to be running in an environment where your program has the privileges to do so (a browser won't normally allow this unless it is suicidal) but you can access objects via C pointers that way.
I sort of wonder where the original question is coming from. At one point long ago I was totally skeptical of the notion that C-style memory management and C-style pointers were not needed, but at this point I am true believer.