When running the following code in Matlab, the physical memory usage of Matlab increases until it reaches 90% of my system memory, and crashes the program. This code is part of a larger optimization scheme that will have many more iterations, but it has been simplified here.
import org.parent.modeling.*
for i = 1:100
rTool = Tool('C:/etc');
model = rTool.getModel();
model.initSystem();
rTool.setModel(model);
rTool.run();
end
rTool and model are stored as Java objects in the workspace.
By selectively removing steps from the bottom to top, I've determined that the rTool.run(); step is causing an incremental memory increase during each iteration.
I've tried using a variety of memory clearing techniques with no success
clear classes, clear java, clear rTool model, clear all
I have checked for fragmentation by running the memory command, but it is insignificant.
Below is the actual code that corresponds to the API documentation (https://simtk.org/api_docs/opensim/api_docs/classOpenSim_1_1RRATool.html)
for i = 1:1:4
import org.opensim.modeling.*
rraTool = RRATool('C:\...filename.xml');
model = rraTool.getModel();
model.initSystem();
rraTool.setModel(model);
rraTool.run();
end
Any suggestions for reducing the physical memory increase?
Matlab: 2013
It is difficult to figure out what you are actually doing here, let alone why it is leaking memory. From what I have been able to deduce, you can call Java from MatLab, and you can call C++ directly from Matlab. I cannot figure out if you are calling the C++ library directly, or calling it via a Java (JNI / JNA) wrapper. And if you are doing the latter ... why.
For the Matlab -> C++ case, this page has some tips on avoiding memory leaks: http://www.mathworks.com.au/help/matlab/matlab_external/memory-management-issues.html
For the Matlab -> Java -> C++ case, the leaks could be due to bugs in the Java wrapper or its associated C++-side wrapper code, or a problem in the way that you are using the wrapper. For instance, there could be some dispose method that you need to call. The details should be in the Java API docs ... which you have not been able to locate ... or in the Java source code itself.
I was hoping that there were some general memory clearing Java commands I could use.
There are no such magic commands. You need to figure out where the memory is leaking and fix that.
Related
I have a lot of road traffic and markup elements, charts, nodes and arcs within my Main agent. When running the simulation it throws the following error:
Description: The code of method _createPersistentElementsBP4_xjal() is exceeding the 65535 bytes limit.
I read this article: https://noorjax.com/2018/10/17/your-agent-is-too-big-memory-problem/
However, I would like to know how much have I exceeded the limit. Is there any way of getting this information? Because if it is not that far from the threshold, I can make some modifications to drop below that threshold. Otherwise it is painful to create so many new agents, etc.
This is a Java Virtual Machine (JVM) restriction on the Java bytecode size for the method body (i.e., the compiled code size) as I understand it (e.g., see Baeldung's description which links to the relevant JVM specification details). Thus, even though you can see the generated Java source code for the offending method, it isn't actually the length of that that is the limitation (though obviously the length of the source code correlates to some degree with the size of the compiled bytecode).
[As such, I'm surprised if Felipe's idea of reducing variable name lengths makes any difference since they're not stored explicitly like that in the bytecode...]
So, no, you can't tell how much you've exceeded it by (unless I guess you actually interrogate compiled class files and know exactly what you're doing). Even though it is AnyLogic's code generation that is 'causing' the problem, any such situation will normally always be something that you could re-architect better (as with Felipe's example) from an object-oriented (or data structuring) design perspective in the model.
I have homework assignment and the limit of memory is 64 MB and my code uses 68 MB.
I already set my arrays to null and ran Runtime.getRuntime().gc();
Is there anything i can do? Delete all stuff from memory that i don't need? How?
I have 3 int arrays and 3 float arrays, 2 double arrays all of the same N given size.
a = null;
size = null;
prize = null;
required = null;
b= null;
potr= null;
result = null;
Runtime.getRuntime().gc();
First, check whether your program really won't run with 64 MB, using the command-line option -Xmx64m (exact syntax might depend on your JRE version). You only have a problem if you get an OutOfMemoryError under that setting.
If Java runs without a command-line limit, it tends to grab more memory than strictly necessary. So I guess, if you see 68 MB in Windows Task Manager or equivalent, everything will be OK.
If you really run into an OutOfMemoryError, you can try to profile your application. At our company, we're using the commercial JProfiler, a tool that can give you detailed information on memory usage. But it takes a learning curve to understand the profiling results.
An alternative is to post the complete code here or at https://codereview.stackexchange.com/, so we can give more specific help.
Presumably it is code that is taking the bulk of the memory up. Make use of fewer obscure libraries classes. Or get hold of JDK 1.00.
what all external libraries your code refers.....? jvm loads all classes in memory before execution....if you remove unnecessary imports and libraries that will reduce your memory footprint ...also you can use earlier version of jvm with less features if it satisfies your requirement .hope that helps .
String val="Hello";
//blocks of code
int c=val.hashCode(); //say i get 101 as memory location.
Say i don't know what is the "val" at mem location 101 .And i want to store val="abc" at this location. How can i use the memory location to append the val ? Is this possible in Java.
Java doesn't allow direct manipulation of memory locations. It is built as a layer between the user and the actual memory.
Not to mention that hashcode has nothing to do with the memory address.
Per #ValekHalfHeart's comment I did a quick search and appearantly there is an API available for unsafe operations (including changes to the the memory location). I have no experience with this, but it might look like a backdoor to do what you want.
Do note that Java is explicitly built to abstract all these things away from you. Trying to use them after all is abusing your tools.
The short answer to your question is "no". Java "protects" you from knowing (or caring) about the memory locations of objects, allocation, destruction, etc.
If you like that kind of thing, might I recommend you use C? It's a much maligned language - but it's great for that kind of "close to the silicon" programming. With all the perils that entails...
I am profiling my Java application using VisualVM
and I have gone through
profiling_with_visualvm_part_1
profiling_with_visualvm_part_2
When I see memory profile result, I see millions of Objects[], Char[], String and other such fundamental objects created which is taking all the memory. I want to know which of my classes (or my code) are actually responsible for creating those Objects[] and String etc, so far I couldn't find it. Once I know the culprit class I can dive-in the code and fix it.
I put a filter com.mypackage.*, but I see all of them are many times smaller (sometimes 0byte) compared to the total size of Objects[],Char[], String objects.
I believe, there should be a way to find the culprit code. Otherwise, profiler won't be of much use.
Let me know if my question is not clear, I will try to clarify further.
If you want to see, which code allocates those instances, go to 'Memory settings' and enable 'Record allocations stack traces'. 'Record allocations stack traces' option is explained 'Profiling with VisualVM part 2'. Once you turned it on, profile your application, take snapshot of profiling results. In the snapshot right-click on the particular class and invoke 'Show allocation stacktraces'.
I have a native C library that runs some algorithms on very large datasets (on the order of hundreds of mb to gb). This is being called from within a Java framework using JNA. Java loads the data and passes it through JNA to the C library.
The problem is, there seems to be an inordinate amount of memory being used. For one dataset, the process uses about 3.0gb after all the loading is finished on the Java end, and 2.0gb is in use by the C library (as determined using in-house memory management). But the process, once the C library is called, ends up maxing out at about 9.5gb!
Specific questions, then:
Is there no overlap between the Java and C side? That is to say, does JNA produce a C-valid copy of the Java data (all int and double arrays, by the way) and pass that to the native library instead of the same blocks that contain the data in Java?
Even presuming there is no overlap, and the native library is using a copy of the data contained within the JVM, where is that extra 4.5gb coming from? That about doubles the amount of system memory taken up by the process, and i can't imagine where it's all going. The documentation on these aspects of JNA seems very limited, but I wonder if someone more familiar than I am with JNA might know why it's using so much memory, and if and how I might be able to reduce its footprint.
EDIT: The JNA-enabled Java class looks like this:
public interface MyNativeLibrary extends Library {
MyNativeLibrary INSTANCE = (MyNativeLibrary) Native.loadLibrary(
"native_library", MyNativeLibrary.class);
int native_library_function(int num_inputs, int inputs[], int max_num_outputs, int preallocated_outputs[]);
}
In this case, the return value of the native function would be the number of outputs returned, or an error code. The C interface is specified with int32_t, to make sure the sizes match.
If you want to allocate memory only on the native side, use the Memory class in JNA and access the memory using java.nio.ByteBuffer (Memory has a getByteBuffer() method). You can pass your arrays of integers as JNA Pointers the allocated memory. This should save you quite a bit of space and improve performance. Just make sure you manage the native memory resources on the Java side.