I have a native C library that runs some algorithms on very large datasets (on the order of hundreds of mb to gb). This is being called from within a Java framework using JNA. Java loads the data and passes it through JNA to the C library.
The problem is, there seems to be an inordinate amount of memory being used. For one dataset, the process uses about 3.0gb after all the loading is finished on the Java end, and 2.0gb is in use by the C library (as determined using in-house memory management). But the process, once the C library is called, ends up maxing out at about 9.5gb!
Specific questions, then:
Is there no overlap between the Java and C side? That is to say, does JNA produce a C-valid copy of the Java data (all int and double arrays, by the way) and pass that to the native library instead of the same blocks that contain the data in Java?
Even presuming there is no overlap, and the native library is using a copy of the data contained within the JVM, where is that extra 4.5gb coming from? That about doubles the amount of system memory taken up by the process, and i can't imagine where it's all going. The documentation on these aspects of JNA seems very limited, but I wonder if someone more familiar than I am with JNA might know why it's using so much memory, and if and how I might be able to reduce its footprint.
EDIT: The JNA-enabled Java class looks like this:
public interface MyNativeLibrary extends Library {
MyNativeLibrary INSTANCE = (MyNativeLibrary) Native.loadLibrary(
"native_library", MyNativeLibrary.class);
int native_library_function(int num_inputs, int inputs[], int max_num_outputs, int preallocated_outputs[]);
}
In this case, the return value of the native function would be the number of outputs returned, or an error code. The C interface is specified with int32_t, to make sure the sizes match.
If you want to allocate memory only on the native side, use the Memory class in JNA and access the memory using java.nio.ByteBuffer (Memory has a getByteBuffer() method). You can pass your arrays of integers as JNA Pointers the allocated memory. This should save you quite a bit of space and improve performance. Just make sure you manage the native memory resources on the Java side.
Related
In my Spark program, I'm interested in allocating and using data that is not touched by Java's garbage collector. Basically, I want to do the memory management of such data myself like you would do in C++. Is this a good case of using off heap memory? Secondly, how do you read and write to off heap memory in Java or Scala. I tried searching for examples, but couldn't find any.
Manual memory management is a viable optimization strategy for garbage collected languages. Garbage collection is a known source of overhead and algorithms can be tailored to minimize it. For example, when picking a hash table implementation one might prefer Open Addressing because it allocates its entries manually on the main array instead of handling them to the language memory allocation and its GC. As another example, here's a Trie searcher that packs the Trie into a single byte array in order to mimimize the GC overhead. Similar optimization can be used for regular expressions.
That kind of optimization, when the Java arrays are used as a low-level storage for the data, goes hand in hand with the Data-oriented design, where data is stored in arrays in order to achieve better cache locality. Data-oriented design is widely used in gamedev, where the performance matters.
In JavaScript this kind of array-backed data storage is an important part of asm.js.
The array-backed approach is sufficiently supported by most garbage collectors used in the Java world, as they'll try to avoid moving the large arrays around.
If you want to dig deeper, in Linux you can create a file inside the "/dev/shm" filesystem. This filesystem is backed by RAM and won't be flushed to disk unless your operating system is out of memory. Memory-mapping such files (with FileChannel.map) is a good enough way to get the off-heap memory directly from the operating system. (MappedByteBuffer operations are JIT-optimized to direct memory access, minus the boundary checks).
If you want to go even deeper, then you'll have to resort to JNI libraries in order to access the C-level memory allocator, malloc.
If you are not able to achieve "Efficiency with Algorithms, Performance with Data Structures", and if efficiency and performance are so critical, you could consider using "sun.misc.Unsafe". As the name suggests it is unsafe!!!
Spark is already using it as mentioned in project-tungsten.
Also, you can start here, to understand it better!!!
Note: Spark provides a highly concurrent for execution of application and with multiple JVMs mostlikely across multiple machines, manual memory management will be extreamly complex. Fundamemtally spark promotes re-computation over global shared memory. So, perhaps, you could store partially computed data/result in another store like HDFS/Kafka/cassandra!!!
Have a look at ByteBuffer.allocateDirect(int bytes). You don't need to memory map files to make use of them.
Off heap can be a good choice if the objects will stick there for a while (i.e. are reused). If you'll be allocating/deallocating them as you go, that's going to be slower.
Unsafe is cool but it's going to be removed. Probably in Java 9.
I have a Java program which is designed to display images in a custom file format, read through a C++ library using JNI. The data is loaded into an char array on the C++ side, and transferred to a BufferedImage on the Java side. Since sections of the image could be removed from memory and need to be reloaded fairly regularly, so I want these operations to be as fast as possible.
The way I'm currently doing this is data gets read from the file into a buffer in the C++ library's memory. In order to populate the BufferedImage, the Java code makes a JNI function call for each pixel to read from this buffer and, if necessary, load another chunk of data into the buffer. This works, but with a higher overhead than I'd like.
What I've been planning to do to improve this is pass the BufferedImage object to the C++ code using a JNI call, and make function calls on it from that side. I've been researching JNI as much as I can, but I haven't been able to find out if there's any cost in modifying Java objects from a C++ library which was loaded by the JVM. Is this a good way to implement this, or is there a faster way to transfer large amounts of byte data with JNI?
The most efficient way to move data across the JNI boundary is probably via direct ByteBuffers. You should move as much as you can at a time, as it is crossing the boundary itself that is inefficient, not the transfer. Certainly more than one pixel, preferably megabytes at a time.
I wrote a small java program which loads data from DB2 database using simple JDBC call. I am using select query to get data and using java statement for this purpose. I have properly closed statement and connection objects. I am using 64 bit JVM for compilation and for running the program.
The query is returning 52 million records, each row having 24 columns, which takes me around 4 minutes to load complete data in Unix (having multiprocessor environment). I am using HashMap as data-structure to load the data: Map<String, Map<String, GridTradeStatus>>. The bean GridTradeStatus is a simple getter/setter bean with 24 properties in it.
The memory required for the program is alarmingly high. Java heap size goes up to 5.8 - 6GB to load complete data while actual used heap size remains between 4.7 - 4.9GB. I know that we should not load this much data into memory but my business requirements are in that way only.
The question is that when I put whole data of my table in a flat file it comes out to be roughly equivalent to ~1.2GB. I want to know why my java program is consuming memory 4 times more that its actual size.
There is nothing surprising here (to me at least).
a.) Strings in java consume double the space compared to most common text formats (because Strings are always represented as UTF-16 in the heap). Also, String as an object has quite some overhead (String object itself, reference to the char[] it contains, hashCode etc.). For small strings the String object costs easily as much memory as the data it contains.
b.) You put stuff into a HashMap. HashMap is not exactly memory efficient. First it uses a default load factor of 75%, which means a map with many entries has also a big bucket array. Then, each entry in the map is an object itself, which costs at least two references (key and value) plus object overhead.
In conclusion you pretty much have to expect the memory requirements to increase quite a bit. A factor of 4 is reasonable if your average data String is relatively short.
If you think you cannot afford a ratio 1:4 between the size of data in a flat file and the memory necessary to load the Strings in a HashMap, you should considere not using Java but a lower level language such as C++ or even C.
Of course there are possible optimizations :
use byte[] instead of String (about half the size)
do not use default HashMap parameters (initial size / load factor) but tweak them to meet your actual requirements.
What follows is mainly experience opinion based. I generally use 4 language levels :
high level scripting language (Python, Ruby, or even bash ...) when performance
is not a requirement and speed of developpement is
mid level language (Java, less frequently high level C++) when performance matters but when I also want simplicity of developpement and robustness (strong typing, ...)
low level language (low level C++, or C) what performance is a high requirement and when I accept to spend much more time in writing and testing individual modules
assembly language for the small parts where performance is critical and has been proved to be by profiling.
IMHO you can tweak Java code to highly reduce the memory footprint, but you risk to lose a great part of the interest of Java by losing the excellent string and collections support. It might be as easy and perhaps more efficient to code a small part of the application in C++ and use JNI to tie all together.
When running the following code in Matlab, the physical memory usage of Matlab increases until it reaches 90% of my system memory, and crashes the program. This code is part of a larger optimization scheme that will have many more iterations, but it has been simplified here.
import org.parent.modeling.*
for i = 1:100
rTool = Tool('C:/etc');
model = rTool.getModel();
model.initSystem();
rTool.setModel(model);
rTool.run();
end
rTool and model are stored as Java objects in the workspace.
By selectively removing steps from the bottom to top, I've determined that the rTool.run(); step is causing an incremental memory increase during each iteration.
I've tried using a variety of memory clearing techniques with no success
clear classes, clear java, clear rTool model, clear all
I have checked for fragmentation by running the memory command, but it is insignificant.
Below is the actual code that corresponds to the API documentation (https://simtk.org/api_docs/opensim/api_docs/classOpenSim_1_1RRATool.html)
for i = 1:1:4
import org.opensim.modeling.*
rraTool = RRATool('C:\...filename.xml');
model = rraTool.getModel();
model.initSystem();
rraTool.setModel(model);
rraTool.run();
end
Any suggestions for reducing the physical memory increase?
Matlab: 2013
It is difficult to figure out what you are actually doing here, let alone why it is leaking memory. From what I have been able to deduce, you can call Java from MatLab, and you can call C++ directly from Matlab. I cannot figure out if you are calling the C++ library directly, or calling it via a Java (JNI / JNA) wrapper. And if you are doing the latter ... why.
For the Matlab -> C++ case, this page has some tips on avoiding memory leaks: http://www.mathworks.com.au/help/matlab/matlab_external/memory-management-issues.html
For the Matlab -> Java -> C++ case, the leaks could be due to bugs in the Java wrapper or its associated C++-side wrapper code, or a problem in the way that you are using the wrapper. For instance, there could be some dispose method that you need to call. The details should be in the Java API docs ... which you have not been able to locate ... or in the Java source code itself.
I was hoping that there were some general memory clearing Java commands I could use.
There are no such magic commands. You need to figure out where the memory is leaking and fix that.
If I have a List<Object>, would it be possible to run some method on each Object to see how much memory each is consuming? I know nothing about each Object it may be an entire video file loaded onto the heap or just a two-byte string. I ultimately would like to know which objects to drop first before running out of memory.
I think Runtime.totalMemory() shows the memory currently used by the JVM, but I want to see the memory used by a single object.
SoftReference looks kinda like what you need. Create a list of soft references to your objects, and if those objects are not referenced anywhere and you run out of memory, JVM will delete some of them. I don't know how smart the algorithm for choosing what to delete is, but it could as well be removing those that will free most memory.
If you are in a container you can use Jconsole http://java.sun.com/developer/technicalArticles/J2SE/jconsole.html
The jdk since 1.5 comes with heap dump ulits... You in a container or in eclipse? Also why do you have a List of Objects??
There is no clean way to do it. You can create a dummy OutputStream which will do nothing but counting number of bytes written. So, you can make some estimation about your object graph size by serializing it to such stream.
I would not advise to do it in production system. I, personally, did it once for experimenting and making estimations.
Actually another possible tactic is just to make a crap load of instance of the class you want to check (like a million in an array).
The sheer number of objects should negate the overhead (as in the overhead of other stuff will be much smaller than your crap load of objects).
You will want to run this in isolation of course (ie public static main()).
I will admit you will need lots of memory for this test.
Something you could do is make a Map<Object, Long> which maps each object to it's memory size.
Then to measure the size of a particular object, you have to do it at instantiation of each object - measure the JVM memory use before (calling Runtime.totalMemory()) and after building the object (calling Runtime.totalMemory()) and take the difference between the two - that is the size of the object in memory. Then add the Object and Long to your map. From there you should be able to loop through all of the keys in the map and find the object using the largest amount of space.
I am not sure there is a way to do it per object after you already have your List<Object>... I hope this is helpful!