I have a similar code:
ArrayList<HashMap<String, String>> myArray = new ArrayList<HashMap<String, String>>();
for(int i=0; i<8000; i++){
HashMap <String, String> hashMap = new HashMap<String, String>();
hashMap.put("key1", string1);
hashMap.put("key2", string2);
myArray.add(hashMap);
}
Sometimes happen that in older Android device, this code leads to OutOfMemory on new HashMap.
There is a way to improve this code?
Thanks
EDIT:
I have this structure in my Application Class to retrive this array around the app and do something like that:
ArrayList<String> allObj1 = new ArrayList<String>();
ArrayList<String> allObj2 = new ArrayList<String>();
for (int i = 0; i<myArray.size(); i++) {
String obj1 = myArray.get(i).get("key1");
String obj2 = myArray.get(i).get("key2");
allObj1.add(obj1);
allObj2.add(obj2);
}
String[] stringObj1 = allObj1.toArray(new String[allObj1.size()]);
String[] stringObj2 = allObj2.toArray(new String[allObj2.size()]);
list.setAdapter(new Adapter(this, stringObj1, stringObj2));
You can improve your code a bit, I'll write two solutions, the first is better, but if you can't use it, use the second one:
In both solutions, use a constructor with initial capacity.
Use SparseArray if you can change your keys in int values:
ArrayList<SparseArray<String>> myArray = new ArrayList<>(8000);
for(int i=0; i<8000; i++) {
SparseArray<String> sp = new SparseArray<>(2);
sp.put(1, string1);
sp.put(2, string2);
myArray.add(sp);
}
Use ArrayMap instead:
ArrayList<ArrayMap<String, String>> myArray = new ArrayList<>(8000);
for (int i = 0; i < 8000; i++) {
ArrayMap<String, String> am = new ArrayMap<>(2);
am.put("key1", string1);
am.put("key2", string2);
myArray.add(am);
}
my answer is coming from here
why not to create an object that holds your properties
like this
class A{
String key1;
String key2;
}
ArrayList<A> myArray = new ArrayList<A>();
for(int i=0; i<8000; i++) {
A a=new A();
a.key1=string1;
a.key2=string2;
myArray.Add(a);
}
what i'm trying to say here that the hashmap object has an overhead that can be shrinked using an object
You code should be changed to:
String[] stringObj1 = new String[myArray.size()]);
String[] stringObj2 = new String[myArray.size()]);
for (int i = 0; i < myArray.size(); i++) {
stringObj1[i] = myArray.get(i).get("key1");
stringObj2[i] = myArray.get(i).get("key2");
}
list.setAdapter(new Adapter(this, stringObj1, stringObj2));
This prevents the intermediate lists and saves memory. The copy operation does not start if the memory for the two arrays is not available.
myArray does not seems to be an array but a list. :o
What You have done is good enough. the only concern is the device doesnt have that much memory to allocate for new hashmap object every time.
When you start JVM you define how much RAM it can use use for processing. JVM divides this into certain memory locations for its processing purpose, two of those are Stack & Heap
OutOfMemoryError is related to Heap. If you have large objects (or) referenced objects in memeory, then you will see OutofMemoryError. If you have strong references to objects, then GC can't clean the memory space allocated for that object. When JVM tries to allocate memory for new object and not enough space available it throws OutofMemoryError because it can't allocate required amount of memory.
How to avoid: Make sure un-necessary objects are available for GC
StackOverflowError is related to stack. All your local variables and methods calls related data will be on stack. For every method call one stack frame will be created and local as well as method call related data will be placed inside the stack frame. Once method execution is completed, stack frame will be removed. ONE WAY to reproduce this is, have infinite loop for method call, you will see stackoverflow error, because stack frame will be populated with method data for every call but it won't be freed (removed).
How to avoid Make sure method calls are ending (not in infinite loop)
ArrayList<HashMap<String, String>> myArray = new ArrayList<HashMap<String, String>>();
HashMap <String, String> hashMap = new HashMap<String, String>();
for(int i=0; i<8000; i++) {
hashMap.put("key1", string1);
hashMap.put("key2", string2);
myArray.add(hashMap);
}
Do like this declare hashMap outside for loop. you are creating object again and again in memory pool.
Related
first of all thanks for the help.
I'm aware of the Reference passing mechanism of java and I need to read one million of lines (a word + a_list_of_integers each line) from a text file and put them in some structures that are class attributes, one hashmap and two arraylist.
The problem is that with the code below, written to save memory reusing the list "termine_frequenza", when I try to get and element from the "frequency" arraylist or the "dictionaryMarTD" hashmap, the list that returns is always the last list that I added.
Adding the declaration of the "Arraylist termine_frequenza" into the While obviously solves the problem but I receive a prevedible "GC overhead limit exceeded" error because of multiple declaration (i tried to increase heap o disable it, but GC fills the cpu capacity trying to free memory.
The question is simple: how can I save memory and at the same time have a correct reading? Thanks.
//Class attributes
private HashMap<String, ArrayList> dictionaryMapTD;
private ArrayList<String> words;
private ArrayList<ArrayList> frequency;
//This is the code of a method of the class that reads from a file
br = new BufferedReader(new FileReader("dictionary.txt"));
s = br.readLine();
String[] splitted;
ArrayList<Integer> termine_frequenza = new ArrayList<>();
while(s!=null)
{
termine_frequenza.clear();
splitted = s.split(" ");
words.add(splitted[0]);
for (int i = 1; i < splitted.length; i++)
{
termine_frequenza.add(Integer.valueOf(splitted[i]));
}
frequency.add(termine_frequenza);
dictionaryMapTD.put(splitted[0], termine_frequenza);
s = br.readLine();
}
//END
Change your XMS/XMX parameters in your eclips.ini file.
I set it -Xms256m-Xmx7024m for 3000000
If it has no effect then try to modify that parameters for application
In your eclips go to
RunConfigurations->Arguments->VM Arguments
for your application and put
-Xms256m
-Xmx7024m
Then in your code move
termine_frequenza = new ArrayList<>();
inside while and remove
termine_frequenza.clear();
GC should not complain
In my case It runs for 7000000 records
Let me know if it helps
I have a number of repetitions of a task I would like to put in a for loop. I have to store a time series object as an IExchangeItem, a special class in openDA (a data assimilation software).
This is one of the tasks (that works):
HashMap<String, TimeSeries> items = new LinkedHashMap<String, TimeSeries>();
...
TimeSeries tsc1Q = new TimeSeries(time,value);
id = "Q1";
tsc1Q.setId(id);
this.items.put(id,tsc1Q);
IExchangeItem c1Q = new TimeSeries(tsc1Q);
What changes across the tasks is the id of the time series object and the name of IExchangeItem. I have to create a new IExchangeItem object for each time series.
This is what I tried in the for loop:
HashMap<String, TimeSeries> items = new LinkedHashMap<String, TimeSeries>();
...
TimeSeries temp;
for (int i = 0; i<readDataDim[0]; i++) {
value[0] = values[i];
id = exchangeItemIDs[i];
temp = new TimeSeries(time,value);
temp.setId(id);
this.items.put(id,temp);
IExchangeItem <??> = new TimeSeries(temp); //* How can I handle this line?
}
I know I cannot use dynamic variable names in java and that arrays, lists, or maps are commonly used to work around this issue (this is why I used <??> in the code snippet above. However, I'm a relative beginner with java and I have no clue how I can work around this specific problem since I have to have a new invocation of IExchangeItem for each time series.
From here I take it that my IExchangeItem created in the for loop will not be accessible outside the for loop so how can I initialise n replicates of IExchangeItem outside the for loop?
Edit:
Does a HashMap create n instances of IExchangeItem if I try something like this?
HashMap<String,IExchangeItem> list = new LinkedHashMap<String,IExchangeItem>();
Just one suggestion, try to write a separate method when you can pass the size of the array or a fixed number (based on array), then you created a hashMap and add that many number of instances with its keys, and values, cannot post this as a comment and hence posting it as an answer.
Try to create a new method using the value of readDataDim[0] value,
public Map<String, IExchangeItem> createAndInitialzeMap(int maxValue) {
Map<String, IExchangeItem> map = new HashMap<>();
String temp = "tempName";
for(int i =0; i < maxValue ; i ++ ) {
map.put(temp+i, new IExchangeItem());
}
return map;
}
return this way you can initialize your map along with its variable name and you can use it in your app anywhere. However I would consider refactoring if such code exists and time permits.
One more thing you should read about hashMap. :) :)
So I've been running into a few errors. When I try to add an event to a date in a month, I try to add an event into the date ArrayList but the list doesn't have that number available. So I want to know how to instantiate a list that will have the amount of days in it.
I know you can instantiate how many months there are by using
public static List<List<String>> events = new ArrayList<List<String>>(12);
which 12 being the number of months, but is there a way to make all 12 of the arrays have 31 slots open for days?
I'd recommend completely changing your design.
Use a map of date against event list - to deal with multiple events on same date
Lazy initialize map. Create a method that returns list for given day/month, if it isn't there, add it.
Code
private class Events {
Map<Date, List<String>> events = new HashMap<Date, List<String>>();
public List<String> getEvents(int month, int day) {
Calendar calendar = GregorianCalendar.getInstance();
calendar.set(0, month, date, 0, 0, 0);
Date date = calendar.getTime();
List<String> list = events.get(date);
if (list == null) {
list = new ArrayList<String>();
events.put(date, list);
}
return list;
}
}
Usage
Events events = new Events();
events.getEvents(11, 25).add("Christmas");
events.getEvents(0, 1).add("New years day");
I think you should change your design plan. Make a class Month with an ArrayLisy<String>(31) which you create 12 times in another class called year or something. For myself I would do this and it makes your code more readable!
Also, when it is a fixed number of slots (and you don't want it to go over that limit), use basic arrays
Why not use
List<Map<String, Integer>> = new List<HashMap<String, Integer>>();
While #Adam's answer is probably the best way to handle this particular problem, the general question of how to initialize a multidimensional list structure should also be addressed.
The standard idiom for constructing and initializing a multi-dimensional data structure like the one you describe is to use a set of for loops, like this:
int[][] myArray = new int[][myFirstDimension];
for(int i=0; i<a.length; i++){
int [] a = new int[mySecondDimension];
for(int j=0; j<b.length; j++){
a[j] = myInitialValue;
}
myArray[i] = a;
}
Nested ArrayLists are constructed in a similar way to arrays, but with one important difference: The parameter passed to the ArrayList is only an initial capacity, not the initial size. In fact, ArrayList actually makes no guarantees about how much space it allocates, although in practice, IIRC, it typically allocates the next power of two. Attempting to index into the ArrayList before you have initialized its contents will throw an IndexOutOfBoundsException.
Thus, when initializing nested ArrayLists, we typically do it like this:
List<List<String>> myList = new ArrayList<>();
for(int i=0; i<myFirstDimension; i++){
List<String> a = new ArrayList<>();
for(int i=0; i<mySecondDimension; i++){
a.add(""); // or `null`, if that is needed
}
myList.add(a);
}
I think doing arraylist1=arraylist2 makes the 2 of them share the same memory. How can I copy an arraylist without them doing that? I want to treat them sepparately.
List<Integer> rez = new ArrayList<>();
List<Integer> rezc = new ArrayList<>();
rez.add(1);
rezc=rez;
rezc.add(2);
for (int s : rez) {
System.out.print(s + " ");
}//this will print 1 2
I think doing arraylist1=arraylist2 makes the 2 of them share the same memory.
Not quite, it makes both of those references refer to the same, single, object.
How can I copy an arraylist without them doing that?
Lots of options:
ArrayList has a copy constructor:
List<Integer> rezc = new ArrayList<>(rez);
List has an addAll method:
List<Integer> rezc = new ArrayList<>();
rezc.addAll(rez);
ArrayList has a clone method, but it's a bit ugly to use if rez is declared as a List because you have to assume it's an ArrayList and cast it, which is probably not a great idea:
List<Integer> rezc = (List<Integer>)((ArrayList<Integer>)rez).clone();
It's well worth reading through the JavaDoc when trying to figure things like this out.
The statement arraylist1=arraylist2 means they are referring to same ArrayList object. Reference variables arraylist1 and arraylist2 are referring to same object and hence, the changes done by arraylist1 will be seen when you are trying to access the object by arraylist2
If you want to make a new ArrayList then, ArrayList rezc = new ArrayList(rez)
Instead of this line rezc=rez;
Use List<Integer> rezc = new ArrayList<>(rez);
The long hand way is a for loop to cycle though one list while adding all the items to the second. Something like
for (int I = 0; I < rez.size() I++) {
rezc.add(rez.get(I)); }
But the previous answers are much more efficient.
If I understand correctly you are talking about Java shallow cloning v/s deep cloning. In this case the below code might help
List<Integer> rez = new ArrayList<>();
List<Integer> rezc = new ArrayList<>();
rez.add(1);
rezc.addAll(rez); // addAll
List<Integer> rezc2 = (List<Integer>)((ArrayList<Integer>)rez).clone(); //clone
The Java code is as follows:
Random r = new Random(1234697890);
HashMap<Integer, List<Integer>> map = new HashMap<Integer, List<Integer>>();
List<Integer> list = new ArrayList<Integer>();
for(int i=0;i<100000;i++){
for(int j=0;j<1000;j++){
list.add(r.nextInt(100000));
}
map.put(i, list);
map.remove(i);
}
when i reaches 37553 , java.lang.OutOfMemoryError: Java heap space happens.
It seems that garbage collection does not happen in the loop.
Now I wonder how to fix the problem.
Try rewriting the code as follows and you should not get OOME's ...
Random r = new Random(1234697890);
HashMap<Integer, List<Integer>> map = new HashMap<Integer, List<Integer>>();
for(int i=0;i<100000;i++){
List<Integer> list = new ArrayList<Integer>();
for(int j=0;j<1000;j++){
list.add(r.nextInt(100000));
}
map.put(i, list);
map.remove(i);
}
The problem with your original code is that:
you only create one list,
you keep adding more and more elements to it, and
that list only becomes garbage when the code completes ... because it is "in scope" the whole time.
Moving the list declaration inside the loop means that a new ArrayList is created and filled in each loop iteration, and becomes garbage when you start the next iteration.
Someone suggested calling System.gc(). It won't help at all in your case because there is minimal1 garbage to be collected. And in general it is a bad idea because:
the GC is guaranteed to have run immediately before an OOME is thrown,
the JVM can figure out better than you can when is the best (i.e. most efficient) time to run the GC,
your call to System.gc() may be totally ignored anyway. The JVM can be configured so that calls to System.gc() are ignored.
1 - The pedant in me would like to point out that map.put(i, list); map.remove(i); is most likely generating an Integer object that most likely becomes garbage. However, this is "chicken feed" compared to your indefinitely growing ArrayList object.
You use the same List all the time, which contains 100000 * 1000 items when the loop exits. To enable GC to get rid of your list, you need to reduce its scope to within the for(i) loop.
In other words, both map and list are reachable at all time in that piece of code and are therefore not eligible for collection.
In your case you keep filling the same list (even though you remove it from that HashMap, it still exists as a local variable).
The JVM promises to do a full garbage collect before throwing an OutOfMemoryError. So you can be sure there is nothing left to clean up.
Your code
List<Integer> list = new ArrayList<Integer>();
for(int i=0;i<100000;i++){
for(int j=0;j<1000;j++){
list.add(r.nextInt(100000));
}
map.put(i, list);
map.remove(i);
}
is the same as
List<Integer> list = new ArrayList<Integer>();
for(int i=0;i<100000 * 1000; i++) {
list.add(r.nextInt(100000));
}
As you can see its the List, not the map which is retaining all the integers. BTW Try this instead and see what happens ;)
list.add(r.nextInt(128));