We Keep Coding

My collection's maximum size is not sufficient for computation, what can I do about it?

Working with ArrayList, its size grows exponentially with each iteration and, over time, the collection becomes too small to continue the computation.
With each iteration, you need to change each element and, according to some condition, add more new elements. My code worked for 100 iterations, but now it needs 256 and it can't handle it anymore.
It turns out a very large flow of calculations, size exceeds int and I need to continue computing in long, but ArrayList cant do it due to size restrictions.
I guess that at a certain stage I need to create new ArrayList's, also iterate over each of them multithreaded, adding new data to the new one, without hitting the limit of ArrayList's sizes.
How can I realize that? Initial size of my ArrayList is 300, each value is a number from 0 to 5.
Here is the code for my methods:
public static void main(String...args) {
ArrayList<Integer> old = new ArrayList<>();
//here is filling my array with 300 elements, too much numbers, don't want to dirty the question with them
//now I call the killer method, the capacity of which is no longer enough
calculateNew(256, old); //it was worked for 100 iterarions, but now i need 256 and he cant do it
old.size(); //accordingly, I need to calculate this value after 256 iterations
}
static void calculateNew(int days, ArrayList<Integer> old){
for(int i = 0; i < days; i++){
oneDayNew(old);
System.out.println("Day " + (i+1<10? i+1 + " " : i+1) + ": size of school: " + old.size());
}
}
static void oneDayNew(ArrayList<Integer> old){
int countNew = 0;
for(int i = 0; i < old.size(); i++){
if(old.get(i) == 0) {
countNew++;
old.set(i, 7);
}
if(old.get(i) > 0) {
int temp = old.get(i);
old.set(i, --temp);
}
}
if(countNew > 0 ){
for(int i = 0; i < countNew; i++){
old.add(8);
}
}
}
What can I do? Multi-threading? Array of ArrayLists? I'm looking for hints, not a ready-made solution.

The size of school after 256 iterations is over 350 billion (starting with 300 random elements sized 0-8). Even if every number was 1 byte, which it isn't, it would need over 350 GB of memory. As it stands, it needs 10* that. Therefore, your approach unfortunately does not scale. This is common for brute force algorithms. You do not want to store every number separately, and you definitely do not want to process them one by one.
Change your data structure.
For example, you can group the numbers together and only store how many of each numbers you have, using a long[] as a histogram. That way, histogram[0] would store how many zeros you have, histogram[1] would store how many ones you have, etc. all the way up to your max value. Of course, a Map<Integer, Long> would work just as well for the purpose. That way, you only use constant memory, and you process much fewer numbers per iteration as they are grouped together.

What is the Default ArrayList size in Java 8 [duplicate]

This question already has answers here:
Distinction between the capacity of an array list and the size of an array
(7 answers)
Closed 5 years ago.
What is the default size for an arraylist in java 8? As far as I remember and searched on google, everywhere I am finding that it is 10 by default but when I am taking output of the below code, it is printing 0. Is there any changes happened in java 8 related to this?
ArrayList<String> arrList = new ArrayList<String>();
System.out.println("size:"+arrList.size());
My second query is: when I add some element to the list, then will the size increase by 1.5 times of previous size or it will increase by only 1?
PS: Is there any way to know the current capacity of the arraylist object?

The initial List has the capacity zero (unless you specify otherwise):
private static final Object[] DEFAULTCAPACITY_EMPTY_ELEMENTDATA = {};
When it has to grow, this is used:
int newCapacity = oldCapacity + (oldCapacity >> 1)
oldCapacity >> 1 is division by two, so it grows by 1.5
The one thing to notice that once you remove elements, the capacity does not change. If you need it to shrink use trimToSize.
The 10 is right actually, but only when you put an element into the ArrayList:
ArrayList<String> ar = new ArrayList<>();
Field f = ar.getClass().getDeclaredField("elementData");
f.setAccessible(true);
int l = ((Object[]) f.get(ar)).length;
System.out.println(l); // 0
ar.add("1");
int x = ((Object[]) f.get(ar)).length;
System.out.println(x); // 10

I think you are confusing size with capacity. Size is the number of elements you have placed into the arrayList while capacity is the max number of elements the arrayList can take. Once you've reached max, the capacity is doubled(if i remember correctly).
So when you add an element into your arrayList, size will always increase by one. If size is equal to capacity, then capacity will increase.

How to increase ArrayList size to 100% just like Vector

While changing some code based on SonarQube suggestions I get to know below lines:
Automatic Increase in Capacity A Vector defaults to doubling size of its array . While when you insert an element into the ArrayList ,it
increases its Array size by 50%.
Now I am wondering if I need to replace the Vector with ArrayList there is a chance of failure of normal execution of the code.
Remember existing Vector is not doing any Thead-safe work.
Question:
Is ArrayList capable enough to resize just like vector?
Is it safe to replace the Vector with ArrayList in any condition except Synchronization??
Is there any exact replacement of Vector (Not expecting the Thread-safety)
Please feel free to update the question or ask anything.

The differences between Vector and ArrayList are some like this:
Vector is synchronized while ArrayList is not synchronized. So, Vector is thread safe.
Vector is slow as it is thread safe . In comparison ArrayList is fast as it is non-synchronized.
A Vector grows as doubling size of its array in default. While when you insert an element into the ArrayList, it increases its Array size by 50% .
ArrayList:
/**
* Increases the capacity to ensure that it can hold at least the
* number of elements specified by the minimum capacity argument.
*
* #param minCapacity the desired minimum capacity
*/
private void grow(int minCapacity) {
// overflow-conscious code
int oldCapacity = elementData.length;
int newCapacity = oldCapacity + (oldCapacity >> 1); // 50%
if (newCapacity - minCapacity < 0)
newCapacity = minCapacity;
if (newCapacity - MAX_ARRAY_SIZE > 0)
newCapacity = hugeCapacity(minCapacity);
// minCapacity is usually close to size, so this is a win:
elementData = Arrays.copyOf(elementData, newCapacity);
}
Vector:
private void grow(int minCapacity) {
// overflow-conscious code
int oldCapacity = elementData.length;
int newCapacity = oldCapacity + ((capacityIncrement > 0) ?
capacityIncrement : oldCapacity); // default 100%
if (newCapacity - minCapacity < 0)
newCapacity = minCapacity;
if (newCapacity - MAX_ARRAY_SIZE > 0)
newCapacity = hugeCapacity(minCapacity);
elementData = Arrays.copyOf(elementData, newCapacity);
}
ArrayList does not define the increment size . Vector defines the increment size .
/**
* The amount by which the capacity of the vector is automatically
* incremented when its size becomes greater than its capacity. If
* the capacity increment is less than or equal to zero, the capacity
* of the vector is doubled each time it needs to grow.
*
* #serial
*/
protected int capacityIncrement;
Based above:
ArrayList capable can not resize just like Vector.
ArrayList is not thread safe. It can not replace Vector by ArrayList in multiple threads directly.
It can replace Vector by ArrayList in single thread mostly. Because The declaration of Vector and ArrayList:
public class Vector<E>
extends AbstractList<E>
implements List<E>, RandomAccess, Cloneable, java.io.Serializable
public class ArrayList<E>
extends AbstractList<E>
implements List<E>, RandomAccess, Cloneable, java.io.Serializable

I don’t see a problem. The exact performance metrics of Vector and ArrayList are not the same, but for most practical purposes this is not important. The ArrayList will extend whenever needed, and more often than Vector (if you don’t tell it the needed capacity beforehand). Go ahead.
For your questions: 1. Yes 2. Yes 3. No

Java Hashmap Internal

I have few doubts about the Java HashMap class. It is my understanding that
transient Entry[] table;
the table array is going to hold the data based on the value of hashCode(). I need to know when this array gets initialized. Is the array length based on the capacity we define during the HashMap's initialization or the default capacity of 16 if it is not defined when calling the constructor?
How is the hashcode scaled to the array index? For example, if the hashcode has a huge value, how it is scaled to array index like 10, 20?
I have read that when the threshold value is reached, rehashing will occur. For example, in the default case, when 16 is the capacity and 0.75 is the load factor, then the threshold value is 16*0.75=12. Once the 12 items are added rehashing will occur and capacity will increase. Does this mean that the table array size gets increased?

since your post has many questions I'm going to enumerate your questions as part of my answer. Also, please note that I'm going off HashMap's source code for Java 1.8 b132 for my answers.
Q: I need to know when this array gets initialized.
A: The table array only gets initialized when data is first entered into the map (e.g. a put() method call). It does not happen as part of the instantiation of the map, itself, unless the copy constructor is called, or the map is being deserialized into an object.
Q: Is the array length based on the capacity we define during the HashMap's initialization or the default capacity of 16 if it is not defined when calling the constructor?
A: Correct, the table array's length is based on the initial capacity your pass to the constructor. When the initial capacity is not specified and the default constructor is called, the default capacity is used.
Q: How is the hashcode scaled to the array index?
A: For the actual code that does this, itself, see the implementation of the putVal() method. Basically what happens is that the code takes the very large hash value and performs a bitwise-AND with the last element index of the table. That effectively randomizes the position of the key/value pair with the table array. For example, if the hash value is 333 (101001101 in base 2) and the table array size is 32 (100000), the last element index would be 31 (11111). Thus the index chosen would be 11111 & 101001101 == 01101 == 13.
Q: I have read that when the threshold value is reached, rehashing will occur. ... Does this mean that the table array size gets increased?
A: More or less, yes. When the threshold is exceeded, the table is resized. Note that by resizing, the existing table array isn't modified. Rather, a new table array is created with the twice the capacity of the first table array. For details, see the implementation of the resize() method.

public HashMap(int initialCapacity, float loadFactor) {
if (initialCapacity < 0)
throw new IllegalArgumentException("Illegal initial capacity: " +
initialCapacity);
if (initialCapacity > MAXIMUM_CAPACITY)
initialCapacity = MAXIMUM_CAPACITY;
if (loadFactor <= 0 || Float.isNaN(loadFactor))
throw new IllegalArgumentException("Illegal load factor: " +
loadFactor);
// Find a power of 2 >= initialCapacity
int capacity = 1;
while (capacity < initialCapacity)
capacity <<= 1;
this.loadFactor = loadFactor;
threshold = (int)(capacity * loadFactor);
table = new Entry[capacity];
init();
}
Above code block explains how and when you populate the table.
Once the rehashing occurs it doesn't increase the table array size as you can declare array size once for ever; It creates a new array every time with the updated size:
void resize(int newCapacity) {
Entry[] oldTable = table;
int oldCapacity = oldTable.length;
if (oldCapacity == MAXIMUM_CAPACITY) {
threshold = Integer.MAX_VALUE;
return;
}
Entry[] newTable = new Entry[newCapacity];
transfer(newTable);
table = newTable;
threshold = (int)(newCapacity * loadFactor);
}

Queue has more items that I put in it

In my Java program, I initialized a Queue with all numbers from 0 to 1000.
emptyFrames = new PriorityQueue<Integer>();
for (int i = 0; i < 1000; i++) {
emptyFrames.add(i);
}
System.out.println("Debug");
However, when I go in to debug, there are 1155 items in the Queue.
Why is this happening?

The indices greater than 1000 are related to the queue's capacity, rather than its size.
Internally, PriorityQueue is backed by an array of objects. When adding objects to a queue with a full backing array, the queue will expand the array by a moderate amount by calling grow, so that it will have internal space (capacity) available for future add calls. This avoids the queue having to expand its array every time add is called, which would be horribly inefficient.
private void grow(int minCapacity) {
int oldCapacity = queue.length;
// Double size if small; else grow by 50%
int newCapacity = oldCapacity + ((oldCapacity < 64) ?
(oldCapacity + 2) :
(oldCapacity >> 1));
// overflow-conscious code
if (newCapacity - MAX_ARRAY_SIZE > 0)
newCapacity = hugeCapacity(minCapacity);
queue = Arrays.copyOf(queue, newCapacity);
}
Code retrieved from Docjar.

The PriorityQueue internally resizes itself according to its capacity to hold more elements. This is a common feature of collections.
From Java:
A priority queue is unbounded, but has an internal capacity governing the size of an array used to store the elements on the queue. It is always at least as large as the queue size. As elements are added to a priority queue, its capacity grows automatically. The details of the growth policy are not specified.

You're looking at two different pieces of information.
First, the formal size of your queue is 1,000 - there are only 1,000 elements in it. You can verify this with emptyFrames.size().
Second, it appears that Eclipse is showing you the backing array, which is not a good reflection of the total number of elements currently present in the queue. That array's size will fluctuate based on its internal resizing rules.
In this scenario, the backing array isn't something you should trust; only inspect the size() of the collection instead.