We Keep Coding

Differences between Java 6 and Java 7 in ArrayList capacity growth

I have a question about how ArrayList's capacity growth (not the size, but the capacity) is managed in Java.
When we initialize an ArrayList using the default constructor without setting a capacity, the capacity is set = 10 by default.
At this point, when we add another element to the list, the Oracle documentation says that "As elements are added to an ArrayList, its capacity grows automatically. The details of the growth policy are not specified beyond the fact that adding an element has constant amortized time cost."
If we look at Java internals the capacity growth policy has changed its function. Until Java 6 it was:
(1) int newCapacity = (oldCapacity * 3)/2 + 1;
From Java 7 (and > 7) it is:
(2) int newCapacity = oldCapacity + (oldCapacity >> 1);
but these two mathematical series are slightly different. Starting from the default value (10) we have:
(1) 10,16,25,38,58,88,133,200,301,452...
(2) 10,15,22,33,49,73,109,163,244,366...
I think that this hasn't any implication on ArrayList use, but why did they change this function? Is there any performance reason? Did they find a flaw or a bug in the old one?

OpenJDK's source control history shows it was changed by Martin Buchholz from Google in changeset 2350 to fix bug JDK-6933217: Huge arrays handled poorly in core libraries.
The new code is careful to avoid unnecessary integer overflow. oldCapacity * 3 can overflow even if oldCapacity * 3 / 2 doesn't. The new line oldCapacity + (oldCapacity >> 1) won't. And if it does overflow and go negative there's additional code to set the capacity to Integer.MAX_VALUE (or near to it).
/**
* The maximum size of array to allocate.
* Some VMs reserve some header words in an array.
* Attempts to allocate larger arrays may result in
* OutOfMemoryError: Requested array size exceeds VM limit
*/
private static final int MAX_ARRAY_SIZE = Integer.MAX_VALUE - 8;
private void grow(int minCapacity) {
// overflow-conscious code
int oldCapacity = elementData.length;
int newCapacity = oldCapacity + (oldCapacity >> 1);
if (newCapacity - minCapacity < 0)
newCapacity = minCapacity;
if (newCapacity - MAX_ARRAY_SIZE > 0)
newCapacity = hugeCapacity(minCapacity);
// minCapacity is usually close to size, so this is a win:
elementData = Arrays.copyOf(elementData, newCapacity);
}
private static int hugeCapacity(int minCapacity) {
if (minCapacity < 0) // overflow
throw new OutOfMemoryError();
return (minCapacity > MAX_ARRAY_SIZE) ?
Integer.MAX_VALUE :
MAX_ARRAY_SIZE;
}
Full details from the bug report:
I've noticed bugs in java.util.ArrayList, java.util.Hashtable and
java.io.ByteArrayOutputStream which arise when the capacities of the
data structures reach a particular threshold. More below.
When the capacity of an ArrayList reaches (2/3)*Integer.MAX_VALUE its
size reaches its capacity and an add or an insert operation is
invoked, the capacity is increased by only one element. Notice that
in the following excerpt from ArrayList.ensureCapacity the new
capacity is set to (3/2) * oldCapacity + 1 unless this value would not
suffice to accommodate the required capacity in which case it is set
to the required capacity. If the current capacity is at least
(2/3)*Integer.MAX_VALUE, then (oldCapacity * 3)/2 + 1 overflows and
resolves to a negative number resulting in the new capacity being set
to the required capacity. The major consequence of this is that each
subsequent add/insert operation results in a full resize of the
ArrayList causing performance to degrade significantly.
int newCapacity = (oldCapacity * 3)/2 + 1;
if (newCapacity < minCapacity)
newCapacity = minCapacity;
...
It is interesting to note that any statements about the amortized time
complexity of add/insert operations, such as the one in the ArrayList
javadoc, are invalidated by the performance related bugs. One
solution to the above situations is to set the new capacity of the
backing array to Integer.MAX_VALUE when the initial size calculation
results in a negative number during a resize.

Difference between Stack.capacity() and Stack.size()

I am currently doing a check with the Stack<E> class to see if it's full. However, List does not have a isFull() implementation, so I am asking to check if capacity() is the same as size(). According to the docs, size() returns the number of components in this vector, and capacity returns the current capacity of the vector. If I understand correctly, are they the same? And if so, how do I go about checking if my Stack<E> is full?

stack.size() - gives the current size i.e., total number of elements pushed to the stack
stack.capacity() - gives the current capacity i.e., array size like 10 or 20 etc... i.e., as soon as you pushes 10 elements to the stack, your stack capacity gets doubled.
Internally Stack uses Vector and Vector is a dynamic growing array.
Also, for a Stack, you can't manually set the capacityIncrement factor, rather the stack itself manages internally, you can look here

The Stack datastructure in Java represents a last-in-first out (LIFO) stack of objects. It extends class Vector with five operation such as
push
pop
peek item at the top of the stack
Check stack is empty and
search for an item in the stack
when the Stack classs would be like as follows
public class Stack extends Vector {
}
When the stack is created it contains no items. Coming to stack capacity and size
Size - Number of elements a stack contains at present
Capacity - Number of elements it is capable of holding
The Push operation is implemented as follows
public E push(E item) {
addElement(item);
return item;
}
addElement method belongs to Vector class which helps to insert a new element into the Vector
public synchronized void addElement(E obj) {
modCount++;
ensureCapacityHelper(elementCount + 1);
elementData[elementCount++] = obj;
}
ensureCapacityHelper allows to check whether the Vector inside is capable of adding a new element or not. If it does not have enough space to hold the new element the Vector grows
private void ensureCapacityHelper(int minCapacity) {
// overflow-conscious code
if (minCapacity - elementData.length > 0)
grow(minCapacity);
}
/**
* The maximum size of array to allocate.
* Some VMs reserve some header words in an array.
* Attempts to allocate larger arrays may result in
* OutOfMemoryError: Requested array size exceeds VM limit
*/
private static final int MAX_ARRAY_SIZE = Integer.MAX_VALUE - 8;
private void grow(int minCapacity) {
// overflow-conscious code
int oldCapacity = elementData.length;
int newCapacity = oldCapacity + ((capacityIncrement > 0) ?
capacityIncrement : oldCapacity);
if (newCapacity - minCapacity < 0)
newCapacity = minCapacity;
if (newCapacity - MAX_ARRAY_SIZE > 0)
newCapacity = hugeCapacity(minCapacity);
elementData = Arrays.copyOf(elementData, newCapacity);
}
Arrays.copyOf is a native method would allocate a new memory space with newCapacity and copies the data from old memory location to new location.

The size is the current number of elements in the stack.
The capacity is an internal detail that tells you the maximum items that would fit in the Vector. However, this is not really relevant as it will expand automatically when the capacity is reached.

Load factor of Arraylist and Vector?

Hi I was trying to find the load factor of Array list and vector but I was not able to find it. I know load factor of HashMap and other Map is 0.75. Can any one help to find me how to check the load factor of Vector and Arraylist.

ArrayList:
Initial Capacity:10
Load Factor:1 (when the list is full)
Growth Rate: current_size + current_size/2
Vector:
Initial Capacity:10
Load Factor:1 (when the list is full)
Growth Rate:
current_size * 2 (if capacityIncrement is not defined)
current_size + capacityIncrement (if capacityIncrement is defined during vector initialization)

I assume you would like to know how ArrayList and Vector increase its size.
For ArrayList, every time you put an element into it, it will check if the nested array needs to be enlarge its size. If yes, generally, its size will grow with:
newCapacity = oldCapacity + (oldCapacity >> 1);
For some special case, for example, add many or huge number of elements, things will be different. Please refer grow(int minCapacity) function in java.util.ArrayList source code.
Regarding Vector, generally, its size will grow with:
newCapacity = oldCapacity + ((capacityIncrement > 0) ?
capacityIncrement : oldCapacity);
For some special cases, please refer grow(int minCapacity) in java.util.Vector.

ArrayList al = new ArrayList();
for(int i=0; i<=10; i++){
al.add(i+1);
}
default capacity = 10
in the above example, we want to add 11 elements so new Capacity of ArrayList is
int newCapacity = (oldcapacity*3)/2+1
(10*3)/2+1 = 16

How to increase ArrayList size to 100% just like Vector

While changing some code based on SonarQube suggestions I get to know below lines:
Automatic Increase in Capacity A Vector defaults to doubling size of its array . While when you insert an element into the ArrayList ,it
increases its Array size by 50%.
Now I am wondering if I need to replace the Vector with ArrayList there is a chance of failure of normal execution of the code.
Remember existing Vector is not doing any Thead-safe work.
Question:
Is ArrayList capable enough to resize just like vector?
Is it safe to replace the Vector with ArrayList in any condition except Synchronization??
Is there any exact replacement of Vector (Not expecting the Thread-safety)
Please feel free to update the question or ask anything.

The differences between Vector and ArrayList are some like this:
Vector is synchronized while ArrayList is not synchronized. So, Vector is thread safe.
Vector is slow as it is thread safe . In comparison ArrayList is fast as it is non-synchronized.
A Vector grows as doubling size of its array in default. While when you insert an element into the ArrayList, it increases its Array size by 50% .
ArrayList:
/**
* Increases the capacity to ensure that it can hold at least the
* number of elements specified by the minimum capacity argument.
*
* #param minCapacity the desired minimum capacity
*/
private void grow(int minCapacity) {
// overflow-conscious code
int oldCapacity = elementData.length;
int newCapacity = oldCapacity + (oldCapacity >> 1); // 50%
if (newCapacity - minCapacity < 0)
newCapacity = minCapacity;
if (newCapacity - MAX_ARRAY_SIZE > 0)
newCapacity = hugeCapacity(minCapacity);
// minCapacity is usually close to size, so this is a win:
elementData = Arrays.copyOf(elementData, newCapacity);
}
Vector:
private void grow(int minCapacity) {
// overflow-conscious code
int oldCapacity = elementData.length;
int newCapacity = oldCapacity + ((capacityIncrement > 0) ?
capacityIncrement : oldCapacity); // default 100%
if (newCapacity - minCapacity < 0)
newCapacity = minCapacity;
if (newCapacity - MAX_ARRAY_SIZE > 0)
newCapacity = hugeCapacity(minCapacity);
elementData = Arrays.copyOf(elementData, newCapacity);
}
ArrayList does not define the increment size . Vector defines the increment size .
/**
* The amount by which the capacity of the vector is automatically
* incremented when its size becomes greater than its capacity. If
* the capacity increment is less than or equal to zero, the capacity
* of the vector is doubled each time it needs to grow.
*
* #serial
*/
protected int capacityIncrement;
Based above:
ArrayList capable can not resize just like Vector.
ArrayList is not thread safe. It can not replace Vector by ArrayList in multiple threads directly.
It can replace Vector by ArrayList in single thread mostly. Because The declaration of Vector and ArrayList:
public class Vector<E>
extends AbstractList<E>
implements List<E>, RandomAccess, Cloneable, java.io.Serializable
public class ArrayList<E>
extends AbstractList<E>
implements List<E>, RandomAccess, Cloneable, java.io.Serializable

I don’t see a problem. The exact performance metrics of Vector and ArrayList are not the same, but for most practical purposes this is not important. The ArrayList will extend whenever needed, and more often than Vector (if you don’t tell it the needed capacity beforehand). Go ahead.
For your questions: 1. Yes 2. Yes 3. No

Java Hashmap Internal

I have few doubts about the Java HashMap class. It is my understanding that
transient Entry[] table;
the table array is going to hold the data based on the value of hashCode(). I need to know when this array gets initialized. Is the array length based on the capacity we define during the HashMap's initialization or the default capacity of 16 if it is not defined when calling the constructor?
How is the hashcode scaled to the array index? For example, if the hashcode has a huge value, how it is scaled to array index like 10, 20?
I have read that when the threshold value is reached, rehashing will occur. For example, in the default case, when 16 is the capacity and 0.75 is the load factor, then the threshold value is 16*0.75=12. Once the 12 items are added rehashing will occur and capacity will increase. Does this mean that the table array size gets increased?

since your post has many questions I'm going to enumerate your questions as part of my answer. Also, please note that I'm going off HashMap's source code for Java 1.8 b132 for my answers.
Q: I need to know when this array gets initialized.
A: The table array only gets initialized when data is first entered into the map (e.g. a put() method call). It does not happen as part of the instantiation of the map, itself, unless the copy constructor is called, or the map is being deserialized into an object.
Q: Is the array length based on the capacity we define during the HashMap's initialization or the default capacity of 16 if it is not defined when calling the constructor?
A: Correct, the table array's length is based on the initial capacity your pass to the constructor. When the initial capacity is not specified and the default constructor is called, the default capacity is used.
Q: How is the hashcode scaled to the array index?
A: For the actual code that does this, itself, see the implementation of the putVal() method. Basically what happens is that the code takes the very large hash value and performs a bitwise-AND with the last element index of the table. That effectively randomizes the position of the key/value pair with the table array. For example, if the hash value is 333 (101001101 in base 2) and the table array size is 32 (100000), the last element index would be 31 (11111). Thus the index chosen would be 11111 & 101001101 == 01101 == 13.
Q: I have read that when the threshold value is reached, rehashing will occur. ... Does this mean that the table array size gets increased?
A: More or less, yes. When the threshold is exceeded, the table is resized. Note that by resizing, the existing table array isn't modified. Rather, a new table array is created with the twice the capacity of the first table array. For details, see the implementation of the resize() method.

public HashMap(int initialCapacity, float loadFactor) {
if (initialCapacity < 0)
throw new IllegalArgumentException("Illegal initial capacity: " +
initialCapacity);
if (initialCapacity > MAXIMUM_CAPACITY)
initialCapacity = MAXIMUM_CAPACITY;
if (loadFactor <= 0 || Float.isNaN(loadFactor))
throw new IllegalArgumentException("Illegal load factor: " +
loadFactor);
// Find a power of 2 >= initialCapacity
int capacity = 1;
while (capacity < initialCapacity)
capacity <<= 1;
this.loadFactor = loadFactor;
threshold = (int)(capacity * loadFactor);
table = new Entry[capacity];
init();
}
Above code block explains how and when you populate the table.
Once the rehashing occurs it doesn't increase the table array size as you can declare array size once for ever; It creates a new array every time with the updated size:
void resize(int newCapacity) {
Entry[] oldTable = table;
int oldCapacity = oldTable.length;
if (oldCapacity == MAXIMUM_CAPACITY) {
threshold = Integer.MAX_VALUE;
return;
}
Entry[] newTable = new Entry[newCapacity];
transfer(newTable);
table = newTable;
threshold = (int)(newCapacity * loadFactor);
}