We Keep Coding

Purpose of new keyword in creating array in Java

I want to know why an array created in Java static even when we use the new keyword to define it.
From what I've read, the new keyword allocates a memory space in the heap whenever it is encountered during run time, so why give the size of the array at all during definition.
e.g. Why can't
int[] array1=new int[20];
simply be:
int[] array1=new int[];
I know that it does not grow automatically and we have ArrayList for that but then what is the use of keyword new in this? It could have been defined as int array1[20]; like we used to do it in C, C++ if it has to be static.
P.S. I know this is an amateurish question but I am an amateur, I tried to Google but couldn't find anything comprehensive.

This may be an amateurish question, but it is one of the best amateurish questions you could make.
In order for java to allow you to declare arrays without new, it would have to support an additional kind of data type, which would behave like a primitive in the sense that it would not require allocation, but it would be very much unlike a primitive in the sense that it would be of variable size. That would have immensely complicated the compiler and the JVM.
The approach taken by java is to provide the bare minimum and sufficient primitives in order to be able to get most things done efficiently, and let everything else be done using objects. That's why arrays are objects.
Also, you might be a bit confused about the meaning of "static" here. In C, "static" means "of file scope", that is, not visible by other object files. In C++ and in Java, "static" means "belongs to the class" rather than "belongs to instances of the class". So, the term "static" is not suitable for describing array allocation. "Fixed size" or "fixed, predefined size" would be more suitable terms.

Well, in Java everything is an object, including arrays (they have length and other data). Thats why you cannot use
int var[20];
In java that would be an int and the compiler would be confused. Instead by using this:
int[] var;
You are declaring that var is of type int[] (int array) so Java understands it.
Also in java the length of the array and other data are saved on the array, for this reason you don't have to declare size of array during declaration, instead when creating an array (using new) the data are saved.
Maybe there is a better reason that oracle may have answered already, but the fact that in Java everything is an object must have something to do with it. Java is quite specific about objects and types, unlike C where you have more freedom but everything is more loose (especially using pointers).

The main idea of the array data structure is that all its elements are located in the sequential row of memory cells. That is why you can not create array with variable size: it should be unbounbed space vector in memory for this purpose, which is impossible.
If you want change size of array, you should recreate it.

Since arrays are fixed-size they need to know how much memory to allocate at the time they are instantiated.
ArrayLists or other resizing data structures that internally use arrays to store data actually re-allocate larger arrays when their inner array data
structure fills up.

My understanding of OP's reasoning is:
new is used for allocating dynamic objects (which can grow like, ArrayList), but arrays are static (can't grow). So one of them is unnecessary: the new or the size of the array.
If that is the question, then the answer is simple:
Well, in Java new is necessary for every Object allocation, because in Java all objects are dynamically allocated.
Turns out that in Java, arrays are objects, different from C/C++ where they are not.

All of Java's variables are at most a single 64bit field. Either primitives like
integer (32bit)
long (64bit)
...
or references to Objects which depending on JVM / config / OS are 64 or 32 bit fields (but unlike 64bit primitives with atomicity guaranteed).
There is no such thing as C's int[20] "type". Neither is there C's static.
What int[] array = new int[20] boils down to is roughly
int* array = malloc(20 * sizeof(java_int))
Each time you see new in Java you can imagine a malloc and a call to the constructor method in case it's a real Object (not just an array). Each Object is more or less just a struct of a few primitives and more pointers.
The result is a giant network of relatively small structs pointing to other things. And the garbage collector's task is to free all the leaves that have fallen off the network.
And this is also the reason why you can say Java is copy by value: both primitives and pointers are always copied.
regarding static in Java: there is conceptually a struct per class that represents the static context of a class. That's the place where static instance variables are anchored. Non-static instance variables are anchored at with their own instance-struct
class Car {
static int[] forAllCars = new int[20];
Object perCar;
}
...
new Car();
translates very loosely (my C is terrible) to
struct Car-Static {
Object* forAllCars;
};
struct Car-Instance {
Object* perCar;
};
// .. class load time. Happens once and this is referenced from some root object so it can't get garbage collected
struct Car-Static *car_class = (struct Car-Static*) malloc(sizeof(Car-Static));
car_class->forAllCars = malloc(20 * 4);
// .. for every new Car();
struct Car-Instance *new_reference = (struct Car-Instance*) malloc(sizeof(Car-Instance));
new_reference.perCar = NULL; // all things get 0'd
new_reference->constructor();
// "new" essentially returns the "new_reference" then

Having two stores of an object in Java

Suppose I have the following code in java
Object object = new Object();
mylist.add(object);
mylist2.add(object);
As far as I understand I have created one object in memory and both mylist and mylist2 have some kind reference to this object. Is that correct?
In which case how much more memory does the program above use compared to if I had just done
Object object = new Object();
mylist.add(object);
I'm wondering because I sometimes feel it would be useful to have two different data structures holding the same information for different purposes.
Ex:
A binary tree and a hash map such that you can easily search for objects in constant time and easily iterate through an ordered list of the objects.

It depends on what type of list you use. If you use an ArrayList, then there is no overhead for each entry except the reference itself (4 bytes on a 32 bit machine, ignoring the empty space in this kind of list ;)). If you use for example a LinkedList then there is a wrapper object around it, which additionally holds a reference to the previous/next element in the list.

On most VMs the size of a reference is the native pointer size (from John Skeet)
So if you use a 32bit VM e.g. it will be 4 bytes.

Java object arrays memory requirements

Suppose there is an Integer array in my class:
public class Foo {
private Integer[] arr = new Integer[20];
.....
}
On a 64 bit architecture the space requirement for this is ~ (20*8+24) + 24*20 {space required for references + some array overhead + space required for objects}.
Why java stores references to all of the 20 Integer objects? Wouldn't knowing that first memory location and the number of items in the array suffice? (assuming and I also as I read somewhere that objects in an array are placed contiguously anyways). I want to know the reason for this sort of implementation. Sorry if this is a noobish question.

Like every other class, Integer is a reference type. This means it can only be accessed indirectly, via a reference. You cannot store an instance of a reference type in a field, a local variable, a slot in a collection, etc. -- you always have to store a reference and allocate the object itself separately. There are a variety of reasons for this:
You need to be able to represent null.
You need to be able to replace it with another instance of a subtype (assuming subtypes are possible, i.e. the class is not final). For example, an Object[] may actually store instances of any number of different classes with wildly varying sizes.
You need to preserve sharing, e.g. after a[0] = a[1] = someObject; all three must refer to the same object. This is much more important (vital even) if the object is mutable, but even with immutable objects the difference can be observed via reference equality checks (==).
You need reference assignment to be atomic (cf. Java memory model), so copying the whole instance is even more expensive than it seems.
With these and many other constraints, always storing references is the only feasible implementation strategy (in general). In very specific circumstances, a JIT compiler may avoid allocating an object entirely and store its directly (e.g. on the stack), but this is an obscure implementation detail, and not widely applicable. I only mention this for completeness and because it's a wonderful illustration of the as-if rule.

Java object which takes the least memory

This is a silly question, but here it goes.
I have a multithreaded program and a "global" Collection of unique elements. I rejected synchronized Set implementations due to performance, for the ConcurrentHashMap. I don't really need the Value part of Map, so I wanted to use the smallest Object in java in terms of memory usage. I solved this issue in a different way (single Boolean object referenced multiple times in the Map), but I am still curious what is the smallest object in Java. I always thought it to be Boolean, but that is not true I think (Java - boolean primitive type - size, Primitive Data Types)

It doesn't really matter, actually, since the value part of each association is fixed to be a reference. You might even use null as value here, but any other (fixed) object reference should be fine (and more convenient sometimes). I'd prefer Boolean.TRUE (or a similar "well known" singleton). You can then test for membership via
if (myMap.get(someKey) != null) { ... }
in addition to
if (myMap.containsKey(someKey)) { ... }

If you want a Set<K> that is backed by a ConcurrentHashMap, you should use Collections.newSetFromMap, e.g.
final Set<K> set = Collections.newSetFromMap(new ConcurrentHashMap<K, Boolean>());
Now, if you really want to reinvent the wheel, and care that much about memory usage, I suggest you merely use a plain Object as your value. Since every object in Java inherits from Object (the universal base class), the size of any object in memory must be greater than or equal to the size of a plain Object. You cannot use a primitives since generic type arguments must be Objects.
EDIT: Actually, allocating a particular object to use as your value here will take more memory than using a preexisting object which will likely be allocated for anyways. You can just use a reference to an object that will more or less always be allocated during VM initialization, e.g. Object.class. I really suggest you just use the first solution, though.

An object's size consists of:
the size of the instance variables it holds
an 8 or 16 bytes header (depending on the Hotspot VM (32/64bit))
a padding: its size is always padded to be a multiple of 8 bytes.
E.g (assuming a 32bit JVM) :
public MyBoolObject {
boolean flag;
}
will take up 16 bytes: 8bytes(header) + 1byte(instance variable) + 7bytes(padding).
Since you are not interested in the map values you can set them to null. This consumes 4 or 8 bytes memory from the stack (32/64bit).
You might also check this good list on cost/elements of well-known Java data structures:
http://code.google.com/p/memory-measurer/wiki/ElementCostInDataStructures

The primitive data types are not objects.
Since all objects in java must inherit from the super class Object. Then the smallest conceivable object in java would be a class that you define that has no members. Such a class would be pretty useless.

The Object class is instantiable and its instances are definitely the smallest objects in Java. However, many other objects have exactly the same footprint, Integer and Boolean being examples on 64-bit VMs. This is due to heap memory alignment.

Is there an expandable list of object references in Java?

In Java, we can always use an array to store object reference. Then we have an ArrayList or HashTable which is automatically expandable to store objects. But does anyone know a native way to have an auto-expandable array of object references?
Edit: What I mean is I want to know if the Java API has some class with the ability to store references to objects (but not storing the actual object like XXXList or HashTable do) AND the ability of auto-expansion.

Java arrays are, by their definition, fixed size. If you need auto-growth, you use XXXList classes.
EDIT - question has been clarified a bit
When I was first starting to learn Java (coming from a C and C++ background), this was probably one of the first things that tripped me up. Hopefully I can shed some light.
Unlike C++, Object arrays in Java do not store objects. They store object references.
In C++, if you declared something similar to:
String myStrings[10];
You would get 10 String objects. At this point, it would be perfectly legal to do something like println(myStrings[5].length); - you'd get '0' - the default constructor for String creates an empty string with length 0.
In Java, when you construct a new array, you get an empty container that can hold 10 String references. So the call:
String[] myStrings = new String[10];
println(myStringsp[5].length);
would throw a null pointer exception, because you haven't actually placed a String reference into the array yet.
If you are coming from a C++ background, think of new String[10] as being equivalent to new (String *)[10] from C++.
So, with that in mind, it should be fairly clear why ArrayList is the solution for an auto expanding array of objects (and in fact, ArrayList is implemented using simple arrays, with a growth algorithm built in that allocates new expanded arrays as needed and copies the content from the old to the new).
In practice, there are actually relatively few situations where we use arrays. If you are writing a container (something akin to ArrayList, or a BTree), then they are useful, or if you are doing a lot of low level byte manipulation - but at the level that most development occurs, using one of the Collections classes is by far the preferred technique.

All the classes implementing Collection are expandable and store only references: you don't store objects, you create them in some data space and only manipulate references to them, until they go out of scope without reference on them.
You can put a reference to an object in two or more Collections. That's how you can have sorted hash tables and such...

What do you mean by "native" way? If you want an expandable list f objects then you can use the ArrayList. With List collections you have the get(index) method that allows you to access objects in the list by index which gives you similar functionality to an array. Internally the ArrayList is implemented with an array and the ArrayList handles expanding it automatically for you.

Straight from the Array Java Tutorials on the sun webpage:
-> An array is a container object that holds a fixed number of values of a single type.
Because the size of the array is declared when it is created, there is actually no way to expand it afterwards. The whole purpose of declaring an array of a certain size is to only allocate as much memory as will likely be used when the program is executed. What you could do is declare a second array that is a function based on the size of the original, copy all of the original elements into it, and then add the necessary new elements (although this isn't very 'automatic' :) ). Otherwise, as you and a few others have mentioned, the List Collections is the most efficient way to go.

In Java, all object variables are references. So
Foo myFoo = new Foo();
Foo anotherFoo = myFoo;
means that both variables are referring to the same object, not to two separate copies. Likewise, when you put an object in a Collection, you are only storing a reference to the object. Therefore using ArrayList or similar is the correct way to have an automatically expanding piece of storage.

There's no first-class language construct that does that that I'm aware of, if that's what you're looking for.

It's not very efficient, but if you're just appending to an array, you can use Apache Commons ArrayUtils.add(). It returns a copy of the original array with the additional element in it.

if you can write your code in javascript, yes, you can do that. javascript arrays are sparse arrays. it will expand whichever way you want.
you can write
a[0] = 4;
a[1000] = 434;
a[888] = "a string";