We Keep Coding

What is an apparently correct example of Java code causing heap pollution?

I'm trying to decide what to do every time I get a Java heap pollution warning when using parameterized varargs such as in
public static <T> LinkedList<T> list(T... elements) {
...
}
It seems to me that if I am confident not to be using some weird casts in my methods, I should just use #SafeVarargs and move on. But is this correct, or do I need to be more careful? Is there apparently correct code that is actually not safe when using parameterized varargs?
Reading about the subject, I notice that the provided examples are quite artificial. For example, the Java documentation shows the following faulty method:
public static void faultyMethod(List<String>... l) {
Object[] objectArray = l; // Valid
objectArray[0] = Arrays.asList(42);
String s = l[0].get(0); // ClassCastException thrown here
}
which is didactic but pretty unrealistic; experienced programmers are not likely to write code doing stuff like this. Another example is
Pair<String, String>[] method(Pair<String, String>... lists) {
Object[] objs = lists;
objs[0] = new Pair<String, String>("x", "y");
objs[1] = new Pair<Long, Long>(0L, 0L); // corruption !!!
return lists;
}
which is again pretty obviously mixing types in an unrealistic way.
So, are there more subtle cases in which heap pollution happens under parameterized varargs? Am I justified in using #SafeVarargs if I am not casting variables in a way that loses typing information, or mixes types incorrectly? In other words, am I justified in treating this warning as a not very important formality?

Good question. This has bothered me quite a while too. There are two things here - you don't care about the actual runtime type of the elements within the array, like the example that you have shown:
public static <T> LinkedList<T> list(T... elements) {
// suppose you iterate over them and add
}
This is where #SafeVarargs is well, safe.
And the second one is where you DO care about the runtime type of the elements within the array (even if so by accident). Arrays, in java, can not be generic, so you can not create a type T [] ts = new T[10], but you can declare a type T[] ts... and because arrays are covariant you can cast an Object[] to a T[] - if you know the types match.
All this becomes interesting when you pass a generic array:
// create a single element "generic" array
static <T> T[] singleElement(T elem) {
#SuppressWarnings("unchecked")
T[] array = (T[]) new Object[] { elem };
return self(array);
}
// #SafeVarargs
static <T> T[] self(T... ts) {
return ts;
}
Invoking this with Integer[] ints = singleElement(1); looks perfectly legal, but will break at runtime, this is where placing #SafeVarargs would be unsafe.
It will break because that cast (T[]) is actually useless and does not enforce any compile time checks. Even if you rewrote that method as:
static <T> T[] singleElement(T elem) {
#SuppressWarnings("unchecked")
T[] array = (T[]) new Object[]{elem};
System.out.println(array.getClass());
return array;
}
it would still not work.

To declare generic arrays T[] in Java is problematic because their type is not known at compile time and as a consequence they can be misused, as the examples in the question show. So the Java compiler issues warnings whenever this is done.
For example, if we declare a generic array as in
T[] tArray = (T[]) new Object[] { 42 };
we get an "unchecked cast" warning.
Besides such casts, the only other way of introducing a generic array into a program is by using a generic varargs. For example, in
void bar() {
foo(new Integer[]{ 42 })
}
void foo(T... args) {
}
Again here a generic array is being introduced, but in a different way than an unchecked cast, so it gets its own specific warning to make sure the user is not misusing it.
Indeed, as long as one is not converting the array to an array of a different type, it seems that using #SafeVarargs should be safe to use, barring atypical type conversions.

Java generics in ArrayList.toArray()

Say you have an arraylist defined as follows:
ArrayList<String> someData = new ArrayList<>();
Later on in your code, because of generics you can say this:
String someLine = someData.get(0);
And the compiler knows outright that it will be getting a string. Yay generics! However, this will fail:
String[] arrayOfData = someData.toArray();
toArray() will always return an array of Objects, not of the generic that was defined. Why does the get(x) method know what it is returning, but toArray() defaults to Objects?

If you look at the implementation of toArray(T[] a) of ArrayList<E> class, it is like:
public <T> T[] toArray(T[] a) {
if (a.length < size)
// Make a new array of a's runtime type, but my contents:
return (T[]) Arrays.copyOf(elementData, size, a.getClass());
System.arraycopy(elementData, 0, a, 0, size);
if (a.length > size)
a[size] = null;
return a;
}
Problem with this method is that you need to pass array of the same generic type. Now consider if this method do not take any argument then the implementation would be something similar to:
public <T> T[] toArray() {
T[] t = new T[size]; // compilation error
return Arrays.copyOf(elementData, size, t.getClass());
}
But the problem here is that you can not create generic arrays in Java because compiler does not know exactly what T represents. In other words creation of array of a non-reifiable type (JLS §4.7) is not allowed in Java.
Another important quote from Array Store Exception (JLS §10.5):
If the component type of an array were not reifiable (§4.7), the Java Virtual Machine could not perform the store check described in the
preceding paragraph. This is why an array creation expression with a
non-reifiable element type is forbidden (§15.10.1).
That is why Java has provided overloaded version toArray(T[] a).
I will override the toArray() method to tell it that it will return an
array of E.
So instead of overriding toArray(), you should use toArray(T[] a).
Cannot Create Instances of Type Parameters from Java Doc might also be interesting for you.

Generic information is erased at runtime. JVM does not know whether your list is List<String> or List<Integer> (at runtime T in List<T> is resolved as Object), so the only possible array type is Object[].
You can use toArray(T[] array) though - in this case JVM can use the class of a given array, you can see it in the ArrayList implementation:
public <T> T[] toArray(T[] a) {
if (a.length < size)
// Make a new array of a's runtime type, but my contents:
return (T[]) Arrays.copyOf(elementData, size, a.getClass());

If you look at the Javadoc for the List interface, you'll notice a second form of toArray: <T> T[] toArray(T[] a).
In fact, the Javadoc even gives an example of how to do exactly what you want to do:
String[] y = x.toArray(new String[0]);

The pertinent thing to note is that arrays in Java know their component type at runtime. String[] and Integer[] are different classes at runtime, and you can ask arrays for their component type at runtime. Therefore, a component type is needed at runtime (either by hard-coding a reifiable component type at compile time with new String[...], or using Array.newInstance() and passing a class object) to create an array.
On the other hand, type arguments in generics do not exist at runtime. There is absolutely no difference at runtime between an ArrayList<String> and a ArrayList<Integer>. It is all just ArrayList.
That's the fundamental reason why you can't just take a List<String> and get a String[] without passing in the component type separately somehow -- you would have to get component type information out of something that doesn't have component type information. Clearly, this is impossible.

I can, and will use an iterator instead of making an array sometimes, but this just always seemed strange to me. Why does the get(x) method know what it is returning, but toArray() defaults to Objects? Its like half way into designing it they decided this wasn't needed here??
As the intention of the question seems to be not just about getting around using toArray() with generics, rather also about understanding the design of the methods in the ArrayList class, I would like to add:
ArrayList is a generic class as it is declared like
public class ArrayList<E> extends AbstractList<E>
implements List<E>, RandomAccess, Cloneable, java.io.Serializable
which makes it possible to use Generic methods such as public E get(int index) within the class.
But if a method such as toArray() is not returning E, rather E[] then things start getting a bit tricky. It would not be possible to offer a signature such as public <E> E[] toArray() because it is not possible to create generic arrays.
Creation of arrays happen at runtime and due to Type erasure, Java runtime has no specific information of the type represented by E. The only workaround as of now is to pass the required type as a parameter to the method and hence the signature public <T> T[] toArray(T[] a) where clients are forced to pass the required type.
But on the other hand, it works for public E get(int index) because if you look at the implementation of the method, you would find that even though the method makes use of the same array of Object to return the element at the specified index, it is casted to E
E elementData(int index) {
return (E) elementData[index];
}
It is the Java compiler which at the compile time replaces E with Object

The very first thing you have to understand is what ArrayList own is just an array of Object
transient Object[] elementData;
When it comes to the reason why T[] is fail, it because you can't get an array of generic type without a Class<T> and this is because java's type erase( there is a more explanation and how to create one). And the array[] on the heap knows its type dynamically and you can't cast int[] to String[]. The same reason, you can't cast Object[] to T[].
int[] ints = new int[3];
String[] strings = (String[]) ints;//java: incompatible types: int[] cannot be converted to java.lang.String[]
public <T> T[] a() {
Object[] objects = new Object[3];
return (T[])objects;
}
//ClassCastException: [Ljava.lang.Object; cannot be cast to [Ljava.lang.Integer;
Integer[] a = new LearnArray().<Integer>a();
But what you put into the array is just a object which type is E(which is checked by compiler), so you can just cast it to E which is safe and correct.
return (E) elementData[index];
In short, you can't get what don't have by cast. You have just Object[], so toArray() can just return Object[](otherwise, you have to give it a Class<T> to make a new array with this type). You put E in ArrayList<E>, you can get a E with get().

An array is of a different type than the type of the array. It's sort of StringArray class instead of String class.
Assuming, it would be possible, an Generic method toArray() would look like
private <T> T[] toArray() {
T[] result = new T[length];
//populate
return result;
}
Now during compilation, the type T gets erased. How should the part new T[length] be replaced? The generic type information is not available.
If you look at the source code of (for example) ArrayList, you see the same. The toArray(T[] a) method either fills the given array (if the size matches) or creates a new new array using the type of the parameter, which is the array-type of the Generic Type T.

It is possible to create a "generic" array of the given(known) type. Normally I use something like this in my code.
public static <T> T[] toArray(Class<T> type, ArrayList<T> arrList) {
if ((arrList == null) || (arrList.size() == 0)) return null;
Object arr = Array.newInstance(type, arrList.size());
for (int i=0; i < arrList.size(); i++) Array.set(arr, i, arrList.get(i));
return (T[])arr;
}

Why is the declaration of the array of generic type correct?

We cannot create the array of generic type, it's a well-known fact, so I'm not going to provide formal references to JLS here. But we can declare such arrays as follows:
static <E> void reduce() {
List<Integer>[] arr; //compiles fine
E[] avv; //compiles fine
avv = new E[10]; //doesn't compile
arr = new List<Integer>[10]; //doesn't compile
}
Anyone know the reason for such declarations?

First of all, I assume you meant
avv = new E[2]; //doesn't compile
arr = new List<Integer>[2]; //doesn't compile
in the last two lines of your method. You have to specify the size of the array when to create a new array. The code still doesn't compile, though.
Arrays predate generics. Arrays are present since the first version of Java, while generics were only added in version 1.5. To break no old code, the Java designers decided to erase generic types at runtime: at runtime, a type parameters are replaced by their upper bound. In your case, at runtime, E is the same as Object. It is not known which type E is really.
This is a problem, because, the array element type is not erased at runtime. Integer[] and String[] are different types, even at runtime. If you write new E[2], the Java runtime doesn't know what kind of array it must create. It could be String[] or Integer[], or any other array type. Therefore, you cannot create new arrays with generic elements.
Arrays with generic elements are still allowed as types, mainly in order to use them in method parameters:
<E> E doSomething(E[] param) { ... }
The actual array is created in another part of the program, where its type is known. You could call this method with
String result = doSomething(new String[2]);
for example.

It allows you to pass arrays of generic types as method parameters.
For example:
public class Foo<T>
{
public void bar (T[] arr) {}
}
...
Foo<String> foo = new Foo<String>();
String[] arr = {"aa","bb");
foo.bar (arr);
If T[] wasn't allowed, the method signature would have to be public void bar (Object[] arr), and the compiler would let you pass any type of array to that method.

Generic Types & the toArrayMethod

I have a class MyStack<T> which defines the following
public T[] toArray(){
int s=size();
#SuppressWarnings("unchecked")
T[] result=(T[])new Object[s];
Node n=first;
for (int i=0; i<s; i++){
result[i]=n.data;
n=n.next;
}
return result;
}
Since this returns an array of type T, I would think that if I declared this instance: MyStack<String> s=new MyStack<>, that the following would be perfectly valid: String[] test=s.toArray(). I think this because since s is of type String, toArray should return an array of type String, since String has basically been substituted in for every T in this class (only for this particular instantiation, I know). The only way this runs without errors is if I do this: Object[] test=s.toArray().
Why is this?

In a word, type erasure. Taken from the Java website:
Replace all type parameters in generic types with their bounds or Object if the type parameters are unbounded. The produced bytecode, therefore, contains only ordinary classes, interfaces, and methods.
What this means is that, when your code is compiled, MyStack<String> is compiled into MyStack<Object>. This is to make sure that generics do not incur an overhead by needing to create new classes. How does this apply to you? Well..
MyStack<String> s = new MyStack<>();
is converted into..
MyStack<Object> s = new MyStack<>();
Now, this means that when you call the toArray method, the only type that can be guarenteed is the Object type. The compiler can't be sure that everything it returns is of type String, so it won't let you treat it as a String, due to the strong typing in Java. So, what is the only variable type left?
Object[] array = s.toArray();
Extra Reading
Type Erasure in Java.

Well, hold on a minute. Suppose your hypothesis were correct that String were substituted for every T.
Would the following cast be valid?
String[] result = (String[])new Object[s];
No, it would not. We can be sure that a new Object[] is not a String[].
Now sometimes you will see something like (T[])new Object[n] but it only works because the cast actually becomes erased inside the generic class. (It is a deceptive idiom.)
When the class gets compiled, what actually happens is that references to T are replaced with its upper bound (probably Object unless you had something like <T extends ...>):
public Object[] toArray(){
int s=size();
Object[] result=new Object[s];
Node n=first;
for (int i=0; i<s; i++){
result[i]=n.data;
n=n.next;
}
return result;
}
And the cast is moved to the call site:
MyStack stack = new MyStack();
String[] arr = (String[])stack.toArray();
So in fact, while the cast is erased inside the class, the cast does happen once the value is returned to outside the class, where ClassCastException is thrown.
The inability to instantiate arrays (and objects in general) generically is why the Collections framework defines their toArray method to take the return array as an argument. A simple version of this for you would be like the following:
public T[] toArray(T[] inArray){
int s = size();
Node n = first;
for (int i = 0; i < s; i++){
inArray[i] = n.data;
n = n.next;
}
return inArray;
}
For some ideas on how to create an array generically, you may see 'How to create a generic array in Java?'; however you will need the caller to pass some argument to the method.

What's the reason I can't create generic array types in Java?

What's the reason why Java doesn't allow us to do
private T[] elements = new T[initialCapacity];
I could understand .NET didn't allow us to do that, as in .NET you have value types that at run-time can have different sizes, but in Java all kinds of T will be object references, thus having the same size (correct me if I'm wrong).
What is the reason?

It's because Java's arrays (unlike generics) contain, at runtime, information about its component type. So you must know the component type when you create the array. Since you don't know what T is at runtime, you can't create the array.

Quote:
Arrays of generic types are not
allowed because they're not sound. The
problem is due to the interaction of
Java arrays, which are not statically
sound but are dynamically checked,
with generics, which are statically
sound and not dynamically checked.
Here is how you could exploit the
loophole:
class Box<T> {
final T x;
Box(T x) {
this.x = x;
}
}
class Loophole {
public static void main(String[] args) {
Box<String>[] bsa = new Box<String>[3];
Object[] oa = bsa;
oa[0] = new Box<Integer>(3); // error not caught by array store check
String s = bsa[0].x; // BOOM!
}
}
We had proposed to resolve this
problem using statically safe arrays
(aka Variance) bute that was rejected
for Tiger.
-- gafter
(I believe it is Neal Gafter, but am not sure)
See it in context here: http://forums.sun.com/thread.jspa?threadID=457033&forumID=316

By failing to provide a decent solution, you just end up with something worse IMHO.
The common work around is as follows.
T[] ts = new T[n];
is replaced with (assuming T extends Object and not another class)
T[] ts = (T[]) new Object[n];
I prefer the first example, however more academic types seem to prefer the second, or just prefer not to think about it.
Most of the examples of why you can't just use an Object[] equally apply to List or Collection (which are supported), so I see them as very poor arguments.
Note: this is one of the reasons the Collections library itself doesn't compile without warnings. If this use-case cannot be supported without warnings, something is fundamentally broken with the generics model IMHO.

The reason this is impossible is that Java implements its Generics purely on the compiler level, and there is only one class file generated for each class.
This is called Type Erasure.
At runtime, the compiled class needs to handle all of its uses with the same bytecode. So, new T[capacity] would have absolutely no idea what type needs to be instantiated.

The answer was already given but if you already have an Instance of T then you can do this:
T t; //Assuming you already have this object instantiated or given by parameter.
int length;
T[] ts = (T[]) Array.newInstance(t.getClass(), length);
Hope, I could Help,
Ferdi265

The main reason is due to the fact that arrays in Java are covariant.
There's a good overview here.

I like the answer indirectly given
by Gafter. However, I propose it is wrong. I changed Gafter's code a little. It compiles and it runs for a while then it bombs where Gafter predicted it would
class Box<T> {
final T x;
Box(T x) {
this.x = x;
}
}
class Loophole {
public static <T> T[] array(final T... values) {
return (values);
}
public static void main(String[] args) {
Box<String> a = new Box("Hello");
Box<String> b = new Box("World");
Box<String> c = new Box("!!!!!!!!!!!");
Box<String>[] bsa = array(a, b, c);
System.out.println("I created an array of generics.");
Object[] oa = bsa;
oa[0] = new Box<Integer>(3);
System.out.println("error not caught by array store check");
try {
String s = bsa[0].x;
} catch (ClassCastException cause) {
System.out.println("BOOM!");
cause.printStackTrace();
}
}
}
The output is
I created an array of generics.
error not caught by array store check
BOOM!
java.lang.ClassCastException: java.lang.Integer cannot be cast to java.lang.String
at Loophole.main(Box.java:26)
So it appears to me you can create generic array types in java. Did I misunderstand the question?

From Oracle tutorial:
You cannot create arrays of parameterized types. For example, the following code does not compile:
List<Integer>[] arrayOfLists = new List<Integer>[2]; // compile-time error
The following code illustrates what happens when different types are inserted into an array:
Object[] strings = new String[2];
strings[0] = "hi"; // OK
strings[1] = 100; // An ArrayStoreException is thrown.
If you try the same thing with a generic list, there would be a problem:
Object[] stringLists = new List<String>[]; // compiler error, but pretend it's allowed
stringLists[0] = new ArrayList<String>(); // OK
stringLists[1] = new ArrayList<Integer>(); // An ArrayStoreException should be thrown,
// but the runtime can't detect it.
If arrays of parameterized lists were allowed, the previous code would fail to throw the desired ArrayStoreException.
To me, it sounds very weak. I think that anybody with a sufficient understanding of generics, would be perfectly fine, and even expect, that the ArrayStoredException is not thrown in such case.

In my case, I simply wanted an array of stacks, something like this:
Stack<SomeType>[] stacks = new Stack<SomeType>[2];
Since this was not possible, I used the following as a workaround:
Created a non-generic wrapper class around Stack (say MyStack)
MyStack[] stacks = new MyStack[2] worked perfectly well
Ugly, but Java is happy.
Note: as mentioned by BrainSlugs83 in the comment to the question, it is totally possible to have arrays of generics in .NET

class can declare an array of type T[], but it cannot directly instantiate such an array. Instead, a common approach is to instantiate an array of type Object[], and then make a narrowing cast to type T[], as shown in the following:
public class Portfolio<T> {
T[] data;
public Portfolio(int capacity) {
data = new T[capacity]; // illegal; compiler error
data = (T[]) new Object[capacity]; // legal, but compiler warning
}
public T get(int index) { return data[index]; }
public void set(int index, T element) { data[index] = element; }
}

It is because generics were added on to java after they made it, so its kinda clunky because the original makers of java thought that when making an array the type would be specified in the making of it. So that does not work with generics so you have to do
E[] array=(E[]) new Object[15];
This compiles but it gives a warning.

There surely must be a good way around it (maybe using reflection), because it seems to me that that's exactly what ArrayList.toArray(T[] a) does. I quote:
public <T> T[] toArray(T[] a)
Returns an array containing all of the
elements in this list in the correct order; the runtime type of the
returned array is that of the specified array. If the list fits in the
specified array, it is returned therein. Otherwise, a new array is
allocated with the runtime type of the specified array and the size of
this list.
So one way around it would be to use this function i.e. create an ArrayList of the objects you want in the array, then use toArray(T[] a) to create the actual array. It wouldn't be speedy, but you didn't mention your requirements.
So does anyone know how toArray(T[] a) is implemented?

If we cannot instantiate generic arrays, why does the language have generic array types? What's the point of having a type without objects?
The only reason I can think of, is varargs - foo(T...). Otherwise they could have completely scrubbed generic array types. (Well, they didn't really have to use array for varargs, since varargs didn't exist before 1.5. That's probably another mistake.)
So it is a lie, you can instantiate generic arrays, through varargs!
Of course, the problems with generic arrays are still real, e.g.
static <T> T[] foo(T... args){
return args;
}
static <T> T[] foo2(T a1, T a2){
return foo(a1, a2);
}
public static void main(String[] args){
String[] x2 = foo2("a", "b"); // heap pollution!
}
We can use this example to actually demonstrate the danger of generic array.
On the other hand, we've been using generic varargs for a decade, and the sky is not falling yet. So we can argue that the problems are being exaggerated; it is not a big deal. If explicit generic array creation is allowed, we'll have bugs here and there; but we've been used to the problems of erasure, and we can live with it.
And we can point to foo2 to refute the claim that the spec keeps us from the problems that they claim to keep us from. If Sun had more time and resources for 1.5, I believe they could have reached a more satisfying resolution.

As others already mentioned, you can of course create via some tricks.
But it's not recommended.
Because the type erasure and more importantly the covariance in array which just allows a subtype array can be assigned to a supertype array, which forces you to use explicit type cast when trying to get the value back causing run-time ClassCastException which is one of the main objectives that generics try to eliminate: Stronger type checks at compile time.
Object[] stringArray = { "hi", "me" };
stringArray[1] = 1;
String aString = (String) stringArray[1]; // boom! the TypeCastException
A more direct example can found in Effective Java: Item 25.
covariance: an array of type S[] is a subtype of T[] if S is a subtype of T

T vals[]; // OK
But, you cannot instantiate an array of T
// vals = new T[10]; // can't create an array of T
The reason you can’t create an array of T is that there is no way for the
compiler to know what type of array to actually create.

Try this:
List<?>[] arrayOfLists = new List<?>[4];