Scala #specialized annotation infinite recursion? - java

Version: scala 2.11.8
I defined a class with specialized type and override method in inheritance:
class Father[#specialized(Int) A]{
def get(from: A): A = from
}
class Son extends Father[Int]{
override def get(from: Int): Int = {
println("Son.get")
super.get(from)
}
}
new Son().get(1) // will cause infinite recursion
So, how to reuse the method of superclass with specialized annotation?

From the article Quirks of Scala Specialization:
Avoid super calls
Qualified super calls are (perhaps fundamentally) broken with specialization. Rewiring the super-accessor methods properly in the specialization phase is a nightmare that has not been solved so far. So, avoid them like the plague, at least for now. In particular, stackable modifications pattern will not work with it well.
So it's most likely a compiler bug, in general you shouldn't use super calls with Scala specialization.
After a bit of investigation:
javap -c Son.class
public class Son extends Father$mcI$sp {
public int get(int);
Code:
0: aload_0
1: iload_1
2: invokevirtual #14 // Method get$mcI$sp:(I)I
5: ireturn
public int get$mcI$sp(int);
Code:
0: getstatic #23 // Field scala/Predef$.MODULE$:Lscala/Predef$;
3: ldc #25 // String Son.get
5: invokevirtual #29 // Method scala/Predef$.println:(Ljava/lang/Object;)V
8: aload_0
9: iload_1
10: invokespecial #31 // Method Father$mcI$sp.get:(I)I
13: ireturn
Son.get(int) calls Son.get$mcI$sp(int) which turns into Father$mcI$sp.get(int):
javap -c Father\$mcI\$sp.class
public class Father$mcI$sp extends Father<java.lang.Object> {
public int get(int);
Code:
0: aload_0
1: iload_1
2: invokevirtual #12 // Method get$mcI$sp:(I)I
5: ireturn
public int get$mcI$sp(int);
Code:
0: iload_1
1: ireturn
Looks like we have found the cause - Father$mcI$sp.get(int) makes a virtual call to get$mcI$sp, which is overloaded in Son! This is what caused the infinite recursion here.
The compiler has to create specialized versions of method get which is get$mcI$sp, in order to support the non-specialized generic version of Father[T], which unfortunately makes it impossible to have super calls with specialized classes.
Now what happens after changing Father to be a trait (with Scala 2.12):
javap -c Son.class
public class Son implements Father$mcI$sp {
public int get$mcI$sp(int);
Code:
0: getstatic #25 // Field scala/Predef$.MODULE$:Lscala/Predef$;
3: ldc #27 // String Son.get
5: invokevirtual #31 // Method scala/Predef$.println:(Ljava/lang/Object;)V
8: aload_0
9: iload_1
10: invokestatic #37 // Method scala/runtime/BoxesRunTime.boxToInteger:(I)Ljava/lang/Integer;
13: invokestatic #43 // InterfaceMethod Father.get$:(LFather;Ljava/lang/Object;)Ljava/lang/Object;
16: invokestatic #47 // Method scala/runtime/BoxesRunTime.unboxToInt:(Ljava/lang/Object;)I
19: ireturn
It looks like instead of calling get$mcI$sp in a parent class, it invokes static method Father.get$:
javap -c Father.class
public interface Father<A> {
public static java.lang.Object get$(Father, java.lang.Object);
Code:
0: aload_0
1: aload_1
2: invokespecial #17 // InterfaceMethod get:(Ljava/lang/Object;)Ljava/lang/Object;
5: areturn
public A get(A);
Code:
0: aload_1
1: areturn
public static int get$mcI$sp$(Father, int);
Code:
0: aload_0
1: iload_1
2: invokespecial #26 // InterfaceMethod get$mcI$sp:(I)I
5: ireturn
public int get$mcI$sp(int);
Code:
0: aload_0
1: iload_1
2: invokestatic #33 // Method scala/runtime/BoxesRunTime.boxToInteger:(I)Ljava/lang/Integer;
5: invokeinterface #17, 2 // InterfaceMethod get:(Ljava/lang/Object;)Ljava/lang/Object;
10: invokestatic #37 // Method scala/runtime/BoxesRunTime.unboxToInt:(Ljava/lang/Object;)I
13: ireturn
What's interesting here, is that it seems like the get method is not getting real specialization since it has to box the value in get$mcI$sp, which might be a bug, or maybe specialization support for traits was dropped in Scala 2.12.

Related

Java overloading with objects

I want to know why the second print statement produces "one implementation" as and output, below is the java code
class Point {
public boolean equals(final Object anObject) {
System.out.println("One implementation.");
return false;
}
public boolean equals(final Point aPoint) {
System.out.println("Another implementation.");
return false;
}
}
public class Main {
public static void main(final String[] args) {
final Point p1 = new Point();
final Point p2 = new Point();
final Object o = p1;
System.out.println(p1.equals(p2));
System.out.println(o.equals(p2));
System.out.println(p1.equals(o));
}
}
The output:-
Another implementation
false
One implementation {concern}
false
One implementation
false
The method
boolean equals(Object other);
overloads a base Object method; this method
boolean equals(Point other);
does not. So, when you call Object.equals(), which is what you're doing in your second case, you will end up in the first implementation you provided. Overriding Object.equals() is what you need to do as a minimum; other implementations are optional, and in my opinion, apt to mislead.
One should annotate overrides with #Override; this might help avoid confusion about whether a purported override really is.
You can use javap utility with -c -v flags to get better understanding what's going on, full command javap -c -v Main, its output (I truncated it) is
public static void main(java.lang.String[]);
Code:
0: new #2 // class Main$Point
3: dup
4: invokespecial #3 // Method Main$Point."<init>":()V
7: astore_1
8: new #2 // class Main$Point
11: dup
12: invokespecial #3 // Method Main$Point."<init>":()V
15: astore_2
16: aload_1
17: astore_3
18: getstatic #4 // Field java/lang/System.out:Ljava/io/PrintStream;
21: aload_1
22: aload_2
23: invokevirtual #5 // Method Main$Point.equals:(LMain$Point;)Z
26: invokevirtual #6 // Method java/io/PrintStream.println:(Z)V
29: getstatic #4 // Field java/lang/System.out:Ljava/io/PrintStream;
32: aload_3
33: aload_2
34: invokevirtual #7 // Method java/lang/Object.equals:(Ljava/lang/Object;)Z
37: invokevirtual #6 // Method java/io/PrintStream.println:(Z)V
40: getstatic #4 // Field java/lang/System.out:Ljava/io/PrintStream;
43: aload_1
44: aload_3
45: invokevirtual #8 // Method Main$Point.equals:(Ljava/lang/Object;)Z
48: invokevirtual #6 // Method java/io/PrintStream.println:(Z)V
51: return
In other words, Java compiler translated 2nd invocation into java/lang/Object.equals because type of o variable is Object. Java is statically typed language so all type resolutions happen at compile time so it does not matter if variable o will be assigned Point at run-time

Do Java enums allocate heap memory in the Android Runtime?

I was watching a video by two Android engineers talking about garbage collection in Android. In the introduction they do a little banter over enums. Romain Guy says: "I have to correct you there. Enums, they don't allocate. That's the whole point." At first, I that Romain just made a joke because enums work like this in other languages. But after this, Chet seems to concede that enums indeed to not allocate, but doing some "memory-related" stuff (implying: living on the stack). This reaction is what confuses me.
https://youtu.be/Zc4JP8kNGmQ?t=96
In my understanding, enums in Java are basically fixed collections of class-instances and seeing as Enum implements Object as good as object instantiations from a memory perspective, and so will be allocated to the heap.
But I can imagine that enums have some special status, owing to the strong properties the compiler can draw about them. Analogously, I know there are various optimizations for String like a shared pool for literals.
I am currently in the situation where I have fixed list of objects that I use as constants in my application. So I can implement this as an enum or as an array of class instantiations. Assuming readability is not an issue, would it be more performant to do the former?
Enums are objects. Like all objects, they live on the heap.
Indeed, if you decompile a simple enum, like:
enum Foo { A, B }
it looks like this (some stuff omitted):
static {};
Code:
0: new #4 // class Foo
3: dup
4: ldc #7 // String A
6: iconst_0
7: invokespecial #8 // Method "<init>":(Ljava/lang/String;I)V
10: putstatic #9 // Field A:LFoo;
13: new #4 // class Foo
16: dup
17: ldc #10 // String B
19: iconst_1
20: invokespecial #8 // Method "<init>":(Ljava/lang/String;I)V
23: putstatic #11 // Field B:LFoo;
26: iconst_2
27: anewarray #4 // class Foo
30: dup
31: iconst_0
32: getstatic #9 // Field A:LFoo;
35: aastore
36: dup
37: iconst_1
38: getstatic #11 // Field B:LFoo;
41: aastore
42: putstatic #1 // Field $VALUES:[LFoo;
45: return
which is basically the same as a class like this:
class Bar {
static final Bar A = new Bar("A");
static final Bar B = new Bar("B");
static final Bar[] $VALUES;
static {
Bar[] array = new Bar[2];
array[0] = A;
array[1] = B;
$VALUES = array;
}
private Bar(String name) {}
}
which decompiles to:
static {};
Code:
0: new #2 // class Bar
3: dup
4: ldc #3 // String A
6: invokespecial #4 // Method "<init>":(Ljava/lang/String;)V
9: putstatic #5 // Field A:LBar;
12: new #2 // class Bar
15: dup
16: ldc #6 // String B
18: invokespecial #4 // Method "<init>":(Ljava/lang/String;)V
21: putstatic #7 // Field B:LBar;
24: iconst_2
25: anewarray #2 // class Bar
28: astore_0
29: aload_0
30: iconst_0
31: getstatic #5 // Field A:LBar;
34: aastore
35: aload_0
36: iconst_1
37: getstatic #7 // Field B:LBar;
40: aastore
41: aload_0
42: putstatic #8 // Field $VALUES:[LBar;
45: return
There are a few other special things you get from an enum (like guarantees that they can't be creayed reflectively); but really, they are just regular objects.
The point that I think they are trying to make is that enums don't get allocated more than once (they are a good way to implement singetons, if you really need a singleton). So, you pay a small, fixed cost to load the enum class; but you can then reuse those same instances over and over.

Where's the code for a lambda located in a java class file?

I've this java source file:
import java.util.function.*;
public class t {
public static void main(String[] args) {
Function<Integer,Integer> r = (a) -> a*a+2*a+1;
System.out.println(r.apply(2));
}
}
I compile it and it works as expected. Here's the output of javap -c -v t, and I can't find the location of lambda in it. Where's the bytecode which tells the jvm to compute the expression with the input Integer whenever the lambda is envoked?
If you want to see the code of your lambda body a*a+2*a+1, you should call javap -c -v -p t to see also the private methods:
private static java.lang.Integer lambda$main$0(java.lang.Integer);
descriptor: (Ljava/lang/Integer;)Ljava/lang/Integer;
flags: ACC_PRIVATE, ACC_STATIC, ACC_SYNTHETIC
Code:
stack=3, locals=1, args_size=1
0: aload_0
1: invokevirtual #7 // Method java/lang/Integer.intValue:()I
4: aload_0
5: invokevirtual #7 // Method java/lang/Integer.intValue:()I
8: imul
9: iconst_2
10: aload_0
11: invokevirtual #7 // Method java/lang/Integer.intValue:()I
14: imul
15: iadd
16: iconst_1
17: iadd
18: invokestatic #4 // Method java/lang/Integer.valueOf:(I)Ljava/lang/Integer;
21: areturn
LineNumberTable:
line 4: 0
More detailed answer about the lambda inner implementation is here: How will Java lambda functions be compiled?

Does looping through an enum create new objects? [duplicate]

I need to convert an ordinal int value to an enum value in Java. Which is simple:
MyEnumType value = MyEnumType.values()[ordinal];
The values() method is implicit, and I cannot locate the source code for it, hence the question.
Does the MyEnumType.values() allocate a new array or not? And if it does, should I cache the array when first called? Suppose that the conversion will be called quite often.
Yes.
Java doesn't have mechanism which lets us create unmodifiable array. So if values() would return same mutable array, we risk that someone could change its content for everyone.
So until unmodifiable arrays will be introduced to Java, for safety values() must return new/separate array holding all values.
We can test it with == operator:
MyEnumType[] arr1 = MyEnumType.values();
MyEnumType[] arr2 = MyEnumType.values();
System.out.println(arr1 == arr2); //false
If you want to avoid recreating this array you can simply store it and reuse result of values() later. There are few ways to do it, like.
you can create private array and allow access to its content only via getter method like
private static final MyEnumType[] VALUES = values();// to avoid recreating array
MyEnumType getByOrdinal(int){
return VALUES[int];
}
you can store result of values() in unmodifiable collection like List to ensure that its content will not be changed (now such list can be public).
public static final List<MyEnumType> VALUES = Collections.unmodifiableList(Arrays.asList(values()));
Theoretically, the values() method must return a new array every time, since Java doesn't have immutable arrays. If it always returned the same array it could not prevent callers muddling each other up by modifying the array.
I cannot locate the source code for it
The values() method has no ordinary source code, being compiler-generated. For javac, the code that generates the values() method is in com.sun.tools.javac.comp.Lower.visitEnumDef. For ECJ (Eclipse's compiler), the code is in org.eclipse.jdt.internal.compiler.codegen.CodeStream.generateSyntheticBodyForEnumValues.
An easier way to find the implementation of the values() method is by disassembling a compiled enum. First create some silly enum:
enum MyEnumType {
A, B, C;
public static void main(String[] args) {
System.out.println(values()[0]);
}
}
Then compile it, and disassemble it using the javap tool included in the JDK:
javac MyEnumType.java && javap -c -p MyEnumType
Visible in the output are all the compiler-generated implicit members of the enum, including (1) a static final field for each enum constant, (2) a hidden $VALUES array containing all the constants, (3) a static initializer block that instantiates each constant and assigns each one to its named field and to the array, and (4) the values() method that works by calling .clone() on the $VALUES array and returning the result:
final class MyEnumType extends java.lang.Enum<MyEnumType> {
public static final MyEnumType A;
public static final MyEnumType B;
public static final MyEnumType C;
private static final MyEnumType[] $VALUES;
public static MyEnumType[] values();
Code:
0: getstatic #1 // Field $VALUES:[LMyEnumType;
3: invokevirtual #2 // Method "[LMyEnumType;".clone:()Ljava/lang/Object;
6: checkcast #3 // class "[LMyEnumType;"
9: areturn
public static MyEnumType valueOf(java.lang.String);
Code:
0: ldc #4 // class MyEnumType
2: aload_0
3: invokestatic #5 // Method java/lang/Enum.valueOf:(Ljava/lang/Class;Ljava/lang/String;)Ljava/lang/Enum;
6: checkcast #4 // class MyEnumType
9: areturn
private MyEnumType(java.lang.String, int);
Code:
0: aload_0
1: aload_1
2: iload_2
3: invokespecial #6 // Method java/lang/Enum."<init>":(Ljava/lang/String;I)V
6: return
public static void main(java.lang.String[]);
Code:
0: getstatic #7 // Field java/lang/System.out:Ljava/io/PrintStream;
3: invokestatic #8 // Method values:()[LMyEnumType;
6: iconst_0
7: aaload
8: invokevirtual #9 // Method java/io/PrintStream.println:(Ljava/lang/Object;)V
11: return
static {};
Code:
0: new #4 // class MyEnumType
3: dup
4: ldc #10 // String A
6: iconst_0
7: invokespecial #11 // Method "<init>":(Ljava/lang/String;I)V
10: putstatic #12 // Field A:LMyEnumType;
13: new #4 // class MyEnumType
16: dup
17: ldc #13 // String B
19: iconst_1
20: invokespecial #11 // Method "<init>":(Ljava/lang/String;I)V
23: putstatic #14 // Field B:LMyEnumType;
26: new #4 // class MyEnumType
29: dup
30: ldc #15 // String C
32: iconst_2
33: invokespecial #11 // Method "<init>":(Ljava/lang/String;I)V
36: putstatic #16 // Field C:LMyEnumType;
39: iconst_3
40: anewarray #4 // class MyEnumType
43: dup
44: iconst_0
45: getstatic #12 // Field A:LMyEnumType;
48: aastore
49: dup
50: iconst_1
51: getstatic #14 // Field B:LMyEnumType;
54: aastore
55: dup
56: iconst_2
57: getstatic #16 // Field C:LMyEnumType;
60: aastore
61: putstatic #1 // Field $VALUES:[LMyEnumType;
64: return
}
However, the fact that the values() method has to return a new array, doesn't mean the compiler has to use the method. Potentially a compiler could detect use of MyEnumType.values()[ordinal] and, seeing that the array is not modified, it could bypass the method and use the underlying $VALUES array. The above disassembly of the main method shows that javac does not make such an optimization.
I also tested ECJ. The disassembly shows ECJ also initializes a hidden array to store the constants (although the Java langspec doesn't require that), but interestingly its values() method prefers to create a blank array then fill it with System.arraycopy, rather than calling .clone(). Either way, values() returns a new array every time. Like javac, it doesn't attempt to optimize the ordinal lookup:
final class MyEnumType extends java.lang.Enum<MyEnumType> {
public static final MyEnumType A;
public static final MyEnumType B;
public static final MyEnumType C;
private static final MyEnumType[] ENUM$VALUES;
static {};
Code:
0: new #1 // class MyEnumType
3: dup
4: ldc #14 // String A
6: iconst_0
7: invokespecial #15 // Method "<init>":(Ljava/lang/String;I)V
10: putstatic #19 // Field A:LMyEnumType;
13: new #1 // class MyEnumType
16: dup
17: ldc #21 // String B
19: iconst_1
20: invokespecial #15 // Method "<init>":(Ljava/lang/String;I)V
23: putstatic #22 // Field B:LMyEnumType;
26: new #1 // class MyEnumType
29: dup
30: ldc #24 // String C
32: iconst_2
33: invokespecial #15 // Method "<init>":(Ljava/lang/String;I)V
36: putstatic #25 // Field C:LMyEnumType;
39: iconst_3
40: anewarray #1 // class MyEnumType
43: dup
44: iconst_0
45: getstatic #19 // Field A:LMyEnumType;
48: aastore
49: dup
50: iconst_1
51: getstatic #22 // Field B:LMyEnumType;
54: aastore
55: dup
56: iconst_2
57: getstatic #25 // Field C:LMyEnumType;
60: aastore
61: putstatic #27 // Field ENUM$VALUES:[LMyEnumType;
64: return
private MyEnumType(java.lang.String, int);
Code:
0: aload_0
1: aload_1
2: iload_2
3: invokespecial #31 // Method java/lang/Enum."<init>":(Ljava/lang/String;I)V
6: return
public static void main(java.lang.String[]);
Code:
0: getstatic #35 // Field java/lang/System.out:Ljava/io/PrintStream;
3: invokestatic #41 // Method values:()[LMyEnumType;
6: iconst_0
7: aaload
8: invokevirtual #45 // Method java/io/PrintStream.println:(Ljava/lang/Object;)V
11: return
public static MyEnumType[] values();
Code:
0: getstatic #27 // Field ENUM$VALUES:[LMyEnumType;
3: dup
4: astore_0
5: iconst_0
6: aload_0
7: arraylength
8: dup
9: istore_1
10: anewarray #1 // class MyEnumType
13: dup
14: astore_2
15: iconst_0
16: iload_1
17: invokestatic #53 // Method java/lang/System.arraycopy:(Ljava/lang/Object;ILjava/lang/Object;II)V
20: aload_2
21: areturn
public static MyEnumType valueOf(java.lang.String);
Code:
0: ldc #1 // class MyEnumType
2: aload_0
3: invokestatic #59 // Method java/lang/Enum.valueOf:(Ljava/lang/Class;Ljava/lang/String;)Ljava/lang/Enum;
6: checkcast #1 // class MyEnumType
9: areturn
}
However, it's still potentially possible that the JVM could have an optimization that detects the fact that the array is copied and then thrown away, and avoids it. To test that, I ran the following pair of benchmark programs that test ordinal lookup in a loop, one which calls values() each time and the other that uses a private copy of the array. The result of the ordinal lookup is assigned to a volatile field to prevent it being optimized away:
enum MyEnumType1 {
A, B, C;
public static void main(String[] args) {
long t = System.nanoTime();
for (int n = 0; n < 100_000_000; n++) {
for (int i = 0; i < 3; i++) {
dummy = values()[i];
}
}
System.out.printf("Done in %.2f seconds.\n", (System.nanoTime() - t) / 1e9);
}
public static volatile Object dummy;
}
enum MyEnumType2 {
A, B, C;
public static void main(String[] args) {
long t = System.nanoTime();
for (int n = 0; n < 100_000_000; n++) {
for (int i = 0; i < 3; i++) {
dummy = values[i];
}
}
System.out.printf("Done in %.2f seconds.\n", (System.nanoTime() - t) / 1e9);
}
public static volatile Object dummy;
private static final MyEnumType2[] values = values();
}
I ran this on Java 8u60, on the Server VM. Each test using the values() method took around 10 seconds, while each test using the private array took around 2 seconds. Using the -verbose:gc JVM argument showed there was significant garbage collection activity when the values() method was used, and none when using the private array. Running the same tests on the Client VM, the private array was still fast, but the values() method became even slower, taking over a minute to finish. Calling values() also took longer the more enum constants were defined. All this indicates that the values() method really does allocate a new array each time, and that avoiding it can be advantageous.
Note that both java.util.EnumSet and java.util.EnumMap need to use the array of enum constants. For performance they call JRE proprietary code that caches the result of values() in a shared array stored in java.lang.Class. You can get access to that shared array yourself by calling sun.misc.SharedSecrets.getJavaLangAccess().getEnumConstantsShared(MyEnumType.class), but it is unsafe to depend on it as such APIs are not part of any spec and can be changed or removed in any Java update.
Conclusion:
The enum values() method has to behave as if it always allocates a new array, in case callers modify it.
Compilers or VMs could potentially optimize that allocation away in some cases, but apparently they don't.
In performance-critical code, it is well worth taking your own copy of the array.

how to get a list of constant mappings in the java class constant pool?

when I run javap -c Address.class > Address.txt on a particular class,
I get a bunch of output, with #<NUMBER> as the index to the constant pool
Compiled from "Address.java"
public class test.Address extends test.Entity {
public test.Address();
Code:
0: aload_0
1: invokespecial #1 // Method test/DefaultEntity."<init>":()V
4: aload_0
5: aconst_null
6: putfield #2 // Field io:Ljava/lang/String;
9: aload_0
10: aconst_null
11: putfield #3 // Field zip4:Ljava/lang/String;
14: aload_0
15: aconst_null
16: putfield #4 // Field zip:Ljava/lang/String;
19: aload_0
How would I be able to get the pool in the form of a hashmap? i.e:
{1 "Method test/DefaultEntity."<init>":()"
2 "Field io:Ljava/lang/String;"}
With ASM, you can use a ClassReader to read a .class file and then use getItemCount() and the various read methods to read all of the constant pool items.
(def cr (clojure.asm.ClassReader "test.Address"))
(.getItemCount cr)
;; etc

Categories