We Keep Coding

If the return type is not a part of method signature in Java, why must an overriding method have the same as its parent?

Is there any particular reason why Java does not allow method overriding by changing the return type of a sub-class?
Since the return type is not part of the method signature, isn't it the case that a method having a different return types (but having the same name and the same number and type of parameters) are seen as the same by the compiler?

My understanding
Java Language Specification does not consider return type as method signature.
Since return type is not present in signature, two methods can't differ only by return type(as they will be ambiguous to runtime). So compiler will not allow it.
This can be considered along the same lines of type erasure of generic parameters. Since type will not be available at runtime, compiler will block any two methods whose signature(parameters) vary only by generic type to avoid ambiguity at runtime.
Similarly, exceptions are not considered in method signature and hence two methods can not just vary by exceptions. Compiler will not allow it.
Similarly, access modifiers are not considered in method signature and hence two methods can not just vary by access modifiers. Compiler will not allow it.
So the basis is, two methods can not vary only by components that are not part of method signature.
Basically compilers are meant to parse the program according to JLS grammar and block anything that does not follow the grammar. Compilers do have luxury to accommodate some deviations. Still it will not outright allow a violation.
All these are done to ensure that the runtime can execute more freely with high confidence based on the output of compiler.
References
Definition: Two of the components of a method declaration comprise the method signature—the method's name and the parameter types.
Ref: Docs
Two methods have the same signature if they have the same name and argument types.
Ref: JLS
Return types may vary among methods that override each other if the return types are reference types. The notion of return-type-substitutability supports covariant returns, that is, the specialization of the return type to a subtype.
If R1 is void then R2 is void.
If R1 is a primitive type, then R2 is identical to R1.
If R1 is a reference type then:
R1 is either a subtype of R2 or R1 can be converted to a subtype of R2 by unchecked conversion (§5.1.9), or
R1 = |R2|
Ref: JLS

Why is method overload resolution determined at compile time?

I am trying to understand why we have polymorphism / dynamic binding with overridden methods, but not overloaded methods. I understand that there’s indirection that allows us to index into a vtable to allow us to do this in a language like C++ or Java. I’m curious why the same isn’t the case for resolving overloaded methods — my intuition leads me to believe we could have a level of indirection that allows us to determine at runtime which overloaded method to invoke based on the runtime type.
I’m wondering if this design decision was made for performance reasons or if there’s extra complexity I’m neglecting to consider.

I have not read the minds of the language designers, so cannot really tell you. I am thinking of it in this way:
Overridden methods are different implementations of the same method at different levels in the superclass/subclass hierarchy. The subclass generally uses the same method signature (it’s allowed to return a more specific type and to declare fewer exceptions to be thrown, but it cannot completely redefine the method header, or it will no longer be an override).
Overloaded methods are really just different methods that happen to have the same name. Then the parameter types are used for distinguishing. Just like the compiler always decides on compile time which method to call, this is also the case with overloaded methods.
As an observation (possibly a minor one), with runtime resolution of overloaded methods we could no longer statically type check the return value. Imagine we have
public boolean foo(Number n);
public String foo(Integer i);
Now I would find it perfectly natural to call the former foo() like this:
boolean result = foo(myNumber);
Now if myNumber happened to be an Integer, the latter foo() would be called instead. It would return a String and I would have a type conversion error happening on runtime. I would not be amazed.
… why then we can still have runtime polymorphism and it be considered
static typing, but it wouldn't be if we did dynamic resolution of
overloaded methods.
Java has both: a static type and a runtime type. When I store an Integer into a variable declared to be a Number, then Number is the static type and Integer is the runtime type (BTW a type is not the same as a class). And you are correct, when I do myObject.foo(arg), then the runtime type of myObject decides which implementation of foo() gets called. And conceivably the runtime type of arg could have been involved in the decision too. It would get more complicated, and I am unsure about the gain.

Overloaded methods are methods that have the same name but take different parameters. You can simply see the types and order of the parameters it takes (also the type of value it returns) as part of the name.
In C++ this is actually somewhat more "exposed" - this language internally tweaks the names to match the parameters, void h(int, char) becomes something like h__Fic and void h(int) like h__Fi. These mangled names can be occasionally seen in places like error messages about them not being resolved. In Java, this is less exposed but internally comparable. This is also called the "signature".
Then you question simplifies to "if the methods have different names, why are they resolved at the compilation time and not a the run time?". The resolution between the two methods with different signatures cannot change over time, so there is simply no reason and no benefit to delay it.

Express a general `MethodType` with return type as any object

Let the following be a class in my problem:
class MyClass {
String name() {
return toString();
}
}
I want to create an instance of MethodType which describes a method with a return that is "any" object. I am trying the following:
MethodType genericTypeWithObjectReturn =
MethodType.methodType(Object.class, new Class[] {});
then try to find a method using
MethodHandle mh =
lookup.findVirtual(MyClass.class, "name", genericTypeWithObjectReturn);
Using the above line, I get an exception:
Caused by: java.lang.NoSuchMethodError: MyClass.name()Ljava/lang/Object;
My observation is that the method return type should be exactly the same type; namely I should have used String.class in the above statement to define MethodType. Is this correct? Is there a way so that I can do this in the way I described and preferrably not using reflection API?

since an answer is still missing...
The MethodType has to be exact. That's a big pain point in the MethodHandles-API, but only because you control the transformation are some of the optimizations possible. The solution is to use reflection to find the method and then use MethodHandles#unreflect* to get the handle.

There's no way to use the find{Virtual,Static} methods to get method handles without specifying the exact method type, including the return type. This is a consequence of the JVM allowing overloading on return type, even though the Java programming language does not. From the documentation for Class.getMethod (emphasis mine):
To find a matching method in a class or interface C: If C declares exactly one public method with the specified name and exactly the same formal parameter types, that is the method reflected. If more than one such method is found in C, and one of these methods has a return type that is more specific than any of the others, that method is reflected; otherwise one of the methods is chosen arbitrarily.
Note that there may be more than one matching method in a class because while the Java language forbids a class to declare multiple methods with the same signature but different return types, the Java virtual machine does not. This increased flexibility in the virtual machine can be used to implement various language features. For example, covariant returns can be implemented with bridge methods; the bridge method and the method being overridden would have the same signature but different return types.
Thus JVM methods can be made inaccessible to Class.getMethod by the presence of other methods with the same signature but different (not-more-specific) return types. Given that the purpose of method handles is to support other languages on the JVM, including languages that either support overloading on return type in the source or that generate return type overloads as part of mapping to the JVM, this hole had to be fixed by requiring the exact type to be specified. (You could imagine a method pow(int, int) with overloads returning int, long, and BigInteger, selected by the source language based on the context type in foo(pow(1000, 100000));.)
Apart from that hard requirement to make all methods accessible, it's arguably better from an API design standpoint to fail fast if the desired method is not found rather than silently select an argument-compatible but different method.
If you're happy with the lookup semantics of Class.getMethod, you can use it and call MethodHandles.Lookup.unreflect on the returned Method.

yes. you should use String.class for resolving right method type.
as i know you can't implement it with method handles api, but you can use Reflection API.

does Java type erasure erase my generic type?

I've thought java erasure wipes generic types out in compile time however when i test it by myself i realized there are some information about generic types in Bytecode.
here is my test :
i wrote 2 classes:
import java.util.*;
public class Test {
List integerList;
}
and
import java.util.*;
public class Test {
List<Integer> integerList;
}
i compiled both classes and somewhere in generic class i saw this line
integerList{blah blah}Ljava/util/List;{blah blah}
Signature{blah blah}%Ljava/util/List<Ljava/lang/Integer;>;{blah blah}<init>
in non generic class :
integerList{blah blah}Ljava/util/List;{blah blah}<init>
so obviously i have generic information inside bytecode so what is this erasure thing ??

what is this erasure thing ??
Erasure is a mapping from generic to raw types. The common phrase "because of erasure" is essentially meaningless. What is significant are the specifications that use the mapping.
There are two interesting uses.
It's used to map method signatures from using generics to raw types. It is the raw-type signatures that used for overloading. This causes the vast majority of the problems with "erasure". For instance, you can't have two methods add(List<String>) and add(List<Integer>) in the same type. Overloading probably isn't a great idea, and there's not a great willingness to add this feature.
The type available for an instance of an object at runtime is the type it was created with erased. So if you cast to, say, (String) that will be checked at runtime, but if you cast to List<String> only the erasure of that type (List) will be checked. You can have the variables of type List<String> and List<Integer> point to exactly the same instance. In practice, you shouldn't be using casts (of reference types) in 1.5 and later.
Where practical, generic information is kept in class files and made available through reflection. So you'll find it on class definitions, supertypes, fields, methods, constructors, etc.

Some Generic type information is stored in Signature attributes . Refer JLS 4.8 and 4.6 and JVM spec 4.3.4. Read here:
Probably the most common complaint about generics in Java is that they are not reified - there is not a way to know at runtime that a List<String> is any different from a List<Long>. I've gotten so used to this that I was quite surprised to run across Neil Gafter's work on Super Type Tokens. It turns out that while the JVM will not track the actual type arguments for instances of a generic class, it does track the actual type arguments for subclasses of generic classes. In other words, while a new ArrayList<String>() is really just a new ArrayList() at runtime, if a class extends ArrayList<String>, then the JVM knows that String is the actual type argument for List's type parameter.
and Neal Gafter's blog.

This is one instance where accurate use of terminology actually matters: Bytecode is the instruction set of the Java Virtual Machine. A class file contains bytecode, but also information used for linking (field signatures, method signatures, ...), for the bytecode verifier, for the debugger, ...
Type erasure means that generic type informations is not translated into byte code; more specifically, all instances of a generic type share the same representation in byte code. Likewise, the dynamic type of an object the runtime keeps track of (as used by the cast and instanceof operators, and available through getClass()) is the same for all instances of a generic class, irrespective of any type parameters supplied in the source code.
Your experiment proves that generic type information is retained in the class file, more specifically, in the types of method and field signatures. That's unsurprising, because the signatures are actually used at compile time. The might also be used at link time, and are even accessible through the reflection api. The crucial difference is that they are the declared types of fields or methods, not the runtime types of actual objects.
That is, since Java 1.5 we must distinguish between a variable's declared type, and the runtime type of the object it refers to. The former supports generics, the latter does not. And yes, this means there isn't a one-to-one correspondence between compile time and runtime types.

Type info will be erased from here
integerList = new ArrayList<Integer>();
in the bytecode it will be equivalent to
integerList = new ArrayList();
and there is no chance to know in runtime from integerList object what was its compile time type.

Erasure means that generic typing is not incorporated in the byte code (when the list is created or used).
The signature you see is used just to indicate that the field is generic.

What is the difference between a class and a type in Scala (and Java)?

Scala
Where can differences between a class and a type be observed in Scala and why is this distinction important?
Is it only a consideration from the language design point-of-view or has it "practical" impact when programming Scala?
Or is it fundamental to "securing the boundaries" of the type system (Nothing, Null come to my mind)?
Java
How many of the considerations/differences/problems mentioned above can also be recognized in Java?
(See What is the difference between Type and Class? as a language-agnostic introduction.)

When you say "type" I'm going to assume you mean static type mostly. But I'll talk about dynamic types shortly.
A static type is a property of a portion of a program that can be statically proven (static means "without running it"). In a statically typed language, every expression has a type whether you write it or not. For instance, in the Cish "int x = a * b + c - d", a,b,c,and d have types, a * b has a type, a * b + c has a type and a * b + c -d has a type. But we've only annotated x with a type. In other languages, such as Scala, C#, Haskell, SML, and F#, even that wouldn't be necessary.
Exactly what properties are provable depends on the type checker.
A Scala style class, on the other hand, is just the specification for a set of objects. That specification includes some type information and includes a lot of implementation and representation details such as method bodies and private fields, etc. In Scala a class also specifies some module boundaries.
Many languages have types but don't have classes and many languages have classes but don't have (static) types.
There are several observable differences between types and classes. List[String] is a type but not a class. In Scala List is class but normally not a type (it's actually a higher kinded type). In C# List isn't a type of any sort and in Java it's a "raw type".
Scala offers structural types. {def foo : Bar} means any object that provably has a foo method that returns a Bar, regardless of class. It's a type, but not a class.
Types can be abstracted using type parameters. When you write def foo[T](x : T) = ..., then inside the body of foo T is a type. But T is not a class.
Types can be virtual in Scala (i.e. "abstract type members"), but classes can't be virtual with Scala today (although there's a boilerplate heavy way to encode virtual classes https://wiki.scala-lang.org/display/SIW/VirtualClassesDesign)
Now, dynamic types. Dynamic types are properties of objects that the runtime automatically checks before performing certain operations. In dynamically typed class-based OO languages there's a strong correlation between types and classes. The same thing happens on JVM languages such as Scala and Java which have operations that can only be checked dynamically such as reflection and casting. In those languages, "type erasure" more or less means that the dynamic type of most objects is the same as their class. More or less. That's not true of, e.g., arrays which aren't typically erased so that the runtime can tell the difference between Array[Int] and Array[String]. But remember my broad definition "dynamic types are properties of objects that the runtime automatically checks." When you use reflection it is possible to send any message to any object. If the object supports that message then everything works out. Thus it makes sense to talk of all objects that can quack like a duck as a dynamic type, even though it's not a class. That's the essence of what the Python and Ruby communities call "duck typing." Also, by my broad definition even "zeroness" is a dynamic type in the sense that, in most languages, the runtime automatically checks numbers to make sure you don't divide by zero. There are a very, very few languages that can prove that statically by making zero (or not-zero) a static type.
Finally, as other's have mentioned, there are types like int which don't have a class as an implementation detail, types like Null and Any which are a bit special but COULD have classes and don't, and types like Nothing which doesn't even have any values let alone a class.

Okay, I'll bite... James has a good answer, so I'm going to try a different tact and give a more down-to-earth viewpoint.
Broadly speaking, a class is something that can be instantiated. singleton objects (scala) traits (Scala) and interfaces (Scala) are also commonly considered to be classes. This makes sense, as singletons are still instantiated (via compiler-generated code) and an interface can be instantiated as part of a subclass.
Which brings us to the second point. classes are the primary unit of design in most object-oriented languages (though not the prototype-based ones like javascript). Polymorphism and subclassing are both defined in terms of classes. classes also provide a namespace and visibility controls.
types are a very different beast, every possible value that the system can express will have one or more types, and these can sometimes be equated to classes, for example:
(Int) => String // both the type and class are Function1[Int,String]
"hello world" // class and type are String
You also get some interesting differences between Scala and Java:
7 // both the class and type are Int in Scala
// in Java there's no class and the type is Integer.TYPE
println("hello world") // the return type is Unit, of class Unit
// Java has void as a type, but no corresponding class
error("oops") // the type and class are both "Nothing"
and the really fun types that aren't classes at all. For example, this.type always refers to the unique type of this. It's unique to a single instance and isn't even compatible with other instances of the same class.
There are also abstract types and type parameters. For example:
type A // 'A' is an undetermined abstract type
// to be made concrete in a subclass
class Seq[T] { ... } // T is a type, but not a class
Seq is interesting as it's a class, but not a type. More accurately, it's a "type constructor"; something that will construct a valid type when supplied with the necessary type parameter. Another term for type constructors is "higher kinded types", I personally don't like this term, as "type constructor" encourages me to think in terms of supplying types like any other form of argument - a mental model that has served me well for Scala.
"higher-kinded" rightly implies that Seq has a "kind", which is * => *, this notation states that Seq will take a single type and yield a single type (this is similar to curried notation for describing functions). By way of comparison, the kind of Map is * => * => * because it takes two type parameters.

A type can be useful by itself, without any instances. One example for this is called "phantom type". Here is an example for Java: http://michid.wordpress.com/2008/08/13/type-safe-builder-pattern-in-java/
In that example we have public static class Initializer<HA, HB>, where HA and HB take some types (represented by the abstract classes TRUE and FALSE), without ever beeing instantiated.
I hope this shows that types and classes are something different, and that types can be useful by itself.

(Java only) I'd say, a type is a set of objects. Object o is type X, if o is a member of set X. Type X is a subtype of Y, if set X is a subset of Y.
For every class C (not interface) there is a set of objects, created from new C(...). Interestingly, we rarely cares about this set. (but every object does belong to a set like this, a fact that may be useful)
For every class C, there is a type t(C), generally refered to as "the type C", which is the set of all objects that can be created from new S(...) where S is C or a subclass of C.
Similarly, for every interface I, there is a type t(I), "the type I", which is the set of all objects that can be created from new S(...) where S implements I.
Obviously, if class S is a subclass of C, type S is a subtype of type C. Similar for interface I
There is a null type, which is the empty set. The null type is a subtype of every type.
There is a set of all objects, which is the type Object. It's a super type of every type.
So far, this formalism is pretty useless. A type is basically the same as a class or an interface, and the subtype relation is basically the subclass/subinterface relation. The triviality is a good thing, the language was understandable! But entering generics, there are more complicated types, and operations like unions and intersections of types. Types are no longer only classes and interfaces, and subtype relations are much richer and harder to understand.