Related
Preface
I have been experimenting with ByteBuddy and ASM, but I am still a beginner in ASM and between beginner and advanced in ByteBuddy. This question is about ByteBuddy and about JVM bytecode limitations in general.
Situation
I had the idea of creating global mocks for testing by instrumenting constructors in such a way that instructions like these are inserted at the beginning of each constructor:
if (GlobalMockRegistry.isMock(getClass()))
return;
FYI, the GlobalMockRegistry basically wraps a Set<Class<?>> and if that set contains a certain class, then isMock(Class<?>> clazz) would return true. The advantage of that concept is that I can (de)activate global mocking for each class during runtime because if multiple tests run in the same JVM process, one test might need a certain global mock, the next one might not.
What the if(...) return; instructions above want to achieve is that if mocking is active, the constructor should not do anything:
no this() or super() calls, → update: impossible
no field initialisations, → update: possible
no other side effects. → update: might be possible, see my update below
The result would be an object with uninitialised fields that did not create any (possibly expensive) side effects such as resource allocation (database connection, file creation, you name it). Why would I want that? Could I not just create an instance with Objenesis and be happy? Not if I want a global mock, i.e. mock objects I cannot inject because they are created somewhere inside methods or field initialisers I do not have control over. Please do not worry about what method calls on such an object would do if its instance fields are not properly initialised. Just assume I have instrumented the methods to return stub results, too. I know how to do that already, the problem are only constructors in the context of this question.
Questions / problems
Now if I try to simulate the desired result in Java source code, I meet the following limitations:
I cannot insert any code before this() or super(). I could mitigate that by also instrumenting the super class hierarchy with the same if(...) return;, but would like to know if I could in theory use ASM to insert my code before this() or super() using a method visitor. Or would the byte code of the instrumented class somehow be verified during loading or retransformation and then rejected because the byte code is "illegal"? I would like to know before I start learning ASM because I want to avoid wasting time for an idea which is not feasible.
If the class contains final instance fields, I also cannot enter a return before all of those fields have been initialised in the constructor. That might happen at the very end of a complex constructor which performs lots of side effects before actually initialising the last field. So the question is similar to the previous one: Can I use ASM to insert my if(...) return; before any fields (including final ones) are initialised and produce a valid class which I could not produce using javac and will not be rejected when loaded or retransformed?
BTW, if it is relevant, we are talking about Java 8+, i.e. at the time of writing this that would be Java versions 8 to 14.
If anything about this question is unclear, please do not hesitate to ask follow-up questions, so I can improve it.
Update after discussing Antimony's answer
I think this approach could work and avoid side effects, calling the constructor chain but avoiding any side effects and resulting in a newly initialised instance with all fields empty (null, 0, false):
In order to avoid calling this.getClass(), I need to hard-code the mock target's class name directly into all constructors up the parent chain. I.e. if two "global mock" target classes have the same parent class(es), multiple of the following if blocks would be woven into each corresponding parent class, one for each hard-coded child class name.
In order to avoid any side effects from objects being created or methods being called, I need to call a super constructor myself, using null/zero/false values for each argument. That would not matter because the next parent class up the chain would have a similar code block so that the arguments given do not matter anyway.
// Avoid accessing 'this.getClass()'
if (GlobalMockRegistry.isMock(Sub.class)) {
// Identify and call any parent class constructor, ideally a default constructor.
// If none exists, call another one using default values like null, 0, false.
// In the class derived from Object, just call 'Object.<init>'.
super(null, 0, false);
return;
}
// Here follows the original byte code, i.e. the normal super/this call and
// everything else the original constructor does.
Note to myself: Antimony's answer explains "uninitialised this" very nicely. Another related answer can be found here.
Next update after evaluating my new idea
I managed to validate my new idea with a proof of concept. As my JVM byte code knowledge is too limited and I am not used to the way of thinking it requires (stack frames, local variable tables, "reverse" logic of first pushing/popping variables, then applying an operation on them, not being able to easily debug), I just implemented it in Javassist instead of ASM, which in comparison was a breeze after failing miserably with ASM after hours of trial & error.
I can take it from here and I want to thank user Antimony for his very instructive answer + comments. I do know that theoretically the same solution could be implemented using ASM, but it would be exceedingly difficult in comparison because its API is too low level for the task at hand. ByteBuddy's API is too high level, Javassist was just right for me in order to get quick results (and easily maintainable Java code) in this case.
Yes and no. Java bytecode is much less restrictive than Java (source) in this regard. You can put any bytecode you want before the constructor call, as long as you don't actually access the uninitialized object. (The only operations allowed on an uninitialized this value are calling a constructor, setting private fields declared in the same class, and comparing it against null).
Bytecode is also more flexible in where and how you make the constructor call. For example, you can call one of two different constructors in an if statement, or you can wrap the super constructor call in a "try block", both things that are impossible at the Java language level.
Apart from not accessing the uninitialized this value, the only restriction* is that the object has to be definitely initialized along any path that returns from the constructor call. This means the only way to avoid initializing the object is to throw an exception. While being much laxer than Java itself, the rules for Java bytecode were still very deliberately constructed so it is impossible to observe uninitialized objects. In general, Java bytecode is still required to be memory safe and type safe, just with a much looser type system than Java itself. Historically, Java applets were designed to run untrusted code in the JVM, so any method of bypassing these restrictions was a security vulnerability.
* The above is talking about traditional bytecode verification, as that is what I am most familiar with. I believe stackmap verification behaves similarly though, barring implementation bugs in some versions of Java.
P.S. Technically, Java can have code execute before the constructor call. If you pass arguments to the constructor, those expressions are evaluated first, and hence the ability to place bytecode before the constructor call is required in order to compile Java code. Likewise, the ability to set private fields declared in the same class is used to set synthetic variables that arise from the compilation of nested classes.
If the class contains final instance fields, I also cannot enter a return before all of those fields have been initialised in the constructor.
This, however, is eminently possible. The only restriction is that you call some constructor or superconstructor on the uninitialized this value. (Since all constructors recursively have this restriction, this will ultimately result in java.lang.Object's constructor being called). However, the JVM doesn't care what happens after that. In particular, it only cares that the fields have some well typed value, even if it is the default value (null for objects, 0 for ints, etc.) So there is no need to execute the field initializers to give them a meaningful value.
Is there any other way to get the type to be instantiated other than this.getClass() from a super class constructor?
Not as far as I am aware. There's no special opcode for magically getting the Class associated with a given value. Foo.class is just syntactic sugar which is handled by the Java compiler.
I always wondered, since a singleton allows use to have just have one reference to an object, which we get by using the static method getInstance, why can’t we decide to make all fields in a singleton static?
Static members are part of class and thus remain in memory till
application terminates and can’t be ever garbage collected. Using
excess of static members sometime predicts that you fail to design
your product and trying to cop of with static / procedural
programming. It denotes that object oriented design is compromised.
This can result in memory over flow. Also there are certain
disadvantages if you make any method static in Java for example you
can not override any static method in Java so it makes testing harder
you can not replace that method with mock. Since static method
maintains global state they can create subtle bug in concurrent
environment which is hard to detect and fix.
So by making every method static we eliminate the purpose of making singleton class which is to Saves memory
I recently read this article by Jake Wharton. This is targeted at Android, but also perfectly valid for java I think.
Consider the following code:
public class A {
private String someField = "abc";
class B {
public void doSomething() {
System.out.println(someField);
}
}
}
We have a simple class A and an inner class B (This has a reference to A and can acccess its members). Class B is accessing the field someField in A even though it is private. According to the article, this is done by the compiler generating synthetic accessor methods which allow the inner class to access the field.
Now my way more basic question: Why does the compiler even care about visiblities when the code is compiled? As we have seen in this example, a new method is generated, which basically just introduces overhead.
Visibilities are a great thing for architecting good software, but if the compiler is done checking that everything is correct according to the declared visibilies, why not optimize those methods away (E.g. just taking everything for being public and allow calls to it)?
First I thought this was due to security reasons, but reflection allows the access of all fields, not caring for visibility as far as I know.
This might be a basic misunderstanding of me, if so, I would be happy if someone could explain it to me.
Why does the compiler even care about visiblities when the code is compiled?
The JVM doesn't allows access to private methods/constructors/fields outside a class. It has no special rule for nested classes which were added after this rule was designed. Instead the compiler adds accessor methods so the language can support a means of access the JVM doesn't.
As we have seen in this example, a new method is generated, which basically just introduces overhead.
Only if the method isn't called very much and isn't optimised.
Adding any simple method (On Hotspot, any method of 35 bytes or less) will be inlined very quickly and has no impact on performance (except if the maximum inline level is reached)
why not optimize those methods away
It does this at runtime so that previous rules continue to be enforced.
reflection allows the access of all fields, not caring for visibility as far as I know.
Though not by default, you have to explicitly want this as an option and not have a SecurityManager which prevents it.
If compiler converts all private fields to public fields in compile time, one problem will be happened when your project is compiled into library and reused by others. In that case, all your private fields will become public.
There are some tools for optimizing this. In Android, there is a tool named ProGuard that will convert all getter/setter to direct field access.
I have a utility class which has non static methods with no instance variables. So I am thinking of converting all the methods to static methods. I doubt there will be any memory or performance impacts. But I just wanted to confirm.
Will changing such a method to be a static have any performance impact on the program?
One final thing to add to what people have said here.
Using a static method has a slightly less overhead due to the fact that you have guaranteed compile time binding. Static method calls will create the bytecode instruction invokestatic. ]
In a typical scenario, instance methods are bound at runtime, and will create the bytecode instruction invokevirtual which has higher overhead than invokestatic.
However, this only becomes relevant in the case of likely millions of iterations, and i would caution against this driving your class design. Do what makes sense from a design perspective. Based on your description, static methods are probably the way to go. In fact, this is relatively standard practice to create a utility class:
public class MyUtilities {
private MyUtilities() { } // don't let anyone construct it.
public static String foo(String s) { ... }
}
EDIT: Addressing the performance aspect: it's cheaper not to have to create an instance of something pointlessly, but the difference is very likely to be completely irrelevant. Focusing on a clear design is much more likely to be important over time.
Utility methods are frequently static, and if all the methods within a class are static it may well be worth making the class final and including a private constructor to prevent instantation. Fundamentally, with utility classes which don't represent any real "thing" it doesn't make logical sense to construct an instance - so prevent it.
On the other hand, this does reduce flexibility: if any of these utility methods contain functionality which you may want to vary polymorphically (e.g. for testing purposes) then consider leaving them as instance methods - and try to extract some meaningful class name to represent the "thing" involved. (For example, a FooConverter makes sense to instantiate - a FooUtil doesn't.)
There are two requirements that must be met for a method to be eligible for conversion into static:
no instance variables accessed (this is met in your case);
will never need to be subject to overriding (for this you may have to think it through).
However, when these requirements are met, it is actually recommended to make the method static because it narrows down the context the method is run within.
Finally, note that there are no performance issues to talk about here and any theoretical difference is in fact in favor of static methods since they don't involve dynamic method resolution. However, instance method invocation is blazing fast in any relevant JVM implementation.
As far as memory, the story is the same: a theoretical difference is in favor of the static method, but there is no practical difference if compared against a singleton utility class.
If the utility class is not subclassed, converting methods that do not access the instance variables to static is a good idea. You should go through the code and convert invocations to static syntax, i.e.
int res = utilityInstance.someMethod(arg1, arg2);
should be converted to
int res = UtilityClass.someMethod(arg1, arg2);
for clarity.
There will be no noticeable performance impact: although theoretically static invocations are slightly less expensive, the difference is too small to consider important in most scenarios.
It is common for utility classes without state(like java.lang.Math for example) to have public static methods. This way you don't need to create an instance of the class to use it.
Static method good idea when you are going to use the particular functionality very often.
The difference is that you need an instance in order to use them, so the user has to make an instance which will be a
Lets assume that a rule (or rule of thumb, anyway), has been imposed in my coding environment that any method on a class that doesn't use, modify, or otherwise need any instance variables to do its work, be made static. Is there any inherent compile time, runtime, or any other disadvantage to doing this?
(edited for further clarifications)
I know the question was somewhat open ended and vague so I apologize for that. My intent in asking was in the context of mostly "helper" methods. Utility classes (with private CTORs so they can't be instantiated) as holders for static methods we already do. My question here was more in line of these little methods that HELP OUT the main class API.
I might have 4 or 5 main API/instance methods on a class that do the real work, but in the course of doing so they share some common functionality that might only be working on the input parameters to the API method, and not internal state. THESE are the code sections I typically pull out into their own helper methods, and if they don't need to access the class' state, make them static.
My question was thus, is this inherently a bad idea, and if so, why? (Or why not?)
In my opinion, there are four reasons to avoid static methods in Java. This is not to say that static methods are never applicable, only to say that they should generally be avoided.
As others have pointed out, static methods cannot be mocked out in a unit test. If a class is depending on, say, DatabaseUtils.createConnection(), then that dependent class, and any classes that depend on it, will be almost impossible to test without actually having a database or some sort of "testing" flag in DatabaseUtils. In the latter case, it sounds like you actually have two implementations of a DatabaseConnectionProvider interface -- see the next point.
If you have a static method, its behavior applies to all classes, everywhere. The only way to alter its behavior conditionally is to pass in a flag as a parameter to the method or set a static flag somewhere. The problem with the first approach is that it changes the signature for every caller, and quickly becomes cumbersome as more and more flags are added. The problem with the second approach is that you end up with code like this all over the place:
boolean oldFlag = MyUtils.getFlag();
MyUtils.someMethod();
MyUtils.setFlag( oldFlag );
One example of a common library that has run into this problem is Apache Commons Lang: see StringUtilsBean and so forth.
Objects are loaded once per ClassLoader, which means that you could actually have multiple copies of your static methods and static variables around unwittingly, which can cause problems. This usually doesn't matter as much with instance methods, because the objects are ephemeral.
If you have static methods that reference static variables, those stay around for the life of the classloader and never get garbage collected. If these accumulate information (e.g. caches) and you are not careful, you can run into "memory leaks" in your application. If you use instance methods instead, the objects tend to be shorter-lived and so are garbage-collected after a while. Of course, you can still get into memory leaks with instance methods too! But it's less of a problem.
Hope that helps!
The main disadvantage is that you cannot swap, override or choose method implementations at runtime.
The performance advantage is likely negligible. Use static methods for anything that's not state dependent. This clarifies the code, as you can immediately see with a static method call that there's no instance state involved.
Disadvantage -> Static
Members are part of class and thus remain in memory till application terminates.and can't be ever garbage collected. Using excess of static members sometime predicts that you fail to design your product and trying to cop of with static /procedural programming. It denotes that object oriented design is compromised.This can result in memory over flow.
I really like this question as this has been a point I have been debating for last 4 years in my professional life. Static method make a lot of sense for classes which are not carrying any state. But lately I have been revised my though somewhat.
Utility classes having static methods is a good idea.
Service classes carrying business logic can be stateless in many cases. Initially I always added static methods in them, but then when I gained more familiarity with Spring framework (and some more general reading), I realized these methods become untestable as an independent unit as u cannot inject mock services easily into this class. E.g. A static method calling another static method in another class, there is no way JUnit test can short circuit tis path by injecting a dummy implementation at run time.
So I kind of settled to the thought that having utility static methods which do not need to call other classes or methods pretty much can be static. But service classes in general should be non static. This allows you to leverage OOPs features like overriding.
Also having a singleton instance class helps us to make a class pretty much like a static class still use OOPs concepts.
It's all a question of context. Some people have already given examples where static is absolutely preferable, such as when writing utility functions with no conceivable state. For example, if you are writing a collection of different sort algorithms to be used on arrays, making your method anything but static just confuses the situation. Any programmer reading your code would have to ask, why did you NOT make it static, and would have to look to see if you are doing something stateful to the object.
public class Sorting {
public static void quiksort(int [] array) {}
public static void heapsort(int[] array) { }
}
Having said that, there are many people who write code of some kind, and insist that they have some special one-off code, only to find later that it isn't so. For example, you want to calculate statistics on a variable. So you write:
public class Stats {
public static void printStats(float[] data) { }
}
The first element of bad design here is that the programmer intends to just print out the results, rather than generically use them. Embedding I/O in computation is terrible for reuse. However, the next problem is that this general purpose routine should be computing max, min, mean, variance, etc. and storing it somewhere. Where? In the state of an object. If it were really a one-off, you could make it static, but of course, you are going to find that you want to compute the mean of two different things, and then it's awfully nice if you can just instantiate the object multiple times.
public class Stats {
private double min,max,mean,var;
public void compute(float data[]) { ... }
public double getMin() { return min; }
public double
}
The knee jerk reaction against static is often the reaction of programmers to the stupidity of doing this sort of thing statically, since it's easier to just say never do that than actually explain which cases are ok, and which are stupid.
Note that in this case, I am actually using the object as a kind of special-purpose pass by reference, because Java is so obnoxious in that regard. In C++, this sort of thing could have been a function, with whatever state passed as references. But even in C++, the same rules apply, it's just that Java forces us to use objects more because of the lack of pass by reference.
As far as performance goes, the biggest performance increase of switching from a regular method is actually avoiding the dynamic polymorphic check which is the default in java, and which in C++ is specified manually with virtual.
When I tried last there was a 3:1 advantage of calling a final method over a regular method, but no discernible for calling static functions over final.
Note that if you call one method from another, the JIT is often smart enough to inline the code, in which case there is no call at all, which is why making any statement about exactly how much you save is extremely dangerous. All you can say is that when the compiler has to call a function, it can't hurt if it can call one like static or final which requires less computation.
The main problem you may face is, you won't be able to provide a new implementation if needed.
If you still have doubts ( whether your implementation may change in the future or not ) you can always use a private instance underneath with the actual implementation:
class StringUtil {
private static StringUtil impl = new DefaultStringUtil();
public static String nullOrValue( String s ) {
return impl.doNullOrValue();
}
... rest omitted
}
If for "some" reason, you need to change the implementation class you may offer:
class StringUtil {
private static StringUtil impl = new ExoticStringUtil();
public static String nullOrValue( String s ) {
return impl.doNullOrValue(s);
}
... rest omitted
}
But may be excessive in some circumstances.
No, actually the reason for that advice is that it provides a performance advantage. Static methods can be called with less overhead so any method that doesn't need a reference to this ought to be made static.
No there is no disadvantages, rather when you are not accessing any instance members in the method then there is no meaning of having it as an instance method. It is good programming skill to have it as a static method.
and adding to that you don't have to create any instances to access these methods and thus saving a memory and garbage collecting time.
In order to call the static methods you don't need to create class objects. The method is available immediately.
Assuming the class is already loaded. Otherwise there's a bit of a wait. :-)
I think of static as a good way to separate the functional code from procedural/state-setting code. The functional code typically needs no extension and changes only when there are bugs.
There's also the use of static as an access-control mechanism--such as with singletons.
One disadvantage is if your static methods are general and distributed in different classes as far as usage is concerned. You might consider putting all static methods that are general in a utility class.
There shouldn't be any disadvantages--there may even be a slight advantage in performance (although it wouldn't be measurable) since the dynamic lookup can be avoided.
It's nice to tag functions as functions instead of having them look like Methods--(and static "Methods" ARE functions, not methods--that's actually by definition).
In general a static method is a bad OO code smell--it probably means that your OO model isn't fully integrated. This happens all the time with libraries that can't know about the code that will be using it, but in integrated non-library code static methods should be examined to evaluate which of it's parameters it's most closely associated with--there is a good chance it should be a member of that class.
If a static method just takes native values, then you're probably missing a handful of classes; you should also keep passing native variables or library objects (like collections) to a minimum--instead containing them in classes with business logic.
I guess what I'm saying is that if this is really an issue, you might want to re-examine your modeling practices--statics should be so rare that this isn't even an issue.
As others have said, it provides a slight performance advantage and is good programming practice. The only exception is when the method needs to be an instance method for overriding purposes, but those are usually easily recognised. For example if a class provides default behaviour of an instance method, that happens not to need instance variables, that clearly can't be made static.
In general:
You should be writing your software to take advantage of interfaces and not implementations. Who's to say that "now" you won't use some instance variable, but in the future you will? An example of coding to interfaces...
ArrayList badList = new ArrayList(); //bad
List goodList = new ArrayList(); //good
You should be allowed to swap implementations, especially for mocking & testing. Spring dependency injection is pretty nice in this respect. Just inject the implementation from Spring and bingo you have pretty much a "static" (well, singleton) method...
Now, those types of APIs that are purely "utility" in purpose (i.e., Apache Commons Lang) are the exception here because I believe that most (if not all) of the implementations are static. In this situation, what are the odds that you will want to ever swap Apache Commons out for another API?
Specifically:
How would you elegantly handle the "staticness" of your implementation when you're targeting, say, a Websphere vs. Tomcat deployment? I'm sure there would be an instance (no pun intended) of when your implementation would differ between the two...and relying on a static method in one of those specific implementations might be dangerous...