We Keep Coding

Java - best way to detect NPE at compile time when not allowed to use #NotNull or #Nullable annotations

I used to use lots of #NotNull/#Nullable annotations to enable IDE to help me find out potential NPE at compile time. However, my new team doesn't allow any use of #NotNull/#Nullable annotations, and nor are custom annotations like those allowed. As a result, I become much more likely to write bugs caused by NPE than before.
I have tried several solutions:
Use Optional<T> in java 8. However, this is not doing well for every case. It's usually not recommended to use Optional<T> as the type of fields or arguments. It's also very frustrating that Optional<T> instance itself could be null. Also, it's difficult to operate control flows inside lambda expressions when calling ifPresent(obj->...)(it's easier in Java 9). And using too many Optional<T>s makes the code a little verbose.（UPDATE: Unfortunately, Optional<T> is also banned from using now)
Make IDE treat every unannotated instance as #Nullable. This solution does help me find out some potential bugs, however, IDE would suggest me to check almost every method invocation, which is really annoying, since many methods are designed intentionally not returning null.
Check every method invocations. This is a viable solution, however it has a severe influence that the possibility of being null would be passed everywhere through method invocations. Under such circumstances, every argument of a method is possible to be null, and when an argument is checked for nullity, the method would usually return a null continuously. Finally, every method is "infected" with the possibility of receiving null arguments and returning null.
Call Objects.requireNonNull() to prevent the problem aforementioned. It slightly reduces the pain of checking null everywhere. However, it provides no guarantee that the caller won't pass a null to the cases when null is not allowed. And once null is passed, the runtime NPE thrown is much more likely to ruin your application.
Switch to kotlin. Of course, it's not allowed:)
Is there other suggestions about detecting NPE at compile time (and save my job)? I think a solution to this problem could be widely used, not only helping myself, since not all teams allow usage of #NotNull/#Nullable annotations and Optional<T>s.

Not a definitive answer but a few way to explore:
Tools like findBug, PMD , Coverity ... are quite good at this exercice.
Check the reason why you cannot use the annotation in your team. Maybe you can make your team change its mind? (There might be a good reason too.)
Eclipse IDE is reasonnably good at this exercice. Did you try the options in 'Windows > Preferences > Java > Compiler > Errors/Warnings > Null analysis'?
Unit testing. Cover the 'should-not-happen' situation in a few test cases. Code coverage tools (like JaCoCo) might be helpfull as wel.
Defensive programming (plain 'if' statement, assertions ...)
A mix of the previous items
Hope this helps.

You did not say what IDE you are using but Eclipse and Intellij support external annotations. With them, you can still annotate your code locally and have the IDE provide null analysis.
Intellij Documentation
Eclipse Documentation

You can probably find a way to use Optional<T> that suits your cases.
Let's say you have to write a method U doStuff(T param), and that param is given to your code by some library from other person and it may be null.
Inside the body of the method doStuffyou can do
return Optional.ofNullable(param)
.map(notNullParam -> doThing(notNullParam))
.orElse(null);
This way, you are sure that your Optionalitself is not null, people can still use your API with parameters that are null, and you return null when it makes sense for them.

I recommend checking out SonarLint which is a very nice tool to detect all kind of bugs and errors including NPE.

What you should do - is come up with a good API design and specify which of the layers are allowed to return null and which not.
In general, you should always try to return something else rather than null - empty collection, an object representing not-found.
So you should check for nulls manually where the return could make sense to be null. Otherwise, just convene to not return null and just don't check.
You can also introduce some utility methods to check for null/empty collections:
Check.isEmpty() would return true if collection is null or is not null but is empty and so on...
You could also use the null-object pattern

I can relate your problem to my use cases as well. In those cases feel free to suggest code coverage tools.
JaCoCo is very handy. If you use IntelliJ, inbuilt Jacoco/IJ's code coverage clearly gives you the violations and possible NPE
Unit tests to support your domain wise non-nullable object scenarios
After all, NPE is still either design based/programmer based fault. Keep your code well tested and proper checks to avoid NPE and throw custom exceptions or IllegalArgumentException if your service expects a valid value.

Error checking in business methods a.k.a doing defensive programming

I am starting to add tests to a large java code base. I often see the following in the session beans I am testing:
public OrderDTO getOrderDTO(Long id) {
Order o = (Order)entityManager.find(Order.class, id);
OrderDTO dto = new OrderDTO(o.getId(), o.getCurrency());
return dto;
}
Its quit easy to write a unit test to break this code (send in a null or a non existing id). When I did that half the developers of the team said:
We are not error checking everything. If you parameter is rubbish you will know fast!
The other half said:
We must add ifs to the id and then to the o and if any of them are null the null is what we return.
Isn't the point of unit testing to find exactly thees kind of issues?
(Yes, I am asking for an opinion!)
Yes, switching from Long to long will remove one if.

While this is somewhat opinion based, few people would say it's correct to return null if given null as a parameter. If I were to add anything, it would be at most a IllegalArgumentException (or even NPE) when null is passed in.
A test could be created to check that the method fails in a consistent fashion, but it would really be testing the behaviour of the JPA provider and not your code.

Returning nulls should be avoided, they are the source of all evil.
You could use the Null Object Pattern.
Or throw an exception, illegalargument or entitynotexsits spring to mind.
If you must return null, at least wrap it in an optional or use guava.

As always, it depends :)
If you are writing library code (code that is shared and used elsewhere or even by others), then you should really aim for handling all thinkable input values in a consistent way. Using well documented exceptions instead of returning null is certainly preferable for library code.
On the other hand there is local code.
I can see the method is public, but that does not eliminate the possibility that it's only used in one isolated portion of the code base.
In that case, you don't work with assumptions about parameters and what the caller expects in return. You work with defined calls. You control what is sent in and how the caller handles the return value. So it's OK not to null-check, if you know the caller never sends null. And it can be OK to return null as well if that simplifies your overall program structure.
ps: what bothers me most in that method is the unhandled NPE if entityManager.find(..) fails :)

Benefits of using Validate in Java over doing manual error checking?

I am modifying some code and I am seeing consistent use of "validate" for error checking.
What is the difference between Validate.notNull(x, "x cannot be empty") and if(null != x){...} and what are the advantages of one over the other?
EDIT: I am using org.apache.commons.lang.Validate

edit: assuming you are referring to org.apache.commons.lang.Validate
The first says that calling the method with a null parameter is entirely illegal, it was a coding mistake on the part of the programmer who used the method.
The second is just saying that there are different behaviours depending on if x is null or not.
It becomes just a readability issue if you're actually doing this:
if(null != x)
{
//...
} else {
throw new IllegalArgumentException( "x cannot be empty");
}
How long is the body of that if going to be? Why is the consequence of calling the method illegally physically separated from the check for if it was called illegally? As soon as you add some more parameters, that have various different legal states, this can get really hard for others to understand. It is much friendlier to other programmers if you explicitly reject every invocation that is known to be absolutely illegal right at the start.

Validate does not belong to the Java API, you are probably using a framework or somebody else's class to do validations.
Regarding the benefits, it depends on what Validate.notNull is doing internally, hopefully it would be consistent with its name and would be doing only what is says, check if a value is not null. A benefit is that it allows you to specify an error message so you can reduce the verbosity of your code, but again it depends on what it does with that message, it could be adding them to a stack of messages or throwing an unchecked exception.

Where to catch and process null arguments?

When I code, I often ask myself the same question :
Do I have to verify all arguments are not null ? So, in each method, I will have something like that :
if (arg1 == null)
{
throw FooException("...");
}
if (arg2 == null)
{
throw FooException("...");
}
If not, in which case is preferable ?
What's the best practices ?

As always, it depends.
If you're writing an API to be used by other teams / organizations, such defensive programming with precondition checks on public functions can really help your users; when using an external library, a meaningful error message like 'argument passed to foo() should not be null' is way better than NullPointerException thrown from some inner class.
Outside of API, though, I think such checks clutter the code too much. Thrown NullPointerExceptions are usually pretty easy to trace with debugger anyway. In languages that support them, you can consider using assertions - their syntax is usually less cumbersome, and you can turn them off on production so the checks won't degrade performance.

Unfortunetly, yes. you should check all arguments. Now ideally, if you code with good design practices one function should not have more than 4 or 5 arguments, at the most.
Having said that, one should always check for null values in function entry and throw appropriate exception or IllegalArgumentException (my fav).
Furhter, one should never pass NULL to a function and should never return a NULL. Sounds simple but it will save lots of code and bugs. Have a look at the NULL Design Pattern too.

Depends, if you want different exceptions i guess you would have to do that for all occasions where you might get a null value. Another way would be to user DATATYP.TryParse(). Look that up.
Hope it helps.

Since you're throwing an exception anyway, not verifying them would probably just lead to a nullpointerexception or something similar. I'm not entirely sure what the best practices are myself.

You should ideally always verify any arguments before you perform any action that might modify any state or data associated with said arguments. It's better to fail early and in a manageable way (by throwing a exception) than to end up with an inconsistent state / data which then also has to be resolved.
Your methods are expecting certain data to be there, there are some cases when it should be safe to assume that it is actually there (say inside a private method, which is called from other methods which validate input). However in general I would recommend validating arguments whenever they are:
Supplied by a user.
Supplied as part of an API.
Passed between modules of a system .
It's might be worth taking a look at this previous StackOverflow question.

I feel it's mostly down to common sense, and a little down to personal preference.
As others have mentioned, if it's a public API then you want to provide clear error messages wherever possible, so it's best to check parameters before they are used and throw exceptions with messages as per your example.
If it's internal code then there are two other options to think about: use assertions, or don't bother with the validation and rely on debugging. As a rule of thumb, I'll put assertions in if it's code that I expect other developers will be calling or if the condition is subtle enough that debugging it might be a pain. Otherwise, I'll just allow it to fail.
Sometimes you can avoid the issue by using the Null Object pattern. If it's a public API I'd still be inclined to include the checks though.

How to show if a method may return null

After posting this question and reading that one I realized that it is very important to know if a method is supposed to return null, or if this is considered an error condition and an exceptions should be thrown. There also is a nice discussion when to return ‘null’ or throw exception .
I'm writing a method and I already know if I want to return null or throw an exception, what is the best way to express my decision, in other words, to document my contract?
Some ways I can think of:
Write it down in the specs / the documentation (will anyone read it?)
Make it part of the method name (as I suggested here)
assume that every method that throws an exception will not return null, and every one that does 'not' throw might return null.
I'm mainly talking about java, but it might apply to other languages, too: Why is there a formal way to express if exceptions will be thrown (the throws keywords) but no formal way to express if null might be returned?
Why isn't there something like that:
public notnull Object methodWhichCannotReturnNull(int i) throws Exception
{
return null; // this would lead to a compiler error!
}
Summary and Conclusion
There are many ways to express the contract:
If your IDE supports it (as IntelliJ), it's best to use an annotation like #NotNull because it is visible to the programmer and can be used for automated compile time checking. There's a plugin for Eclipse to add support for these, but it didn't work for me.
If these are not an option, use custom Types like Option<T> or NotNull<T>, which add clarity and at least runtime checking.
In any way, documenting the contract in the JavaDoc never hurts and sometimes even helps.
Using method names to document the nullability of the return value was not proposed by anyone but me, and though it might be very verbose und not always useful, I still believe sometimes it has its advantages, too.

A very good follow up question. I consider null a truly special value, and if a method may return null it must clearly document in the Javadoc when it does (#return some value ..., or null if ...). When coding I'm defensive, and assume a method may return null unless I'm convinced it can't (e.g., because the Javadoc said so.)
People realized that this is an issue, and a proposed solution is to use annotations to state the intention in a way it can be checked automatically. See JSR 305: Annotations for Software Defect Detection, JSR 308: Annotations on Java Types and JetBrain's Nullable How-To.
Your example might look like this, and refused by the IDE, the compiler or other code analysis tools.
#NotNull
public Object methodWhichCannotReturnNull(int i) throws Exception
{
return null; // this would lead to a compiler error!
}

You can use the Option type, which is very much like a list that has zero or one element. A return type of Option<Object> indicates that the method may return an Object, or it may return a special value of type None. This type is a replacement for the use of null with better type checks.
Example:
public Option<Integer> parseInt(String s) {
try {
return Option.some(Integer.parseInt(s));
}
catch (Exception e) {
return Option.none();
}
}
If you use this consistently, you can turn on IDE null-warnings, or just use grep for null which should not appear in your code at all if you use Option.none() everywhere you would normaly use a null literal.
Option comes standard with Scala, and it is called Maybe in Haskell. The link above is to a library called Functional Java that includes it. That version implements the Iterable interface, and has monadic methods that let you compose things nicely. For example, to provide a default value of 0 in case of None:
int x = optionalInt.orSome(0);
And you can replace this...
if (myString != null && !"".equals(myString))
...with this, if you have an Option<String>...
for (String s : myOptionString)

There's some support for a #Nullable and #NotNull annotation in IntelliJ IDEA. There's also some talk about adding those annotations (or a similar feature) to Java 7. Unfortunately I don't know how far that got or if it's still on track at all.

Indeed: in our framework we have a 'non-null' pointer type, which may be returned to indicate that the method will always return a value.
I see three options:
wait for language support to express it (e.g. the C# ?! thing)
use Aspect Orientation to build your own language extensions to express it
use a custom type to express it
(but builds on developer cooperation) use a naming scheme to indicate it

For Java, one can use the Javadoc description of a method to document the meaning of the returned value, including whether it can be null. As has been mentioned, annotations may also provide assistance here.
On the other hand, I admit that I don't see null as something to be feared. There are situations in which "nobody's home" is a meaningful condition (although the Null Object technique also has real value here).
It is certainly true that attempting a method invocation on a null value will cause an exception. But so will attempting to divide by zero. That doesn't mean that we need to go on a campaign to eliminate zeroes! It just means that we need to understand the contract on a method and do the right thing with the values that it returns.

Have you had a look at Spec#?

You could write your own annotation (Java) or attribute (C#) to indicate that the return value might be null. Nothing will automatically check it (although .NET 4.0 will have code contracts for this sort of thing) but it would at least act as documentation.

Maybe you could define a generic class named "NotNull", so that your method might be like:
public NotNull<Object> methodWhichCannotReturnNull(int i) throws Exception
{
// the following would lead to a run-time error thown by the
// NotNull constructor, if it's constructed with a null value
return new NotNull<Object>(null);
}
This is still a run-time (not a compile-time) check, but:
It's thrown in the implementation of the method (it's not a fault in the calling code)
It's self-documenting (the caller knows he's geting NotNull<T> as a return type)

At all costs, avoid relying on the JavaDocs. People only read them if the signature doesn't appear trivial and self-explanatory (Which is bad to begin with), and these who actually bother to read them are less likely to make a mistake with the nulls since they are currently being more careful.

If you're using Java 5+, you can use a custom Annotation, e.g. #MayReturnNull
UPDATE
All coding philosophy aside (returning null, using exceptions, assertions, yada yada), I hope the above answers your question. Apart from primitives having default values, complex types may or may not be null, and your code needs to deal with it.

Generally speaking, I would assume that a null return value is against the contract of the API by default. It is almost always possible to design your code such that a null value is never returned from your APIs during "normal" flow of execution. (For example, check foo.contains(obj) rather then calling foo.get(obj) and having a separate branch for null. Or, use the Null object pattern.
If you cannot design your API in such a way, I would clearly document when and why a null could be thrown--at least in the Javadoc, and possibly also using a custom #annotation such as several of the other answers have suggested.