Is the Java Module System supposed to prevent modules accessing other modules via reflection, without declaring proper module dependencies?
For example, when compiling this hello world Java 11 class, which calls a class from another module, as expected it won't compile, because dependency on java.xml is missing:
module m1 {}
package p1;
public class C1 {
public static void main(String[] args) throws Exception {
System.out.println(javax.xml.XMLConstants.XML_NS_URI);
}
}
After adding module dependency to java.xml it compiles and runs as expected.
However, this class:
module m1 {}
package p1;
public class C1 {
public static void main(String[] args) throws Exception {
System.out.println(Class.forName("javax.xml.XMLConstants").getField("XML_NS_URI").get(null));
}
}
runs and prints the result, without any need to declare a module dependency to java.xml:
java -version
openjdk version "11.0.2" 2019-01-15
java -p bin -m m1/p1.C1
http://www.w3.org/XML/1998/namespace
So effectively we are able to bypass Java modularity.
This works from any module to any other resolved module.
How is that possible? Should the module system prevent such scenario?
There are two types of readability in the Java module system: static readability and reflective readability. In the first versions of the module system (during its development before it was released in Java 9), these two types were the same. So, your second example indeed would fail with an error because there is no requires clause from m1 to java.xml. Later this policy was revised and the rules for reflective readability were relaxed because such strict policy hadn't played well with many frameworks that heavily relied on reflection (e.g. Spring). Now the module system doesn't mandate readability edges for reflection (but the target types still must be exported and the defining module must be in the module graph).
5.2 Reflective readability
...
To make the provider class accessible to the framework we need to make
the provider’s module readable by the framework’s module. We could
mandate that every framework explicitly add the necessary readability
edge to the module graph at run time, as in an earlier version of this
document, but experience showed that approach to be cumbersome and a
barrier to migration.
We therefore, instead, revise the reflection API simply to assume that
any code that reflects upon some type is in a module that can read the
module that defines that type. This enables the above example, and
other code like it, to work without change. This approach does not
weaken strong encapsulation: A public type must still be in an
exported package in order to be accessed from outside its defining
module, whether from compiled code or via reflection.
Related
I have two jars. One provides a service interfaces and a service loading class and one provides the implementation of that service.
This works perfectly when running this in jdk8, but I get an service type not accessible to unnamed module #3754a4bf error when running on jdk9 or higher.
I migrated that two jars to a module base jar and that works great when running on >= jdk9, but that fails on jdk8 of cause, because of wrong class file version.
So, I don't have problem using java serivceloader api with java 8 or 9.
I'm aware of https://blog.codefx.org/tools/multi-release-jars-multiple-java-versions/ , but I want to avoid to make the build process more complicated. The build already involves class relocation and
other stuff.
My question: Is there a way to use the same jars running on jdk8 and >=jdk9 using java serviceloading api?
In JDK9 and up, all jars are effectively 'modules'. If they do not have a module definition (which you create by making a file named module-info.java and putting a module declaration inside), its name is 'unnamed module #whatever', it exports all its packages, and can access anything exported by any other module (in module-speak: it 'reads' everything). Meaning, if all your classpath dependencies are such unnamed modules, they all export everything and they all read everything thus they all can access anything marked public from anybody else – which is how it worked in JDK8, and thus, why it's all compatible.
To be clear: in JDK9, for code in module (jar) A to access a method, class, or field from module (jar) B, then in addition to the usual access modifiers (the public keyword), B needs to 'export' that package, and A needs to read that module, or it doesn't work. If you aren't explicitly writing module info files for either side, well, then you get the default behaviour which is 'export everything' and 'read everything', bringing us back to the JDK8 scenario of: the thing has to be marked public, and then you can access it.
I actually do not recommend you make that module-info file; that's a big step, one you should only take once you're familiar with the module system.
The error means that the 'service type'[1] is not 'accessible'[2] to 'unnamed module #3754a4bf'[3]. Let's break that into pieces:
[1] serviceloader works by defining an interface that a 'service provider' implements, and the class using the service loader then ends up with a bunch of instances of that interface; each representing one implementation. It's the Foo in ServiceLoader.load(Foo.class).
[2] 'accessible' is module speak for: Either the code that needs this did not explicitly 'read' it, or the code that has this did not export it.
[3] 'unnamed module#3754abf' is the name of the module that is trying to access it.
Taking this all together, it means: You're operating under false assumptions: Whatever jar contains your service interface (that Foo I was talking about), is NOT an unnamed module, or, possibly, isn't public. Note that if it is a public interface inside a not-public class (i.e.: /* package private */ class Example { public interface MyServiceInterface {} }), that probably still counts as 'not public enough'.
If it's a named module (which means: it has a module-info.java file), then export the package that the service interface is in. See any jigsaw (the name of the java module system) tutorial on how to set that up. If it's not, ensure it's a public interface or abstract class, and if its inside some other type, make sure those are public too. If neither is the case, check your classpaths; javac is rather adamant that one of these two is the case.
I have a mock class with a trivial implementation of a service I provide from a module. I'm using OpenJDK 11.03, gradle 5.2.1 and IntelliJ 2019.2.
In /main/code/myPackage/myService.java I have:
package myPackage;
class myService {
public abstract void someFunction();
}
And in my test/code/somePackage/myMockService I have:
package myPackage;
// no import, they're in the same package.
class myMockService extends myService {
#Override
public void someFunction() { System.out.prinln("Hello World"); }
}
In my main/code/module-info.java I have:
module myModule {
exports somePackage;
}
I've tried several variations on a test/code/module-info.java, without success. For example:
// "open module" lets anyone use reflection within (mostly JUnit 5 in my case)
import myPackage.myService;
import myPackage.myMockService;
open module myTestModule {
exports myPackage;
provides myService with myMockService
}
The above module-info.java spews errors about how "module name myTestModule does not match expected name myModule", "package 'myPackage' is not visible" (from myMockModule.java), explaining "package myPackage is declared in module myModule but module myTestModule does not read it"
On the other hand, with the following module-info.java, I get a different batch of errors (below the code)
import myPackage.myService;
import myPackage.myMockService;
open module myModule {
provides myService with myMockService;
}
Without a requires myModule;, every reference to the main code branch from my test code gives an "error: cannot find symbol". With a requires myModule;, I get an "error: cyclic dependence involving myModule".
So... my tests can't be in a different module. AND they can't be the same module! [long string of expletives deleted]
How do I introduce a mock version of a service in test code rather than creating an entirely different module/gradle sub-project?
Or is this simply a case where that's not possible, and while you can have a separate test module-info, you can't do much with it?
Or is there some way to dynamically load things at runtime such that I don't have to put every little mock service in any module-info, test or otherwise? Such that ServiceLoader.load() will find them. Hmm... perhaps extend ServiceLoader and wrap its usage in main code such that it'll use the right one either in production code or test code...
a) Welcome to "Testing in the Modular World"!
TL;DR https://sormuras.github.io/blog/2018-09-11-testing-in-the-modular-world.html
Having one or more dedicated test modules is good. With all bells-and-whistles, read module-info.java declarations. Those test modules are your main modules' first clients. Just make sure, that your build tool packages all main modules before compiling and running the test modules. Otherwise you don't test your main modules as close as possible to reality — others will consume your main modules as JAR files. So should you. This solves all issues with services and multi-release JARs as well.
Now the interesting part: in-module testing, also named white box testing. Or how do test types residing non-exported packages or package-private types in exported packages? Either use a build that knows how to patch test modules into main modules (or vice versa) at test compile and/or test runtime. Like pro or Bach.java (which I maintain), or in your case of using Gradle, see b)elow part of this answer.
b) Gradle and Java main, test, … modules are not friends out-of-the-box, yet
Best plugin-based solution: https://github.com/java9-modularity/gradle-modules-plugin -- which honors the pass theses java command line options at test runtime module-info.test configuration file (which I invented). Here you basically desribe your test module requirements via verbose command line options, although a perfect DSL already exists: module-info-java ... loop back to a) and the module-aware build tools.
c) IntelliJ IDEA and Java test modules are ... improving!
https://youtrack.jetbrains.com/issue/IDEA-171419 module-info.java support in 2019.3
https://youtrack.jetbrains.com/issue/IDEA-222831 module-info.test support, soon?
This question already has answers here:
What's the difference between requires and requires static in module declaration
(2 answers)
Does the Java 9 Module system support optional dependencies?
(1 answer)
Closed 3 years ago.
I started to learn jigsaw java-9 feature and read some articles/video.
I can't understand concept of optional dependencies(requires static)
quote from article:
When a module needs to be compiled against types from another module
but does not want to depend on it at run time, it can use a requires
static clause. If foo requires static bar, the module system behaves
different at compile and run time:
At compile time, bar must be present or there will be an error. During
compilation bar is readable by foo.
At run time, bar might be absent
and that will cause neither error nor warning. If it is present, it is
readable by foo.
So I want to know couple of things:
What the reason to make module dependable on another module during compile time but not in runtime? any examples? instruments like lombok?
Any analogs of optional dependencies in java prior java-9 ?
P.S.
I found one more explanation:
quote from article:
Sometimes we write code that references another module, but that users
of our library will never want to use.
For instance, we might write a utility function that pretty-prints our
internal state when another logging module is present. But, not every
consumer of our library will want this functionality, and they don’t
want to include an extra logging library.
In these cases, we want to use an optional dependency. By using the
requires static directive, we create a compile-time-only dependency:
module my.module {
requires static module.name;
}
But it is absolutely unclear for me. Could anyone explain it in a simple way?
There are a decent number of libraries out there where it only makes sense to have them at compile time. Mostly this deals with annotations that only exist to help during development (e.g. prevent bugs, reduce boilerplate). Some examples include:
java-annotations by JetBrains
spotbugs-annotations by SpotBugs (successor of FindBugs)
Project Lombok (as you mentioned)
jcip-annotations
These annotations tend to have a RetentionPolicy of SOURCE or CLASS, which means they aren't useful (or even available) at runtime. Why ship these dependencies with the rest of your application when you deploy? Without requires static you would be forced to include them when you deploy, otherwise your application would fail to start due to missing dependencies.
You would declare these dependencies as optional pre-Java 9 as well. Many Java projects of any significance use a build tool such as Maven or Gradle. In addition to those tools automatically building and testing your project, a large part of what they do is dependency management. I'm not familiar enough with Maven, but when using Gradle one would use:
dependencies {
compileOnly 'group.id:artifact-id:version'
}
To declare dependencies that are not needed at runtime.
If Dependent Module should be available at compile time but optional at rumtime, then such type of Dependency is called Optional Dependency. We can Specify optional dependency by using static keyword.
Note The static keyword is used to say that "This dependency check is mandatory at compile time and optional at runtime."
Eg.1
module moduleB {
requires moduleA;
}
moudleA should be available at the time of compilation & rumtime. it is not Optional Dependency.
Eg2.
module moduleB {
requires static moduleA;
}
At the time of compilation moduleA should be available, but at runtime it is optional ie, at runtime even moduleA is not avaiable JVM will execute code.
What is the difference between a module with the open keyword before it and without it? For instance:
open module foo {
}
module foo {
}
In order to provide reflective access to your module, Java 9 introduced the open keyword.
You can create an open module by using the open keyword in the module declaration.
An open module grants reflective access to all of its packages to other modules.
For example, if you want to use some framework that heavily relies on reflection, such as Spring, Hibernate, etc, you can use this keyword to enable reflective access for it.
You can enable reflective access for specified packages of your module by using the opens statement in the package declaration:
module foo {
opens com.example.bar;
}
or by using the open keyword in the module declaration:
open module foo {
}
but keep in mind, that you cannot combine them:
open module foo {
opens com.example.bar;
}
results with compile-time error.
Hope it helps.
A bit of a background to the directives in question. The Module System states in the Breaking encapsulation section
It is sometimes necessary to violate the access-control boundaries
defined by the module system, and enforced by the compiler and virtual
machine, in order to allow one module to access some of the unexported
types of another module. This may be desirable in order to, e.g.,
enable white-box testing of internal types, or to expose unsupported
internal APIs to code that has come to depend on them. The
--add-exports option can be used, at both compile time and run time, to do this.
The command line option is equivalent to for example:-
module java.logging {
exports java.util.logging;
}
The --add-exports option enables access to the public types of a specified package.
Though thereafter,
It is sometimes necessary to go further and enable access to all
non-public elements via the setAccessible method of the core
reflection API. The --add-opens option can be used, at runtime, to
do this.
This command line option is equivalent to for example:-
module jdk.unsupported {
opens sun.misc;
}
An ordinary named module, by contrast, is defined explicitly with a module declaration as:-
module foo { }
and these are known as explicit modules. Explicit modules can use directives on packages as stated above for example to export/open packages to provide reflective access to their respective public members and classes.
On the other hand, an OPEN module is a
module that does not declare any open packages but the resulting module is
treated as if all packages are open.
Such that it grants access at runtime to types in all of the module's packages as if all packages are exported which means bytecode or reflection can be used to access every package's classes or members in all packages. The reflection APIs with setAccessible or MethodHandles.privateLookupIn allow for deep reflection, so in short you can reflect on all members of all classes in all packages. Which also pretty much explains the reason why compiler won't allow both open directives to a package while the module is already open.
Assume 'foo' has two packages, com.example and com.example.bar
module foo {
opens com.example;
}
Above module declaration allows reflective access for types inside 'com.example' package at runtime for outside.
But 'com.example' will not be visible at compile time for outside.
open module foo {
}
This means, It implicitly open all packages in the module for the runtime to reflective access.
Above module declaratoin will equilent to follows,
module foo {
opens com.example;
opens com.example.bar;
}
I've read that OSGI uses separate classloaders per module which allows modules to use different versions of their dependencies.. while at the same time running all modules in the same JVM.
How does this work? If module A uses version #1 of a dependency and module B uses version #2, won't you run into trouble if module A passes an instance of the dependency class to module B as a method parameter?
I would think module B would choke if it was expecting a different interface to the dependency class.
You're right that inconsistent dependencies can cause problems. OSGi avoids this by calculating the transitive closure of these dependencies and ensuring that there are none at resolution time.
This allows you to expose a public dependency whilst having internal/hidden private dependencies and, as a result, hide your implementation dependencies to avoid this. The good thing is all thus is checked at Bundle resolution time rather than obscure runtime errors.
Specifically in your example, if A and B uses an incompatible interface, and A depends on B, then A will fail to resolve with a dependency error. So it won't even be able to start to pass it an incompatible type.