We Keep Coding

How does JPMS knows which packages belong to which modules?

I am trying to get a better understanding of how the new java module system works, especially when it comes to bundling together java packages to their respective modules.
I could not find a way of asking this question in an easy to understand way without tying it to a specific example.
If I have the following project structure:
.
└── src
└── main
└── java
├── module-info.java
└── org
└── lb
└── app
└── App.java
where module-info.java is:
module lb.module {}
and App.java is:
package org.lb.app;
public class App {
public static void main (String[] args) {
System.out.println("Hello, world!");
}
}
and then I run the command from the . directory:
javac -d build/classes $(find src/main/ -name "*.java")
which gives me the following result:
.
├── build
│   └── classes
│   ├── module-info.class
│   └── org
│   └── lb
│   └── app
│   └── App.class
└── src
└── ...
and finally I run the command:
java --module-path build/classes/ --module lb.module/org.lb.app.App
My questions are:
How does the javac tool knows that module-info.class should be in the root directory folder build/classes if there is no "package information" (e.g. like there are in normal java files), and not, for instance in a folder that mimics the original location (e.g. build/classes/src/main/java/module-info.class
I did not expect it to work, but it does. How does the JVM knows that the org.lb.app package belongs to the lb.module module? I purposefully "misnamed" the module, making it not have same name as the directory that contains it, and also did not export the package. It seems obvious for a human reader, but I want to understand what the java program assumes in order to bundle these two together.
Thank you.

flyway-run both SQL and Java based scripts in remote location

Trying to run Java-based (compiled) migration files which are not in the project where Flyway is configured. Can anyone tell me is it possible to do so?
I've created a jar which use flyway to do migrations. Jar expects an argument which is migration scripts' location. Migration scripts are in a different location/project. So far all scripts are SQL based. (i.e. XXX.sql). Need to add a java based migration scripts to it, to do some complex logic.
Tried to add pom.xml to the script location and a sample java migration script in db/migration folder. Java-based migration file is ignored by Flyway. Is it due to checksum validation fail?
Java-based migrations are compiled and .class files are in the classpath. My folder structure as below.
C:/
└── database-migration-scripts
└── src/main/java/
└── db
└── migration
└── V1__m1.sql
└── V2__m2.sql
└── V3__SampleJava_script.java
└── target/classes/
└── db
└── migration
└── V3__SampleJava_script.class
└── pom.xml
W:/
└── someLocationOutsideProject
└── flyway-database-migration.jar
NOTE: flyway runner (jar) and scripts will be in different locations in the same or different machine. In the example above, migration scripts in C:/ and jar in W:/

In order to be discovered, Java migrations should go under src/main/java with the package db.migration
e.g.
package db.migration; // <-- classpath:db/migration
import org.flywaydb.core.api.migration.jdbc.JdbcMigration;
import java.sql.Connection;
public class V3__SampleJava_script implements JdbcMigration {
public void migrate(Connection connection) throws Exception {
// your code...
}
}
Difficult to diagnose without seeing your pom.xml and how your jar is packaged but given the folder structure of your target directory above, perhaps either the V3__SampleJava_script.class is added to the jar under classpath:resources/db/migration or is just not included at all.
To check, try unzipping the jar file:
jar -xf flyway-database-migration.jar
or just listing the contents:
jar -tf flyway-database-migration.jar
It is also worth noting that if you have overridden the locations setting with a filesystem: path, the documentation states that the directory "may only contain sql migrations", any java migrations will just be ignored.
Update 2018-06-03
Given that the flyway-database-migration.jar is being pointed to a filesystem: location, only sql migrations will be discovered, not java ones. The database-migration-scripts directory needs to be added to the classpath and the flyway location set to classpath:db/migration.
java -cp C:\database-migration-scripts;<existing classpath> ...
Update 2018-06-09
Because of the way that the flyway-database-migrations.jar is packaged, using the Spring Boot Executable Jar format, all the application dependencies are placed in the BOOT-INF/lib directory inside the executable jar and are loaded by a separate classloader from the org.springframework.boot.loader.JarLauncher main class. So, contrary to the above, I'm not sure if it's possible to pass additional classpath entries to the application using the -cp command line option. I think you would need to remove spring boot and package it as a regular jar file. I certainly encountered class visibility issues and decided to try a different approach.
I downloaded the Flyway Command Line Runner and updated the <RUNNER_DIR>/conf/flyway.conf settings with the following:
flyway.url=jdbc:mariadb://localhost:3306/flyway?createDatabaseIfNotExist=true
flyway.user=root
flyway.password=yourPassword
flyway.locations=classpath:db/migration
I created a directory under <RUNNER_DIR>/jars/ called database-migration-scripts.jar with the following structure (the .jar is important as Flyway will only add files or directories with this suffix to the classpath):
database-migration-scripts.jar/
└── db
└── migration
├── V1__m1.sql
├── V2__m2.sql
└── V3__SampleJava_script.class
Finally, I added all of the runtime dependencies for the database-migration-scripts project to <RUNNER_DIR>/lib:
lib/
├── animal-sniffer-annotations-1.14.jar
├── checker-compat-qual-2.0.0.jar
├── checker-qual-2.3.0.jar
├── error_prone_annotations-2.1.3.jar
├── flyway-commandline-5.1.1.jar
├── flyway-core-5.1.1.jar
├── guava-23.6-jre.jar
├── j2objc-annotations-1.1.jar
├── jcl-over-slf4j-1.7.25.jar
├── jsr305-1.3.9.jar
├── jul-to-slf4j-1.7.25.jar
├── log4j-over-slf4j-1.7.25.jar
├── logback-classic-1.1.11.jar
├── logback-core-1.1.11.jar
├── slf4j-api-1.7.25.jar
├── snakeyaml-1.17.jar
├── spring-aop-4.3.13.RELEASE.jar
├── spring-beans-4.3.13.RELEASE.jar
├── spring-boot-1.5.9.RELEASE.jar
├── spring-boot-autoconfigure-1.5.9.RELEASE.jar
├── spring-boot-starter-1.5.9.RELEASE.jar
├── spring-boot-starter-jdbc-1.5.9.RELEASE.jar
├── spring-boot-starter-logging-1.5.9.RELEASE.jar
├── spring-context-4.3.13.RELEASE.jar
├── spring-core-4.3.13.RELEASE.jar
├── spring-expression-4.3.13.RELEASE.jar
├── spring-jdbc-4.3.13.RELEASE.jar
├── spring-tx-4.3.13.RELEASE.jar
├── tomcat-jdbc-8.5.23.jar
└── tomcat-juli-8.5.23.jar
After that I was able to successfully run:
./flyway migrate
And was able to verify that both sql and java migrations had been successfully applied:
./flyway info
+-----------+---------+-------------------+-------------+---------------------+---------+
| Category | Version | Description | Type | Installed On | State |
+-----------+---------+-------------------+-------------+---------------------+---------+
| Versioned | 1 | m1 | SQL | 2018-06-09 07:41:57 | Success |
| Versioned | 2 | m2 | SQL | 2018-06-09 07:41:57 | Success |
| Versioned | 3 | SampleJava script | SPRING_JDBC | 2018-06-09 07:47:56 | Success |
+-----------+---------+-------------------+-------------+---------------------+---------+
Phew! This is much harder work than packaging the migrations with the application, in my opinion.
One of the other downsides of this approach is that if someone adds a new java migration with an additional dependency (e.g. commons-lang3, or whatever else) that dependency needs to be added to the <RUNNER_DIR>/lib directory as well.
Hope this helps!

"There's nothing here yet" after deploying Java Web App on Azure

I have a web app I want to deploy on Azure. I followed the recommended instructions and deployed my app, and according to the Azure portal, everything is working just fine. However, when I visit the base URL my app should reside in, I see a page that says the following:
This Java based web application has been successfully created
There's nothing here yet, but Microsoft Azure makes it simple to
publish content with GIT and FTP
Also when I visit any one of the endpoints (in this case, the /live endpoint) my app should have, I always see a page with the following message:
HTTP ERROR 404
Problem accessing /live. Reason:
Not Found
Powered by Jetty:// 9.3.13.v20161014
When I look at the directories on the machine, everything seems to be in place. All my files are inside wwwroot. However, there's another directory named webapp under wwwroot and inside it is another directory named ROOT with two files: index.jsp and background.png. index.jsp is the page that shows the aforementioned "There's nothing here yet" message.
I'm using Bitbucket as my source control provider and I use jetty to run my web app. I'm also using javalite as the library to manage my server and different endpoints.

You'll need to put your stuff under wwwroot/webapps/ROOT/, or package as ROOT.war and drop that under wwwroot/webapps - it will get picked up and extracted automagically:
wwwroot
└── webapps
└── ROOT
├── about.jsp
├── Content
│   ├── favicon.ico
│   └── Site.css
├── Images
│   ├── banner_coffee.png
├── index.jsp
├── META-INF
│   ├── context.xml
│   └── MANIFEST.MF
├── orderconfirmation.jsp
├── placeorder.jsp
├── Scripts
│   ├── jquery-1.7.1.min.js
└── WEB-INF
├── classes
├── lib
├── log4j.properties
└── web.xml
From https://github.com/Azure-Samples/app-service-web-java-get-started:
The main thing in the repo is a webapps folder with ROOT.war. The Tomcat/Jetty server in App Service will look inside this folder for web apps to host.
ROOT.war represents the default web app (at the site root). Any WAR file that's otherwise named represents a web app accessbile at ~/<WARfilename>.
Clearing things up
If your application sits in wwwroot/webapps/CoffeeShop/, then you'll access it at http://{site}.azurewebsites.net/CoffeeShop/.
If your application sits in wwwroot/webapps/ROOT/, then you'll access it at http://{site}.azurewebsites.net/.

NiFi-1.0.0 - load lua script

I have a NiFi processor, that uses the redislabs/luascript lib in order to load a lua script and execute it on a redis instance.
The thing is that I don't know where exactly to put the lua script in order to load it using the luascript lib. I've put it into the nifi_proc/src/main/resources/lua/name.lua, but I get an IOException.
I have a nifi controller service for connecting to redis and a processor that uses that service.
My project structure:
.
├── nifi-bundle-nar
│   └── target
├── nifi-redis_cservice
│   ├── src
│   └── target
├── nifi-redis_cservice-api
│   ├── src
│   └── target
├── nifi-redis_cservice-api-nar
│   └── target
├── nifi-redis_cservice-nar
│   └── target
├── redis-processors
│   ├── src
│   └── target
└── target
└── maven-shared-archive-resources
Any ideas?

Can you share more information about how the processor is interacting with the library? Are you passing in an InputStream, calling out to a executable, etc.?
Ensure your resource is in the JAR module of your processor's project, not the processor's NAR module or the parent (that includes both). You should be able to use getResourceAsStream("lua/name.lua") from a Class object that is in the processor's JAR file (such as the processor class itself). I'm not sure what you'd need to do with it after that, is it possible to share the source code or more details around it?
EDIT (reply to comments below): fromResource() uses LuaScript's classloader to get the resource, I wonder if it doesn't have access to the nifi-proc or controller service resources. It seems like, unless the user needs to specify the location of the script, that the controller service should be loading in the Lua script. So an alternative could be to use the controller service class to getResourceAsStream, read the whole thing into a String, and use fromSource instead of fromResource.

how to specify class path for java agent

I am write a Java Agent to instrument a target Method of a target Class.
I use the javassist library to do instrument.
So the java agent (let named CnAgent.class) need its dependency : javassist library to run.
The directory hierarchy is :
.
├── META-INF
│   └── MANIFEST.MF
├── com
│   └── yet
│   └── another
│   └── test
│   └── agent
│   ├── CnAgent.class
│   └── CnTransformer.class
└── lib
└── javassist-3.18.2-GA.jar
and the MANIFEST.MF file content is :
Manifest-Version: 1.0
Class-Path: lib/javassist-3.18.2-GA.jar .
Agent-Class: com.yet.another.test.agent.CnAgent
Created-By: 1.8.0_11 (Oracle Corporation)
Can-Retransform-Classes: true
I create jar ball by following command:
jar cvfm CnAgent.jar META-INF/MENIFIEST.MF . lib
when I load the Agent with Attach API of JVM.
the error prints :
error when transform : javassist/ClassPool
java.lang.NoClassDefFoundError: javassist/ClassPool
which means the javassist library cannot be found by agent code.
So my question is :
How to set Agent library's class path letting it find the dependencies?
Why the Class-Path option in MANIFEST.MF not works , does it only for jar directly ran in command line ?
Thanks your wisdom :)

You can use the option -Xbootclasspath: (sets the path) or -Xbootclasspath/a: (appends the given path to the existing boot class path) (refer to doc from oracle). But, as described in the link, it is non-standard.
As an alternative, you can copy the missing jar file in the %JAVA_HOME%/jre/lib/ext directory.

Per Guido's comment above, you should add Boot-Class-Path to your agent MANIFEST.MF.
See these java.lang.instrumentation docs (Manifest Attributes section)
In my case, I have this in Ant's build.xml:
<manifest file="META-INF/MANIFEST.MF">
<attribute name="Premain-Class" value="de.bodden.tamiflex.playout.Agent"/>
<attribute name="Main-Class" value="de.bodden.tamiflex.playout.Agent"/>
<attribute name="Can-Retransform-Classes" value="true"/>
<attribute name="Implementation-Version" value="${tf.version}"/>
<attribute name="Boot-Class-Path" value="guava-22.0.jar:guice-4.1.0.jar" />
</manifest>
and then copy the guice and guava jars to the directory I run the command from e.g. java -verbose:class -javaagent:poa.jar -jar ExampleProject.jar > loaded.txt
This also lists all the classes loaded to allow you to debug what Java class loader is actually doing.
Neither option from whiskeyspider worked for my case.