We Keep Coding

macOS dynamic linker reports it loaded library which doesn't exist

When running java with DYLD_PRINT_LIBRARIES=TRUE the following lines are contained in the output:
dyld[15078]: <C77B7FE3-7104-362A-B686-168971378ADC> /System/Library/Frameworks/JavaNativeFoundation.framework/Versions/A/JavaNativeFoundation
dyld[15078]: <BE429D77-6080-3D27-B4EE-12EE022386B4> /System/Library/Frameworks/JavaRuntimeSupport.framework/Versions/A/JavaRuntimeSupport
However the repsecitve folders don't actually contain any binary files to load. For example this is the directory structure of JavaNativeFoundation.framework:
/System/Library/Frameworks/JavaNativeFoundation.framework
├── Resources -> Versions/Current/Resources
└── Versions
├── A
│ ├── Resources
│ │ ├── BridgeSupport
│ │ │ └── JavaNativeFoundation.bridgesupport
│ │ ├── Info.plist
│ │ └── version.plist
│ └── _CodeSignature
│ └── CodeResources
└── Current -> A
7 directories, 4 files
Loading a dynamic library which links against the framework JavaNativeFoundation in the java program fails as expected:
java.lang.UnsatisfiedLinkError: <path to the library>: dlopen(<path to the library>, 0x0001): Library not loaded: #rpath/JavaNativeFoundation
Referenced from: <path to the library>
Reason: tried:
'/System/Library/Frameworks/JavaNativeFoundation.framework/Versions/Current/JavaNativeFoundation' (no such file),
'/System/Library/Frameworks/JavaNativeFoundation.framework/Versions/Current/JavaNativeFoundation' (no such file),
'/Library/Java/JavaVirtualMachines/jdk8u232-b09/Contents/Home/jre/lib/server/./JavaNativeFoundation' (no such file),
'/Library/Java/JavaVirtualMachines/jdk8u232-b09/Contents/Home/jre/lib/server/../JavaNativeFoundation' (no such file),
'/Library/Java/JavaVirtualMachines/jdk8u232-b09/Contents/Home/bin
Note that /System/Library/Frameworks/JavaNativeFoundation.framework/Versions/Current/ is a symlink to /System/Library/Frameworks/JavaNativeFoundation.framework/Versions/A/ so both logs actually refer to the same file. For me this looks like a contradiction. How can the dynamic linker load and not load
/System/Library/Frameworks/JavaNativeFoundation.framework/Versions/[Current/A]/JavaNativeFoundation at the same time.
I don't believe this to be specific to Java at all. It's just the context in which this pops up.
Some information regarding the OS (all mentioned machines are Intel x86-64 based):
The above scenario is for macOS 12.1
The binary for JavaNativeFoundation is also missing on macOS 11.5.2
On a different machine running 10.15.7 it exists.

This is due to:
New in macOS Big Sur 11 beta, the system ships with a built-in dynamic linker cache of all system-provided libraries. As part of this change, copies of dynamic libraries are no longer present on the filesystem. Code that attempts to check for dynamic library presence by looking for a file at a path or enumerating a directory will fail. Instead, check for library presence by attempting to dlopen() the path, which will correctly check for the library in the cache.
Apparently this also affects the dynamic linker itself while resolving transitive dependencies of a library. If the load command contains an absolute path it resolves it correctly. It’s only in the case that the path contains #rpath entries that the linker will check for the existence of the path (after substitution) and ignored the linker cache.

flyway-run both SQL and Java based scripts in remote location

Trying to run Java-based (compiled) migration files which are not in the project where Flyway is configured. Can anyone tell me is it possible to do so?
I've created a jar which use flyway to do migrations. Jar expects an argument which is migration scripts' location. Migration scripts are in a different location/project. So far all scripts are SQL based. (i.e. XXX.sql). Need to add a java based migration scripts to it, to do some complex logic.
Tried to add pom.xml to the script location and a sample java migration script in db/migration folder. Java-based migration file is ignored by Flyway. Is it due to checksum validation fail?
Java-based migrations are compiled and .class files are in the classpath. My folder structure as below.
C:/
└── database-migration-scripts
└── src/main/java/
└── db
└── migration
└── V1__m1.sql
└── V2__m2.sql
└── V3__SampleJava_script.java
└── target/classes/
└── db
└── migration
└── V3__SampleJava_script.class
└── pom.xml
W:/
└── someLocationOutsideProject
└── flyway-database-migration.jar
NOTE: flyway runner (jar) and scripts will be in different locations in the same or different machine. In the example above, migration scripts in C:/ and jar in W:/

In order to be discovered, Java migrations should go under src/main/java with the package db.migration
e.g.
package db.migration; // <-- classpath:db/migration
import org.flywaydb.core.api.migration.jdbc.JdbcMigration;
import java.sql.Connection;
public class V3__SampleJava_script implements JdbcMigration {
public void migrate(Connection connection) throws Exception {
// your code...
}
}
Difficult to diagnose without seeing your pom.xml and how your jar is packaged but given the folder structure of your target directory above, perhaps either the V3__SampleJava_script.class is added to the jar under classpath:resources/db/migration or is just not included at all.
To check, try unzipping the jar file:
jar -xf flyway-database-migration.jar
or just listing the contents:
jar -tf flyway-database-migration.jar
It is also worth noting that if you have overridden the locations setting with a filesystem: path, the documentation states that the directory "may only contain sql migrations", any java migrations will just be ignored.
Update 2018-06-03
Given that the flyway-database-migration.jar is being pointed to a filesystem: location, only sql migrations will be discovered, not java ones. The database-migration-scripts directory needs to be added to the classpath and the flyway location set to classpath:db/migration.
java -cp C:\database-migration-scripts;<existing classpath> ...
Update 2018-06-09
Because of the way that the flyway-database-migrations.jar is packaged, using the Spring Boot Executable Jar format, all the application dependencies are placed in the BOOT-INF/lib directory inside the executable jar and are loaded by a separate classloader from the org.springframework.boot.loader.JarLauncher main class. So, contrary to the above, I'm not sure if it's possible to pass additional classpath entries to the application using the -cp command line option. I think you would need to remove spring boot and package it as a regular jar file. I certainly encountered class visibility issues and decided to try a different approach.
I downloaded the Flyway Command Line Runner and updated the <RUNNER_DIR>/conf/flyway.conf settings with the following:
flyway.url=jdbc:mariadb://localhost:3306/flyway?createDatabaseIfNotExist=true
flyway.user=root
flyway.password=yourPassword
flyway.locations=classpath:db/migration
I created a directory under <RUNNER_DIR>/jars/ called database-migration-scripts.jar with the following structure (the .jar is important as Flyway will only add files or directories with this suffix to the classpath):
database-migration-scripts.jar/
└── db
└── migration
├── V1__m1.sql
├── V2__m2.sql
└── V3__SampleJava_script.class
Finally, I added all of the runtime dependencies for the database-migration-scripts project to <RUNNER_DIR>/lib:
lib/
├── animal-sniffer-annotations-1.14.jar
├── checker-compat-qual-2.0.0.jar
├── checker-qual-2.3.0.jar
├── error_prone_annotations-2.1.3.jar
├── flyway-commandline-5.1.1.jar
├── flyway-core-5.1.1.jar
├── guava-23.6-jre.jar
├── j2objc-annotations-1.1.jar
├── jcl-over-slf4j-1.7.25.jar
├── jsr305-1.3.9.jar
├── jul-to-slf4j-1.7.25.jar
├── log4j-over-slf4j-1.7.25.jar
├── logback-classic-1.1.11.jar
├── logback-core-1.1.11.jar
├── slf4j-api-1.7.25.jar
├── snakeyaml-1.17.jar
├── spring-aop-4.3.13.RELEASE.jar
├── spring-beans-4.3.13.RELEASE.jar
├── spring-boot-1.5.9.RELEASE.jar
├── spring-boot-autoconfigure-1.5.9.RELEASE.jar
├── spring-boot-starter-1.5.9.RELEASE.jar
├── spring-boot-starter-jdbc-1.5.9.RELEASE.jar
├── spring-boot-starter-logging-1.5.9.RELEASE.jar
├── spring-context-4.3.13.RELEASE.jar
├── spring-core-4.3.13.RELEASE.jar
├── spring-expression-4.3.13.RELEASE.jar
├── spring-jdbc-4.3.13.RELEASE.jar
├── spring-tx-4.3.13.RELEASE.jar
├── tomcat-jdbc-8.5.23.jar
└── tomcat-juli-8.5.23.jar
After that I was able to successfully run:
./flyway migrate
And was able to verify that both sql and java migrations had been successfully applied:
./flyway info
+-----------+---------+-------------------+-------------+---------------------+---------+
| Category | Version | Description | Type | Installed On | State |
+-----------+---------+-------------------+-------------+---------------------+---------+
| Versioned | 1 | m1 | SQL | 2018-06-09 07:41:57 | Success |
| Versioned | 2 | m2 | SQL | 2018-06-09 07:41:57 | Success |
| Versioned | 3 | SampleJava script | SPRING_JDBC | 2018-06-09 07:47:56 | Success |
+-----------+---------+-------------------+-------------+---------------------+---------+
Phew! This is much harder work than packaging the migrations with the application, in my opinion.
One of the other downsides of this approach is that if someone adds a new java migration with an additional dependency (e.g. commons-lang3, or whatever else) that dependency needs to be added to the <RUNNER_DIR>/lib directory as well.
Hope this helps!

Open a file from the src directory (Java)

I've written a short documentation for my Java program. When clicking on the menu Help -> Documentation the default PDF reader of the OS should open the documentation.pdf.
I'm trying to open the PDF which is located in the directory src/doc with Desktop.getDesktop().open(new File("doc/documentation.pdf")); in Controller.java.
However, Java does not find the file. When I open the icon for the program with primaryStage.getIcons().add(new Image("icon/icon_512x512.png")); it works perfectly in Main.java.
Here you can see layout of my IntelliJ project.
src
├── META-INF
├── de
│   └── myapp
│   ├── model
│ │ └── *.java
│   ├── view
│ │ └── *.java
│ ├── Main.java
│   └── Controller.java
├── doc
│ └── documentation.pdf
└── icon
└── icon_512x512.png
My stack
IntelliJ 2016.2
Java 1.8.0_77

It works with new Image("icon/icon_512x512.png") because internally it gets is from the context ClassLoader which is not the case of new File("doc/documentation.pdf") that gets it from the user working directory in case of a relative path, so you could simply apply the same logic.
ClassLoader contextClassLoader = Thread.currentThread().getContextClassLoader();
URL resource = contextClassLoader.getResource("doc/documentation.pdf");
Desktop.getDesktop().open(new File(resource.toURI()));

3-rd party applications can not access src dir in your application, in case, when your app assemble in jar archive. You should place your file separately from src.
Of course, java find icons, because it's java API.
You can access any resources in src folder through follow methods:
URL url = getClass().getResource("/path/in/src");
InputStream is = getClass().getResourceAsStream("/path/in/src");
If your app is NOT assemble in JAR - try provide full path to file like this:
URL url = getClass().getResource("/path/in/src");
File file = new File(url.toURI());

The files from Classpath can be loaded by using ClassLoader's getResourceAsStream Method.
So you can try with generating an Input stream object
InputStream is = Controller.class.getClassLoader().getResourceAsStream("doc/documentation.pdf");
And After generating Input Stream you can read it by Java Program.

Java jar can't access resources

I need to execute a jar file which uses some files located in some subfolders.
For example the directory tree can be like this:
jar_root/
├── executable.jar
├── folder1/
│ └── required_file1.txt
│
├── folder2/
│ └── required_file2.txt
│
├── other_folder/
│ └── ...
└── other_file.txt
In this example executable.jar needs to access required_file1 and required_file2.
I need to execute the jar from another directory, so I tried this command:
java -cp /path/to/jar_root/ -jar /path/to/jar_root/executable.jar <options>
But what I got is a FileNotFoundException on required_file1 (I guess the same Exception will be raised for required_file2)
How can I make the jar work?
Note that I cannot modify the jar, so I can't use getResourceAsStream, as suggested by this (and other) answer(s).

It depends on how the code in the jar tries to access the files. If by relative path, that can only work if you start the program from the appropriate working directory, for example:
cd /path/to/jar_root/
java -jar executable.jar <options>
An alternative is to reference the files by absolute path, or relative from classpath instead of filesystem path.

Spring Boot JSF packaging as a JAR

I am trying to create a spring boot app with jsf and gradle.
So far, during development everything was fine. When I wanted to run my App I just typed gradle bootRun, the app started and I was able to access it under 'localhost'.
Now I am at a point of time, where I want to deploy the app, therefore I run the command 'gradle clean distTar' which creates the tar file to deploy.
After running the generated script and accessing my app via Browser I just get an 404 with the message.
index.xhtml Not Found in ExternalContext as a Resource
Also in the jar file, there aren't any html files included. I included them in the jar with the command
from ("${projectDir}/src/main/webapp/"){
into('resources')
}
Referring to https://spring.io/blog/2013/12/19/serving-static-web-content-with-spring-boot
this files should be accessible. But still nothing changed.
Does anyone else have a clue? What am I doing wrong?
BR

I've been also struggling with this thing and finally managed to come up with a working solution.
As stated in: https://stackoverflow.com/a/9473487/4250114 if you use Servlet3.x (and you probably are with SpringBoot).
For me structure in maven looks like below worked:
src
|-main
| ...
|-resources
|-META-INF
|-faces-config.xml
|-resources
|-test.xhtml
So in jar it should be:
|-META-INF
|-faces-config.xml
|-resources
|-test.xhtml

Follow the given structure to create fat jar :
Demo
└── src
| ├── main
| │ ├── java
| │ │ └── org
| │ │ └── demo
| │ │ └── App.java
| │ └── resources
| | |
| │ └── application.properties
| | |
| | └── META-INF
| | | └── resources
| | | └── jsp
| | └── static
| | └── css
| | └── js
| |
| └── test
| └── java
| └── org
| └── demo
| └── App.java
├──── pom.xml

Based on information from https://github.com/spring-projects/spring-boot/issues/8299 and also based on the output from SpringBoot+Jetty
2018-01-15 15:57:03 [main] INFO o.j.jetty.JsfJettyServerCustomizer - Setting Jetty classLoader to META-INF/resources directory
I used this in my Gradle file:
jar {
baseName = 'csm'
version = "0.0.1-SNAPSHOT"
from("build/docs/"){
into("generated/docs/")
}
from("src/main/resources/"){
include("**")
}
// JSF and SpringBoot and Jetty require all webapp files to be in a very particular location.
// See: https://github.com/spring-projects/spring-boot/issues/8299
from ("${projectDir}/src/main/webapp/"){
into('META-INF/resources')
}
}
/BOOT-INF
/META-INF/
/resources/
/WEB-INF/
web.xml
index.jsf
blah.xhtml