macOS dynamic linker reports it loaded library which doesn't exist - java

When running java with DYLD_PRINT_LIBRARIES=TRUE the following lines are contained in the output:
dyld[15078]: <C77B7FE3-7104-362A-B686-168971378ADC> /System/Library/Frameworks/JavaNativeFoundation.framework/Versions/A/JavaNativeFoundation
dyld[15078]: <BE429D77-6080-3D27-B4EE-12EE022386B4> /System/Library/Frameworks/JavaRuntimeSupport.framework/Versions/A/JavaRuntimeSupport
However the repsecitve folders don't actually contain any binary files to load. For example this is the directory structure of JavaNativeFoundation.framework:
/System/Library/Frameworks/JavaNativeFoundation.framework
├── Resources -> Versions/Current/Resources
└── Versions
├── A
│ ├── Resources
│ │ ├── BridgeSupport
│ │ │ └── JavaNativeFoundation.bridgesupport
│ │ ├── Info.plist
│ │ └── version.plist
│ └── _CodeSignature
│ └── CodeResources
└── Current -> A
7 directories, 4 files
Loading a dynamic library which links against the framework JavaNativeFoundation in the java program fails as expected:
java.lang.UnsatisfiedLinkError: <path to the library>: dlopen(<path to the library>, 0x0001): Library not loaded: #rpath/JavaNativeFoundation
Referenced from: <path to the library>
Reason: tried:
'/System/Library/Frameworks/JavaNativeFoundation.framework/Versions/Current/JavaNativeFoundation' (no such file),
'/System/Library/Frameworks/JavaNativeFoundation.framework/Versions/Current/JavaNativeFoundation' (no such file),
'/Library/Java/JavaVirtualMachines/jdk8u232-b09/Contents/Home/jre/lib/server/./JavaNativeFoundation' (no such file),
'/Library/Java/JavaVirtualMachines/jdk8u232-b09/Contents/Home/jre/lib/server/../JavaNativeFoundation' (no such file),
'/Library/Java/JavaVirtualMachines/jdk8u232-b09/Contents/Home/bin
Note that /System/Library/Frameworks/JavaNativeFoundation.framework/Versions/Current/ is a symlink to /System/Library/Frameworks/JavaNativeFoundation.framework/Versions/A/ so both logs actually refer to the same file. For me this looks like a contradiction. How can the dynamic linker load and not load
/System/Library/Frameworks/JavaNativeFoundation.framework/Versions/[Current/A]/JavaNativeFoundation at the same time.
I don't believe this to be specific to Java at all. It's just the context in which this pops up.
Some information regarding the OS (all mentioned machines are Intel x86-64 based):
The above scenario is for macOS 12.1
The binary for JavaNativeFoundation is also missing on macOS 11.5.2
On a different machine running 10.15.7 it exists.

This is due to:
New in macOS Big Sur 11 beta, the system ships with a built-in dynamic linker cache of all system-provided libraries. As part of this change, copies of dynamic libraries are no longer present on the filesystem. Code that attempts to check for dynamic library presence by looking for a file at a path or enumerating a directory will fail. Instead, check for library presence by attempting to dlopen() the path, which will correctly check for the library in the cache.
Apparently this also affects the dynamic linker itself while resolving transitive dependencies of a library. If the load command contains an absolute path it resolves it correctly. It’s only in the case that the path contains #rpath entries that the linker will check for the existence of the path (after substitution) and ignored the linker cache.

Related

flyway sql file is not able to open another file

In my Java project, I am using flyway migration (which is working). In one of my migration file, I have to read a svg file from resource folder to save in a table (column). I am using postgres Database. For that operation, I am using pg_read_file() method.
If I provide absolute path to file, it's working. But with relative path, it's not working out.
src
├── main
│ ├── java
│ └── resources
│ ├── local-images
│ │ └──image.svg
│ └── db
│ │ └──migration
│ │ └──V1__populate_table.sql
└── application.properties
application.properties
# https://docs.spring.io/spring-boot/docs/current/reference/html/howto.html#howto-execute-flyway-database-migrations-on-startup
# provides location of the database migrations sql files
spring.flyway.locations=classpath:db/migration
V1__populate_table.sql
INSERT INTO public.image(
image_id, file)
VALUES ('656cc4e8-9e50-1111-1111-303cdc05a058', pg_read_file('resources/local-images/image.svg')::bytea);
Error
SQL State : 58P01
Error Code : 0
Message : ERROR: could not open file "resources/local-images/image.svg" for reading: No such file or directory
I tried putting both .sql and image.svg file in same folder, but still no luck.
Try loading your resources file using the java classloader:
InputStream inputStream = this.getClass().getClassLoader().getResourceAsStream("local-images/image.svg");
byte[] fileBytes = IOUtils.toByteArray(inputStream);
INSERT INTO public.image (image_id, file)
VALUES ('656cc4e8-9e50-1111-1111-303cdc05a058', bytes::bytea);
EDIT
pg_read_file() reads files from the file system relative to the data directory of the cluster, not the application classpath. To read the file from the resources directory you can:
Provide the absolute path of the file
Provide a path relative to the data directory
You could try copying that file from resources to a location on the file system and then use pg_read_file() to read the file from that location before executing the SQL statement.

IntelliJ IDEA Config cant find AWS Lambda Handler in Default 'helloworld' Application

When setting the debug configurations for the default IntelliJ IDEA Java helloworld application (created upon making a new AWS Lambda project) the following response is shown:
Error: Cannot find handler 'helloworld.App::handleRequest' in project."
To fix this I've tried editing 'Handler' element inside template.yaml to include a filepath, though there had been no success.
Resources:
HelloWorldFunction:
Type: AWS::Serverless::Function
Properties:
CodeUri: HelloWorldFunction
Handler: helloworld.App::handleRequest
I noticed the Python AWS Lambda helloworld project (within the PyCharm counterpart) required me to change the root project folder (allowing the handler to be found), however, I cant seem to achieve this with the Java counterpart within IntelliJ.
The default project file structure is as follows:
bash
├── README.mdH
├── HelloWorldFunction
│ ├── pom.xml
│ └── src
│ ├── main
│ │ └── java
│ │ └── helloworld
│ │ ├── App.java
│ │ └── GatewayResponse.java
│ └── test
│ └── java
│ └── helloworld
│ └── AppTest.java
└── template.yaml
The relevant section of the template.yaml file contains:
Resources:
HelloWorldFunction:
Type: AWS::Serverless::Function
Properties:
CodeUri: HelloWorldFunction
Handler: helloworld.App::handleRequest
Runtime: java8
Events:
HelloWorld:
Type: Api
Properties:
Path: /hello
Method: get
The App.java file contains the class
public class App implements RequestHandler<Object, Object> {
...
}
I would like the debug configuration to point to the correct Handler ( being "helloworld.App::handleRequest") so I can run the project on my local machine.
On project files, right click on HelloWorldFunction then find Mark directory as and choose Source Root. That should fix your problems.

Java jar can't access resources

I need to execute a jar file which uses some files located in some subfolders.
For example the directory tree can be like this:
jar_root/
├── executable.jar
├── folder1/
│ └── required_file1.txt
│
├── folder2/
│ └── required_file2.txt
│
├── other_folder/
│ └── ...
└── other_file.txt
In this example executable.jar needs to access required_file1 and required_file2.
I need to execute the jar from another directory, so I tried this command:
java -cp /path/to/jar_root/ -jar /path/to/jar_root/executable.jar <options>
But what I got is a FileNotFoundException on required_file1 (I guess the same Exception will be raised for required_file2)
How can I make the jar work?
Note that I cannot modify the jar, so I can't use getResourceAsStream, as suggested by this (and other) answer(s).
It depends on how the code in the jar tries to access the files. If by relative path, that can only work if you start the program from the appropriate working directory, for example:
cd /path/to/jar_root/
java -jar executable.jar <options>
An alternative is to reference the files by absolute path, or relative from classpath instead of filesystem path.

how to specify class path for java agent

I am write a Java Agent to instrument a target Method of a target Class.
I use the javassist library to do instrument.
So the java agent (let named CnAgent.class) need its dependency : javassist library to run.
The directory hierarchy is :
.
├── META-INF
│   └── MANIFEST.MF
├── com
│   └── yet
│   └── another
│   └── test
│   └── agent
│   ├── CnAgent.class
│   └── CnTransformer.class
└── lib
└── javassist-3.18.2-GA.jar
and the MANIFEST.MF file content is :
Manifest-Version: 1.0
Class-Path: lib/javassist-3.18.2-GA.jar .
Agent-Class: com.yet.another.test.agent.CnAgent
Created-By: 1.8.0_11 (Oracle Corporation)
Can-Retransform-Classes: true
I create jar ball by following command:
jar cvfm CnAgent.jar META-INF/MENIFIEST.MF . lib
when I load the Agent with Attach API of JVM.
the error prints :
error when transform : javassist/ClassPool
java.lang.NoClassDefFoundError: javassist/ClassPool
which means the javassist library cannot be found by agent code.
So my question is :
How to set Agent library's class path letting it find the dependencies?
Why the Class-Path option in MANIFEST.MF not works , does it only for jar directly ran in command line ?
Thanks your wisdom :)
You can use the option -Xbootclasspath: (sets the path) or -Xbootclasspath/a: (appends the given path to the existing boot class path) (refer to doc from oracle). But, as described in the link, it is non-standard.
As an alternative, you can copy the missing jar file in the %JAVA_HOME%/jre/lib/ext directory.
Per Guido's comment above, you should add Boot-Class-Path to your agent MANIFEST.MF.
See these java.lang.instrumentation docs (Manifest Attributes section)
In my case, I have this in Ant's build.xml:
<manifest file="META-INF/MANIFEST.MF">
<attribute name="Premain-Class" value="de.bodden.tamiflex.playout.Agent"/>
<attribute name="Main-Class" value="de.bodden.tamiflex.playout.Agent"/>
<attribute name="Can-Retransform-Classes" value="true"/>
<attribute name="Implementation-Version" value="${tf.version}"/>
<attribute name="Boot-Class-Path" value="guava-22.0.jar:guice-4.1.0.jar" />
</manifest>
and then copy the guice and guava jars to the directory I run the command from e.g. java -verbose:class -javaagent:poa.jar -jar ExampleProject.jar > loaded.txt
This also lists all the classes loaded to allow you to debug what Java class loader is actually doing.
Neither option from whiskeyspider worked for my case.

Accessing gradle resources from Java

I have a gradle-based java project with this structure
.
├── myproject
│ ├── src
│ | └── main
│ | ├── java
│ | └── resources
│ | └── myresource.xml
| ├── build
| | ├── classes
| | | └── main
│ | | └── myresource.xml
| | ├── resources
I'm trying to access some files in the resources folder using a ClassLoader, like this
ClassLoader.getSystemClassLoader().getResoure("/myresource.xml");
but it does not find the file.
The only way I have found to access those files is by exploring the known structure of the project
Path resourcesPath= FileSystems.getDefault().getPath(System.getProperty("user.dir"), "/src/main/resources/");
Any idea on what am I doing wrong?
Well, it seems my difficulties came from another problem (resources not being copied to the proper places). Once I solved that problem, the ClassLoader was able to find my resources using either of these two forms:
ClassLoader.getSystemClassLoader().getResource("./myresource.xml");
ClassLoader.getSystemClassLoader().getResource("myresource.xml");
Edit: When using the jar embedded in other applications, the former solution do not work, use this in that case:
Thread.currentThread().getContextClassLoader().getResource("myresource.xml")
http://docs.oracle.com/javase/7/docs/api/java/lang/Class.html#getResource(java.lang.String)
For example something like MyMain.class.getResource("/config.txt") or use relative path when appropriate.
Maybe you should use it this way:
Thread.currentThread().getContextClassLoader().getResource("myresource.xml")
If i use ClassLoader.getSystemClassLoader().getResource("myresource.xml")
When I use it as a jar package embedded in other applications, I still have no access to resources.
Another solution to access Java resources in the build.gradle with in Gradle in Groovy is to do for example to read VERSION file in the Java resources:
static String getVersion(File rootFile) {
File versionFile = new File(rootFile, "src/main/resources/VERSION")
byte[] versionBytes = Files.readAllBytes(versionFile.toPath())
String version = new String(versionBytes, StandardCharsets.UTF_8).trim()
version
}
version = getVersion(project.projectDir)
It avoids manipulating the ClassLoader.
Note: sourceSets.main.output.resourcesDir does not exist yet in function of the task.

Categories