We Keep Coding

Trouble With Java getResourceAsStream, confusion with how Packages work

I'm revisiting Java after briefly learning it in high school to work through the lwjgl git book. There's a section that kind of loses me where the fragment shader / vertex shader is supposed to be loaded, using the function 'getResourceAsStream'.
I've tried different approaches, and vainly tried to debug / print the locations where the JVM was searching for the file, but I'm kind of at a loss. I am very confused, but would also prefer understanding why this code doesn't work on my machine, and how to get it to work (with this package / class loading approach).
Since the classes both share the 'main' class, I thought the JVM would search starting there for a 'resources' folder. But this does not seem to be the case.
I do think it has to do with the package declarations, but I don't understand enough to proceed.
Here are the relevant files and locations... I tried to only include what I thought important.
// project-root/src/main/java/org/lwjglb/engine/Utils.java
package main.java.org.lwjglb.engine;
// other imports...
public class Utils {
public static String loadResource(String fileName) {
InputStream in = class.forName(Utils.class.getName()).getResourceAsStream(fileName);
// 'in' is null after this call
// other code...
}
}
// project-root/src/main/java/org/lwjglb/game/Renderer.java
import main.java.org.lwjgl.engine.Utils;
// other imports...
public class Renderer {
public void init() {
Utils.loadResource("/vertex.vs"); // fails
}
}
// project-root/src/main/resources/vertex.vs
// vertex shader code...
The issue is that the getResourceAsStream() call returns null, so the following code that depends on a valid result crashes with a java.lang.NullPointerException.
What path should I pass in to getResourceAsStream() so that it will not return null?

Assume the following directory structure:
|-- src
`-- test
|-- java
| `-- shaders
| `-- ShaderParser.java
`-- resources
`-- shaders
`-- fragmentShader.txt
The proper way to get the resource "fragmentShader.txt" from "ShaderParser.java" is:
Class clazz = Class.forName(TestUtils.class.getName());
InputStream in = clazz.getResourceAsStream("../../resources/shaders/fragmentShader.txt");
The ../ means "traverse upward one directory".
The path starts in the src/test/java/shaders/ directory so we use ../../ to traverse upward twice into src/test/ and then tack on the rest of our path resources/shaders/fragmentShader.txt.
It is important to remember that when using Class#getResourceAsStream(), it uses the package that the class resides in as the base directory. It ONLY uses relative paths because it is trying to load the resource using that class' ClassLoader. and will not work if supplied with a fully qualified path or a path starting at the project root.
Please refer to the documentation for more information.

I had erroneusly thought that Java had a reserved directory (e.g. 'resources') where it would look first to load files. Kind of similar to Flask in python.
However, I understand now that the class.getResourceAsStream() function either takes relative paths (to the current 'lowest' level package) or absolute paths (from the project's top-level root).
I had furthermore erroneously thought that I could try doing a relative path load, from one of the higher-level packages. As in, I'm in package main.java.org.lwjgl.engine.Utils, but when looking for the file main.resources.vertex.vs, I could start the search from the 'main' package and pass 'resources/vertex.vs' or 'resources/vertex.vs'. I don't know if this is possible, but it feels erroneous.
Instead, the solution is to pass the file location '/main/resources/vertex.vs'. This will start at the project-root, and correctly load the file.
I'm still confused as to how the lwjgl code worked, but I see that it could be possible that the classPath was set to 'main', so passing an absolute path could work.

Jxbtrowser retrieve platform specific artefact at runtime

I'm writing an intelij plugin and would like to download the platform specific artefact at runtime.
I've loaded the platform specific jar into a class loader but the ChromiumExtractor cannot access the nested resources when prefixed with "/". So I can access the resource as "chromium-mac.zip" but the library cannot.
I've tried to unzip the nested zipped chromium artefact into the correct directory but this does not leading to a working solution. So now I've been trying to piece together the way the library extracts the artefact but it's rather tedious as the code is obfuscated.
Does the jxbrowser plugin have some support for retrieving the artefact at runtime. Could such support be added (jxbtrowser devs use SO for support questions etc, this is a message to them :D ) ?
Approach taken :
// inside intelij plugin . The plugin has the jxbrowser-6.6.jar
// and license.jar loaded into the classloader. the platform specific
// artefact will be retrieved manual).
val cl = URLClassLoader(arrayOf(URL("file://.../jxbrowser-mac-6.6.jar")), Browser::class.java.classLoader)
val backup = Thread.currentThread().contextClassLoader
try {
Thread.currentThread().contextClassLoader = cl
// can access like this
Thread.currentThread().contextClassLoader.getResource("chromium-mac.zip")
val ce = ChromiumExtractor.create()
// cannot access as resource is retrieved "/chromium-mac.zip" ?
ce.extract(BrowserPreferences.getChromiumDir())
browser = Browser()
} finally {
Thread.currentThread().contextClassLoader = backup
}

The following does the trick, The resource jar had to be in the same class loader as the client jar (as well as the license). It would be nice if JxBrowser added a helper for this that is capable of performing the download and initialising chromium, perhaps taking just a path for a persistent storage directory.
private fun initializeJxBrowser(): Browser {
if(ChromiumExtractor.create().shouldExtract(BrowserPreferences.getChromiumDir())) {
val cl = URLClassLoader(arrayOf(
URL("file:.../license.jar"),
URL("file:.../jxbrowser-mac-6.6.jar"),
URL("file:../jxbrowser-6.6.jar")
))
cl.loadClass("com.teamdev.jxbrowser.chromium.BrowserContext")
.getMethod("defaultContext")
.invoke(null)
}
return Browser()
}

JAR indexing and getResources

It appears to me that JAR file indexing breaks the mechanics of ClassLoader.getResources(). Consider the following program:
import java.io.*;
import java.net.*;
import java.util.*;
public class TryIt {
public static void main(String[] args) throws Exception {
URL[] urls = {
(new File("a.jar")).getAbsoluteFile().toURI().toURL(),
(new File("b.jar")).getAbsoluteFile().toURI().toURL()
};
URLClassLoader cl = URLClassLoader.newInstance(urls);
String[] res = { "foo", "foo/", "foo/arb", "foo/bar", "foo/cab" };
for (String r: res) {
System.out.println("'" + r + "':");
for (URL u: Collections.list(cl.getResources(r)))
System.out.println(" " + u);
}
}
}
Now prepare the JAR files mentioned in that program:
mkdir a/foo b/foo
touch a/foo/arb a/foo/bar b/foo/bar b/foo/cab
echo "Class-Path: b.jar" > mf
jar cfm a.jar mf -C a foo
jar cf b.jar -C b foo
If you run java TryIt, you will get output like this:
'foo':
jar:file:…/a.jar!/foo
jar:file:…/b.jar!/foo
'foo/':
jar:file:…/a.jar!/foo/
jar:file:…/b.jar!/foo/
'foo/arb':
jar:file:…/a.jar!/foo/arb
'foo/bar':
jar:file:…/a.jar!/foo/bar
jar:file:…/b.jar!/foo/bar
'foo/cab':
jar:file:…/b.jar!/foo/cab
But if you run jar -i a.jar to create an index, then the same command as above prints this:
'foo':
jar:file:…/a.jar!/foo
'foo/':
jar:file:…/a.jar!/foo/
'foo/arb':
jar:file:…/a.jar!/foo/arb
'foo/bar':
jar:file:…/a.jar!/foo/bar
'foo/cab':
jar:file:…/b.jar!/foo/cab
The index itself looks like this:
JarIndex-Version: 1.0
a.jar
foo
b.jar
foo
Doesn't the contract of getResources imply that all available resources matching the given name should be returned?
Finds all the resources with the given name.
Doesn't the JAR File Specification allow indexed packages to span multiple JAR files?
Normally one package name is mapped to one jar file, but if a particular package spans more than one jar file, then the mapped value of this package will be a list of jar files.
Is there some specification somewhere which says that what I'm observing is indeed correct (or at least permissible) behavior?
Is there some workaround to get all named resources despite the index?

This appears to be a bug.
I've reported it to Oracle, and it's now in their bug database as bug 8150615.
I did some digging around in the OpenJDK sources and found the reson for this behavior in there.
The relevant class here is sun.misc.URLClassPath. It contains a (lazily constructed) list of loaders, and queries each loader in turn to assemble its result. However, if a JAR file contains an index, then the JAR files therein will explicitely be excluded from getting added to the list of loaders. Instead, the loader for the JAR containing the index will query said index for the name in question, and traversed the resulting list. But here is the catch: this happens in a method URLClassPath$JarLoader.getResource which returns a single Resource object. It is not possible for this method to return multiple resources. And as all objects in the index are handled by a single loader, a single resource is all we get.

Error in Udf of pig

I am new to pig. I wrote a UDF in pig and used it in my pig script. But it gives following error
ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1070: Could not resolve UserDefined.PartsOfSpeech using imports: [, java.lang., org.apache.pig.builtin., org.apache.pig.impl.builtin.]
Here is my UDF code
public String exec(Tuple input) throws IOException {
//my code here
}
Here is my pig script
REGISTER /home/bigdata/NetBeansProjects/UserDefined/dist/UserDefined.jar
a = load '/user/bigdata/json' using TextLoader() as (input:chararray);
b = foreach a GENERATE UserDefined.PartsOfSpeech(input);
In the above code UserDefined is my package name and PartsOfSpeech is my class name

The error message says that Pig cannot find UserDefined.PartsOfSpeech.
What package declaration does PartsOfSpeech.java have at the top of the file?
If the package declaration is package com.my.company; try this instead:
REGISTER /home/bigdata/NetBeansProjects/UserDefined/dist/UserDefined.jar
a = load '/user/bigdata/json' using TextLoader() as (input:chararray);
b = foreach a GENERATE com.my.company.PartsOfSpeech(input);
That is, replace UserDefined.PartsOfSpeech(input) with com.my.company.PartsOfSpeech(input) since the UDF is located in the package com.my.company.
Also, consider using the DEFINE keyword in your Pig script so you don't need to repeat com.my.company every time you use PartsOfSpeech.
DEFINE PartsOfSpeech UserDefined.dist.PartsOfSpeech();
REGISTER /home/bigdata/NetBeansProjects/UserDefined/dist/UserDefined.jar
a = load '/user/bigdata/json' using TextLoader() as (input:chararray);
b = foreach a GENERATE PartsOfSpeech(input);
There is more information about DEFINE in Chapter 5 of Alan Gates' Programming Pig: http://chimera.labs.oreilly.com/books/1234000001811/ch05.html#udf_define.
Here is an example of DEFINE from Gates' book:
--define.pig
register 'your_path_to_piggybank/piggybank.jar';
define reverse org.apache.pig.piggybank.evaluation.string.Reverse();
divs = load 'NYSE_dividends' as (exchange:chararray, symbol:chararray,
date:chararray, dividends:float);
backwards = foreach divs generate reverse(symbol);

Before compiling your UDF(java class) make sure you have mentioned package name properly. for example if you have mentioned package name-
package com.pig.udf;
It means you need to take care of directory in your linux box as well.
you can follow below mentioned steps to create jar -
Create directory using
mkdir -p com/pig/udf
Create your java class with package com.pig.udf
Compile your java source code using command
javac -cp /usr/lib/pig-0.12.0.2.0.6.0-76.jar YourClass.java
Then go to the directory where you want to create jar for now -
cd ../../..
Now create jar using below command
jar -cvf yourJarName.jar com/
Register the jar in your script using keyword "register" followed by path of the jar
Now use your jar with keyword com.pig.udf.YourJavaClassName
for your scenerio -
REGISTER /home/bigdata/NetBeansProjects/UserDefined/dist/UserDefined.jar
a = load '/user/bigdata/json' using TextLoader() as (input:chararray);
b = foreach a GENERATE com.pig.udf.PartsOfSpeech(input);

Package name breaks JNI classpath?

I'm working on a JNI application. It is a C program that calls some Java methods.
I wrote it following some internet examples.
First I created a simple example in Eclipse Indigo (it created the Java files in a "default package"). Then I used something like this in the C code:
options.optionString = "-Djava.class.path=/home/elias/workspace/Funciones/bin";
All worked fine but then I re-made the Java proyect to have the Java code in a package called "Funciones", so I modified to:
options.optionString = "-Djava.class.path=/home/elias/workspace/Funciones/bin/Funciones";
But now I doesn't work... I supposed it is something I have wrong in the classpath.
Can someone help me please?
Thanks.

You want the class path to point at the directory (or directories, or jar files) holding the top-level packages you're using, not inside the packages. So if your code is laid out like this:
~/Funciones/bin/
|
>----Funciones
| |
| >----Funciones.class – this is the class Funciones.Funciones
|
>----some other package
|
:
you need to add ~/Funciones/bin to your classpath. To find the class, you need to use its fully qualified name – the class name prefixed with the package name:
Class clsFunciones = Class.forName("Funciones.Funciones");
or, in JNI, the class descriptor:
jclass clsFunciones = (*env)->FindClass(env, "Funciones/Funciones");