ND4J NoAvailableBackendException despite following all documentation steps - java

I have an annoying issue with ND4J while attempting to implement deeplearning4j into a project. As the title already says, it is a NoAvailableBackendException. I am getting this error with maven,
the full pom.xml is here.
The stacktrace is here.
The only thing I can think of that might have an effect is the fact that I am attempting to run this on a minecraft server. I do not quite know how to account for this or how to solve this issue if this is the cause.
I have attempted to use the maven import for nd4j;
<dependency>
<groupId>org.nd4j</groupId>
<artifactId>nd4j-native-platform</artifactId>
<version>${dl4j-master.version}</version>
</dependency>
In my IDE this does not give any errors, and it correctly packages.
I have the NoAvailableBackendException when using the packaged jar. I have followed all the troubleshooting steps the documentation offers:
You haven't configured an ND4J backend at all.
It is in maven without any issues
You have a jar file that doesn't contain a backend for your platform.
I am importing the nd4j-native-platform, the nd4j-native also does not give a different result.
You have a jar file that doesn't contain service loader files.
I have decompiled the packaged jar, it contains the necessary service loader files according to the documentation.
I have also attempted to use a different version of nd4j (and of course matching it with deeplearning4j-core). Also to no avail.
Any help is highly appreciated.

Related

How to add a dependency to module path?

I am very new to programming, and while working on my first software development project I came across the error: "package javax.activation is not visible." I have read other posts that have said that in order to fix the error, one must add a dependency to the module path?
Because I'm just starting out, I really don't know what this means and how to go about that, and was wondering if someone could point me in the right direction? Thanks in advance. (I'm also using JGrasp if that matters)
Broadly speaking, a dependency is code (often written by someone else) which your application needs to compile.
A dependency in your case, is a java library which has classes that need to be on your class path. You can find more about class paths here: https://docs.oracle.com/javase/tutorial/essential/environment/paths.html. In order to overcome error mentioned in a question, you need to have javax.activation module dependency on your class path.
You could do it in few ways. In the majority of IDEs (in your case we are talking about JGrasp) there is a way to add a dependency to the project directly. Then, your IDE would compile the code with given dependency on a class path and problem would be solved. And that would be the first and most beginner-friendly way, unfortunately I haven't a faintest idea about JGrasp so I'm going to focus on other solutions.
The second way you could do this is to build your program with build automation tool, such as Apache Maven or Gradle. You should definitely check those guys out, as they are insanely useful when it comes to building Java code and, sooner or later, you will probably start using them anyway. Let's say you have chosen Apache Maven. In your project you would then have a pom.xml file and you would simply look-up the needed dependency in Maven Central repository, add it to your dependencies section in pom.xml file and build the application. Your pom would look something like this:
<project>
...
<dependencies>
<!-- https://mvnrepository.com/artifact/javax.validation/validation-api -->
<dependency>
<groupId>javax.validation</groupId>
<artifactId>validation-api</artifactId>
<version>2.0.1.Final</version>
</dependency>
</dependencies>
...
</project>
Of course Apache Maven is not a lightweight tool so you would have to take some time to learn how to build code with it. I recommend starting with this tutorial:
https://maven.apache.org/guides/getting-started/maven-in-five-minutes.html
I also encourage you to get your hands on Apache Maven docs, as it is quite readable and transparent.
And the last way I can think of is to manually compile your application and include the required dependency during the compilation process. There are plenty of tutorials on SO that tell you how to do this, so I'll simply summarize and indicate the resources. What you need to do is to find the required dependency jar package. You will want to search the maven central repository (see: https://mvnrepository.com/) and from there download your .jar file. The next thing you need to do is to learn how to compile your Java code to .class files including the downloaded jar. To acquire such a wonderful skill, please see this one: How to include jar files with java file and compile in command prompt
Amongst those three ways, the recommended one is to get to know with build tools such as Apache Maven or Gradle. Hope I helped you! Good luck

How come my pom.xml dependency isn't found for my Jenkins plugin?

My pom.xml has this dependency:
<dependency>
<groupId>net.sf.json-lib</groupId>
<artifactId>json-lib</artifactId>
<version>2.4</version>
<classifier>jdk15</classifier>
</dependency>
When I use the XMLSerializer it throws an exception: java.lang.NoClassDefFoundError: nu/xom/Node
If I run the class locally and add the JAR to my classpath, everything works as expected. I'm running this class as a Jenkins plugin so I don't expect to be manually defining classpath - I figured that's what Maven should be handling.
It's important to note that Jenkins plugins require me uploading an hpi file that is created from Maven. It is not running based on the output jar. If I go on Jenkins box and manually put the xom JAR into WEB-INF/libs, it works. But obviously that means this plugin wouldn't for other people, which is self-defeating.
Here is minimal code causing error: https://github.com/DaveStein/parser-sample
The Readme has exact repro steps.
Note on chosen answer
The PR to my sample repo got me most of the way to where I needed to be. I did have a few other issues that had to get resolved, but the JSONObject conflict was the core problem. I took out all GlobalConfiguration as Jesse's PR suggested. The only other issue that might concern a future viewer was some glitch when using xom as explicit dependency while also using a higher version than 1.626 for org.jenkins-ci.plugins at the time of this post.
Jenkins core bundles json-lib. (A forked copy, not that it matters for purposes of this question.) It does not bundle the optional dependency¹ XOM, whatever that is. When your plugin loads XmlSerializer.class, it gets defined by the class loader for Jenkins core, which then attempts to link against classes such as nu.xom.Node. Since this is not available in the defining loader of XmlSerializer—the Jenkins core class loader (more or less jenkins.war!/WEB-INF/lib/*.jar)—you get an error. The fact that a class by that name is accessible in your plugin class loader is of no import, excuse the pun.
If your plugin needs to use its own versions of classes which are normally bundled in Jenkins core and exposed to plugins implicitly, then it needs to not only bundle those JARs (a regular compile-scoped Maven dependency suffices for that purpose), but to also use the pluginFirstClassLoader option. Before attempting to do so, you had better understand Java class loading semantics thoroughly, or you will be lost in a maze of cryptic² ClassCastExceptions and LinkageErrors.
By the way the mvn hpi:run command normally used to test plugin code iteratively does not simulate a realistic class loading regime. So if you are using pluginFirstClassLoader or any other tricks in this space, always double-check the resulting class loading behavior by (re-)installing an *.hpi in a sample Jenkins instance, for example using /pluginManager/advanced, or the install-plugin CLI command. Judging by your description, you were already doing that (and perhaps unaware of hpi:run).
¹The original sin here is use of optional dependencies. json-lib should rather have defined a distinct artifact json-lib-xom with hard dependencies on json-lib and xom. That would ensure that any given class loader can either see XmlSerializer and its dependencies, or neither.
²No progress on JDK-6273389, alas. Marked as a duplicate, but what it is a duplicate of, I am not sure. Theoretically Java 9 modules make questions like this obsolete—by imposing such onerous restrictions that applications like Jenkins could not use that module system to begin with.
please google "noclassdeffounderror vs class not found" , this error means the class dependency is in fact found but is not available in run time.
Try these steps:
Run mvn clean package and mvn clean install
Check if your maven environment is correct and has latest jars
Check if the installed target project contains the required jars
Check if dependency type is selected as runtime and not only as
compile time in pom.xml
Here is an example of using runtime dependency:
<dependency>
<groupId>group-a</groupId>
<artifactId>artifact-b</artifactId>
<version>1.0</version>
<type>bar</type>
<scope>runtime</scope>
</dependency>
My guess is that the local version of the XOM jar is not the same as the one that is used in your Maven version. To verify use the dependency:list Maven command to list out all your dependencies. Verify if the XOM dependency listed is the same version as the local jar.
Probably error on jenkins occurred while this dependency has been loaded for the first time and now it's considered as complete. Try to remove dependency from jenkins' maven local repository and re-run. That might help you

Getting Spring (and other libraries) in a Maven world

I am following an online tutorial, and am stuck trying to get Spring libraries to run things with. The instructions are either a little out of date or assume I know how to do things in a Maven world that I do not.
They give web addresses on the Spring site from which they say I can "download the jars". But you can't download jars from any of them; evidently what you can do is find pom.xml fragments and use them to download jars. If you know how.
For example: the tutorial says:
Finally, following is the list of Spring and other libraries to be
included in your web application. You simply drag these files and drop
them in WebContent/WEB-INF/lib folder.
commons-logging-x.y.z.jar
org.springframework.asm-x.y.z.jar
org.springframework.beans-x.y.z.jar
org.springframework.context-x.y.z.jar
org.springframework.core-x.y.z.jar
org.springframework.expression-x.y.z.jar
org.springframework.web.servlet-x.y.z.jar
org.springframework.web-x.y.z.jar
spring-web.jar
I love the "simply" here.
I have many but not all of these (version 3.2.4). I have googled the library names and used jarfinder for the ones I don't have, but haven't found them all. It seems to me I'm flailing around a bit, just trying this and that until something works. I hate that.
I'm hoping someone can give me a set of steps I should follow when faced with this sort of thing, since we're faced with it all the time. Do I need to stop my current study of programming and learn Maven inside and out so that I can configure the bloody system so I can program again?
Is it a maven project you have?
If so in the pom.xml you just need to place the “pom fragments” in the xml.
i.e. between the dependencies tags. For example,
<dependencies>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.8.2</version>
</dependency>
</dependencies>
Once you have done that run maven install. Under Eclipse it is: run as > maven install.
That will download all the dependences i.e. jars you have listed into the you local maven repository.
I think the default is C:\Users\yourName\.m2\repository
Found most of the needed jar files here:
http://www.java2s.com/Code/Jar/CatalogJar.htm
while I downloaded commons-logging here:
http://mvnrepository.com/artifact/commons-logging/commons-logging/1.2
My advice is, stop study you program language, Java I guess and learn how to integrate maven dependencies of Spring in your Maven project.
http://spring.io/blog/2009/12/02/obtaining-spring-3-artifacts-with-maven/
You cannot start the house from the roof

Which JAR does Java pick for a certain import?

Dependency issues, we've all dealt with them, but I'm mostly used to C# and now working in Java so I have some questions.
Let's say I add a library to my project, called ExtLib.
ExtLib has a certain library included in its lib-folder, let's call it LogLib-1.0.
I'm using Eclipse and I've made a User Library for ExtLib, included its main jar file and all of the files in its lib-folder. So far so good.
But now I want to do some logging of my own, so I make another User Library and add the newer LogLib-1.1 to it, because it has some new features I want to use.
Can I ever be sure I'm not breaking ExtLib this way?
I know .NET uses the Global Assembly Cache and methods like that, but I have no clue how Java handles this. I tried Googling, but didn't find much, a few mentions of the Classloader here and there, but nothing helpful.
Can anyone tell me what a proper way to deal with this issue is? Or is it no issue at all?
In this specific case (LogLib-1.0 and LogLib-1.1) we're dealing with the same library that is both a direct dependency of your application, and a "transitive" dependency via the ExtLib. In this situation, dependency management can come to help.
It will probably reason that LogLib-1.1 is a backward compatible release of LogLib-1.0, and it will decide that your application can run fine using only LogLib-1.1.
In the Java world, tools like Maven, Gradle or SBT exist to help you in this. Maven is the most widespread, and other tools often are compatible with Maven.
Usage
To solve this situation using Maven, you would add a file called pom.xml to your application, stating it depends on LogLib version 1.1. That might look like this (note that this example is pure fiction):
<dependency>
<groupId>org.loglib</groupId>
<artifactId>loglib</artifactId>
<version>1.1</version>
</dependency>
The ExtLib you're using also has a pom.xml shipped with it, and it might state
<dependency>
<groupId>org.loglib</groupId>
<artifactId>loglib</artifactId>
<version>1.0</version>
</dependency>
Maven (or any other tool) would decide that including LogLib-1.1 is sufficient to get your application running. When using Maven, mvn depedency:tree helps you visualise that.
Deployment
With respect to the packaging / deployment question: mvn package will package your application to a jar, war or ear archive, including only the dependencies you need (and not two versions of the same lib). This makes you don't have to worry about the order in which your application server reads the jar files.

NoClassDefFoundError using jcifs library

I have a Maven project (ejb container) where I need to use the jcifs library.
I made the entries in pom.xml like:
<dependency>
<groupId>jcifs</groupId>
<artifactId>jcifs</artifactId>
<version>1.3.17</version>
<type>jar</type>
</dependency>
Everything is okay, I see the jar file inside the ear package, I see it also in the dependencies of the project, I can use the classes but at runtime I get:
javax.ejb.EJBException: java.lang.RuntimeException: java.lang.NoClassDefFoundError: jcifs/smb/SmbFile
at org.jboss.ejb3.tx.Ejb3TxPolicy.handleExceptionInOurTx(Ejb3TxPolicy.java:63)
What am I doing wrong?
Right-click the webproject and go to properties -> Java EE Module Dependencies. Check the Maven dependencies is checked.If not check and redeploy .
I would like to help the people who are as stupid as I am. This is not really the about the same scenario as asked for in this question, but this question helped me to figure out my problem.
I wrote a little code in a scratch file, in my case in Intellij. Before that I added the jcifs dependency to my maven project, assuming that the scratch file would find it there, which in hindsight is a bit stupid, because how would the scratch space know.
So my hint is to have a look where the dependency will actually have an effect, i.e. is available - compile time, runtime, scratch space?
Also I wanted to mention that I used this library, which works very fine for me https://github.com/hierynomus/smbj.
So in my case I edited the run configuration so that it would take the maven modules configuration into the classpath.
So my guess is that #SANN3 's answer is correct for the actual question.

Categories