We Keep Coding

Java Compiler API (Linux): cannot find symbol for custom classes

I have a rather specific problem with the Java Compiler API.
For my Usecase I have to generate, compile and load a java-class at runtime in my Web-Application (using Tomcat).
In order to do that, I create a .java-File on my disk, compile it, using the Compiler API and and then load it via a custom classloader. All of that works just fine on windows, but once I try running it on the Ubuntu-system it is supposed to be installed on, the compiler cannot find any custom classes that are referenced in the class.
(Both systems use java 8 and the same version of Tomcat.)
The file is created correctly and stored in a subdirectory of the webapplication. ('catalina.home/webapps/projectname/resources/')
All the .class-files of my application are stored in 'catalina.home/webapps/projectname/WEB-INF/classes/'.
This directory is later added to the classpath of the compiler:
JavaCompiler compiler = ToolProvider.getSystemJavaCompiler();
File classesFolder = new File(TestFile.class.getProtectionDomain().getCodeSource().getLocation().getFile());
List<String> optionList = new ArrayList<String>();
//Adding the classpath to the OptionList
optionList.addAll(Arrays.asList("-classpath", System.getProperty("java.class.path") + ";" + Settings.projectPath + "WEB-INF/classes/;" + classesFolder.getAbsolutePath() ));
CompilationTask task = compiler.getTask(null, fileManager, diagnostics, optionList, null, files); //a task is initialized with the values previously specified and saved
task.call();
The assigned values for the classpath do point to the directory, where the .class-Files are. But for some reason I keep getting
cannot find symbol
symbol: class *nameOfReferencedClass*
location: class testfiles.TestFile
for every single class that is referenced... (I did not forget to import things, the packages are declared correctly and there are no syntax errors or misspelled names. I double checked that and as I mentioned, it works fine if the application is running on windows).
I even tried to save the src-File inside of /WEB-INF/classes/, where the referenced classes reside and compile it there, but that didn't help either.
I do not have any idea, why the compiler would not find these classes and what could be different for linux. Could it be related to the file permissions or to the way Tomcat is configured (e.g. if it is run as root instead of a user)?

Try using a colon instead of a semi-colon as the classpath separator. On windows, the semi-colon is correct, but on *nix a semi-colon actually ends a shell command, so a colon is used instead.
See the comment by #Slaw for a platform independent way to do this.

How to load My Jar to ClassPath at runtime through Java Coding?

I am working with Swing in Net Beans. I have my own jar which contains classes and methods inside it. I will call those classes and methods using JAVA Reflection API but before that I want to load my Jar into class path at run time. I have a J Button and on click of that I am getting Jar Name and Jar path. But I am failing to load Jar to classpath at run time. Got some links but were not helpful. Please provide me with simple example. I should load my jar to classpath. That's the only problem for me.I will take care of that. Please help.

You can load classes at run time through the use of a ClassLoader, take a look at URLClassLoader for example
File yourJarFile = ...;
URLClassLoader classLoader = new URLClassLoader(new URL[]{yourJarFile.toURI().toURL()});
This will then allow you to load classes and instantiate them...
Class class = classLoader.loadClass("fully.qualified.packagename.to.your.AwesomeClass");
You can then instantiate them using something like...
Object obj = class.newInstance();
Or reflection if you want to use a specific constructor. Just remember, you won't be able to reference these classes directly within the current class loader context, as the current class loader knows nothing about them

In Java, is there a logical reason why I cannot say "java MyClass.class " from command line?

Maybe this question may be splitting hairs, but when I compile a file from command line like :
javac MyClass.java
then afterward I cannot run it by saying
java MyClass.class
I have to call:
java MyClass
What is the motivation for this notation?

Because you run a class on the classpath, which may be contained inside a jar for example. You couldn't use your syntax in that case.

Java compiler needs a compilation unit; this is by default (at least) a java source file, with the whole of classes defined in it and its dependencies.
Java interpreter (the jvm) needs a single class with a main method as entry point of the execution - it must start somewhere.

You'd have to ask Sun (now Oracle) for the development history, but I do want to point out that for folks who are just using Java rather than developing Java, "java DoSomething" is easier to remember, and to type, than "java DoSomething.class"

There is no way to run a Java program that is not a class. For that reason, there is no reason to mandate typing the ".class". You might also invoke a class from within a JAR on your path, or directly, but it's still instantiating a class (possibly a "default" class from the Manifest).

Because the name of the class is MyClass and not MyClass.class. And when running java you specify the CLASS NAME and not the PATH to the actual compiled file.
For more in depth knowledge I guess Sun & Oracle will have to answer :)

Imagine that you have a class named package and you have a class named Class, in a package named package,
--CurrentFolder
--package
Class.class
package.class
so executing java package.class may lead to an undecidability to the compiler!

Best way to override MATLAB's default static javaclasspath

MATLAB is configured to search its static java class path before searching the user-modifiable dynamic path. Unfortunately, the static path contains quite a number of very old public libraries, so if you are trying to use a new version you may end up loading the wrong implementation and get errors.
For instance, the static path contains an old copy of the google-collections.jar, which has long been supplanted by Google's guava library and which has some of the same class names (e.g. com.google.common.base.Objects). As a result, if you invoke a Guava method that uses a newer method of one of such a class, you will end up getting surprising NoSuchMethodErrors because the google-collections jar is found first.
As of R2012b, MATLAB lets you specify additional jars to add to the static path by putting a javaclasspath.txt file in your preferences folder, but that adds jars to the end of the path, and doesn't let you override jars that are built into MATLAB.
So what is the best way around this?

I got an official response from Mathworks:
As of MATLAB R2013a (also in R2012b), classes can be added to the front of the static Java class path by including the following line in javaclasspath.txt:
<before>
Any directory that is after this line in javaclasspath.txt will be added to the front of the static Java class path. This is an undocumented use of javaclasspath.txt as of R2013a.
But overall in MATLAB, the ability to add classes to the front of the static Java classpath is not available through javaclasspath.txt in MATLAB 8.0 (R2012b).
MATLAB searches for classpath.txt in the following order:
In the startup directory. As of MATLAB 8.0 (R2012b) a warning will be shown if the file is found there and it will be ignored.
In the first directory on the MATLABPATH environment variable. (This environment variable is used in the bin/matlab shell script on Linux and in general is not used by the end-user).
In the toolbox/local directory.
Although the MATLABPATH environment variable of point 2 is normally not used by end-users we can use it in a workaround to allow reading a custom classpath.txt outside of the toolbox/local directory.
On Windows:
You will need to create the MATLABPATH environment variable. The first directory on it should be your directory with the custom classpath.txt AND you will also need to add the toolbox\local directory as second option. So from a cmd prompt you could do:
set MATLABPATH=c:\Users\user\Documents\myMATLABClasspath;c:\Program Files\MATLAB\R2012b
\toolbox\local
matlab.exe

One hack that appears to work is to add the jar to the top of the classpath.txt file that can be found in your MATLAB installations toolbox/local folder. Unfortunately, this is automatically generated and may get rewritten at some unspecified time, such as when you install new toolboxes, so this approach would require you to have some way to notice when this happens and reapply the hack.

If you're distributing a jar that's intended to be used with matlab, it may be better to use proguard as described at http://code.google.com/p/guava-libraries/wiki/UsingProGuardWithGuava.
If you specify that all of your classes and their (public) fields and methods are to be preserved and include guava as a program jar (not a library), then it will rename all of guava's methods and update your compiled bytecode to reference the new names.
It seems a bit hackish, but depending on the audience, it may be significantly easier than teaching your users about static vs. dynamic classpath, and it won't break any matlab code that depends on the old behavior.

Instead of obfuscating the package as suggested by #user2443532, I have found it easier to "shade" the conflicting package instead of obfuscating it - unless you actually need obfuscation. One easy way to do this is to build your package using Maven and use the maven-shade-plugin. Internal calls are modified automatically, so you don't need to modify any of the Java code.
Direct calls from Matlab will need to be modified - for example, calls to com.opensource.Class become shaded.com.opensource.Class.
For more info on shading, see What is the maven-shade-plugin used for, and why would you want to relocate Java packages?

how to avoid "duplicate class" in Java

Suppose I have have a java project myProject and am using an external library jar (someJar.jar), which has a class com.somepackage.Class1.class.
Now I find an updated version of Class1.java which fixes a bug in the original jar.
I include the new Class1.java in my source code under package com.somepackage
When I build the project (e.g., using Netbeans), there is a dist\myProject.jar which contains the classcom.somepackage.Class1.class and a dist\lib\someJar.jar which also contains a class with the same name.
When I run the file (e.g, using java -jar dist\myProject.jar), the new version of Class1.class is used (as I want).
How does Java decide which class file to run in case of such duplicates? Is there any way I can specify precedence ?
Is there any 'right' way to avoid such clashes?
In Proguard, when I try to compress my code, I get a duplicate class error. How do I eliminate this?

Java decides which one to use based on the order of the classpath. List yours first and you'll be fine.
The "right" way would be to fix the orignal source, but sometimes that's not always an option.
I haven't used ProGuard, but I have re-jarred libaries before that had duplicate classes. The solution in my case was to tell Ant to ignore duplicate classes. I would assume ProGuard would have that support too.

Can you not create an updated jar file which contains the bug fix? It's going to make things a lot simpler if you don't have two versions of the same fully-qualified class around.

1) Updated Jar is a better solution.
2) Use a different class name. Is there a reason, why you want to use the same class name and same packing? I don't think there is a reason.
3) create a wrapper/ proxy class, that encapsulate all the calls to the jar and you can decide to call this new class that fixes the bug ( provided it has a different name and packaging)