We Keep Coding

How can you set the search path for JavaToRascal from Java

We tried to import a test rascal module and a module from the standard library using JavaToRascal.
The test module is stored in C:\Users\Klemens\workspace\RascalInterop\src\MyTest.rsc and contains:
module MyTest
Te java code containing the JavaToRascal invocation is as follows:
import java.io.PrintWriter;
import java.net.URISyntaxException;
import org.rascalmpl.interpreter.JavaToRascal;
import org.rascalmpl.interpreter.load.IRascalSearchPathContributor;
import org.rascalmpl.interpreter.load.StandardLibraryContributor;
import org.rascalmpl.interpreter.load.URIContributor;
import org.rascalmpl.uri.URIUtil;
public class RascalInterop {
public static void main(String[] args) throws URISyntaxException {
JavaToRascal j2r = new JavaToRascal(new PrintWriter(System.out), new PrintWriter(System.err));
IRascalSearchPathContributor modulePath = new URIContributor(URIUtil.createFileLocation("C:\\Users\\Klemens\\workspace\\RascalInterop\\src\\MyTest.rsc"));
j2r.getEvaluator().addRascalSearchPathContributor(modulePath);
try {
j2r.eval("import MyTest;").toString(); // Could not import module MyTest: can not find in search path
} catch (Exception e) {
System.out.println(e.getMessage());
}
try {
j2r.getEvaluator().addRascalSearchPathContributor(StandardLibraryContributor.getInstance());
j2r.eval("import IO;").toString(); // null pointer exception
} catch (Exception e) {
System.out.println(e.getClass());
}
}
}
The print in the first try block that tries to import our MyTest.rsc module results in:
Could not import module MyTest: can not find in search path
?[Advice](http://tutor.rascal-mpl.org/Errors/Static/ModuleImport/ModuleImport.html)
The second import attempting to import the IO module from the standard library results in:
class java.lang.NullPointerException
Any ideas how to use properly set the search path from a Java program?
We tried to use j2r.getEvaluator().addRascalSearchPathContributor in various ways but did not succeed in loading a MyTest.rsc module from the given directory.

Despite that these API will change in the near future (due to the compilation process and related changes), here's an answer. Two answers actually, one for Rascal files and one for Java code that it needs
For Rascal:
j2r.getEvaluator().addRascalSearchPathContributor
What you used is the correct way of doing things. So if it did not work, please provide more code so we can diagnose what goes wrong. So where is your module? Is it in a jar file or a binary folder? If its in a jar, you need some additional wiring I'm glad to explain.
The Rascal search path is distinguished from the Classpath for Java classes which are used by Rascal. So you have different API for that. We use classloaders to find Java files (such that it also works for situations like OSGI bundles in Eclipse):
Evaluator x = ctx.getEvaluator();
x.addClassLoader(getClass().getClassLoader());
This will make sure that the class loader used to load the current class is also used to load the class linked mentioned in the Rascal file. Of course you can also provide other class loaders. Note that if the libraries you depend on are loaded via OSGI, make sure you get a classloader from a class that is in a bundle that has access to these classes. The simple case is when everything is in the same jar file, then any classloader will do.

I think you should change the path to refer to the src directory instead of the source file:
new URIContributor(URIUtil.createFileLocation("C:\\Users\\Klemens\\workspace\\RascalInterop\\src"));
Also: probably you should use forward slashes without C:\, so /Users/.../src
AFAIK The null pointer exception is expected, evaluating import returns null, and you try to call toString().

Access classes from package

I'm developing an android test app and i'm going to access all internal class of android.view package. android.view is a package that is present in jar file. I tried by loading package name but it doesn't display the classes if any one tried
this already, please help.
Here's what I tried so far:
public static void main() throws ClassNotFoundException{
Class o =Class.forName("android.view");
Class[] C=o.getDeclaredClasses();
for(int i=0;i<C.length;i++) {
Classname = C[i].getName();
ClassesDisplayActivity.your_array_list3.add(Classname);
Log.i("Ramu","classname "+ C[i].getName());
}
}
}

It is not possible to determine at runtime all of the classes that are in a package using a standard class loader.
You might have some luck with this library though:
https://code.google.com/p/reflections/

Package is not a class. You cannot call Class.forName() for package and access classes that belong to class using getDelcaredClasses().
I do not know what do you really need, so I'd recommend you to explain this in separate question. probably you will receive better solutions.
However if you really need this you have to do the following:
Get your classpath by calling System.getProperty(java.class.path)
split this property to its elements by colon
iterate over the list and read each resource. If resource is jar you can use ZipInputStream, if it is a directory use File class.
filter list of resources you got at #3.
Fortunately you can use 3rd party library named Reflections that helps you to do all this without writing code.

Is possible to use .class files that are not in the build?

I wanted to add a system of adding more "commands" to my program from out side of it.
Basically, a folder that would be in the same directory as my jar (executable) that can have a varying number of "extensions"
Each "extension" would be a .class file that extends a certain abstract class that has methods defined for identification, etc.
However, I can't seem to find out if it's even possible to control these .class files from outside my main package.
Is this possible? If so how? And if not is there an alternative I could try?
Thanks!

As far as I understand you. You want to build a plugin system.
If so you should think about the java service provider approach.
In this approach you define a service provider interface (SPI) in your main
package and use the java META-INF/services location to lookup implementations.
In this case you can add extensions just by putting a jar file on the classpath.
As of java 1.6 you can use java.util.ServiceLoader.
Or you take a look at apache discovery
http://commons.apache.org/proper/commons-discovery/apidocs/org/apache/commons/discovery/tools/Service.html.
It's more powerful than the java.util.SerivceLoader since it lets you pass constructor arguments the the services it instantiates.
A possible plugin loader implementation could look like:
public interface ServiceInterface {
}
public class PluginHost {
public ServiceLoader<ServiceInterface> loadPlugins() {
File[] pluginLibraries = getPluginLibraries();
URL[] pluginLibUrls = new URL[pluginLibraries.length];
for (int i = 0; i < pluginLibUrls.length; i++) {
try {
pluginLibUrls[i] = pluginLibraries[i].toURI().toURL();
} catch (MalformedURLException e) {
throw new IllegalStateException("Unable to load plugin: "
+ pluginLibraries[i], e);
}
}
URLClassLoader pluginsClassLoader = new URLClassLoader(pluginLibUrls,
ServiceInterface.class.getClassLoader());
ServiceLoader<ServiceInterface> serviceLoader = ServiceLoader.load(
ServiceInterface.class, pluginsClassLoader);
return serviceLoader;
}
private File[] getPluginLibraries() {
// please implement
}
}

It's possible.
Jars are just zipped class files.
As long as the class files are in the classpath you're fine.
Unzip a jar file (rename from .jar to .zip and unzip) to see what the directory structure should be for the java packages.

Sure you can load a class at runtime and create a Class object also:
See this Class#forName(String, boolean, ClassLoader) API:
See this tutorial for examples.

For a simple extension mechanism have a look at ServiceLoader.
For a more complete extension mechanism have a look at OSGi.

Implementing Spring-like package scanning in Android

I'm attempting to implement a package-scanning feature, similar to Spring's component-scan, for the Android framework I'm developing. Basically, I would like to be able to specify a base package, e.g. com.foo.bar and retrieve all Class instances that have a particular annotation. I don't want to have to register every component with my framework as that would defeat the purpose of the auto scanning.
Based on my research, it seems that it's not possible with Java to retrieve resources given a package name using reflection. However, I briefly looked into the Reflections framework, and I'm wondering if there is an Android-compatible equivalent. If not, perhaps there is a slightly less obvious way to accomplish what I want to do.
I looked into the Spring source a bit to see how they achieved this, but I don't think what they are doing would work within the Dalvik runtime.
Update
Currently, the below code has been the best I can do to retrieve all classes that contain a specific annotation, but frankly it's a pretty poor solution. It makes some really unsafe assumptions about the ClassLoader plus it scans (and loads) all application classes.
public Set<Class<?>> getClassesWithAnnotation(Class<? extends Annotation> annotation) {
Set<Class<?>> classes = new HashSet<Class<?>>();
Field dexField = PathClassLoader.class.getDeclaredField("mDexs");
dexField.setAccessible(true);
PathClassLoader classLoader = (PathClassLoader) Thread.currentThread().getContextClassLoader();
DexFile[] dexs = (DexFile[]) dexField.get(classLoader);
for (DexFile dex : dexs) {
Enumeration<String> entries = dex.entries();
while (entries.hasMoreElements()) {
String entry = entries.nextElement();
Class<?> entryClass = dex.loadClass(entry, classLoader);
if (entryClass != null && entryClass.isAnnotationPresent(annotation)) {
classes.add(entryClass);
}
}
}
return classes;
}

I wanted to find all the subclass at runtime.
So I've been looking for android class scanning.
This is my final code from what I gathered in web.
You will get the idea.
public static void findSubClasses(Context context, Class parent) {
ApplicationInfo ai = context.getApplicationInfo();
String classPath = ai.sourceDir;
DexFile dex = null;
try {
dex = new DexFile(classPath);
Enumeration<String> apkClassNames = dex.entries();
while (apkClassNames.hasMoreElements()) {
String className = apkClassNames.nextElement();
try {
Class c = context.getClassLoader().loadClass(className);
if (parent.isAssignableFrom(c)) {
android.util.Log.i("nora", className);
}
} catch (ClassNotFoundException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
// android.util.Log.i("nora", className);
}
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} finally {
try {
dex.close();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}

I share the opinion of Joop Eggen and find his approach a good one. In Android I try to avoid the usual web app features which lead to a long lasting application start. I do not use reflection or package scanning.
But if you want to .... if I understand it correctly you want to have an annotation for a class. Instead of using annotations you could also use marker interfaces (to just have more possibilites).
1) Look at
Annotation: Java custom annotation and dynamic loading
Has an implementation in the question which just answers your question.
Annotation: Scanning Java annotations at runtime
Interface: Find Java classes implementing an interface
Interface: Is something similar to ServiceLoader in Java 1.5?
Interface: How can I get a list of all the implementations of an interface programmatically in Java?
Interface: Since the approach is expensive, maybe the ServiceLoader is a compromise between execution time and comfort, since it loads only the classes given in the services file. On the other hand if only classes with a certain interface are in your package then the ServiceLoader isn't that faster.
2) AndroidAnnotations
I would prefer the way AndroidAnnotations work (maybe an integration in AndroidAnnotations is the preferable way): It automatically adds an extra compilation step that generates source code, using the standard Java Annotation Processing Tool. So instead of runtime scanning you execute code based on the annotations generated during compile time.
I think the Bean/EBean annotation could work for you (only single class): https://github.com/excilys/androidannotations/wiki/Enhance%20custom%20classes
A scan-feature is not available, see this thread
3) Writing your own annotation processor
See APT (Annotation Processing Tool). The idea would be to generate a static function which returns a list of classes which are annotated, so that no class scanning is needed.
A very good ressource is http://javadude.com/articles/annotations/index.html

Take a look at Vogar's ClassPathScanner. It uses it to find test cases on the class path.

EDIT:
I found this issue in the Android issue tracker. It appears that ClassLoader.getResource(String) is 'working as expected', in that it returns null. This is expected because the DalvikVM does not keep the resources around after compiling. There are workarounds listed in the issue, but there may be another way to access the classes you desire.
Use the PackageManager to get a hold of an instance of ApplicationInfo. ApplicationInfo has a public field called sourceDir which is the full path (a String) to the location of the source directory for that application. Create a File from this String, and you should be able to navigate to your package within the source directory. Once there, you can use the method from my original answer to find the classes you are looking for.
String applicationSourceDir =
getPackageManager().getApplicationInfo(androidPackageName, 0).sourceDir;
/EDIT
You should be able to use the ClassLoader.getResource(String) to get a URL to your specific package (the passed in String being the package name you are interested in delimited by path separators rather than periods). With this URL you can then call getFile(), from which you can create a Java File to the package folder. Call packageFile.listFiles() from there, and you have your classes/subpackages.
Be recursive with the subpackages, and with the classes find the Class object using the static Class.forName(String) method.

In your java build process incorporate the class path scanning, generating injection data/code. This could then be ported too to Dalvik. It is even more efficient that dynamic scanning.

Java - Get a list of all Classes loaded in the JVM

I would like to get a list of all the classes belonging to a certain package as well as all of their children. The classes may or may not be already loaded in the JVM.

It's not a programmatic solution but you can run
java -verbose:class ....
and the JVM will dump out what it's loading, and from where.
[Opened /usr/java/j2sdk1.4.1/jre/lib/rt.jar]
[Opened /usr/java/j2sdk1.4.1/jre/lib/sunrsasign.jar]
[Opened /usr/java/j2sdk1.4.1/jre/lib/jsse.jar]
[Opened /usr/java/j2sdk1.4.1/jre/lib/jce.jar]
[Opened /usr/java/j2sdk1.4.1/jre/lib/charsets.jar]
[Loaded java.lang.Object from /usr/java/j2sdk1.4.1/jre/lib/rt.jar]
[Loaded java.io.Serializable from /usr/java/j2sdk1.4.1/jre/lib/rt.jar]
[Loaded java.lang.Comparable from /usr/java/j2sdk1.4.1/jre/lib/rt.jar]
[Loaded java.lang.CharSequence from /usr/java/j2sdk1.4.1/jre/lib/rt.jar]
[Loaded java.lang.String from /usr/java/j2sdk1.4.1/jre/lib/rt.jar]
See here for more details.

using the Reflections library, it's easy as:
Reflections reflections = new Reflections("my.pkg", new SubTypesScanner(false));
That would scan all classes in the url/s that contains my.pkg package.
the false parameter means - don't exclude the Object class, which is excluded by default.
in some scenarios (different containers) you might pass the classLoader as well as a parameter.
So, getting all classes is effectively getting all subtypes of Object, transitively:
Set<String> allClasses =
reflections.getStore().getSubTypesOf(Object.class.getName());
(The ordinary way reflections.getSubTypesOf(Object.class) would cause loading all classes into PermGen and would probably throw OutOfMemoryError. you don't want to do it...)
If you want to get all direct subtypes of Object (or any other type), without getting its transitive subtypes all in once, use this:
Collection<String> directSubtypes =
reflections.getStore().get(SubTypesScanner.class).get(Object.class.getName());

There are multiple answers to this question, partly due to ambiguous question - the title is talking about classes loaded by the JVM, whereas the contents of the question says "may or may not be loaded by the JVM".
Assuming that OP needs classes that are loaded by the JVM by a given classloader, and only those classes - my need as well - there is a solution (elaborated here) that goes like this:
import java.net.URL;
import java.util.Enumeration;
import java.util.Iterator;
import java.util.Vector;
public class CPTest {
private static Iterator list(ClassLoader CL)
throws NoSuchFieldException, SecurityException,
IllegalArgumentException, IllegalAccessException {
Class CL_class = CL.getClass();
while (CL_class != java.lang.ClassLoader.class) {
CL_class = CL_class.getSuperclass();
}
java.lang.reflect.Field ClassLoader_classes_field = CL_class
.getDeclaredField("classes");
ClassLoader_classes_field.setAccessible(true);
Vector classes = (Vector) ClassLoader_classes_field.get(CL);
return classes.iterator();
}
public static void main(String args[]) throws Exception {
ClassLoader myCL = Thread.currentThread().getContextClassLoader();
while (myCL != null) {
System.out.println("ClassLoader: " + myCL);
for (Iterator iter = list(myCL); iter.hasNext();) {
System.out.println("\t" + iter.next());
}
myCL = myCL.getParent();
}
}
}
One of the neat things about it is that you can choose an arbitrary classloader you want to check. It is however likely to break should internals of classloader class change, so it is to be used as one-off diagnostic tool.

I'd also suggest you write a -javagent agent, but use the getAllLoadedClasses method instead of transforming any classes.
To synchronize with your client code (Normal Java code), create a socket and communicate with the agent through it. Then you can trigger a "list all classes" method whenever you need.

An alternative approach to those described above would be to create an external agent using java.lang.instrument to find out what classes are loaded and run your program with the -javaagent switch:
import java.lang.instrument.ClassFileTransformer;
import java.lang.instrument.IllegalClassFormatException;
import java.security.ProtectionDomain;
public class SimpleTransformer implements ClassFileTransformer {
public SimpleTransformer() {
super();
}
public byte[] transform(ClassLoader loader, String className, Class redefiningClass, ProtectionDomain domain, byte[] bytes) throws IllegalClassFormatException {
System.out.println("Loading class: " + className);
return bytes;
}
}
This approach has the added benefit of providing you with information about which ClassLoader loaded a given class.

List of all Classes loaded in the JVM
From Oracle doc you can use -Xlog option that has a possibility to write into file.
java -Xlog:class+load=info:classloaded.txt

Run your code under a JRockit JVM, then use JRCMD <PID> print_class_summary
This will output all loaded classes, one on each line.

One way if you already know the package top level path is to use OpenPojo
final List<PojoClass> pojoClasses = PojoClassFactory.getPojoClassesRecursively("my.package.path", null);
Then you can go over the list and perform any functionality you desire.

This program will prints all the classes with its physical path. use can simply copy this to any JSP if you need to analyse the class loading from any web/application server.
import java.lang.reflect.Field;
import java.util.Vector;
public class TestMain {
public static void main(String[] args) {
Field f;
try {
f = ClassLoader.class.getDeclaredField("classes");
f.setAccessible(true);
ClassLoader classLoader = Thread.currentThread().getContextClassLoader();
Vector<Class> classes = (Vector<Class>) f.get(classLoader);
for(Class cls : classes){
java.net.URL location = cls.getResource('/' + cls.getName().replace('.',
'/') + ".class");
System.out.println("<p>"+location +"<p/>");
}
} catch (Exception e) {
e.printStackTrace();
}
}
}

You might be able to get a list of classes that are loaded through the classloader but this would not include classes you haven't loaded yet but are on your classpath.
To get ALL classes on your classpath you have to do something like your second solution. If you really want classes that are currently "Loaded" (in other words, classes you have already referenced, accessed or instantiated) then you should refine your question to indicate this.

There's another possibility using VM.class_hierarchy, available since JDK 8 (tested on 1.8.0_322).
$ jcmd 44506 VM.class_hierarchy
This will give a result like that :
44506:
java.lang.Object/null
|--com.intellij.psi.impl.source.tree.JavaElementType$$Lambda$1163/0x0000000800cd86d8/0x0000600002f012c0
|--com.intellij.ide.IdeTooltipManager/0x0000600002f2c6e0
|--sun.security.ssl.SSLBasicKeyDerivation$SecretSizeSpec/null
|--com.intellij.openapi.editor.impl.view.EditorSizeManager$$Lambda$2094/0x0000000801774c38/0x0000600002f2c6e0
|--com.intellij.psi.util.CachedValueProfiler$EventPlace/0x0000600002f2c6e0 (intf)
|--io.ktor.utils.io.internal.ReadWriteBufferStateKt/0x0000600002fcd680
|--com.intellij.javascript.nodejs.library.core.codeInsight.NodePredefinedReferenceErrorUpdater/0x0000600002f13660
|--com.intellij.openapi.fileEditor.impl.FileDocumentManagerImpl$MyAsyncFileListener/0x0000600002f2c6e0
|--java.lang.management.ManagementFactory$PlatformMBeanFinder$$Lambda$89/0x000000080013e0b0/null
|--org.intellij.plugins.markdown.ui.preview.MarkdownHtmlPanelProvider$AvailabilityInfo/0x0000600002f0ada0
| |--org.intellij.plugins.markdown.ui.preview.MarkdownHtmlPanelProvider$AvailabilityInfo$2/0x0000600002f0ada0
| |--org.intellij.plugins.markdown.ui.preview.MarkdownHtmlPanelProvider$AvailabilityInfo$1/0x0000600002f0ada0
|--java.lang.invoke.LambdaForm$DMH/0x000000080012a800/null
|--git4idea.status.GitStagingAreaHolder$$Lambda$2907/0x0000000801d2e690/0x0000600002f41cc0
|--com.intellij.lang.javascript.refactoring.extractSuper.JSCustomExtractInterfaceHandler/0x0000600002f13660 (intf)
|--javax.xml.transform.stream.StreamSource/null
...

If you don't want any libraries and need this information given to you at runtime, you can use this for Java 11+. It finds all system modules loaded at runtime, iterates over their entries (path names) and collects class items.
public static List<String> getSystemClasses() {
// Errorables is a util class to ignore exceptions on lambda types, easy enough to implement yourself
return ModuleFinder.ofSystem().findAll().stream()
.map(modRef -> Errorables.silent(modRef::open)) // open reader to each module
.flatMap(modReader -> Errorables.silent(modReader::list)) // list all items in the module
.filter(s -> s.endsWith(".class") && s.indexOf('-') == -1) // retain only classes (except module-info or package-info)
.map(s -> s.substring(0, s.length() - 6)) // cut off '.class' from the path
.collect(Collectors.toList());
}
If you need non-system classes then there are some conditions to consider. In both situations you can use a class reference to something in your project to get the needed context to find other classes.
If you want only the currently loaded classes, you can use reflection to access the Vector<Class<?>> classes in ClassLoader. This will not include classes from libraries and such that have not yet been initialized.
If you want all classes of all libraries you have in a project then you'll want to use reflection to access the AppClassLoader's URLClassPath ucp. This will hold a ArrayList<URL> path containing URL's pointing to every directory/jar/etc holding referenced resources. Navigating those you can use path-walking to collect the names of class entries similar to the code snippet above.

Well, what I did was simply listing all the files in the classpath. It may not be a glorious solution, but it works reliably and gives me everything I want, and more.