We Keep Coding

Generating an AST report from Java source code without actually running it

So I spend the whole day trying to figure out how to work with ANTLR. I have a bunch of Java source code files. My goal is to use ANTLR in order to create an AST for each one of those files. In the end, the AST will be converted to JSON so it will be easier to parse each one of them with a dynamic language like python. The conversion will use the solution in this topic (although if someone have a better solution I will be glad to hear). I have managed to run the basic example in the getting-started page. Now I'm trying to figure out how to create an AST for one of my Java source code files. As it is said in the getting-started page, I can use the Java grammar for that. I have downloaded JavaParser.g4 and JavaLexer.g4. I tried to run:
java -Xmx500M -cp "/p/antlr-4.8-complete.jar:$CLASSPATH" org.antlr.v4.Tool JavaParser.g4
But I get the error:
JavaParser.g4:32:21: cannot find tokens file ./JavaLexer.tokens
I guess I need to create that file somehow but I could not understand how, from the docs. So I tried to find that file in Github and I found it here. I have downloaded it and it looked ok:
java -Xmx500M -cp "/p/antlr-4.8-complete.jar:$CLASSPATH" org.antlr.v4.Tool JavaParser.g4
javac JavaP*.java // Also tried javac Java*.java
java -Xmx500M -cp "/p/antlr-4.8-complete.jar:$CLASSPATH" org.antlr.v4.gui.TestRig JavaParser r -gui
But I get:
Exception in thread "main" java.lang.ClassCastException: class JavaParser
at java.lang.Class.asSubclass(Class.java:3404)
at org.antlr.v4.gui.TestRig.process(TestRig.java:135)
at org.antlr.v4.gui.TestRig.main(TestRig.java:119)
Not sure what I do wrong or whether I'm doing the right thing in order to solve my problem.
All I want to do is to create an AST of the file /tmp/file.java and I'm not even sure where I should include it in the commands.
So my questions are:
How do I create an AST for the file /tmp/file.java?
Is it possible to create an AST file without actually writing Java code? I saw many places where they suggest to write Java code and compile it wtih Maven or some other tool. I prefer not to do it if possible, rather use the command-line option.
It's been a while since the JSON topic (mentioned above) was updated. Have something changed? Is it possible to create a JSON report without actually running Java?
As you can see I'm really confused so I appreciate any help!
EDIT: A small clarification - I don't need the AST image, rather to create a JSON report. If there is no easy way to do it, I'm sure that ANTLR has some raw file with that info that I could parse and convert it into JSON. Although, where is that raw data located?

I hope your classpath is correct. It needs the current directory in addition to the ANTLR tool jar. So something like .;C:\...\antlr.jar. As for using ANTLR on the Java grammar, you can do this:
$ antlr4 JavaLexer.g4 JavaParser.g4
$ javac Java*.java
$ grun Java compilationUnit -tree -gui
-gui will pop a window which may take some time. Remember to send an EOF to the ANTLR runtime after you type something to stdin. (CTLR+Z on Windows, CTLR+D on Linux) Alternatively, you can input a file as an extra argument to grun.
From the two .g4 files, I couldn't tell that the grammar name was Java. I just guessed it out based on the context. compilationUnit is the name of the first grammar rule all java files start with. This can be deduced by opening the JavaParser.g4 and reading the first rule.
Here are the aliases I used on Git Bash.
$ alias
alias antlr4='java -Xmx500M org.antlr.v4.Tool'
alias grun='java -Xmx500M org.antlr.v4.gui.TestRig'

Easy way to migrate from JAXB to Castor?

I have created a library which includes some 200 Java classes generated from an existing XSD with JAXB like this:
xjc -no-header -d schemas -b xsd/binding.xml xsd
Alas, JAXB is not supported on Android, and the general suggestion seems to be to use a different library. Castor seems to be a suitable alternative—especially as it offers conversion of .xsd into Java classes. However, doing so seems to be more complex than in Java, and I have no idea of how much the result differs from xjc output.
My use case is unmarshaling and reading the unmarshaled data (changing data or marshaling are not needed). That is, there is a vast amount of code which relies on the resulting Java class schema, therefore any difference between xjc-generated classes and their Castor counterparts would mean a lot of refactoring.
Is there a simple recipe on how to generate Java classes from .xsd in Castor and get a result that is as close as possible to what xjc produces?

The following approach is as close as I got, though some refactoring is still required:
Place a file named castorbuilder.properties on your classpath, with the following content:
org.exolab.castor.builder.javaclassmapping=type
org.exolab.castor.builder.javaVersion=5.0
# Replace the following lines with your schema(s)
org.exolab.castor.builder.nspackages=\
http://example.com/schema/foo=com.example.schema.foo,\
http://example.org/schema/bar=org.example.schema.bar
org.exolab.castor.builder.primitivetowrapper=true
Then run the following command line:
java -cp "*" org.exolab.castor.builder.SourceGeneratorMain -i schema.xsd -types j2
In the above, replace the classpath with the path to the Castor JAR files and castorbuilder.properties, and replace schema.xsd with the path to your XSD file.
Of course, you can invoke Castor via Ant or Maven—in this case, be sure to use the respective equivalent to the command line options above and make sure the properties file is picked up.
Differences which require refactoring:
Enum types are now in their own types sub-package
Where XML Enum literals are in camelCase, they now become CAMELCASE rather than CAMEL_CASE
Castor generates array properties where JAXB has Collection properties
Some class names get an underscore as a prefix; apparently JAXB drops those while Castor preserves them
Date fields are now Date instances
In one instance a property changed its name from value to content
However—there seems to be a bug in Castor, as some of the generated classes invoke a non-existent constructor. This has stopped me from successfully trying this out; I only got as far as refactoring my existing code to the point of being free from syntax and compiler errors.

Loading the class from wrong jar and getting class cast exception

I am using several different schemas in my project. They are each compiled into a separate jar, each using a separate package, using the xmlbeans ant task. I can only seem to successfully parse xml (using the .Factory.parse(String xml) method) for the schema jar that is first in the classpath, otherwise I get a ClassCastException as described in this bug. If I change the jar ordering, a different schema will be able to parse successfully, and the ClassCastException will be thrown for a different class.
I've done some debugging, and I'm coming to the conclusion that the structure of the schemaorg_apache_xmlbeans.namespace package is probably responsible. Since my schemas do not have namespaces, each of jars I've built share some files that are identically named in identical packages. Specifically, I've seen that each jar has a schemaorg_apache_xmlbeans.namespace._nons.xmlns.xsb file that seems to point to the actual schema for that jar. If the factory uses this file to determine some of the classes it will use to parse the xml it has, this may explain the ClassCastException, as it's only looking at the first file on the classpath and not the correct one for the XML it has.
Whether there is any option to specify the namespaces for the schemas generated (like java namespaces) in the wsdls or xsds or in the ant task "wsdl to java" compilation?

I think the problem is that XMLBeans uses some kind of internal schema cache that mixes them up.
Have you tried giving your schemas (xsds) different namespaces?

This can be resolved by using the XMLOptions argument to the parse method.
Example:
XmlOptions opts = new XmlOptions();
opts.setDocumentType(YourDocument.Factory.newInstance().schemaType());
YourDocument.parse(String xml, opts);
see xmloptions.setDocumentType

Could not locate Clojure resource on classpath

I am trying to access some of my clojure functions in my Java code (I am using Eclipse and CounterClockWise). Firstly, I was not sure how to do it? But then I found some answer here.
I am using clojure.lang.RT package for calling .clj files in my Java code. While naming namespaces, .clj files proper conventions has been followed like ns test.clojure.core is in test.clojure package where-in core is .clj file. Also upon following some information, I have added java-source-path (i.e. java-source-path src/test/java) and source-path (i.e. source-path src/test/clojure) in my project.clj.
But, when I run my Java file following error is given - a) Could not locate Clojure resource on classpath b) Attempting to call unbound fn.
Please if anyone in community could help me (assuming I am absolute beginner) through detailed procedure or tutorial I would be grateful, as only information I could get (for using clojure.lang.RT) is very little.
Looking forward to get a complete answer to end my frustration.

I guess you are using RT.loadResourceScript which is causing the proble.
You can try out something like this:
//Load the namespace
RT.var("clojure.core","eval").invoke(RT.var("clojure.core","read-string").invoke("(use 'test.clojure.core)"));
//Find a function in namespace
IFn fn = (IFn)RT.var("test.clojure.core","hello");
//Call that function
fn.invoke();

Make instance of class in java at runtime

In my program I generate classes dynamically but when I try:
String[] args = {"-d","D:\\path\\build\\classes","-s","D:\\path\\src","http://services.aonaware.com/DictService/DictService.asmx?WSDL"};
WsImport.doMain(args);
URL url = new URL("file:D:/path/build/classes/com/aonaware/services/webservices/");
URLClassLoader urlClassLoader = new URLClassLoader(new URL[]{url});
Class service = Class.forName("com.MyClass",true,urlClassLoader );
I recieve java.lang.ClassNotFoundException
If I run one more time the program (in Eclipse), then it is working. Actually I only have to refresh the project in Eclipse and then its working
Does anybody see the problem

Sounds like a classpath issue. Make sure that the class in question (or the jar containing it) is compiled and is in the classpath.
What exactly do you mean by "generating classes dynamically"? If you generate the java source file for a class, it needs to be compiled first into a class file, before the classloader can pick it up. Maybe Eclipse does that in the second round, that's why it is working then.

You would generally use a ClassLoader like URLClassLoader to load classes dynamically at runtime.

Use the three argument form of Class.forName() which requires you specify the ClassLoader.
[Java API Link][1]
[1]: http://java.sun.com/javase/6/docs/api/java/lang/Class.html#forName(java.lang.String, boolean, java.lang.ClassLoader)

Read the docs on URLClassLoader: Any URL that ends with a '/' is assumed to refer to a directory.
Also, you probably should use '/' as a separator instead of \\.
Addendum:
Well, your code works perfectly for me -- if there are actual compiled Java classes in the directory specified as an URL. But when you say
In my program I generate classes
dynamically
do you generate Java source or bytecode directly? If it's source code, do you also compile it from your app?