I want to use a math-expression parser of java code. In particular I would like to convert a math-expression given as String to an abstract syntax tree consisted of separate nodes.
Is there anyone to recommend me a relevant open source tool?
If no, how do you reckon the possibility to exploit Intellij source code to do this work?
Which classes are responsible for code parsing and analysis?
Are they included in idea.jar? How can I easily infiltrate their functionality (methods etc)?
I am speaking exclusively for Intellij.
Take a look at MVEL library.
If you only want the results of the math-expression you should revise the question and the answer i selected months ago:
Java 1.5: mathematical formula parser
Brieff description: use the java integration with dinamyc languajes like javascript to let them do the work for you
I would not use IntelliJ, as much as I love it.
If you need an AST, look no further than ANTLR. If you can write a grammar for your equations, ANTLR can generate a lexer/parser to create it for you.
Related
This page describes how I can use the code generator in javac to generate code given that I can build an AST (using a separate parser which I wrote). The technique involves editing javac's source code to basically bypass the Java parser, so that one could supply his/her own AST to the code generator. This could work, but I was hoping to do it in a slightly cleaner way. I want to include the code generating part of javac as a library in my project so I can use it to generate code, without bringing with it the rest of javac's source.
Is there a way to do this with javac, or is there perhaps a better library?
Also, feel free to change the question's title. I couldn't think of a better one, but it's a little ambiguous. If you suggest an edit for a better title, I'll accept it.
I think what you might be interested in is a java library like BCEL(ByteCode Engineering Library)
I played around with it back when I took a class on compiler construction, basically, it has a nice wrapper for generating the constant pool, inserting named bytecode instructions into a method and whatnot, then when you are done, you can either load the class at runtime with a custom classloader, or write it out to a file in the normal way.
With BCEL, it should be relatively easy to go from the syntax tree to the java bytecodes, albeit a bit tedious, but you may want to just use BCEL to generate the raw bytecode without building the tree as well in some cases.
Another cool framework is ASM, a bytecode analysis and manipulation framework.
In case you do not want to use a framework, as of now (2014), it is not possible to generate bytecode from a tree using the arbitrary representations of com.sun.source.tree.* as said here.
I'm looking for a sample or possibly even code I can use which scans Java files and tells me key pieces of information about each class, which i can then use much like the NetBeans refactoring and go to source features do.
Instead of reusing the Netbeans sources, you should probably just find a good library.
tells me key pieces of information about each class
Depending on your definition of "key pieces", I would recommend QDox:
http://qdox.codehaus.org/
QDox is a high speed, small footprint
parser for extracting
class/interface/method definitions
from source files complete with
JavaDoc #tags.
If you are looking for reusing Netbeans code which parses Java file, I don't know.
If you are looking for how to parse a Java file, you can try ANTLR. ANTLR is a parser generator. There exists Java grammar which you can use right away. Once, you generate a Java parser, you can use the parser to parse your Java file. You will have to learn how to use ANTLR.
I don't understand very well your question.
If you want to navigate inside source java code with netbeans, push CTRL and with your mouse go over the word you want to explore ; netbeans highligth the word, and if you clic you go to the source.
As the title says, I want to parse some Java source code in Java. I'm pretty sure there are other java libraries that already perform this, but I couldn't find any.
Antlr has a grammar file for Java. See this.
Janino can parse, compile and execute Java code.
You may be looking for something like ANTLR: http://www.antlr.org/
Eclipse exposes the Syntax Tree of it's own Java compiler. You can simply access the elements.
See here.
JavaCC has a Java 1.5 grammar including generics.
the tokens are probably more
fine-grained then what I want
Your first requirement is to get an accurate parse, and you will realistically only get that from an existing parser. What you do with the result is up to you.
We want to design a simple domain specific language for writing test scripts to automatically test a XML-based interface of one of our applications. A sample test would be:
Get an input XML file from network shared folder or subversion repository
Import the XML file using the interface
Check if the import result message was successfull
Export the XML corresponding to the object that was just imported using the interface and check if it correct.
If the domain specific language can be declarative and its statements look as close as my sentences in the sample above as possible, it will be awesome because people won't necessarily have to be programmers to understand/write/maintain the tests. Something like:
newObject = GET FILE "http://svn/repos/template1.xml"
reponseMessage = IMPORT newObject
newObjectID = GET PROPERTY '/object/id/' FROM responseMessage
(..)
But then I'm not sure how to implement a simple parser for that languange in Java. Back in school, 10 years ago, I coded a language parser using Lex and Yacc for the C language. Maybe an approach would be to use some equivalent for Java?
Or, I could give up the idea of having a declarative language and choose an XML-based language instead, which would possibly be easier to create a parser for? What approach would you recommend?
You could try JavaCC or Antlr for creating a parser for your domain specific language. If the editors of that file are not programmers, I would prefer this approach over XML.
Take a look at Xtext - it will take a grammar definition and generate a parser as well as a fully-featured eclipse editor pluging with syntax highlighting and -checking.
ANTLR should suffice
ANTLR, ANother Tool for Language Recognition, is a language tool that provides a framework for constructing recognizers, interpreters, compilers, and translators from grammatical descriptions containing actions in a variety of target languages. ANTLR provides excellent support for tree construction, tree walking, translation, error recovery, and error reporting.
Look at Antlr library. You'll have to use EBNF grammatic to describe your language and then use Antlr to make java classes from your grammatic.
Have a look at how Cucumber defines its test cases:
(source: cukes.info)
http://cukes.info/ - can run in JRuby.
Or, I could give up the idea of having a declarative language and
choose an XML-based language instead,
which would possibly be easier to
create a parser for? What approach
would you recommend?
This could be easily done using XML to describe your test scenarios.
< GETFILE object="newObject" file="http://svn/repos/template1.xml"/ >
Since your example of syntax is quite simple, it should also be possible to simply use StringTokenizer to tokenize and parse these kind of scripts.
If you want to introduce more complex expressions or control structures you probably better choose ANTLR
I realize this thread is 3 years old but still feel prompted to offer my take on it. The questioner asked if Java could be used for a DSL to look as closely as possible like
Get an input XML file from network shared folder or subversion repository
Import the XML file using the interface
Check if the import result message was successfull
Export the XML corresponding to the object that was just imported
using the interface and check if it correct.
The answer is yes it can be done, and has been done for similar needs. Many years ago I built a Java DSL framework that - with simple customization - could allow the following syntax to be used for compilable, runnable code:
file InputFile
message Message
get InputFile from http://<....>
import Message from InputFile
if validate Message export Message
else
begin
! Signal an error
end
In the above, the keywords file, message, get, import, validate and export are all custom keywords, each one requiring two simple classes of less than a page of code to implement their compiler and runtime functions. As each piece of functionality is completed it is dropped into the framework, where it is immediately available to do its job.
Note that this is just one possible form; the exact syntax can be freely chosen by the implementor. The system is effectively a DIY high-level assembly language, using pre-written Java classes to perform all the functional blocks, both for compiling and for the runtime. The framework defines where these bits of functionality have to be placed, and provides the necessary abstract classes and interfaces to be implemented.
The system meets the primary need of clarity, where non-programmers can easily see what's happening. Changes can be made quickly and run immediately as compilation is almost instantaneous.
Complete (open) source code is available on request. There's a generic Java version and also one for Android.
For C#, I have often used CodeSmith and lately the T4 generator which is part of Visual Studio.
I'm looking for something similar for Java, in particular an Eclipse add-in since I do all my Java development using the Eclipse IDE.
I've found that freemarker does a pretty good job for generating any type of code. From the website:
FreeMarker is a "template engine"; a generic tool to generate text output (anything from HTML to autogenerated source code) based on templates. It's a Java package, a class library for Java programmers. It's not an application for end-users in itself, but something that programmers can embed into their products.
It is used by struts2 a lot. The website has a long list of other products that use freemarker.
I have worked with both Velocity and StringTemplate. Velocity is a bit more conventional (think JSP/ASP concepts), while StringTemplate seems a bit cleaner. in a sense described in this Artima interview. Both are pure templating engines, and will require you to write some code around them, if you want to do full-blown code generation from a model.
StringTemplate is used internally by ANTLR, which may be useful to your effort.
As an alternative approach, you could go with Eclipse, using EMF and JET.
You should try Telosys Tools, an Eclipse plugin for code generation working from an existing database with customizable Velocity templates
See: http://www.telosys.org/
Eclipse Marketplace : http://marketplace.eclipse.org/content/telosys-tools
The tutorials are here : https://sites.google.com/site/telosystutorial/
See this other question about CodeSmith : Is there any freeware tool available which is like Codesmith?
I use JavaForger to generate code from templates. It parses existing classes so that you can use that class-data inside your templates. It can both create new classes or insert code into existing classes. You can determine where generated code will be inserted based on a string conversion rule (e.g. myProject/dao/ProductDao.java => myProject/service/ProductService.java).
JavaForger is open source and uses FreeMarker as template engine and JavaParser as parser.
This is an old question but the only thing that comes close (for Java) to do what CodeSmith Generator does is Spring Roo.
The reason is that Java does not have Partial Classes like C# does. Spring Roo gets around this by using AspectJ ITDs.
My answer is to use StringTemplate, but there is a bit more to it than just what tool to use.
Is it the issue to generate java code? Or is it to use java tools? Programmers would be normally very comfortable writing code. Therefore, it would not be a leap to write some java classes and write a walk that would generate code using StringTemplate. I personally think it is a good exercise to create example models, generate your java code from the models. And depending on your use case you could end up writing JSON models by hand and never having to write any java code to produce the java code. Or you could end up writing Java classes that produce equivalent models.
You could use the StringTemplate based STST, which reads JSON. STST is command line based, and I am sure you could hook it to both eclipse and/or Visual Studio.
I personally think about portability, JSON is an extremely simple language. And almost every language has libraries that support it.
I'm not a C# man so I don't know what the equivalents would be, however I've found xdoclet to be very good in the past. I don't think it integrates with eclipse as such but you can run it from an ant script. Does things like generating Hibernate mapping files from annotated Java classes. Useful if that's what you're looking for :)
eclipse has a built-in template system.
look in window -> preferences -> java -> code style -> code templates
You might look at my plugin : http://fast-code.sourceforge.net/. It allows one to select multiple fields and generate code using user specified velocity templates.
Take a look at my project https://github.com/karajdaar/templator and see if that helps.
I wrote a simple web based application for my use.
its available at https://github.com/harish2704/templates
and a demo is available at http://templates-harish2704.rhcloud.com/
Its language independent tool. GUI supports several languages ( highlighting, snippet completion ect )