I'm doing a project which in essence is a way for a user to create macros for very long/repetitive code so they can then put these macros into a their code to increase readability and encourage re-usability.
The project is split up into two areas: an RCP application where the user constructs their macros and saves them to file, and a plugin-in in eclipse where a user can insert a macro.
The RCP application has come along nicely, but now I'm getting to the real bit where I want java to look at a piece of code and at compile time go 'Oh, that piece of code means this'.
For example a user could have created a macro with an identifier macro1 that represents the code System.out.println("Hello World");. They want to use it in their class foo which will look like:
class foo{
//lots of java....
macro1
//lots more java
}
I want the user to run their class, and when Java compiles it replace macro1 with it's value.
I have been looking into this for a while and read up a little bit on JavaCC and looked into Ants Replace Task but I feel I want to go down the road of creating a custom annotation to inform Java that this string below the annotation should be transformed into/replaced by a piece of java code located in a specific text file. Hopefully ending up with something like:
#ExpandMacro
macro1
To me it seems like it could be possible but I don't know if I'm just being too optimistic! :)
Also one of my concerns is how deployable would it be as a plugin? If I create a custom annotation how would I make it available to the user to use?
Any advice/insights would be very helpful!
Edit: I have also looked into Project Lombok which is looks extremely insteresting. But again I'm concerned about deployability because of the amount of setup required for a lombok project.
The best way for doing this is Annotation Processor (something like Lombokg). With JSR 269 Pluggable Annotation Processing API configuration is easy, because jar with processor is autodiscovered.
Look at: http://www.slideshare.net/PolymathicCoder/the-art-of-metaprogramming-in-java page 34
Look at this differently. I think better for you is to use dynamic language (groovy for example). You can write code in Java and run it dynamically with Groovy (without compilation) - because java code is compatible with groovy. So you can simple replace macro1 with what you need and run this code. With groovy you can
write code than looks like java
replace macros with real code
run code without compilation
Links to read:
http://groovy.codehaus.org/Embedding+Groovy
http://groovy.codehaus.org/Differences+from+Java
Related
I am in the process of moving some old ColdFusion source code across to Java. Most of it is in CFScript format and this does a good enough job of converting CFML to CFScript: https://cfscript.me/ (Thanks Pete Freitag)
I wonder if there are utilities to convert CFScript source to Java source code. Note: I am NOT talking about decompiling java class bytecode.
The CFScript syntax is pretty close to Java syntax and I am looking for a way to do some of the heavy lifting to reduce the amount of hand-editing.
Any thoughts? If not, I might have a go at writing something.
Thanks,
Murray
EDIT: More background:
I want to migrate an application completely from CF to Java. But, it is a big app and my idea is to incrementally create "drop in" java classes to replace existing CFCs and write new code as Java classes in the meantime. I can execute a Java class in CF and use CFCs in Java via createDynamicProxy().
At its most simple form I can then change from
someObj = createObject("component","SomeCFC");
msg = someObj.getMessage();
to
someObj = createObject("java","SomeClass");
msg = someObj.getMessage();
I am not looking to "port" it (as we would say in the old days ;-) ), just to run the CFC through a "parser" that did some of the obvious code changes before I then work through and fix the harder stuff and refactor. That way, at least I have the business logic and same methods and I can run it through the tests.
This might be pie-in-the-sky but I imagine I am not the first person who needed to go down this particular migration path.
I wrote a simple program taking some commands from a XML file, executing it, checkings for errors of the commands, reporting back by mail etc. An entry for a command looks e.g. like this:
<command>C:\Program Files\Test\test.exe /xyz</command>
Now I want to introduce some kind of plugins:
<command myPlugin1="abc,1" myPlugin2="def">C:\Program Files\Test\test.exe /xyz</command>
I thought about writing the plugins as a Java class implementing some interface or subclassing some Plugin class, then having a method like
pluginExec(List<String> parameters, String command)
that gets passed the parameters (e.g. abc,1) and the command (C:\Program Files\Test\test.exe /xyz).
Then maybe take all attribute names of the command tag, therefore myPlugin1 and myPlugin2, then search for a class with the same name as the attribute on myapp.plugins namespace and call pluginExec(...).
Any better ideas on this? I guess with my idea I need some kind of reflection, is there a way without it? The application is a private tool, no need to be able to add plugins without recompiling the software. I just don't want to change the code everytime, I just want to plug in my new plugin class.
If your plugin based program you develop will not grow a lot in the next future then your design is fine.
However, if you expect it to grow and you expect to add multiple features to it than I suggest you try Java standards like OSGi.
I'm looking for a way to automatically generate source code for new methods within an existing Java source code file, based on the fields defined within the class.
In essence, I'm looking to execute the following steps:
Read and parse SomeClass.java
Iterate through all fields defined in the source code
Add source code method someMethod()
Save SomeClass.java (Ideally, preserving the formatting of the existing code)
What tools and techniques are best suited to accomplish this?
EDIT
I don't want to generate code at runtime; I want to augment existing Java source code
What you want is a Program Transformation system.
Good ones have parsers for the language you care about, build ASTs representing the program for the parsed code, provide you with access to the AST for analaysis and modification, and can regenerate source text from the AST. Your remark about "scanning the fields" is just a kind of traversal of the AST representing the program. For each interesting analysis result you produce, you want to make a change to the AST, perhaps somewhere else, but nonetheless in the AST.
And after all the chagnes are made, you want to regenerate text with comments (as originally entered, or as you have constructed in your new code).
There are several tools that do this specifically for Java.
Jackpot provides a parser, builds ASTs, and lets you code Java procedures to do what you want with the trees. Upside: easy conceptually. Downside: you write a lot more Java code to climb around/hack at trees than you'd expect. Jackpot only works with Java.
Stratego and TXL parse your code, build ASTs, and let you write "surce-to-source" transformations (using the syntax of the target language, e.g., Java in this case) to express patterns and fixes. Additional good news: you can define any programming language you like, as the target language to be processed, and both of these have Java definitions.
But they are weak on analysis: often you need symbol tables, and data flow analysis, to really make analyses and changes you need. And they insist that everything is a rewrite rule, whether that helps you or not; this is a little like insisting you only need a hammer in toolbox; after all, everything can be treated like a nail, right?
Our DMS Software Reengineering Toolkit allows the definition of an abitrary target language (and has many predefined langauges including Java), includes all the source-to-source transformation capabilities of Stratego, TXL, the procedural capability of Jackpot,
and additionally provides symbol tables, control and data flow analysis information. The compiler guys taught us these things were necessary to build strong compilers (= "analysis + optimizations + refinement") and it is true of code generation systems too, for exactly the same reasons. Using this approach you can generate code and optimize it to the extent you have the knowledge to do so. One example, similar to your serialization ideas, is to generate fast XML readers and writers for specified XML DTDs; we've done that with DMS for Java and COBOL.
DMS has been used to read/modify/write many kinds of source files. A nice example that will make the ideas clear can be found in this technical paper, which shows how to modify code to insert instrumentation probes: Branch Coverage Made Easy.
A simpler, but more complete example of defining an arbitrary lanauges and transformations to apply to it can be found at How to transform Algebra using the same ideas.
Have a look at Java Emitter Templates. They allow you to create java source files by using a mark up language. It is similar to how you can use a scripting language to spit out HTML except you spit out compilable source code. The syntax for JET is very similar to JSP and so isn't too tricky to pick up. However this may be an overkill for what you're trying to accomplish. Here are some resources if you decide to go down that path:
http://www.eclipse.org/articles/Article-JET/jet_tutorial1.html
http://www.ibm.com/developerworks/library/os-ecemf2
http://www.vogella.de/articles/EclipseJET/article.html
Modifying the same java source file with auto-generated code is maintenance nightmare. Consider generating a new class that extends you current class and adds the desired method. Use reflection to read from user-defined class and create velocity templates for the auto-generating classes. Then for each user-defined class generate its extending class. Integrate the code generation phase in your build lifecycle.
Or you may use 'bytecode enhancement' techniques to enhance the classes without having to modify the source code.
Updates:
mixing auto-generated code always pose a risk of someone modifying it in future to just to tweak a small behavior. It's just the matter of next build, when this changes will be lost.
you will have to solely rely on the comments on top of auto-generated source to prevent developers from doing so.
version-controlling - Lets say you update the template of someMethod(), now all of your source file's version will be updated, even if the source updates is auto-generated. you will see redundant history.
You can use cglib to generate code at runtime.
Iterating through the fields and defining someMethod is a pretty vague problem statement, so it's hard to give you a very useful answer, but Eclipse's refactoring support provides some excellent tools. It'll give you constructors which initialize a selected set of the defined members, and it'll also define a toString method for you.
I don't know what other someMethod()'s you'd want to consider, but there's a start for you.
I'd be very wary of injecting generated code into files containing hand-written code. Hand-written code should be checked into revision control, but generated code should not be; the code generation should be done as part of the build process. You'd have to structure your build process so that for each file you make a temporary copy, inject the generated source code into it, and compile the result, without touching the original source file that the developers work on.
Antlr is really a great tool that can be used very easily for transforming Java source code to Java source code.
I've found LINQPad to be extremely useful when answering StackOverflow questions for C# or VB.NET. It allows me to write up some quick code, run it, and (if I want) see a nicely-formatted dump of the results. That way I can be sure that the code I post actually runs. Thus far I haven't seen anything that I can use to achieve the same result with Java. Is there anything like that out there?
I am not looking for something to query data sources; I just want a light-weight IDE. These are the features I'm particularly interested in:
The ability to write and run short snippets of code without establishing a whole project or file structure.
Reporting of compiler and runtime errors in the code when it is run.
The ability to add references to a particular editor instance.
Syntax highlighting and Autocomplete/Intellisense would be a plus.
JPad - A java scratchpad for running snippets
Since I also couldn't find one I've decided to write one. Currently it can:
Run java snippets (no class / imports / public blah... needed).
Contains drivers for MS/MySQL/Postgres.
Output results as HTML tables
It's very rough but I will add to it over time. Feedback is definitely welcome.
This may help : http://www.browxy.com:9000/codeRunner
EDIT: Url seems to have changed to http://www.browxy.com
You can use the Groovy web console ; it's possible to speak java in groovy land.
Java Snippet Runner:
Does something similar to Linqpad (jar file, not just for macs)
http://mac.softpedia.com/get/Development/Java/Java-Snippet-Runner.shtml
Code Runner (Commercial):
for Mac's only, it'll run code snippets in Java, and lots of other languages too (e.g. Objective C)
http://krillapps.com/coderunner/
http://ideone.com is an online service that has the features you want.
I've been using JEdit for a long time, which is a very powerful cross-platform editor, NOT an IDE. It does have plugins to execute Java code right in the editor, and even uses BSH for macros.
I was looking for a "Java LinqPad" also, and i came across :
this
I've been using IntelliJ IDEA and it works really well as a Groovy scratchpad. The Community Edition is free too.
You need to create a new project, but then can add Groovy scripts to it and run them on the fly. Not had any luck with the actual Scratch File functionality though.
Being a Jetbrains editor it's pretty slick too. (Unlike some of the other options)
Nothing beats LinqPad though.
We want to design a simple domain specific language for writing test scripts to automatically test a XML-based interface of one of our applications. A sample test would be:
Get an input XML file from network shared folder or subversion repository
Import the XML file using the interface
Check if the import result message was successfull
Export the XML corresponding to the object that was just imported using the interface and check if it correct.
If the domain specific language can be declarative and its statements look as close as my sentences in the sample above as possible, it will be awesome because people won't necessarily have to be programmers to understand/write/maintain the tests. Something like:
newObject = GET FILE "http://svn/repos/template1.xml"
reponseMessage = IMPORT newObject
newObjectID = GET PROPERTY '/object/id/' FROM responseMessage
(..)
But then I'm not sure how to implement a simple parser for that languange in Java. Back in school, 10 years ago, I coded a language parser using Lex and Yacc for the C language. Maybe an approach would be to use some equivalent for Java?
Or, I could give up the idea of having a declarative language and choose an XML-based language instead, which would possibly be easier to create a parser for? What approach would you recommend?
You could try JavaCC or Antlr for creating a parser for your domain specific language. If the editors of that file are not programmers, I would prefer this approach over XML.
Take a look at Xtext - it will take a grammar definition and generate a parser as well as a fully-featured eclipse editor pluging with syntax highlighting and -checking.
ANTLR should suffice
ANTLR, ANother Tool for Language Recognition, is a language tool that provides a framework for constructing recognizers, interpreters, compilers, and translators from grammatical descriptions containing actions in a variety of target languages. ANTLR provides excellent support for tree construction, tree walking, translation, error recovery, and error reporting.
Look at Antlr library. You'll have to use EBNF grammatic to describe your language and then use Antlr to make java classes from your grammatic.
Have a look at how Cucumber defines its test cases:
(source: cukes.info)
http://cukes.info/ - can run in JRuby.
Or, I could give up the idea of having a declarative language and
choose an XML-based language instead,
which would possibly be easier to
create a parser for? What approach
would you recommend?
This could be easily done using XML to describe your test scenarios.
< GETFILE object="newObject" file="http://svn/repos/template1.xml"/ >
Since your example of syntax is quite simple, it should also be possible to simply use StringTokenizer to tokenize and parse these kind of scripts.
If you want to introduce more complex expressions or control structures you probably better choose ANTLR
I realize this thread is 3 years old but still feel prompted to offer my take on it. The questioner asked if Java could be used for a DSL to look as closely as possible like
Get an input XML file from network shared folder or subversion repository
Import the XML file using the interface
Check if the import result message was successfull
Export the XML corresponding to the object that was just imported
using the interface and check if it correct.
The answer is yes it can be done, and has been done for similar needs. Many years ago I built a Java DSL framework that - with simple customization - could allow the following syntax to be used for compilable, runnable code:
file InputFile
message Message
get InputFile from http://<....>
import Message from InputFile
if validate Message export Message
else
begin
! Signal an error
end
In the above, the keywords file, message, get, import, validate and export are all custom keywords, each one requiring two simple classes of less than a page of code to implement their compiler and runtime functions. As each piece of functionality is completed it is dropped into the framework, where it is immediately available to do its job.
Note that this is just one possible form; the exact syntax can be freely chosen by the implementor. The system is effectively a DIY high-level assembly language, using pre-written Java classes to perform all the functional blocks, both for compiling and for the runtime. The framework defines where these bits of functionality have to be placed, and provides the necessary abstract classes and interfaces to be implemented.
The system meets the primary need of clarity, where non-programmers can easily see what's happening. Changes can be made quickly and run immediately as compilation is almost instantaneous.
Complete (open) source code is available on request. There's a generic Java version and also one for Android.