We Keep Coding

JavaCC creating custom Token class

I'm working on a school assignment for my compiler and interpreters course and our current task is to create a scanner and a set of tokens using JavaCC. I have a pretty solid understanding of how JavaCC works but my problem is finding resources online to help me out when I get stuck. I am working on creating a custom Token class, let's call it NewToken.Java. I know that the base Token class has an image variable and a kind variable but I want to implement my own variable "value". Furthermore I want to figure out how I can assign this value. I want the value variable to hold the literal value of what I scan, for example, my NewToken is being matched to the following
< IDENTIFIER:(< LETTER >)+ ( < LETTER > | < DIGIT >)* >
< #LETTER:["a" - "z"] >
< #DIGIT: ["0" - "9"] >
so something along the lines of Name123Name would get caught and when it does I want to store the string "Name123Name" into the 'value' variable of my NewToken object. I hope this makes sense, I am still new to JavaCC and may be calling things by there wrong name here.
public NewToken(){}
public NewToken(int kind){
this(kind,null);
}
public NewToken(int kind, String image){
this.kind=kind;
this.image=image;
this.value=image;
}
public String toString(){
return image;
}
public static Token newToken(int ofKind, String image){
switch(ofKind){
default : return new Token(ofKind, image);
}
}
public static Token newToken(int ofKind){
return newToken(ofKind, null);
}
}
Above is part of my code for the NewToken class, I have it extending Token and implementing java.io.serializable. I created by using the code generated for Token.java. I also have my variable declarations and my getValue() function which are not listed here to save space. I'm not looking for anyone to do my work for me I just need some guidance on how I would get this working, thank you in advance.

First off, I think the newToken routine should return objects of type NewToken rather than Token.
public static Token newToken(int ofKind, String image){
return new NewToken(ofKind, image);
}
public static Token newToken(int ofKind){
return new NewToken(ofKind, null);
}
(I don’t think you need that second method. But, I’m not completely sure, so I’ll leave it.)
It’s a bit unclear to me how you want value to differ from image, but I’m going to assume that you can compute the desired value for value from the image and the kind. And I’ll further assume that you have implemented this function as a static method.
private static String computeValue(int kind, String image) {...}
Delete the first two constructors and the remaining one should be:
private NewToken(int kind, String image){
this.kind = kind;
this.image = image;
this.value = computeValue( kind, image );
}

The answer that Professor Norvell is giving you is based on using a very old, obsolete version of JavaCC. The way he's suggesting you go about things is probably about the best way of doing it, if you were going to use the legacy JavaCC.
However, the most advanced version of JavaCC is JavaCC 21 and it handles this sort of use case straight out of the box in a very clean, elegant manner. See here for information on this specifically.
As you can see, you can put annotations in your grammar file that cause the various Token subclasses to be generated and used.
Also, JavaCC 21 has code injection that allows you to inject code directly into the any generated files, including the Token subclasses. That feature is not at all present in legacy JavaCC either. But using that, you could just inject your computeValue method right into the appropriate Token subclass.
INJECT NewToken :
{
private static String computeValue(int kind, String image) {...}
}
You put that in your grammar and the computeValue method just gets inserted into the generated NewToken.java file.
By the way, there is an article about JavaCC 21 that appeared recently on dzone.com.

Validating BCP-47 language tag using Java APIs

As far as I know, the only way to validate whether a given BCP-47 language tag is valid is to use the following idiom:
private static boolean isValid(String tag) {
try {
new Locale.Builder().setLanguageTag(tag).build();
return true;
} catch (IllformedLocaleException e) {
return false;
}
}
However, the downside of this approach is that setLanguageTag throws an exception which has noticeable (in a profile) performance overhead in workloads where locales are checked often.
The setLanguageTag function is implemented using sun.util.locale APIs, and as far as I can tell it's the only place where sun.util.locale.ParseStatus is checked.
What I would like to be able to do is to use a method which has the following semantics:
import sun.util.locale.LanguageTag;
import sun.util.locale.ParseStatus;
private static boolean isValid(String tag) {
ParseStatus sts = new ParseStatus();
LanguageTag.parse(tag, sts);
return !sts.isError();
}
However, it's not possible to check the locale in the above way since it's using sun.* classes directly since it requires additional JDK options to export sun.util.locale from the java.base module.
Is there a way to validate a language tag without using private sun.* APIs while being consistent with the implementation of sun.util.locale.LanguageTag#parse?

The simplest solution should be:
boolean isValidBCP47 = "und".equals(Locale.forLanguageTag(tag))
The best solution is to make use of java.util.Locale's filtering to handle this for you.
Often we need to fallback to locales. For example; en-JP is what you want for English Speakers visiting a theme park in Japan. However; when en-JP is not present you likely want to fall back to just en. In addition, your platform likely doesn't support every locale and would require a check against a list of supported locales.
Using com.glide.Locale you can do the following:
ArrayList<Locale.LanguageRange> priorityList = new ArrayList<>();
priorityList.add(new Locale.LanguageRange("en-JP"));
priorityList.add(new Locale.LanguageRange("joking")); // invalid tag
priorityList.add(new Locale.LanguageRange("fr")); // unsupported tag
priorityList.add(new Locale.LanguageRange("en"));
ArrayList<String> supportedTags = new ArrayList<>();
supportedTags.add("ja");
supportedTags.add("en-JP");
supportedTags.add("en");
Locale.filterTags(priorityList, supportedTags, Locale.FilteringMode.AUTOSELECT_FILTERING);
// returns ArrayList ["en-JP", "en"]

JSP like templating for simple text

I've got a program that currently has a mass of code that I would like to design away. This code takes a number of text files and passes it through an interestingly written interpreter to produce a plain text file report that goes on to other systems. In theory this allows a non-programmer to be able to modify the report without having to understand the inner workings of Java and the interpreter. In practice, any minor change likely necessitates going into the interpreter and tweaking it (and the domain specific language isn't exactly friendly even to other programmers).
I would love to redesign this code. As a primarily web programmer the first thing that came to mind when thinking of "non-programmer being able to modify the report ..." I replaced report with web page and said to myself "ah ha! Jsp." This would give me a nice What You See Is Almost What You Get approach for people along with taglibs and java scriptlets (as undesirable as the later may be) rather than awkwardly written DSL statements.
While it is possible to use jspc to compile a jsp into java (another part of the application runs ejbs on a jboss server so jspc isn't too far away), the boilerplate code that it uses tries to hook up the output to the pagecontext from the servletcontext. It would involve tricking the code into thinking it was running inside a web container (not an impossibility, but a kluge) and then removing the headers.
Is there a different templateing approach (or library) for java that could be used to print to a text file? Every one that I've looked at so far appears to either be optimized for web or tightly coupled to a particular application server (and designed for web work).

So you need a slim down version of JSP.
See if this one (JSTP) works for you
http://jstp.sourceforge.net/manual.html

Give Apache Velocity a try. It is incredibly simple and does not assume it is running in the context of a web application.
This is totally subjective, but I would argue it's syntax is easier for a non-programmer to understand than JSP and tag libraries.

If you want to be a real tread setter in your company, you could create a Grails application to do it and use Groovy templating (maybe in combination with the Quartz plugin for scheduling), it might be a bit of a hard sell if there is alot of existing code to be replaced but I love it...
http://groovy.codehaus.org/Groovy+Templates
If you want the safe bet, then (the also excellent) Velocity has to be it:
http://velocity.apache.org/

Probably you want to check Rythm template engine, with good performance (2 to 3 times faster than velocity) and elegant syntax (.net Razor like) and designed specifically to Java programmer.
Template, generate a string of user names separated by "," from a list of users
#args List<User> users
#for (User user: users) {
#user.getName() #user_sep
}
Template: if-else demo
#args User user
#if (user.isAdmin()) {
<div id="admin-panel">...</div>
} else {
<div id="user-panel">...</div>
}
Invoke template using template file
// pass render args by name
Map<String, Object> renderArgs = ...
String s = Rythm.render("/path/to/my/template.txt", renderArgs);
// or pass render arguments by position
String s = Rythm.render("/path/to/my/template.txt", "arg1", 2, true, ...);
Invoke template using inline text
User user = ...;
String s = Rythm.render("#args User user;Hello #user.getName()", user);
Invoke template with String interpolation mode
User user = ...;
String s = Rythm.render("Hello #name", user.getName());
ToString mode
public class Address {
public String unitNo;
public String streetNo;
...
public String toString() {
return Rythm.toString("#_.unitNo #_.streetNo #_.street, #_.suburb, #_.state, #_.postCode", this);
}
}
Auto ToString mode (follow apache commons lang's reflectionToStringBuilder, but faster than it)
public class Address {
public String unitNo;
public String streetNo;
...
public String toString() {
return Rythm.toString(this);
}
}
Document could be found at http://www.playframework.org/modules/rythm. Full demo app running on GAE: http://play-rythm-demo.appspot.com.
Note, the demo and doc are created for play-rythm plugin for Play!Framework, but most of the content also apply to the pure rythm template engine.
Source code:
Rythm template engine: https://github.com/greenlaw110/rythm/
Play Rythm Plugin: https://github.com/greenlaw110/play-rythm

What does the #sign do?

I have seen the at (#) sign in Groovy files and I don't know if it's a Groovy or Java thing. I have tried to search on Google, Bing, and DuckDuckGo for the mystery at sign, but I haven't found anything. Can anyone please give me a resource to know more about what this operator does?

It's a Java annotation. Read more at that link.

As well as being a sign for an annotation, it's the Groovy Field operator
In Groovy, calling object.field calls the getField method (if one exists). If you actually want a direct reference to the field itself, you use #, ie:
class Test {
String name = 'tim'
String getName() {
"Name: $name"
}
}
def t = new Test()
println t.name // prints "Name: tim"
println t.#name // prints "tim"

'#' is an annotations in java/ Groovy look at the demo :Example with code
Java 5 and above supports the use of annotations to include metadata within programs. Groovy 1.1 and above also supports such annotations.
Annotations are used to provide information to tools and libraries.
They allow a declarative style of providing metadata information and allow it to be stored directly in the source code.
Such information would need to otherwise be provided using non-declarative means or using external files.

It can also be used to access attributes when parsing XML using Groovy's XmlSlurper:
def xml = '''<results><result index="1"/></results>'''
def results = new XmlSlurper().parseText(xml)
def index = results.result[0].#index.text() // prints "1"
http://groovy.codehaus.org/Reading+XML+using+Groovy's+XmlSlurper

Get declared methods in order they appear in source code

The situation seems to be abnormal, but I was asked to build serializer that will parse an object into string by concatenating results of "get" methods. The values should appear in the same order as their "get" equivalent is declared in source code file.
So, for example, we have
Class testBean1{
public String getValue1(){
return "value1";
}
public String getValue2(){
return "value2";
}
}
The result should be:
"value1 - value2"
An not
"value2 - value1"
It can't be done with Class object according to the documentation. But I wonder if I can find this information in "*.class" file or is it lost? If such data exists, maybe, someone knows a ready to use tool for that purpose? If such information can't be found, please, suggest the most professional way of achieving the goal. I thought about adding some kind of custom annotations to the getters of the class that should be serialized.

If you want that you have to parse the source code, not the byte code.
There are a number of libraries that parse a source file into a node tree, my favorite is the javaparser (hosted at code.google.com), which, in a slightly modified version, is also used by spring roo.
On the usage page you can find some samples. Basically you will want to use a Visitor that listens for MethodDefinitions.

Although reflection does not anymore (as of java 7 I think) give you the methods in the order in which they appear in the source code, the class file appears to still (as of Java 8) contain the methods in the order in which they appear in the source code.
So, you can parse the class file looking for method names and then sort the methods based on the file offset in which each method was found.
If you want to do it in a less hacky way you can use Javassist, which will give you the line number of each declared method, so you can sort methods by line number.

I don't think the information is retained.
JAXB, for example, has #XmlType(propOrder="field1, field2") where you define the order of the fields when they are serialized to xml. You can implemenet something similar

Edit: This works only on concrete classes (the class to inspect has its own .class file). I changed the code below to reflect this. Until diving deeper into the ClassFileAnalyzer library to work with classes directly instead of reading them from a temporary file this limitation exists.
Following approach works for me:
Download and import following libarary ClassFileAnalyzer
Add the following two static methods (Attention! getClussDump() needs a little modification for writing out the class file to a temporary file: I removed my code here because it's very special at this point):
public static String getClassDump(Class<?> c) throws Exception {
String classFileName = c.getSimpleName() + ".class";
URL resource = c.getResource(classFileName);
if (resource == null) {
throw new RuntimeException("Works only for concreate classes!");
}
String absolutePath = ...; // write to temp file and get absolute path
ClassFile classFile = new ClassFile(absolutePath);
classFile.parse();
Info infos = new Info(classFile, absolutePath);
StringBuffer infoBuffer = infos.getInfos();
return infoBuffer.toString();
}
public static <S extends List<Method>> S sortMethodsBySourceOrder(Class<?> c, S methods) throws Exception {
String classDump = getClassDump(c);
int index = classDump.indexOf("constant_pool_count:");
final String dump = classDump.substring(index);
Collections.sort(methods, new Comparator<Method>() {
public int compare(Method o1, Method o2) {
Integer i1 = Integer.valueOf(dump.indexOf(" " + o1.getName() + lineSeparator));
Integer i2 = Integer.valueOf(dump.indexOf(" " + o2.getName() + lineSeparator));
return i1.compareTo(i2);
}});
return methods;
}
Now you can call the sortMethodsBySourceOrder with any List of methods (because sorting arrays is not very comfortable) and you will get the list back sorted.
It works by looking at the class dumps constant pool which in turn can be determined by the library.
Greetz,
GHad

Write your custom annotation to store ordering data, then use Method.getAnnotation(Class annotationClass)