Visitor file issue in ANTLR4 parser - java

What to write within Visitor class?
We already made a grammar for our language. We don't need to perform any operations on it. If language is passed through written grammar, then we just want to take some of the objects from them.
given language as input:
Dec 17 14:00:00 103.56.229.11 firewall,info FFFW forward: in:<pppoe-mm.demo.649> out:sfp-sfpplus1.vlan113, proto TCP (ACK,PSH), 10.0.15.245:49831->103.235.46.39:443, NAT (10.0.15.245:49831->202.173.127.253:49831)->103.235.46.39:443, len 250
desired output:
Dec, 17, 14:00:00, 103.56.229.11, pppoe-mm.demo.649, TCP, 10.0.15.245:49831, 103.235.46.39:443, 202.173.127.253:49831
Our grammar (File name: sys.g): ( which is working well, We attested it using ANTLRWorks2 )
/*
* To change this license header, choose License Headers in Project Properties.
* To change this template file, choose Tools | Templates
* and open the template in the editor.
*/
grammar sys;
r: IDENT NUM time ip x+ user xout proto xuser ipfull xtra ipfull xtra1 ipfull xtra ipfull xtra2 ipfull xtra3;
time: NUM SEP NUM SEP NUM;
ip: NUM USER NUM USER NUM USER NUM ;
ipfull: NUM USER NUM USER NUM USER NUM SEP NUM ;
x: (IDENT | SEP | NUM)+ LTHAN;
user: (IDENT | USER | NUM)+ ;
xuser: (IDENT | SEP | NUM)+ ;
xout: GTHAN IDENT+ SEP IDENT+ USER IDENT+ USER IDENT SEP IDENT;
proto: IDENT ;
xtra: USER GTHAN ;
xtra1: SEP IDENT SEP;
xtra2: SEP xtra;
xtra3: SEP IDENT NUM;
IDENT: ('a'..'z' | 'A'..'Z')('a'..'z' | 'A'..'Z' | '0'..'9')* ;
NUM: ('0'..'9')+ ;
LTHAN: '<' ;
GTHAN: '>' ;
SEP: ':' | ',' | '(' | ')' ;
USER: '-' | '.' ;
WS : (' ' | '\t' | '\r' | '\n')+ -> skip ;
Generated tree for the given language:
Generated tree for the language
Question 1:
We compiled our grammar file using antlr4.5 and we also used visitor. So our problem is how to print specific objects in another file?
Question 2:
Is it required to make another class named "value" which returns the value to the visitor?
EvalVisitor.java file:
public class EvalVisitor extends sysBaseVisitor{
//
}
Our main java file i.e. SysLogCheck.java, in which we are using Lexer (SysLexer.java) and Parser(SysParser.java) generated by our grammar sys.g file.
import org.antlr.v4.runtime.ANTLRFileStream;
import org.antlr.v4.runtime.CommonTokenStream;
import org.antlr.v4.runtime.tree.ParseTree;
import java.io.*;
import org.antlr.v4.runtime.*;
public class SysLogCheck {
public static void main(String[] args) throws Exception {
ANTLRInputStream input = new ANTLRInputStream(new FileInputStream(new File("input.txt")));
sysLexer lexer = new sysLexer(input);
CommonTokenStream tokens = new CommonTokenStream(lexer);
sysParser parser = new sysParser(tokens);
ParseTree tree = parser.r();
EvalVisitor visitor = new EvalVisitor();
visitor.visit(tree);
}
}

As for your first question:
Here is an example of a crude visitor, which outputs Dec, 17, 14:00:00:
In the line which reads /* do something with the results */ you can place some code which saves the results.
import org.antlr.v4.runtime.tree.ParseTree;
public class EvalVisitor extends sysBaseVisitor{
class LogEntry {
String ident1;
String dayNum;
String ip;
/*
...
*/
}
static LogEntry logEntry;
#Override
public Object visit(ParseTree tree) {
/* Setup logentry used by all visitors (this case, there is only a single visitor...)*/
logEntry = new LogEntry();
/* visit */
final Object o = super.visit(tree);
/* do something with the results */
System.out.println(logEntry.ident1 + ", " + logEntry.dayNum + ", " + logEntry.ip);
return o;
}
StringBuilder stringBuilder;
#Override
public Object visitR(sysParser.RContext ctx) {
logEntry.ident1 = ctx.IDENT().getText();
logEntry.dayNum = ctx.NUM().getText();
return super.visitR(ctx);
}
#Override
public Object visitTime(sysParser.TimeContext ctx) {
logEntry.ip = ctx.getText();
return super.visitTime(ctx);
}
#Override
public Object visitIp(sysParser.IpContext ctx) {
return super.visitIp(ctx);
}
#Override
public Object visitIpfull(sysParser.IpfullContext ctx) {
return super.visitIpfull(ctx);
}
#Override
public Object visitX(sysParser.XContext ctx) {
return super.visitX(ctx);
}
#Override
public Object visitUser(sysParser.UserContext ctx) {
return super.visitUser(ctx);
}
#Override
public Object visitXuser(sysParser.XuserContext ctx) {
return super.visitXuser(ctx);
}
#Override
public Object visitXout(sysParser.XoutContext ctx) {
return super.visitXout(ctx);
}
#Override
public Object visitProto(sysParser.ProtoContext ctx) {
return super.visitProto(ctx);
}
#Override
public Object visitXtra(sysParser.XtraContext ctx) {
return super.visitXtra(ctx);
}
#Override
public Object visitXtra1(sysParser.Xtra1Context ctx) {
return super.visitXtra1(ctx);
}
#Override
public Object visitXtra2(sysParser.Xtra2Context ctx) {
return super.visitXtra2(ctx);
}
#Override
public Object visitXtra3(sysParser.Xtra3Context ctx) {
return super.visitXtra3(ctx);
}
//
}

Related

Picocli required options selection based on a primary option

I would like to parse options with picocli in the following format:
application -mode CLIENT -c aaaa -d bbbb
application -mode SERVER -e xxxx -f yyyy
mode is an enum with values { CLIENT, SERVER }
If mode == CLIENT, -c and -d options are mandatory, and -e, -f must not be used.
If mode == SERVER, -e and -f options are mandatory, and -c, -d must not be used.
In other words, I would like to choose the required options based on a key option. Is this possible in picocli?
Yes, this is possible. One way is simple programmatic validation:
import picocli.CommandLine;
import picocli.CommandLine.Command;
import picocli.CommandLine.Model.CommandSpec;
import picocli.CommandLine.Option;
import picocli.CommandLine.ParameterException;
import picocli.CommandLine.Spec;
import java.util.Objects;
import java.util.function.Predicate;
#Command(name = "application", mixinStandardHelpOptions = true)
public class MyApp implements Runnable {
enum Mode {CLIENT, SERVER}
#Option(names = "-mode", required = true)
Mode mode;
#Option(names = "-c") String c;
#Option(names = "-d") String d;
#Option(names = "-e") String e;
#Option(names = "-f") String f;
#Spec CommandSpec spec;
public static void main(String[] args) {
System.exit(new CommandLine(new MyApp()).execute(args));
}
#Override
public void run() {
validateInput();
// business logic here...
}
private void validateInput() {
String INVALID = "Error: option(s) %s cannot be used in %s mode";
String REQUIRED = "Error: option(s) %s are required in %s mode";
if (mode == Mode.CLIENT) {
check(INVALID, "CLIENT", Objects::isNull, e, "-e", f, "-f");
check(REQUIRED, "CLIENT", Objects::nonNull, c, "-c", d, "-d");
} else if (mode == Mode.SERVER) {
check(INVALID, "SERVER", Objects::isNull, c, "-c", d, "-d");
check(REQUIRED, "SERVER", Objects::nonNull, e, "-e", f, "-f");
}
}
private void check(String msg, String param, Predicate<String> validator, String... valuesAndLabels) {
String desc = "";
String sep = "";
for (int i = 0; i < valuesAndLabels.length; i += 2) {
if (validator.test(valuesAndLabels[i])) {
desc = sep + valuesAndLabels[i + 1];
sep = ", ";
}
}
if (desc.length() > 0) {
throw new ParameterException(spec.commandLine(), String.format(msg, desc, param));
}
}
}
Alternatively, if you are willing to change your requirements a little bit, we can use picocli's argument groups for a more declarative approach:
import picocli.CommandLine;
import picocli.CommandLine.ArgGroup;
import picocli.CommandLine.Command;
import picocli.CommandLine.Option;
#Command(name = "application", mixinStandardHelpOptions = true)
public class MyApp2 implements Runnable {
static class ClientArgs {
#Option(names = "-clientMode", required = true) boolean clientMode;
#Option(names = "-c", required = true) String c;
#Option(names = "-d", required = true) String d;
}
static class ServerArgs {
#Option(names = "-serverMode", required = true) boolean serverMode;
#Option(names = "-e", required = true) String e;
#Option(names = "-f", required = true) String f;
}
static class Args {
#ArgGroup(exclusive = false, multiplicity = "1", heading = "CLIENT mode args%n")
ClientArgs clientArgs;
#ArgGroup(exclusive = false, multiplicity = "1", heading = "SERVER mode args%n")
ServerArgs serverArgs;
}
#ArgGroup(exclusive = true, multiplicity = "1")
Args args;
public static void main(String[] args) {
System.exit(new CommandLine(new MyApp2()).execute(args));
}
#Override
public void run() {
// business logic here...
}
}
When invoked with just -serverMode, this second example will show this error message, followed by the usage help message:
Error: Missing required argument(s): -e=<e>, -f=<f>
Usage: application ((-clientMode -c=<c> -d=<d>) | (-serverMode -e=<e> -f=<f>))
...
Note that this declarative approach cannot be achieved with a single -mode option: each argument group needs its own option (I chose -clientMode and -serverMode in this example). This allows the picocli parser to figure out which options must occur together and which are mutually exclusive.

How to have same slf4j log with JDK8 and JDK11 (with String.format inside)?

How to have same slf4j log with JDK8 and JDK11?
My java Slf4j logger:
log.info("---> {} {}", "When", String.format(matcher.group(1).replaceAll("\\{\\S+\\}", "{%s}").replace("(\\?)", ""), invocation.getArguments()));
My trace in java 8 by JDK8:
---> When I update text {bakery.DemoPage-input_text_field} with {Jenkins T5}
My trace in java 8 by JDK11:
---> When "I update text {bakery.DemoPage-input_text_field} with {Jenkins T5}"
EDIT:
I try this but same result:
String message = MessageFormat.format("---> {0} {1}",
stepAnnotation.annotationType().getSimpleName(),
String.format(matcher.group(1).replaceAll("\\{\\S+\\}", "{%s}").replace("(\\?)", ""), invocation.getArguments())
);
log.info(message);
EDIT (if you want a more simple case):
log.info("---> {} {}", "When", String.format("I update text {%s} with {%s}", "bakery.DemoPage-input_text_field", "Jenkins T5"));
EDIT with #M. Deinum proposal but do not work
log.info("---> {} " + matcher.group(1).replaceAll("\\{\\S+\\}", "{}").replace("(\\?)", ""), stepAnnotation.annotationType().getSimpleName(), invocation.getArguments());
---> When "I update text [bakery.DemoPage-input_text_field, Jenkins T5, []] with {}"
EDIT: I try other proposal with external replace:
String mes = String.format(matcher.group(1).replaceAll("\\{\\S+\\}", "{%s}").replace("(\\?)", ""), invocation.getArguments());
log.info("---> {} {}", stepAnnotation.annotationType().getSimpleName(), mes);
---> When "I update text {bakery.DemoPage-input_text_field} with {Jenkins T5}"
the problem is not come from Slf4j but from stepAnnotation.toString() different in JDK8 and JDK11)
openjdk11 and oraclejdk11 do not respect javadoc:
/**
* Returns a string representation of this annotation. The details
* of the representation are implementation-dependent, but the following
* may be regarded as typical:
* <pre>
* #com.acme.util.Name(first=Alfred, middle=E., last=Neuman)
* </pre>
*
* #return a string representation of this annotation
*/
String toString();
Solution:
import java.lang.annotation.Annotation;
import java.lang.reflect.InvocationTargetException;
import java.lang.reflect.Method;
import java.lang.reflect.Modifier;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import io.cucumber.java.en.When;
public class Sof {
private static final Logger log = LoggerFactory.getLogger(Sof.class);
#When(value = "I update text {string} with {string}(\\?)")
public static void main(String[] args) {
Object as[] = { "a", "b" };
Class c = Sof.class;
Method[] methods = c.getMethods();
Method method = null;
for (Method m : methods) {
if (m.getName().equals("main")) {
method = m;
}
}
Annotation stepAnnotation = method.getAnnotation(When.class);
Class<? extends Annotation> annotationClass = stepAnnotation.annotationType();
try {
Method valueMethods = annotationClass.getDeclaredMethod("value");
if (Modifier.isPublic(valueMethods.getModifiers())) {
log.info("---> {} " + String.format(valueMethods.invoke(stepAnnotation).toString().replaceAll("\\{\\S+\\}", "{%s}").replace("(\\?)", ""), as),
stepAnnotation.annotationType().getSimpleName());
}
} catch (NoSuchMethodException | SecurityException | IllegalAccessException | IllegalArgumentException | InvocationTargetException e1) {
e1.printStackTrace();
}
}
}

RxJava: combine two optional observables

I have two Observables, let's call them PeanutButter and Jelly. I'd like to combine them to a Sandwich Observable. I can do that using:
Observable<PeanutButter> peanutButterObservable = ...;
Observable<Jelly> jellyObservable = ...;
Observable<Sandwich> sandwichObservable = Observable.combineLatest(
peanutButterObservable,
jellyObservable,
(pb, j) -> makeSandwich(pb, j))
The problem is that RX waits for the first PeanutButter and the first Jelly to be emitted before emitting the first combined Sandwich but Jelly may never be emitted which means I never get the first Sandwich.
I'd like to combine the two feeds such that a combined item is emitted as soon as the first item from either feed is emitted, regardless of whether the other feed has yet to emit anything, how do I do that in RxJava?
one possible approach would be to use the startWith operator to trigger an emission of a known value from each stream upon subscription. this way combineLatest() will trigger if either stream emits a value. you'd just have to be mindful of looking out for the initial/signal values in the onNext consumer.
something like this...:
#Test
public void sandwiches() {
final Observable<String> peanutButters = Observable.just("chunky", "smooth")
.startWith("--initial--");
final Observable<String> jellies = Observable.just("strawberry", "blackberry", "raspberry")
.startWith("--initial--");
Observable.combineLatest(peanutButters, jellies, (peanutButter, jelly) -> {
return new Pair<>(peanutButter, jelly);
})
.subscribe(
next -> {
final String peanutButter = next.getFirst();
final String jelly = next.getSecond();
if(peanutButter.equals("--initial--") && jelly.equals("--initial--")) {
// initial emissions
} else if(peanutButter.equals("--initial--")) {
// jelly emission
} else if(jelly.equals("--initial--")) {
// peanut butter emission
} else {
// peanut butter + jelly emissions
}
},
error -> {
System.err.println("## onError(" + error.getMessage() + ")");
},
() -> {
System.out.println("## onComplete()");
}
);
}
I think this problem can be approached by using merge and scan operators:
public class RxJavaUnitTestJava {
public Observable<Sandwich> getSandwich(Observable<Jelly> jelly, Observable<PeanutButter> peanutButter) {
return Observable.merge(jelly, peanutButter)
.scan(new Sandwich(null, null), (BiFunction<Object, Object, Object>) (prevResult, newItem) -> {
Sandwich prevSandwich = (Sandwich) prevResult;
if (newItem instanceof Jelly) {
System.out.println("emitted: " + ((Jelly) newItem).tag);
return new Sandwich((Jelly) newItem, prevSandwich.peanutButter);
} else {
System.out.println("emitted: " + ((PeanutButter) newItem).tag);
return new Sandwich(prevSandwich.jelly, (PeanutButter) newItem);
}
})
.skip(1) // skip emitting scan's default item
.cast(Sandwich.class);
}
#Test
public void testGetSandwich() {
PublishSubject<Jelly> jelly = PublishSubject.create();
PublishSubject<PeanutButter> peanutButter = PublishSubject.create();
getSandwich(jelly, peanutButter).subscribe(new Observer<Sandwich>() {
#Override
public void onSubscribe(Disposable d) {
System.out.println("onSubscribe");
}
#Override
public void onNext(Sandwich sandwich) {
System.out.println("onNext: Sandwich: " + sandwich.toString());
}
#Override
public void onError(Throwable e) {
System.out.println("onError: " + e.toString());
}
#Override
public void onComplete() {
System.out.println("onComplete");
}
});
jelly.onNext(new Jelly("jelly1"));
jelly.onNext(new Jelly("jelly2"));
peanutButter.onNext(new PeanutButter("peanutButter1"));
jelly.onNext(new Jelly("jelly3"));
peanutButter.onNext(new PeanutButter("peanutButter2"));
}
class Jelly {
String tag;
public Jelly(String tag) {
this.tag = tag;
}
}
class PeanutButter {
String tag;
public PeanutButter(String tag) {
this.tag = tag;
}
}
class Sandwich {
Jelly jelly;
PeanutButter peanutButter;
public Sandwich(Jelly jelly, PeanutButter peanutButter) {
this.jelly = jelly;
this.peanutButter = peanutButter;
}
#Override
public String toString() {
String jellyResult = (jelly != null) ? jelly.tag : "no jelly";
String peanutButterResult = (peanutButter != null) ? peanutButter.tag : "no peanutButter";
return jellyResult + " | " + peanutButterResult;
}
}
}
Output:
onSubscribe
emitted: jelly1
onNext: Sandwich: jelly1 | no peanutButter
emitted: jelly2
onNext: Sandwich: jelly2 | no peanutButter
emitted: peanutButter1
onNext: Sandwich: jelly2 | peanutButter1
emitted: jelly3
onNext: Sandwich: jelly3 | peanutButter1
emitted: peanutButter2
onNext: Sandwich: jelly3 | peanutButter2
The fact that Jelly, PeanutButter and Sandwich are all independent types makes it a bit more complex around casting and nullability in scan. If you have control over these types, this solution can be further improved.

Apache Velocity: non-latin names of the custom directives

how to do custom velocity directive with localized name.
Such directives is ignored now.
For example:
class MyDirective extends Directive {
#Override
public String getName() {
return "Пример";
}
#Override
public boolean render(InternalContextAdapter context, Writer writer, Node node)
throws IOException, ResourceNotFoundException, ParseErrorException, MethodInvocationException {
writer.write("example");
return true;
}
}
Input text: #Пример()
Expected example, but got unmodified text #Пример()
It's a velocity parser limitation according to this Identifier definition
<PRE_REFERENCE,REFMODIFIER,REFMOD2>
TOKEN :
{
<#ALPHA_CHAR: ["a"-"z", "A"-"Z", "_"] >
| <#IDENTIFIER_CHAR: [ "a"-"z", "A"-"Z", "0"-"9", "_" ] >
| <IDENTIFIER: ( <ALPHA_CHAR> ) (<IDENTIFIER_CHAR>)* >
{
if (curLexState == PRE_REFERENCE)
{
SwitchTo(REFERENCE);
}
}
}

Grails 2.2.2 UrlMappings testing with exceptions

Is it possible to implement unit test against such url mapping?
"500" (controller: 'error', action: 'handle', exception: MyCustomException)
I've tried to write such unit test
#TestFor(UrlMappings)
#Mock(ErrorController)
class UrlMappingsTest {
void test() {
assertForwardUrlMapping(500, controller: "error", action: "handle", exception: MyCustomException)
}
}
but got junit.framework.AssertionFailedError: url '500' did not match any mappings
If I remove exception: MyCustomException from UrlMapping.groovy and from unit test it works. But I cannot do it.
I know it has been almost 2 years but I too faced this problem yesterday, although using Grails 2.4.4, so I'll post a solution here in case anyone bumps into this.
As if it was not enough that you have to write tests, it is even more disturbing when writing tests takes 1000% more time than writing the actual code! :)
Anyway, the problem is that UrlMappingsUnitTestMixin does not handle this scenario.
There is an official bug reported here:
https://github.com/grails/grails-core/issues/10226 and a fix has been pushed just last month.
If you want to avail from the fix in Grails 2.4.4 you can create a mixin UrlMappingsUnitTestMixinBugFix and then use it in UrlMappingsTest.
UrlMappingsUnitTestMixin.groovy
package com.example.util.test
import grails.util.Holders
import junit.framework.AssertionFailedError
import org.codehaus.groovy.grails.commons.ControllerArtefactHandler
import org.codehaus.groovy.grails.commons.GrailsControllerClass
import org.codehaus.groovy.grails.web.mapping.UrlMappingsHolder
import static junit.framework.Assert.assertEquals
/**
* Methods below are copied from {#link grails.test.mixin.web.UrlMappingsUnitTestMixin}
*
* <p>This method is here because the {#link grails.test.mixin.web.UrlMappingsUnitTestMixin} in Grails 2.4.4 has a bug and this code fixes it.
* <p>Link to the bug: here.
*
* <p>To use this class just write UrlMappingsTest as per Grails guidelines and add a static code block to the Test:
* <p>
* <code>
* static {
* UrlMappingsTest.mixin(UrlMappingsUnitTestMixinBugFix)
* }
* </code>
*/
class UrlMappingsUnitTestMixinBugFix {
void assertForwardUrlMapping(assertions, url) {
assertForwardUrlMapping(assertions, url, null)
}
void assertForwardUrlMapping(assertions, url, paramAssertions) {
def assertionKeys = ["controller", "action", "view"]
final String KEY_EXCEPTION = 'exception'
UrlMappingsHolder mappingsHolder = Holders.applicationContext.getBean("grailsUrlMappingsHolder", UrlMappingsHolder)
if (assertions.action && !assertions.controller) {
throw new AssertionFailedError("Cannot assert action for url mapping without asserting controller")
}
if (assertions.controller) assertController(assertions.controller, url)
if (assertions.action) assertAction(assertions.controller, assertions.action, url)
if (assertions.view) assertView(assertions.controller, assertions.view, url)
def mappingInfos
if (url instanceof Integer) {
mappingInfos = []
// -------- START FIX --------
// -------- OLD CODE (below) --------
// def mapping = mappingsHolder.matchStatusCode(url)
// if (mapping) mappingInfos << mapping
// -------- FIXED CODE (below) --------
def mapping
if (assertions."$KEY_EXCEPTION") {
mapping = mappingsHolder.matchStatusCode(url, assertions."$KEY_EXCEPTION" as Throwable)
} else {
mapping = mappingsHolder.matchStatusCode(url)
}
if (mapping) mappingInfos << mapping
// -------- END FIX --------
} else {
mappingInfos = mappingsHolder.matchAll(url)
}
if (mappingInfos.size() == 0) throw new AssertionFailedError("url '$url' did not match any mappings")
def mappingMatched = mappingInfos.any { mapping ->
mapping.configure(webRequest)
for (key in assertionKeys) {
if (assertions.containsKey(key)) {
def expected = assertions[key]
def actual = mapping."${key}Name"
switch (key) {
case "controller":
if (actual && !getControllerClass(actual)) return false
break
case "view":
if (actual[0] == "/") actual = actual.substring(1)
if (expected[0] == "/") expected = expected.substring(1)
break
case "action":
if (key == "action" && actual == null) {
final controllerClass = getControllerClass(assertions.controller)
actual = controllerClass?.defaultAction
}
break
}
assertEquals("Url mapping $key assertion for '$url' failed", expected, actual)
}
}
if (paramAssertions) {
def params = [:]
paramAssertions.delegate = params
paramAssertions.resolveStrategy = Closure.DELEGATE_ONLY
paramAssertions.call()
params.each { name, value ->
assertEquals("Url mapping '$name' parameter assertion for '$url' failed", value, mapping.params[name])
}
}
return true
}
if (!mappingMatched) throw new IllegalArgumentException("url '$url' did not match any mappings")
}
private GrailsControllerClass getControllerClass(controller) {
return grailsApplication.getArtefactByLogicalPropertyName(ControllerArtefactHandler.TYPE, controller)
}
}
UrlMappingsTest.groovy
import com.example.controller.ErrorController
import com.example.util.test.UrlMappingsUnitTestMixinBugFix
import grails.test.mixin.Mock
import grails.test.mixin.TestFor
#TestFor(UrlMappings)
#Mock(ErrorController)
class UrlMappingsTest {
static {
UrlMappingsTest.mixin(UrlMappingsUnitTestMixinBugFix)
}
void test() {
assertForwardUrlMapping(500, controller: "error", action: "handle", exception: MyCustomException)
}
}

Categories