We Keep Coding

Context variable is null in tMysqlInput query

I have a problem with my job when I want to make a query with 2 context variables. I attached photos with my job and my components and when I run the job, it's giving me this error:
Exception in component tMysqlInput_1 (facebook_amazon_us)
java.lang.NullPointerException
at mava.facebook_amazon_us_0_1.facebook_amazon_us.tWaitForFile_1Process(facebook_amazon_us.java:2058)
at mava.facebook_amazon_us_0_1.facebook_amazon_us.tMysqlConnection_1Process(facebook_amazon_us.java:798)
at mava.facebook_amazon_us_0_1.facebook_amazon_us.runJobInTOS(facebook_amazon_us.java:5363)
at mava.facebook_amazon_us_0_1.facebook_amazon_us.main(facebook_amazon_us.java:5085)
What I want to do in this job: I have a csv file with multiple columns. The first one is called Reporting_Starts. I want to get the first registration from that column and put it in the query for a select like:
SELECT * FROM my_table WHERE MONTH(my_table.Reporting_Starts)='"+context.month+"'.
I cannot get why my tJava_4 sees the variables and tMysqlInput don't.
In my tJava_4 I have the following code:
System.out.println(context.month);[My job][1][after running the job][1][tJava_3][1][tJavaRow_1][1][tMysqlInput_1 query][1]
Please let me know if you need any additional information about the job.
Thanks!

With all the iterate links you have, I'm guessing the code isn't executing in the order you expect. Could you please make the following changes:
Remove all the iterate links from tFileList_1
Reorganize your jobs as :
tMysqlConnection_1
|
OnSubjobOk
|
tWaitForFile_1
|
Iterate
|
tFileList_1 -- Iterate -- tJava_3
|
OnSubjobOk
|
tFileInputDelimited_1 -- Main -- tJavaRow_1
|
OnSubjobOk
|
tMysqlInput -- tMap -- tMysqlOutput (delete mode, set a column as delete key)
|
tFileInputDelimited -- tMap -- tMysqlOutput (insert csv)
|
OnSubjobOk
|
tFileCopy
First test with just this part. Then if it works, you can add the rest of your job.

ANTLR 4: Parsing grammar

I want to parse some data from AppleSoft Basic script.
I choose ANTLR and download this grammar: jvmBasic
I'm trying to extract function name without parameters:
return parser.prog().line(0).amprstmt(0).statement().getText();
but it returns PRINT"HELLO" e.g full expression except the line number
Here is string i want to parse:
10 PRINT "Hello!"

I think this question really depends on your ANTLR program implementation but if you are using a treewalker/listener you probably want to be targeting the rule for the specific tokens not the entire "statement" rule which is circular and encompasses many types of statement :
//each line can have one to many amprstmt's
line
: (linenumber ((amprstmt (COLON amprstmt?)*) | (COMMENT | REM)))
;
amprstmt
: (amperoper? statement) //encounters a statement here
| (COMMENT | REM)
;
//statements can be made of 1 to many sub statements
statement
: (CLS | LOAD | SAVE | TRACE | NOTRACE | FLASH | INVERSE | GR | NORMAL | SHLOAD | CLEAR | RUN | STOP | TEXT | HOME | HGR | HGR2)
| prstmt
| printstmt1 //the print rule
//MANY MANY OTHER RULES HERE TOO LONG TO PASTE........
;
//the example rule that occurs when the token's "print" is encountered
printstmt1
: (PRINT | QUESTION) printlist?
;
printlist
: expression (COMMA | SEMICOLON)? printlist*
;
As you can see from the BNF type grammar here the statement rule in this grammar includes the rules for a print statement as well as every other type of statement so it will encompass 10, PRINT and hello and subsequently return the text with the getText() method when any of these are encountered in your case, everything but linenumber which is a rule outside of the statement rule.
If you want to target these specific rules to handle what happens when they are encountered you most likely want to add functionality to each of the methods ANTLR generates for each rule by extending the jvmBasiListener class as shown here
example:
-jvmBasicListener.java
-extended to jvmBasicCustomListener.java
void enterPrintstmt1(jvmBasicParser.Printstmt1Context ctx){
System.out.println(ctx.getText());
}
However if all this is setup and you are just wanting to return a string value etc using the single line you have then trying to access the methods at a lower level by addressing the child nodes of statement may work amprstmt->statement->printstmt1->value :
return parser.prog().line().amprstmt(0).statement().printstmt1().getText();
Just to maybe narrow my answer slightly, the rules specifically that address your input "10 PRINT "HELLO" " would be :
linenumber (contains Number) , statement->printstmt1 and statement->datastmt->datum (contains STRINGLITERAL)
So as shown above the linenumber rule exists on its own and the other 2 rules that defined your text are children of statement, which explains outputting everything except the line number when getting the statement rules text.
Addressing each of these and using getText() rather than an encompassing rule such as statement may give you the result you are looking for.
I will update to address your question since the answer may be slightly longer, the easiest way in my opinion to handle specific rules rather than generating a listener or visitor would be to implement actions within your grammar file rules like this :
printstmt1
: (PRINT | QUESTION) printlist? {System.out.println("Print"); //your java code }
;
This would simply allow you to address each rule and perform whichever java action you would wish to carry out. You can then simply compile your code with something like :
java -jar antlr-4.5.3-complete.jar jvmBasic.g4 -visitor
After this you can simply run your code however you wish, here is an example:
import JVM1.jvmBasicLexer;
import org.antlr.v4.runtime.ANTLRInputStream;
import org.antlr.v4.runtime.CommonTokenStream;
import org.antlr.v4.runtime.tree.ParseTree;
public class Jvm extends jvmBasicBaseVisitor<Object> {
public static void main(String[] args) {
jvmBasicLexer lexer = new jvmBasicLexer(new ANTLRInputStream("10 PRINT \"Hello!\""));
jvmBasicParser parser = new jvmBasicParser(new CommonTokenStream(lexer));
ParseTree tree = parser.prog();
}
}
The output for this example would then be just :
Print
You could also incorporate whatever Java methods you like within the grammar to address each rule encountered and either develop your own classes and methods to handle it or directly print it out a result.
Update
Just to address the latest question now :
parser.line().linenumber().getText() - For line Number, as line is not part of a statement
parser.prog().line(0).amprstmt(0).statement().printstmt1().PR‌​INT().getText() - For PRINT as it is isolated in printstmt1, however does not include CLR in the rule
parser.prog().line(0).amprstmt(0).statement().printstmt1().pr‌intlist().expression().getText() - To get the value "hello" as it is part of an expression contained within the printstmt1 rule.
:) Good luck

Hibernate PersistentMap returning wrong values

I am working on an application running Grails 2.4.5 and Hibernate 3.6.10.
There is a domain object that has a child PersistentMap. This map stores 4
key-value pairs where the value is always a String.
In our dev and test environments everything works fine and then occasionaly
the persistent map starts returning "1" for either the key or the value.
The other values in the parent domain object are fine. The problem has been
resolved when it occurs by updating one of the records for the map directly
in the database. This makes me think its a cacheing issue of some sort,
but I haven't been able to recreate it in a local environment.
The database underneath is MySQL.
The following is not the actual code but is representative of the structure.
class MyDomain {
static belongsTo = [owner: Owner]
static hasMany = [relatedDomains: RelatedDomain]
Set relatedDomains = []
Map flags = [:]
String simpleItem
String anotherItem
static constraints = {
owner(nullable: true)
relatedDomains(nullable: true)
flags(nullable: true)
simpleItem(nullable: true)
anotherItem(nullable: true)
}
}
This results in a couple of tables (ignoring RelatedDomain and Owner):
mydomain table
| id | version |owner_id|simple_item|another_item |
|-------|-----------|--------|-----------|-------------|
| 1 | 1 | 1 | A value |Another value|
mydomain_flags table
|flags| flags_ids | flags_elt |
|-----|-----------|-------------|
| 1 | KEY_ONE | VALUE_ONE |
| 1 | KEY_TWO | VALUE_TWO |
| 1 | KEY_THREE | VALUE_THREE |
When the MyDomain instance is retrieved the flags map will have:
[ "KEY_ONE": "VALUE_ONE", "KEY_TWO": "VALUE_TWO", "KEY_THREE" :"VALUE_THREE"]
Occasionally the map contains:
[ "KEY_ONE": "1", "KEY_TWO": "1", "KEY_THREE" :"1"]<br/>
OR
[ "1": "VALUE_ONE", "1": "VALUE_TWO", "1" :"VALUE_THREE"]
The rest of the data in the MyDomain instance is correct. It is just the flags map that seems to have an issue. The application only reads the information for the mydomain and flags, it never updates the data. It's basically configuration data for the application.
Has anyone else seen behavior like this? I don't know if its related to hibernate (version 3.6.10) or Grails/Gorm or both. I've been unable to reproduce it locally but it has happened in two separate environments.

I tracked it down to an issue with hibernate. The aliases generated for the persistent map resulted in the same alias for the key and the element. This is because the aliases are based on a static counter in the org.hibernate.mapping.Table class (in 3.6.10). The reason it was sporadic was because Grails loads all the domain classes into a HashSet and then iterates over the set binding each one. Since the Set is unordered sometimes the domain class with the persistent map would be the 3rd class mapped resulting in a key alias identical to the element alias.
This problem was fixed in Hibernate version 4.1.7
https://hibernate.atlassian.net/browse/HHH-7545
To get around the problem in Grails, I subclassed the GrailsAnnotationConfiguration class and in the constructor, created and discarded 10 Hibernate Table instances. This incremented the static counter to a safer seed value prior to loading the Grails domain classes.

How can I determine which alternative node was chosen in ANTLR

Suppose I have the following:
variableDeclaration: Identifier COLON Type SEMICOLON;
Type: T_INTEGER | T_CHAR | T_STRING | T_DOUBLE | T_BOOLEAN;
where those T_ names are just defined as "integer", "char" etc.
Now suppose I'm in the exitVariableDeclaration method of a test program called LittleLanguage. I can refer to LittleLanguageLexer.T_INTEGER (etc.) but I can't see how determine which of these types was found through the context.
I had thought it would be context.Type().getSymbol().getType() but that doesn't return the right value. I know that I COULD use context.Type().getText() but I really don't want to be doing string comparisons.
What am I missing?
Thanks

You are loosing information in the lexer by combining the tokens prematurely. Better to combine in a parser rule:
variableDeclaration: Identifier COLON type SEMICOLON;
type: T_INTEGER | T_CHAR | T_STRING | T_DOUBLE | T_BOOLEAN;
Now, type is a TerminalNode whose underlying token instance has a unique type:
variableDeclarationContext ctx = .... ;
TerminalNode typeNode = (TerminalNode) ctx.type().getChild(0);
switch(typeNode.getSymbol().getType()) {
case YourLexer.T_INTEGER:
...

How do you access related entries co-located with data affinity in Hazelcast?

I am trying to use Hazelcast's map-reduce feature to perform an aggregate operation, which needs to access the co-located entries. The co-location is controlled using data-affinity.
Imagine the classic customer/order model as used in the Hazelcast documentation on data-affinity. In my example I want to return a customer summary that has the customer and the sum of all their orders, for example, given this data set:
customer_id | name
------------------
1 | Dave
2 | Kate
order_id | customer_id | value
------------------------------
1 | 1 | 5
2 | 1 | 10
3 | 2 | 12
I want to return:
customer_id | name | value
--------------------------
1 | Dave | 15
2 | Kate | 12
This is simple enough, however the reason for using data-affinity is to be able to perform the summing logic within the respective partition holding the data by simply getting all the orders within that partition and therefore avoiding any cross JVM communication.
And so my question, from within a Mapper or similar, how do you get the co-located entries in another cache?
EDIT:
After #noctarius' answer and comments, here's some code (I've tried to make it as brief as possible) that highlights the point at which I only want the orders from the current partition.
The order key class looks like this:
public class OrderKey implements PartitionAware<CustomerIdentity>
{
...
#Override
public CustomerIdentity getPartitionKey()
{
return this.customerIdentity;
}
...
}
And the Mapper like this:
public class OrderSumMapper implements Mapper<CustomerKey, Customer, CustomerKey, CustomerOrderTotal>, HazelcastInstanceAware
{
...
#Override
public void map(CustomerKey customerKey, Customer customer, Context<CustomerKey, CustomerOrderTotal> context)
{
Predicate ordersForCustomer = new OrdersForCustomerPredicate(customerKey);
int totalValue = 0;
//******************************************************************
//
// Given orders are co-located with the customer, how do you ensure
// this call to get the orders only runs in the current partition?
//
//******************************************************************
for (Order order : hazelcastInstance.getMap("orders").values(ordersForCustomer))
{
totalValue += order.getValue();
}
context.emit(customerKey, new CustomerOrderTotal(customer, total));
}
...
}
The highlighted call hazelcastInstance.getMap("orders").values(ordersForCustomer) would ordinarily hit all nodes in the cluster, but because the data is co-located this is an unncessary overhead.
And so back to me original question, how you I get the orders such that only those in the current partition are returned?

You just inject the current node's HazelcastInstance into your Mapper and retrieve the second data structure to read the data.
See a basic example here:
https://github.com/noctarius/hazelcast-mapreduce-presentation/blob/master/src/main/java/com/hazelcast/examples/tutorials/impl/SalaryMapper.java

I've worked it out, which will hopefully this prove useful to others (and so I'm shamelessly answering and accepting my own question).
After some experimentation it is possible to get to the objects held in another map within the partition from a Mapper running in that partition.
The first thing is to have the Mapper implement NodeAware, which causes Hazelcast to inject a reference to the Node the Mapper is running in.
Once you have the Node, you can write a method like this to access data in other maps within a given partition like this:
private Collection<Order> getCustomerOrders(CustomerKey customerKey)
{
List<Order> orders = new ArrayList<>();
MapService mapService = node.getClusterService().getNodeEngine().getService(MapService.SERVICE_NAME);
RecordStore recordStore = mapService.getRecordStore(node.getPartitionService().getPartitionId(customerKey), "orders");
for (Data key : recordStore.keySet())
{
OrderKey orderKey = mapService.getSerializationService().toObject(key);
if (customerKey.equals(orderKey.getCustomerKey()))
{
orders.add(mapService.getSerializationService().toObject(recordStore.get(key)));
}
}
return orders;
}
There's a bit of a deserialization overhead, but that'd be the case using a Predicate and working this way keeps all the processing performed by the Mapper in the JVM containing the data being mapped therefore avoiding any expensive process/network hops - basically it 'should' be faster and will definately cut down on the network traffic caused by inter-node communication.