(I'm jumping from Esper to Siddhi)
I'm trying to deploy several queries in the same stream. The example here https://docs.wso2.com/display/CEP400/Using+Siddhi+as+a+Library shows how to deploy a query in a stream, altogether in one ExecutionPlan. So what if I want to add a query to the same execution plan, o same stream?
I'm interest to do this grammatically, Java code.
UPDATE
I want to add queries in an existing execution plan. This means while streams are arriving, and not before.
Using an example posted as answer:
SiddhiManager siddhiManager = new SiddhiManager();
String executionPlan = "" +
"#Plan:name('demo') " +
"" +
"define stream cseEventStream (symbol string, price float, volume long);" +
"" +
"from cseEventStream[symbol==\"WSO2\"] " +
"insert into wso2Stream;" +
"" +
"from cseEventStream[symbol==\"ABC\"] " +
"insert into abcStream;";
ExecutionPlanRuntime executionPlanRuntime = siddhiManager.createExecutionPlanRuntime(executionPlan);
StreamCallback streamCallback = new StreamCallback() {
#Override
public void receive(Event[] events) {
EventPrinter.print(events);
}
};
executionPlanRuntime.addCallback("wso2Stream", streamCallback);
//Similarly, we can add another call back for abcStream
streamCallback.startProcessing();
InputHandler inputHandler = executionPlanRuntime.getInputHandler("cseEventStream");
executionPlanRuntime.start();
// HERE: ADD new query <---------------------------------------------
inputHandler.send(new Object[]{"WSO2", 700f, 100l});
inputHandler.send(new Object[]{"ABC", 60.5f, 200l});
inputHandler.send(new Object[]{"WSO2", 60.5f, 200l});
streamCallback.stopProcessing();
executionPlanRuntime.shutdown();
Following sample adds multiple Queries, for same Stream (cseEventStream), within the same Execution Plan.
SiddhiManager siddhiManager = new SiddhiManager();
String executionPlan = "" +
"#Plan:name('demo') " +
"" +
"define stream cseEventStream (symbol string, price float, volume long);" +
"" +
"from cseEventStream[symbol==\"WSO2\"] " +
"insert into wso2Stream;" +
"" +
"from cseEventStream[symbol==\"ABC\"] " +
"insert into abcStream;";
ExecutionPlanRuntime executionPlanRuntime = siddhiManager.createExecutionPlanRuntime(executionPlan);
StreamCallback streamCallback = new StreamCallback() {
#Override
public void receive(Event[] events) {
EventPrinter.print(events);
}
};
executionPlanRuntime.addCallback("wso2Stream", streamCallback);
//Similarly, we can add another call back for abcStream
streamCallback.startProcessing();
InputHandler inputHandler = executionPlanRuntime.getInputHandler("cseEventStream");
executionPlanRuntime.start();
inputHandler.send(new Object[]{"WSO2", 700f, 100l});
inputHandler.send(new Object[]{"ABC", 60.5f, 200l});
inputHandler.send(new Object[]{"WSO2", 60.5f, 200l});
streamCallback.stopProcessing();
executionPlanRuntime.shutdown();
I've used Siddhi version 3.0.6-beta2.
This code sample is a modification made to one of the Siddhi Passthrough test cases (PassThroughTest4).
Update:
Looking at ExecutionPlanRuntime class, it does not seem like it is possible to add queries "on the run".
So, to my understanding, you will have to shutdown current execution plan runtime, add those new queries and start it back.
Related
I started Java for plugin codding in minecraft. now I'm trying to add role names as prefix and show it in chat.
ArrayList<String> userRoles = new ArrayList<String>();
if(player.hasPermission("chat.rank.admin")){
userRoles.add("[Admin]");
} if(player.hasPermission("chat.rank.vip")) {
userRoles.add("[VIP]");
}
event.setFormat(userRoles<>(1) + " " + player.getDisplayName() + "§7: " + msg);
// In this line, the expected output is "[Admin] user: msg" or both [Admin] [VIP] user: msg"
// But it gives "([Admin],[VIP]) user: msg"
// I'm sure it has a simple solution but as I said, I'm new here. thanks from now
It seems like you are trying to create a list which only stores one value.
You might want to try creating a function that get the rank name of the player outside of your PlayerChatEvent listener.
Here's a demo code:
public String getPlayerRankName(Player p){
if (p.hasPermission("chat.rank.admin"))
return "[Admin]";
else if (p.hasPermission("chat.rank.vip"))
return "[VIP]";
else
return "";
}
And in your PlayerChatEvent event listener, call this function in your chat line:
event.setFormat(getPlayerRankName(event.getPlayer()) + " " + player.getDisplayName() + "§7: " + msg);
I am a relative newbie to Spark. I need to read from a Mongo collection in Java using Spark, change some field values, let's say I am appending "123" to one field value and write into another collection. Accordingly I had 2 separate Mongo URIs as the input and output URIs configured in Spark. I am then proceeding to read from the input collection. However, what I am not understanding is how would I make the same RDD of documents as output to another collection. This is the input code:
String inputUri = "mongodb://" + kp.getProperty("source.mongo.userid") + ":"
+ Encryptor.decrypt(kp.getProperty("source.mongo.cache")) + "#"
+ kp.getProperty("source.mongo.bootstrap-servers") + "/" + kp.getProperty("source.mongo.database")
+ "." + kp.getProperty("source.mongo.inputCollection") + "?ssl=true&connectTimeoutMS="
+ kp.getProperty("source.mongo.connectTimeoutMS") + "&socketTimeoutMS="
+ kp.getProperty("source.mongo.socketTimeoutMS") + "&maxIdleTimeMS="
+ kp.getProperty("source.mongo.maxIdleTimeMS");
String outputUri = "mongodb://" + kp.getProperty("source.mongo.userid") + ":"
+ Encryptor.decrypt(kp.getProperty("source.mongo.cache")) + "#"
+ kp.getProperty("source.mongo.bootstrap-servers") + "/" + kp.getProperty("source.mongo.database")
+ "." + kp.getProperty("source.mongo.outputCollection") + "?ssl=true&connectTimeoutMS="
+ kp.getProperty("source.mongo.connectTimeoutMS") + "&socketTimeoutMS="
+ kp.getProperty("source.mongo.socketTimeoutMS") + "&maxIdleTimeMS="
+ kp.getProperty("source.mongo.maxIdleTimeMS");
SparkSession spark = SparkSession.builder().master("local[3]").appName(kp.getProperty("spark.app.name"))
.config("spark.mongodb.input.uri", inputUri)
.config("spark.mongodb.output.uri", outputUri)
...;
JavaSparkContext sc = new JavaSparkContext(spark.sparkContext());
JavaMongoRDD<Document> rdd = MongoSpark.load(sc);
System.out.println("Count: " + rdd.count());
System.out.println(rdd.first().toJson());
Please help me in this regard.
I have got the answer myself. I went the Dataset route instead of RDDs which made the modification simpler. So, to load the Mongo colection, I use
Dataset<Row> df = MongoSpark.load(sc).toDF();
Then I create a temporary view upon it in orcder to be able to use Spark SQL:
df.createOrReplaceTempView("Customer");
I register an UDF for operating upon each column value:
spark.udf().register("Test", new TestUDF(), DataTypes.StringType);
the UDF definition is as follows:
public class TestUDF implements UDF1<String, String> {
#Override
public String call(String customer) throws Exception {
return customer + "123";
}
}
Then I call the UDF using the same column name as the original so that the values in the original dataset are replaced:
df = df.withColumn("CustomerName", functions.callUDF("Test", functions.col("CustomerName")));
Then I write it back to Mongo in a separate collection:
MongoSpark.write(df).option("collection", "myCollection").save();
I have a Java file that query data and I'm trying to add a filtering function. By using criteria builder and criteria query, I manage to get data that I want to filter.
As starter, this is the column to display my data:
Name Host
Fikrie ubuntu
Fikrie2 unix
Fikrie3 ulalala
Fikrie4 ugagaga
There are 3 variable used here. The Name column, is displaying data from name. For the Host column, it is a bit tricky. It will display the hostname, but if there is a logAsHost display, this data will overwrite the hostname.
So this is how my data really look like:
Name Host
Fikrie ubuntu <-- hostname = 1, logAsHost = ubuntu
Fikrie2 unix <-- hostname = 123, logAsHost = unix
Fikrie3 ulala <-- hostname = ulala, logAsHost = no value
Fikrie4 ugaga <-- hostname = ugaga, logAsHost = no value
When I try to filter just 1 variable, I manage to do so. (Eg. filter by Name). When I try to filter out 2 variable, then a problem happen. I didnt manage to get any data.
This is the code that I use:
public List<Connection> retrieveAll(String nameFilter, String hostFilter,
int start, int length) {
ServiceUtil.requireAdmin();
CriteriaBuilder cb = em.getCriteriaBuilder();
CriteriaQuery<Connection> q = cb.createQuery(Connection.class);
Root<Connection> c = q.from(Connection.class);
q.select(c);
logger.info("nameFilter = [" + nameFilter + "]");
logger.info("hostFilter = [" + hostFilter + "]");
//This is the line that I use to query data.
//when I replace Connection_.hostname with Connection_.logAsHost, or Connection_.name
//It works just fine. So I use either one of these line to query.
//q.where(cb.like(c.get(Connection_.hostname), "%" + hostFilter + "%"));
//q.where(cb.like(c.get(Connection_.logAsHost), "%" + hostFilter + "%"));
//q.where(cb.like(c.get(Connection_.name), "%" + nameFilter + "%"));
//This is the problem part.
//When I add cb.or, it cannot get any data.
//From the documentation, it should just be q.where(cb.or(A, B))
//Where A is the first expression and B is the second.
// I have confirm both the expression are working by calling it separately
q.where(cb.or(cb.like(c.get(Connection_.hostname), "%" + hostFilter + "%")), cb.like(c.get(Connection_.logAsHost), "%" + hostFilter + "%"));
List<Connection> results = em.createQuery(q).setFirstResult(start)
.setMaxResults(length).getResultList();
for (Connection conn : results) {
logger.info("Name=" + conn.getName() + ", hostname=["
+ conn.getHostname() + "]" + ", logAsHost =["
+ conn.getLogAsHost() + "]");
}
return results;
}
This the log to show that the data is available or not:
I used c.get(Connection_.hostname), and passed u to hostFilter,
INFO nameFilter = []
INFO hostFilter = [u]
INFO Name=fikrie3, hostname=[ulala], logAsHost =[]
INFO Name=testt, hostname=[ugaga], logAsHost =[]
I used c.get(Connection_.logAsHost), and passed u to hostFilter,
INFO nameFilter = []
INFO hostFilter = [u]
INFO Name=fikrie, hostname=[192.168.56.90], logAsHost =[ubuntu]
INFO Name=fikrie2, hostname=[192.168.56.90], logAsHost =[unix]
I combine both and passed u to hostFilter,
INFO nameFilter = []
INFO hostFilter = [u]
I'm assuming the cb.or() is causing this error. If so, how do I use OR condition properly in criteriaquery? I'm following this part
or(Expression<java.lang.Boolean> x, Expression<java.lang.Boolean> y) from the documentation.
For me it looks like only a problem with your parenthesis. Your predicates are not parameter to the or method
you have
q.where(
cb.or(
cb.like(c.get(Connection_.hostname), "%" + hostFilter + "%")
),
cb.like(c.get(Connection_.logAsHost), "%" + hostFilter + "%")
);
Which lead to an AND of both predicates.
It should rather be
q.where(
cb.or(
cb.like(c.get(Connection_.hostname), "%" + hostFilter + "%"),
cb.like(c.get(Connection_.logAsHost), "%" + hostFilter + "%")
)
);
I'm trying to run a mapReduce query on Riak 1.4 that queries by secondary index, sorts the records by date, and then limits the results to the first record.
I've got the secondary index query working. The sort doesn't seem to do anything. No errors on the sort, just returns the results unsorted. The limit on the number of records returned yields a 'bad_json' error returned by the server.
Here's what I have. It is suppose to query the "cars" bucket for the most recent car owned by "john_doe". (some names have been changed to protect the innocent;) :
JSSourceFunction dateSortFunction = new JSSourceFunction(
"function(v) {" +
"return v.sort(function(a, b) {" +
"return a.issueDate - b.issueDate ;" +
"}" +
");" +
"}");
IndexQuery iq = new BinValueQuery(BinIndex.named("person"), "cars", "john_doe");
MapReduceResult response = session.mapReduce(iq)
.addMapPhase(NamedErlangFunction.MAP_OBJECT_VALUE)
.addReducePhase(dateSortFunction)
.addReducePhase(new NamedJSFunction("Riak.reduceLimit"), 1)
.execute();
I've seen a number of posts on sorting and am hoping to figure it out eventually. However, I haven't seen any help on how the LIMIT function might work.
Thanks in advance!
Update:
Thanks to Joe, he put me on the right track. Here's what ended up working for me. My date format is ISO 8601 (eg. 2011-05-18T17:00:00-07:00). So, I can lexically compare for the correct sorting. Also, I found javascript's array shortening method and updated the code to return up-to the first 5 objects.
JSSourceFunction sortLimitFunction = new JSSourceFunction(
"function(v) {" +
"v.sort(function(a, b) {" +
"return a.issueDate < b.issueDate" +
"}" +
");" +
"if (v.length > " + "5" + ") { " +
"v.length = " + "5" + ";" +
"}" +
"return v;" +
"}");
IndexQuery iq = new BinValueQuery(BinIndex.named("person"), "cars", "john_doe");
MapReduceResult response = session.mapReduce(iq)
.addMapPhase(new NamedJSFunction("Riak.mapValuesJson"))
.addReducePhase(sortLimitFunction)
.execute();
For the sorting, there is a mailing list post that covers this topic. The main difference I see between that implementation and yours is the use of the JavaScript Riak.mapValuesJson function in the map phase.
For the limiting, if you want just the first item from the sorted list, try having your sort function return only the first element. While the reduce function can (and probably is) called multiple times as partial result sets arrive from the various vnodes, the first element in the consolidated list must also be the first element in the partial list where it originated, so this should give you what you are looking for:
JSSourceFunction dateSortFunction = new JSSourceFunction(
"function(v) {" +
"var arr = v.sort(function(a, b) {" +
"return a.issueDate - b.issueDate ;" +
"}" +
");" +
"if (arr.length == 0) { " +
"return [];" +
"} else {"
"return arr[0];" +
"}"
"}"
);
I have a java function that is meant to take strings from jlist called "readyList" and pulling data from mysql workbench tables with the intent to write a line for each string in a .csv file. With the current code it sucessfully pulls the data one at a time like i intended but it only writes the last line instead of all the lines. I want to have all the lines written in the .csv file. Please help!
int[] selectedIx = readyList.getSelectedIndices();
for (int i = 0; i < selectedIx.length; i++) {
// while (i < selectedIx.length) {
Object sel = readyList.getModel().getElementAt(selectedIx[i]);
Statement s1 = DBConnect.connection.createStatement();
String selTable01 = "SELECT Sku As s, Qty As q, Orig_Retail As prce, Orig_Sku As orgsk, Store As strcd "
+ "FROM completed_lines WHERE Form_Name = '" + sel + "' AND Warranty = 'true'";
s1.execute(selTable01);
try (ResultSet rs01 = s1.getResultSet()) {
fWriter = new FileWriter("Exports/Transfers/" + /* frmNm.replace(":", "_") */"EBW_" + GtDates.fdate + ".csv", false);
writer = new BufferedWriter(fWriter);
String header = "slip_number,slip_type,out_store,in_store,item_number,quantity,price,comment,to_num";
writer.write(header);
writer.newLine();
while (rs01.next()) {
String strcode = rs01.getString("strcd");
String sku = rs01.getString("s");
String qty = rs01.getString("q");
String price = rs01.getString("prce");
String orgsku = rs01.getString("orgsk");
//System.out.println(frmNm.split("_")[1] + qty + "," + sku + "," + vendor + "," + desc1 + "," + reas + "," + descdmg + "," + orgR + "," + nwsku + "," + desc2 + "," + qtyI);
String line = ""+","+"out"+","+strcode+","+"RTV"+","+sku+","+qty+","+price+","+"EBW"+","+orgsku;
writer.write(line);
writer.newLine();
}
}
// JOptionPane.showMessageDialog(frame, "All Data from Selected Forms has been Exported");
}
// FormCompelted();
writer.close();
}
}
A few issues with this code. The reason you're only getting the last result is because of this line:
fWriter = new FileWriter("Exports/Transfers/" + /* frmNm.replace(":", "_") */"EBW_" + GtDates.fdate + ".csv", false);
This line is inside your loop. The false as the last parameter tells FileWriter not to append. In other words, a false means overwrite the file if it exists. Since this is in your loop, each result overwrites the file that the last result created. You should create the FileWriter outside of your loop, probably in a try with resources. That will allow you to remove your writer.close() call, which should have been in a finally block anyway.
Not related to your original question but something you should be aware of: You're creating a new Statement with each loop iteration. This can be an expensive operation. You should use a PreparedStatement instead. Create it outside your loop and then just set the parameter and execute it inside the loop. It also implements AutoCloseable, so you can create it in a try with resources too, probably the same one you create your FileWriter in.