Intention is to rule to activate when TradeEvent with specific book name appears, and PanicButtonManager panic mode is enabled:
when
$tradeEvent : TradeEvent(
bookShortName == "FMBTHQLA")
p : PanicButtonManager(
panicModeEnabled)
The thing is that when I change field panicModeEnabled value in drools it's not updated. The field value is constant from time of PanicButtonManager creation. But when I try to "print" it it's working fine:
when
$tradeEvent : TradeEvent(
bookShortName == "FMBTHQLA");
p : PanicButtonManager(
panicModeEnabled)
then
modify ($tradeEvent){
messageCode = "PM003",
message = "HQLA: FMBTHQLA is restricted to HQLA mode. Panic status: " + p.isPanicModeEnabled(),
tradeValidationStatus = STATUS.ERROR
}
This code activated whenever new TradeEvent appears with specific book name, but second condition stays same even tho I change panicMode field. But at the same time in message it prints correct status(it changes).
Why in "when" section evaluation of panicModeEnabled stays same? It keeps value from creation of class PanicButtonManager and doesn't update with change of field, but field prints correctly in message.
How do I make PanicButtonManager evaluate correctly in when section(Not keep state from creation time but update with field update).
From the comments, the state of PanicButtonManager is modified outside of Drools. Therefore to make the changed state visible, you need to call update to trigger a reevaluation of the rules with the new value. The modify is only making the new state of the TradeEvent visible.
The syntax would be like this:
rule "..."
when
// ...
p: PanicButtonManager( panicModeEnabled )
then
// ...
update(p);
end
Note that update will reevaluate all of the rules with the new condition(s), not just the remaining rules in the previous set of matches.
Right now, in my drools project I have two groups of rules in separate DRL files which are split by agenda groups. For the agenda group "preCheck" I am setting auto focus to true for each rule in that agenda group. Example:
rule "preCheckDuplicate"
agenda-group "preCheck"
auto-focus true
no-loop true
salience 50
when
$f : IngestFileMetadata(isDuplicate.equalsIgnoreCase("True"))
then
$f.setIsDuplicate("True");
end
For the other agenda group - "defaultRules" - the rules do NOT have the auto focus attribute set. Example:
rule "duplicate file default"
agenda-group "defaultRules"
activation-group "isDuplicate"
no-loop true
salience 0
when
$f : IngestFileMetadata(isDuplicate.equals("True"))
then
insert(createResponse($f));
end
When invoking the rules via the rest API, I am also trying to set focus to the "preCheck" agenda group through the JSON payload. Example:
{
"lookup": "defaultStatelessKieSession",
"set-focus": "preCheck",
"commands": [
{
"insert": {
"out-identifier": "IngestFileMetadata",
"return-object": "true",
"entry-point": "DEFAULT",
"object": {
"com.hms.ingestion.rules.IngestFileMetadata": {
* * * * * data attributes here * * * * *
}
}
}
},
{
"fire-all-rules": {"out-identifier": "fired"}
},
{
"query": {"name": "rulesResponses", "out-identifier": "rulesResponses"}
}
]
}
However, when the rules are executed, it seems like the rules in the "defaultRules" agenda group are being evaluated first. I have no idea why. I'm relatively new to drools so it's entirely possible I'm not correctly understanding the concept of agenda groups, but I was sure this design would ensure the "preCheck" rules would evaluate first.
Can anyone provide any insight on why this is not happening? If I need to provide more details I can.
Thanks in advance.
Agenda groups allow you to place rules into groups, and to place those groups onto a stack. The
stack has push/pop behavior.
Before going into how to use agenda group firstly, I want to say that configuring agenda group is depends on what type of KieSession you are using in your rule engine. For stateful Session, you can directly configure it by calling ksession.getAgenda().getAgendaGroup( "preCheck" ).setFocus();.
For Stateless Session, you have to declare an explicit rule to set the focus of the session to the particular Agenda. You can use the below rule to set agenda in Stateless Session:
rule "global"
salience 100
when
$f : IngestFileMetadata()
then
drools.setFocus($f.getAgenda());
end
Note : You have to find some way to get the agenda variable in your rule file. In the above example, getAgenda() is a method in your IngestFileMetadata class and it returns agenda value of String type
Turns out my issue was having to issue an explicit update to my fact once I updated the attributes during my pre-check rule. That solved the problem.
The situation is easy. I created a rules file:
package org.domain.rules;
dialect "mvel"
import eu.ohim.fsp.core.configuration.domain.xsd.Section;
global java.lang.String sectionName;
rule "rule 1"
salience 1000
when
Section($name : nameOfTheSection)
eval(sectionName == null)
then
System.out.println("Section: " + $name+ "("+$name.length()+")");
System.out.println("Section Name: " + sectionName + "("+sectionName.length()+")");
System.out.println("Mark Details: " + sectionName.equals(null));
end
And before firing the rules I added the Section object with a valid coreName and the globals:
public void fireInserted(Section section1) {
kstateful.insert(section1);
kstateful.setGlobal("sectionName", new String("markudetails"));
kstateful.fireAllRules();
}
The result is:
Section: markudetails(12)
Section Name: markudetails(12)
Mark Details: false
QUESTION: How can it be possible? in when part is null and in then part is not null!!!
Global vars are not a part of the knowledge base, but a separate channel to push some context into the rule execution. It is not appropriate to use them in a when clause. The exact reason why it was null in your case may be hard to trace, since rule activation is completely decoupled from rule execution. The variable may simply not be bound at when clause evaluation time, but is bound at then clause execution time.
To summarize: don't use globals in a when clause, that's not what they are for.
Your problem has an easy general solution: you can insert a configuration object into the knowledge. That object can have your desired "sectionName" property which you will then find easy to test in a when.
As an aside, it is meaningless to test for object.equals(null) -- this can never produce true. There is also no need to use new String("markudetails"). Instead use just "markudetails".
From the wouldn't-it-be-cool-if category of questions ...
By "queue-like-thing" I mean supports the following operations:
append(entry:Entry) - add entry to tail of queue
take(): Entry - remove entry from head of queue and return it
promote(entry_id) - move the entry one position closer to the head; the entry that currently occupies that position is moved in the old position
demote(entry_id) - the opposite of promote(entry_id)
Optional operations would be something like:
promote(entry_id, amount) - like promote(entry_id) except you specify the number of positions
demote(entry_id, amount) - opposite of promote(entry_id, amount)
of course, if we allow amount to be positive or negative, we can consolidate the promote/demote methods with a single move(entry_id, amount) method
It would be ideal if the following operations could be performed on the queue in a distributed fashion (multiple clients interacting with the queue):
queue = ...
queue.append( a )
queue.append( b )
queue.append( c )
print queue
"a b c"
queue.promote( b.id )
print queue
"b a c"
queue.demote( a.id )
"b c a"
x = queue.take()
print x
"b"
print queue
"c a"
Are there any data stores that are particularly apt for this use case? The queue should always be in a consistent state even if multiple users are modifying the queue simultaneously.
If it weren't for the promote/demote/move requirement, there wouldn't be much of a problem.
Edit:
Bonus points if there are Java and/or Python libraries to accomplish the task outlined above.
Solution should scale extremely well.
Redis supports lists and ordered sets: http://redis.io/topics/data-types#lists
It also supports transactions and publish/subscribe messaging. So, yes, I would say this can be easily done on redis.
Update: In fact, about 80% of it has been done many times: http://www.google.co.uk/search?q=python+redis+queue
Several of those hits could be upgraded to add what you want. You would have to use transactions to implement the promote/demote operations.
It might be possible to use lua on the server side to create that functionality, rather than having it in client code. Alternatively, you could create a thin wrapper around redis on the server, that implements just the operations you want.
Python: "Batteries Included"
Rather than looking to a data store like RabbitMQ, Redis, or an RDBMS, I think python and a couple libraries have more than enough to solve this problem. Some may complain that this do-it-yourself approach is re-inventing the wheel but I prefer running a hundred lines of python code over managing another data store.
Implementing a Priority Queue
The operations that you define: append, take, promote, and demote, describe a priority queue. Unfortunately python doesn't have a built-in priority queue data type. But it does have a heap library called heapq and priority queues are often implemented as heaps. Here's my implementation of a priority queue meeting your requirements:
class PQueue:
"""
Implements a priority queue with append, take, promote, and demote
operations.
"""
def __init__(self):
"""
Initialize empty priority queue.
self.toll is max(priority) and max(rowid) in the queue
self.heap is the heap maintained for take command
self.rows is a mapping from rowid to items
self.pris is a mapping from priority to items
"""
self.toll = 0
self.heap = list()
self.rows = dict()
self.pris = dict()
def append(self, value):
"""
Append value to our priority queue.
The new value is added with lowest priority as an item. Items are
threeple lists consisting of [priority, rowid, value]. The rowid
is used by the promote/demote commands.
Returns the new rowid corresponding to the new item.
"""
self.toll += 1
item = [self.toll, self.toll, value]
self.heap.append(item)
self.rows[self.toll] = item
self.pris[self.toll] = item
return self.toll
def take(self):
"""
Take the highest priority item out of the queue.
Returns the value of the item.
"""
item = heapq.heappop(self.heap)
del self.pris[item[0]]
del self.rows[item[1]]
return item[2]
def promote(self, rowid):
"""
Promote an item in the queue.
The promoted item swaps position with the next highest item.
Returns the number of affected rows.
"""
if rowid not in self.rows: return 0
item = self.rows[rowid]
item_pri, item_row, item_val = item
next = item_pri - 1
if next in self.pris:
iota = self.pris[next]
iota_pri, iota_row, iota_val = iota
iota[1], iota[2] = item_row, item_val
item[1], item[2] = iota_row, iota_val
self.rows[item_row] = iota
self.rows[iota_row] = item
return 2
return 0
The demote command is nearly identical to the promote command so I'll omit it for brevity. Note that this depends only on python's lists, dicts, and heapq library.
Serving our Priority Queue
Now with the PQueue data type, we'd like to allow distributed interactions with an instance. A great library for this is gevent. Though gevent is relatively new and still beta, it's wonderfully fast and well tested. With gevent, we can setup a socket server listening on localhost:4040 pretty easily. Here's my server code:
pqueue = PQueue()
def pqueue_server(sock, addr):
text = sock.recv(1024)
cmds = text.split(' ')
if cmds[0] == 'append':
result = pqueue.append(cmds[1])
elif cmds[0] == 'take':
result = pqueue.take()
elif cmds[0] == 'promote':
result = pqueue.promote(int(cmds[1]))
elif cmds[0] == 'demote':
result = pqueue.demote(int(cmds[1]))
else:
result = ''
sock.sendall(str(result))
print 'Request:', text, '; Response:', str(result)
if args.listen:
server = StreamServer(('127.0.0.1', 4040), pqueue_server)
print 'Starting pqueue server on port 4040...'
server.serve_forever()
Before that runs in production, you'll of course want to do some better error/buffer handling. But it'll work just fine for rapid-prototyping. Notice that this doesn't require any locking around the pqueue object. Gevent doesn't actually run code in parallel, it just gives that impression. The drawback is that more cores won't help but the benefit is lock-free code.
Don't get me wrong, the gevent SocketServer will process multiple requests at the same time. But it switches between answering requests through cooperative multitasking. This means you have to yield the coroutine's time slice. While gevents socket I/O functions are designed to yield, our pqueue implementation is not. Fortunately, the pqueue completes it's tasks really quickly.
A Client Too
While prototyping, I found it useful to have a client as well. It took some googling to write a client so I'll share that code too:
if args.client:
while True:
msg = raw_input('> ')
sock = gsocket.socket(gsocket.AF_INET, gsocket.SOCK_STREAM)
sock.connect(('127.0.0.1', 4040))
sock.sendall(msg)
text = sock.recv(1024)
sock.close()
print text
To use the new data store, first start the server and then start the client. At the client prompt you ought to be able to do:
> append one
1
> append two
2
> append three
3
> promote 2
2
> promote 2
0
> take
two
Scaling Extremely Well
Given your thinking about a data store, it seems you're really concerned with throughput and durability. But "scale extremely well" doesn't quantify your needs. So I decided to benchmark the above with a test function. Here's the test function:
def test():
import time
import urllib2
import subprocess
import random
random = random.Random(0)
from progressbar import ProgressBar, Percentage, Bar, ETA
widgets = [Percentage(), Bar(), ETA()]
def make_name():
alphabet = 'abcdefghijklmnopqrstuvwxyz'
return ''.join(random.choice(alphabet)
for rpt in xrange(random.randrange(3, 20)))
def make_request(cmds):
sock = gsocket.socket(gsocket.AF_INET, gsocket.SOCK_STREAM)
sock.connect(('127.0.0.1', 4040))
sock.sendall(cmds)
text = sock.recv(1024)
sock.close()
print 'Starting server and waiting 3 seconds.'
subprocess.call('start cmd.exe /c python.exe queue_thing_gevent.py -l',
shell=True)
time.sleep(3)
tests = []
def wrap_test(name, limit=10000):
def wrap(func):
def wrapped():
progress = ProgressBar(widgets=widgets)
for rpt in progress(xrange(limit)):
func()
secs = progress.seconds_elapsed
print '{0} {1} records in {2:.3f} s at {3:.3f} r/s'.format(
name, limit, secs, limit / secs)
tests.append(wrapped)
return wrapped
return wrap
def direct_append():
name = make_name()
pqueue.append(name)
count = 1000000
#wrap_test('Loaded', count)
def direct_append_test(): direct_append()
def append():
name = make_name()
make_request('append ' + name)
#wrap_test('Appended')
def append_test(): append()
...
print 'Running speed tests.'
for tst in tests: tst()
Benchmark Results
I ran 6 tests against the server running on my laptop. I think the results scale extremely well. Here's the output:
Starting server and waiting 3 seconds.
Running speed tests.
100%|############################################################|Time: 0:00:21
Loaded 1000000 records in 21.770 s at 45934.773 r/s
100%|############################################################|Time: 0:00:06
Appended 10000 records in 6.825 s at 1465.201 r/s
100%|############################################################|Time: 0:00:06
Promoted 10000 records in 6.270 s at 1594.896 r/s
100%|############################################################|Time: 0:00:05
Demoted 10000 records in 5.686 s at 1758.706 r/s
100%|############################################################|Time: 0:00:05
Took 10000 records in 5.950 s at 1680.672 r/s
100%|############################################################|Time: 0:00:07
Mixed load processed 10000 records in 7.410 s at 1349.528 r/s
Final Frontier: Durability
Finally, durability is the only problem I didn't completely prototype. But I don't think it's that hard either. In our priority queue, the heap (list) of items has all the information we need to persist the data type to disk. Since, with gevent, we can also spawn functions in a multi-processing way, I imagined using a function like this:
def save_heap(heap, toll):
name = 'heap-{0}.txt'.format(toll)
with open(name, 'w') as temp:
for val in heap:
temp.write(str(val))
gevent.sleep(0)
and adding a save function to our priority queue:
def save(self):
heap_copy = tuple(self.heap)
toll = self.toll
gevent.spawn(save_heap, heap_copy, toll)
You could now copy the Redis model of forking and writing the data store to disk every few minutes. If you need even greater durability then couple the above with a system that logs commands to disk. Together, those are the AFP and RDB persistence methods that Redis uses.
Websphere MQ can do almost all of this.
The promote/demote is almost possible, by removing the message from the queue and putting it back on with a higher/lower priority, or, by using the "CORRELID" as a sequence number.
What's wrong with RabbitMQ? It sounds exactly like what you need.
We extensively use Redis as well in our Production environment, but it doesn't have some of the functionality Queues usually have, like setting a task as complete, or re-sending the task if it isn't completed in some TTL. It does, on the other hand, have other features a Queue doesn't have, like it is a generic storage, and it is REALLY fast.
Use Redisson it implements familiar List, Queue, BlockingQueue, Deque java interfaces in distributed approach provided by Redis. Example with a Deque:
Redisson redisson = Redisson.create();
RDeque<SomeObject> queue = redisson.getDeque("anyDeque");
queue.addFirst(new SomeObject());
queue.addLast(new SomeObject());
SomeObject obj = queue.removeFirst();
SomeObject someObj = queue.removeLast();
redisson.shutdown();
Other samples:
https://github.com/mrniko/redisson/wiki/7.-distributed-collections/#77-list
https://github.com/mrniko/redisson/wiki/7.-distributed-collections/#78-queue https://github.com/mrniko/redisson/wiki/7.-distributed-collections/#710-blocking-queue
If you for some reason decide to use an SQL database as a backend, I would not use MySQL as it requires polling (well and would not use it for lots of other reasons), but PostgreSQL supports LISTEN/NOTIFY for signalling other clients so that they do not have to poll for changes. However, it signals all listening clients at once, so you still would require a mechanism for choosing a winning listener.
As a sidenote I am not sure if a promote/demote mechanism would be useful; it would be better to schedule the jobs appropriately while inserting...
I am trying to schedule a quartz job according to the following plan:
Job runs daily and should only be executed between 9:30am and 6:00pm. I am trying to achieve this via DailyCalendar. Here what my DailyCalendar looks like:
DailyCalendar dCal = new DailyCalendar(startTimeString, endTimeString);
dCal.setTimeZone(TimeZone.getDefault());
dCal.setInvertTimeRange(true);
where start and end time strings are of the format HH:MM
Next, I try to schedule this job:
Scheduler myscheduler = StdSchedulerFactory.getDefaultScheduler();
SimpleTrigger trigger = new SimpleTrigger();
myscheduler.addCalendar("todcal", cal, true, true);
trigger.setName("TRIGGER " + alertName);
trigger.setJobName(alertName);
trigger.setJobGroup(alertName);
trigger.setCalendarName("todcal");
logger.info("Adding TOD job");
myscheduler.scheduleJob(trigger); // line causing exception
myscheduler.start();
As soon as scheduleJob is called I see the following Exception:
Based on configured schedule, the given trigger will never fire.
The configuration seems fine to me but I cant find any sample code for using DailyCalendar so I could be wrong here. Please help
You don't seem to be setting a repeat count or repeat interval on your trigger. So it will only fire once at the current moment (because you did not set a future start time), which probably happens to be during the calendar's exclusion time - which is why it would be calculated that it will never fire.
Job runs daily and should only be
executed between 9:30am and 6:00pm.
How often should the job be executed within that timeframe? Once? Once an hour? Every 10 seconds?
You need to define the repeat interval for your trigger. Look at setRepeatInterval(long repeatInterval) method of SimpleTrigger. It defines in milliseconds the interval with which the trigger will repeat.