How can I limit the rate of emails a Logback SMTPAppender, so that it would email me at most once every n minutes?
I have setup my logging according to the Logback appender, but I don't quite see how it be configured or subclassed to implement that.
Is there a hidden feature? Did someone develop a subclass to handle this?
Based on the documentation it appears that the way to do this is to write an EventEvaluator (see example 4.14 and 4.15) which looks at the time stamp for each event to only accept an event when "enough time" has passed since the last event was accepted.
You can use System.currentTimeMillis to get a number you can do math on to calculate time differences. http://java.sun.com/javase/6/docs/api/java/lang/System.html#currentTimeMillis%28%29
As Thorbjørn, it's easy to create an EventEvaluator that limit the rate by which an appender fires a message.
However, I found Logback to support DuplicateMessageFilter, that solves my problem probably in a bitter way: "The DuplicateMessageFilter merits a separate presentation. This filter detects duplicate messages, and beyond a certain number of repetitions, drops repeated messages."
Have a look at the new Whisper appender. It does smart suppression. Available via Maven and github here
Statutory disclaimer: I'm the author.
This tool would do exactly what you want but it's not threadsafe at all: http://code.google.com/p/throttled-smtp-appender/wiki/Usage
I've written a threadsafe version but haven't open sourced it yet.
The reason you would have trouble finding good tools for this is that SMTP isn't a real endpoint. Use a service like loggly, airbrake, or dozens of others, or run your own server using something like logstash.
To solve same problem I've written custom evaluator. It extends ch.qos.logback.classic.boolex.OnMarkerEvaluator, but you can use any other evaluator as base. If there will many acceptable messages in silence interval evaluator will discard these. For my use case it's ok, but if you need different behavior - just add extra checks to the second if.
public class LimitingOnMarkerEvaluator extends OnMarkerEvaluator {
private long lastSend = 0, interval = 0;
#Override
public boolean evaluate(ILoggingEvent event) throws EvaluationException {
if (super.evaluate(event)) {
long now = System.currentTimeMillis();
if (now - lastSend > interval) {
lastSend = now;
return true;
}
}
return false;
}
public long getInterval() {
return interval;
}
public void setInterval(long interval) {
this.interval = interval;
}
}
Config to send maximum one message every 1000 second (about 17 mins):
<evaluator class="package.LimitingOnMarkerEvaluator">
<marker>FATAL</marker>
<interval>1000000</interval>
</evaluator>
I suggest filing a jira item requesting this feature. It is likely to be implemented if only asked.
Btw,
Logback v0.9.26 allows now to set the size of SMTPAppender message buffer. Until yesterday it would send the current contens of the buffer which was up to 256 messages which imho was a pain in the neck as I wanted to show only the last one in the email. Thus it's now possible to implement periodically recurring email warnings that carry only one particular error as per my interpretation of this question.
http://logback.qos.ch/manual/appenders.html#cyclicBufferSize
Have fun.
Related
I have a following flow that I would like to implement using Spring Integration Java DSL:
Poll a table in a database every 2 hours which returns id of documents that need to be processed
For each id, process a document through an HTTP gateway
Store a response in a database
I have a working Java code that does exactly these steps. An additional requirement that I'm struggling with is that the polling for the next round of documents shouldn't happen until all the documents from the last polling has been processed and stored in the database.
Is there any pattern in Spring Integration that I could use for this additional requirement?
Here is a simplified code - it will get more complex and I'll split processing of the documents (HTTP outbound and persisting) into separate classes / flows:
return IntegrationFlows.from(Jpa.inboundAdapter(this.targetEntityManagerFactory)
.entityClass(ProcessingMetadata.class)
.jpaQuery("select max(p.modifiedDate) from ProcessingMetadata p " +
"where p.status = com.test.ProcessingStatus.PROCESSED")
.maxResults(1)
.expectSingleResult(true),
e -> e.poller(Pollers.fixedDelay(Duration.ofSeconds(10))))
.handle(Jpa.retrievingGateway(this.sourceEntityManagerFactory)
.entityClass(DocumentHeader.class)
.jpaQuery("from DocumentHeader d where d.modified > :modified")
.parameterExpression("modified", "payload"))
.handle(Http.outboundGateway(uri)
.httpMethod(HttpMethod.POST)
.expectedResponseType(String.class))
.handle(Jpa.outboundAdapter(this.targetEntityManagerFactory)
.entityClass(ProcessingMetadata.class)
.persistMode(PersistMode.PERSIST),
e -> e.transactional(true))
.get();
UPDATE
Following Artem's suggestion, I'm trying to implement it using a SimpleActiveIdleMessageSourceAdvice
class WaitUntilCompleted extends SimpleActiveIdleMessageSourceAdvice {
public WaitUntilCompleted(DynamicPeriodicTrigger trigger) {
super(trigger);
}
#Override
public boolean beforeReceive(MessageSource<?> source) {
return false;
}
}
If I understand it correctly, above code would stop polling. Now I have no idea how to attach this Advice to the Jpa.inboundAdapter... It doesn't seem to have a proper method (neither Advice nor Spec Handler). Do I miss something obvious here? I've tried attaching the Advice to the Jpa.retrievingGateway but it doesn't change the flow at all.
UPDATE2
Check this question for a complete solution: Spring Integration: how to unit test an advice
I have answered today for similar question: How to poll from a queue 1 message at a time after downstream flow is completed in Spring Integration.
You also may have a trick on database level do not let to see new records in the table while others are locked. Or you can have some UPDATE in the end of flow while your SELECT won't see appropriate records until they are updated respectively.
But anyway any of those approaches I suggest for that question should be applied here as well.
Also you indeed can consider to rely on the SimpleActiveIdleMessageSourceAdvice since your solution is already based on a MessageSource implementation.
UPDATE
For your use-case it is probably would be better to extend that SimpleActiveIdleMessageSourceAdvice and override its beforeReceive() to check some state that you are able to read more data or not. The idlePollPeriod and activePollPeriod could be the same value: doesn't look like it make sense to change it in between since you are going to the idle state just after reading the next set of data.
For the state to check it really might be a simple AtomicBoolean bean which you should change after you process the current set of documents. That might be something after an aggregator or anything else you can use in your solution.
UPDATE 2
To use a WaitUntilCompleted for your Jpa.inboundAdapter you should have a configuration like this:
IntegrationFlows.from(Jpa.inboundAdapter(this.targetEntityManagerFactory)
.entityClass(ProcessingMetadata.class)
.jpaQuery("select max(p.modifiedDate) from ProcessingMetadata p " +
"where p.status = com.test.ProcessingStatus.PROCESSED")
.maxResults(1)
.expectSingleResult(true),
e -> e.poller(Pollers.fixedDelay(Duration.ofSeconds(10)).advice(waitUntilCompleted())))
Pay attention to the .advice(waitUntilCompleted()) which is a part of the pller configuration and points to your advice bean.
Given some code using streams to process a large number of items, what's the best way to instrument the various steps for logging and performance/profiling?
Actual example:
ReactiveSeq.fromStream(pairs)
.filter(this::satisfiesThreshold)
.filter(this::satisfiesPersistConditions)
.map((pair) -> convertToResult(pair, jobId))
.flatMap(Option::toJavaStream)
.grouped(CHUNK_SIZE)
.forEach((chunk) ->
{
repository.save(chunk);
incrementAndReport();
});
reportProcessingTime();
Logging progress is important so I can trigger progress events in another thread that update a user interface.
Tracking the performance characteristics of the filtering and mapping steps in this stream is desireable to see where optimizations can be made to speed it up.
I see three options:
put logging/profiling code in each function
use peek around each step without actually using the value
some sort of annotation based or AOP solution (no idea what)
Which is the best? Any ideas on what #3 would look like? Is there another solution?
You have a couple of options here (if I have understood correctly) :-
We can make use of the elapsed operator to track the elapsed time between element emissions e.g.
ReactiveSeq.fromStream(Stream.of(1,2))
.filter(this::include)
.elapsed()
.map(this::logAndUnwrap)
Long[] filterTimeTakenMillis = new Long[maxSize];
int filterIndex = 0;
private <T> T logAndUnwrap(Tuple2<T, Long> t) {
//capture the elapsed time (t.v2) and then unwrap the tuple
filterTimeTakenMillis[filterIndex++]=t.v2;
return t.v1;
}
This will only work on cyclops-react Streams.
We can make use of the AOP-like functionality in FluentFunctions
e.g.
ReactiveSeq.fromStream(Stream.of(1,2))
.filter(this::include)
.elapsed()
.map(this::logAndUnwrap)
.map(FluentFunctions.of(this::convertToResult)
.around(a->{
SimpleTimer timer = new SimpleTimer();
String r = a.proceed();
mapTimeTakenNanos[mapIndex++]=timer.getElapsedNanos();
return r;
}));
This will also work on vanilla Java 8 Streams.
Can someone provide an example of how the Progressable interface might be implemented for use when calling FileSystem.create()? I saw the following code snippet in another post, but it did not show where bytesWritten came from:
OutputStream os = hdfs.create( file,
new Progressable() {
public void progress() {
out.println("...bytes written: [ "+bytesWritten+" ]");
} });
The documentation of this interface says it is for reporting progress to the Hadoop framework to avoid timeout in the case of a lengthy operation, but "Hadoop: The Definitive Guide" says it is for notifying the application of the progress of the data being written to the data nodes, which doesn't make much sense since it is a create.
Thanks, RF
If you have an implementation of Mapper where an invocation of map() may take a long time (like more than several minutes), then you can periodically call progress() on the provided context object to let Hadoop know that your code isn't hung. That's what they mean by "explicitly reporting progress" - it works when you're using an object provided by the framework that implements Progressable, it obviously doesn't work that way when you write your own implementation of Progressable.
I should have read the Hadoop book further -- here is the example they gave later on:
OutputStream out = fs.create(new Path(dst), new Progressable() {
public void progress() {
System.out.print(".");
}
The accompanying text says " We illustrate progress
by printing a period every time the progress() method is called by Hadoop, which is after each 64 KB packet of data is written to the datanode pipeline".
I guess my question becomes, how does this "explicitly report progress to the Hadoop framework" as stated by the documentation of Progressable?
Our app suddenly got a lot of traffic and there were some design flaws in the system (or rather we never thought it would get this much traffic so we just skipped it by choice).
As the topic states I'm looking for a way to prevent the error: org.datanucleus.exceptions.NucleusDataStoreException: Concurrent Modification
Currently I have an entity called Group that looks like this:
#PersistenceCapable
public class Group extends PersistableString {
private static final long serialVersionUID = 6215353466976945628L;
#Persistent
private int yesCount;
#Persistent
private int noCount;
public void increaseYesCount()
{
yesCount++;
}
public void increaseNoCount()
{
noCount++;
}
}
The following code is how the update of the entity is done:
int answer = Integer.parseInt(req.getParameter("answer"))
try {
PersistenceManager pm = PMF.getPersistenceManager();
for(String groupId : allGroupsToBeUpdated)
{
Group group = pm.getObjectById(Group.class, groupId);
if(answer == 0)
group.increaseNoCount();
else
group.increaseYesCount();
}
pm.close();
} catch (Exception e) {
e.printStackTrace();
}
allGroupsToBeUpdated is a list that contains around 30 string-ids. Is there some way I can avoid the Concurrent Modification-error? Can I check if the entity that I retrieve is being updated and then just discard(/ignore) the update? It's not SUPER important that the write actually succeeds, I just wanna make sure I don't get the error (or that it keeps trying to succeed with the write), because it's causing the request to take between 10-30seconds.
Should I maybe open (get new PM-instance) and close the connection (pm.close()) between each update instead of waiting for all of the 30ish updates to go through?
I know of sharded counters and should have (obviously) used them, but right now I'm looking for a "quick-fix" to this problem.
EDIT:
I'm using:
App Engine SDK 1.8.9
JDO 3.0
Stacktrace can be found at:
http://pastebin.com/TWnmkpPU
Posting as an answer due to length.
Transactions probably aren't good in your case since you are really just looking to hide the issue from the user which is manifesting itself in slow request times. Perhaps kicking off an async push task to do the writes in the background outside of the request would be your best bet.
I really would not recommend design based on hiding errors and swallowing exceptions though. Looking to "prevent" an exception that is doing what it is supposed to (signaling a failed write due to contention), means you should avoid the condition which caused it in the first place.
I totally understand needing to get things working fast early on, but it may be a good idea to start adopting best practices now once the bad design decisions just start making their mark. Continuing to rely on "quick-fixes" and hiding problems can land you in a real mess later on.
I am implementing REST through RESTlet. This is an amazing framework to build such a restful web service; it is easy to learn, its syntax is compact. However, usually, I found that when somebody/someprogram want to access some resource, it takes time to print/output the XML, I use JaxbRepresentation. Let's see my code:
#Override
#Get
public Representation toXml() throws IOException {
if (this.requireAuthentication) {
if (!this.app.authenticate(getRequest(), getResponse()))
{
return new EmptyRepresentation();
}
}
//check if the representation already tried to be requested before
//and therefore the data has been in cache
Object dataInCache = this.app.getCachedData().get(getURI);
if (dataInCache != null) {
System.out.println("Representing from Cache");
//this is warning. unless we can check that dataInCache is of type T, we can
//get rid of this warning
this.dataToBeRepresented = (T)dataInCache;
} else {
System.out.println("NOT IN CACHE");
this.dataToBeRepresented = whenDataIsNotInCache();
//automatically add data to cache
this.app.getCachedData().put(getURI, this.dataToBeRepresented, cached_duration);
}
//now represent it (if not previously execute the EmptyRepresentation)
JaxbRepresentation<T> jaxb = new JaxbRepresentation<T>(dataToBeRepresented);
jaxb.setFormattedOutput(true);
return jaxb;
}
AS you can see, and you might asked me; yes I am implementing Cache through Kitty-Cache. So, if some XML that is expensive to produce, and really looks like will never change for 7 decades, then I will use cache... I also use it for likely static data. Maximum time limit for a cache is an hour to remain in memory.
Even when I cache the output, sometimes, output are irresponsive, like hang, printed partially, and takes time before it prints the remaining document. The XML document is accessible through browser and also program, it used GET.
What are actually the problem? I humbly would like to know also the answer from RESTlet developer, if possible. Thanks