I have set up a logstash with redis architecture to handle my logs. The way I have organized it is:
logstash ---> redis ---> logstash ---> elasticsearch
but the problem that occurred is that after parsing nearly 1.25 million logs a java exception is thrown.
In my logstash.err log file, the exception appears as
Exception in thread "<file" java.lang.UnsupportedOperationException
at java.lang.Thread.stop(Thread.java:869)
at org.jruby.RubyThread.exceptionRaised(RubyThread.java:1221)
at org.jruby.internal.runtime.RubyRunnable.run(RubyRunnable.java:112)
at java.lang.Thread.run(Thread.java:745)
I think that this exception might be thrown because of logstash unable to open/close a file. So what can I do to rectify this error? The way that my input configuration is set for my first logstash server to send the logs is:
input {
file {
start_position => "beginning"
path => [
"/var/logstash_logs/child1/nginx/*log*",
"/var/logstash_logs/child2/nginx/*log*",
"/var/logstash_logs/child3/nginx/*log*"
]
}
}
And the way output is sent is like this:
output {
redis {
host => "X.X.X.X"
key => "logstash"
data_type => "list"
}
}
There are no errors in the logs of logstash server with redis installed.
Well, one problem here is with JRuby, which is trying to call Thread.stop(Throwable obj);, a deprecated method that throws UnsupportedOperationException and totally messes up the actual source of the error (the Throwable parameter).
So currently you can only guess what the actual problem is, and guessing is never good.
One idea is to set a breakpoint on RubyThread.exceptionRaised(); and run it through a debugger. That should allow you to find out what the original Throwable is, and then you can get to the source of the problem.
You should also check if there exists a bug ticket for JRuby about this, and possibly update your JRuby.
Related
Hey i try to using Ktor with the Kotlin Hands on. But I have a Problem when access the Web Page with "/".
embeddedServer(Netty, port) {
install(FreeMarker) {
templateLoader = ClassTemplateLoader(this::class.java.classLoader, "templates")
outputFormat = HTMLOutputFormat.INSTANCE
}
routing {
static("/static") {
resources("files")
}
get("/") {
call.respond(FreeMarkerContent("index.ftl", mapOf("entries" to blogEntries), ""))
}
}
}.start(wait = false)
There is my code i use. I cant find an error there is only this message in the console:
[eventLoopGroupProxy-4-1] INFO ktor.application - Unhandled: GET - /, io failed
Is there anyone who has an idea how can I fix that error?
The exact line where that message is generated is in io.ktor.server.engine.logFailure: https://github.com/ktorio/ktor/blob/2e1b0046e4235865da7cbbfbf2e887f410775278/ktor-server/ktor-server-host-common/jvm/src/io/ktor/server/engine/DefaultEnginePipeline.kt#L107
private fun ApplicationEnvironment.logFailure(call: ApplicationCall, cause: Throwable) {
...
is IOException -> log.info("$status: $logString, io failed: ${cause.message ?: "unknown error"}")
...
}
Add a breakpoint on this line. Then when your app fails, cause will contain the real exception.
In my case it was an error in the FreeMarker template. Which is weird, because usually those exceptions print normally. This time it only printed Unhandled: <route>, io failed.
It seems that this is a bug in Ktor: these exceptions are thrown in normal behavior, and frequently enough that you don't want a full stacktrace. But sometimes (like in this case) the exception isn't normal and you do want a stack trace.
I suggest using the CallLogging feature and routes tracing to troubleshoot your problem.
I am using Server-Sent events on one browser, and a spring boot application on the back end. When I shot down the client, I get the next exception:
14:35:09.458 [http-nio-8084-exec-25] ERROR o.a.c.c.C.[Tomcat].[localhost] - Exception Processing ErrorPage[errorCode=0, location=/error]
org.apache.catalina.connector.ClientAbortException: java.io.IOException: Broken pipe
I understand this is the expected behavior; on the other hand, my application works fine, but I have awful logs full of those exceptions. I guess this is caused by Tomcat. Is there a way to catch these exceptions, or at least to prevent Tomcat from writing this exception stack trace to the log? I mean, without modifying Tomcat's code.
To prevent this exception in logs you can try some changes in your code that performs push on client. Here is my example. I listen to api called and then it called I push socket to the client. I think you could understand the code:
#GetMapping("/api/status/{groupId}/{groupstatusId}")
#ResponseStatus(HttpStatus.NO_CONTENT)
#ExceptionHandler(IOException.class)
public void listenToNewStatus(#PathVariable Long groupId, #PathVariable String groupstatusId, IOException e) {
Group group = groupDTOService.findById(groupId);
if (group != null) {
if (StringUtils.containsIgnoreCase(ExceptionUtils.getRootCauseMessage(e), "Broken pipe")) {
logger.info("broken pipe");
} else {
template.convertAndSend("/topic/callstatus/" + group.getUser().getId(), groupstatusId);
}
}
In this code to prevent broken pipe I add annotation #ExceptionHandler(IOException.class) and check if exception contains broken pipe then nothing else send message to client.
I see that this question is quite old, but in case someone is still looking for an answer, there are a few blog posts on how to mute ClientAbortException so it doesn't flood your logs.
https://tutorial-academy.com/jersey-workaround-clientabortexception-ioexception/
I use Java to index some documents with a BulkRequestinto Elasticsearch 1.4.2.
Some of these docs only need to be written when they are not already in the index, so I set the CREATE-opType like this:
indexRequestBuilder.opType(IndexRequest.OpType.CREATE)
Now the docs which were already in the index fail in the BulkResponse.
Error message bulkItemResponse.getFailureMessage():
DocumentAlreadyExistsException[...]
I want to ignore this class of exception but retry writing the docs for all other type of exceptions.
So how can I catch just the DocumentAlreadyExistsException?
I can get the Failure with bulkItemResponse.getFailure(), but I cannot find any information about the type of the Exception beside the error message.
I could look in the error-message for the exception name, but this may be rather fragile with new Elasticsearch versions:
if(bulkItemResponse.getFailureMessage().startsWith("DocumentAlreadyExistsException[")
Is there a better way?
This cant be possible. The bulk request is actaully executed on the server side and not client side. And hence all it can do is to sent the stacktrace back and not the Exception object.
I'm using Square's Tape library to queue uploads of data to the server.
The queue is stored in File in JSON format. When the app starts I init the queue and start uploading (i.e if on Wifi) However on some devices on users I'm seeing EOFException with 'null' message (logged in crashlytics).
The error occurs when creating a FileObjectQueue object from an existing file - from the debug info gather the actual file is ~1MB.
Any ideas what's causing this or how to prevent it? - maybe I need to dust up on my java.io.
Edit: using Tape v1.2.1
Caused by: java.io.EOFException
at java.io.RandomAccessFile.readFully(RandomAccessFile.java:419)
at java.io.RandomAccessFile.readInt(RandomAccessFile.java:439)
at com.squareup.tape.QueueFile.readElement(:182)
at com.squareup.tape.QueueFile.readHeader(:162)
at com.squareup.tape.QueueFile.(:110)
at com.squareup.tape.FileObjectQueue.(:35)
at com.myapp.queue.MyUploadTaskQueue.create(:125)
Updated - Also seeing this error since upgrading to 1.2.2
Caused by: java.io.IOException: File is corrupt; length stored in header is 0.
at com.squareup.tape.QueueFile.readHeader(:165)
at com.squareup.tape.QueueFile.<init>(:117)
at com.squareup.tape.FileObjectQueue.<init>(:35)
The EOFException shows that End Of File has been reached, that is, there are no more bytes to read. This exception is just another way to signal that there is nothing more to read, whereas other methods return a value, like -1. As you can see in your error stack trace, the methods throwing the exception are read methods; java.io.RandomAccessFile.readFully(RandomAccessFile.java:419) and com.squareup.tape.QueueFile.readHeader(:165). As such, it can't be "prevented" unless you don't read all the bytes (which you typically want to), just catch it like so; catch(EOFException e) { /* ignore */ } :)
https://docs.oracle.com/javase/7/docs/api/java/io/EOFException.html
I am using GWT. if any server side exception is generated, we are sending an email with error details(have used log4j SMTPAppender). Based on the line number, we can fix the issue..
My scenario is, if any exception is generated in the client package code, as of now, we are giving generic message saying "Some Exception has Occured". But is it possible to display error stack trace along with the exception cause and line number? my code is as below:
GWT.setUncaughtExceptionHandler(new
GWT.UncaughtExceptionHandler() {
public void onUncaughtException(Throwable e) {
Window.alert("Some Exception has Occured");
}
});
I dont think it is possible as client package is converted into Javascript in web mode. Please suggest me if there is any approach to display exception cause and line number where it has occured.
You can read this page
Basically, you have to use JUL to do your logging, and it's client logging : firebug, JS console, etc... You may do some smarter things with the RemoteLogging but i can't help you on that.
The main problem is that log4j is not supported. Maybe with a bridge between JUL and log4j you will be able to achieve everything you want
I would recommend using gwt-log:
Project Page
gwt-log has support for an 'UncaughtExceptionHandler' and a RemoteLogger to send messages/exception to the server.
in gwt-log, you can also turn on the "emulated stack", which is exactly what you want to do:
Wiki Page - Emulated Stack
please note however that this adds a LOT of code to the compiled JS-script