We Keep Coding

XML data display in grid using kibana and logstash

I wanted to display XML data using logstash and Kibana in grid format. using below conf file I am able to display data into grid but not able to split row data.
Example:
Output
logstash.conf file :
input {
file {
path => "C:/ELK Stack/logstash-8.2.0-windows-x86_64/logstash-8.2.0/Test.xml"
start_position => "beginning"
sincedb_path => "NUL"
codec => multiline {
pattern => "^<?stations.*>"
negate => "true"
what => "previous"
auto_flush_interval => 1
max_lines => 3000
}}}
filter
{
xml
{
source => "message"
target => "parsed"
store_xml => "false"
xpath => [
"/stations/station/id/text()", "station_id",
"/stations/station/name/text()", "station_name"
]
}
mutate {
remove_field => [ "message"]
}
}
output {
elasticsearch {
action => "index"
hosts => "localhost:9200"
index => "logstash_index123xml"
workers => 1
}
stdout {
codec => rubydebug
}
}

xpath will always return arrays, to associate the members of the two arrays you are going to need to use a ruby filter. To get multiple events you can use a split filter to split an array which you build in the ruby filter. If you start with
<stations>
<station>
<id>1</id>
<name>a</name>
<id>2</id>
<name>b</name>
</station>
</stations>
then if you use
xml {
source => "message"
store_xml => "false"
xpath => {
"/stations/station/id/text()" => "[#metadata][station_id]"
"/stations/station/name/text()" => "[#metadata][station_name]"
}
remove_field => [ "message" ]
}
ruby {
code => '
ids = event.get("[#metadata][station_id]")
names = event.get("[#metadata][station_name]")
if ids.is_a? Array and names.is_a? Array y and ids.length == names.length
a = []
ids.each_index { |x|
a << { "station_name" => names[x], "station_id" => ids[x] }
}
event.set("[#metadata][theData]", a)
end
'
}
if [#metadata][theData] {
split {
field => "[#metadata][theData]"
add_field => {
"station_name" => "%{[#metadata][theData][station_name]}"
"station_id" => "%{[#metadata][theData][station_id]}"
}
}
}
You will get two events
{
"station_name" => "a",
"station_id" => "1",
...
}
{
"station_name" => "b",
"station_id" => "2",
...
}

Failed in loading csv data to Elasticsearch, translation issue

I am trying to import CSV file to elastic file, but it is failed and threw an error
Pipeline aborted due to error {:pipeline_id=>"main",
:exception=>#,
:backtrace=>["/usr/local/Cellar/logstash/7.6.1/libexec/vendor/bundle/jruby/2.5.0/gems/logstash-filter-mutate-3.5.0/lib/logstash/filters/mutate.rb:222:in
block in register'", "org/jruby/RubyHash.java:1428:ineach'",
"/usr/local/Cellar/logstash/7.6.1/libexec/vendor/bundle/jruby/2.5.0/gems/logstash-filter-mutate-3.5.0/lib/logstash/filters/mutate.rb:220:in
register'",
"org/logstash/config/ir/compiler/AbstractFilterDelegatorExt.java:56:in
register'",
"/usr/local/Cellar/logstash/7.6.1/libexec/logstash-core/lib/logstash/java_pipeline.rb:200:in
block in register_plugins'", "org/jruby/RubyArray.java:1814:in
each'",
"/usr/local/Cellar/logstash/7.6.1/libexec/logstash-core/lib/logstash/java_pipeline.rb:199:in
register_plugins'",
"/usr/local/Cellar/logstash/7.6.1/libexec/logstash-core/lib/logstash/java_pipeline.rb:502:in
maybe_setup_out_plugins'",
"/usr/local/Cellar/logstash/7.6.1/libexec/logstash-core/lib/logstash/java_pipeline.rb:212:in
start_workers'",
"/usr/local/Cellar/logstash/7.6.1/libexec/logstash-core/lib/logstash/java_pipeline.rb:154:in
run'",
"/usr/local/Cellar/logstash/7.6.1/libexec/logstash-core/lib/logstash/java_pipeline.rb:109:in
`block in start'"],
"pipeline.sources"=>["/Users/user/Document/Esk-Data/xudaxia.conf"],
:thread=>"#"}
Below is the conf file
input
{
file{
path => ["/test.csv"]
start_position => "beginning"
}
}
filter{
csv{
separator => ","
columns => ["comment_time","comment", "id", "video_time"]
}
mutate{
convert => {
"comment_time" => "date_time"
"comment" => "string"
"id" => "integer"
"video_time" => "float"
}
}
}
output{
elasticsearch{
hosts => ["localhost:9200"]
index => "test"
}
}
test.csv
comment_time comment id video_time
2020/03/22 15:59:41 バイ a 123.100
2020/03/22 15:59:45 บาย b 100.100
2020/04/22 15:59:50 ByeBye c 80.210
can anyone help?

According the documentation the option date_time doesn't exist for convert action on mutate plugin - doc here.
However this plugin is used to cast a type into another one, that it isn't your use case. If comment_time is not recognized as date field you should trasform it with date plugin - doc here.
So you should remove this block:
mutate{
convert => {
"comment_time" => "date_time"
"comment" => "string"
"id" => "integer"
"video_time" => "float"
}
}
and replace it with this one:
date {
match => [ "comment_time", "yyyy/MM/dd HH:mm:ss"
}

Wildfly Logs are not saved

After restarting my Ubuntu VM, Wildfly 18 automatically starts.
From ps aux
wildfly 1031 0.0 0.0 20048 3508 ? Ss Dez18 0:00 /bin/bash /opt/wildfly/bin/launch.sh standalone standalone.xml 0.0.0.0
wildfly 1067 0.0 0.0 4628 1756 ? S Dez18 0:00 /bin/sh /opt/wildfly/bin/standalone.sh -c standalone.xml -b 0.0.0.0
wildfly 1482 35.2 7.0 1658040 572176 ? Sl Dez18 0:36 /opt/jdk-13.0.1/bin/java -D[Standalone] -server -Xms64m -Xmx512m -XX:MetaspaceSize=96M -XX:MaxMetaspaceSize=256m -Djava.net.preferIPv4Stack=true -Djboss.modules.system.pkgs
But my server.log is empty (cat /opt/wildfly/standalone/log/server.log gives me no messages about startup process of wildfly etc.).
When I make a "service wildfly restart" no additional Entries are inserted into server.log. How can I get access to my Log?
I think I`ve changed nothing in comparison to the standard config.
/standalone/configuration/logging.properties
loggers=sun.rmi,io.jaegertracing.Configuration,org.jboss.as.config,com.arjuna
logger.level=INFO
logger.handlers=FILE,CONSOLE
logger.sun.rmi.level=WARN
logger.sun.rmi.useParentHandlers=true
logger.io.jaegertracing.Configuration.level=WARN
logger.io.jaegertracing.Configuration.useParentHandlers=true
logger.org.jboss.as.config.level=DEBUG
logger.org.jboss.as.config.useParentHandlers=true
logger.com.arjuna.level=WARN
logger.com.arjuna.useParentHandlers=true
handler.CONSOLE=org.jboss.logmanager.handlers.ConsoleHandler
handler.CONSOLE.level=INFO
handler.CONSOLE.formatter=COLOR-PATTERN
handler.CONSOLE.properties=enabled,autoFlush,target
handler.CONSOLE.enabled=true
handler.CONSOLE.autoFlush=true
handler.CONSOLE.target=SYSTEM_OUT
handler.FILE=org.jboss.logmanager.handlers.PeriodicRotatingFileHandler
handler.FILE.level=ALL
handler.FILE.formatter=PATTERN
handler.FILE.properties=append,autoFlush,enabled,suffix,fileName
handler.FILE.append=true
handler.FILE.autoFlush=true
handler.FILE.enabled=true
handler.FILE.suffix=.yyyy-MM-dd
handler.FILE.fileName=/opt/wildfly/standalone/log/server.log
formatter.PATTERN=org.jboss.logmanager.formatters.PatternFormatter
formatter.PATTERN.properties=pattern
formatter.PATTERN.pattern=%d{yyyy-MM-dd HH\:mm\:ss,SSS} %-5p [%c] (%t) %s%e%n
formatter.COLOR-PATTERN=org.jboss.logmanager.formatters.PatternFormatter
formatter.COLOR-PATTERN.properties=pattern
formatter.COLOR-PATTERN.pattern=%K{level}%d{HH\:mm\:ss,SSS} %-5p [%c] (%t) %s%e%n
Output from standalone-cli
[standalone#localhost:9990 /] /subsystem=logging:read-resource(recursive=true)
{
"outcome" => "success",
"result" => {
"add-logging-api-dependencies" => true,
"use-deployment-logging-config" => true,
"async-handler" => undefined,
"console-handler" => {"CONSOLE" => {
"autoflush" => true,
"enabled" => true,
"encoding" => undefined,
"filter" => undefined,
"filter-spec" => undefined,
"formatter" => "%d{HH:mm:ss,SSS} %-5p [%c] (%t) %s%e%n",
"level" => "INFO",
"name" => "CONSOLE",
"named-formatter" => "COLOR-PATTERN",
"target" => "System.out"
}},
"custom-formatter" => undefined,
"custom-handler" => undefined,
"file-handler" => undefined,
"filter" => undefined,
"json-formatter" => undefined,
"log-file" => undefined,
"logger" => {
"com.arjuna" => {
"category" => "com.arjuna",
"filter" => undefined,
"filter-spec" => undefined,
"handlers" => undefined,
"level" => "WARN",
"use-parent-handlers" => true
},
"io.jaegertracing.Configuration" => {
"category" => "io.jaegertracing.Configuration",
"filter" => undefined,
"filter-spec" => undefined,
"handlers" => undefined,
"level" => "WARN",
"use-parent-handlers" => true
},
"org.jboss.as.config" => {
"category" => "org.jboss.as.config",
"filter" => undefined,
"filter-spec" => undefined,
"handlers" => undefined,
"level" => "DEBUG",
"use-parent-handlers" => true
},
"sun.rmi" => {
"category" => "sun.rmi",
"filter" => undefined,
"filter-spec" => undefined,
"handlers" => undefined,
"level" => "WARN",
"use-parent-handlers" => true
}
},
"logging-profile" => undefined,
"pattern-formatter" => {
"PATTERN" => {
"color-map" => undefined,
"pattern" => "%d{yyyy-MM-dd HH:mm:ss,SSS} %-5p [%c] (%t) %s%e%n"
},
"COLOR-PATTERN" => {
"color-map" => undefined,
"pattern" => "%K{level}%d{HH:mm:ss,SSS} %-5p [%c] (%t) %s%e%n"
}
},
"periodic-rotating-file-handler" => {"FILE" => {
"append" => true,
"autoflush" => true,
"enabled" => true,
"encoding" => undefined,
"file" => {
"relative-to" => "jboss.server.log.dir",
"path" => "server.log"
},
"filter" => undefined,
"filter-spec" => undefined,
"formatter" => "%d{HH:mm:ss,SSS} %-5p [%c] (%t) %s%e%n",
"level" => "ALL",
"name" => "FILE",
"named-formatter" => "PATTERN",
"suffix" => ".yyyy-MM-dd"
}},
"periodic-size-rotating-file-handler" => undefined,
"root-logger" => {"ROOT" => {
"filter" => undefined,
"filter-spec" => undefined,
"handlers" => [
"CONSOLE",
"FILE"
],
"level" => "INFO"
}},
"size-rotating-file-handler" => undefined,
"socket-handler" => undefined,
"syslog-handler" => undefined,
"xml-formatter" => undefined
}
}

My problem was that server.log has owner root because of prior testings ... just deleted it and now its re-created with owner wildfly.

How to get Wildfly server status via CLI?

I've been trying to validate if our server has started in Wildfly using the jboss-cli.bat
This is the command i'm using:
/host=slave-1/server-config=REST-server-one:read-resource(include-runtime=true)
and this is what i'm getting from the command
{
"outcome" => "success",
"result" => {
"auto-start" => true,
"cpu-affinity" => undefined,
"group" => "wildfly-server-group",
"name" => "wildfly-server",
"priority" => undefined,
"socket-binding-default-interface" => undefined,
"socket-binding-group" => undefined,
"socket-binding-port-offset" => 0,
"status" => "STARTED",
"update-auto-start-with-server-status" => false,
"interface" => undefined,
"jvm" => undefined,
"path" => undefined,
"ssl" => undefined,
"system-property" => undefined
}
Is there a command that will return the value of the status in that response?

You should be able to use the read-attribute operation.
/host=slave-1/server-config=REST-server-one:read-attribute(name=status)

I end up using this
BufferedReader reader = new BufferedReader(new InputStreamReader(p.getInputStream()));
String line = reader.readLine();
while (line != null) {
String[] value = line.split("=>");
if(value.length > 1){
if(value[0].contains("\"status\"")){
System.out.println(value[1]);
}
}
line = reader.readLine();
}
If anyone can suggest a better method would be greatly appreciated.

Send log4j2 stack traces over syslog

I am trying to log stack traces into Logstash.
The logging stack is ELK (ElasticSearch, Logstash, Kibana).
The application producing logs is a Java application, using slf4j as a logging interface, and log4j2 as the logging implementation.
The log4j2.xml declares this syslog Appender, with the RFC5424 format:
<Appenders>
<Syslog name="RFC5424" format="RFC5424" host="localhost" port="8514"
protocol="TCP" appName="MyApp" includeMDC="true" mdcId="mdc"
facility="LOCAL0" enterpriseNumber="18060" newLine="true"
messageId="Audit" id="App">
<LoggerFields>
<KeyValuePair key="thread" value="%t"/>
<KeyValuePair key="priority" value="%p"/>
<KeyValuePair key="category" value="%c"/>
<KeyValuePair key="exception" value="%ex{full}"/>
</LoggerFields>
</Syslog>
</Appenders>
I log a Throwable from the Java application like so:
org.slf4j.LoggerFactory.getLogger("exception_test").error("Testing errors", new RuntimeException("Exception message"));
When an exception is logged, Logstash traces something like this to show me what it persists:
{
"#timestamp":"2016-11-08T11:08:10.387Z",
"port":60397,
"#version":"1",
"host":"127.0.0.1",
"message":"<131>1 2016-11-08T11:08:10.386Z MyComputer.local MyApp - Audit [mdc#18060 category=\"exception_test\" exception=\"java.lang.RuntimeException: Exception message",
"type":"syslog",
"tags":[
"_grokparsefailure"
]
}
And I confirm that Kibana displays exactly the same JSON within the _source field of one of its log entries.
There's a problem here: no stack trace is saved. And the message, "Testing errors", is lost.
The "tags":["_grokparsefailure"] is unfortunate but not related to this question.
I tried adding <ExceptionPattern/> to see if it would change anything:
<Syslog name="RFC5424" format="RFC5424" host="localhost" port="8514"
protocol="TCP" appName="MyApp" includeMDC="true" mdcId="mdc"
facility="LOCAL0" enterpriseNumber="18060" newLine="true"
messageId="Audit" id="App">
<LoggerFields>
<KeyValuePair key="thread" value="%t"/>
<KeyValuePair key="priority" value="%p"/>
<KeyValuePair key="category" value="%c"/>
<KeyValuePair key="exception" value="%ex{full}"/>
</LoggerFields>
<ExceptionPattern>%ex{full}</ExceptionPattern>
</Syslog>
<ExceptionPattern/> replaces the log message, and also (sadly) omits all loggerFields. But it does give me a class name and line number:
{
"#timestamp":"2016-11-08T11:54:03.835Z",
"port":60397,
"#version":"1",
"host":"127.0.0.1",
"message":"at com.stackoverflow.LogTest.throw(LogTest.java:149)",
"type":"syslog",
"tags":[
"_grokparsefailure"
]
}
Again: no stack trace. And again: the message, "Testing errors", is lost.
How can I use log4j2 to log stack traces into Logstash? I don't necessarily have to use the syslog appender.
Essentially the constraints are:
Not be locked in to any particular logging infrastructure (this is why I used syslog)
Multi-line stack traces need to be understood as being a single log entry. It's undesirable for "each line of the stack trace" to be "a separate log message"
Stack traces must be able to be subjected to filters. A typical exception of mine can have a page-long stack trace. I want to filter out frames like Spring.

Log4j 2.5's SyslogAppender can only send stack traces over UDP.
<Syslog name="RFC5424" format="RFC5424" host="localhost" port="8514"
protocol="UDP" appName="MyApp" includeMDC="true" mdcId="mdc"
facility="LOCAL0" enterpriseNumber="18060" newLine="true"
messageId="LogTest" id="App">
<LoggerFields>
<KeyValuePair key="thread" value="%t"/>
<KeyValuePair key="priority" value="%p"/>
<KeyValuePair key="category" value="%c"/>
<KeyValuePair key="exception" value="%ex{full}"/>
</LoggerFields>
<ExceptionPattern>%ex{full}</ExceptionPattern>
</Syslog>
With UDP: both ExceptionPattern and LoggerFields.KeyValuePair["exception"] start working as solutions for multiline stack traces.
This is what logstash prints when I sent an exception over UDP via syslog:
{
"#timestamp" => 2016-11-14T13:23:38.304Z,
"#version" => "1",
"host" => "127.0.0.1",
"message" => "<131>1 2016-11-14T13:23:38.302Z BirchBox.local MyApp - LogTest [mdc#18060 category=\"com.stackoverflow.Deeply\" exception=\"java.lang.RuntimeException: Exception message\n\tat com.stackoverflow.Deeply.complain(Deeply.java:10)\n\tat com.stackoverflow.Nested.complain(Nested.java:8)\n\tat com.stackoverflow.Main.main(Main.java:20)\n\" priority=\"ERROR\" thread=\"main\"] Example error\njava.lang.RuntimeException: Exception message\n\tat com.stackoverflow.Deeply.complain(Deeply.java:10)\n\tat com.stackoverflow.Nested.complain(Nested.java:8)\n\tat com.stackoverflow.Main.main(Main.java:20)",
"type" => "syslog",
"tags" => [
[0] "_grokparsefailure"
]
}
Inside [mdc#18060 exception=\"…\"] we get the LoggerFields.KeyValuePair["exception"] stack trace.
In addition to this: the stack trace is inserted into the logged message itself, thanks to ExceptionPattern.
For reference: this is what logstash prints when I send the exception over TCP via syslog (i.e. the same SyslogAppender as described above, but with protocol="TCP" instead):
{
"#timestamp" => 2016-11-14T19:56:30.293Z,
"port" => 63179,
"#version" => "1",
"host" => "127.0.0.1",
"message" => "<131>1 2016-11-14T19:56:30.277Z BirchBox.local MyApp - Audit [mdc#18060 category=\"com.stackoverflow.Deeply\" exception=\"java.lang.RuntimeException: Exception message",
"type" => "syslog",
"tags" => [
[0] "_grokparsefailure"
]
}
{
"#timestamp" => 2016-11-14T19:56:30.296Z,
"port" => 63179,
"#version" => "1",
"host" => "127.0.0.1",
"message" => "at com.stackoverflow.Deeply.complain(Deeply.java:10)",
"type" => "syslog",
"tags" => [
[0] "_grokparsefailure"
]
}
{
"#timestamp" => 2016-11-14T19:56:30.296Z,
"port" => 63179,
"#version" => "1",
"host" => "127.0.0.1",
"message" => "at com.stackoverflow.Nested.complain(Nested.java:8)",
"type" => "syslog",
"tags" => [
[0] "_grokparsefailure"
]
}
{
"#timestamp" => 2016-11-14T19:56:30.296Z,
"port" => 63179,
"#version" => "1",
"host" => "127.0.0.1",
"message" => "at com.stackoverflow.Main.main(Main.java:20)",
"type" => "syslog",
"tags" => [
[0] "_grokparsefailure"
]
}
{
"#timestamp" => 2016-11-14T19:56:30.296Z,
"port" => 63179,
"#version" => "1",
"host" => "127.0.0.1",
"message" => "\" priority=\"ERROR\" thread=\"main\"] Example error",
"type" => "syslog",
"tags" => [
[0] "_grokparsefailure"
]
}
{
"#timestamp" => 2016-11-14T19:56:30.296Z,
"port" => 63179,
"#version" => "1",
"host" => "127.0.0.1",
"message" => "java.lang.RuntimeException: Exception message",
"type" => "syslog",
"tags" => [
[0] "_grokparsefailure"
]
}
{
"#timestamp" => 2016-11-14T19:56:30.297Z,
"port" => 63179,
"#version" => "1",
"host" => "127.0.0.1",
"message" => "at com.stackoverflow.Deeply.complain(Deeply.java:10)",
"type" => "syslog",
"tags" => [
[0] "_grokparsefailure"
]
}
{
"#timestamp" => 2016-11-14T19:56:30.298Z,
"port" => 63179,
"#version" => "1",
"host" => "127.0.0.1",
"message" => "at com.stackoverflow.Nested.complain(Nested.java:8)",
"type" => "syslog",
"tags" => [
[0] "_grokparsefailure"
]
}
{
"#timestamp" => 2016-11-14T19:56:30.298Z,
"port" => 63179,
"#version" => "1",
"host" => "127.0.0.1",
"message" => "at com.stackoverflow.Main.main(Main.java:20)",
"type" => "syslog",
"tags" => [
[0] "_grokparsefailure"
]
}
{
"#timestamp" => 2016-11-14T19:56:30.299Z,
"port" => 63179,
"#version" => "1",
"host" => "127.0.0.1",
"message" => "",
"type" => "syslog",
"tags" => [
[0] "_grokparsefailure"
]
}
It looks like TCP does actually "work", but splits the single log message into many syslog messages (for example when \n is encountered).