We Keep Coding

Splunk HTTP Event Collector Log4j2 SpringBoot unable to send data

I tried integrating my SpringBoot Application with Splunk, but could not get the logs. There is no Error in Console. Kindly help
<?xml version="1.0" encoding="UTF-8"?>
<Configuration>
<Appenders>
<Console name="console" target="SYSTEM_OUT">
<PatternLayout
pattern="%style{%d{ISO8601}} %highlight{%-5level }[%style{%t}{bright,blue}] %style{%C{10}}{bright,yellow}: %msg%n%throwable" />
</Console>
<SplunkHttp
name="student-service-token"
url="https://prd-p-wyalk.splunkcloud.com:8080"
token="*********************************"
host="localhost"
index="student_idex_api"
type="raw"
source="http-event-logs"
sourcetype="log4j"
messageFormat="text"
disableCertificateValidation="true">
<PatternLayout pattern="%m" />
</SplunkHttp>
</Appenders>
<Loggers>
<!-- LOG everything at INFO level -->
<Root level="info">
<AppenderRef ref="console" />
<AppenderRef ref="splunkhttp" />
</Root>
</Loggers>
</Configuration>

Log4j2 custom Filter xml to property file

Created the custom filter by extending AbstractFilter.
Created log4j2.xml as per below code
<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="WARN" monitorInterval="30"
packages="com.smp.marker">
<Properties>
<Property name="DEFAULT_LOG_PATTERN">%d %-5p %m%n [:%L]</Property>
</Properties>
<Appenders>
<Console name="CLOG" target="SYSTEM_OUT">
<PatternLayout alwaysWriteExceptions="false" pattern="%d %-5p
[%c{5}.%M():%L] %m%n" />
<Filters>
<SmpClogAppenderFilter level="ALL" onMatch="ACCEPT"
onMismatch="DENY"/>
</Filters>
</Console>
<Console name="DEFAULT" target="SYSTEM_OUT">
<PatternLayout alwaysWriteExceptions="false"
pattern="${DEFAULT_LOG_PATTERN}" />
<Filters>
<SmpClogAppenderFilter level="ALL" onMatch="DENY"
onMismatch="ACCEPT"/>
</Filters>
</Console>
</Appenders>
<Loggers>
<Root level="info">
<AppenderRef ref="CLOG" />
<AppenderRef ref="DEFAULT" />
</Root>
</Loggers>
</Configuration>
This is working as expetced.
But in our project we need log4j2.properties.
Unable to create the property file with custom filter.Mainly need propety value for filter section.
<Filters>
<SmpClogAppenderFilter level="ALL" onMatch="ACCEPT"
onMismatch="DENY"/>
Could some one please help me to resolve this issue.

How to save logs into file with timestamp with log4j2

i have a little problem. I had to change my logging implementation from org.apache.commons.logging to org.apache.logging.log4j. I saved my logs into a "log" directory and each file got a appended timestamp.
I used the following .xml file:
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<include resource="org/springframework/boot/logging/logback/base.xml"/>
<logger name="org.springframework.web" level="INFO"/>
<timestamp key="myTimestamp" timeReference="contextBirth" datePattern="yyyy-MM-dd'_'HH-mm-ss"/>
<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<pattern>%d{HH:mm:ss.SSS} - %msg%n</pattern>
</encoder>
</appender>
<appender name="FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>/logs/app-${myTimestamp}.log</file>
<encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
<Pattern>%d{yyyy-MM-dd_HH:mm:ss} - %msg%n</Pattern>
</encoder>
<rollingPolicy class="ch.qos.logback.core.rolling.FixedWindowRollingPolicy">
<FileNamePattern>app.%i{yyyy-MM-dd_HH:mm:ss.SSS}}.log</FileNamePattern>
</rollingPolicy>
<triggeringPolicy class="ch.qos.logback.core.rolling.SizeBasedTriggeringPolicy">
<MaxFileSize>30MB</MaxFileSize>
</triggeringPolicy>
</appender>
<logger name="com....." level="INFO" additivity="false">
<appender-ref ref="STDOUT" />
<appender-ref ref="FILE" />
</logger>
<root level="INFO">
<appender-ref ref="STDOUT" />
</root>
</configuration>
Now i wanted to extend my xml file with log4j2, because of some new environments. But i also want to save it in the "logs" directory again. I tried different methods via the documentation and http://www.baeldung.com, but i don't get. My application always writes in the my app.log again without timestamp. How can i realize it? Where is the mistake in my new .xml ?
I thought with this
filePattern="logs/ldapmicroservice.%d{MM-dd-yyyy-HH-mm}.log.gz"
i define the new pattern of my file?
This is my new .xml:
<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="WARN" monitorInterval="30"
packages="com......">
<Properties>
<Property name="LOG_PATTERN">
%d{yyyy-MM-dd HH:mm:ss.SSS} %5p %X{mdcData} --- %-40.40c{1.} : %m%n%ex
</Property>
</Properties>
<Appenders>
<RollingFile
name="FILE" fileName="/logs/app.log"
filePattern="logs/app.%d{MM-dd-yyyy-HH-mm}.log.gz"
ignoreExceptions="false">
<PatternLayout>
<Pattern>%d{yyyy-MM-dd HH:mm:ss} %p %m%n</Pattern>
</PatternLayout>
<TimeBasedTriggeringPolicy />
</RollingFile>
<Console name="ConsoleAppender" target="SYSTEM_OUT" follow="true">
<PatternLayout pattern="${LOG_PATTERN}" />
</Console>
<CloudantAppender name="CloudantAppender"
database="logs"
......
<PatternLayout pattern="${LOG_PATTERN}"/>
</CloudantAppender>
</Appenders>
<Loggers>
<Logger name="com...." level="debug" additivity="false">
<AppenderRef ref="FILE" />
<AppenderRef ref="ConsoleAppender" />
<AppenderRef ref="CloudantAppender" />
</Logger>
<Root level="info">
<AppenderRef ref="FILE" />
<AppenderRef ref="ConsoleAppender" />
<AppenderRef ref="CloudantAppender" />
</Root>
</Loggers>
</Configuration>

I found the solution on my own:
<RollingFile
name="FILE" fileName="/logs/app-${date:yyyy-MM-dd'_'HH-mm-ss}.log"
filePattern="logs/app-%d{yyyy-MM-dd}-%i.log"
ignoreExceptions="false">
<PatternLayout>
<Pattern>%d{yyyy-MM-dd'_'HH-mm-ss} %p %m%n</Pattern>
</PatternLayout>
<TimeBasedTriggeringPolicy />
</RollingFile>

Change log4j2 log level from string to int

within the log4j2 configuration I am trying to convert the loglevels from string to an int.
Following the directions located in log4j2 Patterns section
image snippet of example log4j2 pattern mapping
Following the example above I created in log4j2.xml config:
<JMS name="jmsQueue"
destinationBindingName="${sys:env}.logging"
factoryName="org.apache.activemq.jndi.ActiveMQInitialContextFactory"
factoryBindingName="ConnectionFactory"
providerURL="${sys:log4j.providerurl}"
userName="log"
password="log">
<PatternLayout pattern='{"Message":"%m","LogLevel":%level{Debug=1,Info=2,Warn=3,Error=4,Fatal=5,Trace=6},"Type":"middleware","App":"${app_name}","Env":"${sys:env}","data":{"Event_Time":"%d{ISO8601}","Thread":"%t","Class":"%c"}}'/>
</JMS>
The edited PatternLayout I would now expect would replace the string debug,info,warn etc. with the mapped int values of 1,2,3 . .
This is not the case, I still receive string value. If someone could suggest where I am making the mistake I would appreciate it. thank you.
edit:
<?xml version="1.0" encoding="utf-8"?>
<Configuration>
<!-- replace > app_name_here_only < with the distinct name of your service. you will use this name to search for within kibana (ex: type:middleware app:<app_name>) -->
<Properties>
<Property name="app_name">middleware_united</Property>
</Properties>
<!-- do not configure below this line -->
<Appenders>
<RollingFile name="file" fileName="${sys:mule.home}${sys:file.separator}logs${sys:file.separator}${app_name}.log" filePattern="${sys:mule.home}${sys:file.separator}logs${sys:file.separator}${app_name}-%i.log">
<PatternLayout pattern="%d [%t] %-5p %c - %m%n"/>
<SizeBasedTriggeringPolicy size="10 MB"/>
<DefaultRolloverStrategy max="10"/>
</RollingFile>
<JMS name="jmsQueue" destinationBindingName="${sys:env}.logging" factoryName="org.apache.activemq.jndi.ActiveMQInitialContextFactory" factoryBindingName="ConnectionFactory" providerURL="${sys:log4j.providerurl}" userName="log" password="log">
<PatternLayout pattern='{"Type":"middleware","Env":"${sys:env}","App":"${app_name}","LogLevel":%level{Debug=1,Info=2,Warn=3,Error=4,Fatal=5,Trace=6},"Message":"%m","data":{"Event_Time":"%d{ISO8601}","Thread":"%t","Class":"%c"}}'/>
</JMS>
</Appenders>
<Loggers>
<!-- CXF is used heavily by Mule for web services -->
<AsyncLogger name="org.apache.cxf" level="WARN"/>
<!-- Apache Commons tend to make a lot of noise which can clutter the log-->
<AsyncLogger name="org.apache" level="WARN"/>
<!-- Reduce startup noise -->
<AsyncLogger name="org.springframework.beans.factory" level="WARN"/>
<!-- Mule classes -->
<AsyncLogger name="org.mule" level="INFO"/>
<AsyncLogger name="com.mulesoft" level="INFO"/>
<!-- Reduce DM verbosity -->
<AsyncLogger name="org.jetel" level="WARN"/>
<AsyncLogger name="Tracking" level="WARN"/>
<AsyncRoot level="INFO">
<AppenderRef ref="file"/>
<AppenderRef ref="jmsQueue"/>
</AsyncRoot>
</Loggers>
</Configuration>

I've tested your pattern but it worked well.
Here's my log4j2.xml
<?xml version="1.0" encoding="UTF-8"?>
<Configuration>
<Appenders>
<Console name="STDOUT" target="SYSTEM_OUT">
<PatternLayout
pattern='{"Type":"middleware","Env":"${sys:env}","App":"${app_name}","LogLevel":%level{Debug=1,Info=2,Warn=3,Error=4,Fatal=5,Trace=6},"Message":"%m","data":{"Event_Time":"%d{ISO8601}","Thread":"%t","Class":"%c"}}'/>
</Console>
</Appenders>
<Loggers>
<Root level="DEBUG">
<AppenderRef ref="STDOUT"/>
</Root>
</Loggers>
</Configuration>
And here's the output:
{"Type":"middleware","Env":"${sys:env}","App":"${app_name}","LogLevel":2,"Message":"b","data":{"Event_Time":"2017-03-29T11:28:14,477","Thread":"main","Class":"test.Log4j2"}}

Warnings while running an application

I configured log4j ,and I tested it on an application creating log files for threads but I am getting log4j warnings while it is running and log files are not being created for threads.
log4j:WARN No appenders could be found for logger(com.amazonaws.internal.config.InternalConfig).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN
See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="WARN">
<properties>
<property name="logMsgPattern">%d{HH:mm:ss} %-5level - %msg%n</property>
<property name="logDir">logs</property><!-- ${sys:testLogDir} -->
</properties>
<Appenders>
<Console name="Console" target="SYSTEM_OUT">
<PatternLayout pattern="${logMsgPattern}" />
</Console>
<Routing name="Routing">
<Routes pattern="$${ctx:threadId}">
<Route>
<RollingFile name="RollingFile-${ctx:threadId}"
fileName="${logDir}/${ctx:threadId}.log"
filePattern="${logDir}/%d{yyyy-MM-dd}/${ctx:threadId}-%d{yyyy-MM-dd}.log">
<PatternLayout pattern="${logMsgPattern}" />
<Policies>
<TimeBasedTriggeringPolicy interval="1" modulate="true"/>
</Policies>
</RollingFile>
</Route>
</Routes>
</Routing>
</Appenders>
<Loggers>
<Root level="error">
<AppenderRef ref="Console" />
</Root>
<logger name="test" level="INFO" />
<Root level="INFO">
<AppenderRef ref="Console" />
<AppenderRef ref="Routing" />
</Root>
</Loggers>
</Configuration>

I tried with logback instead of log4j and its solved. i can create multiple log files for multiple threads.
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<statusListener class="ch.qos.logback.core.status.NopStatusListener" />
<property name="USER_HOME" value="\logs" />
<appender name="FILE-THREAD" class="ch.qos.logback.classic.sift.SiftingAppender">
<!-- This is MDC value -->
<!-- We will assign a value to 'logFileName' via Java code -->
<discriminator>
<key>logFileName</key>
<defaultValue>head0</defaultValue>
</discriminator>
<sift>
<!-- A standard RollingFileAppender, the log file is based on 'logFileName'
at runtime -->
<appender name="FILE-${logFileName}"
class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${USER_HOME}/${logFileName}.log</file>
<encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
<Pattern>
%d{yyyy-MM-dd HH:mm:ss} %mdc [%thread] %level %logger{35}
- %msg%n
</Pattern>
</encoder>
<rollingPolicy
class="ch.qos.logback.core.rolling.FixedWindowRollingPolicy">
<FileNamePattern>${USER_HOME}/${logFileName}.%i.log.zip
</FileNamePattern>
<MinIndex>1</MinIndex>
<MaxIndex>10</MaxIndex>
</rollingPolicy>
<triggeringPolicy
class="ch.qos.logback.core.rolling.SizeBasedTriggeringPolicy">
<MaxFileSize>10MB</MaxFileSize>
</triggeringPolicy>
</appender>
</sift>
</appender>