We Keep Coding

Apache Camel library will holding memory for long time?

I am using the verbosegc to capture some data and try to analyze the memory usage of my application.
I have a module that will pulling data from database or third party and put it into a list object then only return to front end for display.
When I choose the date to be date range, it will pull the data from database.
When I choose the date to be today date, then my application will send a request to MQ server, and the MQ server will response my application with xml message. The I will use Apache camel library to handle it.
Here is my verbosegc screen shot when pulling data from database:
As you can see, everytime when I trigger the search function, the memory usage will increase, and then drop back. So this is normal, and also what I expected.
And this is the verbosegc screen shot when pulling data from third party.
As you can see, after the memory increase, it will will horizontal there for a period, and then only drop back.
I suspect that the org.apache.camel.Exchange or org.apache.camel.Message or those object in Apache will holding the memory for longer time.
Here is some of my code to handle the xml message from third party:
/**
* Camel Exchange producer template
*/
protected ProducerTemplate< Exchange > template;
#SuppressWarnings("unchecked")
private < T > T doSend(final Object request, final String headerName,
final Object headerObject,
final SendEaiMessageTemplateCallBack callback)
throws BaseRuntimeException {
log.debug( "doSend START >> {} ", request );
if ( this.requestObjectValidator != null
&& requestObjectValidator
.requiredValidation( requestObjectValidator ) ) {
requestObjectValidator.validateRequest( request );
}
final Exchange exchange = template.request( to, new Processor( ) {
public void process(final Exchange exchange) throws Exception {
exchange.getIn( ).setBody( request );
if ( headerName != null && headerObject != null ) {
exchange.getIn( ).setHeader( headerName, headerObject );
}
}
} );
log.debug( "doSend >> END >> exchange is failed? {}",
exchange.isFailed( ) );
Message outBoundMessage = null;
if ( callback != null ) {
// provide the callBack method to access exchange
callback.exchangeCallBack( exchange );
}
if ( exchange.isFailed( ) ) {
failedHandler.handleExchangeFailed( exchange, request );
} else {
outBoundMessage = exchange.getOut( false );
}
// handler outbound message
if ( this.outboundMessageHandler != null ) {
this.outboundMessageHandler.handleMessage( outBoundMessage );
}
if ( outBoundMessage != null ) {
if ( outBoundMessage.getBody( ) != null ) {
log.debug( "OutBoundMessage body {}", outBoundMessage.getBody( ) );
}
return (T) outBoundMessage.getBody( );
} else {
return null;
}
}
Because of this, my application was hitting Out Of Memory Exception. I am not sure is it because of Apache Camel library or not, kindly advise.
Other than that, when I open the heapdump file, there is 52% complain on the com/ibm/xml/xlxp2/scan/util/SimpleDataBufferFactory$DataBufferLink
And the other are complain on the "Java heap is used by this char[] alone", which is some sub category under DataBufferLink as well.
I google on this, all is talking about the xml message too large.
I have no idea on which way or which direction I should continue to troubleshoot, can kindly advise on this?
FYI, I am using camel-core-1.5.0.jar

hibernate optimistic lock mechanism

I am so curious about the hibernate optimistic lock (dedicated version way), I checked hibernate source code which tells that it checks version before current transaction commits, but if there is another transaction happens to committed after it query the version column from DB(in a very short time gap), then current transaction considers there is no change, so the old transaction would be replaced wrongly.
EntityVerifyVersionProcess.java
#Override
public void doBeforeTransactionCompletion(SessionImplementor session) {
final EntityPersister persister = entry.getPersister();
if ( !entry.isExistsInDatabase() ) {
// HHH-9419: We cannot check for a version of an entry we ourselves deleted
return;
}
final Object latestVersion = persister.getCurrentVersion( entry.getId(), session );
if ( !entry.getVersion().equals( latestVersion ) ) {
throw new OptimisticLockException(
object,
"Newer version [" + latestVersion +
"] of entity [" + MessageHelper.infoString( entry.getEntityName(), entry.getId() ) +
"] found in database"
);
}
}
is such case possible?
Hope there are DB domain experts who would help me on this.
Many thanks.

Based on a quick glance of the code, EntityVerifyVersionProcess is used for read transactions, so there's no potential for data loss involved. This would only check that when the transaction commits, it's not returning data that's already stale. With a READ COMMITTED transaction, I suppose this might return data that is instantly going stale, but hard to say without going into details.
Write transactions on the other hand use EntityIncrementVersionProcess, which is a completely different beast and leaves no chance for race conditions.
public void doBeforeTransactionCompletion(SessionImplementor session) {
final EntityPersister persister = entry.getPersister();
final Object nextVersion = persister.forceVersionIncrement( entry.getId(), entry.getVersion(), session );
entry.forceLocked( object, nextVersion );
}

More info about save in GORM

I have a simple block of code, that return false
PromoDebit promoDebit = new PromoDebit();
promoDebit.promoCode=promoCode;
promoDebit.userId=userId;
promoDebit.countUsages=countUsages;
promoDebit.endDate=endDate;
promoDebit.startDate=startDate;
promoDebit.status=1;
promoDebit.calcValue=Float.parseFloat(p.getProperty("promoPercent"));
if(promoDebit.save(flush: true)){
log.info "GOOD!"
} else {
log.info "BAD!"
}
How can I get more info about GORM.save fail? Stdrout log show nothning even if I turn on logSql in DataSource.groovy

First ,make sure that your Log4j is configured correctly.
Here http://grails.org/doc/2.3.1/guide/conf.html#logging
And ,If you wan to know more detail info on GORM.save fail like
PromoDebit promoDebit = new PromoDebit();
promoDebit.promoCode=promoCode;
promoDebit.userId=userId;
promoDebit.countUsages=countUsages;
promoDebit.endDate=endDate;
promoDebit.startDate=startDate;
promoDebit.status=1;
promoDebit.calcValue=Float.parseFloat(p.getProperty("promoPercent"));
Then ,
if (!promoDebit.save()) {
log.warn myDomainObj.errors.allErrors.join(' \n')
//each error is an instance of org.springframework.validation.FieldError
}
And , I like this one
if (!promoDebit.save()) {
promoDebit.errors.each {
println it
}
}
If you want to throw an exception for EVERY domain classes, then simply set grails.gorm.failOnError to true in grails-app/conf/Config.groovy
or Simply
promoDebit.save(failOnError:true)
Cheers!

log.warn "error occurred by saving: $promoDebit.errors"
you will see the validation failures in the log. If some SQL-constraint gets broken you will get a full-blown Exception like DataIntegrityException

Get tomcat log file programmatically within webapp

I'm looking for a way to retrieve the Tomcat log within a webapp. In the past I've seen this feature provided in other webapps, usually dumping the log in a Servlet.
I'm using slf4j (with log4j) and Tomcat 6. I haven't found anything relevant in the Tomcat docs, although the JMX API looks like it might provide something useful? I'm not too concerned whether the output represents just the webapp logging or the entire Tomcat log, either will suffice.
Ideally, I'm hoping for a solution that does not involve scraping the log from the filesystem, although if that is the only way, it would be great if the log directory could be calculated at runtime...

Scraping the log from the filesystem is probably the easiest way to go. You can get the log directly programatically using System.getProperty("catalina.base") + "/logs".
Otherwise you could set up an additional appender in your log4j configuration to log to something like JDBC, JMS, a Writer, etc. Whatever makes sense for your app.

This function will get the most recent log file matching a given prefix. You do not need to know what directory the logs are written to.
public static File locateLogFile( final String prefixToMatch ) {
File result = null;
Handler[] handlers = LogManager.getLogManager().getLogger( "" ).getHandlers();
try {
for( Handler handler : handlers ) {
Field directoryField;
Field prefixField;
try {
//These are private fields in the juli FileHandler class
directoryField = handler.getClass().getDeclaredField( "directory" );
prefixField = handler.getClass().getDeclaredField( "prefix" );
directoryField.setAccessible( true );
prefixField.setAccessible( true );
} catch( NoSuchFieldException e ) {
continue;
}
String directory = (String)directoryField.get( handler );
if( prefixToMatch.equals( prefixField.get( handler ) ) ) {
File logDirectory = new File( directory );
File[] logFiles = logDirectory.listFiles( new FileFilter() {
public boolean accept( File pathname ) {
return pathname.getName().startsWith( prefixToMatch );
}
} );
if( logFiles.length == 0 ) continue;
Arrays.sort( logFiles );
result = logFiles[ logFiles.length - 1 ];
break;
}
}
} catch( IllegalAccessException e ) {
log.log( Level.WARNING, "Couldn't get log file", e );
}
return result;
}

How do I access Windows Event Viewer log data from Java

Is there any way to access the Windows Event Log from a java class. Has anyone written any APIs for this, and would there be any way to access the data from a remote machine?
The scenario is:
I run a process on a remote machine, from a controlling Java process.
This remote process logs stuff to the Event Log, which I want to be able to see in the controlling process.
Thanks in advance.

http://www.j-interop.org/ is an open-source Java library that implements the DCOM protocol specification without using any native code. (i.e. you can use it to access DCOM objects on a remote Windows host from Java code running on a non-Windows client).
Microsoft exposes a plethora of system information via Windows Management Instrumentation (WMI). WMI is remotely accessible via DCOM, and considerable documentation on the subject exists on Microsoft's site. As it happens, you can access the Windows Event Logs via this remotely accessible interface.
By using j-interop you can create an instance of the WbemScripting.SWbemLocator WMI object remotely, then connect to Windows Management Instrumentation (WMI) services on the remote Windows host. From there you can submit a query that will inform you whenever a new event log entry is written.
Note that this does require that you have DCOM properly enabled and configured on the remote Windows host, and that appropriate exceptions have been set up in any firewalls. Details on this can be searched online, and are also referenced from the j-interop site, above.
The following example connects to a remote host using its NT domain, hostname, a username and a password, and sits in a loop, dumping every event log entry as they are logged by windows. The user must have been granted appropriate remote DCOM access permissions, but does not have to be an administrator.
import java.io.IOException;
import java.util.logging.Level;
import org.jinterop.dcom.common.JIException;
import org.jinterop.dcom.common.JISystem;
import org.jinterop.dcom.core.JIComServer;
import org.jinterop.dcom.core.JIProgId;
import org.jinterop.dcom.core.JISession;
import org.jinterop.dcom.core.JIString;
import org.jinterop.dcom.core.JIVariant;
import org.jinterop.dcom.impls.JIObjectFactory;
import org.jinterop.dcom.impls.automation.IJIDispatch;
public class EventLogListener
{
private static final String WMI_DEFAULT_NAMESPACE = "ROOT\\CIMV2";
private static JISession configAndConnectDCom( String domain, String user, String pass ) throws Exception
{
JISystem.getLogger().setLevel( Level.OFF );
try
{
JISystem.setInBuiltLogHandler( false );
}
catch ( IOException ignored )
{
;
}
JISystem.setAutoRegisteration( true );
JISession dcomSession = JISession.createSession( domain, user, pass );
dcomSession.useSessionSecurity( true );
return dcomSession;
}
private static IJIDispatch getWmiLocator( String host, JISession dcomSession ) throws Exception
{
JIComServer wbemLocatorComObj = new JIComServer( JIProgId.valueOf( "WbemScripting.SWbemLocator" ), host, dcomSession );
return (IJIDispatch) JIObjectFactory.narrowObject( wbemLocatorComObj.createInstance().queryInterface( IJIDispatch.IID ) );
}
private static IJIDispatch toIDispatch( JIVariant comObjectAsVariant ) throws JIException
{
return (IJIDispatch) JIObjectFactory.narrowObject( comObjectAsVariant.getObjectAsComObject() );
}
public static void main( String[] args )
{
if ( args.length != 4 )
{
System.out.println( "Usage: " + EventLogListener.class.getSimpleName() + " domain host username password" );
return;
}
String domain = args[ 0 ];
String host = args[ 1 ];
String user = args[ 2 ];
String pass = args[ 3 ];
JISession dcomSession = null;
try
{
// Connect to DCOM on the remote system, and create an instance of the WbemScripting.SWbemLocator object to talk to WMI.
dcomSession = configAndConnectDCom( domain, user, pass );
IJIDispatch wbemLocator = getWmiLocator( host, dcomSession );
// Invoke the "ConnectServer" method on the SWbemLocator object via it's IDispatch COM pointer. We will connect to
// the default ROOT\CIMV2 namespace. This will result in us having a reference to a "SWbemServices" object.
JIVariant results[] =
wbemLocator.callMethodA( "ConnectServer", new Object[] { new JIString( host ), new JIString( WMI_DEFAULT_NAMESPACE ),
JIVariant.OPTIONAL_PARAM(), JIVariant.OPTIONAL_PARAM(), JIVariant.OPTIONAL_PARAM(), JIVariant.OPTIONAL_PARAM(), new Integer( 0 ),
JIVariant.OPTIONAL_PARAM() } );
IJIDispatch wbemServices = toIDispatch( results[ 0 ] );
// Now that we have a SWbemServices DCOM object reference, we prepare a WMI Query Language (WQL) request to be informed whenever a
// new instance of the "Win32_NTLogEvent" WMI class is created on the remote host. This is submitted to the remote host via the
// "ExecNotificationQuery" method on SWbemServices. This gives us all events as they come in. Refer to WQL documentation to
// learn how to restrict the query if you want a narrower focus.
final String QUERY_FOR_ALL_LOG_EVENTS = "SELECT * FROM __InstanceCreationEvent WHERE TargetInstance ISA 'Win32_NTLogEvent'";
final int RETURN_IMMEDIATE = 16;
final int FORWARD_ONLY = 32;
JIVariant[] eventSourceSet =
wbemServices.callMethodA( "ExecNotificationQuery", new Object[] { new JIString( QUERY_FOR_ALL_LOG_EVENTS ), new JIString( "WQL" ),
new JIVariant( new Integer( RETURN_IMMEDIATE + FORWARD_ONLY ) ) } );
IJIDispatch wbemEventSource = (IJIDispatch) JIObjectFactory.narrowObject( ( eventSourceSet[ 0 ] ).getObjectAsComObject() );
// The result of the query is a SWbemEventSource object. This object exposes a method that we can call in a loop to retrieve the
// next Windows Event Log entry whenever it is created. This "NextEvent" operation will block until we are given an event.
// Note that you can specify timeouts, see the Microsoft documentation for more details.
while ( true )
{
// this blocks until an event log entry appears.
JIVariant eventAsVariant = (JIVariant) ( wbemEventSource.callMethodA( "NextEvent", new Object[] { JIVariant.OPTIONAL_PARAM() } ) )[ 0 ];
IJIDispatch wbemEvent = toIDispatch( eventAsVariant );
// WMI gives us events as SWbemObject instances (a base class of any WMI object). We know in our case we asked for a specific object
// type, so we will go ahead and invoke methods supported by that Win32_NTLogEvent class via the wbemEvent IDispatch pointer.
// In this case, we simply call the "GetObjectText_" method that returns us the entire object as a CIM formatted string. We could,
// however, ask the object for its property values via wbemEvent.get("PropertyName"). See the j-interop documentation and examples
// for how to query COM properties.
JIVariant objTextAsVariant = (JIVariant) ( wbemEvent.callMethodA( "GetObjectText_", new Object[] { new Integer( 1 ) } ) )[ 0 ];
String asText = objTextAsVariant.getObjectAsString().getString();
System.out.println( asText );
}
}
catch ( Exception e )
{
e.printStackTrace();
}
finally
{
if ( null != dcomSession )
{
try
{
JISession.destroySession( dcomSession );
}
catch ( Exception ex )
{
ex.printStackTrace();
}
}
}
}
}
~

On the Java side, you'll need a library that allows you to make native calls. Sun offers JNI, but it sounds like sort of a pain. Also consider:
https://github.com/twall/jna/
http://johannburkard.de/software/nativecall/
http://www.jinvoke.com/
On the Windows side, the function you're after is OpenEventLog. This should allow you to access a remote event log. See also Querying for Event Information.
If that doesn't sound right, I also found this for parsing the log files directly (not an approach I'd recommend but interesting nonetheless):
http://msdn.microsoft.com/en-us/library/bb309026.aspx
http://objectmix.com/java/75154-regarding-windows-event-log-file-parser-java.html

Read this article.
JNA 3.2.8 has both methods to read and write from the Windows event log.
You can see an example of write in log4jna.
Here's an example of read:
EventLogIterator iter = new EventLogIterator("Application");
while(iter.hasNext()) {
EventLogRecord record = iter.next();
System.out.println(record.getRecordNumber()
+ ": Event ID: " + record.getEventId()
+ ", Event Type: " + record.getType()
+ ", Event Source: " + record.getSource());
}

If you want true event log access from a remote machine, you will have to find a library which implements the EventLog Remoting Protocol Specification. Unfortunately, I have not yet found any such library in Java. However, much of the foundation for implementing this protocol has already been laid by the JCIFS and JARAPAC projects. The protocol itself (if I'm not mistaken) runs on top of the DCE/RPC protocol (implemented by JARAPAC) which itself runs on top of the SMB protocol (implemented by JCIFS).
I have already been using JCIFS and JARAPAC to implement some of EventLog's cousin protocols, such as remote registry access. I may be blind, but documentation seemed a little scarce regarding JARAPAC. If you are interested in implementing this, I can share with you what I have learned when I get some spare time!
Later!

there are a million (and one) options here ;)
you could look at sigar
http://cpansearch.perl.org/src/DOUGM/hyperic-sigar-1.6.3-src/docs/javadoc/org/hyperic/sigar/win32/EventLog.html
mind the licensing though....
or you could be quick and dirty and just periodically execute (and capture the output)
D:>cscript.exe c:\WINDOWS\system32\eventquery.vbs /v
then use event filtering params to refine the results etc...
http://technet.microsoft.com/en-us/library/cc772995(WS.10).aspx