I have a large number of Java bean classes in my web application, and I am trying to find a simple way to implement the toString() methods in these beans. The toString() method would be used for logging throughout the application, and should print the attribute-value pairs of all attributes in the bean.
I am trying out two alternatives:
1. BeanUtils.describe() (Apache commons-beanutils)
2. ReflectionToStringBuilder.toString() (Apache commons-lang)
Since this is a web application expected to have high traffic, the implementation has to be lightweight and should not impact performance. (Memory use, processor use, etc are main considerations).
I'd like to know which of these performs better according the criteria mentioned above. As far as I know, reflection is a heavy operation, but more details and insight into both these options would help me choose the optimal solution.
We use ToStringBuilder.reflectionToString() in our objects' toString() methods. We have not had any issues running like this in a production environment. Granted, we rarely use the toString() method.
We also use BeanUtils.describe(), but for another purpose. BeanUtils uses PropertyUtilsBean which keeps an internal cache of beans for which it has performed introspection. It would seem that this would give it a performance advantage over the other, but a little poking around in the reflectionToString source and it seems that since it ultimately relies on the implementation of java.lang.Class, caching comes into play there as well.
Either looks like a viable choice, but BeanUtils.describe() will return a Map of properties where reflectionToString will return a formatted String. I guess it depends on what you want to do with the output.
I would suggest that if your application is heavily dependent on calling toString() on your objects, having a specific implementation might be more beneficial.
Personally, I prefer to generate the toString() method using Eclipse/IntelliJ and then modify it as necessary (only include important fields).
Right click -> Source -> Generate toString(). Select fields. Done.
It takes less time than even writing the Builder code.
It will execute faster.
It doesn't use permgen space (reflection can tend to eat up permgen)
That's the path I would go if you're concerned about performance.
Be careful, as this is reflection based it will be slow.
In a recent web project our Base entity toString() method was ToStringBuilder.reflectionToString(this)
This method is called in hibernate during save (via Spring Data JPA respository). We have quite a large object tree with nested lists, leading to a large in memory and CPU hit during save.
It almost sank the project.
Just use code generator in your IDE to generate toString() method. This way you will avoid the overhead caused by using reflections. In real production system your toString() method can be called very often (100/sec) causing garbage collector to work hard and pause your JVM. These pauses can be seconds or tens of seconds.
Related
I was wondering how the serialization of MicroStream works in detail.
Since it is described as "Super-Fast" it has to rely on code-generation, right? Or is it based on reflections?
How would it perform in comparison to the Protobuf-Serialization, which relies on Code-generation that directly reads out of the java-fields and writes them into a bytebuffer and vice-versa.
Using reflections would drastically decrease the performance when serializing objects on a huge scale, wouldn't it?
I'm looking for a fast way to transmit and persist objects for a multiplayer-game and every millisecond counts. :)
Thanks in advance!
PS: Since I don't have enough reputation, I can not create the "microstream"-tag. https://microstream.one/
I am the lead developer of MicroStream.
(This is not an alias account. I really just created it. I'm reading on StackOverflow for 10 years or so but never had a reason to create an account. Until now.)
On every initialization, MicroStream analyzes the current runtime's versions of all required entity and value type classes and derives optimized metadata from them.
The same is done when encountering a class at runtime that was unknown so far.
The analysis is done per reflection, but since it is only done once for every handled class, the reflection performance cost is negligible.
The actual storing and loading or serialization and deserialization is done via optimized framework code based on the created metadata.
If a class layout changes, the type analysis creates a mapping from the field layout that the class' instances are stored in to that of the current class.
Automatically if possible (unambiguous changes or via some configurable heuristics), otherwise via a user-provided mapping. Performance stays the same since the JVM does not care if it (simplified speaking) copies a loaded value #3 to position #3 or to position #5. It's all in the metadata.
ByteBuffers are used, more precisely direct ByteBuffers, but only as an anchor for off-heap memory to work on via direct "Unsafe" low-level operations. If you are not familiar with "Unsafe" operations, a short and simple notion is: "It's as direct and fast as C++ code.". You can do anything you want very fast and close to memory, but you are also responsible for everything. For more details, google "sun.misc.Unsafe".
No code is generated. No byte code hacking, tacit replacement of instances by proxies or similar monkey business is used. On the technical level, it's just a Java library (including "Unsafe" usage), but with a lot of properly devised logic.
As a side note: reflection is not as slow as it is commonly considered to be. Not any more. It was, but it has been optimized pretty much in some past Java version(s?).
It's only slow if every operation has to do all the class analysis, field lookups, etc. anew (which an awful lot of frameworks seem to do because they are just badly written). If the fields are collected (set accessible, etc.) once and then cached, reflection is actually surprisingly fast.
Regarding the comparison to Protobuf-Serialization:
I can't say anything specific about it since I haven't used Protocol Buffers and I don't know how it works internally.
As usual with complex technologies, a truly meaningful comparison might be pretty difficult to do since different technologies have different optimization priorities and limitations.
Most serialization approaches give up referential consistency but only store "data" (i.e. if two objects reference a third, deserialization will create TWO instances of that third object.
Like this: A->C<-B ==serialization==> A->C1 B->C2.
This basically breaks/ruins/destroys object graphs and makes serialization of cyclic graphs impossible, since it creates and endlessly cascading replication. See JSON serialization, for example. Funny stuff.)
Even Brian Goetz' draft for a Java "Serialization 2.0" includes that limitation (see "Limitations" at http://cr.openjdk.java.net/~briangoetz/amber/serialization.html) (and another one which breaks the separation of concerns).
MicroStream does not have that limitation. It handles arbitrary object graphs properly without ruining their references.
Keeping referential consistency intact is by far not "trying to do too much", as he writes. It is "doing it properly". One just has to know how to do it properly. And it even is rather trivial if done correctly.
So, depending on how many limitations Protobuf-Serialization has ("pacts with the devil"), it might be hardly or even not at all comparable to MicroStream in general.
Of course, you can always create some performance comparison tests for your particular requirements and see which technology suits you best. Just make sure you are aware of the limitations a certain technology imposes on you (ruined referential consistency, forbidden types, required annotations, required default constructor / getters / setters, etc.).
MicroStream has none*.
(*) within reason: Serializing/storing system-internals (e.g. Thread) or non-entities (like lambdas or proxy instances) is, while technically possible, intentionally excluded.
I am currently having some performance issues using PropertyUtilsBean.getProperty to evaluate some property-expressions like:
obj.propert1.coolMap[1].property
I am currently using PropertyUtilsBean.getProperty for this, but it has to be executed a lot of times and I can see the enormous amount of wasted CPU time in JProfiler.
As the expression for each occurance never changes, I would like to sort of pre-initialize the property accessor. In an older version of my software I used reflection to save an instance of the getMethod so all I had to do, was to call the method for a given object. In the current version we added support for more complex expressions and therefore switched to PropertyUtilsBean ... seems this was a bad decision from a performance point of view.
Is there a way to have reusable access to a property which is able to interpret such property expressions as PropertyUtilsBean understands them? I would really like to be able to do this without implementing it manually.
I have a requirement, where support in my application a lot of processing is happening, at some point of time an exception occrured, due to an object. Now I would like to know the whole history of that object. I mean whatever happened with that object over the period of time since the application has started.
Is this peeping into this history of Object possible thru anyway using JMX or anything else ?
Thanks
In one word: No
With a few more words:
The JVM does not keep any history on any object past its current state, except for very little information related to garbage collection and perhaps some method call metrics needed for the HotSpot optimizer. Doing otherwise would imply a huge processing and memory overhead. There is also the question of granularity; do you log field changes only? Every method call? Every CPU instruction during a method call? The JVM simply takes the easy way out and does none of the above.
You have to isolate the class and/or specific instance of that object and log any operation that you need on your own. You will probably have to do that manually - I have yet to find a bytecode instrumentation library that would allow me to insert logging code at runtime...
Alternatively, you might be able to use an instrumenting profiler, but be prepared for a huge performance drop when doing that.
That's not possible with standard Java (or any other programming language I'm aware of). You should add sufficient logging to your application, which will allow you to get some idea of what's happened. Also, learn to use your IDE's debugger if you don't already know how.
I generally agree with #thkala and #artbristol (+1 for both).
But you have a requirement and have no choice: you need a solution.
I'd recommend you to try to wrap your objects with dynamic proxies that perform auditing, i.e. write all changes that happen to object.
You can probably use AspectJ for this. The aspect will note what method was called and what are the parameters that were sent. You can also use other, lower level tools, e.g. Javasist or CgLib.
Answer is No.JVM doesn't mainatain the history of object's state.Maximum what you can do you can keep track of states of your object that could be some where in-memory and when you get exception you can serialize that in-memory object and then i think you can do analysis.
I very much like the simplicity of calling remote methods via Java's RMI, but the verbosity of its serialization format is a major buzz kill (Yes, I have benchmarked, thanks). It seems that the architects at Sun did the obvious right thing when designing the RPC (speaking loosely) component, but pulled an epic fail when it came to implementing serialization.
Conversely, it seems the architects of Thrift, Avro, Kryo (especially), protocol buffers (not so much), etc. generally did the obvious right thing when designing their serialization formats, but either do not provide a RPC mechanism, provide one that is needlessly convoluted (or immature), or else one that is more geared toward data transfer than invoking remote methods (perfectly fine for many purposes, but not what I'm looking for).
So, the obvious question: How can I use RMI's method-invocation loveliness but employ one of the above libraries for the wire protocol? Is this possible without a lot of work? Am I evaluating one of the aforementioned libraries too harshly (N.B. I very much dislike code generation, in general; I dislike unnecessary annotations somewhat, and XML configuration quite a bit more; any sort of "beans" make me cringe--I don't need the weight; ideally, I'm looking to just implement an interface for my remote objects, as with RMI).
Once upon a time, I did have the same requirement. I had changed rmi methods arguments and return types to byte[].
I had serialized objects with my preferred serializer to byte array, then called my modified rmi methods.
Well, as you mentioned java serialization is too verbose, therefore 5 years ago I did implement a space efficient serialization algorithm. It saves too much space, if you are sending a very complex object graph.. Recently, I have to port this serialization implementation to GWT, because GWT serialization in Dev mode is incredibly slow.
As an example;
rmi method
public void saveEmployee(Employee emp){
//business code
}
you should change it like below ,
public void saveEmployee(byte[] empByte) {
YourPreferredSerializer serialier = YourPreferredSerializerFactory.creteSerializer();
Employee emp = (Employee) serializer.deSerialize(empByte);
//business code
}
EDIT :
You should check MessagePack . it looks promising.
I don't think there is a way to re-wire RMI, but it might be that specific replacement projects -- I am specifically thinking of DiRMI -- might? And/or project owners might be interest in helping with this (Brian, its author, is a very competent s/w engineer from Amazon.com).
Another interesting project is Protostuff -- its author is building a RPC framework too (I think); but even without it supports an impressive range of data formats; and does this very efficiently (as per https://github.com/eishay/jvm-serializers/wiki/).
Btw, I personally think biggest mistake most projects have made (like PB, Avro) is not keeping proper separation between RPC and serialization aspects nicely separate.
So ability to do RPC using a pluggable data format or serialization providers seems like a good idea to me.
writeReplace() and readResolve() is probably the best combo for doing so. Mighty powerful in the right hands.
Java serialization is only verbose where it describes the classes and fields it's serializing. Overall, the format is as "self describing" as XML. You can can actually override this and replace it with something else. This is what the writeClassDescriptor and readClassDescriptor methods are for. Dirmi overrides these methods, and so it is able to use standard object serialization with less wire overhead.
The way it works is related to how its sessions work. Both endpoints may have different versions of the object, and so simply throwing away the class descriptors won't work. Instead, additional data is exchanged (in the background) so that the serialized descriptor is replaced with a session-specific identifier. Upon seeing the identifier, a lookup table is examined to find the descriptor object. Because the data is exchanged in the background, there's a brief "warm up period" after a session is created and for every time an object type is written for the first time.
Dirmi has no way to replace the wire format at this time.
For example:
Security.setProperty("ocsp.enable", "true");
And this is used only when a CertPathValidator is used. I see two options for imporement:
again singleton, but with getter and setter for each property
an object containing the properties relevant to the current context:
CertPathValidator.setValidatorProperties(..) (it already has a setter for PKIXParameters, which is a good start, but it does not include everything)
Some reasons might be:
setting the properties from the command line - a simple transformer from command-line to default values in the classes suggested above would be trivial
allowing additional custom properties by different providers - they can have public Map getProviderProperties(), or even public Object .. with casting.
I'm curious, because these properties are not always in the most visible place, and instead of seeing them while using the API, you have to go though dozens of google results before (if lucky) getting them. Because - in the first place - you don't always know what exactly you are looking for.
Another fatal drawback I just observed is that this is not thread-safe. For example if two threads want to check a revocation via ocsp, they have to set the ocsp.responderURL property.. and perhaps override the settings of each other.
This is actually a great question that forces you to think about design decisions you may have made in the past. Thanks for asking a question that should have occurred to me years ago!
It sounds like the objection is not so much the singleton aspect of this (although an entirely different discussion could occur about that) - but the use of string keys.
I've worked on APIs that used this sort of scheme, and the reasons you outline above were definitely the driving factors - it makes it crazy simple to parse a command line or properties file, and it allows for 3rd party extensibility without impact to the official API.
In our library, we actually had a class with a bunch of static final String entries for each of the official parameters. This gave us the best of both worlds - the developer could still use code completion where it made sense to do so. It also becomes possible to construct hierarchies of related settings using inner classes.
All that said, I think that the first reason (easy parsing of command line) doesn't really cut it. Creating a reflection driven mechanism for pushing settings into a bunch of setters would be fairly easy, and it would prevent the cruft of String->object transformation from drifting into the main application classes.
Extensibility is a bit trickier, but I think it could still be handled using a reflection driven system. The idea would be to have the main configuration object (the one with all the setters in it) also have a registerExtensionConfiguration(xxx) method. A standard notation (probably dot separated) could be used to dive into the resultant acyclic graph of configuration objects to determine where the setter should be called.
The advantage of the above approach is that it puts all of the command line argument/properties file parsing exception handling in one place. There isn't a risk of a mis-formatted argument floating around for weeks before it gets hit.