I'm currently losing myself in hibernate understanding. Firstly let me explain, my problem.
We decide to add hibernate shared/clustered cache for a product. We got no problem in testing environnements but since this modification is publish in production we got heavy overload on CPU (at least + 1 CPU use), RAM (+1 to 3 Go) and network (bandwidth *2). This even if I disable hibernate L2 cache and query cache.
EDIT : 60 to 65 % of CPU time is spent in thread.sleep in RMI Synchronization calls.
My technical stacks :
Liferay portal CE 6.1.1 GE2
Ehcache 2.3 (not terracota it seems)
Hibernate 3.X (load from liferay dependancies)
Using liferay build service for most part of services which automatically create hibernate stack.
3 nodes in cluster
Now I will describe What I understand at the moment :
Hibernate L2 cache and query cache permit to save database query by using the result in cache to avoid useless queries
I'm using Ehcache RMI replication to shared caches
I'm noob in hibernate ;)
Let see my hibernate - clustering configurations hibernate-clustered.xml:
<!--
Hibernate will use the defaultCache unless custom configurations are defined
below for individual domain objects, collection, queries, etc.
-->
<defaultCache
maxElementsInMemory="10000"
eternal="false"
timeToIdleSeconds="600"
overflowToDisk="false"
>
<cacheEventListenerFactory
class="net.sf.ehcache.distribution.RMICacheReplicatorFactory"
properties="replicatePuts=false,replicateUpdatesViaCopy=false"
propertySeparator=","
/>
<bootstrapCacheLoaderFactory class="net.sf.ehcache.distribution.RMIBootstrapCacheLoaderFactory" />
</defaultCache>
<bootstrapCacheLoaderFactory class="net.sf.ehcache.distribution.RMIBootstrapCacheLoaderFactory" />
</cache>
I search & read for more than an entire day and I continue to have doubt (and probably misunderstanding) on my thinking process. I think it's time to ask for some help !
Firstly, I read a lot about replication and clustering. In my mind these concept are not clearly defined. Is these process the same ? What the different, pro and cons?
I read in some places that ehcache (not terracota ehcache) is not compatible with cluster, what the consequences about using it like i do in small cluster, is it scalable?
What do you think about the issue I experiment? How to explain this overhead ?
Let me know if you need more information.
Thanks!
Related
I have a problem when using cache across different modules. At first, I was using Ehcache 2.10.6 so in my configuration ehcache.xml file I had something like this:
<bootstrapCacheLoaderFactory
class="net.sf.ehcache.distribution.RMIBootstrapCacheLoaderFactory"/>
and I was using it as cache.bootstrap();
After upgrading to 3.x version (3.8.1) , this is not available nor I can see any replacements. Is there any replacement to do or something else to replicate the configuration above?
Thanks.
RMI based replication has been discontinued in Ehcache 3.x. Its implementation in Ehcache 2.x was lacking in semantics: writes could be lost, reads were not guaranteed to be consistent, etc ...
Instead you should look into cache clustering with Terracotta.
I'm using JCache annotations with Spring and EhCache. If I use #CacheResult on a method without specifying the cache name (or a #CacheDefaults on the class), then the default cache name resolves to the fully qualified method name. However such a cache is not found, unless explicitly created using the CacheManager. This may be manageable for a few such cache-enabled methods, but not if I have to create 50 different caches manually.
Is there a way to tell Spring (or any JCache implementer) to automatically create caches with the default name, if not found? This would allow me to use #CacheResult on any method without having to go update the cache configuration every time.
Is there a way to tell Spring (or any JCache implementer) to automatically create caches with the default name, if not found?
This problem is covered in cache2k. You can configure cache2k to use a default configuration, if the requested cache name is not known.
Here is an example XML configuration for this scenario, which is placed at /cache2k.xml in the class path:
<cache2k xmlns:xsi='http://www.w3.org/2001/XMLSchema-instance'
xmlns='https://cache2k.org/schema/v1.x'
xsi:schemaLocation="https://cache2k.org/schema/v1.x https://cache2k.org/schema/cache2k-core-v1.x.xsd">
<version>1.0</version>
<ignoreMissingCacheConfiguration>true</ignoreMissingCacheConfiguration>
<defaults>
<!-- default settings for every cache -->
<cache>
<entryCapacity>100_000</entryCapacity>
<expireAfterWrite>5m</expireAfterWrite>
</cache>
</defaults>
<templates>
<cache>
<name>shortExpiry</name>
<expireAfterWrite>5m</expireAfterWrite>
</cache>
<cache>
<name>lessResilient</name>
<resilienceDuration>1m</resilienceDuration>
</cache>
</templates>
<caches>
<cache>
<name>products</name>
<entryCapacity>10_000</entryCapacity>
<include>shortExpiry,lessResilient</include>
</cache>
<cache>
<name>users</name>
<entryCapacity>1000</entryCapacity>
<include>shortExpiry</include>
</cache>
</caches>
</cache2k>
By default, according to this configuration, a cache without a specific configuration entry will get 100K entry limit and 5 minutes expiry. The configuration may be suitable for the trails in production. Later on, you, or a system operator, can check what caches are existing and their performance via JMX and then add a more specific cache configuration as soon as there is need. If you have many caches, the configuration has a template mechanism to avoid repetition.
It may happen that caches need a specific configuration upfront and some don't. I suggest separating them into different cache managers.
If you need an in process cache, cache2k is a a good alternative to EHCache. It is way faster and more memory efficient.
You can use cache2k via JCache, however there is also direct Spring framework support. See the User Guide Spring Framework Section for details.
I am the author of cache2k and not a heavy Spring user at the moment, however, I am happy to help if there are any issues.
How can I configure EhCache 3.5 (xml-config, ehcache.xml) to allow only 100 cached instances for:
org.company.Foo? Basically, I have a dozen of entities with separate caching limits wanted.
I've read some sources where they state this is possible, like this one source
This syntax doesn't work for versions 3.x (and I doubt it is the correct config for 2.x too)
<ehcache>
<cache name="org.company.Foo" maxElementsInMemory="100" />
</ehcache>
The official documentation is silent (or I missed it somehow, but scrutinized pretty thoroughly) on this topic for versions 2.x as well as for versions 3.x
If not possible, what is the preferred idiom for configuring EhCache?
Thank you.
P.S. I am using EhCache 3.5 with Hibernate 5.3 and able to monitor the cache state via VisualVM console through MBeans plugin.
The correct config would be like this:
<config
xmlns:xsi='http://www.w3.org/2001/XMLSchema-instance'
xmlns:jsr107='http://www.ehcache.org/v3/jsr107'
xmlns='http://www.ehcache.org/v3'
xsi:schemaLocation="http://www.ehcache.org/v3 http://www.ehcache.org/schema/ehcache-core.xsd">
<service>
<jsr107:defaults enable-management="true" enable-statistics="true"/>
</service>
<cache alias="org.company.Foo">
<heap>1000</heap>
</cache>
currently we are aiming to do load balance 2 active servers 50/50.
Java application uses hibernate search locally, I have centralized this directory so both server uses the same directory.
I want to share hibernate indexing with multiple servers
I have set following so there is no locking between read/writefrom the servers
property name="hibernate.search.default.locking_strategy" value="none"
Does anyone know if this will be an issue?
I can't really answer your question but I'd like to share some considerations.
We used this kind of configuration during years in production (no custom lock strategy) and we experimented so much problem (stale nfs file handle, dead lock and index corruption).
We tried to defer all the index updates operation to a single server using JMS but even in this mode we experimented some problems (much less than in the mode where update operation occurs on many servers however)
Note also that putting the index files on NFS is strongly discouraged
We finally gave up hibernate search, for distributed indexes I'll personally advise to use elastic search.
However this is theoretically possible as stated there : https://docs.jboss.org/hibernate/search/5.1/reference/en-US/html/search-architecture.html#_lucene
This mode targets non clustered applications, or clustered applications where the Directory is taking care of the locking strategy.
I don't really know how the "Directory" is expected to handle the locking strategy
The previously used datasource configuration regarding hibernate search :
<!-- hibernate search configuration -->
<!-- index root folder location -->
<prop key="hibernate.search.default.indexBase">${env.ftindex.location.root}/${datasource.database}/${ftindex.folder}</prop>
<!-- global analyzer -->
<prop key="hibernate.search.analyzer">custom_analyzer</prop>
<!-- asynchronous indexing for performance considerations -->
<prop key="org.hibernate.worker.execution">async</prop>
<!-- max number of indexing operation to be processed asynchronously (before session flush) to avoid OutOfMemoryException -->
<prop key="hibernate.search.worker.buffer_queue.max">100</prop>
I'm using Hibernate second level cache in my application, for certain business reason I can't change the entity annotation any more.
In my project, apart from changing the Database from Hibernate, there exist also other native SQL that do not go through Hibernate. Therefore, the Hibernate second-level cache data could be stale after database being updated from native SQL. That's why I want to disable the second-level cache for certain entities (programmatically or other way than changing annotation).
Thanks in advance!
WARNING: As Jens Schauder noted, it is impossible to configure Ehcache to store 0 elements in memory by setting maxElementsInMemory="0" as it effectively causes opposite effect - sets unlimited size for the cache. This behaviour is not mentioned on the Hibernate Caching page but is documented on Cache Configuration page.
I have quickly reviewed the documentation and haven't found alternative approach yet. I am unable to delete this answer by myself. :-(
My original suggestion:
You can configure the implementation provider of second level cache to short TTL times and/or to store 0 entries of particular entity type.
E.g. if you are using the Ehcache, you can configure it in ehcache.xml:*
<cache
name="com.problematic.cache.EntityName"
maxElementsInMemory="0" <<== this should effectively disable caching for EntityName
overflowToDisk="false" <<== Do not overflow any entries to disk
/>
See Hibernate Caching in Ehcache documentation.
In Terracotta 3.1 and above, you can enable/disable Hibernate 2nd Level Caches on a per region basis, both in the configuration (statically) and at runtime, using the Terracotta Developer Console.
You can also monitor in realtime statistics about the cache and Hibernate, for individual nodes in a cluster or cluster-wide.
Terracotta is open source. For more details, check out Terracotta for Hibernate.