We're trying to horizontally scale a JPA based application, but have encountered issues with the second level cache of JPA. We've looked at several solutions (EhCache, Terracotta, Hazelcast) but couldn't seem to find the right solution. Basically what we want to achieve is to have multiple application servers all pointing to a single cache server that serves as the JPA's second level cache.
From a non java perspective, it would look like several PHP servers all pointing to one centralised memcache server as it's cache service. Is this currently possible with Java?
Thanks
This is in response to the comment above.
Terracotta will be deployed in it's own server
Each of the app server will have terracota drivers which will store/retrieve data to-fro terracotta server.
Ehcache api present in the application war, will invoke the terracota drivers to store data into terracotta server.
Hibernate api will maintain the L1 cache, in addition it will use the ehcache api to save/retrieve data to-fro L2 cache. Blissfully unaware about how ehcache api performs the task.
Related
I have recently started taking a look into Infinispan as our caching layer. After reading through the operation modes in Infinispan as mentioned below.
Embedded mode: This is when you start Infinispan within the same JVM as your applications.
Client-server mode: This is when you start a remote Infinispan instance and connect to it using a variety of different protocols.
Firstly, I am confuse now which will be best suited to my application from the above two modes.
I have a very simple use case, we have a client side code that will make a call to our REST Service using the main VIP of the service and then it will get load balanced to individual Service Server where we have deployed our service and then it will interact with the Cassandra database to retrieve the data basis on the user id. Below picture will make everything clear.
Suppose for example, if client is looking for some data for userId = 123 then it will call our REST Service using the main VIP and then it will get load balanced to any of our four service server, suppose it gets load balanced to Service1, and then service1 will call Cassandra database to get the record for userId = 123 and then return back to Client.
Now we are planning to cache the data using Infinispan as compaction is killing our performance so that our read performance can get some boost. So I started taking a look into Infinispan and stumble upon two modes as I mentioned below. I am not sure what will be the best way to use Infinispan in our case.
Secondly, As from the Infinispan cache what I will be expecting is suppose if I am going with Embedded Mode, then it should look like something like this.
If yes, then how Infinispan cache will interact with each other? It might be possible that at some time, we will be looking for data for those userId's that will be on another Service Instance Infinispan cache? Right? So what will happen in that scenario? Will infinispan take care of those things as well? if yes, then what configuration setup I need to have to make sure this thing is working fine.
Pardon my ignorance if I am missing anything. Any clear information will make things more clear to me to my above two questions.
With regards to your second image, yes, architecture will exactly look like this.
If yes, then how Infinispan cache will interact with each other?
Please, take a look here: https://docs.jboss.org/author/display/ISPN/Getting+Started+Guide#GettingStartedGuide-UsingInfinispanasanembeddeddatagridinJavaSE
Infinispan will manage it using JGroups protocol and sending messages between nodes. The cluster will be formed and nodes will be clustered. After that you can experience expected behaviour of entries replication across particular nodes.
And here we go to your next question:
It might be possible that at some time, we will be looking for data for those userId's that will be on another Service Instance Infinispan cache? Right? So what will happen in that scenario? Will infinispan take care of those things as well?
Infinispan was developed for this scenario so you don't need to worry about it at all. If you have for example 4 nodes and setting distribution mode with numberOfOwners=2, your cached data will live on exactly 2 nodes in every moment. When you issue GET command on NON owner node, entry will be fetched from the owner.
You can also set clustering mode to replication, where all nodes contain all entries. Please, read more about modes here: https://docs.jboss.org/author/display/ISPN/Clustering+modes and choose what is the best for your use case.
Additionally, when you add new node to the cluster there will StateTransfer take place and synchronize/rebalance entries across cluster. NonBlockingStateTransfer is implemented already so your cluster will be still capable of serving responses during that joining phase. See: https://community.jboss.org/wiki/Non-BlockingStateTransferV2
Similarly for removing/crashing nodes in your cluster. There will be automatic rebalancing process so for example some entries (numOwners=2) which after crash live only at one node will be replicated respectively to live on 2 nodes according to numberOfOwners property in distribution mode.
To sum it up, your cluster will be still up to date and this does not matter which node you are asking for particular entry. If it does not contain it, entry will be fetched from the owner.
if yes, then what configuration setup I need to have to make sure this thing is working fine.
Aforementioned getting started guide is full of examples plus you can find some configuration file examples in the Infinispan distribution: ispn/etc/config-samples/*
I would suggest you to take a look at this source too: http://refcardz.dzone.com/refcardz/getting-started-infinispan where you can find even more basic and very quick configuration examples.
This source also provides decision related information for your first question: "Should I use embedded mode or remote client-server mode?" From my point of view, using remote cluster is more enterprise-ready solution (see: http://howtojboss.com/2012/11/07/data-grid-why/). Your caching layer is very easily scalable, high-available and fault tolerant then and is independent of your database layer and application layer because it simply sits between them.
And you could be interested about this feature as well: https://docs.jboss.org/author/display/ISPN/Cache+Loaders+and+Stores
I think in newest Infinispan release supports to work in a special, compatibility mode for those users interested in accessing Infinispan in multiple ways .
follow below link to configure your cache environment to support either embedded or remotely.
Interoperability between Embedded and Remote Server Endpoints
My DAL is implemented with Hibernate and I want to use EHCache as its second level cache with its distributed capabilities (for scalability and HA).
Seeing as EHCache provides distributed caching only with Terracotta my question is what is the role of the Terracotta server instance? Does it also hold data? Does it only coordinate the distribution between the partitioned cache parts?
My confusion derives mainly from this explanation regarding TSA which says the server holds the data but I think that maybe in my scenario the cache and the Terracotta server are sort of merged. Am I correct?
If the server does hold data then why shouldn't the bottleneck just move from the db to the Terracotta server?
Update:
Affe's answer answered the second part of my question which was the important part but just in case someone comes by looking for the first part I'll say that the TC server has to hold all the data that the EHCache in memory holds and so if you want a distributed cache (not replicated) then the L2 (TC server) must hold all the objects itself as well.
Thanks in advance,
Ittai
The idea is it's still significantly faster to contact the terracotta cluster via the terracotta driver and do what's basically a Map lookup, than to acquire a database connection and execute an SQL statement. Even if that does become the application's choke point, overall throughput would be expected to still be significantly higher than a JDBC Connection + SQL choke point. Open connections and open cursors are big resource hogs in the database, an open socket to the terracotta cluster is not!
You can get ehcache clustered without using terracotta. They have documentation for doing it via RMI, JGroups and JMS. We are using JMS since we have a significant JMS infrastructure to handle the communication already. I don't know how well it will scale in the long term, but our current concern is just HA.
Preface: I'm not a java developer.
I have a question about Tomcat / jBOSS and other java application servers. Where are sessions (session data) stored? In PHP, sessions are usually stored in the database which means you can easily share session data in a load balanced environment. In Tomcat and other application servers, session seem to be stored in memory by default which would not apply in a load balanced env. While it is true that PHP stores sessions in files by default, it takes a few lines to hook it up to a DB. Is the same true of applications servers?
Basically, what's the pros for storying sessions in memory? Is this still standard practice for application servers? Thanks all!
I have a question about Tomcat / jBOSS and other java application servers. Where are sessions (session data) stored?
By default, I'd say in memory. Details are actually... implementation details which are application server specific.
In PHP, sessions are usually stored in the database which means you can easily share session data in a load balanced environment. In Tomcat and other application servers, session seem to be stored in memory by default which would not apply in a load balanced env.
Well, not exactly. What this means is that a client request has to be sent to the same node in a clustered environment (this is referred to as "session stickiness") and this is not an issue from a load balancing point of view. But this is an issue from a failover point of view: in case of failure of a node in a cluster, session state managed by the node can be lost. To solve this, almost all application server providers implement session failover (using various mechanism such as in-memory replication, JDBC-based persistence, etc). But, again, implementation details are application server specific. See for example how Tomcat or WebLogic deal with that. The Under the Hood of J2EE Clustering article on The Server Side is a very interesting reading too.
While it is true that PHP stores sessions in files by default, it takes a few lines to hook it up to a DB. Is the same true of applications servers?
As I said, not all application servers will offer JDBC-based persistence. Having that said, and to answer your question, configuration is in general simple when they do. But using a database is really not the preferred solution (actually, I avoid it at all price).
Basically, what's the pros for storying sessions in memory? Is this still standard practice for application servers?
Simply: performance! Serializing data, calling a database, writing to disk, all this has a cost. In-memory replication obviously allows to avoid some overhead. But it has some limitation too. For example, it doesn't allow WAN HTTP Session State Replication with WebLogic. But well, only few people need this :)
With the provided Session Manager, the session is always in memory but it has a manager to persist the session to JDBC store,
http://tomcat.apache.org/tomcat-5.5-doc/config/manager.html
Unlike in PHP, the session is still accessed from memory and it's only persisted to DB when the memory limit is reached or at server shutdown. So your load-balancer must have sticky routing for this to work.
The benefit of in-memory session is performance because db access doesn't occur for every transaction.
You can write your own session manager to simulate the PHP behavior.
The JavaEE spec does not dictate this, it is up to the individual implementations to decide. For example, the usual way to handle load balancing in Tomcat is to use replicated sessions, where the session data is multicast between the nodes. Storing session data in the database is a huge performance killer, and while Tomcat may support it, I really wouldn't recommend it.
We have an infrastructure set up where in the webservers are clustered and the application servers are not. The webservers route the request to the application servers based on round-robin policy.
In this scenario, the session data available in one application server is not available in the other application server. Is there anyway by which the session data from first application server can be made available in the second application ? The two application servers are physically separate boxes in different cells.
One approach could be to use the database - is there any other means of accomplishing this session replication ?
In WebSphere there are essentially two ways to replicate session data:
Persisting to a database
Memory-To-Memory transfers
Which one is appropriate for your needs is highly dependent on your application scenario:
How important is the persistence of your session data, when all your application servers go down?
How many session objects do you have at any one time simultaneously?
In a DB you can store many sessions without much problems, the other option is always a question of how much memory is available.
I would go with the database, if you already got one set up, which all application servers use anyway.
Here is the link to the WebSphere Information Center with the necessary details.
One obvious solution is to enable clustering of your application servers. I assume from the way you worded your question you have rejected this option. Another option is to change the routing used by the web servers to use session affinity (requests for the same session go to the same app server).
Other that that, I'd second the answer by dertoni.
maybe you can look at 'terracota'. its an caching framework, which can cache sessions and runs on a seperate server
There are two options for clustering within WebSphere, session replication or database. If you have large session objects you are best off using database because it allows you to offload stale sessions to disk. If they are then represented then they can be extracted from the database, if you use session replication then those sessions need to stay in memory on not just your target server but also the other servers in the replication group. With large sessions this can lead to an out of memory condition.
With database session handling it is also very customisable and doesn't performance noticeably in the environments that I have used it.
don't forget oracle coherence.
Tomcat (version 5 here) stores session information in memory. When clustering this information is periodically broadcast to other servers in the cluster to keep things in sync. You can use a database store to make sessions persistant but this information is only written periodically as well and is only really used for failure-recovery rather than actually replacing the in-memory sessions.
If you don't want to use sticky sessions (our configuration doesn't allow it unfortunately) this raises the problem of the sessions getting out of sync.
In other languages, web frameworks tend to allow you to use a database as the primary session store. Whilst this introduces a potential scaling issue it does make session management very straightforward. I'm wondering if there's a way to get tomcat to use a database for sessions in this way (technically this would also remove the need for any clustering configuration in the tomcat server.xml).
There definitely is a way. Though I'd strongly vote for sticky sessions - saves so much load for your servers/database (unless something fails)...
http://tomcat.apache.org/tomcat-5.5-doc/config/manager.html has information about SessionManager configuration and setup for Tomcat. Depending on your exact requirements you might have to implement your own session manager, but this starting point should provide some help.
Take a look at Terracotta, I think it can address your scaling issues without a major application redesign.
I've always been a fan of the Rails sessions technique: store the sessions (zipped+encrypted+signed) in the user's cookie. That way you can do load balancing to your hearts content, and not have to worry about sticky sessions, or hitting the database for your session data, etc. I'm just not sure you could implement that easily in a java app without some sort of rewriting of your session-access code. Anyway just a thought.
Another alternative would be the memcached-session-manager, a memcached based session failover and session replication solution for tomcat 6.x / 7.x. It supports both sticky sessions and non-sticky sessions.
I created this project to get the best of performance and reliability and to be able to scale out by just adding more tomcat and memcached nodes.