We have a requirement with one of the source applications which allows very few connections per user to its database.
Since we have multiple spring batch interfaces connected to same source Db, we run out of connections, Spring batches run as individual java programs and we do not have a container.
Please suggest a way to have a datasource with multiple users or a data source confirguration that can help us maintain a common pool with multiple users connected to that database.
One ugly workaround, I think of is to have a common service to fetch connections from a databases and in that service we can use some container to maintain a pool for multiple users, by having a list/array of datasource connections per user.
To solve this we had to add a container. Tomcat to be precise. It managed all connections. All applications were deployed on tomcat and tomcat manager was used to start stop the applications, when needed.
Related
in my project I have different applications that all need a database connection (all "apps" are running on the same server) now my question is, what is better:
one "backend" that get requested from the apps through netty or something and has the one and only mongodb connection and cache with redis
or
all apps have mongodb connection and global cache with redis
Thanks in advance
TG
//edit
all applications are for the same project so they will need the same data
I would suggest you to write separate Backends for each Application as tomorrow you might want to have different connection requirements from each application. For eg : One application might decide it doesn't want to use Mongo DB at all . One application might want to use more connections and might be a noisy neighbour for others. Unless you are willing to write a Full Policy based server which can cater to the unique requirements of each application.
I have a spring boot application which uses embedded tomcat. The app is hosted on multiple EC2 instances, which auto scale if required and some of which may be killed/restarted. So, effectively there are 3 instances of the app running , and requests are routed from the load balancer to any of these instances.
I am trying to track user sessions on my app. I started with implementing container level session management using tomcat HttpSession. But it is not able to track sessions across instances. On researching a bit, I got to know that i need something like session replication.
My app is not running a tomcat cluster, it has 3 independent instances of the API which do not talk to each other in anyway. I am not planning to change that and not sure if it is possible with AWS as it does not encourage multicast communication for this purpose.
Also, I do not want to setup/manage a separate DB (like redis with spring session) just for this purpose, because I only need session Ids for logging, and I need to do that in a lightweight manner.
Is there any other way to manage sessions across instances ? or for my purpose, would it be better to just implement some custom code which can check for session id/token passed to and fro between the frontend and backend.
The goal is to externalize the sessions from your application server so that you can autoscale, restart, load balance etc. without worrying about breaking a User's session.
Honestly on AWS using the Spring stack, I would recommend Spring Session + Redis. I've used it countless times and it is very easy to implement. You can leverage AWS Elasticache which manages the Redis cluster for you (like RDS does for relational DBs).
You could write your own custom implementation of Spring Session with a backing store of S3, Dynamo, etc. But is that really any better than the Redis implementation? I'd recommend the path of least resistance.
I'm new to java in web application design and I'm getting surprised how many things I don't now.
In particular I'm having problems in understending how servlet containers manage resources like connection pooling classes.
Assuming that I choose a pooling library (let's say c3p0), I read that there are many ways to use and manage the connection pooling classes.
For instance in many examples I saw that a certain class (let's say ComboPooledDataSource) is instantiated in the init() method of the servlet and here I'm getting a bit confusing. I mean, I think that a connection pooling system must exist and have a separate life with respect to all the servlets that will need a connection, otherwise it make no sense. So I think that the class below may be a Thread that is started once from the first servlet that call the init method and then it continues to exist until someone doesn not interrupt it. Is that correct? If not how does it work?
Anyway once I start this class , is it shared among all the servlets in the context (I mean all the servlets that call it in the init method)?
Other examples set up the connection pooling system as a resource by for instance defining it in the context.xml and then any servlet that want a connection just have to access it via JNDI (JNDI is correct?). What i understood (or I think to) is that in this case the thread that will execute the pooling system is started when the application is started and each servlet can access it when it wants to. Is that correct?
In this case can I modify the connection pooling system properties by a servlet or a background thread runtime? (for example if I want to change the number of connections as a function of the statistics on the number of requests and so on)
If I want to create different pools (for example I want to subdivide the database access to N different databases or I want to access using different usernames ) do I need to create as many resources as are the different kind of connection I want?
Is there a "better" way among these two or they are equivalent?
It comes down to ease of use of the webapp and (Tomcat) server maintenance.
You describe 2 use cases:
a connection pool used by one webapp
a connection pool shared between webapps
The first case is suitable for "ease of use": you can drop the war-file in any Tomcat server and it will work (e.g. Jenkins delivers such a war-file) since the webapp contains all the code needed to access a database. No need to change the configuration of the Tomcat server and restart it.
If I can, I like to deliver these kind of war-files since less things can go wrong (e.g. no clashes with configuration for other webapps). You can take it a step further by delivering Tomcat embedded with an application (so you don't need a Tomcat server to start with, an example project is here).
Note that opening a database connection pool (preferably via ServletContextListener.contextInitialized, not a servlet) and closing it (via contextDestroyed) does not involve starting and stopping a thread. The pool implementation may decide to start a background thread (e.g. to remove idle and/or abandoned connections), but it does not have to.
The second case is suitable for several webapps that share a common ground. For example, if several webapps all running in the same Tomcat server all use the same database, it saves time and effort if the Tomcat server already has a connection pool available for the different webapps (via JNDI). The Tomcat server can contain the JDBC driver software and connection pool software in the "lib" directory and configuration is done once in the context.xml of the server. If the database changes or different (patched) software components are required, only the tomcat server needs to be updated and all the webapps can remain the same. In this case, updating the Tomcat server instead of each webapp is a lot easier.
The second case is also more suitable for monitoring: there is a good chance you can monitor the connection pool via JMX. This kind of monitoring may not be available in the first case.
Changing "the number of connections as a function of the statistics on the number of requests " is not needed with connection pools: you set a maximum amount of connections to use and a timeout to remove idle connections. The connection pool will then grow and shrink with the number of requests.
Subdividing the database access to N different databases would require a different connection pool for each database, unless this is "read-only" in which case you could also use a load-balancer.
Access using different usernames is a bit tricky. I read in another question (which I cannot find anymore, sorry) that you can change the schema during runtime, but changing username/password might require a new connection in which case connection pooling is out.
Are you using Tomcat to run your servlets? Tomcat 7 has a new pooling solution that you could investigate. Tomcat also includes the dbcp libraries you can use them by setting the factory="org.apache.tomcat.dbcp.dbcp.BasicDataSourceFactory". The DBCP pools are self managed and you can define the configuration values in the context.xml file I am not aware of how to change these values on the fly. I believe the Tomcat 7 pool use the same configuration as the DBCP with a couple of added options to make the conversion between the two simple. We use DBCP in all our applications and have not experienced problems so I have not used the Tomcat 7 pooling. I am thinking you would need to create many pools to handle most of your requirements.
Is it safe to run a database connection pool (like Commons DBCP or c3p0) as part of an application deployed to an application server like Glassfish or Websphere? Are there any additional steps over a standalone application that should be taken to ensure safety or performance?
Update, clarification of reason - the use case I have in mind could need new data sources to be defined at runtime by skilled end users - changing the data sources is part of the application's functionality, if you like. I don't think I can create abnd use container-managed pools on the fly?
AFAIK it works, but it will of course escape the app. server management features.
Also, I'm not entirely sure how undeployment or redeployment happens and whether the connections are correctly disposed. But that can be considered as a minor safety detail: if disposed improperly, connections will simply time out I guess. I'm also not entirely sure whether it works for XA data source which integrates with the distributed transaction manager.
That said, using the app. server pool is usually a matter of configuring the JNDI name in a configuration file. Then you get monitoring, configuration from the admin console, load management, etc. for free.
In fact, you can create container-managed datasources depending on AS you use.
For example, Weblogic has an extensive management API that is used, for example, by their own WLST (Weblogic Shell) to configure servers by scripts. This is, of course, Java API. It has methods to create and configure datasources too.
Another route is JMX-based configuration. All modern AS expose themselves as JMX containers. You can create datasources via JMX too.
All you need is to grant your application admin privileges (i.e. provide with username/password).
The benefit of container-managed DS is it can be clustered. Also, it can be managed by human being using standard AS UI.
If that doesn't work for you, why, sure you can create application-managed DS any time and in any numbers. Just keep in mind that it will be bound to a specific managed server (unless you implement a manual clustering of it's definition).
I don't see why you'd want to. Why not use the connection pool that the app server provides for you?
Update: I don't believe it's possible to create new pools on the fly without having to bounce the app server, but I could be wrong. If that's correct, I don't believe that Commons DBCP or C3P0 will help.
We have an infrastructure set up where in the webservers are clustered and the application servers are not. The webservers route the request to the application servers based on round-robin policy.
In this scenario, the session data available in one application server is not available in the other application server. Is there anyway by which the session data from first application server can be made available in the second application ? The two application servers are physically separate boxes in different cells.
One approach could be to use the database - is there any other means of accomplishing this session replication ?
In WebSphere there are essentially two ways to replicate session data:
Persisting to a database
Memory-To-Memory transfers
Which one is appropriate for your needs is highly dependent on your application scenario:
How important is the persistence of your session data, when all your application servers go down?
How many session objects do you have at any one time simultaneously?
In a DB you can store many sessions without much problems, the other option is always a question of how much memory is available.
I would go with the database, if you already got one set up, which all application servers use anyway.
Here is the link to the WebSphere Information Center with the necessary details.
One obvious solution is to enable clustering of your application servers. I assume from the way you worded your question you have rejected this option. Another option is to change the routing used by the web servers to use session affinity (requests for the same session go to the same app server).
Other that that, I'd second the answer by dertoni.
maybe you can look at 'terracota'. its an caching framework, which can cache sessions and runs on a seperate server
There are two options for clustering within WebSphere, session replication or database. If you have large session objects you are best off using database because it allows you to offload stale sessions to disk. If they are then represented then they can be extracted from the database, if you use session replication then those sessions need to stay in memory on not just your target server but also the other servers in the replication group. With large sessions this can lead to an out of memory condition.
With database session handling it is also very customisable and doesn't performance noticeably in the environments that I have used it.
don't forget oracle coherence.