I'm implementing several JavaSE applications on single server. Is it possible to setup a single connection pool (e.g. C3P0) and share among these applications? I just want to have an easy way to manage the total number of DB connections.
Is there any drawbacks using such centralized connection pool?
Thank you,
Wilson
You can simply use the same data source defined in the server for all application to share the same DB connection pool easily.
One obvious drawback would be that performance of independent application may degrade due to a load on totally unrelated application which would be hard to figure out.
Related
I am making an app that use RMI and DATABSE,but i wanted to follow a good resource utilization for my connection to the database. I wanted to have only one single instance to be created when the server start and I need other class needs to get the connection and do their executeQuery(),createStatement() and return results to the client which be liter bind to combobox,table.
The proper way is to use a JDBC connection pool library such as boneCP. Do not attempt a do-it-yourself implementation of the connection pool: it is ridden with difficulties, and the ready-made alternatives always work due to the simple contract they implement.
I should also mention that using plain RMI is an obsoleted communication technique which was mainly associated with Java applets of the early 2000's. Today you would be much better served by a REST/JSON combination if you are doing something lightweight. Even on the enterprise level, REST/XML is gaining popularity, while for techniques based on Java Serialization it is waning. These newer technologies are preferred (among many other aspects) for the greater transparency of their line protocol, which helps diagnostics, debugging, and general predictability of the system as a whole.
I currently have a Java EE application in which I have implemented my own connection pool class. Each method I use does a simple query (Statement and ResultSet). In the finally block of each method I use JDBC/my pool in, I first close the ResultSet, and then I close the Statement as many, many resources in books and online indicate should be done. Finally, I return the connection back to the pool.
While watching the memory of the JVM, I notice that the memory never really releases after I make a call which uses JDBC through my connection pool, or it takes a very long time to do so. I have checked my Garbage Collection settings and I am using gencon (IBM WebSphere), which many resources online have indicated is very good as well. I am using Spring Framework in my application too.
The connection pool class I wrote is very simple. Upon initialization, it creates a certain number of connections to the database and adds them to a Queue (I tried another implementation with just a simple Vector, but same results with the memory). When you request a connection, it checks to make sure there's an available connection, and if so it gives one to the caller. At the end, you return it back and it puts it back into the Queue/Vector.
I am wondering if there's anything else that could be done about this? Should I let Spring Framework handle my connection pool instead, or is there something else that handles the memory better? To me, it does make sense, but I'm not too familiar with implementing connection pooling. All the resources say to do what I am doing, but I'm assuming they might be using some built in pooling implementation. I do know that closing the connection works, but since this is a custom pooled solution, I cannot really do that.
Thanks!
Is your application running inside the WebSphere application server? Is it getting its datasource from the app server, rather than creating it itself? If so, then the connections and statements are already being pooled, and you don't need to pool them yourself.
OK, first thing's first: unless you have a very good reason, implementing your own connection pooling mechanism is an exercise in unnecessary risk. There's simply many connection pooling mechanisms out there to pick from - Spring would be one of them, Jakarta's DBCP being another. If you have the option of using a third-party utility for connection pooling - do it. If you're running inside a container, let the container do the work for you (WebSphere already does this, through its various "helper classes").
Regarding the symptoms you're experiencing: the fact that the memory isn't released, doesn't mean that there's a memory leak. It is up to the JVM to decide whether and when to actually free-up unreferenced object instances. Are you encountering OutOfMemory errors as well?
Start by issuing some heap dumps (right after JDBC resources' release) to see what's going on in there.
About the code you wrote - without posting the code, it's very hard to guesstimate whether you have hidden bugs there; but the flow you're describing appears correct.
My Java application will need to gather information from different MySQL-tables on startup. Without the database information, the application cannot be used. Hence, it takes up to a few seconds to start up (reducing this time with cache when possible).
Would it be bad practise to preform each of these SQL-queries in a
separate thread, allowing computers with multiple CPU cores to start
the application eventually even faster (no guarantees, I know)?
Is there any cons that I need to know about before implementing this
"system"?
It's something your going to have to try.
If your bringing back relatively few rows from each table, it would probably take longer to establish the database connections in each of the threads (or jdbc connection pool) than to establish it once and do the queries.
Fortunately it's not a lot of code, so you should be able to try it out pretty quickly.
No, certainly not. For example, a Java web server like Tomcat makes it all the time, when multiple users access your web application concurrently.
Just make sure you manage properly your data integrity using transactions.
Executing the request by parallelizing them instead of executing them serially may be a good idea.
Take care to use a Datasource and each request must use its own connection to the database (never share a conenction between different threads simultaneously).
Be sure that your connection pool and your thread pool size is well adapted
Database sessions are relatively expensive objects. Parallelizing to a few threads is no problem, but don't create 1000 threads if you have 1000 tables.
Furthermore, multithreading comes with complexity and potentially huge maintenance costs (for example, unreproducible issues resulting from race conditions). So, do your measurements, and if you find out that the speed up is just a few percent, just put everything back.
There are more ways to avoid the latency you see. For example, you can send multiple queries in a single command batch, thus reducing the number of roundtrips between your code and the database.
I am writing a ETL project in JAVA. I will connect to the source database, get the data only once, do some transformations and Load the data to a target database.
The point is that I am not connecting to the source or the target database multiple times repeatedly. I just connect once (using JDBC), get the data I need and close the connection.
Should I still use the connection pooling?
Thank you for your views!
Connection pooling is used to get around the fact that many database drivers take a long time to create a connection. If you only need to use it shortly, and then discard it, the overhead might be substantial (both in time and cpu) if you need many connections. It is simply faster to reuse than to create a new.
If you do not have that need, there is no reason to set up a connection pool if you don't have it already. If you happen to have one already, then just use that.
My guess is that in some circonstances, using several threads and concurrent connections could improve the overvall throughput of your software allowing for exemple to use all CPU of your RDBMS server, or of the client ETL. This also could help using the fact that several tables could sit physically on differents hardware and thus could be accessed in parallel.
The real impact would really depend of the computers you use and the architecture of the database.
Be carefull that typically ETL have ordering constraints and doing several things at the same time should not violate theses constraints.
Edit : An exemple of this. You can configure Oracle to execute each requests using several cores or not. (Depending of configuration and licence if I understand right). So if one request is allowed to use only one core, using several connections at the same time will allow several requests as the same time and better use the CPU resources of the server.
I have to write an architecture case study but there are some things that i don't know, so i'd like some pointers on the following :
The website must handle 5k simultaneous users.
The backend is composed by a commercial software, some webservices, some message queues, and a database.
I want to recommend to use Spring for the backend, to deal with the different elements, and to expose some Rest services.
I also want to recommend wicket for the front (not the point here).
What i don't know is : must i install the front and the back on the same tomcat server or two different ? and i am tempted to put two servers for the front, with a load balancer (no need for session replication in this case). But if i have two front servers, must i have two back servers ? i don't want to create some kind of bottleneck.
Based on what i read on this blog a really huge charge is handle by one tomcat only for the first website mentionned. But i cannot find any info on this, so i can't tell if it seems plausible.
If you can enlight me, so i can go on in my case study, that would be really helpful.
Thanks :)
There are probably two main reasons for having multiple servers for each tier; high-availability and performance. If you're not doing this for HA reasons, then the unfortunate answer is 'it depends'.
Having two front end servers doesn't force you to have two backend servers. Is the backend going to be under a sufficiently high load that it will require two servers? It will depend a lot on what it is doing, and would be best revealed by load testing and/or profiling. For a site handling 5000 simultaneous users, though, my guess would be yes...
It totally depends on your application. How heavy are your sessions? (Wicket is known for putting a lot in the session). How heavy are your backend processes.
It might be a better idea to come up with something that can scale. A load-balancer with the possibility to keep adding new servers for scaling.
Measurement is the best thing you can do. Create JMeter scripts and find out where your app breaks. Built a plan from there.
To expand on my comment: think through the typical process by which a client makes a request to your server:
it initiates a connection, which has an overhead for both client and server;
it makes one or more requests via that connection, holding on to resources on the server for the duration of the connection;
it closes the connection, generally releasing application resources, but generally still hogging a port number on your server for some number of seconds after the conncetion is closed.
So in designing your architecture, you need to think about things such as:
how many connections can you actually hold open simultaneously on your server? if you're using Tomcat or other standard server with one thread per connection, you may have issues with having 5,000 simultaneous threads; (a NIO-based architecture, on the other hand, can handle thousands of connections without needing one thread per connection); if you're in a shared environment, you may simply not be able to have that many open connections;
if clients don't hold their connections open for the duration of a "session", what is the right balance between number of requests and/or time per connection, bearing in mind the overhead of making and closing a connection (initialisation of encrypted session if relevant, network overhead in creating the connection, port "hogged" for a while after the connection is closed)
Then more generally, I'd say consider:
in whatever architecture you go for, how easily can you re-architecture/replace specific components if they prove to be bottlenecks?
for each "black box" component/framework that you use, what actual problem does it solve for you, and what are its limitations? (Don't just use Tomcat because your boss's mate's best man told them about it down the pub...)
I would also agree with what other people have said-- at some point you need to not be too theoretical. Design something sensible, then run a test bed to see how it actually copes with your expected volumes of data. (You might not have the whole app built, but you can start making predictions about "we're going to have X clients sending Y requests every Z minutes, and p% of those requests will take n milliseconds and write r rows to the database"...)