<property name="hibernateProperties">
<props>
<prop key="hibernate.c3p0.max_size">?</prop>
</props>
</property>
is it just a random number guess ? or are there any studies that suggest to use a particular range for a specific use case?
Well, assuming that you are talking about a production system, then "random number guess" is definitely not the answer. Actually, "random number guess" is never the answer for any sort of configuration in a production environment. Same goes for "just accepting the product's defaults".
Connection pool properties (such as max_size) depend on a few factors:
Anticipated use. For example, if you are expecting a typical usage pattern of 10 concurrent users, with an occasional burst of 20 users, there is little point in setting the number 50 as a maximum size even if you think that your machine can handle it. While it seems harmless at first sight, you have to remember that database connections are an expensive resource and, sometimes, you might want to actually cap the usage to match your expectation, if only to put your own assumptions into test and getting to know the real, production-like, usage pattern of the system and to prevent your application from being a resource hog, potentially affecting other applications.
Available resources. If you know (and that is easy to verify) that your database can only accept 30 connections at a time, then setting a number larger than 30 is senseless, regardless of the application's usage pattern.
Application design. Is your application going to mainly use short-lived connections? long-lived connections? Are you setting up a timeout for JDBC calls, so your JDBC calls are time-limited to begin with? For example, there's a difference in how you would configure a connection pool when you know that you're setting a timeout of 30 seconds per operation, comparing to how you would define that pool knowing that you set a timeout of 2 minutes.
Specific product considerations. I am not sure about c3p0, but certain containers that provide connection pooling mechanisms carry their own factors into the equation. If you are using the connection pooling functionality provided by a container, you should read that container's documentation to see whether the container's vendor has some insight with regards to configuring the connection pooling mechanism they provide you with.
... Just don't guesstimate a number.
... And don't just assume product's defaults.
Related
Essentially this part of this question:
-> It seems ehcache/ hibernate creates 1 cache per Entity. This is fine, however when replication is in place it create 1 thread / cache
to replicate. Is this the intended behavious ? As our domain is big,
it creates about 300 threads, which seems to me really big
Is there no option to pool or consolidate the number of replication threads?
The code around replication has not changed in a number of years, so the limitation is still there. As described in the accepted answer to the linked question you will have to code what you need yourself.
Now Ehcache considers replication to be deprecated and favours Terracotta clustering in order to provide some consistency guarantees. Details of that are beyond the scope of the question.
Trying to understand the best possible Java ThreadPoolTaskExecutor that I can define when transferring to OkHttpClient, latency wise. Currently our definition is the following:
<property name="corePoolSize" value="#{ T(java.lang.Math).max(32,numCpu) * 2 }" />
<property name="maxPoolSize" value="#{ T(java.lang.Math).max(32,numCpu) * 8 }" />
<property name="queueCapacity" value="200"/>
That is maximal queue capacity (at which new Thread will be opened) is 200, minimal thread count is max(32,numCpu) * 2 and maximal thread count is max(32,numCpu) * 8. In our case numCpu could vary from 16 to 24 (though if hyper-threading is taken into account then multiply that number *2, right? ).
But when you think about it - I am not sure that the number of Threads here should be somehow connected to CPU count. Those are sending/receiving threads of HTTP client, not BusinessLogic threads. So perhaps CPU count shouldn't be even factor here.
Any opinions/advice?
It sounds to me like your thread pool is being used to concurrently make lots of HTTP connections, meaning your performance is limited not by CPU usage but by I/O (and potentially memory too). The "optimal" number of threads is going to be limited by a number of other factors ...
1. Link speed between your client and endpoints.
Let's say your client is connected to a 1Gbps link but somewhere down the line, all of your endpoints can only serve you data at 1Mbps. To max out your local bandwidth, you would need to run 1000 connections concurrently to fully utilize your 1Gbps link, meaning your thread pool needs to run 1000 threads. But this could be problematic as well because of another issue ...
2. Memory usage per-thread is non-zero, even if they aren't doing anything intensive.
The default amount of stack space allocated to a Java varies by vendor, but it's on the order of 1MB. This doesn't sound like a whole lot, but if you need to run thousands of threads to keep as many client connections active at a time, you will need to allocate gigabytes of RAM for the stack space alone. You can adjust the stack space allocated per thread using the -Xss[size] VM argument, but this is global to the VM, so shrinking the stack size may cause problems in other areas of your program, depending on what you are doing.
3. Average HTTP request size.
Sometimes, it's going to boil down to how much data you expect to transfer per POST/GET call. Recall that each TCP connection requires an initial handshake before any data can be sent. If the amount of data you expect to transmit over the life of an HTTP call is very small, you may not be able to keep thousands of connections running concurrently, even if you have thousands of threads at your disposal. If the amount is very large, it may only take a few concurrent connections to max out the total bandwidth available to your client.
Finally ...
You may not be able to predict the link speed of every connection if all of your endpoints are running out there on the web. I think the best you can do is benchmark the performance of different configurations, while considering each of these factors, and choose the configuration that seems to give the best performance in your typical operating environment. It will likely be somewhere between N and 1000, where N is the number of cores you run, but nailing that number down to something specific will take a little bit of elbow grease :)
<property name="hibernateProperties">
<props>
<prop key="hibernate.c3p0.max_size">?</prop>
</props>
</property>
is it just a random number guess ? or are there any studies that suggest to use a particular range for a specific use case?
Well, assuming that you are talking about a production system, then "random number guess" is definitely not the answer. Actually, "random number guess" is never the answer for any sort of configuration in a production environment. Same goes for "just accepting the product's defaults".
Connection pool properties (such as max_size) depend on a few factors:
Anticipated use. For example, if you are expecting a typical usage pattern of 10 concurrent users, with an occasional burst of 20 users, there is little point in setting the number 50 as a maximum size even if you think that your machine can handle it. While it seems harmless at first sight, you have to remember that database connections are an expensive resource and, sometimes, you might want to actually cap the usage to match your expectation, if only to put your own assumptions into test and getting to know the real, production-like, usage pattern of the system and to prevent your application from being a resource hog, potentially affecting other applications.
Available resources. If you know (and that is easy to verify) that your database can only accept 30 connections at a time, then setting a number larger than 30 is senseless, regardless of the application's usage pattern.
Application design. Is your application going to mainly use short-lived connections? long-lived connections? Are you setting up a timeout for JDBC calls, so your JDBC calls are time-limited to begin with? For example, there's a difference in how you would configure a connection pool when you know that you're setting a timeout of 30 seconds per operation, comparing to how you would define that pool knowing that you set a timeout of 2 minutes.
Specific product considerations. I am not sure about c3p0, but certain containers that provide connection pooling mechanisms carry their own factors into the equation. If you are using the connection pooling functionality provided by a container, you should read that container's documentation to see whether the container's vendor has some insight with regards to configuring the connection pooling mechanism they provide you with.
... Just don't guesstimate a number.
... And don't just assume product's defaults.
How would you go about dynamically configuring the maximum number of connections in a DB connection pool?
I've all but given up on using a "hard coded" (configuration file, but still) number of connections. Some of the time, more connections provide better performance. On other times, less connections do a better job.
What measurement would you use to determine if you've opened too many connections and are actually hurting performance by it? Please keep in mind I can't just "stop the world" to run a performance test - I need something that I could my own query responses (of which I have no specific measurement - some are slow, some are fast, and I can't know in advance which is which) to determine.
(please note I'm using Java JDBC with underlying DataDirect drivers)
Is this approach used somewhere (and was it successful)? If not, how would you go about solving the "what is the optimal number of connections" when you have to support both Oracle and MS SQL, both for several versions and the queries vary wildly in nature (indexed lookup / non-indexed lookup / bulk data fetching / condition matching (indexed and non indexed, with and without wildcards))?
[I know this is similar to optimal-number-of-connections-in-connection-pool question, but I'm asking about dynamic configuration while he's asking about static one]
If you queue users to wait for a free database connection, they are waiting on something unknown or artificially imposed.
If you let them through to the database, you'll at least find out what resource is being fought over (if any). For example, if it is disk I/O, you may move your files around to spread activity against more or different disks or buy some SSD or more memory for cache. But at least you know what your bottleneck is and can take steps to address it.
If there is some query or service hogging resource, you should look into resource manager to segregate/throttle those sessions.
You probably also want to close off unused sessions (so you may have a peak of 500 sessions at lunch, but drop that to 50 overnight when a few bigger batch jobs are running).
You need free flowing connection pool which auto adjusts according to the load. So it should have:-
1) Min size: 0
2) Max size: as per ur DB configuration
3) increment by 1 if available connections are out of stock
4) abandon connection if it is idel for X (configured time) seconds
5) Connection pool should release the abandoned connections.
Witht this settings the connection pool should manage the number of connections based on the load dynamically.
closing to lack of interest. We ended up using a high maximal value and it didn't seem to bother the DB much.
I'm in charge of developing and maintaining a group of Web Applications that are centered around similar data. The architecture I decided on at the time was that each application would have their own database and web-root application. Each application maintains a connection pool to its own database and a central database for shared data (logins, etc.)
A co-worker has been positing that this strategy will not scale because having so many different connection pools will not be scalable and that we should refactor the database so that all of the different applications use a single central database and that any modifications that may be unique to a system will need to be reflected from that one database and then use a single pool powered by Tomcat. He has posited that there is a lot of "meta data" that goes back and forth across the network to maintain a connection pool.
My understanding is that with proper tuning to use only as many connections as necessary across the different pools (low volume apps getting less connections, high volume apps getting more, etc.) that the number of pools doesn't matter compared to the number of connections or more formally that the difference in overhead required to maintain 3 pools of 10 connections is negligible compared to 1 pool of 30 connections.
The reasoning behind initially breaking the systems into a one-app-one-database design was that there are likely going to be differences between the apps and that each system could make modifications on the schema as needed. Similarly, it eliminated the possibility of system data bleeding through to other apps.
Unfortunately there is not strong leadership in the company to make a hard decision. Although my co-worker is backing up his worries only with vagueness, I want to make sure I understand the ramifications of multiple small databases/connections versus one large database/connection pool.
Your original design is based on sound principles. If it helps your case, this strategy is known as horizontal partitioning or sharding. It provides:
1) Greater scalability - because each shard can live on separate hardware if need be.
2) Greater availability - because the failure of a single shard doesn't impact the other shards
3) Greater performance - because the tables being searched have fewer rows and therefore smaller indexes which yields faster searches.
Your colleague's suggestion moves you to a single point of failure setup.
As for your question about 3 connection pools of size 10 vs 1 connection pool of size 30, the best way to settle that debate is with a benchmark. Configure your app each way, then do some stress testing with ab (Apache Benchmark) and see which way performs better. I suspect there won't be a significant difference but do the benchmark to prove it.
If you have a single database, and two connection pools, with 5 connections each, you have 10 connections to the database. If you have 5 connection pools with 2 connections each, you have 10 connections to the database. In the end, you have 10 connections to the database. The database has no idea that your pool exists, no awareness.
Any meta data exchanged between the pool and the DB is going to happen on each connection. When the connection is started, when the connection is torn down, etc. So, if you have 10 connections, this traffic will happen 10 times (at a minimum, assuming they all stay healthy for the life of the pool). This will happen whether you have 1 pool or 10 pools.
As for "1 DB per app", if you're not talking to an separate instance of the database for each DB, then it basically doesn't matter.
If you have a DB server hosting 5 databases, and you have connections to each database (say, 2 connection per), this will consume more overhead and memory than the same DB hosting a single database. But that overhead is marginal at best, and utterly insignificant on modern machines with GB sized data buffers. Beyond a certain point, all the database cares about is mapping and copying pages of data from disk to RAM and back again.
If you had a large redundant table in duplicated across of the DBs, then that could be potentially wasteful.
Finally, when I use the word "database", I mean the logical entity the server uses to coalesce tables. For example, Oracle really likes to have one "database" per server, broken up in to "schemas". Postgres has several DBs, each of which can have schemas. But in any case, all of the modern servers have logical boundaries of data that they can use. I'm just using the word "database" here.
So, as long as you're hitting a single instance of the DB server for all of your apps, the connection pools et al don't really matter in the big picture as the server will share all of the memory and resources across the clients as necessary.
Excellent question. I don't know which way is better, but have you considered designing the code in such a way that you can switch from one strategy to the other with the least amount of pain possible? Maybe some lightweight database proxy objects could be used to mask this design decision from higher-level code. Just in case.
Database- and overhead-wise, 1 pool with 30 connections and 3 pools with 10 connections are largely the same assuming the load is the same in both cases.
Application-wise, the difference between having all data go through a single point (e.g. service layer) vs having per-application access point may be quite drastic; both in terms of performance and ease of implementation / maintenance (consider having to use distributed cache, for example).
Well, excellent question, but it's not easy to discuss using a several data bases (A) approach or the big one (B):
It depends on the database itself. Oracle, e.g. behaves differently from Sybase ASE regarding the LOG (and therefore the LOCK) strategy. It might be better to use several different & small data base to keep lock contention rate low, if there is a lot of parallel writes and the DB is using a pessimistic lock strategy (Sybase).
If the table space of the small data bases aren't spread over several disks, it might better be using one big data base for using the (buffer/cache) memory only for one. I think this is rarely the case.
Using (A) is scales better for a different reason than performance. You're able moving a hot spot data base on a different (newer/faster) hardware when needed without touching the other data bases. In my former company this approach was always cheaper than variant (B) (no new licenses).
I personally prefer (A) for reason 3.
Design, architecture, plans and great ideas falls short when there is no common sense or a simple math behind the. Some more practice and/or experience helps ... Here is a simple math of why 10 pools with 5 connections is not the same as 1 pool with 50 connection:
each pool is configured with min & max open connections, fact is that it will usually use (99% of the time) 50% of the min number (2-3 in case of 5 min) if it is using more that that this pool is mis-configured since it is opening and closing connections all the time (expensive) ... so we 10 pools with 5 min connections each = 50 open connections... means 50 TCP connections; 50 JDBC connections on top of them ... (have you debug a JDBC connection? you will be surprise how much meta data flows both ways ...)
If we have 1 pool (serving the same infrastructure above) we can set the min to 30 simple because it will be able to balance the extras more efficiently ... this means 20 less JDBS connections. I don't know about you but for me this is a lot ...
The devil s in the detail - the 2-3 connections that you leave in each pool to make sure it doesn't open/close all the time ...
Don't even want to go in the overhead of 10 pool management ... (I do not want to maintain 10 pools every one ever so different that the other, do you?)
Now that you get me started on this if it was me I would "wrap" the DB (the data source) with a single app (service layer anyone?) that would provide diff services (REST/SOAP/WS/JSON - pick your poison) and my applications won't even know about JDBC, TCP etc. etc. oh, wait google has it - GAE ...