We have a java application that connects to mysql database. We are now going to change the database to replication mode by adding another DB instance. The idea is to provide DB high availability. The application should be able to switch to the standby DB in case it is unable to connect to the main database. One way to implement this is to maintain 2 sets of connections, keep monitoring both the databases and in case the app is unable to connect to the main DB, switch to the next set of connections and continue.
My question is is there a transparent way of switching the connections through mysql connector itself? OR is there any utility app that can sit between my app and the mysql connector and do this job?
To clarify , we are planning to do master - master replication. Both writes and reads happen frequently.
Yes, Connector/J (the MySQL JDBC driver) offers connection failover.
It's not trivial to set up, though. This should get you started.
http://dev.mysql.com/doc/refman/5.5/en/connector-j-usagenotes-j2ee-concepts-load-balancing-failover.html
#Mike Brant's point is good. If your data is write-infrequently / read-frequently, then you are best off only having your app write to a master DBMS, and read from a pool of slave DBMSs. It's good programming practice to use a different Connection for the parts of your app that write, and for those parts that will read the data. You can set up the read-only Connection with loadbalancing and failover, while leaving the write Connection pointing to the master.
Related
I plan on making a distributed application where 10-15 computers are connected in a LAN and there is one server where the database will be stored, also inside the LAN.
For the purpose of the question lets say the application will be made using Java and the database will be MySQL, but that is not yet decided.
So the question is, what do I need to realize this?
I have ofcourse worked with MySQL databases, but on a single computer so I am not quite sure how to connect the computers to the database server. Is it enough to setup the server local IP on all client computers and connect to it using the JDBC MySQL driver, and after that I can work with it like the DB is on my machine?
Also, do I need a server application to manage connections to the database, so that there are no conflicting entries? And if I do, then the connection part changes, because it needs to be established over the server application, so how do I do that?
I know it is a large question, but a lot of things seem unclear, because I have never attempted a project this big.
Thank you all!
Don't worry just made your application, you can connect to your database via an #IP, so just make a part of configuration in your application to change some information in future:
String db_url = "jdbc:mysql://192.168.0.1/db_test";
Is it enough to setup the server local IP on all client computers and
connect to it using the JDBC MySQL driver, and after that I can work
with it like the DB is on my machine
You don't need a server local of every computer, you can connect to the server directly.
Also, do I need a server application to manage connections to the
database, so that there are no conflicting entries? And if I do, then
the connection part changes, because it needs to be established over
the server application, so how do I do that?
If you are using a desktop application, the you don't need a server application, else if you are using a web application, yes you need one.
You need to shouse a server application, this dippend of your project or your company, there are free servers like GlassFish, Payara, Apache Tomcat, Wildfly and more.
Now the connection to your database, there are many ways to connect to your database, if you are using JPA to connect to your database, then you will hear
something about Entities, Facades, JNDI so this generally be configure in your server
application.
Hope you get an idea.
We do have a distributed (note: not microservices) system which connects to one huge database (PostgresSQL - 9.3.13). We had in the last year different outages on this DB. The DB is kind of HA as well and the virtual IP floats correctly to the slave.
Our application is using Java, and as soon as the Master is not reachable anymore, our application starts to scream (via logs, which is fine) but unfortunately does not reconnect to the new master.
How can someone do this in a more resilient way? The best would be if the connection is kept active and the CP does connect to the new master as soon as it is available and (best case) all running updates/inserts are rolled back and started again on the new master.
For me the problem seems to be the Virtual IP and the connection pool plainly does not reconnect.
We do use DBCP2 as a connection pool. The data layer is hibernate.
I have two independent Java apps which I would like to communicate with each other through an in-memory H2-DB. In theory very straight forward, but I cannot get the connection to work.
What I am trying to do:
I create an in-memory DB executing jdbc:h2:mem:test.
With the client(s), I try connecting to it. I tried jdbc:h2:tcp://localhost/~/test and similar connection strings, but all without success.
Is it possible to connect to an in-memory DB? What should the connection strings look like to make this work? Thanks a bunch.
After a ton of reading and trial-and-error with H2 options (http://h2database.com/html/features.html and http://h2database.com/html/advanced.html), I found that it is possible to access an in-memory database from multiple processes on the same machine (or remotely) using TCP/IP or SSL/TLS. The connection string for an in-memory database test is jdbc:h2:tcp://localhost/mem:test.
H2 database can be shared but not in memory, Your may please refer to official documentation:
http://h2database.com/html/features.html#auto_mixed_mode
First application with open it in embedded mode and other application will use server mode.
I am developing an online mobile game. I have several server machines running numerous instances of a Java socket server application.
Player data has to be stored somewhere (their profiles, items etc). I want to use the H2 database for this purpose.
Now, here's the tricky part: I want all the player data to be stored in the same H2 database. That is, all my server applications will access the data by remotely connecting to one particular machine over TCP, out of convenience.
The thing is, we are expecting a very large amount of clients on launch. For each client, a connection to the H2 database is created. The obvious concern here is whether one single H2 database process can handle so many connections concurrently.
From the website:
There is no limit on the number of database open concurrently per
server, or on the number of open connections.
Given the above fact, in theory, if our server machine has enough resources (memory, space, CPUs, etc), then yes, the H2 database should be able to handle as many concurrent connections as our resources allow.
But there is something unclear to me:
Does the H2 process create a thread for each remote connection? I ask this because I once read that in Windows (our VPS' OS), a thread is stored as a short type, and hence the max amount of threads an application can spawn is roughly 32,000 (I don't know the math they used to get that number). In that case, then the H2 process does have a limit of concurrent connections - which is troubling because I do indeed expect more than 32,000 clients connected.
Of course, it would seem wise to discard the idea of having one single H2 database for all my clients. But I'd like to know if the above statement is correct: can H2 handle more than 32,000 remote database connections?
Let take this by parts:
"Does the H2 process create a thread for each remote connection?"
An application should normally use one connection per thread. An H2 database synchronizes access to the same connection, but other databases may not do this.
"can H2 handle more than 32,000 remote database connections?"
If you want to access the same database at the same time from different processes or computers, you need to use the client / server mode. The JdbcConnectionPool class has the default maximum number of connections set to 10, but it provides a setter to change it if you want. In theory, you can set it to Integer.MAX_VALUE, but I don't think this is wise. Why? For starters, the synchronization point made on the previous section. Another point to consider is if your application opens and closes connections a lot (for example, for each request), you should consider using a connection pool. Opening a DB connection is very slow.
"Of course, it would seem wise to discard the idea of having one single H2 database for all my clients"
It might be, but you have to keep in mind that the number of open database is limited by the memory available. If you are running on a powerful server, it might be a good option to consider. Then again, it might not.
I have a very short Java application that just opens a connection to a remote MySQL database, reads some data, prints it, and exits. The most time-consuming part of the application is the database connection.
Currently I have only a single thread, and my only concern is to save the time of opening the connection.
I thought of several ways to make it faster, but it turned out they do not help:
Connection Pooling - doesn't help because the pool lives only only during a single run of the application. When the application is terminated, the pool is gone, and when I re-run the application, I have to re-open all the connections in the pool.
mysql-proxy - connects only to the local server: mysql-proxy for a remote MySQL server
TCP/IP server - I thought of holding a local TCP/IP server that will keep a persistent open connection and send it to a TCP/IP client on request. However, Connection objects cannot be serialized, so I have no way to pass the Connection object from client to server.
Any other option?
Generally connection to a DB is a most time-consuming operation. If the application is to be started and stopped then there is little that you can do.
Using connection-pooling in a web-server and call that by running your app which talks to the web server using JSON might be an option.
You said you have a very short application so your 3rd option might work if you put the database logic into you "option 3 TCP/IP server" and just forward the results to your connecting client. This is a typical application server pattern.
Another thing you should consider about network look up https://stackoverflow.com/q/3641155/1055715 which Marc B has mentioned in his comment.
It turns out the best solution is to use mysql-proxy with a script that handles connection pooling (a combination of my first two options). I found one such script here:
http://forge.mysql.com/tools/tool.php?id=151
It was probably written for an older version of mysql-proxy, so I had to fix it (if anyone need the fixed version - write me).
It works like a charm - I run the exact same application as before, the only change is in the connection string: instead of connecting to "qa-srv:3308" (the remote server) I connect to "127.0.0.1:4040" (the proxy server).