How Do I properly Manage mongo connections using multiple classes ?
For example I have 4 classes that manages 4 collections.
Collection1.class
Collection2.class
Etc..
What I do is creating a connect and close method in each class which by the time slows the connection of some transactions in the app
What would be the best way to connect the app to db once and start using all the classes instances other than creating object of each class and connecting each one separately ?
In the application, a single MongoClient object with a required number of connections, using connection pooling, will work in this case. The default value of the connection pool of 100, and can be modified (or configured) as needed.
The mongo client object can be created at the start of the application and is closed only when the application is closed. This saves the resources related to creating a connection with the mongo client objects in each collection access class.
The same mongo client object can be used throughout the application. A singleton class (which maintains one instance of the mongo client object) can be accessed by any other object in the application which needs a connection to the MongoDB database server.
What is connection pooling?
In software engineering, a connection pool is a cache of database
connections maintained so that the connections can be reused when
future requests to the database are required. Connection pools are
used to enhance the performance of executing commands on a database.
Opening and maintaining a database connection for each user,
especially requests made to a dynamic database-driven website
application, is costly and wastes resources. In connection pooling,
after a connection is created, it is placed in the pool and it is used
again so that a new connection does not have to be established. If all
the connections are being used, a new connection is made and is added
to the pool. Connection pooling also cuts down on the amount of time a
user must wait to establish a connection to the database.
Example Code:
/*
* Manages the MongoClient object and its settings like host, port, connection pool, etc.
*/
public class DBAccess {
private static MongoClient mongoClient;
private static DBAccess dbAccess;
// MongoClient with default settings
// NOTE: the code will have only one of the constructors
//private DBAccess() {
// final String connectionString = "mongodb://localhost:27017";
// this.mongoClient = MongoClients.create(connectionString);
//}
// MongoClient with custom settings.
// Private constructor, so that the class can be instantiated outside this class.
// NOTE: the code will have only one of the constructors
private DBAccess() {
MongoClientSettings settings =
MongoClientSettings.builder()
.applyToConnectionPoolSettings(builder ->
builder.maxSize(40).minSize(10))
.applyToClusterSettings(builder ->
builder.hosts(Arrays.asList(new ServerAddress("localhost", 27017))))
.build();
mongoClient = MongoClients.create(settings);
}
public static MongoClient getConnection() {
if (dbAccess == null) {
dbAccess = new DBAccess();
}
return mongoClient;
}
public static void closeDatabase() {
mongoClient.close();
}
}
/*
* Class manages a collection.
*/
public class CollectionOneAccess {
public static String COLLECTION_ONE = "collection_one";
private MongoCollection<Document> collection;
public CollectionOneAccess(MongoDatabase db) {
collection = db.getCollection(COLLECTION_ONE);
}
public void printOneDocument() {
Document myDoc = collection.find().first();
System.out.println(myDoc.toJson());
}
// other CRUD operations ...
}
// Usage of DBAcess and CollectionOneAccess classes:
private static final String APP_DATABASE = "abc_db";
public static void main(String [] args) {
MongoDatabase database = DBAccess.getConnection().getDatabase(APP_DATABASE);
CollectionOneAccess one = new CollectionOneAccess(database);
one.printOneDocument();
// ...
}
Mongo Client
MongoClient object is used to connect to the MongoDB server, get access to a database using the getDatebase() method and work with collections.
com.mongodb.client.MongoClient interface:
A client-side representation of a MongoDB cluster. Instances can
represent either a standalone MongoDB instance, a replica set, or a
sharded cluster. Instance of this class are responsible for
maintaining an up-to-date state of the cluster, and possibly cache
resources related to this, including background threads for
monitoring, and connection pools.
From the MongoDB Java documentation:
The MongoClient instance represents a pool of connections to the database; you will only need one instance of class MongoClient even with multiple threads.
IMPORTANT: Typically you only create one MongoClient instance for a
given MongoDB deployment (e.g. standalone, replica set, or a sharded
cluster) and use it across your application. However, if you do create
multiple instances:
All resource usage limits (e.g. max connections, etc.) apply per MongoClient instance.
To dispose of an instance, call MongoClient.close() to clean up resources.
The following code creates a MongoDB client connection object with default settings, like the host ("localhost") and port (27017), connection pooling, etc., and connects to a MongoDB instance and gets access to the testDB database.
MongoClient mongoClient = MongoClients.create();
MongoDatabase database = mongoClient.getDatabase("testDB");
Mongo Client Settings:
You can explicitly specify other settings with the MongoClientSettings to control the behavior of a MongoClient.
MongoClient mongoClient = MongoClients.create(MongoClientSettings settings)
The ConnectionPoolSettings object specifies all settings that relate to the pool of connections to a MongoDB server. The application creates this connection pool when the client object is created. ConnectionPoolSettings.Builder is a builder for ConnectionPoolSettings, has methods to specify the connection pool properties. E.g., maxSize(int maxSize): The maximum number of connections allowed (default is 100). Other methods include, minSize, maxConnectionIdleTime, etc.
Code to instantiate a MongoClient with connection pool settings:
MongoClientSettings settings = MongoClientSettings.builder()
.applyToConnectionPoolSettings(builder ->
builder.maxSize(20))
.build();
MongoClient mongoClient = MongoClients.create(settings);
// ...
// Verify the connection pool settings
System.out.println("Pool size: " +
settings.getConnectionPoolSettings().getMaxSize());
You are correct that each class (representing a MongoDB collection) should not be managing its own connection to the database. Rather you should be passing the database connection into the class - generally in the constructor. Something like this:
class Animal {
private String species;
private String name;
private int age;
public Animal(DBObject dbObject) { ... }
}
class AnimalCollection {
private final DBCollection collection;
public AnimalCollection(Database database) {
collection = database.getCollection("animals");
}
public List<Animal> getAll() {
List<Animal> animals
try (DBCursor cursor = collection.find(query)) {
while (cursor.hasNext()) {
animals.add(new Animal(cursor.next());
}
}
return animals;
}
}
Your code to create all the collections should get the MongoClient, connect to the DB and managing closing the connection at exit. That way you have a single connection you manage.
So the class to manage the collections might look like:
class CollectionManager implements AutoCloseable {
private final Database database;
private final AnimalCollection animals;
public CollectionManager(MongoClient client) {
database = client.getDB("Zoo");
animals = new AnimalCollection(database);
}
#Override
public void close() {
database.close();
}
}
The reason to have this class extend AutoCloseable is that close is called automatically when you exit a try-with-resources block. That'll make your code easier to read and safer.
This approach has another big advantage. You can unit test your classes by passing a mocked Database in the constructor and test behaviour in response to various DB outputs without needing an actual DB with any data in it.
Related
Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 8 years ago.
Improve this question
I'm developing a java based application using NetBeans. My app opens with a window with asks the user to enter their credentials and based on data entered, a connection is established between the app and my MySQL client (I'm using JDBC for this purpose).
My issue: I want the connection object (which is declared and initialized after checking the credentials of the user) to be available for use in all my form. Previously, i have being doing this by passing the connection object from one form to another. But i don't want to do that! I want once this connection object is declared, it's made available to all the forms in the app.
I want the connection object (...) to be available for use in all my form
You should not have an open connection while your application lives. Instead, use a database connection pool of 1 or 2 connections that will be available for all the application and add a shutdown hook to close this data source when the application finishes. The connection pool will take care to maintain the connections alive and use low resources for it.
For example: your user opens the application and enters its credentials, then leaves the room because he/she has to do some paperwork and takes 30 mins, then goes back to the pc and try to use an option. If using a static Connection con object, you manually opened a physical connection to the database and you're in charge to control the connectivity for all these 30 minutes, and if you don't do any action in that time then probably the physical connection was closed by the database engine. If using a connection pool, this will take care of opening/closing physical connections and maintaining them in sleep state so your connection won't be lost.
Note that your Connection object and related resources (PreparedStatement, ResultSet, etc). should be in the narrowest possible scope.
Here's a minimal example of doing this using BoneCP as database connection pool.
public class ConnectionProvider {
private static DataSource dataSource;
private static boolean initialized = false;
public static void init(Map<String, String> conf) {
if (!initialized) {
//synchronization to avoid multiple threads accesing to this part of the method
//at the "same time"
synchronized(DataSourceProvider.class) {
//double validation in case of multi threaded applications
if (!initialized) {
//you may add more validations here
//in case you want to use another datasource provider
//like C3PO, just change this part of the code
BoneCPDataSource bds = new BoneCPDataSource();
bds.setDriverClass(conf.get("driver"));
bds.setJdbcUrl(conf.get("url"));
bds.setUsername(conf.get("user"));
bds.setPassword(conf.get("password"));
//this should be obtained as configuration parameter
bds.setMaxConnectionsPerPartition(2);
//you can add more BoneCP specific database configurations
dataSource = bds;
initialized = true;
}
}
}
}
public static Connection getConnection() {
if (dataSource == null) {
//this should be a custom exception in your app
throw new RuntimeException("Data Source was not initialized.");
}
return dataSource.getConnection();
}
}
And the client (once you have called the init method and provided the database configurations). I'm avoiding exception handling for brevity:
public class SomeDao {
private Connection con;
//using Dependency Injection by composition for DAO classes with connection
public SomeDao(Connection con) {
this.con = con;
}
public SomeEntity getSomeEntity(int id) {
String sql = "SELECT id, col1, col2 FROM someEntity WHERE id = ?";
//PreparedStatement and ResultSet go on the narrowest possible scope
PreparedStatement pstmt = con.prepareStatement(sql);
pstmt.setInt(1, id);
ResultSet rs = pstmt.executeQuery();
SomeEntity someEntity = new SomeEntity();
if (rs.hasNext()) {
someEntity.setId(rs.getInt("id");
//similar for other columns...
}
//don't forget to close the resources after its usage
return someEntity;
}
}
public class SomeService {
public SomeEntity getSomeEntity(int id) {
//retrieving the connection at this level
//a service may access to several daos
Connection con = ConnectionProvider.getConnection();
//performing the operations against DAO layer
SomeDao someDao = new SomeDao(con);
SomeEntity someEntity = someDao.getSomeEntity(id);
//closing the connection. This is A MUST
//here the connection pool won't close the physical connection
//instead put it to sleep
con.close();
//return the proper data at a single point of the method
return someEntity;
}
}
Don't use the same Connection in your application! But what you want to achieve could be done using static variable. For example, add the following code to any of your classes, or create a new class for it:
private static Connection con = null;
public static Connection getConnection (String url)
{
if (con == null)
con = DriverManager.getConnection(url);
return con;
}
Then, call MyClass.getConnection("jdbc:mysql://localhost:3306/") or whatever the connection string is, and it will return one Connection that you could use for all classes.
For university, it is my excercise to develop a multiplayer game with Java. The communication between the clients shall not be handled with sockets or the like, but with the help of a MySQL database where the clients are adding their steps in the game. Because it is a game of dice, not a lot of queries are needed. (approximiately 30 queries per gaming session are needed).
I never used MySQL in connection with Java before, so this maybe is a beginner's fault. But actually, I often get an exception during the execution of my java project.
com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: User my_username already has more than 'max_user_connections' active connections
My queries are executed in a DatabaseHelper.java class. The results are returned and evaluated in another class of the project. Since I use an MVC pattern, I evaluate the results in a controller or model class.
This for example is one of my quers in the DatabaseHelper.java class. The other queries are similar:
private static Connection conn;
private Connection getConn() {
return conn;
}
public void db_connect() throws ClassNotFoundException, SQLException{
// JDBC Klassen laden
Class.forName(dbClassName);
// Verbindungsversuch auf 5 Sekunden setzen
DriverManager.setLoginTimeout(5);
this.setConn(DriverManager.getConnection(CONNECTION,p)); // p contains the username and the database
}
public void db_close(){
try {
this.getConn().close();
} catch (SQLException e) {
if(GLOBALVARS.DEBUG)
e.printStackTrace();
}
}
public String[] query_myHighscores(int gameid, PlayerModel p) throws SQLException{
List<String> rowValues = new ArrayList<String>();
PreparedStatement stmnt;
if(gameid == GLOBALVARS.DRAGRACE)
stmnt = this.getConn().prepareStatement("SELECT score FROM highscore WHERE gid = ? and pname = ? ORDER BY score ASC LIMIT 0,3");
else
stmnt = this.getConn().prepareStatement("SELECT score FROM highscore WHERE gid = ? and pname = ? ORDER BY score DESC LIMIT 0,3");
stmnt.setInt(1, gameid);
stmnt.setString(2, p.getUname());
ResultSet rs = stmnt.executeQuery();
rs.beforeFirst();
while(rs.next()){
rowValues.add(rs.getString(1));
}
stmnt.close();
rs.close();
return (String[])rowValues.toArray(new String[rowValues.size()]);
}
The CONNECTION string is a string which looks like jdbc:mysql://my_server/my_database
In the HighscoreGUI.java class, I request the data like this:
private void actualizeHighscores(){
DatabaseHelper db = new DatabaseHelper();
try{
db.db_connect();
String[] myScoreDragrace = db.query_myHighscores(GLOBALVARS.GAME1); // id of the game as parameter
// using the string
} finally {
db.db_close();
}
So I tried:
Closing the statement and the ResultSet after each query
Used db_close() to close the connection to the dabase in the finally-block
Never returning a ResultSet (found out this may become a performance leak)
The stacktrace leads in the DatabaseHelper.java class to the line
this.setConn(DriverManager.getConnection(CONNECTION,p));
But I cannot find my mistake why I still get this exception.
I cannot change every settings for the database since this is a shared host. So I'd prefer a solution on Java side.
The problem is that you exceed your allowed set of connections to that database. Most likely this limit is exactly or very close to "1". So as soon as you request your second connection your program crashes.
You can solve this by using a connection pooling system like commons-dbcp.
That is the recommended way of doing it and the other solution below is only if you may not use external resources.
If you are prohibited in the external code that you might use with your solution you can do this:
Create a "Database" class. This class and only this class ever connects to the DB and it does so only once per program run. You set it up, it connects to the database and then all the queries are created and run through this class, in Java we call this construct a "singleton". It usually has a private constructor and a public static method that returns the one and only instance of itself. You keep this connection up through the entire livetime of your program and only reactivate it if it gets stall. Basically you implement a "Connection Pool" for the specific case of the pool size "1".
public class Database {
private static final Database INSTANCE = new Database();
private Database() {}
public static Database getInstance() {
return INSTANCE;
}
// add your methods here.
}
When the program terminates, close the Connection (using a shutdown hook).
I am developing a java web service as well as a java client for the service and I am wondering if my design is even possible within the constraints of a SOAP-based, document style web service. On the server side I have a database which I am using a utility class to connect to and a couple other database "handler" classes that manipulate the database. My web service has WebMethods that return the handler classes. Each of the handler classes is annotated with the #XmlType annotation so that they can be bound to XML types. My question is if my client calls one of the webmethods and gets one of the database handler can I in turn manipulate the database on the server side through the methods of the handler (which are not annotated #WebMethod). Is something like this possible with SOAP-based web services?
I have example code below.
Here is the functionality of my utility class.
public class DBUtil {
public static getConnection() {
return DriverManager.getConnection(url, username, password);
}
}
Here is an example of one of my database handlers.
#XmlType
public class Handler1 {
public LinkedList<String> listDatabases() {
Connection con = DBUtil.getConnection();
LinkedList<String> list = new LinkedList<String>();
//use con to query database and fill list with know database names
return list;
}
public LinkedList<UserData> listUser(String database) {
//UserData is a user defined type that store information about user names
//and their privileges
Connection con = DBUtil.getConnection();
//query database and fill a LinkedList list with one UserData object
//for each database user
return list;
}
}
My other handler classes contain similar methods for querying and modifying tables.
Below is my web service.
#WebService()
#SOAPBinding(style = SOAPBinding.Style.DOCUMENT)
public class WSDatabaseImp implements WSDatabase {
//WSDatabase is my SEI
public WSDatabaseImp() {}
#WebMethod
public Handler1 getHandler1() {
return new Handler1();
}
//more get HandlerX methods
}
What I want to do is use Handler1 and such in my client to manipulate the database. The code below demonstrates what I want to do and assumes that I have run run wsimport that generate the artifacts.
public class Client {
public static void main(String[] args) {
WSDatabaseImpService service = new WSDatabaseImpService();
WSDatabaseImp port = service.getWSDatabaseImpPort();
Handler1 h = service.getHandler1();
LinkedList<String> list = h.listDatabases();
for(String s: list)
System.out.println(s);
}
}
Is this design possible? Thank you so much for your help.
First i show you the code then asked few questions. i have a class database connectivity like this (please ignore syntax error if any)
class DatabaseConnection {
private static Connection connection = null;
private static String driverName="";
private static String userName="";
private static String passwrod="";
private static String url="";
private DatabaseConnection() { }
public static void createConnection() {
if ( connection == null ) {
// read database credentials from xml file and set values of driverName, userName, passowrd and url
//create connection with database and set store this connection in connection object created a class level.
}
}
public static void closeConnection1() throws Exception{
if ( connection != null ) {
connection.close();
connection == null
}
}
public static void closeConnection2() throws Exception{
if ( connection != null ) {
connection.close();
}
}
public void insertData(Object data) {
// insetData in database
}
}
I want to know which close connection is more optimize in database connection. Lets suppose I have test class like this
class Test {
public static void main(String args[]) {
DatabaseConnection.createConnection();
DatabaseConnection.insertData(data);
DatabaseConnection.closeConnection2(); // we call also called close connection method within the insertData method after inserting the data
}
}
After creating database connection i insert data in database and then close the connection using closeConnection2 method. in this way connection has been close so if i want to insert some more method then i have to recreate connection with the database but i can't do this because connection object is not null and createConnection didn't execute the code inside the if statement. Now if I called closeConnection1 method for closing connection then in doing this i have to parse xml file again for credential which is not a optimize solution. can you tell me which method is good and if both are worse then please tell me more efficient way for creating and closing database connection.
I see two major problems with this:
The fact that everything (including the Connection object) is static means that you can't ever use this class from more than one thread at once.
parsing the configuration data and opening the connection are separate concerns and should not be mixed. At least move them into separate methods, the configuration could probably even go in another class.
The second thing alone will avoid having to parse the connection information multiple times.
An even better approach would be to use a DataSource instead of opening the connections each time. And then use a DataSource that's actually a connection pool!
I am in the process of designing a simple Java application which deals with insert/delete/update of records in a MySQL database using JDBC. I have a class Member which deals with the member's details.
class Member {
... // Private Members
... // Accessors
}
..and I have a handler to deal with the member records
class MemberHandler {
public MemberHandler(){...}
public void addMember(Member mem){...}
public void removeMember(Member mem){...}
public Member[] getMembers(){...}
}
All I am worried about is the method in which I establish connection to the database and disconnect. I can do it in two ways -
Method 1:
I can have a member in MemberHandler Connection conn, establish connection while instantiating the class and close the connection when the object is no more needed. Here I would have a connection per object and I need not establish a connection whenever I need to do any database related activity. In this case, the disadvantage seems to be - when there is a loss in network connection, conn could become invalid.
class MemberHandler {
private java.sql.Connection conn;
... // Other members
private void createConnection(){/*creates the connection*/}
private void closeConnection(){conn.close(); /*called when conn is no more needed*/}
}
Method 2:
I can establish a connection when needed and close it when I am done with the activity. Disadvantage: everytime, I need to establish a connection and close it. For eg.,
...
...
private void addMember() {
//establish connection
//update database
//close connection
}
...
...
Which of these two ways seems to be better? Or is there a third better way?
Thanks!
I would suggest you to go for connection-pooling
c3po