I am writing a moderately complex Java desktop app, including an embedded database. I do not see any reason why, after the app establishes a connection to the database, why it should close the connection until the app is going to shut down.
Practically everything one does with the database requires a connection; transactions can be started and completed serially in the connection, the app is not doing anything fantastically complicated with the database.
Is there any reason why I should not create the connection and put a reference to it in a static variable in a class known and used by database-specific classes? It would save having the connection have to be passed around among all kinds of methods without ever changing value.
Is there a design-level consideration I'm missing somewhere?
rc
I would suggest using a library such as c3p0 or dbcp which handles connection pooling for you. It gives you the flexibility to scale up your application later if necessary.
Anything static usually makes it harded to write proper test cases, since you never know if the static resource has been altered or not.
Three months down the road, you're going to want to be able to connect to two databases at the same time - maybe you're doing some import / export work, or an upgrade job, or merging two customers together. And then you're going to want two of them. And now suddenly that static field everyone uses is a nightmare.
You could look into an IoC container like Guice or Spring to ensure that you can keep track of "singleton" objects without abusing static fields to enforce their "Singleton"ness.
Avoid statics. Think on concurrency and multithread issues with this kind of variables. A good point is handle your connections with a database pool. Spring is your friend to reach a simple and nice configuration
I do not see any reason why, after the app establishes a connection to the
database, why it should close the connection until the app is going to shut down.
That seems completely fine to me. It's an embedded database; it is at the service
of your application. Create the connection when you start, use it as long as you
need, shut it down when your application closes down.
Related
Referring to similar question :
Pattern for connecting to different databases using JDBC
I am using different Connection Strings/Drivers for each database.This is what I am doing, not very sure if it's the most efficient way to do it:
Create separate classes for each db's Connection with a getConnection(String URl,String userid,String password) method in it
In main class get connection object for DB1,DB2,DB3, open connections
Fetch data from DB1, write it to a flat file, repeat for DB2 and DB3
Close all three connections.
NOTE:I read about using Spring/Hibernate/DataSources/ConnectionPooling Dont know what shoud be the best option
The way I understand it is that you want your application to run some (SELECT?) queries on different databases and dump the results. I presume this is a part of a larger application since otherwise you would probably get results quicker by simply writing a command-line script that automates the client tools for the specific databases.
Hibernate, Data Sources (in the Java DataSource object sense) and Connection Pooling won't solve your problem - I guess it's the same for Spring but I don't know which part of Spring you're referring to. The reason for this is that they all are designed to abstract over a single (or a pool/collection of connections) to a single database - connection pooling simply allows you to keep a pool of ready-to-use (TCP) connections to a given database in order to improve performance, for example by avoiding connection and authentication overhead. Hibernate does the same in the sense that it abstracts a connection to a single database (and can use connection pooling for performance reasons on top of that).
I would suggest to maybe take a different approach to thinking about your problem:
Since you want to run some queries on some datasource and write the results to some destination, why don't you start your design this way: Come up with an interface/class DataExtractionTask that requires a database connection, a set of queries to run and some output stream. Instead of using java.sql.Connection directly you could choose some framework to make your life easier, there are heavy-weights like Hibernate and light-weights like jdbi. Then come up with code that establishes your database connection, decides which queries to run and the outputs to write to and feed all of that into your thought-out DataExtractionTask to run the logic of processing (orchestrating the individual parts).
Once you have the basic stuff in place you can add other features on top of it, you could make it configurable, you could choose to run multiple DataExtractionTasks in parallel instead of sequentially, et cetera.
This way you can generalize the processing logic and then focus on getting everything (database connections, query definitions, etc.) ready for processing. I realize that this is very broad-picture but maybe it makes things a bit easier.
Regarding efficiency: If you mean high performance (relative terms!), the best way would be what #Elliott Frisch wrote -- keeping it all in a single database that you connect to using a single connection pool.
You don't need to use separate classes just for connecting, just build up a util class which holds all the JDBC URLs and obtain a connection from it.
Besides that, you should consider using JPA instead, which you can do as well in Java SE as in Java EE. With that, you can abstract from the low level connection and define a named datasource. See for example this Oracle tutorial.
I have to create an mysql database to be used by several applications in parallel for the first time. Up until this point my only experience with mysql databases have been single programs (for example webservers) querying the database.
Now i am moving into a scenario where i will have several CXF java servlet type programs, as well as a background server editing and reading on the same schemas.
I am using the Connector/J JDBC driver to connect to the database in all instances.
My question is this: What do i need to do in order to make sure that the parallel access does not become a problem. I realize that i need to use transactions where appropriate, but where i am truly lost is in the management.
For example.
Do i need to close the connection every time a servlet is done with a job?
Do i need a unique user for each program accessing the database?
Do i have to do something with my Connector/J objects?
Do i have to declare my tables in a different way?
Did i miss anything or is there something i failed to think about?
I have a pretty good idea about how to handle transactions and the SQL itself, but i am pretty lost when it comes to what i need to do when setting up my database.
You should maintain a pool of connections. Connections are really expensive to create think on the order of of several hundred milliseconds. So for high volume apps it makes sense to cache and reuse them.
For your servlet it depends on what container you are using. Something like JBoss will provide pooling as part of the container. It can be defined through the datasource definition and accessed through JNDI. Other containers like tomcat may rely on something like C3PO.
Most of these frameworks return custom implementations of JDBC connections that implement the close() methods with logic that returns the connection to the pool. You should familiarize yourself with the details of your concrete implementation to make sure you are doing things in a way that is supported
As for the concurrency considerations, you should familiarize yourself with concepts of optimistic/pessimistic locking and transaction isolation levels. These have trade offs where the correct answer can only be determined given the operational context of your application.
Considering the user, Most applications have one user that represents the application called the read/write user. This user should only have privilege to read and write records from the tables,indices,sequences, etc. that are associated with your application. All the instances of the application will specify this user in their connection string.
If you familiarize yourself with the concepts above, you'll be about 95% of the way there.
One more thing. As pointed out in the comments on the administration side your database engine is a huge consideration. You should familiarize yourself with the differences and the tuning/configuration options.
I have a java application that is using MSSQL server through the JDBC driver. Is there some kind of stub that I can use for testing? For example I want to test how my application handle cases of connection errors, SQL server out of disk, and other exceptions. It's pretty hard and complex to simulate this with real SQL server.
Thanks
You could write unit tests against your DAOs or repositories returning mock Connection objects using a mock library such as https://mocquer.dev.java.net/.
You'd need a really clean and decoupled application architecture though in order to make this work correctly and provide you with actual test coverage.
You could (assuming the system is architected in a way to make this easy) create your own versions of the DB Access classes (I assume you are using teh statement/preparedstatement interfaces), which would hold the real DB calls and that you can modify to do exactly what you want.
I've done this - it takes a day or so of really boring work.
I don't think there's something like that.
You'd be better off setting up your own database and testing on your machine/lan.
All I know there is out there, is:
freeSQL
db4free
Both support MySQL, but none MS-SQL. I do think that has to do with licensing issues and limitations. So I'm afraid you won't find a similar service for MS-SQL db.
Answering myself with an option I thought of, I'll be glad to hear your inputs on it.
After crawling around, I got to HyperSQLDB, a java-implemented database.
How feasible do you think is to take the source code of HSQLDB, and adding another layer to it, so I can control it and inject pre-defined behaviors to it.
For example, I'll make it run all queries slowly, I'll make it disconnect, etc.
Do you think this idea is worth pursuing? Is it doable in a reasonable amount of time?
If you use something other than MS-SQL, you may cause more testing problems due to incompatibilities and lack of functionality (e.g., transactions) than you solve. So I'm with Carl - use a shim.
If you were looking for unit-test coverage of ordinary behavior, I might think differently.
I haven't used them personally, but the stuff you're talking about sounds like a really good fit for a mocking framework, such as Mockito(docs) or PowerMock. They appear to provide good support for the kind of failure injection you're after. Can someone with experience with either of them (or similar) weigh in? See also How to stub/mock JDBC ResultSet to work both with Java 5 and 6?
execute procedure sp_who2 it will generate the all the current connections and process in your db you can see a column named spid corresponding to each db connection. just type: kill <<spid>> and execute it to terminate any users..etc. but if the spid is less than 50 it means it is a system process and dont kill it. This can help you replicate connection drops.
you can also say ALTER DATABASE dbname SET SINGLE_USER WITH ROLLBACK_IMMEDIATE this will drop all connections to the said db immediately.
Select ##MAX_Connections as Max_Connections would give you the max connections which can be made to a database (you can set it to a low number to test connection unavailability).
to replicate query timeout.. set the query timeout to a very low number & execute a fairly large query.
to create disk space error, simply redice the size of the db file & do not allow it to grow... then insert data to the database (you'll get an exception).
altert database xxx (file= maxsize= filegrowth=)
i am trying to upload a file into the server and storing the information of the file into an Access database, is there any need to handle the threads while database connectivity for multiple user. If yes how to do it?
Your webserver is inheritly multithreaded, that saves you from implementing you own threads to handle the uploads.
Do however make sure that multiple requests dont use same resources (dont write all uploaded file in the same tmp file,....)
To avoid problems saving the data to your db, use a Connection Pool.
So yes you need threads but if your design is good then all the threading will be handled by your frameworks
Exactly. Each HTTP request is already a thread at its own. Keep in mind that the web container will create only one servlet instance during application's lifetime and that the servlet code is been shared among all requests. This implies that any class-level variables or static variables are going to be shared among all requests. If you have such one variable, it is not threadsafe. You need to declare request-specific variables threadlocal at method-level.
As to JDBC: just write solid code and everything should go well. Using a connection pool is only useful to improve connecting performance (which is really worth the effort, believe me, connecting the DB is a fairly expensive task which may account up to at least 200ms or even more, while reusing a connection from the pool costs almost nothing). It only doesn't change anything to the threadsafety of the code you write, it's still in your control/hands. To get a clear picture of how to do the basic JDBC coding the right way, you may find this article useful.
I know this is a subjective question, but why does Hibernate seem to be designed for short lived sessions? Generally in my apps I create DAOs to abstract my data layer, but since I can't predict how the entity objects are going to be used some of its collections are lazy loaded, or I should say fail to load once the session is closed.
Why did they not design it so that it would automatically re-open the session, or have sessions always stay open?
Becuase once you move out of your transaction boundary you can't hit the database again without starting a new transaction. Having long running transactions 'just in case' is a bad thing (tm).
I guess you want to lazy load object from your view - take a look here for some options. I prefer to define exactly how much of the object map is going to be returned by my session facade methods. I find this makes it easier to unit test and to performance test my business tier.
I worked on a desktop app that used EJB and Hibernate. We had to set lazy=false everywhere, because when the objects get serialized, they lose their ability to be fetched from the backend. That's just how it goes, unfortunately.
If you are concerned with performance, you could use caching on the backend so that your non-lazy fetches are not as painful.
You're looking for the OpenSessionInView pattern, which is essentially a conceptual filter (and sometimes implemented as a servlet filter) that detects when a session needs to be transparently reopened. Several frameworks implement this so it handles it automagically.
I'm writing a desktop application so using a filter isn't applicable.
Connections are a scarce resource that need to be recycled as soon as you are done using them. If you are also using connection pooling, getting another one when you need it should be quick. This is the architecture that you have to use to make websites scale -- even though you are a desktop app, their use-cases probably concentrate on scalable sites.
If you look at MS ADO.NET, you will see a similar focus on keeping connections open for a short time -- they have a whole offline model for updating data disconnected and then applying to a database when you are ready.
Hibernate is designed as a way to map Objects to Relational Database tables. It accomplishes that job very well. But, it can't please everybody all of the time. I think there is some complexity in learning how initialization works but once you get the hang of it it makes sense. I don't know if it was necessarily "designed" to specifically to anger you, it's just the way it happened.
If it was going to magically reopen sessions in non-webapps I think the complexity of learning the framework would far outweight the benefits.