I have an java-spring web application, which will read, write and delete information from a user upload SQLite DB. I am using JDBCtemplate to set connection, query the DB and update the information.
I observed one behavior during my tests:
Every time,after users uploaded a new SQLite db file(it will has the same name, place at the same directory as the old DB file), if they do not reboot/restart tomcat, jdbcquery will report the db was corrupted exception.
To me this looked like the JDBCtemplate somehow cached the connection and is trying to resume the connection with the old db?
If so, do you know anyway to refresh the connection without rebooting the application?
final SingleConnectionDataSource dataSource = new singleConnectionDataSource();
try {
Class.forName("org.sqlite.JDBC");
} catch (ClassNotFoundException e) {
throw new applicationException(MessageTemplateNames.GENERAL_UNKNOWN_ERROR, e);
}
createDirectoryForDbIfNotExists();
dataSource.setUrl(String.format("%s%s", JDBC.PREFIX, getDbFileLocation()));
dataSource.setAutoCommit(true);
JDBCTemplate does not handle connection.It obtains the connection from the datasource set to it.
From the reference documentation for SingleConnectionDataSource
Implementation of SmartDataSource that wraps a single JDBC Connection
which is not closed after use. .....
This is primarily intended for testing. For example, it enables easy
testing outside an application server, for code that expects to work
on a DataSource. In contrast to DriverManagerDataSource, it reuses the
same Connection all the time, avoiding excessive creation of physical
Connections.
A DriverManagerDataSource will suite your requirement to get a new connection without reboot
Simple implementation of the standard JDBC DataSource interface,
configuring the plain old JDBC DriverManager via bean properties, and
returning a new Connection from every getConnection call.
Update
My answer might not solve your original problem , but will only answer the question for new connection without reboot. Please go through #Thilo's comment on the same.
Related
My server app uses prepared statements in almost all cases, to prevent sql injection. Nevertheless a possibility is needed providing special users executing raw SELECT queries.
How can I more or less securely make sure the query does not modify the database? Is it possible to execute a query read only, or is there any other 'secure' way making sure noone tries any sql injection?
(Using sqlite3, so I cannot use any privileges)
Thanks a lot!
JDBC supports read-only connections by calling Connection.setReadOnly(true). However the javadoc says:
Puts this connection in read-only mode as a hint to the driver to enable database optimizations.
Some JDBC drivers will enforce the read-only request, others will use it for optimizations only, or simply ignore it. I don't know how sqlite3 implements it. You'll have to test that.
Otherwise, you could do a "simple" parse of the SQL statement, to ensure that it's a single valid SELECT statement.
I'm not aware of a general JBDC configuration which specifies readonly. But Sqlite does have special database open modes and this can be leveraged in your connection to your sqlite database. Eg.
Properties config = new Properties();
config.setProperty("open_mode", "1"); //1 == readonly
Connection conn = DriverManager.getConnection("jdbc:sqlite:sample.db", config);
Credit: https://stackoverflow.com/a/18092761/62344
FWIW All supported open modes can be seen here.
If you use some sort of factory class to create or return connections to the database, you can individually set connections to be read-only:
public Connection getReadOnlyConnection() {
// Alternatively this could come from a connection pool:
final Connection conn = DriverManager.getConnection("jdbc:sqlite:sample.db");
conn.setReadOnly(true);
return conn;
}
If you're using a connection pool, then you may also want to provide a method for getting writeable connections too:
public Connection getWriteableConnection() {
final Connection conn = getPooledConnection(); // I'm assuming this method exists!
conn.setReadOnly(false);
return conn;
}
You could also provide just a single getConnection(boolean readOnly) method and simply pass the parameter through to the setReadOnly(boolean) call. I prefer the separate methods personally, as it makes your intent much clearer.
Alternatively, some databases like Oracle provide a read only mode that can be enabled. SQLite doesn't provide one, but you can emulate it by simply setting the actual database files (including directories) to read only on the filesystem itself.
Another way of doing it is as follows (credit goes to deadlock for the below code):
public Connection getReadOnlyConnection() {
SQLiteConfig config = new SQLiteConfig();
config.setReadOnly(true);
Connection conn = DriverManager.getConnection("jdbc:sqlite:sample.db",
config.toProperties());
}
I have Java application with Hibernate framework(no spring) connect to MySQL DB , manage connection pooling via c3p0
i try to configure my apllication to read from slave db and write to master db , i have following this link to some extend Master/Slave load balance
let's say if the application already got a session with connection in pool and it need to execute a read-only method , like this
public someReadOnlyMethod()
{
Session session = (get session from current Thread)
//set read-only so that it read from slave db
session.connection().setReadOnly(true);
(...connect to db to do something...)
//set it back in case of this method is followed by write method so that it go to master db
session.connection().setReadOnly(false);
}
Is the pooling create a new connection to connect to db 2 times for read-only and write operation(if so,this will heavily impact performance) or it smart enough to swap the operation to already existing read-only and writable connection pool ?
thx for your advice.
so this has nothing to do with the pool; it's all in the mysql driver. c3p0 will pass your call to setReadOnly (whether true or false) to the underlying Connection, and the Connection will route to the master or the slaves accordingly.
if you don't like how your Connections default (probably by default they are not read only), you can set the read-only property in the onAcquire method of a c3p0 ConnectionCustomizer, and the value use set (true or false) will become th default that c3p0 resets Connections to.
good luck!
tl;dr: It will re-use existing connections whenever you switch setReadOnly(true/false).
JDBC will connect to all servers listed in your connection URL when you do ReplicationDriver().connect(url). Those connections will remain open for re-use no matter how many times you switch setReadOnly().
Source: I just tested Connector/J version 5.1.38 with com.mysql.jdbc.ReplicationDriver.
I have an application that connects to MySQL using JDBC. There are cases where the JDBC connection lies idle for hours (maybe even days) and its loosing its connection to MySQL and then excepts when it tries to execute a query. What is the best solution for this?
Keeping the connection open for an undertemined time is a bad practice. The DB will force a close when it's been open for a too long time. You should write your JDBC code so that it always closes the connection (and statement and resultset) in the finally block of the very same try block where you've acquired them in order to prevent resource leaking like this.
However, acquiring the connection on every hiccup is indeed a pretty expensive task, so you'd like to use a connection pool. Decent connection pools will manage the opening, testing, reusing and closing the connections themselves. This does however not imply that you can change your JDBC code to never close them. You still need to close them since that would actually release the underlying connection back to the pool for future reuse.
There are several connection pools, like Apache DBCP which is singlethreaded and thus poor in performance, C3P0 which is multithreaded and performs better, and Tomcat JDBC for the case that you're using Tomcat and wouldn't like to use the builtin DBCP due to bad performance.
You can create connection pools programmatically, here's an example with C3P0:
ComboPooledDataSource dataSource = new ComboPooledDataSource();
dataSource.setDriverClass("com.mysql.jdbc.Driver");
dataSource.setJdbcUrl("jdbc:mysql://localhost:3306/dbname");
dataSource.setUser("username");
dataSource.setPassword("password");
Do it once during application's startup, then you can use it as follows:
Connection connection = null;
// ...
try {
connection = dataSource.getConnection();
// ...
} finally {
// ...
if (connection != null) try { connection.close(); } catch (SQLException ignore) {}
}
When you're running inside a JNDI-capable container like a servletcontainer (e.g. Tomcat), then you can also declare it as a java.sql.DataSource (Tomcat specific manual here). It will then use the servletcontainer-provided connection pooling facilities. You can then acquire the datasource as follows:
DataSource dataSource = (DataSource) new InitialContext().lookup("java:comp/env/jdbc/YourDataSourceName");
There are libraries, such as Apache's DBCP which can do connection pooling. A part of this is they can be setup to automatically test the connection when you go to use it (such as "SELECT NOW() FROM DUAL", or something else harmless) and automatically re-establish the connection transparently if necessary, allowing your application to pretend that the connection is everlasting.
Check here:
http://download.oracle.com/javase/tutorial/jdbc/basics/connecting.html
Basically, you should use DataSource and always do a getConnection() before using it. DataSource, unless there's something terribly wrong, will reconnect if necessary.
I have a website, in which currently I am getting 1000 page views. I am expecting it will go around 30k per day in future. Now the problem for me to manage the DB connections.
At present I am just connecting to DB directly from java program. I know it is worst design in the world. But for time being I have written like that.
I have plan to manage connection pooling using JNDI. But the problem is my hosting provider is not supporting JNDI.
Can anyone suggest me how to manage DB connections without jndi?
Connection pooling does not per se require the connections to be obtained by JNDI. You can also just setup and use a connection pool independently from JNDI. Let's assume that you'd like to use C3P0, which is one of the better connection pools, then you can find "raw" JNDI-less setup details in this tutorial.
Here's an extract of the tutorial:
ComboPooledDataSource cpds = new ComboPooledDataSource();
cpds.setDriverClass( "org.postgresql.Driver" ); //loads the jdbc driver
cpds.setJdbcUrl( "jdbc:postgresql://localhost/testdb" );
cpds.setUser("swaldman");
cpds.setPassword("test-password");
Create the datasource once during application's startup and store it somewhere in the context. The connection can then be acquired and used as follows:
Connection connection = null;
// ...
try {
connection = cpds.getConnection();
// ...
} finally {
// ...
if (connection != null) try { connection.close(); } catch (SQLException ignore) {}
}
Yes, closing in finally is still mandatory, else the connection pool won't be able to take the connection back in pool for future reuse and it'll run out of connections.
I just want to make sure that if I use the following, I am opening a separate DB session and not resuing the same one (for testing I need individual sessions).
Connection connection = DriverManager.getConnection(URL,USER,PASSWORD);
each time I do the above code, I run my query, then do a connection.close()
So for example:
while(some condition) {
Connection connection = DriverManager.getConnection(URL,USER,PASSWORD);
//now use the connection to generate a ResultSet of some query
connection.close();
}
So, each iteration of the loop (each query) needs its own session.
Is this properly opening separte sessions as I need (and if not, what would I need to add/change)? thanks
The javadoc says:
Attempts to establish a connection to
the given database URL
Slightly woolly language, and I suspect that this is up to the JDBC driver, but I'd be surprised if this did anything other than open a new connection.
I suppose it's possible for a JDBC driver to perform connection pooling under the hood, but I'd be surprised to see that.
In the case of the Oracle JDBC driver, this will open a new connection every time. This is a relatively slow process in Oracle, you may want to consider using a connection pool (e.g. Apache Commons DBCP, or c3p0) to improve performance.