Hi I have a java program with multiple threads querying an HSQL database. I'm getting some very weird results from the queries and my guess is that the database is not set up to handle many queries at the same time. (Is this even possible in a database?).
What I would like to do then is queue any database queries.
As far as I can tell this is done using the synchronize keyword when making functions in my database object?
My database object looks something like this:
public class Database(){
public Database(){
connect();
}
private void connect(){
//connect to HSQL database
}
public void executeOneTypeOfQuery(){
...
ResultSet rs1 = someStatement.executeQuery();
//do something with a query
}
public void executeAnotherTypeOfQuery(){
...
ResultSet rs2 = anotherStatement.executeQuery();
//do something with a query
}
}
At the moment I may be calling any combination of executeOneTypeOfQuery() and executeAnotherTypeOfQuery() at the same time in different threads.
Should those two functions have the synchronized keyword? Or will that only stop them calling JUST THAT method twice at the same time?
Am I right in thinking another solution could be making use of callbacks (I don't really understand the syntax here)?
So anywhere in my project I call one of those database functions, I call an intermediate synchronized function, stating which database function I really want to use.
So threads call something like:
public synchronized void executeAnyQuery(Function theFunctionIWantToCall, Object[] args){
//do theFunctionIWantToCall
}
Is this the right way to do it? If so what is the syntax with callbacks?
The synchronized keyword locks the function from access by another thread until the function terminates (although it does not guarantee the tread will execute uninterrupted)
For the callback, you would do something like this
public class Foo {
public static void main(String[] args) {
new Foo.doExample();
}
}
Related
I'm trying to multi thread a Result Set. I want to make sure whenever I call the next() within one of the many threads, all other threads are locked out. This is important , because if many threads call the next() method simultaneously, this will result in skipping the rows. Here is what I did
public class MainClass {
private static ResultSet rs;
public static void main (String [] args) {
Thread thread1 = new Thread(new Runnable() {
#Override
public void run() {
runWhile();
}});
Thread thread2 = new Thread(new Runnable() {
#Override
public void run() {
runWhile();
}});
thread1.start();
thread2.start();
thread1.join();
thread2.join();
System.exit(0);
}
private static void runWhile () {
String username = null;
while ((username = getUsername()) != null) {
// Use username to complete my logic
}
}
/**
* This method locks ResultSet rs until the String username is retrieved.
* This prevents skipping the rows
* #return
* #throws SQLException
*/
private synchronized static String getUsername() throws SQLException {
if(rs.next()) {
return rs.getString(1).trim();
}
else
return null;
}
}
Is this a correct way of using synchronized. Does it lock the ResutSet and makes sure other thread do not interfere ?
Is this a good approach ?
JDBC objects shouldn't be shared between threads. That goes for Connections, Statements, and ResultSets. The best case here would be that the JDBC vendor follows the spec and does internal locking so that you can get by with this, in which case all the threads are still trying to acquire the same lock and only one can make progress at a time. This will be slower than using a single thread, because on top of doing the same work to read from the database there is extra overhead from managing all the threading.
(Locking done by the driver could be for the driver's benefit, so the provider doesn't have to deal with bug reports of race conditions caused by users misusing their software. That it does locking doesn't necessarily imply the software should actually be used by multiple threads.)
Multithreading works when threads can make progress concurrently, see Amdahl's Law. If you have a situation where you can read the ResultSet and use the results to create tasks which you submit to an ExecutorService (as Peter Lawrey recommends in a comment) then that would make more sense (as long as those tasks can work independently and don't have to wait on each other).
I will suggest to create the ResultSet, then copy all the data into a DTO (Data Transfer Object) or a DAO (Data Access Object). After having the data on the DTO or DAO, close your ResultSet, Statement and Connection.
A very simple structure to creat a DTO/DAO to store records in order, its fields, and parsing capabilities is this:
ArrayList<HashMap<String, Object>> table = new ArrayList<HashMap<String, Object>>();
HashMap<String, Object> record = new HashMap<String, Object>();
String field1 = "something";
Integer field2 = new Integer(45);
record.put("field1", field1);
record.put ("field2", field2);
table.add(record);
You may (and probably you should) automate and make the DTO/DAO flexible enough to use the same class in any table, without hard code or fixed names.
Remember that you will need to create a wrapper and the methods for storing/reading the data, and that these methods should be thread safe.
Keep in mind that this design only works if you have enough memory to store all the records of your ResultSet.
Im attempting to make async writes to a Cassandra cluster using ListenableFuture as follows:
private static Cluster cluster = null;
private ListeningExecutorService executorService;
private PreparedStatement preparedStatement;
private Session session = null;
...
executorService = MoreExecutors.listeningDecorator(Executors.newFixedThreadPool(POOL_SIZE));
...
public void writeValue(Tuple tuple) {
ListenableFuture<String> future = executorService.submit(new Callable<String>() {
#Override
public String call() throws Exception {
if(session == null) {
session = getCluster().connect("dbname");
preparedStatement = session.prepare(queryString);
}
try {
BoundStatement boundStatement = preparedStatement.bind(tuple values);
session.execute(boundStatement);
} catch(Exception exception) {
// handle exception
}
return null;
}
});
If I set POOL_SIZE to 1 everything works.
If I set POOL_SIZE to > 1 I get errors as follows:
Caused by: com.datastax.driver.core.exceptions.InvalidQueryException: Tried to execute unknown prepared query : 0x75c5b41b9f07afa5384a69790503f963. You may have used a PreparedStatement that was created with another Cluster instance.
So I session and preparedStatement into local vars. Then I get warnings about Re-preparing already prepared query ... plus it's creating a new session every time.
I want to reuse as much as possible. What am I doing wrong and what are my options?
Would it help to make this class static?
You have all sorts of race conditions here and execution isn't thread safe.
Each of Cluster, Session, and PreparedStatement are meant to be application scoped singletons, ie. you only need one (one for each query for PreparedStatement).
However, you are recreating a Session and potentially preparing PreparedStatement multiple times.
Don't. Initialize your Session once, in a constructor or some location that only runs once and prepare your statements at the same time. Then use the Session and PreparedStatement where appropriate.
Using a single threaded executor, everything runs as if it was synchronous. When you add more threads, many of them may call
session.prepare(queryString);
at the same time. Or the PreparedStatement you use here
BoundStatement boundStatement = preparedStatement.bind(tuple values);
session.execute(boundStatement);
might be different from the one you initialized
preparedStatement = session.prepare(queryString);
even within the same thread of execution. Or you might be attempting to execute the PreparedStatement with a different Session than the one used to initialize it.
Here are some things you should be doing when using CQL drivers.
Is a prepared statement bound on one session or is it useable on another session?
A prepared statement is derived from a particular session instance.
So when you prepare a statement and it is sent over to the server, it
is sent to the cluster with which this session instance is associated
with.
The javadoc of Session states
Session instances are thread-safe and usually a single instance is
enough per application.
You might also want to use the driver's asynchronous API. Instead of calling execute (which will block your thread for the duration of the query), call executeAsync and register a callback on the resulting future to process the result.
If that callback is expensive and you don't want to block the driver's internal I/O thread, then you can provide your own executor:
ListenableFuture<ResultSet> future = session.executeAsync(statement);
Futures.addCallback(future, new FutureCallback<ResultSet>() {
public void onSuccess(ResultSet rs) { ... }
public void onFailure(Throwable t) { ... }
},
executorService);
This page in the documentation has a few tips on async programming.
I have a stored function that will remove something from the database, but since it could be a very long task, I want to make a thread to execute this function, and let the user keep on doing what he is doing.
Currently I have the following:
The DAO:
#Override
#Transactional
public void deleteAll()
{
Session session = (Session) entityManager.getDelegate();
session.doWork(new Work()
{
#Override
public void execute(Connection connection) throws SQLException
{
try
{
// stored function is currently named delete_function()
CallableStatement deleteAll= connection.prepareCall("{call delete_function()}");
purgeArchived.execute();
purgeArchived.close();
}
catch (SQLException exception)
{
LOGGER.warn(exception);
}
}
});
}
Im afraid when i call getDao.deleteAll() in the rest service, this will be in working for a really long time if the database has lot of stuff to delete. How do I create a thread to do the same thing? Or will this create a thread and execute the function?
Yes, you will need to make your own thread for this. Probably the simplest thing to do is to copy the entire body of your current deleteAll() method to the run() method of a new class, that extends Thread. Assuming you've called that class DeleteAllThread, you'd then replace your method above with
#Override
public void deleteAll() {
new DeleteAllThread().start();
}
Another option is to take a look at using a ExecutorService. This may make things a little cleaner for you. Here is a simple example of how to use an ExecutorService.
I'm new to Java programming. I have a use case where I have to execute 2 db queries parallely. The structure of my class is something like this:
class A {
public Object func_1() {
//executes db query1
}
public Object func_2() {
//executes db query1
}
}
Now I have a add another function func_3 in the same class which calls these 2 functions but also makes sure that they execute parallely. For this, I'm making use callables and futures. Is it the right way to use it this way? I'm storing the this variable in a temporary variable and then using this to call func_1 and func_2 from func_3(which I'm not sure is correct approach). Or is there any other way to handle cases like these?
class A {
public Object func_1() {
//executes db query1
}
public Object func_2() {
//executes db query1
}
public void func_3() {
final A that = this;
Callable call1 = new Callable() {
#Override
public Object call() {
return that.func_1();
}
}
Callable call2 = new Callable() {
#Override
public Object call() {
return that.func_2();
}
}
ArrayList<Callable<Object>> list = new ArrayList<Callable<Object>>();
list.add(call1);
list.add(call2);
ExecutorService executor = Executors.newFixedThreadPool(2);
ArrayList<Future<Object>> futureList = new ArrayList<Future<Object>>();
futureList = (ArrayList<Future<Object>>) executor.invokeAll(list);
//process result accordingly
}
}
First of all, you do NOT need to store this in another local variable: outer functions will be available just as func_1() or func_2() and when you want to get this of outer class you just use A.this.
Secondly, yes, it is common way to do it. Also, if you are going to call func_3 often - avoid creating of fixed thread pool, you should just pass it as params, since thread creation is rather 'costly'.
The whole idea of Executor(Service) is to use small number of threads for many small tasks. Here you use 2-threaded executor for 2 tasks. I would either create globally defined executor, or just spawn 2 threads for 2 tasks.
I guess, DAO is thread safe, does not use any class members.
So can it be used without any problem as a private field of a Servlet ? We need only one copy, and
multiple threads can access it simultaneously, so why bother creating a local variable, right?
"DAO" is just a general term for database abstraction classes. Whether they are threadsafe or not depends on the specific implementation.
This bad example could be called a DAO, but it would get you into trouble if multiple threads call the insert method at the same time.
class MyDAO {
private Connection connection = null;
public boolean insertSomething(Something o) throws Exception {
try {
connection = getConnection()
//do insert on connection.
} finally {
if (connection != null) {
connection.close();
}
}
}
}
So the answer is: if your DAO handles connections and transactions right, it should work.