We Keep Coding

org.h2.jdbc.JdbcSQLException after shutdownNow() in main thread

I have an ExecutorService (thread pool size = 4) that is handling a number of Callables. Each of them opens a database connection (Hikari Connection pool) and closes it again.
If I now call shutdownNow() on the ExecutorService I do also wait for the termination of the currently running tasks. However, eventhough awaitTermination does not produce a timeout - thus all running tasks should have been terminated, and all database operations should have finished - I get an org.h2.jdbc.JdbcSQLException stating the following:
General error: "java.lang.IllegalStateException: Reading from nio:database.mv.db failed; file length -1 read length 256 at 711665 [1.4.196/1]"; SQL statement: SELECT * FROM PERSON WHERE id = ? [50000-196]
In addition, I do close the Hikari connection pool far later than shutting down the ExecutorService. Do you have any ideas what I could search for?
EDIT:
Here is the basic code structure - I think I have mentioned all necessary items. Note, that the exception mentioned does not get thrown every time - but most of the time:
class DatabaseManager {
private HikariDataSource datasource;
private static DatabaseManager instance;
public static DatabaseManager getInstance() {
if (instance == null) {
instance = new DatabaseManager();
}
return instance;
}
public Connection getConnection() { datasource.getConnection(); }
private DatabaseManager() {
// initialize with parameters for a H2 database
this.datasource = new HikariDataSource();
}
public void shutdown() {
if (this.datasource != null) {
this.datasource.shutdown();
}
this.datasource = null;
}
}
class SimpleCallable extends Callable<SomeType> {
String information;
public SomeCallable(String info) { this.information = info; }
public SomeType call() {
// omitted try-catch
Connection connection = DatabaseManager.getInstance().getConnection();
// doing some things with connection (reading and writing data), the Method invoked is static and synchronized
// within this method the exception mentioned above is thrown
SomeType someType = SomeTypeHelper.transferToDB(connection, information);
connection.close();
return someType;
}
}
class SimpleTask extends Runnable {
public void run() {
ExecutorService service = new Executors.newFixedThreadPool(4);
for (i=0; i<1000; i++) {
SimpleCallable callable = new SimpleCallable("random text");
FutureTask task = new FutureTask(callable);
service.submit(task);
}
try {
Thread.sleep(1000);
}
catch (InterruptedException e) {
// nothing to do
}
service.shutdownNow();
try {
if (!service.awaitTermination(60, TimeUnit.SECONDS)) {
System.out.println("timeout"); // but will never be printed
}
}
catch (InterruptedException e) {
// nothing to do
}
}
}
class App {
public static void main(String[] args) {
SimpleTask task = new SimpleTask();
new Thread(task).start();
DatabaseManager.getInstance().shutdown();
}
}

Why is my boolean not being changed?

So I'm trying to create a client/server program. I want to know when my client disconnects of his own accord, so I've setup a heartbeat system. Every 6 seconds my client sends a ping to my server, if the client doesn't send a ping for a total of 30 seconds the client is considered disconnected and removed from the current connections list (for which I plan to implement a GUI). Or at least, that's the plan.
ConnectionManager.java
public class ConnectionManager implements Runnable{
static Socket connection;
private ArrayList<Thread> allConnections;
private ArrayList<Connection> allConnectionList;
private ServerSocket server;
private int id = 0;
public ConnectionManager() {
allConnections = new ArrayList<Thread>();
allConnectionList = new ArrayList<Connection>();
}
#Override
public void run() {
try {
server = new ServerSocket(5555);
System.out.println("Server is running!");
while(true) {
connection = server.accept();
Connection a = new Connection(connection, id);
Runnable runnable = a;
allConnectionList.add(a);
allConnections.add(new Thread(runnable));
allConnections.get(allConnections.size() - 1).start();
id++;
}
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
public void removeConnection(int id) {
allConnections.remove(id);
allConnectionList.remove(id);
}
Connection.java
public class Connection implements Runnable {
private Socket a;
public boolean amIActive;
private int id;
public Connection(Socket a, int id) {
amIActive = true;
this.a = a;
this.id = id;
}
public void onConnect() {
try {
String TimeStamp = new java.util.Date().toString();
String formattedAddress = a.getInetAddress().toString().replace("/", "");
System.out.println("Received connection from: " + formattedAddress + " at " + TimeStamp);
Runnable runnable = new ConnectionListener(this);
Thread connectionThread = new Thread(runnable);
connectionThread.start();
String returnCode = "Server repsonded to " + a.getInetAddress().toString().replace("/", "") + " at "+ TimeStamp + (char) 13;
BufferedOutputStream os = new BufferedOutputStream(a.getOutputStream());
OutputStreamWriter osw = new OutputStreamWriter(os, "US-ASCII");
osw.write(returnCode);
osw.flush();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
#Override
public void run() {
onConnect();
System.out.println("We got this far!");
while(amIActive) {
whileTrue();
}
System.out.println("This code never gets run because we get stuck in the while loop above");
Main.b.removeConnection(id);
System.out.println("Connection was closed from " + a.getInetAddress());
}
public void setOffline(boolean state) {
this.amIActive = state;
}
public void whileTrue() {
}
public Socket getSocket() {
return a;
}
ConnectionListener.java
public class ConnectionListener implements Runnable{
public Connection myConnection;
public boolean receivedHeartbeat;
public int missedHeartbeats = 0;
public ConnectionListener(Connection a) {
this.myConnection = a;
}
#Override
public void run() {
Runnable runnable = new Heartbeat(this);
Thread thread = new Thread(runnable);
thread.start();
while(myConnection.amIActive) {
try {
BufferedInputStream is;
is = new BufferedInputStream(myConnection.getSocket().getInputStream());
InputStreamReader isr = new InputStreamReader(is);
StringBuffer process = new StringBuffer();
int character;
while((character = isr.read()) != 13) { //GETTING STUCK HERE BECAUSE STUPID.
if(character == -1) {
myConnection.setOffline(true);
} else {
process.append((char)character);
}
}
handleInput(process);
} catch (Exception e) {
e.printStackTrace();
}
}
}
public void handleInput(StringBuffer process) {
String messageSent = process.toString();
if(messageSent.equals("Ping!")) {
receivedHeartbeat = true;
}
}
Heartbeat.java
public class Heartbeat implements Runnable{
private ConnectionListener b;
public Heartbeat(ConnectionListener a) {
b = a;
}
#Override
public void run() {
while(true) {
try {
Thread.sleep(1000);
if(b.missedHeartbeats > 5) {
b.myConnection.amIActive = false;
System.out.println("Setting amIActiveToFalse!");
}
if(b.receivedHeartbeat) {
b.receivedHeartbeat = false;
} else {
b.missedHeartbeats++;
}
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
My console is spammed with System.out.println("Setting amIActiveToFalse!"); from Heartbeat.java. But the while loop in Connection.java keeps running. I believe this might be something to do with my threading, but I can't figure it out.

When you have a non-volatile variable, there is no guarentee of visability of a change in one thread to another. In particular, if the JVM detects that a thread doesn't alter a boolean it can inline it, meaning you will never see the value change.
The simple solution is to make the boolean volatile and it will not be inlined and one thread will see when another changes it.
For more details http://vanillajava.blogspot.com/2012/01/demonstrating-when-volatile-is-required.html

The trivial answer to this is: make the variable volatile.
Without this, it is allowed for the thread changing the value to basically keep its updates in cache, committing them to main memory some time later.
This allows threaded code to run much faster, since it can keep its variables in cache rather than having to fetch from main memory. However, the consequence of this is that other threads don't see the update.
Making the variable volatile prevents this from happening: a thread always reads the value from main memory, and writes are immediately committed.
I say that this is the trivial answer because it doesn't necessarily fix all of your problems. There may also be an atomicity issue: in between one thread reading the variable and writing it again, another thread might sneak in and change its value, which may or may not put the first thread into an undefined state from the perspective of its invariants.
Specifically:
if(b.receivedHeartbeat) { b.receivedHeartbeat = false;
It is possible that some other thread can change b.receivedHeartbeat to false after this thread evaluates it to true, so this iteration is erroneously counted as a "non-missed" heartbeat.
This can be fixed by making the variable a (non-volatile) AtomicBoolean, on which there is an atomic compare-and-set method, which avoids such race conditions.
Java Concurrency In Practice is a great reference on these issues, I wholeheartedly recommend it. Look for the topics "visibility" and "atomicity".
Also read the advanced chapter on the Java Memory Model. That made me doubt myself at first, but made me a much stronger programmer after I digested it.

There are a couple issues I saw while debugging the code you posted, but I was able to successfully get the heartbeat functionality working.
In the Connection Listener class I don't think the if statement with .equals("Ping!") will match, because of the newline character at the end of each line.
In the Connection Listener class I would probably put the socket's Input Stream at the top of the loop not inside the loop. (I don't think this will break it but it's probably nicer this way)
ConnectionListener Updates:
public void run() {
Runnable runnable = new Heartbeat(this);
Thread thread = new Thread(runnable);
thread.start();
BufferedReader br = null;
try {
//is = new BufferedInputStream(myConnection.getSocket().getInputStream());
br = new BufferedReader(new InputStreamReader(myConnection.getSocket().getInputStream()));
} catch (IOException e1) {
// TODO Auto-generated catch block
e1.printStackTrace();
}
while(myConnection.amIActive) {
try {
String processLine = br.readLine();
System.out.println("handleInput:" + processLine);
handleInput(processLine);
} catch (Exception e) {
System.out.println("Exception!");
e.printStackTrace();
}
}
}
public void handleInput(String messageSent) {
if(messageSent.startsWith("Ping!")) { //Need to use startsWith, or add newline character
receivedHeartbeat = true;
System.out.println("receivedHeartbeat!");
}
}
Also, in your Heartbeat class make sure you reset the missedHeartbeats counter to 0 on true:
if(b.receivedHeartbeat) {
b.receivedHeartbeat = false;
b.missedHeartbeats = 0;
} else {
b.missedHeartbeats++;
}

How can I make the thread sleep for a while and then process all the messages?

I'm writing an Android app that uses two threads. One is UI thread and the other handles server communication. Is it possible for the other thread to wait for a specified amount of time and then process all the messages that have arrived and then wait again?
I need this so that I can collect different data and send it to server in one session.
I've build my thread with HandlerThread but now I'm stuck. Can anyone point me to the right direction?
This is the code I'm using inside the second thread:
public synchronized void waitUntilReady() {
serverHandler = new Handler(getLooper()){
public void handleMessage(Message msg) { // msg queue
switch(msg.what) {
case TEST_MESSAGE:
testMessage(msg);
break;
case UI_MESSAGE:
break;
case SERVER_MESSAGE:
break;
default:
System.out.println(msg.obj != null ? msg.obj.getClass().getName() : "is null");
break;
}
}
};
}
EDIT:
I resolved my issue by going with Thread instead of HandlerThread and using queue.
I'm new to programming so I apologize for any horrenous errors but here's the code I ended up using.
public class ServiceThread extends Thread {
// TODO maybe set the thread priority to background?
static ServiceThread sThread = new ServiceThread(); // service thread instance
private volatile Handler mainHandler;
//
public Thread mainThread;
private boolean OK = true;
public Queue<MessageService> msgQueue;
private ThreadPoolExecutor exec;
private ServiceThread() { }
#Override
public void run() {
synchronized (this){
msgQueue = new ConcurrentLinkedQueue<MessageService>();
notifyAll();
}
mainHandler = new Handler(Looper.getMainLooper());
ThreadPoolExecutor exPool = (ThreadPoolExecutor) Executors.newFixedThreadPool(2);
exec = exPool;
// MAIN LOOP
try {
while(OK) {
getMessagesFromQueue();
Thread.sleep(3000);
}
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
//end of loop
}
public void ProcessMessage(MessageService message) {
System.err.println("ProcessMessage with command: " + message.command);
}
/** Called from the Main thread. Waits until msgQueue is instantiated and then passes the reference
* #return Message Queue
*/
public Queue<MessageService> sendQueue() {
synchronized (this){
while(msgQueue == null) {
try {
wait();
} catch (InterruptedException e) {
// TODO Auto-generated catch block -- move the try block!
e.printStackTrace();
}
}
}
return msgQueue;
}
public void setOkFalse () {
if (OK == true)
OK = false;
}
// Message handling methods
/** Priority message from UI thread, processed in another thread ASAP.
* Should be used on commands like getBigPicture or getPics when cached pics are running out
* or upload picture etc.
* #param message - Message should always be MessageService class
* TODO check that it really is.
*/
public void prioTask (MessageService message) {
final MessageService taskMsg = message;
Runnable task = new Runnable() {
#Override
public void run(){
ProcessMessage(taskMsg);
}
};
exec.execute(task);
}
/**
* Gets messages from queue, puts them in the list, saves the number of messages retrieved
* and sends them to MessageService.handler(int commands, list messageList)
* (method parameters may change and probably will =) )
*/
public void getMessagesFromQueue() {
int commands = 0;
ArrayList <MessageService> msgList = new ArrayList <MessageService>();
while(!msgQueue.isEmpty()) {
if(msgQueue.peek() instanceof MessageService) {
//put into list?
msgList.add(msgQueue.remove());
commands++;
} else {
//Wrong type of message
msgQueue.remove();
System.err.println("getMessagesFromQueue: Message not" +
" instanceof MessageService, this shouldn't happen!");
}
}
if (commands > 0) {
HTTPConnection conn;
try {
conn = new HTTPConnection();
MessageService.handleSend(commands, msgList, conn);
conn.disconnect();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
}
P.S. This is also my first post here. Should I mark it solved or something? How?

How to Update the dabtabase using for loop and getting the last sequence value

I am trying to do multi-threading here, now I have to update my database using DbHandler class
The program execution begins in a controller class which has a main method and a thread pool:
public class RunnableController {
// Main method
public static void main(String[] args) throws InterruptedException {
try {
RunnableController controller = new RunnableController();
controller.initializeDb();
controller.initialiseThreads();
System.out.println("Polling");
} catch (Exception e) {
e.printStackTrace();
}
}
private void initialUpdate()
{
DBhandler dbhandler = new DBhandler();
dbhandler.updateDb(getOutgoingQueue());
}
private void initialiseThreads() {
try {
threadExecutorRead = Executors.newFixedThreadPool(10);
PollingSynchronizer read = new PollingSynchronizer(incomingQueue, dbConncetion);
threadExecutorRead.submit(read);
} catch (Exception e){
e.printStackTrace();
}
}
}
My poller class which fetches new data and should do updating simulateously:
public class PollingSynchronizer implements Runnable {
public PollingSynchronizer(Collection<KamMessage> incomingQueue,
Connection dbConnection) {
super();
this.incomingQueue = incomingQueue;
this.dbConnection = dbConnection;
}
private int seqId;
public int getSeqId() {
return seqId;
}
public void setSeqId(int seqId) {
this.seqId = seqId;
}
// The method which runs Polling action and record the time at which it is done
public void run() {
int seqId = 0;
while (true) {
List<KamMessage> list = null;
try {
list = fullPoll(seqId);
if (!list.isEmpty()) {
seqId = list.get(0).getSequence();
incomingQueue.addAll(list);
this.outgoingQueue = incomingQueue;
System.out.println("waiting 3 seconds");
System.out.println("new incoming message");
Thread.sleep(3000);//at this wait I should execute run()
//when I debug my execution stops here and throws " Class not found Exception "
// its does not enters the message processor class
MessageProcessor processor = new MessageProcessor() {
//the run method which should fetch the message processor class.
final public void run() {
RunnableController.setOutgoingQueue(generate(outgoingQueue));
}
};
new Thread(processor).start();
}
} catch (Exception e1) {
e1.printStackTrace();
}
}
}
}
My message processor class:
public class MessageProcessor implements Runnable {
private Collection<KpiMessage> fetchedMessages;
private Connection dbConnection;
Statement st = null;
ResultSet rs = null;
PreparedStatement pstmt = null;
private Collection<KamMessage> outgoingQueue;
public Collection<KamMessage> MessageProcessor(Collection<KamMessage> outgoingQueue){
this.outgoingQueue = outgoingQueue;
this.dbConnection = dbConnection;
return outgoingQueue;
}
/**
* Method for updating new values into database in preference for dummy processing of message
* #param outgoingQueue
* #return
*/
#SuppressWarnings("javadoc")
public Collection<KamMessage> generate(Collection<KamMessage> outgoingQueue)
{
for (KamMessage pojoClass : outgoingQueue) {
KamMessage updatedValue = createKamMsg804(pojoClass);
System.out.print(" " + pojoClass.getSequence());
System.out.print(" " + pojoClass.getTableName());
System.out.print(" " + pojoClass.getAction());
System.out.print(" " + updatedValue.getKeyInfo1());
System.out.print(" " + updatedValue.getKeyInfo2());
System.out.println(" " + pojoClass.getEntryTime());
}
return outgoingQueue;
}
/**
*
* #param pojoClass
* #return msg
*/
public KamMessage createKamMsg804(KamMessage pojoClass)
{
if(pojoClass.getAction() == 804){
pojoClass.setKeyInfo1("ENTITYKEY9");
pojoClass.setKeyInfo2("STATUSKEY9");
}
return pojoClass;
}
private KamMessage convertRecordsetToPojo(ResultSet rs) throws SQLException {
KamMessage msg = new KamMessage();
int sequence = rs.getInt("SEQ");
msg.setSequence(sequence);
String tablename = rs.getString("TABLENAME");
msg.setTableName(tablename);
Timestamp entrytime = rs.getTimestamp("ENTRYTIME");
Date entryTime = new Date(entrytime.getTime());
msg.setEntryTime(entryTime);
Timestamp processingtime=rs.getTimestamp("PROCESSINGTIME");
if(processingtime!=null){
Date processingTime = new Date(processingtime.getTime());
msg.setProcessingTime(processingTime);
}
String keyInfo1 = rs.getString("KEYINFO1");
msg.setKeyInfo1(keyInfo1);
String keyInfo2 = rs.getString("KEYINFO2");
msg.setKeyInfo2(keyInfo2);
return msg;
}
#Override
public void run() {
// TODO Auto-generated method stub
}
}
This is my DBhandler Class, which should do updating in database
public class DBhandler {
Connection conn = null;
Statement st = null;
ResultSet rs = null;
PreparedStatement pstmt = null;
public DBhandler(){
super();
}
/**
* Method to initialize the database connection
* #return conn
* #throws Exception
*
*/
public Connection initializeDB() throws Exception {
System.out.println("JDBC connection");
DriverManager.registerDriver(new oracle.jdbc.driver.OracleDriver());
conn = DriverManager.getConnection("jdbc:oracle:thin:#VM-SALES-
MB:1521:SALESDB1","bdeuser", "edb"); // Connection for Database SALES-DB1
return conn;
}
//The method for updating Database
public void updateDb(Collection<KpiMessage> updatedQueue){
for(KpiMessage pojoClass : updatedQueue){
//**How the query should be used so that it gets last sequence vale and Updates into
Database**
String query = "UPDATE msg_new_to_bde Set KEYINFO1= ?, KEYINFO2 = ? WHERE SEQ = and
action = 804";
}
}
/**
* Method for Closing the connection
* #throws Exception
*
*/
public void closeDB() throws Exception {
st.close();
conn.close();
}
}
I just need to Update the database using update query in this class(DbHAndler) by calling the updatedQueue in the controller class.
My program flow - I have three classes: 1.Controller 2.PollerSynchro 3.Msgprocessor
I have database records, which are converted into POJO form and stored in a Collection. With these POJOs my classes try to do multiprocessing and updating in a single stretch.
Controller - has the thread pool, initiates poller class with poll method - done
Poller - should poll for new incoming messages and stores it in incoming queue - done
MsgProcessor - should look for new incoming messages and pass them from outgoing queue to incoming queue - also done
DbHandler- which should update in the database.
Problem:
Now my problem is
I have to implement this update while the poll thread sleeps for 3 sec -Done
In my code for the second void run() method in the Poller class, the outgoing queue is not passed and fed to the messageprocessor class for updating. My flow of execution only just loops back to first run method and am getting Class exception-Resolved
How to Update this in the database in Dbhanler class
Please help me to solve these problems.

The exception seems to come from this line (is this MessageProcessor.java line 38?)
return (KpiMsg804) fetchedMessages;
The fetchedMessages at this point seem to be an ArrayList.

How pass these new messages to another class

Now basically I have created three classes.
public void run() {
int seqId = 0;
while(true) {
List<KamMessage> list = null;
try {
list = fullPoll(seqId);
} catch (Exception e1) {
e1.printStackTrace();
}
if (!list.isEmpty()) {
seqId = list.get(0).getSequence();
incomingMessages.addAll(list);
System.out.println("waiting 3 seconds");
System.out.println("new incoming message");
}
try {
Thread.sleep(3000);
System.out.println("new incoming message");
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
public List<KamMessage> fullPoll(int lastSeq) throws Exception {
Statement st = dbConnection.createStatement();
ResultSet rs = st.executeQuery("select * from msg_new_to_bde where ACTION = 804 and SEQ >" +
lastSeq + "order by SEQ DESC");
List<KamMessage> pojoCol = new ArrayList<KamMessage>();
while (rs.next()) {
KamMessage filedClass = convertRecordsetToPojo(rs);
pojoCol.add(filedClass);
}
for (KamMessage pojoClass : pojoCol) {
System.out.print(" " + pojoClass.getSequence());
System.out.print(" " + pojoClass.getTableName());
System.out.print(" " + pojoClass.getAction());
System.out.print(" " + pojoClass.getKeyInfo1());
System.out.print(" " + pojoClass.getKeyInfo2());
System.out.println(" " + pojoClass.getEntryTime());
}
return pojoCol;
}
The following are the classes:
1.Poller- does the Polling and Passes the new data from db to controller
2.Controller- this class has a thread Pool, which simultaneously calls the Poller and has the new data to be requested from processor
3.Processor- this class has to look for new data, process it and return it to controller.
So now my problem is how to implement the third phase...
Here is my controller class:
public class RunnableController {
/** Here This Queue initializes the DB and have the collection of incoming message
*
*/
private static Collection<KpiMessage> incomingQueue = new ArrayList<KpiMessage>();
private Connection dbConncetion;
public ExecutorService threadExecutor;
private void initializeDb()
{
//catching exception must be adapted - generic type Exception prohibited
DBhandler conn = new DBhandler();
try {
dbConncetion = conn.initializeDB();
} catch (Exception e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
private void initialiseThreads()
{
try {
threadExecutor = Executors.newFixedThreadPool(10);
PollingSynchronizer read = new PollingSynchronizer(incomingQueue, dbConncetion);
threadExecutor.submit(read);
}catch (Exception e){
e.printStackTrace();
}
}
#SuppressWarnings("unused")
private void shutDownThreads()
{
try {
threadExecutor.shutdown();
//DB handling should be moved to separate DB class
dbConncetion.close();
}catch (Exception e){
e.printStackTrace();
}
}
/** Here This Queue passes the messages and have the collection of outgoing message
*
*/
//private Collection<KpiMessage> outgingQueue = new ArrayList<KpiMessage>();
//have to implement something here for future
public static void main(String[] args) throws InterruptedException {
RunnableController controller = new RunnableController();
System.out.println(incomingQueue.size());
controller.initializeDb();
controller.initialiseThreads();
Thread.sleep(3000);
System.out.println("Polling");
}
}

I would recommend using a BlockingQueue for doing so, instead of a simple ArrayList. Just change the type of your incomingQueue variable. Then you can have another thread (or a thread pool) doing something like
//pseudocode
while (true) {
// it polls data from the incomingQueue that shares with the producers
KpiMessage message = this.incomingQueue.take()
//Then process the message and produces an output... you can put that output in a different queue as well for other part of the code to pick it up
}
A good example on BlockingQueues can be found here http://www.javamex.com/tutorials/blockingqueue_example.shtml