I'm blocking with a problem from a few days. I've found some similar posts but still didn't understanding the problem of my code.
In fact, I'm reading a file (18,4Kbytes) which is containing SQL queries. So the only thing I want to do is read the file and execute the queries.
I've no problem reading the file, the problem occurs when after having executed all the queries (if I don't execute it, it works but it's not the deal!)
So here's my code (between try / catch for IO Exception):
InputStream in = ctx.getAssets().open("file.sql");
ByteArrayBuffer queryBuff = new ByteArrayBuffer(in.available());
String query = null;
int curent;
while (-1 != (curent = in.read())) {
queryBuff.append((char) curent);
if (((char) curent) == ';') {
query = new String(queryBuff.toByteArray());
db.execSQL(query);
queryBuff.clear();
query = null;
}
}
in.close();
queryBuff.clear();
And my GC_CONCURENT occurs when there is "new String" in the loop, after the end of the loop.
Thanks !
EDIT :
I'm a little annoyed, because my memory-leak didn't occurs in this part of code but in a part of code executed laterly (don't know what for now) but my problem wasn't a problem, app worked properly in fact...
Sorry !
Related
I'm having a problem in my Java application where Apache Jena will never stop a SPARQL query until it's finished, even if I explicitly tell it to stop. Here's the code that gets called to run a query:
try {
Model union = null;
union = ModelFactory.createOntologyModel(OntModelSpec.OWL_MEM_RULE_INF);
if (ontologies != null)
for (OntModel om : ontologies)
union = ModelFactory.createUnion(union, om);
Reasoner reasoner = ReasonerRegistry.getOWLReasoner();
reasoner = reasoner.bindSchema(union);
InfModel infmodel = ModelFactory.createInfModel(reasoner, triples);
query_running = true;
Query query = QueryFactory.create(query_string);
query_execution = QueryExecutionFactory.create(query, infmodel);
ResultSet rs = query_execution.execSelect();
for ( ; rs.hasNext(); ) {
// do stuff with results
}
} catch (Exception e) {}
finally {
stopQuery();
}
stopQuery() gets called at the end, but the method is also called when the user hits a "cancel" button. Here's what that method looks like:
public void stopQuery() {
try {
if (query_execution != null) {
//query_execution.close();
query_execution.abort();
query_execution = null;
System.out.println("STOPPED");
}
} catch (Exception e) { e.printStackTrace(); }
query_running = false;
}
When the cancel button is hit while a query is running (10+ minutes on a relatively small dataset...?), the method gets called, but the query continues to run in the background. I know it's still running because I can see the application in task manager taking up 30%+ CPU until the query presumably completes. I've tried .abort(), .close(), and both at the same time, but I cannot figure out how to stop the query mid-execution. I've even tried wrapping the query code in a separate thread, but that doesn't work either. It makes sense that threading wouldn't solve the problem because the thread needs to see the interrupt request, but the code is freezing on a particular line. The code seems to freeze on rs.hasNext(), but not the first check. It will run quickly with the first x results of the ResultSet (which are likely explicit statements it finds easily), but then it will seem to freeze for a long while after that, likely searching for implicit results with the reasoner. How can I force the query to stop? I don't want to use a timeout -- I want the user to have the option to stop the query or let it play out. This problem is not specific to any one query or dataset.
Thanks.
I have a problem, the below code runs fine if I run it without the autoCommit property, however I would prefer to run it as a transaction, the code basically inserts an article's header information and then the list each articles associated with it (so it's like a one-to-many relationship), so I could like to commit everything in one go rather than first the article information and then its items. The issue is that when I reach to the cn.commit() line, I get an exception that says "Closed Statement"
database insertion method
public static void addArticle(Article article) throws SQLException {
Connection cn = null;
PreparedStatement ps = null;
StringBuffer insert = new StringBuffer();
StringBuffer itemsSQL = new StringBuffer();
try {
article.setArticleSortNum(getNextArticleNum(article.getShopId()));
article.setArticleId(DAOHelper.getNextId("article_id_sequence"));
cn = DBHelper.makeConnection();
cn.setAutoCommit(false);
insert.append("insert query for article goes here");
ps = cn.prepareStatement(insert.toString());
int i = 1;
ps.setLong(i, article.getArticleId()); i++;
ps.setLong(i, article.getShopId()); i++;
ps.setInt(i, article.getArticleNum()); i++;
// etcetera...
ps.executeUpdate();
itemsSQL.append("insert query for each line goes here");
itemStatement = cn.prepareStatement(itemsSQL.toString());
for(Article item : article.getArticlesList()) {
item.setArticleId(article.getArticleId());
i= 1;
itemStatement.setLong(i, item.getArticleId()); i++;
itemStatement.setInt(i, item.getItemsOnStock()); i++;
itemStatement.setInt(i, item.getQuantity()); i++;
// etcetera...
itemStatement.executeUpdate();
}
cn.commit();
} catch (SQLException e) {
cn.rollback();
log.error(e.getMessage());
throw e;
}
finally {
DBHelper.releasePreparedStatement(ps);
DBHelper.releasePreparedStatement(itemStatement);
DBHelper.releaseConnection(cn);
}
}
I also had the items insertion where the For is running with addBatch() then executeBatch but also the same Closed Statement error upon reaching cn.commit()... I dont understand why its closing, all connections and everything is released in the finally clause, so I get the feeling I'm making some fundamental error I'm not aware of... Any ideas? Thanks in advance!
EDIT: Below is the stack trace:
java.sql.SQLException: Closed Statement at
oracle.jdbc.dbaccess.DBError.throwSqlException(DBError.java:189) at
oracle.jdbc.dbaccess.DBError.throwSqlException(DBError.java:231) at
oracle.jdbc.dbaccess.DBError.throwSqlException(DBError.java:294) at
oracle.jdbc.driver.OracleStatement.ensureOpen(OracleStatement.java:6226)
at
oracle.jdbc.driver.OraclePreparedStatement.sendBatch(OraclePreparedStatement.java:592)
at
oracle.jdbc.driver.OracleConnection.commit(OracleConnection.java:1376)
at com.evermind.sql.FilterConnection.commit(FilterConnection.java:201)
at
com.evermind.sql.OrionCMTConnection.commit(OrionCMTConnection.java:461)
at com.evermind.sql.FilterConnection.commit(FilterConnection.java:201)
at com.dao.ArticlesDAO.addArticle(ArticlesDAO.java:571) at
com.action.registry.CustomBaseAction.execute(CustomBaseAction.java:57)
at
org.apache.struts.action.RequestProcessor.processActionPerform(RequestProcessor.java:431)
at
org.apache.struts.action.RequestProcessor.process(RequestProcessor.java:236)
at
org.apache.struts.action.ActionServlet.process(ActionServlet.java:1196)
at
org.apache.struts.action.ActionServlet.doPost(ActionServlet.java:432)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:760) at
javax.servlet.http.HttpServlet.service(HttpServlet.java:853) at
com.evermind.server.http.ServletRequestDispatcher.invoke(ServletRequestDispatcher.java:765)
at
com.evermind.server.http.ServletRequestDispatcher.forwardInternal(ServletRequestDispatcher.java:317)
at
com.evermind.server.http.HttpRequestHandler.processRequest(HttpRequestHandler.java:790)
at
com.evermind.server.http.HttpRequestHandler.run(HttpRequestHandler.java:270)
at
com.evermind.server.http.HttpRequestHandler.run(HttpRequestHandler.java:112)
at
com.evermind.util.ReleasableResourcePooledExecutor$MyWorker.run(ReleasableResourcePooledExecutor.java:192)
at java.lang.Thread.run(Unknown Source)
EDIT 2:
These are the parameters in the driver's datasource config, I thought the debugging process might be making it time out, but even finishing in less than a second throws the closed statement exception
min-connections="20"
max-connections="200"
inactivity-timeout="20"
stmt-cache-size="40"/>
It's usually best to create a statement, use it and close it as soon as possible, and it does no harm to do so before the transaction gets committed. From reading the Oracle tuturial about the batch model it's sounding like it could be a problem to have multiple statements open at one time. I would try closing the ps object before working with the itemStatement, then moving the initialization
itemStatement = cn.prepareStatement(itemsSQL.toString());
to directly above the for loop, and also move where you close the itemStatement to immediately after the for loop:
PreparedStatement itemStatement = cn.prepareStatement(itemsSQL.toString());
try {
for(Article item : article.getArticlesList()) {
item.setArticleId(article.getArticleId());
i= 1;
itemStatement.setLong(i, item.getArticleId()); i++;
itemStatement.setInt(i, item.getItemsOnStock()); i++;
itemStatement.setInt(i, item.getQuantity()); i++;
// etcetera...
itemStatement.executeUpdate();
}
} finally {
DBHelper.releasePreparedStatement(itemStatement);
}
It looks like what is going on is you have some batching parameter set on the connection that is causing the connection to try to look for unfinished business in the statement to finish up; it's finding the statement is already closed and the connection is complaining about it. This is weird because at the point the commit blows up on you the code hasn't reached the finally where the statement gets closed.
Reading up on Oracle batching models may be helpful. Also check the JDBC driver version and make sure it's right for the version of Oracle you're using, and see if there are any updates available for it.
I have a java application which read files and writes to oracle db row by row.
We have come across a strange error during batch insert which does not occur during sequential insert. The error is strange because it occurs only with IBM JDK7 on AIX platform and I get this error on different rows every time. My code looks like below:
prpst = conn.prepareStatement(query);
while ((line = bf.readLine()) != null) {
numLine++;
batchInsert(prpst, line);
//onebyoneInsert(prpst, line);
}
private static void batchInsert(PreparedStatement prpst, String line) throws IOException, SQLException {
prpst.setString(1, "1");
prpst.setInt(2, numLine);
prpst.setString(3, line);
prpst.setString(4, "1");
prpst.setInt(5, 1);
prpst.addBatch();
if (++batchedLines == 200) {
prpst.executeBatch();
batchedLines = 0;
prpst.clearBatch();
}
}
private static void onebyoneInsert(PreparedStatement prpst, String line) throws Exception{
int batchedLines = 0;
prpst.setString(1, "1");
prpst.setInt(2, numLine);
prpst.setString(3, line);
prpst.setString(4, "1");
prpst.setInt(5, 1);
prpst.executeUpdate();
}
I get this error during batch insert mode :
java.sql.BatchUpdateException: ORA-01461: can bind a LONG value only for insert into a LONG column
at oracle.jdbc.driver.OraclePreparedStatement.executeBatch(OraclePreparedStatement.java:10345)
I already know why this Ora error occurs but this is not my case. I am nearly sure that I am not setting some large data to a smaller column. May be I am hitting some bugs in IBM jdk7 but could not prove that.
My question if there is a way that I can avoid this problem ? One by one insert is not an option because we have big files and it takes too much time.
Try with
prpst.setInt(5,new Integer(1))
What is the type of variable "numLine"?
Can you share type of columns corresponding to the fields you set in PreparedStatement?
Try once by processing with "onebyoneInsert". Share the output for this case. It might help identifying root cause.
Also print value of "numLine" to console.
This is my FULL test code with the main method:
public class TestSetAscii {
public static void main(String[] args) throws SQLException, FileNotFoundException {
String dataFile = "FastLoad1.csv";
String insertTable = "INSERT INTO " + "myTableName" + " VALUES(?,?,?)";
Connection conStd = DriverManager.getConnection("jdbc:xxxxx", "xxxxxx", "xxxxx");
InputStream dataStream = new FileInputStream(new File(dataFile));
PreparedStatement pstmtFld = conStd.prepareStatement(insertTable);
// Until this line everything is awesome
pstmtFld.setAsciiStream(1, dataStream, -1); // This line fails
System.out.println("works");
}
}
I get the "cbColDef value out of range" error
Exception in thread "main" java.sql.SQLException: [Teradata][ODBC Teradata Driver] Invalid precision: cbColDef value out of range
at sun.jdbc.odbc.JdbcOdbc.createSQLException(Unknown Source)
at sun.jdbc.odbc.JdbcOdbc.standardError(Unknown Source)
at sun.jdbc.odbc.JdbcOdbc.SQLBindInParameterAtExec(Unknown Source)
at sun.jdbc.odbc.JdbcOdbcPreparedStatement.setStream(Unknown Source)
at sun.jdbc.odbc.JdbcOdbcPreparedStatement.setAsciiStream(Unknown Source)
at file.TestSetAscii.main(TestSetAscii.java:21)
Here is the link to my FastLoad1.csv file. I guess that setAsciiStream fails because of the FastLoad1.csv file , but I am not sure
(In my previous question I was not able to narrow down the problem that I had. Now I have shortened the code.)
It would depend on the table schema, but the third parameter of setAsciiStream is length.
So
pstmtFld.setAsciiStream(1, dataStream, 4);
would work for a field of length 4 bytes.
But I dont think it would work as you expect it in the code. For each bind you should have separate stream.
This function setAsciiStream() is designed to be used for large data values BLOBS or long VARCHARS. It is not designed to read csv file line by line and split them into separate values.
Basicly it just binds one of the question marks with the inputStream.
After looking into the provided example it looks like teradata could handle csv but you have to explicitly tell that with:
String urlFld = "jdbc:teradata://whomooz/TMODE=ANSI,CHARSET=UTF8,TYPE=FASTLOADCSV";
I don't have enough reputation to comment, but I feel that this info can be valuable to those navigating fast load via JDBC for the first time.
This code will get the full stack trace and is very helpful for diagnosing problems with fast load:
catch (SQLException ex){
for ( ; ex != null ; ex = ex.getNextException ())
ex.printStackTrace () ;
}
In the case of the code above, it works if you specify TYPE=FASTLOADCSV in the connection string, but when run multiple times will fail due to the creation of the error tables _ERR_1 and _ERR_2. Drop these tables and clear out the destination tables to run again.
Nearly 1 month my program worked fine, but today im getting the error above when starting it.
The error comes up in this line:
sounds.put(key, (Clip)AudioSystem.getLine(new Line.Info(Clip.class)));
I have no idea why.
After i read the accepted answer here, i reinstalled eclipse, deleted metadata and so on, but the error still comes up.
The code is to 100% right, but any way ill post the code near the error:
protected SoundArchive(String soundArchive, boolean debugFrame){
f = new File(this.getClass().getResource("../" + soundArchive).toString().substring(5)).listFiles();
sounds = new HashMap<String, Clip>();
try{
for(int i = 0; i < f.length; i++){
String key = f[i].toString().split("\\\\")[f[i].toString().split("\\\\").length - 1].split("\\.")[0];
sounds.put(key, (Clip)AudioSystem.getLine(new Line.Info(Clip.class)));
sounds.get(key).open(AudioSystem.getAudioInputStream(f[i]));
}
}
catch(Exception e){
e.printStackTrace(System.out);
}
if(debugFrame) debugFrame();
}
So, does anyone know what i have to do now? I've my back to the wall....