FYI - I'm not a developer, but write code when I have to :) Trying to write some java code to update a database in a batched fashion for multiple records. As I'm inserting new rows, I'm querying another table to find relevant data to add relevant date.
The code seems to work, but my problem is performance. I'm seeing that the full batch of dml statements take about 1 second per statement to execute. I'm updating several thousand records, so this job will take quite awhile to execute. So, what I'm looking for is any other ideas on how I can do this while maximizing performance.
Here's what I'm doing right now.
for(Referrer_UpdateSet i : referrerUpdateSet)
{
String dmlStatement = "INSERT INTO TempRefURL (firstTouchDate) " +
"(SELECT activityDateTime as firstTouch "+
"FROM referrer_URL_backup_10292014 "+
"WHERE mktPersonId = ? "+
"ORDER BY activityDateTime ASC LIMIT 1)";
stmt = mktoUTMConn.prepareStatement(dmlStatement);
stmt.setInt(1, i.id);
//System.out.println(stmt+" \n");
stmt.executeUpdate();
}
mktoUTMConn.commit();
I'm also trying preparedStatements.addBatch, but it doesn't seem to be working (only 1 row inserted..)
System.out.println("updating temp table with referrer URL data");
//iterate through array of parsed referrer URLs
String dmlStatement = "UPDATE dml_sandbox.TempRefURL SET Referrer_URL = ? " + "WHERE id = ?";
for(Referrer_UpdateSet i : referrerUpdateSet){
stmt = mktoUTMConn.prepareStatement(dmlStatement);
stmt.setInt(2, i.id);
stmt.setString(1, i.cleanURL);
//System.out.println(stmt+" \n");
stmt.addBatch();
//stmt.executeUpdate();
//System.out.println(stmt+" \n");
}
stmt.executeBatch();
System.out.println("Done updating temp table with referrer URL data");
mktoUTMConn.commit();
Any suggestions would be greatly appreciated. Thanks!
Simple fix. See my comment above. Here's the new code:
String dmlStatement = "UPDATE dml_sandbox.TempRefURL SET Referrer_URL = ? " + "WHERE id = ?";
stmt = mktoUTMConn.prepareStatement(dmlStatement);
//iterate through array of parsed referrer URLs
for(Referrer_UpdateSet i : referrerUpdateSet){
stmt.setInt(2, i.id);
stmt.setString(1, i.cleanURL);
stmt.addBatch();
stmt.executeUpdate();
}
System.out.println(stmt+" \n");
int[] recordsAffected = stmt.executeBatch();
System.out.println("Done updating temp table with referrer URL data");
System.out.println(recordsAffected.length + " records affected");
mktoUTMConn.commit();
Related
I am trying to make a java program program can takes just one row/line from phpMyAdmin database. what I mean is in the photo
The problem is that I couldnt figure out how to take just the first line because its always keep giving me all the id'S , dafat's , etc. Is there any way I can get every line alone or even every data alone (jsut 1 id from the first line for example). I would appreciate who can help me with this.
Connection con = myConnection.getconnection();
try{
PreparedStatement ps = con.prepareStatement("SELECT `id`, `dafat`, `sinif`, `adet`, `price`, `type`, `total` FROM "+ff1+" WHERE 1 ");
ResultSet resultset = ps.executeQuery();
System.out.println(resultset.getString("id"));
}
catch (Exception e) {
}
Use the LIMIT clause:
PreparedStatement ps = con.prepareStatement(
"SELECT `id`, `dafat`, `sinif`, `adet`, `price`, `type`, `total` FROM "
+ ff1 + " LIMIT 1");
If you want the first id value (the smallest one) you can combine with ORDER BY, as in:
PreparedStatement ps = con.prepareStatement(
"SELECT `id`, `dafat`, `sinif`, `adet`, `price`, `type`, `total` FROM "
+ ff1 + " ORDER BY id LIMIT 1");
Replace WHERE whit LIMIT
PreparedStatement ps = con.prepareStatement("SELECT `id`, `dafat`, `sinif`, `adet`, `price`, `type`, `total` FROM "+ff1+" ORDER BY id LIMIT 1 ");
Use Rownum in your where clause like:
"SELECT * FROM all_objects WHERE rownum < 2"
https://mfaisal1521.blogspot.com/2020/04/rownum-in-sql.html
And also resultset.getString("id") start fetch result from first index of query untill we dont move our cursor to next resultset.
I've encountered a strange problem. I've got a table, where I want insert data to from java servlet.
The table consits of 3 fields - id - serial (auto inc), name - text, path - text.
So, when I try to insert data with this code:
PreparedStatement statement = null;
ResultSet rs = null;
statement = context.getDBConnection().prepareStatement("INSERT INTO systems (name, path) VALUES ('" + request.getParameter("system_name") + "','" + request.getParameter("system_path") + "')");
rs = statement.executeQuery();
rs.close();
Nothing happens then. Respone from postresql is "No results were returned by the query" BUT when I insert rows manualy through pgAdmin, their id evaulated like the rows were inserted. For example, I've manualy inserted row, so it got id = 1. Then, after a couple of tries to insert data through servlet, I try to insert data manualy again, and row id is 4, not 2.
UPDATE
Modified code, now getting no error, but nothing happens.
PreparedStatement statement = null;
// ResultSet rs = null;
statement = context.getDBConnection().prepareStatement("INSERT INTO systems (system_name, folder_path) VALUES (?,?)");
statement.setString(1, request.getParameter("system_name"));
statement.setString(2, request.getParameter("system_path"));
int i = statement.executeUpdate();
I am making a class in Java to connect and otherwise talk to a database, that I have set up elsewhere. I want to use the executeQuery method for a statement, and recieve a ResultSet, from where I will retrieve the information received in the ResultSet.
My issue is with the SQL command, as in; the query I'm sending to the database. Something appears to be wrong, and I get an SQLException at the very point where I send the command, meaning I must have done something wrong.
Maybe I'm writing something wrong? I can't tell, since I've simply tried to follow the example guidance provided for this course the best way I can.
Here is the query I'm trying to send:
"SELECT fr.DepartureLocation, fr.Destination, d.Date, d.Time FROM `FlightRoute` fr,
`Departure` d WHERE d.FlightRouteId = fr.Id AND d.Date > " + dateFrom + " AND
d.Date < " + dateTo + " AND fr.Destination = `" + destination + "`;"
The "dateFrom", "dateTo" and "destination" are all parameters for the method I'm calling, and I'm trying to limit the results I get from this statement, to those within a certain date-span as well as having a specific destination.
I might add that the dates are integers lined up such as this: 20131205
This should make it so that later dates are a higher number.
Is the way that I use the parameters with the SQL code wrong somehow, or did I make generally faulty SQL code here?
Thanks a lot, to anyone who might be able to provide a correct SQL code for me to use, so that I may see what I did wrong (since I'll have to make a few more similarly working SQL statements)!
Thanks in advance! :)
EDIT:
Here is the requested code where I am announcing the query and trying to execute it:
try
{
ArrayList<Departure> departures = new ArrayList<>();
Statement stmt = con.createStatement();
System.out.println("derp0");
String query = "SELECT fr.DepartureLocation, fr.Destination, d.Date, d.Time FROM FlightRoute fr, Departure d WHERE d.FlightRouteId = fr.Id AND d.Date > " + dateFrom + " AND d.Date < " + dateTo + " AND fr.Destination = `" + destination + "`;";
System.out.println("derp0.1");
ResultSet rs = stmt.executeQuery(query);
System.out.println("derp1");
As a short explaination, the souts are to check where the SQLException occured, and I can only say that the exception happens right after "derp0.1".
I suggest the following change:
PreparedStatement ps = connection.prepareStatement(
"SELECT fr.DepartureLocation, fr.Destination, d.Date, d.Time FROM `FlightRoute` fr,"+
"`Departure` d WHERE d.FlightRouteId = fr.Id AND d.Date > ? AND d.Date < ? "+
" AND fr.Destination = ?");
ps.setDate(1, dateFrom);
ps.setDate(2, dateTo);
ps.setString(3, destination); //assuming destination is a String
ResultSet rs = ps.executeQuery();
In JDBC you should always use parametrisation, as above. It might also solve your problem, which could be caused by an invalid date format.
This is the code block in question:
String sq = "INSERT INTO survey (session_id, character_id, timestamp) VALUES (?,?,?)";
PreparedStatement sadd = conn.prepareStatement(sq, PreparedStatement.RETURN_GENERATED_KEYS);
sadd.setLong(1, sessionId);
sadd.setLong(2, character_id);
sadd.setString(3, dateTime);
int affectedrows = sadd.executeUpdate();
//get the ID
long resultId = 0;
ResultSet key = sadd.getGeneratedKeys();
if (key.next()) {
resultId = key.getLong(1);
}
This query worked fine without the PreparedStatement.RETURN_GENERATED_KEYS option, but when I add it suddenly executeUpdate() throws an exception:
com.microsoft.sqlserver.jdbc.SQLServerException: A result set was generated for update.
If I take the PreparedStatement.RETURN_GENERATED_KEYS out, it works again fine. Out of frustration, I changed executeUpdate() to executeQuery() just to see if I could get the key back and got an exception that it can't get keys because the statement must be executed first.
How can I get the generated key? I am using SQL Server 2008 and the latest JDBC driver.
Looks like a driver bug to me.
You should try a newer 4.0 driver from here -> http://www.microsoft.com/download/en/details.aspx?id=11774
If that does not work, one work around would be to create an 'insert' stored procedure and return the generated id as a stored procedure output parameter.
Looks like a bug. Could you give the uglier alternative a try?
String dateTimeS = new SimpleDateFormat("yyyy-MM-dd'T'HH:mm").format(dateTime);
String sq = "INSERT INTO survey (session_id, character_id, timestamp) "
+ "VALUES (" + sessionId + ", " + character_id + ", '" + dateTimeS + "')";
Statement sadd = conn.createStatement();
int affectedrows = sadd.executeUpdate(sq, Statement.RETURN_GENERATED_KEYS);
I'm having the same issue with the 4.0 & 4.1 JDBC drivers. After a while an insert on a autonumber table would give a "A result set was generated for update." at random. I use connection pooling and somehow the driver can get into a state where executeUpdate in combination with Statement.RETURN_GENERATED_KEYS doesn't work anymore. I found out that in this state an executeQuery does the trick, but in the initial state executeQuery does not work. This lead me to the following workaround:
PreparedStatement psInsert = connection.prepareStatement("INSERT INTO XYZ (A,B,C) VALUES(?,?,?)", Statement.RETURN_GENERATED_KEYS);
ResultSet rs = null;
try {
psInsert.setString(1, "A");
psInsert.setString(2, "B");
psInsert.setString(3, "C");
Savepoint savePoint = connection.setSavepoint();
try {
psInsert.executeUpdate();
rs = psInsert.getGeneratedKeys();
} catch (SQLServerException sqe)
{
if (!sqe.getMessage().equals("A result set was generated for update."))
throw sqe;
connection.rollback(savePoint);
rs = psInsert.executeQuery();
}
rs.next();
idField = rs.getInt(1);
} finally {
if(rs != null)
rs.close();
psInsert.close();
}
I am attempting to pull results from an Oracle database. I have written a query that is correct, and produces accurate results when issued manually in sqlplus. Furthermore, the code works as expected when when the query matches only one row (In other words, when the ResultSet has only one row, everything works). However, when more than one row match the query, the ResultSet returned by the Oracle JDBC is empty.
public Component[] getAllComponents(int typeId, int osId) throws SQLException
{
String query= "SELECT c.component_id, c.component_name, c.component_version, c.type_id, c.post_download_instructions, "
+ "o.os_id, o.os_name, o.description AS os_description, "
+ "i.file_location, i.release_date, i.patch_number, i.file_id, "
+ "i.description AS i_description "
+ "FROM components c, installation_files i, operating_systems o "
+ "WHERE c.type_id = ? "
+ "AND i.os_id = ? "
+ "AND c.component_id = i.component_id "
+ "AND i.os_id = o.os_id";
ResultSet results = null;
PreparedStatement stmt = null;
ArrayList<Component> found = new ArrayList<Component>();
try {
stmt = dbConn.prepareStatement(query); //dbConn is member variable
stmt.setInt(1, typeId);
stmt.setInt(2, osId);
results = stmt.executeQuery();
while(results.next()){
//Some logic
}
} finally {
if(results != null) results.close();
if(stmt != null) stmt.close();
dbConn.close();
}
//More Code
//etc. etc.
Inspecting the ResultSet shows that calling ResultSet.next() never produces true when the fetched results should contain more than one row. However, issuing the query manually does produce results, and when only one row is returned, everything works fine. Does anyone know what's going on? I'm using Oracle's ojdbc6.jar.
Thanks!
Before that query you can check if there really are some components with COUNT(*) instead of all fields. Then run your query only if COUNT(*) is one or more.