I was wondering if any of you could help me identify why my simple test applications are giving me so different results. I made two test programs that makes a simple database query and loop through the results. The first one was made by using PHP the second was made with Java.
I knew that Java would perform better, but I have hard time believing that java would perform almost 20 times better (see results below).
PHP:
$sql = "select * from message where user_id=20";
$db = get_PDO();
$stm = $db->prepare($sql);
for($i=0;$i<10000;$i++)
{
echo $i."\n";
$res = $stm->execute();
$rows = $stm->fetchAll();
}
echo "done";
The get_PDO is just a function that connects to the database and returns an pdo object.
Java:
public class Connect
{
public static void main (String[] args)
{
Connection conn = null;
Statement st= null;
try
{
String userName = ""; // db username
String password = ""; // db password
String url = "jdbc:mysql://localhost/test"; //test = db name
Class.forName ("com.mysql.jdbc.Driver").newInstance ();
conn = DriverManager.getConnection (url, userName, password);
System.out.println ("Database connection established");
}
catch (Exception e)
{
System.err.println ("Cannot connect to database server: "+e.toString());
}
finally
{
if (conn != null)
{
String query="select * from message where user_id=20";
try{
st = conn.createStatement();
}
catch(Exception e)
{
e.printStackTrace();
try{
conn.close();
}
catch(Exception er) {}
return;
}
for(int i=0;i<10000;i++)
{
System.out.println(i);
try
{
ResultSet rs = st.executeQuery(query);
while(rs.next())
{
}
}
catch(Exception e)
{
e.printStackTrace();
}
}
try
{
conn.close ();
System.out.println ("Database connection terminated");
}
catch (Exception e) { /* ignore close errors */ }
}
}
}
}
The results:
I measured the performance using time. (ie. time php test.php and time java Connect)
PHP:
real 1m34.337s
user 1m32.564s
sys 0m0.716s
Java:
real 0m5.388s
user 0m4.428s
sys 0m0.972s
Is java really that much faster or have I done something stupid? :)
Depending on the number of messages, PHP might need much more memory because the complete result set is held after calling fetchAll().
AFAIK PHP creates an associative Array for the result, which might also be time consuming.
It could be that in PHP, you retrieve all the data from the database by calling fetchAll() whereas in Java, you just move through the result set using rs.next() without actually reading. Depending on the JDBC driver implementation, this might give an opportunity for optimization that's not possible in the way the PHP version is implemented.
Related
I get an exception that the database is locked and I try to close connection and statement, but here is the problem unreachable statement in try block.
public static ResultSet getData (String query){
try {
Connection conn = ConnectionProvider.connect();
Statement st = conn.createStatement();
ResultSet rs = st.executeQuery(query);
return rs;
try {
conn.close();
st.close();
rs.close();
} catch (Exception e) {
JOptionPane.showMessageDialog(null, e);
return null;
}
} catch (Exception e) {
JOptionPane.showMessageDialog(null, e);
System.out.println(e);
return null;
}
Everything you do after the first return statement in the outer try block can never be reached. The second try block will never be executed, therefore you have some unreachable code here.
I think what you want to do is this:
try {
Connection conn = ConnectionProvider.connect();
Statement st = conn.createStatement();
ResultSet rs = st.executeQuery(query);
try {
conn.close();
st.close();
rs.close();
} catch (Exception e) {
JOptionPane.showMessageDialog(null, e);
return null;
}
return rs;
} catch (Exception e) {
JOptionPane.showMessageDialog(null, e);
System.out.println(e);
return null;
}
Although I will say that this may just not be the correct place to use the try-catch. Maybe what you should do here is throw the possible exceptions back to whoever calls this method instead - usually nested try-catch blocks are very rarely actually used/needed in this way. Also if you encounter an exception, you just return null, instead of handling what this means for the rest of your application.
What you could also try is, assuming that at this point your query was successful:
try {
Connection conn = ConnectionProvider.connect();
Statement st = conn.createStatement();
ResultSet rs = st.executeQuery(query);
try {
conn.close();
st.close();
rs.close();
} catch (Exception e) {
throw e;
} finally {
return rs;
}
} catch (Exception e) {
throw e;
}
Though for this approach you should really think about what you need to happen in each error case.
As an additional side note, even though I'm aware that many people ignore this:
Always try to use the proper class for the Exceptions you expect in a catch-block, and if it's only for readabilities sake.
There are several things to improve in the piece of code you post.
First, you need to close the resources (Connection, Statement and ResultSet) in reverse order with respect to the opening order. So, first you should close the ResultSet, second the Statement, and finally the Connection.
Closing in the order you are doing might cause problems when closing Statement / ResultSet with the Connection already closed.
By other hand, starting in Java 7, you have the try-with-resources construct, that closes resources for you. You can take a look at The try-with-resources Statement docs.
As far as I understood, closing the connection objects in finally block is the best practice. However, if rs.close() / ps.close() throws an exception at the finally block, it won't execute the conn.close(). Therefore I used to close connections at two positions (as in sample provided), once directly after use, and secondly additionally in a finally-block using a check for null. But some consider block 1 as redundant code. Is it really redundant or is there proper way to address this issue without closing the connection in two places?
Connection conn = null;
PreparedStatement ps = null;
ResultSet rs = null;
try {
conn = dataSource.getConnection(); // geting the connection object
ps = connection.prepareStatement(INSERT_QUERY);
rs = ps.executeQuery();
// some logic here ...
// ---- block 1 ----
ps.close()
ps = null;
rs.close();
rs = null;
conn.close();
conn = null;
// ---- end block 1 ----
} catch (SQLException e) {
// exception handling ...
} finally {
closeQuietly(conn, ps, rs);
}
private void closeQuietly(Connection connection, PreparedStatement ps, ResultSet rs) {
if (rs != null) {
try {
rs.close();
} catch (SQLException e) {}
}
if (ps != null) {
try {
ps.close();
} catch (SQLException e) {}
}
if (connection != null) {
try {
connection.close();
} catch (SQLException e) {}
}
}
is there proper way to address this issue without closing the connection in two places?
Yes:
try (Connection conn = dataSource.getConnection(); // geting the connection object
Prepared Statement ps = connection.prepareStatement(INSERT_QUERY);
ResultSet rs = ps.executeQuery();) {
// ...
}
This is the 'try-with-resources' syntax. Everything declared inside the () after the try is guaranteed to be closed.
Yes, it would be called twice if everything goes fine with your code. That's the reason, people prefer to close any sort of connections (jdbc, stream etc) in the finally block.
As you know, The finally block gets executed whether program executed correctly or not.
So, I would recommend that you should not closing code after the use.
Jitendra
Block 1 is indeed redundant, as closeQuietly will always run due to the finally block.
closeQuietly does the right thing:
As each resource is surrounded by its own try-catch block, the code to cleanup the connection will run even if the block closing the statement or resultset throw exceptions: Those exceptions will be caught and ignored by their try-catch blocks.
I am using Netbeans( I have jar file in my library) to connect to SQL Server and wondering why nothing is happening. I am can see the following message printing on my screen Loading the Driver for SQL Server at xx.xx.x.xxx and nothing happens after that:
I am also trying to test whether the connection is null or not but for some reason System.out.println("Connection Successful !"); is not printing anything and hence I suspect it's not getting connected. Could anyone please see what's going on here:
My code is here:
try {
String QueryStringGlobalAlertLogs = "";
try{
System.out.println("Loading the Driver for SQL Server at xx.xx.x.xxx");
Class.forName("com.microsoft.sqlserver.jdbc.SQLServerDriver");
}
catch(java.lang.ClassNotFoundException e){
return 0;
}
connRemoteforGlobal = java.sql.DriverManager.getConnection("jdbc:sqlserver://xx.xx.x.xxx;databaseName=Alerts",RemoteSQLServerUser,RemoteSQLServerPass);
if(connRemoteforGlobal != null)
{
System.out.println("Connection Successful !");
}
QueryStringGlobalAlertLogs = // My INSERT Query here
}// End of first try
catch(SQLException ex2){
ex2.printStackTrace();
System.out.println("Error Trace in Connection : " + ex2.getMessage());
}
You must catching Class.forName and getConnection together.
try this:
try{
Class.forName("com.microsoft.sqlserver.jdbc.SQLServerDriver");
DriverManager.getConnection("jdbc:sqlserver://xx.xx.x.xxx;databaseName=Alerts",RemoteSQLServerUser,RemoteSQLServerPass);
}catch(SQLException e1){
// handle error
}catch(ClassNotFoundException e2){
// handle error
}
I have a jsp wherein for each row of the table i need to display the image present in the database. I retrieve all the table row data from database including image as Blob and store it in a bean. The image is stored in the bean as byte array like this:
photo = rs.getBlob("PHOTO");
photoByteArray = photo.getBytes(1, (int)photo.length());
While looping over the list of beans in jsp, the src attribute points to a servlet like this:
<img class="img" width="55" height="50" src="displayThumbnail?photoData=${part.photoData}">
which serves the image like shown below but they don't show up however upon debugging the byte array do seem to have data.
protected void processRequest(HttpServletRequest request, HttpServletResponse response) throws IOException {
response.setContentType("image/jpeg");
OutputStream o = response.getOutputStream();
String photoDataStr = request.getParameter("photoData");
byte[] photoData = null;
if(photoDataStr != null) {
photoData = photoDataStr.getBytes();
}
o.write(photoData);
o.close();
}
However the image doesn't show up. Now, if i query the database for each individual image as shown below, the images do show up fine in that case.
protected void processRequest(HttpServletRequest request, HttpServletResponse response) {
PreparedStatement pstmt = null;
ResultSet rs = null;
Connection conn = null;
try {
if(conn == null) {
conn = open();
}
pstmt = conn.prepareStatement("select photo from PART_PHOTOS where id = ?");
String id = request.getParameter("id");
pstmt.setString(1, id);
rs = pstmt.executeQuery();
if (rs.next()) {
Blob b = rs.getBlob("photo");
response.setContentType("image/jpeg");
response.setContentLength((int) b.length());
InputStream is = b.getBinaryStream();
OutputStream os = response.getOutputStream();
byte buf[] = new byte[(int) b.length()];
is.read(buf);
os.write(buf);
os.close();
is.close();
}
} catch (Exception ex) {
System.out.println(ex.getMessage());
ex.printStackTrace();
} finally {
if (rs != null) {
try {
rs.close();
} catch (SQLException e) {
e.printStackTrace();
}
rs = null;
}
if (pstmt != null) {
try {
pstmt.close();
} catch (SQLException e) {
e.printStackTrace();
}
pstmt = null;
}
//check if it's the end of the loop
if (conn != null) {
try {
conn.close();
} catch (SQLException e) {
e.printStackTrace();
}
conn = null;
}
}
}
I would highly appreciate if anyone can provide any recommendations around the same.
You're assuming that you can put random binary data into an HTML file, and it will be parsed correctly, and sent back to your server intact. This is a bad assumption! If nothing else, the byte that corresponds to the ASCII for the quote character is going to cause problems, right? Not to mention encoding issues, and the fact that the parameters to a URL must be urlencoded. This is just doomed to fail.
To make this work, you'd have to have some kind of explicit text encoding of the binary data when you serve the page (base64, maybe), and then decode the servlet parameter back to binary image data after the URL is posted back.
Your first processRequest() snippet is only sending back the byte representation of the photoData request parameter and not the photo data identified by the parameter. Looks like a bug in your code.
It seems you are trying to solve your problem in the wrong manner. When you first create the HTML table, storing the image in your "bean" from your first query gives you nothing unless you cache the data, and the subseqent displayThumbnail request retrieves the image from the cache, avoiding the database query.
If you do not want to mess with caching, then there is no need to store the image in your initial bean since it gives you nothing, and just do something like your second processRequest() snippet to fetch the image directly when the browser asks for it.
Your ${part.photoData} expression must return some ID. In the processRequest() method you must get that ID value (by using request.getParameter("photoData")) and by that value retrieve image from database (or better from cache or from file system) and send the binary data to web client.
There is a table phonenumbers with two columns: id, and number. There are about half a million entries in the table. Database is MySQL.
The requirement is to develop a simple Java EE application, connected to that database, that allows a user to download all numbervalues in comma separated style by following a specific URL.
If we get all the values in a huge String array and then concatenate them (with comma in between all the values) in a String and then send it down to the user, does it sound a proper solution?
The application is not public and will be used by a limited no. of people.
Your best bet is to not store the data in Java's memory in any way, but just write the obtained data to the response immediately as the data comes in. You also need to configure the MySQL JDBC driver to serve the resultset row-by-row by Statement#setFetchSize() as per the MySQL JDBC driver documentation, otherwise it will cache the whole thing in memory.
Assuming you're familiar with Servlets, here's a kickoff example which takes that all into account:
protected void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
response.setContentType("text/plain");
response.setHeader("Content-Disposition", "attachment;filename=numbers.txt"); // Force download popup.
Connection connection = null;
Statement statement = null;
ResultSet resultSet = null;
Writer writer = response.getWriter();
try {
connection = database.getConnection();
statement = connection.createStatement(ResultSet.TYPE_FORWARD_ONLY, ResultSet.CONCUR_READ_ONLY);
statement.setFetchSize(Integer.MIN_VALUE);
resultSet = statement.executeQuery("SELECT number FROM phonenumbers");
while (resultSet.next()) {
writer.write(resultSet.getString("number"));
if (!resultSet.isLast()) {
writer.write(",");
}
}
} catch (SQLException e) {
throw new ServletException("Query failed!", e);
} finally {
if (resultSet != null) try { resultSet.close; } catch (SQLException logOrIgnore) {}
if (statement != null) try { statement.close; } catch (SQLException logOrIgnore) {}
if (connection != null) try { connection.close; } catch (SQLException logOrIgnore) {}
}
}
There's a bit more to properly formatting CSV output. It would be easiest to use an existing library such as this one to generate the output file.
You can generate output to a file on disk (on the web server) and then redirect the browser to that file (with a cron job or whatever to clean up old data) or just stream the result directly back to the user.
If you are streaming directly be sure and set the MIME type to something that will trigger a download in the user's browser (e.g. text/csv or text/comma-separated-values)
If using Mysql 5.1+, I would simply use the proprietary syntax to dump the file somewhere and stream it in a Servlet response.
SELECT a,b,a+b INTO OUTFILE '/tmp/result.txt'
FIELDS TERMINATED BY ',' OPTIONALLY ENCLOSED BY '"'
LINES TERMINATED BY '\n'
FROM test_table;
http://dev.mysql.com/doc/refman/5.1/en/select.html
For so many records, if you still want to use JDBC, you may try the following:
fetch the number of records fetch few
records( using a query
limit ) and write them
if you reach the number of
records in a chunk, you fetch another
one until you reach the maximum
number of records