We Keep Coding

Apache POI 3.14 Writing SXSSF to ServletResponse OutputStream is corrupting workbook JSF

In my webapp, I'm building SXSSFWorkbook objects to generate reports with roughly 30-60k records each. I kick off a process through a request to first fetch and build each SXSSFWorkbook. I'm able to generate the report and open a FileOutputStream to export my object to my desktop(locally of course). However, I want to let user choose which report to download through a request(JSF)to the server. When I feed the OutputStream from the servlet response, I can download the .xlsx file but it tells me it's been corrupted. I've done some research, tried some different workarounds all with no results. Posted is my code. I'm kind of at a loss for what's going on here.
p.s. I was previously generating HSSFWorkbook objects and downloading them but they were starting to causing heap space issues. Hence, the switch to SXSSFWorkbook.
My command button
<h:commandButton value="Download Report" styleClass="secondary-button"
action="#{backingBean.getLatestReport}"
id="thirdReportButton">
</h:commandButton>
My action
public void getLatestReport() throws Exception {
FacesContext faces = FacesContext.getCurrentInstance();
String templateName = "Report.xlsx";
HttpServletResponse response = null;
OutputStream outputStream = null;
//workbookForLatestReport is a SXSSFWorkbook object
try {
if (workbookForLatestReport != null) {
response = (HttpServletResponse) faces.getExternalContext()
.getResponse();
response.reset();
response.setContentType("application/vnd.ms-excel");
response.setHeader("Content-disposition",
"attachment; filename=\"" + templateName + "\"");
outputStream = response.getOutputStream();
workbookForLatestReport.write(outputStream);
outputStream.close();
workbookForLatestReport.dispose();
}
faces.renderResponse();
} catch (Exception e) {
throw e;
}
}

Recently I successfully accomplished similar task so I might be able to help you.
I've just ran your code (on Payara 4.1 using Chrome as browser) adding part that you omitted in your post
#ManagedBean(name = "backingBean")
#ViewScoped
public class BackingBean {
//your command button should call getLatestReport and not getSecondReport() as in your original post
public void getLatestReport() throws Exception {
FacesContext faces = FacesContext.getCurrentInstance();
String templateName = "Report.xlsx";
HttpServletResponse response = null;
OutputStream outputStream = null;
//workbookForLatestReport is a SXSSFWorkbook object
//MY ADDITION START
//I've created SXSSFWorkbook object since your post did't contain this part
//100K rows, 100 columns
SXSSFWorkbook workbookForLatestReport = new SXSSFWorkbook(SXSSFWorkbook.DEFAULT_WINDOW_SIZE);
workbookForLatestReport.setCompressTempFiles(true);
SXSSFSheet sheet = workbookForLatestReport.createSheet();
for (int rowNumber = 0; rowNumber < 100000; rowNumber++) {
SXSSFRow row = sheet.createRow(rowNumber);
for (int columnNumber = 0; columnNumber < 100; columnNumber++) {
SXSSFCell cell = row.createCell(columnNumber);
cell.setCellValue("ROW " + rowNumber + " COLUMN " + columnNumber);
}
}
//MY ADDITION END
try {
if (workbookForLatestReport != null) {
response = (HttpServletResponse) faces.getExternalContext()
.getResponse();
response.reset();
response.setContentType("application/vnd.ms-excel");
response.setHeader("Content-disposition",
"attachment; filename=\"" + templateName + "\"");
outputStream = response.getOutputStream();
workbookForLatestReport.write(outputStream);
outputStream.close();
workbookForLatestReport.dispose();
}
faces.renderResponse();
} catch (Exception e) {
throw e;
}
}
}
It was working just fine and as expected.
I have 2 suggestions:
your command button "calls" action="#{backingBean.getSecondReport}"
but your managed bean action is named public void getLatestReport.
Check out if it is typing error or not.
compare your SXSSFWorkbook object creation code with my example. Are
there any crucial differences?

I was able to come to a solution with the aid of omifaces and changing some code around.
After I create my workbook, I use a ByteArrayStream to set a byte[] attribute on my model bean that can be referenced when the download listener from the jsf is clicked.
ByteArrayOutputStream bos;
byte[] workbookForLatestBytes;
XSSFXWorkbook workbookForLatestReport;
.
.
.
workbookForLatestReport = <generate the report here>
if(workbookForLatestReport != null){
bos = new ByteArrayOutputStream();
workbookForLatestReport.write(bos);
workbookForLatestBytes = bos.toByteArray();
workbookForPreviousReport.dispose();
}
Here is the action being fired from my JSF code.
<h:commandButton value="Download Report"
styleClass="secondary-button"
action="#{productRuleAuditCompareBackingBean.getSecondReport}"
id="thirdReportButton"
rendered="#{not empty productRuleAuditCompareModelBean.workbookForLatestBytes}">
</h:commandButton>
My backing bean action is as follows. I reset the HTTP Response before writing the byte array to the response output stream.
public void getSecondReport() {
FacesContext faces = FacesContext.getCurrentInstance();
try {
faces.getExternalContext().responseReset();
this.prAuditCompareModelBean.getLatestReport();
faces.responseComplete();
} catch (Exception e) {
PODBException.wrap(e, PODBExceptionInformation.create(
PODBExceptionType.ERROR,
ProductRuleAuditCompareBackingBean.class.getName(),
"getting first report",
"Error while getting second report for download", "1"));
}
}
Here I'm using Omniface Faces#sendFile method to write the workbook to the response outputstream.
public void getLatestReport() throws Exception {
try {
if (getWorkbookForLatestBytes() != null) {
Faces.sendFile(getWorkbookForLatestBytes(), reportName, true);
}
} catch (Exception e) {
throw e;
}
}

I want to retrieve image from database and display it in JSP using <img> tag. I am using my sql

I want to retrieve image from database and display it in JSP using img tag. I am using mySql.
public class DisplayImage extends HttpServlet {
public void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
//PrintWriter pw = response.getWriter();
String connectionURL = "jdbc:mysql://localhost:3306/demodb";;
java.sql.Connection con=null;
try{
Class.forName("com.mysql.jdbc.Driver").newInstance();
con=DriverManager.getConnection(connectionURL,"root","0321");
Statement st1=con.createStatement();
ResultSet rs1 = st1.executeQuery("select photo from contacts where contact_id='2'");
String imgLen="";
if(rs1.next()){
imgLen = rs1.getString(1);
System.out.println(imgLen.length());
}
rs1 = st1.executeQuery("select photo from contacts where contact_id='2'");
if(rs1.next()){
int len = imgLen.length();
byte [] rb = new byte[len];
InputStream readImg = rs1.getBinaryStream(1);
int index=readImg.read(rb, 0, len);
System.out.println("index"+index);
st1.close();
response.reset();
response.setContentType("image/jpg");
response.getOutputStream().write(rb,0,len);
response.getOutputStream().flush();
}
}
catch (Exception e){
e.printStackTrace();
}
}
}
This Code display image directly. But I want the path. So I can display it wherever I want. Are there any other ways to display this image in a dynamic web page?

The path depends on how your servlet is mapped in web.xml (as I don't see any annotations in your code). The other - but rather discouraged - way is to use data URI. See I want to retrieve image from mysql database using servlet and show that image along with that user details in table

Best structure for connection pooling with Java EE

I am trying to figure out the best structure for connection pools so that I can access the connection pool from any servlet and establish a connection to my database. I have been following some tutorials in setting up and configuring the datasource and connection pool and they all have them initialized and accessed in classes that extend HttpServlet. So it looks something like this:
public class DatabaseConnector extends HttpServlet {
private static final long serialVersionUID = 1L;
private DataSource dataSource;
private Connection connection;
private Statement statement;
public void init() throws ServletException {
try {
// Get DataSource
Context initContext = new InitialContext();
Context envContext = (Context)initContext.lookup("java:/comp/env");
dataSource = (DataSource)envContext.lookup("jdbc/test");
} catch (NamingException e) {
e.printStackTrace();
}
}
/**
* #see HttpServlet#HttpServlet()
*/
public DatabaseConnector() {
super();
}
/**
* #see HttpServlet#doGet(HttpServletRequest request, HttpServletResponse response)
*/
protected void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
ResultSet resultSet = null;
try {
// Get Connection and Statement
connection = dataSource.getConnection();
statement = connection.createStatement();
String query = "SELECT * FROM STUDENT";
resultSet = statement.executeQuery(query);
while (resultSet.next()) {
System.out.println(resultSet.getString(1) + resultSet.getString(2) + resultSet.getString(3));
}
} catch (SQLException e) {
e.printStackTrace();
}finally {
try {
if (resultSet != null) {resultSet.close();}
if (statement != null) {statement.close();}
if (connection != null) {connection.close();}
} catch (SQLException e) {
e.printStackTrace();
}
}
}
/**
* #see HttpServlet#doPost(HttpServletRequest request, HttpServletResponse response)
*/
protected void doPost(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
// TODO Auto-generated method stub
}
}
It looks like to me that this connection pool is only for this servlet which would only distribute connections when a get request is sent the this servlets URL. What if I want to have another servlet that needs to access the database. I was thinking of just removing the doGet and doPost methods in this servlet and leave the init so that the connection pool would be initialized at runtime, and then have a singleton reference to this servlet which could be used in other servlets. However, this doesn't seem like the right way to me. What is the best way to structure a connection pool that can be accessed from all servlets and listeners?
Thanks for your help!

Completely wrong.
The correct way to access a connection is to use a JNDI connection pool.
Servlets are HTTP listeners. They shouldn't have anything to do with databases.
A properly layered Java EE solution will restrict data sources to the service tier. It will check connections in and out, know about units of work and transactions, and interact with data access objects.
Servlets should deal with services, not data sources.

The logic to create a connection could be placed within a simple class.
public class ConnectionManager{
public static Connection getConnection(){
Connection connection = null;
try {
// Get DataSource
Context initContext = new InitialContext();
Context envContext = (Context)initContext.lookup("java:/comp/env");
dataSource = (DataSource)envContext.lookup("jdbc/test");
connection = dataSource.getConnection();
} catch (NamingException e) {
e.printStackTrace();
}
if(connection == null){
throw new RuntimeException("Cannot connect");
}
return connection;
}
}

The Jasper Reports servlet stopped working after calling response.getOutputStream()

I have code such as below. The program stopped working at line servletOutputStream = response.getOutputStream();. I don't know how to resolve this? Can anybody help me with this problem?
protected void processRequest(HttpServletRequest request, HttpServletResponse response)
throws ServletException, IOException, SQLException, JRException, ParserConfigurationException, SAXException {
response.setContentType("text/html;charset=UTF-8");
PrintWriter out = response.getWriter();
try {
out.println ("<html>");
out.println (" <head>");
out.println (" <meta http-equiv='Content-Type' content='text/html; charset=UTF-8'>");
out.println (" <title>JSP Page</title>");
out.println (" </head>");
out.println (" <body>");
out.println (" <h1>Hello iReport!</h1>");
String resourceName = "D:/classic.jrxml";
response.setContentType("application/pdf");
ServletOutputStream servletOutputStream = null;
servletOutputStream = response.getOutputStream(); // <--
InputStream reportStream = getServletConfig().getServletContext().getResourceAsStream(resourceName);
try {
Driver driver = new org.gjt.mm.mysql.Driver();
DriverManager.registerDriver(driver);
String conString = "jdbc:mysql://localhost:3306/quanlynhasach";
Properties info = new Properties();
info.setProperty("characterEncoding", "utf8");
info.setProperty("user", "root");
info.setProperty("password", "");
Connection con = DriverManager.getConnection(conString, info);
JasperRunManager.runReportToPdfStream(reportStream, servletOutputStream,new HashMap<Object, Object>(), con);
con.close();
}catch(Exception e){
StringWriter stringWriter = new StringWriter();
PrintWriter printWriter = new PrintWriter(stringWriter);
e.printStackTrace(printWriter);
response.setContentType("text/plain");
response.getOutputStream().print(stringWriter.toString());
}
out.println (" </body>");
out.println ("</html>");
} finally {
out.close();
}
} // <editor-fold defaultstate="collapsed" desc="HttpServlet methods. Click on the + sign on the left to edit the code.">
/**
* Handles the HTTP <code>GET</code> method.
* #param request servlet request
* #param response servlet response
* #throws ServletException if a servlet-specific error occurs
* #throws IOException if an I/O error occurs
*/
#Override
protected void doGet(HttpServletRequest request, HttpServletResponse response)
throws ServletException, IOException {
try {
try {
processRequest(request, response);
} catch (ParserConfigurationException ex) {
Logger.getLogger(iReport.class.getName()).log(Level.SEVERE, null, ex);
} catch (SAXException ex) {
Logger.getLogger(iReport.class.getName()).log(Level.SEVERE, null, ex);
}
} catch (SQLException ex) {
Logger.getLogger(iReport.class.getName()).log(Level.SEVERE, null, ex);
} catch (JRException ex) {
Logger.getLogger(iReport.class.getName()).log(Level.SEVERE, null, ex);
}
}
/**
* Handles the HTTP <code>POST</code> method.
* #param request servlet request
* #param response servlet response
* #throws ServletException if a servlet-specific error occurs
* #throws IOException if an I/O error occurs
*/
#Override
protected void doPost(HttpServletRequest request, HttpServletResponse response)
throws ServletException, IOException {
try {
try {
processRequest(request, response);
} catch (ParserConfigurationException ex) {
Logger.getLogger(iReport.class.getName()).log(Level.SEVERE, null, ex);
} catch (SAXException ex) {
Logger.getLogger(iReport.class.getName()).log(Level.SEVERE, null, ex);
}
} catch (SQLException ex) {
Logger.getLogger(iReport.class.getName()).log(Level.SEVERE, null, ex);
} catch (JRException ex) {
Logger.getLogger(iReport.class.getName()).log(Level.SEVERE, null, ex);
}
}
/**
* Returns a short description of the servlet.
* #return a String containing servlet description
*/
#Override
public String getServletInfo() {
return "Short description";
}// </editor-fold>

Look here:
PrintWriter out = response.getWriter();
// *snip*
servletOutputStream = response.getOutputStream();
You're getting both the Writer and OutputStream from the response. This is not allowed. Read their javadocs:
getOutputStream()
ServletOutputStream getOutputStream() throws java.io.IOException
Returns a ServletOutputStream suitable for writing binary data in the response. The servlet container does not encode the binary data.
Calling flush() on the ServletOutputStream commits the response. Either this method or getWriter() may be called to write the body, not both.
and
getWriter()
java.io.PrintWriter getWriter() throws java.io.IOException
Returns a PrintWriter object that can send character text to the client. The PrintWriter uses the character encoding returned by getCharacterEncoding(). If the response's character encoding has not been specified as described in getCharacterEncoding (i.e., the method just returns the default value ISO-8859-1), getWriter updates it to ISO-8859-1.
Calling flush() on the PrintWriter commits the response.
Either this method or getOutputStream() may be called to write the body, not both.
(emphasis mine)
The problem is in your particular case however much bigger. You're attempting to inline the PDF result of a Jasper Report between those HTML tags within a HTML response. I'm not sure what you thought or smoked while you wrote the code, but that is definitely not going to work. You need to rewrite the servlet that way so that it only returns the PDF and not that bunch of HTML noise. You should move all that HTML out the servlet into some JSP file. Then, you can call that servlet by a simple download link in the JSP
Download PDF
or inside an <iframe> (yes, in JSP)
<iframe src="yourServletUrl" style="width: 500px; height: 300px;"></iframe>
or in an <object> (also here, just in JSP)
<object data="yourServletUrl" type="application/pdf" width="500" height="300" />
Just put that HTML in a JSP page, open the JSP in browser and the webbrowser will take care that the servlet will be invoked and that the PDF will be represented the way you intended.
Your other problem is that the exception handling is not really good. You'll see completely nothing this way as the response buffer is not been resetted. You should instead be doing a
} catch (Exception e) {
throw new ServletException("descriptive message here", e);
}
as the container knows perfectly how to handle exceptions.
That both your doGet() and doPost() are doing exactly the same is by the way also a design smell. The JDBC driver which you used there is completely outdated and deprecated. The way how you registered the driver is clumsy. That the DB connection is not closed in finally is prone to resource leaking. Okay, I'll stop...

I presume that you are getting an IllegalStateException because you are calling getWriter() and getOutputStream() on the same response. Which you're not allowed to do.

Java Servlet DB Query with Ajax - slow query time and querystring not always fully passed to the servlet

I'm trying to create a AJAX based SQL query with Java EE and Servlets. I'm using Glassfish 3.01 and MS SQL server with Jquery on the client side.
I put everything together, and bind my ajax function to the textfield's onkeyup event. But sometimes When I put 'teststring' into the textbox only "teststrin" passed to the Servlet. So basically the last char disappears and therefore the query result is not correct.
Not to mention when the resultset contains large amount of data the query is pretty slow. Could you please check if I'm doing something wrong on the server and client side?
On the client side I have this JQuery function:
function ajaxSearch(sstring) {
if (sstring.length < 3)
{
$("#external").html("at least 3 chars please....")
}
else
{
$('#loading').ajaxStart(function() {
$(this).show()
$("#external").hide()
});
$('#loading').ajaxComplete(function() {
$(this).hide()
$("#external").show()
});
$.ajax({
type:"GET",
url: "/myApp/getStd",
dataType: "application/x-www-form-urlencoded",
data: "sstring="+escape(sstring),
async: true,
success: function(data){
$("#external").html(data);
}
})
}
}
On the server side I have this:
#WebServlet(name="getStd", urlPatterns={"/getStd"})
public class getStd extends HttpServlet {
#Override
public void doGet (HttpServletRequest req, HttpServletResponse res) throws ServletException, IOException {
Connection conn = null;
Statement stmt = null;
ResultSet rs = null;
ArrayList rows = new ArrayList();
res.setCharacterEncoding("UTF-8");
res.setContentType("text/html");
PrintWriter out = res.getWriter();
String sql=null;
String test= req.getParameter("sstring");
try{
InitialContext cxt = new InitialContext();
if (cxt == null) {
throw new Exception("Uh oh -- no context!");}
DataSource ds = (DataSource) cxt.lookup( "jdbc/Sample" );
conn = ds.getConnection();
stmt = conn.createStatement();
sql="Select * from MYDB.dbo.testdb where myField like '%"+req.getParameter("sstring")+"%';";
rs = stmt.executeQuery(sql);
while(rs.next()){
stdRecord cols = new stdRecord();
cols.setTeljel(rs.getString("Field1"));
cols.setTitle(rs.getString("Field2"));
cols.setICS(rs.getString("Field3"));
cols.setREF(rs.getString("Field4"));
rows.add(cols);
}
req.setAttribute("std", rows);
req.setAttribute("query",test );
req.getRequestDispatcher("/showRes.jsp").forward(req, res);
// close everything to free up resources
rs.close();
rs = null;
stmt.close();
stmt = null;
conn.close(); /
conn = null;
rows=null;
} catch (SQLException e) {
e.printStackTrace(out);
} catch (Exception e) {
e.printStackTrace(out);
} finally {
if (rs != null) {
try { rs.close(); } catch (SQLException e) { ; }
rs = null;
}
if (stmt != null) {
try { stmt.close(); } catch (SQLException e) { ; }
stmt = null;
}
if (conn != null) {
try { conn.close(); } catch (SQLException e) { ; }
conn = null;
}
}
}
}
Thanks in advance.

As to the lag in keyup, I think this is related to the performance issue, so let's fix that first and then review afterwards.
As to the performance, you've given very little information about your setup, but two common solutions which are often overlooked by starters are the following:
Use a connection pooled DataSource instead of DriverManager. This saves the cost of connecting the DB on every query (which can take over 200ms while a pooled connection is returned in no-time). Consult the JNDI resource config documentation of the application server in question for details (hint: admin console).
Limit the resultset size in SQL side instead of in Java side. This saves the cost of transferring irrelevant data over network. Just return the top 10 results or something instead of the entire table. Consult the SQL manual of the database in question for details (hint: SET ROWCOUNT).