We Keep Coding

java GZIPOutputStream blocks ChunkedOutput http response

I've written a basic REST Server using Jersey2 on top of Jetty, to test out HTTP Chunked Transfer-Encoding, and gzip Content-Encoding. However I've found that the recommended method of implementing a WriterInceptor to apply a GZIPOutputStream for gzip encoding results in the server blocking instead of sending through a gzip'd chunk.
I believe it is the GZIPOutputStream waiting for it's own buffer to fill up, so I tried overriding write() method in the WriterInterceptor to force a flush() after every write (as my server always writes one chunk at a time) but that made no difference. Is there a way of forcing the flush to occur whenever a write occurs?
App.java
public class App
{
public static int lineCount=0;
public static void main( String[] args ) {
System.out.println( "Hello World!" );
ResourceConfig config = new ResourceConfig();
config.packages("com.example.mockAPIjava");
ServletHolder servlet = new ServletHolder(new ServletContainer(config));
EncodingFilter.enableFor(config, GZipEncoder.class);
Server server = new Server(2222);
ServletContextHandler context = new ServletContextHandler(server, "/*");
context.addServlet(servlet, "/*");
try {
server.start();
server.join();
} catch (Exception e) {
throw new RuntimeException(e);
} finally {
server.destroy();
}
}
}
GZIPWriterInterceptor.java
#Provider
#Compress
public class GZIPWriterInterceptor implements WriterInterceptor {
#Override
public void aroundWriteTo(WriterInterceptorContext context)
throws IOException, WebApplicationException {
MultivaluedMap<String,Object> headers = context.getHeaders();
headers.add("Content-Encoding", "gzip");
final OutputStream outputStream = context.getOutputStream();
context.setOutputStream(new GZIPOutputStream(outputStream) {
#Override
public void write(final int b) throws IOException {
out.write(b);
out.flush();
}
#Override
public void write(final byte[] b) throws IOException {
out.write(b);
out.flush();
}
#Override
public void write(final byte[] b, final int off, final int len) throws IOException {
out.write(b, off, len);
out.flush();
}
});
context.proceed();
}
}
Resource.java
#Path("stream")
public class Resource {
#GET
#Path("test")
#Compress
#Produces(MediaType.APPLICATION_JSON)
public ChunkedOutput<String> helloWorld(#Context HttpHeaders header, #Context HttpServletResponse response) {
final ChunkedOutput<String> output = new ChunkedOutput<String>(String.class, "\r\n");
new Thread() {
public void run() {
BufferedReader br = null;
try {
String chunk;
// open file for reading
File file = new File("/tmp/stream.txt");
FileReader fr = new FileReader(file);
br = new BufferedReader(fr);
while ((chunk = getNextString(br)) != null) {
// write a chunk every second
output.write(chunk);
try {
Thread.sleep(1 * 1000);
} catch(InterruptedException ex) {
Thread.currentThread().interrupt();
}
}
} catch (Exception e) {
// IOException thrown when writing the
// chunks of response: should be handled
e.printStackTrace();
} finally {
try {
output.close();
// simplified: IOException thrown from
// this close() should be handled here...
if (br!=null) { br.close(); }
} catch (IOException e1){
e1.printStackTrace();
}
}
}
}.start();
// the output will be probably returned even before
// a first chunk is written by the new thread
return output;
}
private String getNextString(BufferedReader br) throws IOException, ParseException {
App.lineCount++;
return br.readLine();;
}
}
Compress.java
//#Compress annotation is the name binding annotation for the GZIPWriterInterceptor
#NameBinding
#Retention(RetentionPolicy.RUNTIME)
public #interface Compress {}

By overriding the write methods of GZIPOutputStream, you have just stopped it from gzipping!
public void write(final int b) throws IOException {
out.write(b);
out.flush();
}
Because you've overridden it to not invoke super.write (which you should have done), but rather out.write, you're sending directly to the context OutputStream, uncompressed.
Presumably, the receiving side is expecting gzip data and not receiving it, which may lead to all kinds of wrong behaviour.
Change the code to invoke super.write and flush:
public void write(final int b) throws IOException {
super.write(b);
flush();
}
etc.

Read http request without losing data in it

Is there any way to read inputStream of a request without losing the data in it.
I am trying to take a raw copy of my request into string before processing it. But once I read the inputstream from request, the inputstream is changing to null so, I can't get Parameter from my request later. I tried using CustomHttpServletRequestWrapper but it did not work. Below is the snippet of my code.
The InboundHandler is my handler class which has the processRequest(HttpServletRequest request, HttpServletResponse response) method that is invoked from my servlet class.
public class InboundHandler {
protected void processRequest(HttpServletRequest request, HttpServletResponse response)
throws ServletException, IOException {
try {
CustomHttpServletRequestWrapper requestWrapper = new CustomHttpServletRequestWrapper(request);
String body = getRequestBody(requestWrapper.getInputStream());
String from = request.getParameter("from"); //which I'm getting null here
// I also tried using
// String from = requestWrapper.getParameter("from"); // Even this did not work
} catch (Exception e) {
e.printStackTrace();
}
}
private String getRequestBody(InputStream inputStream) {
StringBuilder stringBuilder = new StringBuilder();
BufferedReader bufferedReader = null;
try {
if (inputStream != null) {
bufferedReader = new BufferedReader(new InputStreamReader(inputStream));
char[] charBuffer = new char[128];
int bytesRead = -1;
while ((bytesRead = bufferedReader.read(charBuffer)) != -1) {
stringBuilder.append(charBuffer, 0, bytesRead);
}
}
} catch (IOException ex) {
ex.printStackTrace();
//throw new AuthenticationException("Error reading the request payload", ex);
} finally {
if (bufferedReader != null) {
try {
bufferedReader.close();
} catch (IOException iox) {
// ignore
}
}
}
return stringBuilder.toString();
}
public class CustomHttpServletRequestWrapper extends HttpServletRequestWrapper {
private final String body;
public CustomHttpServletRequestWrapper(HttpServletRequest request) {
super(request);
StringBuilder stringBuilder = new StringBuilder();
BufferedReader bufferedReader = null;
try {
InputStream inputStream = request.getInputStream();
if (inputStream != null) {
bufferedReader = new BufferedReader(new InputStreamReader(inputStream));
char[] charBuffer = new char[128];
int bytesRead = -1;
while ((bytesRead = bufferedReader.read(charBuffer)) > 0) {
stringBuilder.append(charBuffer, 0, bytesRead);
}
} else {
stringBuilder.append("");
}
} catch (IOException ex) {
ex.printStackTrace();
//log.error("Error reading the request body...");
} finally {
if (bufferedReader != null) {
try {
bufferedReader.close();
} catch (IOException ex) {
ex.printStackTrace();
//log("Error closing bufferedReader...");
}
}
}
body = stringBuilder.toString();
}
#Override
public ServletInputStream getInputStream() throws IOException {
final ByteArrayInputStream byteArrayInputStream = new ByteArrayInputStream(body.getBytes());
ServletInputStream inputStream = new ServletInputStream() {
public int read() throws IOException {
return byteArrayInputStream.read();
}
};
return inputStream;
}
}
}

Q: Is there any way to read inputStream of a request without losing the data in it?
A: Yes - simply "save" what you "read"!
As Perdomoff correctly said, "Requests aren't meant to be reused". You can't put the water back in the hose once you've watered your garden :)
You can save the data. Perdomoff suggested using a session variable.
Another alternative might be to use a Servlet Filter. Here is a good tutorial:
http://www.journaldev.com/1933/java-servlet-filter-example-tutorial

Filter causing POST Request Body to become empty

I am using a HMAC Authentication filter. In the filter when I access my POST Request Body, I am able to get the XML in it. When I try to access the XML in the controller I am get a blank string. The xmlString in the filter is giving the proper XML but the xmlString in the controller is giving blank string. I am using Spring MVC.
My filter is:
public void doFilter(ServletRequest request, ServletResponse response, FilterChain chain) throws IOException, ServletException {
HttpServletRequest httpRequest = (HttpServletRequest) request;
HttpServletResponse httpResponse = (HttpServletResponse) response;
String authorisation = httpRequest.getHeader("authorization");
String accessKeyId = authorisation.split(":")[0].split(" ")[1];
String signature = authorisation.split(":")[1];
String secretAccessKey = getSecretAccessKey(accessKeyId);
InputStream xmlStream = httpRequest.getInputStream();
String xmlString = IOUtils.toString(xmlStream, "UTF-8");
String encodedXml = new String();
try {
encodedXml = HMACEncoder.calculateHMAC(xmlString, secretAccessKey);
} catch (SignatureException e) {
e.printStackTrace();
}
if (!signature.equals(encodedXml))
httpResponse.sendError(HttpServletResponse.SC_UNAUTHORIZED);
else
chain.doFilter(request, response);
}
and my controller is:
#RequestMapping(value = "/user", method = RequestMethod.POST)
public String fetchUserString(HttpServletRequest request) {
InputStream xml = null;
String xmlString = null;
try {
xml = request.getInputStream();
xmlString = IOUtils.toString(xml, "UTF-8");
} catch (IOException e) {
e.printStackTrace();
}
return xmlString;
}

As praki suggested, I put the XML in a request attribute and in my controller I got it from the request attribute and then put the XML in a ModelAttribute. Here is my code:
My filter:
#Override
public void doFilter(ServletRequest request, ServletResponse response, FilterChain chain) throws IOException, ServletException {
HttpServletRequest httpRequest = (HttpServletRequest) request;
HttpServletResponse httpResponse = (HttpServletResponse) response;
if (httpRequest.getMethod().equalsIgnoreCase("POST")) {
String authorisation = httpRequest.getHeader("authorization");
String accessKeyId = authorisation.split(":")[0].split(" ")[1];
String signature = authorisation.split(":")[1];
String secretAccessKey = getSecretAccessKey(accessKeyId);
ServletInputStream servletStream = httpRequest.getInputStream();
String xmlString = IOUtils.toString(servletStream);
String encodedXml = new String();
try {
encodedXml = HMACEncoder.calculateHMAC(xmlString, secretAccessKey);
} catch (SignatureException e) {
e.printStackTrace();
}
httpRequest.setAttribute("content", xmlString);
if (!signature.equals(encodedXml))
httpResponse.sendError(HttpServletResponse.SC_UNAUTHORIZED);
else
chain.doFilter(request, response);
} else
chain.doFilter(request, response);
}
and my controller:
#RequestMapping(value = "/add", method = RequestMethod.POST, produces = "application/xml")
public #ResponseBody Catalog addCatalog(#ModelAttribute("catalog") Catalog catalog) {
logger.info("In addCatalog with " + catalog);
catalog.getHeader().setOrganizationId(IConstant.ORGANIZATION_ID_BSL);
try {
catalog = catalogService.addCatalogIntegration(catalog);
} catch (Exception e) {
e.printStackTrace();
}
logger.info("Returning from addCatalog with " + catalog);
return catalog;
}
#ModelAttribute("catalog")
private Catalog getCatalog(HttpServletRequest request) throws Exception {
Catalog catalog = new Catalog();
try {
String xml = (String) request.getAttribute("content");
if (xml != null) {
JAXBContext jaxbContext = JAXBContext.newInstance(Catalog.class);
Unmarshaller unmarshaller = jaxbContext.createUnmarshaller();
catalog = (Catalog) unmarshaller.unmarshal(new StringReader(xml));
}
} catch (Exception e) {
e.printStackTrace();
}
return catalog;
}

You can't read the stream twice, comment the code in filter and read only in controller.

Spring: Logging outgoing HTTP requests

I am trying to log all the outgoing Http requests in my spring based web application. Is there is interceptor for this purpose? I want to log all outgoing the contents and headers before it leaves the application. I am using spring-ws to send SOAP requests. So basically, I want to log not only the SOAP request xml (as mentioned here How can I make Spring WebServices log all SOAP requests?) but the http request as a whole.

Intercept the request/response using a ClientInterceptor on the WebServiceGatewaySupport:
// soapClient extends WebServiceGatewaySupport
soapClient.setInterceptors(new ClientInterceptor[]{new ClientInterceptor() {
#Override
public boolean handleRequest(MessageContext messageContext) throws WebServiceClientException {
ByteArrayOutputStream os = new ByteArrayOutputStream();
try {
messageContext.getRequest().writeTo(os);
} catch (IOException e) {
throw new WebServiceIOException(e.getMessage(), e);
}
String request = new String(os.toByteArray());
logger.trace("Request Envelope: " + request);
return true;
}
#Override
public boolean handleResponse(MessageContext messageContext) throws WebServiceClientException {
ByteArrayOutputStream os = new ByteArrayOutputStream();
try {
messageContext.getResponse().writeTo(os);
} catch (IOException e) {
throw new WebServiceIOException(e.getMessage(), e);
}
String response = new String(os.toByteArray());
logger.trace("Response Envelope: " + response);
return true;
}
...
To get the headers as well you need an instance of TransportOutputStream.
Unfortunately the class is abstract, so you need to subclass is. Here's how it might look:
class ByteArrayTransportOutputStream extends TransportOutputStream {
private ByteArrayOutputStream outputStream;
#Override
public void addHeader(String name, String value) throws IOException {
createOutputStream();
String header = name + ": " + value + "\n";
outputStream.write(header.getBytes());
}
public byte[] toByteArray() {
return outputStream.toByteArray();
}
#Override
protected OutputStream createOutputStream() throws IOException {
if (outputStream == null) {
outputStream = new ByteArrayOutputStream();
}
return outputStream;
}
}

Java- Servlet - redirecting request with a new body

I'm writing a Servlet in Java, that basically, gets a request with a XML in the Requests body, and then changes a few things in the XML and redirect/foreword the request with The new XML to a different Servlet that's on the same server, but its on a different web app.
I'm using doPost.
How do i do that? can i find code example any where?
Plus, whats the correct method to use:?
request.getRequestDispatcher().include /request.getRequestDispatcher().foreword / response.sendRedirect() or do i need to use the: HttpServletRequestWrapper?
this is what i have so far:
protected void doPost(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
String body = getBody(request);
MapXml mapXml = new MapXml(body, "C:\\Projects\\XmlMapper\\output.xml","C:\\Projects\\XmlMapper\\output\\");
String outputXml = mapXml.getOutputXml();
}
public static String getBody(HttpServletRequest request) throws IOException {
String body = null;
StringBuilder stringBuilder = new StringBuilder();
BufferedReader bufferedReader = null;
try {
InputStream inputStream = request.getInputStream();
if (inputStream != null) {
bufferedReader = new BufferedReader(new InputStreamReader(inputStream));
char[] charBuffer = new char[128];
int bytesRead = -1;
while ((bytesRead = bufferedReader.read(charBuffer)) > 0) {
stringBuilder.append(charBuffer, 0, bytesRead);
}
} else {
stringBuilder.append("");
}
} catch (IOException ex) {
throw ex;
} finally {
if (bufferedReader != null) {
try {
bufferedReader.close();
} catch (IOException ex) {
throw ex;
}
}
}
body = stringBuilder.toString();
return body;
}
And i have no idea how to continue on from here. I'm new to the servlet world..
Thanks!!!
Cheers:)