We Keep Coding

How to decode repeatly incoming byte messages after a HTTP handler with Netty?

I am using the feature of Icecast to add a repeatedly message containing the current metadata of the stream. It is enabled with following header:
request.headers().set("Icy-MetaData", "1");
After this, in the HTTP response there is a header containing the interval, which defines that the server sends metadata again after x bytes of the normal stream content. In addition, I want to store the stream content between the metadata tags.
My question is now, how is the best way to achieve this? One thaught was to use the ByteToMessageDecoder to "filter" the metadata tags, but the bytes were already decoded to an HttpObject. My current handler looks like this:
public class StreamClientHandler extends SimpleChannelInboundHandler<HttpObject> {
private int metadataInterval = 0;
#Override
protected void messageReceived(ChannelHandlerContext handlerContext, HttpObject message) throws Exception {
if(message instanceof HttpResponse) {
HttpResponse response = (HttpResponse) message;
this.metadataInterval = Integer.parseInt(response.headers().get("icy-metaint").toString());
}
if(message instanceof HttpContent) {
HttpContent content = (HttpContent) message;
if(content instanceof LastHttpContent) {
// Close connection
handlerContext.channel().close();
}
}
}
}
Thank you!

Limit output payload response in CXF JAX-RS based service

I have multiple jax-rs services built using cxf/spring. I want to control the output payload response size of all services. For simplicity sake, let's say none of api's in any of the services should ever return a JSON response payload more than 500 characters and I want to control this in one place instead of relying on individual services to adhere to this requirement. (We already have other features built into the custom framework/base component that all services depend on).
I have tried implementing this using JAX-RS's WriterInterceptor, ContainerResponseFilter and CXF's Phase Interceptor, but none of the approaches seem to be completely satisfy my requirement. More details on what I've done so far:
Option 1: (WriterInteceptor) In the overridden method, I get the ouputstream and set the max size of the cache to 500. When I invoke an api that returns more than 500 characters in the response payload, I get an HTTP 400 Bad Request status, but the response body contains the entire JSON payload.
#Provider
public class ResponsePayloadInterceptor implements WriterInterceptor {
private static final Logger LOGGER = LoggerFactory.getLogger(ResponsePayloadInterceptor.class);
#Override
public void aroundWriteTo(WriterInterceptorContext context) throws IOException, WebApplicationException {
final OutputStream outputStream = context.getOutputStream();
CacheAndWriteOutputStream cacheAndWriteOutputStream = new CacheAndWriteOutputStream(outputStream);
cacheAndWriteOutputStream.setMaxSize(500);
context.setOutputStream(cacheAndWriteOutputStream);
context.proceed();
}
}
Option 2a: (CXF Phase Inteceptor) In the overridden method, I get the response as String from the ouputstream and check it's size. If it's greater than 500, I create a new Response object with only the data Too much data and set it in the message. Even if the response is > 500 characters, I get an HTTP 200 OK status with the entire JSON. Only when I use the phase as POST_MARSHAL or a later phase, I'm able to get hold of the JSON response and check it's length, but by that time the response has already been streamed to the client.
#Provider
public class ResponsePayloadInterceptor extends AbstractPhaseInterceptor<Message> {
private static final Logger LOGGER = LoggerFactory.getLogger(ResponsePayloadInterceptor.class);
public ResponsePayloadInterceptor() {
super(Phase.POST_MARSHAL);
}
#Override
public void handleMessage(Message message) throws Fault {
LOGGER.info("handleMessage() - Response intercepted");
try {
OutputStream outputStream = message.getContent(OutputStream.class);
...
CachedOutputStream cachedOutputStream = (CachedOutputStream) outputStream;
String responseBody = IOUtils.toString(cachedOutputStream.getInputStream(), "UTF-8");
...
LOGGER.info("handleMessage() - Response: {}", responseBody);
LOGGER.info("handleMessage() - Response Length: {}", responseBody.length());
if (responseBody.length() > 500) {
Response response = Response.status(Response.Status.BAD_REQUEST)
.entity("Too much data").build();
message.getExchange().put(Response.class, response);
}
} catch (IOException e) {
LOGGER.error("handleMessage() - Error");
e.printStackTrace();
}
}
}
Option 2b: (CXF Phase Inteceptor) Same as above, but only the contents of if block is changed. If response length is greater than 500, I create a new output stream with the string Too much data and set it in message. But if the response payload is > 500 characters, I still get an HTTP 200 OK status with an invalid JSON response (entire JSON + additional text) i.e., the response looks like this: [{"data":"", ...}, {...}]Too much data (the text 'Too much data' is appended to the JSON)
if (responseBody.length() > 500) {
InputStream inputStream = new ByteArrayInputStream("Too much data".getBytes("UTF-8"));
outputStream.flush();
IOUtils.copy(inputStream, outputStream);
OutputStream out = new CachedOutputStream();
out.write("Too much data".getBytes("UTF-8"));
message.setContent(OutputStream.class, out);
}
Option 3: (ContainerResponseFilter) Using the ContainerResponseFilter, I added a Content-Length response header with value as 500. If response length is > 500, I get an HTTP 200 OK status with an invalid JSON response (truncated to 500 characters). If the response length is < 500, still get an HTTP 200 OK status, but the client waits for more data to be returned by the server (as expected) and times out, which isn't a desirable solution.
#Provider
public class ResponsePayloadFilter implements ContainerResponseFilter {
private static final Logger LOGGER = LoggerFactory.getLogger(ResponsePayloadFilter.class);
#Override
public void filter(ContainerRequestContext requestContext, ContainerResponseContext responseContext) throws IOException {
LOGGER.info("filter() - Response intercepted");
CachedOutputStream cos = (CachedOutputStream) responseContext.getEntityStream();
StringBuilder responsePayload = new StringBuilder();
ByteArrayOutputStream out = new ByteArrayOutputStream();
if (cos.getInputStream().available() > 0) {
IOUtils.copy(cos.getInputStream(), out);
byte[] responseEntity = out.toByteArray();
responsePayload.append(new String(responseEntity));
}
LOGGER.info("filter() - Content: {}", responsePayload.toString());
responseContext.getHeaders().add("Content-Length", "500");
}
}
Any suggestions on how I can tweak the above approaches to get what I want or any other different pointers?

I resolved this partially using help from this answer. I say partially because I'm successfully able to control the payload, but the not the response status code. Ideally, if the response length is greater than 500 and I modify the message content, I would like to send a different response status code (other than 200 OK). But this is a good enough solution for me to proceed at this point. If I figure out how to update the status code as well, I'll come back and update this answer.
import org.apache.commons.io.IOUtils;
import org.apache.cxf.interceptor.Fault;
import org.apache.cxf.io.CachedOutputStream;
import org.apache.cxf.message.Message;
import org.apache.cxf.phase.AbstractPhaseInterceptor;
import org.apache.cxf.phase.Phase;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
public class ResponsePayloadInterceptor extends AbstractPhaseInterceptor<Message> {
private static final Logger LOGGER = LoggerFactory.getLogger(ResponsePayloadInterceptor.class);
public ResponsePayloadInterceptor() {
super(Phase.PRE_STREAM);
}
#Override
public void handleMessage(Message message) throws Fault {
LOGGER.info("handleMessage() - Response intercepted");
try {
OutputStream outputStream = message.getContent(OutputStream.class);
CachedOutputStream cachedOutputStream = new CachedOutputStream();
message.setContent(OutputStream.class, cachedOutputStream);
message.getInterceptorChain().doIntercept(message);
cachedOutputStream.flush();
cachedOutputStream.close();
CachedOutputStream newCachedOutputStream = (CachedOutputStream) message.getContent(OutputStream.class);
String currentResponse = IOUtils.toString(newCachedOutputStream.getInputStream(), "UTF-8");
newCachedOutputStream.flush();
newCachedOutputStream.close();
if (currentResponse != null) {
LOGGER.info("handleMessage() - Response: {}", currentResponse);
LOGGER.info("handleMessage() - Response Length: {}", currentResponse.length());
if (currentResponse.length() > 500) {
InputStream replaceInputStream = IOUtils.toInputStream("{\"message\":\"Too much data\"}", "UTF-8");
IOUtils.copy(replaceInputStream, outputStream);
replaceInputStream.close();
message.setContent(OutputStream.class, outputStream);
outputStream.flush();
outputStream.close();
} else {
InputStream replaceInputStream = IOUtils.toInputStream(currentResponse, "UTF-8");
IOUtils.copy(replaceInputStream, outputStream);
replaceInputStream.close();
message.setContent(OutputStream.class, outputStream);
outputStream.flush();
outputStream.close();
}
}
} catch (IOException e) {
LOGGER.error("handleMessage() - Error", e);
throw new RuntimeException(e);
}
}

Codename One - How to get a response after file upload with multipart request

I'm using Codename One to upload videos to Vimeo using their respective API's. I'm using a multipart request to actually upload the file, but a response is required to find the upload status. How can I get the response?
public void doFileUpload(String url, String filename) throws IOException {
MultipartRequest req = new MultipartRequest() {
int chr;
StringBuffer sb = new StringBuffer();
public String response = "";
public void readResponse(InputStream input) throws IOException {
response = Util.readToString(input);
Log.p("File Response->" + response);
}
protected void handleErrorResponseCode(int code, String message)
{
Log.p("Error Response->" + message);
}
protected void readHeaders(Object connection) throws IOException {
String val = getHeader(connection, "MyHeaderName");
Log.p("Header Response->" + val);
}
protected void handleException(Exception err) {
Dialog.show(
"Connection Err!!",
"Are you connected to the internet? Check your connection",
"Ok", null);
}
};
req.setUrl(url);
req.setPost(true);
String mime = "application/mp4";
req.addData("file_data", filename, mime);
req.setFilename("file_data", filename);
req.setReadResponseForErrors(true);
req.addResponseCodeListener(new ActionListener() {
public void actionPerformed(ActionEvent ev) {
try {
NetworkEvent event = (NetworkEvent) ev;
Log.p("Err Rsp Code->" + event.getResponseCode());
Log.p("ResponseCodeListener:");
Log.p(ev.toString() );

I suggest you use the addData method that accepts a file URL rather than a byte array since you might exceed device memory with the byte array version (e.g. for a larger video or low memory device).
You can read the response just like any other connection request:
Derive the connection request and override: protected void readResponse(InputStream input)
Use addResponseListener to bind a listener to the users response.
Use addToQueueAndWait to detect that the upload finished then just invoke getResponseData() to get the byte array data.

I had the same problem! on the back-end, I am using Spring boot, my upload controller was simply tagged as #Controller when changed to #RestController, it worked fine and I got readResponse(InputStream input) callback, where I handle the response as JSON.

I can't get cross domain ajax working with Weblogic/Spring 3

I am having trouble trying to get cross-domain ajax request working and despite the many solutions I have found on Stack Overflow I am unable to get it working.
$.ajax({
url : 'http://SERVER:PORT/CONTEXT/RESOURCE.html?someParameter=1234',
dataType : 'json',
success: function(xhr) {
alert('ok '+JSON.stringify(xhr));
},
error : function(xhr) {
alert('error '+JSON.stringify(xhr));
}
});
Doing just a standard $.ajax call with datatype "json" the server responds with a blank response and statusText "error", like so:
error {"readyState":0,"responseText":"","status":0,"statusText":"error"}
So I tried simply changing datatype to "jsonp" as suggested in other threads but this time it still goes to error condition with the following response:
error {"readyState":4,"status":200,"statusText":"success"}
and an error message of "parsererror"
Yet no data.
What gives?
Do I need to do something special on the server side because it is Spring MVC in Weblogic?
EDIT:
jQuery version 1.9.1
Spring-3 MVC
EDIT2: Oh yes I also tried $.getJSON but this command seems to do nothing - when I run the code replacing $.ajax with $.getJSON nothing happens. No response and I dont see any error occurring in console and no Network request seen going to the URL. I did also change the syntax in a 2nd try where I called it like $.getJSON(url, callback); but that did not change anything
EDIT3: I should also mention when I run the original code using "json" datatype and look in Firebug's Response tab, it is empty. But when I run the second code using "jsonp" I do see the JSON text in the Response tab. So it is strange why it still throws an error.

OK, upon more research I finally found the cause - yes I did need to do something on the Server side to support jsonp. I ended up writing a servlet filter which wraps the returning json string in the appropriate callback.
Learn something new everyday!
public class JsonPCallbackFilter extends OncePerRequestFilter {
Logger logger = Logger.getLogger(JsonPCallbackFilter.class);
#Override
protected void doFilterInternal(HttpServletRequest request, HttpServletResponse response, FilterChain chain)
throws ServletException, IOException {
//logger.debug("Filter: "+request.getRequestURI());
#SuppressWarnings("unchecked")
Map<String, String[]> parms = request.getParameterMap();
if(parms.containsKey("callback")) {
logger.debug("Wrapping response with JSONP callback '" + parms.get("callback")[0] + "'");
OutputStream out = response.getOutputStream();
ByteResponseWrapper wrapper = new ByteResponseWrapper(response);
chain.doFilter(request, wrapper);
StringBuffer sb = new StringBuffer();
sb.append(parms.get("callback")[0] + "(");
sb.append(new String(wrapper.getBytes()));
sb.append(new String(");"));
out.write(sb.toString().getBytes());
wrapper.setContentType("text/javascript;charset=UTF-8");
response.setContentLength(sb.length());
out.close();
} else {
chain.doFilter(request, response);
}
}
}
static class ByteOutputStream extends ServletOutputStream {
private ByteArrayOutputStream bos = new ByteArrayOutputStream();
#Override
public void write(int b) throws IOException {
bos.write(b);
}
public byte[] getBytes() {
return bos.toByteArray();
}
}
static class ByteResponseWrapper extends HttpServletResponseWrapper {
private PrintWriter writer;
private ByteOutputStream output;
public byte[] getBytes() {
writer.flush();
return output.getBytes();
}
public ByteResponseWrapper(HttpServletResponse response) {
super(response);
output = new ByteOutputStream();
writer = new PrintWriter(output);
}
#Override
public PrintWriter getWriter() {
return writer;
}
#Override
public ServletOutputStream getOutputStream() throws IOException {
return output;
}
}
<filter>
<filter-name>jsonpFilter</filter-name>
<filter-class>com.blahblah.JsonPCallbackFilter</filter-class>
</filter>
<filter-mapping>
<filter-name>jsonpFilter</filter-name>
<url-pattern>/*</url-pattern>
</filter-mapping>

dealing with Streams and ContentProducer - Java

I'm pretty new to coding with streams but now I have to do it for more efficient Http coding.
Here is code that I wrote(not working) to get ContentProducer for HttpClient:
public static ContentProducer getContentProducer(final Context context, final UUID gId)
{
return new ContentProducer()
{
public void writeTo(OutputStream outputStream) throws IOException
{
outputStream = new Base64.OutputStream(new FileOutputStream(StorageManager.getFileFromName(context, gId.toString())));
outputStream.flush();
}
};
}
I'm using Base64 streaming encoder from here: http://iharder.sourceforge.net/current/java/base64/
My goal is to use this function to provide data that I read from binary file to HttpClient as base64 encoded stream.
This is how I consume content producers:
private MyHttpResponse processPOST(String url, ContentProducer requestData)
{
MyHttpResponse response = new MyHttpResponse();
try
{
HttpPost request = new HttpPost(serviceURL + url);
HttpEntity entity = new EntityTemplate(requestData);
request.setEntity(entity);
ResponseHandler<String> handler = new BasicResponseHandler();
response.Body = mHttpClient.execute(request, handler);
}
catch (HttpResponseException e)
{
}
catch (Throwable e)
{
}
return response;
}
I have another ContentProducer which works with GSON streamer(and it's working):
public ContentProducer getContentProducer(final Context context)
{
return new ContentProducer()
{
public void writeTo(OutputStream outputStream) throws IOException
{
Gson myGson = MyGsonWrapper.getMyGson();
JsonWriter writer = new JsonWriter(new OutputStreamWriter(outputStream, "UTF-8"));
writer.beginObject();
// stuff
writer.endObject();
writer.flush();
}
};
}
My question is: How to make my first example work. Am I doing it correctly? Right now I get empty post on server side, so it seems like no data coming through.

EDIT:
I believe that the issue is that you are being passed an OutputStream in your ContentProviders writeTo() method, and you are overwriting it with your own OutputStream. The contract of that class/method probably requires you to write your data to the OutputStream passed to you.
Based on looking at the Android Documentation, I do not see a way for you to specify the OutputStream to use, so you will probably need to just write out the data of the file to the OutputStream that is passed in.
Instead, you should do something like this:
public void writeTo(OutputStream outputStream) throws IOException
{
byte[] buf = createByteBufferFromFile(StorageManager.getFileFromName(context, gId.toString()));
outputStream.write(buf);
outputStream.flush();
}
Of course you will need to provide an implementation to the createByteBufferFromFile(...) method that I mention. Again, you should note that it is not likely that you will be using the Base64 OutputStream, so if that is a necessity, then you may have to find a different approach.