We Keep Coding

Log response body in case of exception

I am using retrofit for http calls with gson as a converter.
In some cases I get exceptions thrown when gson tries to convert response to object and I would like to know what is the actual response in such case.
For example:
This is the exception message I get:
Expected a string but was BEGIN_OBJECT at line 1 column 26 path $[0].date
The code that execute the call is like this:
Gson gson = gsonBuilder.create();
Retrofit retrofit = (new retrofit2.Retrofit.Builder()).baseUrl(baseUrl).addConverterFactory(GsonConverterFactory.create(gson)).client(httpClient).build();
MyService service = retrofit.create(clazz);
...
Response<T> response = service.call().execute();
When this code throws exception I would like to log the raw response body somehow. How can I do that?

I don't think it can be accomplished easily. Retrofit does not seem to provide an easy way of tracking input streams (the most natural place I was thinking of was CallAdapter.Factory but it does not allow invalid response tracking).
Basically, illegal response conversion should be detected in a particular converter whose only responsibility is logging invalid payloads. Sounds pretty much like the Decorator design pattern. Since Java (unlike Kotlin?) does not support decorators as a first-class citizen, forwarding implementations can be implemented similarly to Google Guava Forwarding*** classes:
ForwardingInputStream.java
#SuppressWarnings("resource")
abstract class ForwardingInputStream
extends InputStream {
protected abstract InputStream inputStream();
// #formatter:off
#Override public int read() throws IOException { return inputStream().read(); }
// #formatter:on
// #formatter:off
#Override public int read(final byte[] b) throws IOException { return inputStream().read(b); }
#Override public int read(final byte[] b, final int off, final int len) throws IOException { return inputStream().read(b, off, len); }
#Override public long skip(final long n) throws IOException { return inputStream().skip(n); }
#Override public int available() throws IOException { return inputStream().available(); }
#Override public void close() throws IOException { inputStream().close(); }
#Override public void mark(final int readlimit) { inputStream().mark(readlimit); }
#Override public void reset() throws IOException { inputStream().reset(); }
#Override public boolean markSupported() { return inputStream().markSupported(); }
// #formatter:on
}
ForwardingResponseBody.java
#SuppressWarnings("resource")
abstract class ForwardingResponseBody
extends ResponseBody {
protected abstract ResponseBody responseBody();
// #formatter:off
#Override public MediaType contentType() { return responseBody().contentType(); }
#Override public long contentLength() { return responseBody().contentLength(); }
#Override public BufferedSource source() { return responseBody().source(); }
// #formatter:on
// #formatter:off
#Override public void close() { super.close(); }
// #formatter:on
}
ForwardingBufferedSource.java
abstract class ForwardingBufferedSource
implements BufferedSource {
protected abstract BufferedSource bufferedSource();
// #formatter:off
#Override public Buffer buffer() { return bufferedSource().buffer(); }
#Override public boolean exhausted() throws IOException { return bufferedSource().exhausted(); }
#Override public void require(final long byteCount) throws IOException { bufferedSource().require(byteCount); }
#Override public boolean request(final long byteCount) throws IOException { return bufferedSource().request(byteCount); }
#Override public byte readByte() throws IOException { return bufferedSource().readByte(); }
#Override public short readShort() throws IOException { return bufferedSource().readShort(); }
#Override public short readShortLe() throws IOException { return bufferedSource().readShortLe(); }
#Override public int readInt() throws IOException { return bufferedSource().readInt(); }
#Override public int readIntLe() throws IOException { return bufferedSource().readIntLe(); }
#Override public long readLong() throws IOException { return bufferedSource().readLong(); }
#Override public long readLongLe() throws IOException { return bufferedSource().readLongLe(); }
#Override public long readDecimalLong() throws IOException { return bufferedSource().readDecimalLong(); }
#Override public long readHexadecimalUnsignedLong() throws IOException { return bufferedSource().readHexadecimalUnsignedLong(); }
#Override public void skip(final long byteCount) throws IOException { bufferedSource().skip(byteCount); }
#Override public ByteString readByteString() throws IOException { return bufferedSource().readByteString(); }
#Override public ByteString readByteString(final long byteCount) throws IOException { return bufferedSource().readByteString(byteCount); }
#Override public int select(final Options options) throws IOException { return bufferedSource().select(options); }
#Override public byte[] readByteArray() throws IOException { return bufferedSource().readByteArray(); }
#Override public byte[] readByteArray(final long byteCount) throws IOException { return bufferedSource().readByteArray(byteCount); }
#Override public int read(final byte[] sink) throws IOException { return bufferedSource().read(sink); }
#Override public void readFully(final byte[] sink) throws IOException { bufferedSource().readFully(sink); }
#Override public int read(final byte[] sink, final int offset, final int byteCount) throws IOException { return bufferedSource().read(sink, offset, byteCount); }
#Override public void readFully(final Buffer sink, final long byteCount) throws IOException { bufferedSource().readFully(sink, byteCount); }
#Override public long readAll(final Sink sink) throws IOException { return bufferedSource().readAll(sink); }
#Override public String readUtf8() throws IOException { return bufferedSource().readUtf8(); }
#Override public String readUtf8(final long byteCount) throws IOException { return bufferedSource().readUtf8(byteCount); }
#Override public String readUtf8Line() throws IOException { return bufferedSource().readUtf8Line(); }
#Override public String readUtf8LineStrict() throws IOException { return bufferedSource().readUtf8LineStrict(); }
#Override public int readUtf8CodePoint() throws IOException { return bufferedSource().readUtf8CodePoint(); }
#Override public String readString(final Charset charset) throws IOException { return bufferedSource().readString(charset); }
#Override public String readString(final long byteCount, final Charset charset) throws IOException { return bufferedSource().readString(byteCount, charset); }
#Override public long indexOf(final byte b) throws IOException { return bufferedSource().indexOf(b); }
#Override public long indexOf(final byte b, final long fromIndex) throws IOException { return bufferedSource().indexOf(b, fromIndex); }
#Override public long indexOf(final ByteString bytes) throws IOException { return bufferedSource().indexOf(bytes); }
#Override public long indexOf(final ByteString bytes, final long fromIndex) throws IOException { return bufferedSource().indexOf(bytes, fromIndex); }
#Override public long indexOfElement(final ByteString targetBytes) throws IOException { return bufferedSource().indexOfElement(targetBytes); }
#Override public long indexOfElement(final ByteString targetBytes, final long fromIndex) throws IOException { return bufferedSource().indexOfElement(targetBytes, fromIndex); }
#Override public InputStream inputStream() { return bufferedSource().inputStream(); }
#Override public long read(final Buffer sink, final long byteCount) throws IOException { return bufferedSource().read(sink, byteCount); }
#Override public Timeout timeout() { return bufferedSource().timeout(); }
#Override public void close() throws IOException { bufferedSource().close(); }
// #formatter:on
}
Trivial forwarding implementations just override all methods of their parent classes and delegate the job to a delegated object. Once a forwarding class is extended, some of the parent methods can be overridden again.
IConversionThrowableConsumer.java
This is just a listener used below.
interface IConversionThrowableConsumer {
/**
* Instantiating {#link okhttp3.ResponseBody} can be not easy due to the way of how {#link okio.BufferedSource} is designed -- too heavy.
* Deconstructing its components to "atoms" with some lack of functionality may be acceptable.
* However, this consumer may need some improvements on demand.
*/
void accept(MediaType contentType, long contentLength, InputStream inputStream, Throwable ex)
throws IOException;
}
ErrorReportingConverterFactory.java
The next step is implementating the error-reporting converter factory that can be injected to Retrofit.Builder and listen to any errors occurring in downstream converters. Note how it works:
For every response converter an intermediate converter is injected. It allows to listen to any error in downstream converters.
Downstream converters obtain a non-closeable resources in order not to close underlaying I/O resources prematurely...
Downstream converters convert whilst the intermediate converter collects the real input stream content into a buffer in order to respond with an input stream that may cause GsonConverter fail. This should be considered a bottleneck here due to possibly large size of the grown buffer (however, it may be limited), its internal array is copied when requested from the converter and so on.
If IOException or RuntimeException occur, the intermediate converter concatenates the buffered input stream content and the real input stream in order to let a consumer to accept input streams from the very beginning.
The intermediate converter takes care of closing resources itself.
final class ErrorReportingConverterFactory
extends Factory {
private final IConversionThrowableConsumer consumer;
private ErrorReportingConverterFactory(final IConversionThrowableConsumer consumer) {
this.consumer = consumer;
}
static Factory getErrorReportingConverterFactory(final IConversionThrowableConsumer listener) {
return new ErrorReportingConverterFactory(listener);
}
#Override
public Converter<ResponseBody, ?> responseBodyConverter(final Type type, final Annotation[] annotations, final Retrofit retrofit) {
return (Converter<ResponseBody, Object>) responseBody -> {
final ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream();
final InputStream realInputStream = responseBody.byteStream();
try {
final ForwardingResponseBody bufferingResponseBody = new BufferingNoCloseResponseBOdy(responseBody, byteArrayOutputStream);
final Converter<ResponseBody, Object> converter = retrofit.nextResponseBodyConverter(this, type, annotations);
return converter.convert(bufferingResponseBody);
} catch ( final RuntimeException | IOException ex ) {
final InputStream inputStream = concatInputStreams(new ByteArrayInputStream(byteArrayOutputStream.toByteArray()), realInputStream);
consumer.accept(responseBody.contentType(), responseBody.contentLength(), inputStream, ex);
throw ex;
} finally {
responseBody.close();
}
};
}
private static class BufferingInputStream
extends ForwardingInputStream {
private final InputStream inputStream;
private final ByteArrayOutputStream byteArrayOutputStream;
private BufferingInputStream(final InputStream inputStream, final ByteArrayOutputStream byteArrayOutputStream) {
this.inputStream = inputStream;
this.byteArrayOutputStream = byteArrayOutputStream;
}
#Override
protected InputStream inputStream() {
return inputStream;
}
#Override
public int read()
throws IOException {
final int read = super.read();
if ( read != -1 ) {
byteArrayOutputStream.write(read);
}
return read;
}
#Override
public int read(final byte[] b)
throws IOException {
final int read = super.read(b);
if ( read != -1 ) {
byteArrayOutputStream.write(b, 0, read);
}
return read;
}
#Override
public int read(final byte[] b, final int off, final int len)
throws IOException {
final int read = super.read(b, off, len);
if ( read != -1 ) {
byteArrayOutputStream.write(b, off, read);
}
return read;
}
}
private static class BufferingNoCloseResponseBOdy
extends ForwardingResponseBody {
private final ResponseBody responseBody;
private final ByteArrayOutputStream byteArrayOutputStream;
private BufferingNoCloseResponseBOdy(final ResponseBody responseBody, final ByteArrayOutputStream byteArrayOutputStream) {
this.responseBody = responseBody;
this.byteArrayOutputStream = byteArrayOutputStream;
}
#Override
protected ResponseBody responseBody() {
return responseBody;
}
#Override
#SuppressWarnings("resource")
public BufferedSource source() {
final BufferedSource source = super.source();
return new ForwardingBufferedSource() {
#Override
protected BufferedSource bufferedSource() {
return source;
}
#Override
public InputStream inputStream() {
return new BufferingInputStream(super.inputStream(), byteArrayOutputStream);
}
};
}
/**
* Suppressing close due to automatic close in {#link ErrorReportingConverterFactory#responseBodyConverter(Type, Annotation[], Retrofit)}
*/
#Override
public void close() {
// do nothing
}
}
}
Note that this implementation uses forwarding classes heavily and only overrides what's necessary.
Also there are some utilities like concatenating input streams and adapting iterators to enumerations.
IteratorEnumeration.java
final class IteratorEnumeration<T>
implements Enumeration<T> {
private final Iterator<? extends T> iterator;
private IteratorEnumeration(final Iterator<? extends T> iterator) {
this.iterator = iterator;
}
static <T> Enumeration<T> iteratorEnumeration(final Iterator<? extends T> iterator) {
return new IteratorEnumeration<>(iterator);
}
#Override
public boolean hasMoreElements() {
return iterator.hasNext();
}
#Override
public T nextElement() {
return iterator.next();
}
}
InputStreams.java
final class InputStreams {
private InputStreams() {
}
static InputStream concatInputStreams(final InputStream... inputStreams) {
return inputStreams.length == 2
? new SequenceInputStream(inputStreams[0], inputStreams[1])
: new SequenceInputStream(iteratorEnumeration((Iterator<? extends InputStream>) asList(inputStreams).iterator()));
}
}
OutputStreamConversionThrowableConsumer.java
Trivial logging implementation.
final class OutputStreamConversionThrowableConsumer
implements IConversionThrowableConsumer {
private static final int BUFFER_SIZE = 512;
private final PrintStream printStream;
private OutputStreamConversionThrowableConsumer(final PrintStream printStream) {
this.printStream = printStream;
}
static IConversionThrowableConsumer getOutputStreamConversionThrowableConsumer(final OutputStream outputStream) {
return new OutputStreamConversionThrowableConsumer(new PrintStream(outputStream));
}
static IConversionThrowableConsumer getSystemOutConversionThrowableConsumer() {
return getOutputStreamConversionThrowableConsumer(System.out);
}
static IConversionThrowableConsumer getSystemErrConversionThrowableConsumer() {
return getOutputStreamConversionThrowableConsumer(System.err);
}
#Override
public void accept(final MediaType contentType, final long contentLength, final InputStream inputStream, final Throwable ex)
throws IOException {
printStream.print("Content type: ");
printStream.println(contentType);
printStream.print("Content length: ");
printStream.println(contentLength);
printStream.print("Content: ");
final byte[] buffer = new byte[BUFFER_SIZE];
int read;
while ( (read = inputStream.read(buffer)) != -1 ) {
printStream.write(buffer, 0, read);
}
printStream.println();
}
}
Put all together
final Gson gson = new Gson();
final Retrofit retrofit = new Retrofit.Builder()
.baseUrl(...)
.addConverterFactory(getErrorReportingConverterFactory(getSystemOutConversionThrowableConsumer()))
.addConverterFactory(GsonConverterFactory.create(gson))
.build();
final IWhateverService service = retrofit.create(IWhateverService.class);
final Call<...> call = service.getWhatever("test.json");
call.enqueue(new Callback<...>() {
#Override
public void onResponse(final Call<...> call, final Response<...> response) {
System.out.println(response.body());
}
#Override
public void onFailure(final Call<...> call, final Throwable throwable) {
throwable.printStackTrace(System.err);
}
});
Note that ErrorReportingConverterFactory must registered before the GsonConverterFactory. Let's assume the service requests for a JSON that's eventually illegal:
{"foo":1,###"bar":2}
In such case, the error reporting converter will produce the following dump to stdout:
Content type: application/json
Content length: -1
Content: {"foo":1,###"bar":2}
I'm not an expert in Log4j, and could not find an efficient way to get the output streams to redirect the input stream to. Here is the closest thing what I've found:
final class Log4jConversionThrowableConsumer
implements IConversionThrowableConsumer {
private static final int BUFFER_SIZE = 512;
private final Logger logger;
private Log4jConversionThrowableConsumer(final Logger logger) {
this.logger = logger;
}
static IConversionThrowableConsumer getLog4jConversionThrowableConsumer(final Logger logger) {
return new Log4jConversionThrowableConsumer(logger);
}
#Override
public void accept(final MediaType contentType, final long contentLength, final InputStream inputStream, final Throwable ex) {
try {
final StringBuilder builder = new StringBuilder(BUFFER_SIZE)
.append("Content type=")
.append(contentType)
.append("; Content length=")
.append(contentLength)
.append("; Input stream content=");
readInputStreamFirstChunk(builder, inputStream);
logger.error(builder.toString(), ex);
} catch ( final IOException ioex ) {
throw new RuntimeException(ioex);
}
}
private static void readInputStreamFirstChunk(final StringBuilder builder, final InputStream inputStream)
throws IOException {
final Reader reader = new InputStreamReader(inputStream);
final char[] buffer = new char[512];
final int read = reader.read(buffer);
if ( read >= 0 ) {
builder.append(buffer, 0, read);
}
}
}
Unfortunately, collecting the whole string may be expensive, so it only takes the very first 512 bytes. This may require callibrating the joined streams in the intermediate converter in order to "shift" the content "to the left" a bit.

Netty Channel fail when write and flush too many and too fast

When I write a producer to publish message to my server. I've seen this:
java.io.IOException: Connection reset by peer
at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
at sun.nio.ch.IOUtil.read(IOUtil.java:192)
at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:384)
at io.netty.buffer.UnpooledUnsafeDirectByteBuf.setBytes(UnpooledUnsafeDirectByteBuf.java:447)
at io.netty.buffer.AbstractByteBuf.writeBytes(AbstractByteBuf.java:881)
at io.netty.channel.socket.nio.NioSocketChannel.doReadBytes(NioSocketChannel.java:242)
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:119)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:511)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354)
at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111)
at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
at java.lang.Thread.run(Thread.java:745)
I've searched all around and was told that because of channel is closed.
But, in my code. I'm just close my channel when my channel pool destroy the channel.
Here my code:
public static class ChannelFactory implements PoolableObjectFactory<Channel> {
private final Bootstrap bootstrap;
private String host;
private int port;
public ChannelFactory(Bootstrap bootstrap, String host, int port) {
this.bootstrap = bootstrap;
this.host = host;
this.port = port;
}
#Override
public Channel makeObject() throws Exception {
System.out.println("Create new channel!!!");
bootstrap.validate();
return bootstrap.connect(host, port).channel();
}
#Override
public void destroyObject(Channel channel) throws Exception {
ChannelFuture close = channel.close();
if (close.isSuccess()) {
System.out.println(channel + " close successfully");
}
}
#Override
public boolean validateObject(Channel channel) {
System.out.println("Validate object");
return (channel.isOpen());
}
#Override
public void activateObject(Channel channel) throws Exception {
System.out.println(channel + " is activated");
}
#Override
public void passivateObject(Channel channel) throws Exception {
System.out.println(channel + " is passivated");
}
/**
* #return the host
*/
public String getHost() {
return host;
}
/**
* #param host the host to set
* #return
*/
public ChannelFactory setHost(String host) {
this.host = host;
return this;
}
/**
* #return the port
*/
public int getPort() {
return port;
}
/**
* #param port the port to set
* #return
*/
public ChannelFactory setPort(int port) {
this.port = port;
return this;
}
}
And here is my Runner:
public static class Runner implements Runnable {
private Channel channel;
private ButtyMessage message;
private MyChannelPool channelPool;
public Runner(MyChannelPool channelPool, Channel channel, ButtyMessage message) {
this.channel = channel;
this.message = message;
this.channelPool = channelPool;
}
#Override
public void run() {
channel.writeAndFlush(message.content()).syncUninterruptibly().addListener(new ChannelFutureListener() {
#Override
public void operationComplete(ChannelFuture future) throws Exception {
channelPool.returnObject(future.channel());
}
});
}
}
And my main:
public static void main(String[] args) throws InterruptedException {
final String host = "127.0.0.1";
final int port = 8080;
int jobSize = 100;
int jobNumber = 10000;
final Bootstrap b = func(host, port);
final MyChannelPool channelPool = new MyChannelPool(new ChannelFactory(b, host, port));
ExecutorService threadPool = Executors.newFixedThreadPool(1);
for (int i = 0; i < jobNumber; i++) {
try {
threadPool.execute(new Runner(channelPool, channelPool.borrowObject(), new ButtyMessage()));
} catch (Exception ex) {
System.out.println("ex = " + ex.getMessage());
}
}
}
With ButtyMessage extends ByteBufHolder.
In my Runner class, if I sleep(10) after writeAndFlush it run quite OK. But I don't want to reply on sleep. So I use ChannelFutureListener, but the result is bad. If I send about 1000 to 10.000 messages, it will crash and throw exception above. Is there any way to avoid this?
Thanks all.
Sorry for my bad explain and my English :)

You have several issues that could explain this. Most of them are related to wrong usage of asynchronous operations and future usage.
I don't know if it could be in link with your issue but, if you really want to print when the channel is really closed, you have to wait on the future, since the future on close() (or any other operations) immediately returns, without waiting for the real close. Therefore your test if (close.isSuccess()) shall be always false.
public void destroyObject(final Channel channel) throws Exception {
channel.close().addListener(new ChannelFutureListener() {
#Override
public void operationComplete(ChannelFuture close) {
if (close.isSuccess()) {
System.out.println(channel + " close successfully");
}
}
});
}
However, as I suppose it is only for debug purpose, it is not mandatory.
Another one: you send back to your pool a channel that is not already connected (which could explain your sleep(10) maybe?). You have to wait on the connect().
public Channel makeObject() throws Exception {
System.out.println("Create new channel!!!");
//bootstrap.validate(); // this is implicitely called in connect()
ChannelFuture future = bootstrap.connect(host, port).awaitUninterruptibly();
if (future.isSuccess()) {
return future.channel();
} else {
// do what you need to do when the connection is not done
}
}
third one: validation of a connected channel might be better using isActive():
#Override
public boolean validateObject(Channel channel) {
System.out.println("Validate object");
return channel.isActive(); // instead of isOpen()
}
fourth one: in your runner, you wrongly await on the future while you should not. You can remove your syncUninterruptibly() and let the rest as is.
#Override
public void run() {
Channel.writeAndFlush(message.content()).addListener(new ChannelFutureListener() {
#Override
public void operationComplete(ChannelFuture future) throws Exception {
channelPool.returnObject(future.channel());
}
});
}
And finally, I suppose you know your test is completely sequential (1 thread in your pool), such that each client will reuse over and over the very same channel?
Could you try to change the 4 points to see if it corrects your issue?
EDIT: after requester comment
For syncUntinterruptibly(), I did not read carefully. If you want to block on write, then you don't need the extra addListener since the future is done once the sync is over. So you can directly call your channelPool.returnObject as next command just after your sync.
So you should write it this way, simpler.
#Override
public void run() {
Channel.writeAndFlush(message.content()).syncUntinterruptibly();
channelPool.returnObject(future.channel());
}
For fireChannelActive, it will be called as soon as the connect finished (so from makeObject, sometime in the future). Moreover, once disconnected (as you did have notice in your exception), the channel is no more usable and must be recreated from zero. So I would suggest to use isActive however, such that, if not active, it will be removed using destroyObject...
Take a look at the channel state model here.

Finally, I've found a solution for myself. But, I'm still think about another solution. (this solution is exactly copy from 4.0.28 netty release note)
final String host = "127.0.0.1";
final int port = 8080;
int jobNumber = 100000;
final EventLoopGroup group = new NioEventLoopGroup(100);
ChannelPoolMap<InetSocketAddress, MyChannelPool> poolMap = new AbstractChannelPoolMap<InetSocketAddress, MyChannelPool>() {
#Override
protected MyChannelPool newPool(InetSocketAddress key) {
Bootstrap bootstrap = func(group, key.getHostName(), key.getPort());
return new MyChannelPool(bootstrap, new _AbstractChannelPoolHandler());
}
};
ChannelPoolMap<InetSocketAddress, FixedChannelPool> poolMap1 = new AbstractChannelPoolMap<InetSocketAddress, FixedChannelPool>() {
#Override
protected FixedChannelPool newPool(InetSocketAddress key) {
Bootstrap bootstrap = func(group, key.getHostName(), key.getPort());
return new FixedChannelPool(bootstrap, new _AbstractChannelPoolHandler(), 10);
}
};
final ChannelPool myChannelPool = poolMap.get(new InetSocketAddress(host, port));
final CountDownLatch latch = new CountDownLatch(jobNumber);
for (int i = 0; i < jobNumber; i++) {
final int counter = i;
final Future<Channel> future = myChannelPool.acquire();
future.addListener(new FutureListener<Channel>() {
#Override
public void operationComplete(Future<Channel> f) {
if (f.isSuccess()) {
Channel ch = f.getNow();
// Do somethings
ch.writeAndFlush(new ButtyMessage().content()).addListener(new ChannelFutureListener() {
#Override
public void operationComplete(ChannelFuture future) throws Exception {
if (future.isSuccess()) {
System.out.println("counter = " + counter);
System.out.println("future = " + future.channel());
latch.countDown();
}
}
});
// Release back to pool
myChannelPool.release(ch);
} else {
System.out.println(f.cause().getMessage());
f.cause().printStackTrace();
}
}
});
}
try {
latch.await();
System.exit(0);
} catch (InterruptedException ex) {
System.out.println("ex = " + ex.getMessage());
}
As you can see, I use SimpleChannelPool and FixedChannelPool (an implementation of SimpleChannelPool provided by netty).
What it can do:
SimpleChannelPool: open channels as much as it need ---> if you has 100.000 msg -> cuz error, of course. Many socket open, then IOExeption: Too many file open occur. (is that really pool? Create as much as possible and throw exception? I don't call this is pooling)
FixedChannelPool: not work in my case (Still study why? =)) Sorry for my stupidness)
Indeed, I want to use ObjectPool instead. And I may post it as soon as when I finish. Tks #Frederic Brégier for helping me so much!

Timeout-based BufferedWriter flush

I'm using BufferedWriter with the default size of 8192 characters to write lines to a local file. The lines are read from socket inputstream using BufferedReader readLine method, blocking I/O.
Average line length is 50 characters. It all works well and fast enough (over 1 mln lines per second) however if the client stops writing, lines that are currently stored in BufferedWriter buffer won't be flushed to disk. In fact the buffered characters won't be flushed to disk until the client resumes writing or the connection is closed. This translates into a delay between the time line is transmitted by client and the time this line is committed to file, so long-tail latency goes up.
Is there a way to flush incomplete BufferedWriter buffer on timeout, e.g. within 100 milliseconds?

What about something like this? It's not a real BufferedWriter, but it's a Writer. It works by periodically checking on on the last writer to the underlying, hopefully unbuffered writer, then flushing the BufferedWriter if it's been longer than the timeout.
public class PeriodicFlushingBufferedWriter extends Writer {
protected final MonitoredWriter monitoredWriter;
protected final BufferedWriter writer;
protected final long timeout;
protected final Thread thread;
public PeriodicFlushingBufferedWriter(Writer out, long timeout) {
this(out, 8192, timeout);
}
public PeriodicFlushingBufferedWriter(Writer out, int sz, final long timeout) {
monitoredWriter = new MonitoredWriter(out);
writer = new BufferedWriter(monitoredWriter, sz);
this.timeout = timeout;
thread = new Thread(new Runnable() {
#Override
public void run() {
long deadline = System.currentTimeMillis() + timeout;
while (!Thread.interrupted()) {
try {
Thread.sleep(Math.max(deadline - System.currentTimeMillis(), 0));
} catch (InterruptedException e) {
return;
}
synchronized (PeriodicFlushingBufferedWriter.this) {
if (Thread.interrupted()) {
return;
}
long lastWrite = monitoredWriter.getLastWrite();
if (System.currentTimeMillis() - lastWrite >= timeout) {
try {
writer.flush();
} catch (IOException e) {
}
}
deadline = lastWrite + timeout;
}
}
}
});
thread.start();
}
#Override
public synchronized void write(char[] cbuf, int off, int len) throws IOException {
this.writer.write(cbuf, off, len);
}
#Override
public synchronized void flush() throws IOException {
this.writer.flush();
}
#Override
public synchronized void close() throws IOException {
try {
thread.interrupt();
} finally {
this.writer.close();
}
}
private static class MonitoredWriter extends FilterWriter {
protected final AtomicLong lastWrite = new AtomicLong();
protected MonitoredWriter(Writer out) {
super(out);
}
#Override
public void write(int c) throws IOException {
lastWrite.set(System.currentTimeMillis());
super.write(c);
}
#Override
public void write(char[] cbuf, int off, int len) throws IOException {
lastWrite.set(System.currentTimeMillis());
super.write(cbuf, off, len);
}
#Override
public void write(String str, int off, int len) throws IOException {
lastWrite.set(System.currentTimeMillis());
super.write(str, off, len);
}
#Override
public void flush() throws IOException {
lastWrite.set(System.currentTimeMillis());
super.flush();
}
public long getLastWrite() {
return this.lastWrite.get();
}
}
}

#copeg is right - flush it after every line. It is easy to flush it at time period but what is the sense to have only half record and not be able to proceed it?

You might apply Observer, Manager, and Factory patterns here and have a central BufferedWriterManager produce your BufferedWriters and maintain a list of active instances. An internal thread might wake periodically and flush the active instances. This might also be an opportunity for Weak references so there is no requirement for your consumers to explicitly free the object. Instead, the GC will do the work and your Manager simply needs to handle the case when its internal reference becomes null (i.e. when all strong references are dropped).

Don't try this complex scheme, it's too hard. Just reduce the size of the buffer, by specifying it when constructing the BufferedWriter. Reduce it till you find the balance between performance and latency that you need.

Faster detection of interrupted connections during PUT operation with Apache 'HttpClient'

I am using Apache HttpClient 4 to communicate with a REST API and most of the time I do lengthy PUT operations. Since these may happen over an unstable Internet connection I need to detect if the connection is interrupted and possibly need to retry (with a resume request).
To try my routines in the real world I started a PUT operation and then I flipped the Wi-Fi switch of my laptop, causing an immediate total interruption of any data flow. However it takes a looong time (maybe 5 minutes or so) until eventually a SocketException is thrown.
How can I speed up to process? I'd like to set a timeout of maybe something around 30 seconds.
Update:
To clarify, my request is a PUT operation. So for a very long time (possibly hours) the only operation is a write() operation and there are no read operations. There is a timeout setting for read() operations, but I could not find one for write operations.
I am using my own Entity implementation and thus I write directly to an OutputStream which will pretty much immediately block once the Internet connection is interrupted. If OutputStreams had a timeout parameter so I could write out.write(nextChunk, 30000); I could detect such a problem myself. Actually I tried that:
public class TimeoutHttpEntity extends HttpEntityWrapper {
public TimeoutHttpEntity(HttpEntity wrappedEntity) {
super(wrappedEntity);
}
#Override
public void writeTo(OutputStream outstream) throws IOException {
try(TimeoutOutputStreamWrapper wrapper = new TimeoutOutputStreamWrapper(outstream, 30000)) {
super.writeTo(wrapper);
}
}
}
public class TimeoutOutputStreamWrapper extends OutputStream {
private final OutputStream delegate;
private final long timeout;
private final ExecutorService executorService = Executors.newSingleThreadExecutor();
public TimeoutOutputStreamWrapper(OutputStream delegate, long timeout) {
this.delegate = delegate;
this.timeout = timeout;
}
#Override
public void write(int b) throws IOException {
executeWithTimeout(() -> {
delegate.write(b);
return null;
});
}
#Override
public void write(byte[] b) throws IOException {
executeWithTimeout(() -> {
delegate.write(b);
return null;
});
}
#Override
public void write(byte[] b, int off, int len) throws IOException {
executeWithTimeout(() -> {
delegate.write(b, off, len);
return null;
});
}
#Override
public void close() throws IOException {
try {
executeWithTimeout(() -> {
delegate.close();
return null;
});
} finally {
executorService.shutdown();
}
}
private void executeWithTimeout(final Callable<?> task) throws IOException {
try {
executorService.submit(task).get(timeout, TimeUnit.MILLISECONDS);
} catch (TimeoutException e) {
throw new IOException(e);
} catch (ExecutionException e) {
final Throwable cause = e.getCause();
if (cause instanceof IOException) {
throw (IOException)cause;
}
throw new Error(cause);
} catch (InterruptedException e) {
throw new Error(e);
}
}
}
public class TimeoutOutputStreamWrapperTest {
private static final byte[] DEMO_ARRAY = new byte[]{1,2,3};
private TimeoutOutputStreamWrapper streamWrapper;
private OutputStream delegateOutput;
public void setUp(long timeout) {
delegateOutput = mock(OutputStream.class);
streamWrapper = new TimeoutOutputStreamWrapper(delegateOutput, timeout);
}
#AfterMethod
public void teardown() throws Exception {
streamWrapper.close();
}
#Test
public void write_writesByte() throws Exception {
// Setup
setUp(Long.MAX_VALUE);
// Execution
streamWrapper.write(DEMO_ARRAY);
// Evaluation
verify(delegateOutput).write(DEMO_ARRAY);
}
#Test(expectedExceptions = DemoIOException.class)
public void write_passesThruException() throws Exception {
// Setup
setUp(Long.MAX_VALUE);
doThrow(DemoIOException.class).when(delegateOutput).write(DEMO_ARRAY);
// Execution
streamWrapper.write(DEMO_ARRAY);
// Evaluation performed by expected exception
}
#Test(expectedExceptions = IOException.class)
public void write_throwsIOException_onTimeout() throws Exception {
// Setup
final CountDownLatch executionDone = new CountDownLatch(1);
setUp(100);
doAnswer(new Answer<Void>() {
#Override
public Void answer(InvocationOnMock invocation) throws Throwable {
executionDone.await();
return null;
}
}).when(delegateOutput).write(DEMO_ARRAY);
// Execution
try {
streamWrapper.write(DEMO_ARRAY);
} finally {
executionDone.countDown();
}
// Evaluation performed by expected exception
}
public static class DemoIOException extends IOException {
}
}
This is somewhat complicated, but it works quite well in my unit tests. And it works in real life as well, except that the HttpRequestExecutor catches the exception in line 127 and tries to close the connection. However when trying to close the connection it first tries to flush the connection which again blocks.
I might be able to dig deeper in HttpClient and figure out how to prevent this flush operation, but it is already a not too pretty solution, and it is just about to get even worse.
UPDATE:
It looks like this can't be done on the Java level. Can I do it on another level? (I am using Linux).

Java blocking I/O does not support socket timeout for write operations. You are entirely at the mercy of the OS / JRE to unblock the thread blocked by the write operation. Moreover, this behavior tends to be OS / JRE specific.
This might be a legitimate case to consider using a HTTP client based on non-blocking I/O (NIO) such as Apache HttpAsyncClient.

You can configure the socket timeout using RequestConfig:
RequestConfig myRequestConfig = RequestConfig.custom()
.setSocketTimeout(5000) // 5 seconds
.build();
When, when you do the call, just assign your new configuration. For instance,
HttpPut httpPut = new HttpPut("...");
httpPut.setConfig(requestConfig);
...
HttpClientContext context = HttpClientContext.create();
....
httpclient.execute(httpPut, context);
For more information regarthing timeout configurations, here there is a good explanation.

Her is one of the link i came across which talks connection eviction policy : here
public static class IdleConnectionMonitorThread extends Thread {
private final HttpClientConnectionManager connMgr;
private volatile boolean shutdown;
public IdleConnectionMonitorThread(HttpClientConnectionManager connMgr) {
super();
this.connMgr = connMgr;
}
#Override
public void run() {
try {
while (!shutdown) {
synchronized (this) {
wait(5000);
// Close expired connections
connMgr.closeExpiredConnections();
// Optionally, close connections
// that have been idle longer than 30 sec
connMgr.closeIdleConnections(30, TimeUnit.SECONDS);
}
}
} catch (InterruptedException ex) {
// terminate
}
}
public void shutdown() {
shutdown = true;
synchronized (this) {
notifyAll();
}
}}
I think you might want to look at this.

Simple Netty handler, iterate through message byte by byte

Is there an example of a very barebones Netty handler which simply gets whatever data is sent on the wire and writes it to a file?
I was thinking of something along the lines of this:
public class SimpleHandler extends SimpleChannelUpstreamHandler {
public void messageReceived(ChannelHandlerContext ctx, MessageEvent e) throws Exception {
HttpRequest request = (HttpRequest) e.getMessage();
// get data from request and write to a file
}
Any ideas? Thanks for any thoughts

You could also just use this method to transfer the content of the ChannelBuffer to an OutputStream:
http://netty.io/docs/stable/api/org/jboss/netty/buffer/ChannelBuffer.html#readBytes(java.io.OutputStream,%20int)
So something like this:
public class FileWriterHandler extends SimpleChannelHandler {
private final String filename;
public FileWriterHandler(String filename) {
this.filename = filename;
}
#Override
public void messageReceived(ChannelHandlerContext context, MessageEvent event) throws Exception{
ChannelBuffer buffer = (ChannelBuffer)event.getMessage();
FileOutputStream out = null;
try {
out = new FileOutputStream(filename, true)
buffer.readBytes(out, buffer.readableBytes());
} finally {
if (out != null) out.close();
}
}
}

I wrote one of these for testing a while back:
public class FileWriterHandler extends SimpleChannelHandler {
private final String filename;
public FileWriterHandler(String filename) {
this.filename = filename;
}
#Override
public void messageReceived(ChannelHandlerContext context, MessageEvent event) {
ChannelBuffer buffer = (ChannelBuffer)event.getMessage();
byte[] bytes = new byte[buffer.readableBytes()];
buffer.readBytes(bytes);
try {
DataOutputStream stream = new DataOutputStream(new FileOutputStream(filename, true));
stream.write(bytes, 0, bytes.length);
stream.flush();
stream.close();
} catch (IOException ex) {
throw runtime(ex);
}
}
}
This is just for test hence i'm just re-throwing exceptions and not really dealing with them properly. Hope that's helpful.
You would set this up as follows:
ServerBootstrap bootstrap = initializedSomehow();
bootstrap.setPipelineFactory(new ChannelPipelineFactory() {
#Override
public ChannelPipeline getPipeline() {
return pipeline(new FileWriterHandler("yourfile.txt"));
}
});