I am new to netty and I try to receive byte response from a server (non-netty). However, I belvie I missundersteand the reference count. While reading the response I am running into the IllegalReferenceCountException: refCnt:0
What I do is not that complicated. I set up the client, then I send a message and I am awaiting a response.
My ResponseHandler looks like that:
public class ResponseHandler extends DriverResponseHandler
{
private ByteBuf received;
#Override
public void handlerAdded(ChannelHandlerContext ctx) throws Exception
{
super.handlerAdded(ctx);
received = ctx.alloc().buffer(32);
}
#Override
public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception
{
super.channelRead(ctx, msg);
received.writeBytes((ByteBuf) msg);
}
#Override
public void channelReadComplete(ChannelHandlerContext ctx) throws Exception
{
super.channelReadComplete(ctx);
if (received.writerIndex() == received.capacity() || received.readerIndex() == received.capacity()) {
received.clear();
received = ctx.alloc().buffer(32);
}
processResponse(received);
}
}
while reading the response in channelRead the exception occurres. Why does it happen. Could someone explain it to me?
Okay, I solved the problem. Everything I need to do is add ByteToMessageDedcoder, otherwise the incoming response will never be allocated so it won't be accessible.
Related
I've been working on a Java application that utilizes the openshift api. Specifically OpenShift deployment configuration
I have tried to set up a watcher, but my response body is never called. I am already able to get a response from the 'non watcher' APIcalls. I am using the groovy httpbuilder library to fulfill my request
def http = new HTTPBuilder(<<URL TO OPENSHIFT>>)
try {
http.get(path: '/oapi/v1/watch/namespaces/myproject/deploymentconfigs', contentType: "application/json") { resp, reader ->
println(resp)
return reader
}
} catch (HttpResponseException e) {
System.out.println(e)
}
Please Advise on a path forward to set up OpenShift watchers in my application.
An error message is never thrown. minishift logs -f are not providing any feedback either.
Also note that I have gotten this to work with the curl command, documented in the api
You can use the OKHttpClient to handle the http websocket upgrade protocol for you. Note legacy versions of minishift require the query parameter "access_token" when trying to make a websocket connection request
OkHttpClient client = new OkHttpClient();
def token = token
Request request = new Request.Builder()
.get()
.url("https://<<IP>>/oapi/v1/watch/namespaces/<<namespace>>/deploymentconfigs?watch=true&access_token=<<token>>")
.addHeader("Accept", "application/json")
.addHeader("Connection", "close")
.addHeader("Sec-WebSocket-Protocol",'base64url.bearer.authorization.k8s.io.' + Base64.getEncoder().encodeToString(token.getBytes()))
.addHeader('Origin', 'https://<<IP>>')
.build()
WebSocketListener websocketListener= new WebSocketListenerImpl()
client.newWebSocket(request, websocketListener)
WebSocketListenerImpl Class
import okhttp3.Response;
import okhttp3.WebSocket;
import okhttp3.WebSocketListener;
import okio.ByteString;
public class WebSocketListenerImpl extends WebSocketListener {
public WebSocketListenerImpl() {
super();
}
#Override
public void onOpen(WebSocket webSocket, Response response) {
super.onOpen(webSocket, response);
print "WEBSOCKET OPEN"
}
#Override
public void onMessage(WebSocket webSocket, String text) {
super.onMessage(webSocket, text);
print "WEBSOCKET RECEIVED"
}
#Override
public void onMessage(WebSocket webSocket, ByteString bytes) {
super.onMessage(webSocket, bytes);
print "WEBSOCKET OPEN"
}
#Override
public void onClosing(WebSocket webSocket, int code, String reason) {
super.onClosing(webSocket, code, reason);
print "WEBSOCKET CLOSING"
}
#Override
public void onClosed(WebSocket webSocket, int code, String reason) {
super.onClosed(webSocket, code, reason);
print "WEBSOCKET CLOSED"
}
#Override
public void onFailure(WebSocket webSocket, Throwable t, #javax.annotation.Nullable Response response) {
super.onFailure(webSocket, t, response);
println "WEBSOCKET FAILED"
}
}
I have a small java application that does the following:
reads messages (json strings) from a queue and sends it to a sender
sender accumulates the received messages and when it reaches a specific size (lets says 5k) makes a http call and posts those messages
apache http client is used to post the messages and it is asynchronous, meaning I don't wait for the response and respective call back methods are invoked upon completing the post
Here is the pseudo code
Reader class: SomeProcessor.java
public class SomeProcessor extends Processor {
#Override
public void process(Messages m) {
// some processing on m
String jsonMessage = convertToJSON(m);
getSender().send(jsonMessage);
}
}
Base class: Processor.java
public class Processor {
private HttpSender sender = null ;
public Processor() {
setSender(new HttpSender());
}
public HttpEventCollectorSender getSender() {
return sender;
}
public void setSender(HttpEventCollectorSender sender) {
this.sender = sender;
}
}
Sender class: HttpSender.java
public class HttpSender {
private List<String> eventsBatch = new ArrayList<String>(5000);
public synchronized void send(final String message) {
eventsBatch.add(message);
if (eventsBatch.size() >= 5000) {
flush(); // calls http post
}
}
public synchronized void flush() {
if (eventsBatch.size() > 0) {
postEventsAsync(eventsBatch);
}
// since the above call is asynchronous after the post is called, I am assuming I should re-init the list, instead of clear. Is this correct?
eventsBatch = new ArrayList<String>(5000);
}
public void postEventsAsync(final List<String> events) {
startHttpClient(); // make sure http client is started
final String encoding = "utf-8";
// create http request
final HttpPost httpPost = new HttpPost(url);
httpPost.setHeader(AuthorizationHeaderTag, String.format(AuthorizationHeaderScheme, token));
StringEntity entity = new StringEntity(String.join("", events), encoding);
entity.setContentType(HttpContentType);
httpPost.setEntity(entity);
httpClient.execute(httpPost, new FutureCallback<HttpResponse>() {
#Override
public void completed(HttpResponse response) {
// log to console
}
#Override
public void failed(Exception ex) {
// just log to console
}
#Override
public void cancelled() {
}
});
}
}
Overall, I see a very high utilization of memory and noticed that even when processing has been stopped heap is not getting cleared. I looked into the heap dump and I see my "json string" messages represented as char[]. I suspect that something funky is happening with all those String not being GC'ed.
Thoughts?
Update-1: Based on the comments from below, attaching a Heap snapshot where the processing was paused and the heap space is still 4GB
Update-2: GC Report
http://gceasy.io/my-gc-report.jsp?p=c2hhcmVkLzIwMTcvMDcvNS8tLXN0cmVhbWdlc3RfZ2MuemlwLS0xNy03LTMx
I have a Echo server example from Official Netty
Echo Server
How to add ability to connect and streaming to it from websocket?
here is my ServerHandler code:
public class ServerHandler extends ChannelInboundHandlerAdapter
{
#Override
public void channelRegistered(ChannelHandlerContext ctx) throws Exception {
super.channelRegistered(ctx);
// !!!!! Think here should be WebSocket Handshake?
}
#Override
public void channelRead(ChannelHandlerContext ctx, Object msg)
{
System.out.println(msg);
ctx.write(msg);
}
#Override
public void channelReadComplete(ChannelHandlerContext ctx)
{
ctx.flush();
}
#Override
public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause)
{
// Close the connection when an exception is raised.
cause.printStackTrace();
}
}
Right now Chrome connection says: WebSocket connection to 'ws://127.0.0.1:8080/' failed: Error during WebSocket handshake: Invalid status line
Netty servers do not automatically handle all protocols, so you would need to add support for WebSockets.
I find the best place to start is to examine the relevant examples in Netty's xref page. Scroll down the package list until you get to the io.netty.example packages. In that list you will find a package called io.netty.example.http.websocketx.server. There is a fairly simple and well laid out example on how to implement a websocket server, or just the handler.
Websocket servers are slightly more complicated than other servers in that they must start life as an HTTP server because the protocol specifies that websockets must be initiated by "upgrading" an HTTP connection, but as I said, the example referenced above makes this fairly clear.
So, I found solution! It is not compliant to native documentation of web-socket, but who cares it works as I expected!
public void channelRead(ChannelHandlerContext ctx, Object msg)
{
DefaultHttpRequest httpRequest = null;
if (msg instanceof DefaultHttpRequest)
{
httpRequest = (DefaultHttpRequest) msg;
// Handshake
WebSocketServerHandshakerFactory wsFactory = new WebSocketServerHandshakerFactory("ws://127.0.0.1:8080/", null, false);
final Channel channel = ctx.channel();
final WebSocketServerHandshaker handshaker = wsFactory.newHandshaker(httpRequest);
if (handshaker == null) {
} else {
ChannelFuture handshake = handshaker.handshake(channel, httpRequest);
}
}
}
Do not forget to add
p.addLast(new HttpRequestDecoder(4096, 8192, 8192, false));
p.addLast(new HttpResponseEncoder());
to your pipeline.
I've a flow where on CXF client I've jaxrs-in-interceptor, provider and exception mapper. In my case I'm catching bad response from client through in-interceptor and then I would like abort the cxf bus chain and throw a fault. Unfortunately I couldn't do it, cause in every situation exception thrown from interceptor is being only logged, but the main error (wrong json format) is propagated to exception mapper. I would like to avoid Exception mapper, but I don't know how. I'm using WebClient to implement interceptors like this:
#Component
public class MyInterceptor extends AbstractPhaseInterceptor<Message> {
public MyInterceptor() {
super(POST_STREAM);
}
#Override
public void handleMessage(Message message) throws Fault {
if (message != null) {
//message.getExchange().setOneWay(true);
//message.getExchange().put(Exception.class, new MyException());
//message.getInterceptorChain().abort();
//message.setContent(Exception.class, new MyException());
//Endpoint ep = message.getExchange().get(Endpoint.class);
//message.getInterceptorChain().abort();
//if (ep.getInFaultObserver() != null) {
// ep.getInFaultObserver().onMessage(message);
//}
//throw new WebApplicationException( new MyException());
//message.setContent(Response.class, response);
throw new Fault(new MyException());
}
}
I read that I should implement jaxrs-filter cause exceptions thrown by interceptor are not propagated to exception mapper. Is it any way to do that in java thanks to WebClient implementation?
S client = create(url, clazz, list(jsonProvider(), providers));
WebClient.getConfig(client).getInInterceptors().add(new MyInterceptor());
I've also tried to use different phases on interceptor, but it also didn't work.
I have been researching and testing with your issue. The problem is that the exceptions thrown from the CXF interceptors escape the JAX-RS flow (see the answer of CXF team)
A Fault generated from interceptor can be catched implementing handleFault in the interceptor itself
public void handleFault(Message message) {
Exception e = message.getContent(Exception.class);
}
Or implementing a FaultListener and registering it at CXF Bus
WebClient.getConfig(client).getBus().getProperties().put("org.apache.cxf.logging.FaultListener",new MyFaultListener());
public class MyFaultListener implements FaultListener{
public boolean faultOccurred(final Exception exception,final String description,final Message message) {
//return false to avoid warning of default CXF logging interceptor
return false;
}
}
But you can not return custom response from interceptor or respond a Fault to client.
The workaround I have found to achieve the desired behaviour consist in replacing the Response with a custom object that could be processed by your usual method invokation, like an exceptionMapper
See CXF/ JAX-RS : Return Custom response from interceptor
Into Interceptor.handleMessage check the conditions you need and create a Response with custom status and entity. After this, stop the chain
public class MyInterceptor extends AbstractPhaseInterceptor<Message> {
public MyInterceptor() {
super(Phase.POST_STREAM);
}
#Override
public void handleMessage(Message message) throws Fault {
if (message != null) {
//check the condition to raise the error
//build the custom Response replacing service call
Response response = Response
.status(Response.Status.BAD_REQUEST)
.entity("custom error")
.build();
message.getExchange().put(Response.class, response);
//abort interceptor chain in you want to stop processing or throw a Fault (catched by handleFault)
//message.getInterceptorChain().abort();
//throw new Fault (new MyException());
}
public void handleFault(Message messageParam) {
}
}
Add the ResponseExceptionMapper as provider when creating the JAXRS client
providers.add(new ResponseExceptionMapper<WebApplicationException>() {
#Override
public WebApplicationException fromResponse(Response r) {
return new WebApplicationException(r);
}
});
YourService proxy = JAXRSClientFactory.create(url, clazz,providers);
Client client = WebClient.client(proxy);
WebClient.getConfig(client).getInInterceptors().add(new MyInterceptor());
After this, a call to proxy.yourService() will raise a WebApplicationException if acomplish the interceptor check. You can catch it or rethrow in the desired way
try{
proxy.yourService();
}catch (WebApplicationException e){
}
Hope this helps
I fully agree with previous answer. My implementation looks like:
#Component
public class ServiceFailureInterceptor extends AbstractPhaseInterceptor<Message> {
private static final Logger LOG = LoggerFactory.getLogger(ServiceFailureInterceptor.class);
public ServiceFailureInterceptor() {
super(PRE_STREAM);
}
#Override
public void handleMessage(Message message) {
if (message != null) {
int responseCode = (int) message.get(Message.RESPONSE_CODE);
LogicException logicException = ErrorMapper.HTTP_STATUS_CODE_MAPPER.get(responseCode);
InputStream is = b2stream(MapperUtils.json().toBytes(logicException));
// clear old message & exchange
Exchange exchange = message.getExchange();
for (Class<?> contentFormat : message.getContentFormats()) {
message.setContent(contentFormat, null);
}
resetOrigInterceptorChain(message);
resetFault(exchange);
message.setContent(InputStream.class, is);
Message outMessage = createOutMessage(exchange, is);
prepareMessage(outMessage);
prepareMessage(message);
}
}
private void prepareMessage(Message message) {
message.put(Message.REQUESTOR_ROLE, true);
message.put(HttpHeaders.CONTENT_TYPE, MediaType.APPLICATION_JSON);
}
private Message createOutMessage(Exchange exchange, InputStream logicException) {
Endpoint ep = exchange.get(Endpoint.class);
Message outMessage = ep != null ? ep.getBinding().createMessage() : new MessageImpl();
outMessage.setContent(InputStream.class, logicException);
exchange.setOutMessage(outMessage);
outMessage.setExchange(exchange);
return outMessage;
}
private void resetFault(Exchange exchange) {
exchange.put(Exception.class, null);
}
private void resetOrigInterceptorChain(Message message) {
InterceptorChain chain = message.getInterceptorChain();
if (chain != null) {
for (Interceptor<?> interceptor : chain) {
chain.remove(interceptor);
}
chain.reset();
}
}
}
After setting this exception manually I'm going to ExceptionMapper implementation where my LogicException is consumed and response with exception is building. I cannot avoid Exception mapper when is declared as a provider through WebClient, so I've decided to use it and remapped Exception later.
I have a websocket server using Netty (4.0.17) that answers requests from a JavaScript client.
The communication works fine but I have an exception when I try to immediately send a greeting message when a client connects.
My code looks like that :
public class LiveOthelloWSHandler extends SimpleChannelInboundHandler<TextWebSocketFrame> {
#Override
public void channelActive(ChannelHandlerContext ctx) throws Exception {
super.channelActive(ctx);
ChannelFuture f = ctx.channel().writeAndFlush(new TextWebSocketFrame("(gameID=0)[LiveOthelloServer="+ VERSION_NUMBER + "]\n"));
}
// ...
#Override
protected void channelRead0(ChannelHandlerContext ctx, TextWebSocketFrame frame) throws Exception {
final String request = frame.text();
Channel thisChannel = ctx.channel();
// Do something with request
// Write back
thisChannel.writeAndFlush(new TextWebSocketFrame(response + "\n"));
}
}
The channelRead0() is ok, the client sends messages and the server answers back without any issue.
What doesn't work is the "greetings" part. I would like to send a welcoming message to the client (the string using VERSION_NUMBER in the ChannelActive() method) but I always get an exception :
java.lang.UnsupportedOperationException: unsupported message type: TextWebSocketFrame
I guess this is maybe because the channelActive() gets invoked as soon as the connection is established but before the websocket handshake is complete. How can I wait for the handshake to be finished and then send the greeting message (without the client having sent any request yet)?
For information, my initialization is:
#Override
public void initChannel(SocketChannel ch) throws Exception {
ChannelPipeline pipeline = ch.pipeline();
pipeline.addLast(
new HttpRequestDecoder(),
new HttpObjectAggregator(65536),
new HttpResponseEncoder(),
new WebSocketServerProtocolHandler("/websocket"),
myLiveOthelloWSHandler);
Just RTFM...
http://netty.io/4.0/api/io/netty/handler/codec/http/websocketx/WebSocketServerProtocolHandler.html
Best way to detect handshake is to override ChannelInboundHandler.userEventTriggered...
So I just had to add:
#Override
public void userEventTriggered(ChannelHandlerContext ctx, Object evt) throws Exception {
super.userEventTriggered(ctx, evt);
if (evt == WebSocketServerProtocolHandler.ServerHandshakeStateEvent.HANDSHAKE_COMPLETE) {
ChannelFuture f = ctx.channel().writeAndFlush(new TextWebSocketFrame("(gameID=0)[LiveOthelloServer="+ VERSION_NUMBER + "]\n"));
}
}