The goal is :develop a custom Kafka connector that read ,messages from the websocket in a loop method. I try to give you an example on what I've realized:
I create an interface IWebsocketClientEndpoint
public interface IWebsocketClientEndpoint {
IWebsocketClientEndpoint Connect() ;
void Disconnect() throws IOException;
IWebsocketClientEndpoint addMessageHandler(IMessageHandler msgHandler);
void SendMessage(String message) throws Exception;
void SendMessage(ByteBuffer message) throws Exception;
void SendMessage(Object message) throws Exception;
boolean isOpen();
void Dispose()throws IOException;
}
and a class that implement above interface:
#ClientEndpoint
public class WebsocketClientEndpoint implements IWebsocketClientEndpoint {
private WebSocketContainer _container;
private Session _userSession = null;
private IMessageHandler _messageHandler;
private URI _endpointURI;
private WebsocketClientEndpoint(URI endpointURI) {
try {
_endpointURI = endpointURI;
_container = ContainerProvider.getWebSocketContainer();
} catch (Exception e) {
throw new RuntimeException(e);
}
}
private WebsocketClientEndpoint(URI endpointURI, int bufferSize) {
try {
_endpointURI = endpointURI;
_container = ContainerProvider.getWebSocketContainer();
_container.setDefaultMaxBinaryMessageBufferSize(bufferSize);
_container.setDefaultMaxTextMessageBufferSize(bufferSize);
} catch (Exception e) {
throw new RuntimeException(e);
}
}
public static IWebsocketClientEndpoint Create(URI endpointURI){
return new WebsocketClientEndpoint(endpointURI);
}
public static IWebsocketClientEndpoint Create(URI endpointURI,int bufferSize){
return new WebsocketClientEndpoint(endpointURI,bufferSize);
}
public IWebsocketClientEndpoint Connect() {
try {
_container.connectToServer(this, _endpointURI);
return this;
} catch (Exception e) {
throw new RuntimeException(e);
}
}
#OnOpen
public void onOpen(Session userSession) {
this._userSession = userSession;
if (this._messageHandler != null) {
this._messageHandler.handleOpen("Web socket "+ _endpointURI +" opened");}
}
#OnClose
public void onClose(Session userSession, CloseReason reason) {
this._userSession = null;
if (this._messageHandler != null) {
this._messageHandler.handleClose("Web socket "+ _endpointURI +" closed. Reason: " + reason.getReasonPhrase());}
}
public void Disconnect() throws IOException {
CloseReason reason = new CloseReason(CloseReason.CloseCodes.NORMAL_CLOSURE,"Web socket closed by user");
this._userSession.close(reason);
this._userSession = null;
//close notification to caller
if (this._messageHandler != null) {
this._messageHandler.handleClose("Web socket "+ _endpointURI +" closed. Reason: " + reason.getReasonPhrase());}
}
#Override
public IWebsocketClientEndpoint addMessageHandler(IMessageHandler msgHandler) {
this._messageHandler = msgHandler;
return this;
}
#OnMessage
public void onMessage(String message) {
if (this._messageHandler != null) {
this._messageHandler.handleMessage(message);
}
}
#OnMessage
public void onMessage(ByteBuffer bytes) {
if (this._messageHandler != null) {
this._messageHandler.handleMessage(bytes);
}
}
public void SendMessage(String message) throws Exception {
try{
this._userSession.getAsyncRemote().sendText(message);
}catch (Exception ex){
throw ex;
}
}
public void SendMessage(ByteBuffer message) throws Exception {
try{
this._userSession.getAsyncRemote().sendBinary(message);
}catch (Exception ex){
throw ex;
}
}
public void SendMessage(Object message) throws Exception {
this._userSession.getAsyncRemote().sendObject(message);
}catch (Exception ex){
throw ex;
}
}
#Override
public boolean isOpen() {
if (this._userSession != null){
return this._userSession.isOpen();
}
return false;
}
}
The class WebsocketClientEndpoint is dedicated to the creation of websocket and manage of connection, disconnection, send and receive message.
The goal is: how can I adapt the my websocket structure in the kafka connect structure? I could queue the message received ("public void handleMessage(String s)) from the socket in a ConcurrentLinkedQueue, and then, in the kafka connect loop method, unqueue them. But is it the best solution?
Below, the implementation of my Kafka custom connector
My kafka Connector
public class MySourceTask extends SourceTask {
IWebsocketClientEndpoint _clientEndPoint;
#Override
public void start(Map<String, String> props) {
_clientEndPoint = WebsocketClientEndpoint
.Create(new URI(socket))
.Connect();
_clientEndPoint.addMessageHandler(new IMessageHandler() {
#Override
public void handleMessage(String s) {
}
#Override
public void handleMessage(ByteBuffer byteBuffer) {
}
#Override
public void handleClose(String s) {
}
#Override
public void handleOpen(String s) {
}
});
}
#Override
public List<SourceRecord> poll() throws InterruptedException {
return null;
}
#Override
public void stop() {
_clientEndPoint.Dispose();
}
}
Thanks in advance to anyone
I'd suggest adding the interface to the class
extends SourceTask implements IMessageHandler
Then
_clientEndPoint.addMessageHandler(this);
And when you implement handleMessage, add those strings to some queue. Inside the poll method, you would pop data off that queue to create SourceRecord objects to return.
Inside of stop, call this.handleClose and clean up other resources.
Related
A RMI server which works fine without the stopServer functionality.
public class HelloServer extends UnicastRemoteObject implements HelloInterface
{
private final static int PORT=1102;
private final String serverName="server";
private Timer timer;
public HelloServer() throws RemoteException
{
timer = new Timer(); //At this line a new Thread will be created
timer.schedule(new StopServerTask(), 5000);
}
#Override
public String serverResponse(String request) throws RemoteException
{
return "Hello"+request;
}
public static void main(String[] args)
{
try
{
HelloServer skeleton=new HelloServer();
System.out.println("Starting server");
skeleton.startServer();
System.out.println("Server started");
}
catch (RemoteException ex)
{
ex.printStackTrace();
}
}
public void startServer()
{
try {
HelloServer skeleton=new HelloServer();
Registry reg=LocateRegistry.createRegistry(PORT);
reg.rebind(serverName, skeleton);
System.out.println("Server is ready");
} catch (RemoteException ex)
{
Logger.getLogger(HelloInterface.class.getName()).log(Level.SEVERE, null, ex);
}
}
public void stopServer()
{
System.out.println("Stopping server");
try {
Registry rmiRegistry = LocateRegistry.getRegistry(PORT);
HelloInterface myService = (HelloInterface) rmiRegistry.lookup(serverName);
rmiRegistry.unbind(serverName);
UnicastRemoteObject.unexportObject(rmiRegistry, true);
} catch (NoSuchObjectException e)
{
e.printStackTrace();
} catch (NotBoundException e)
{
e.printStackTrace();
} catch (RemoteException ex) {
Logger.getLogger(HelloServer.class.getName()).log(Level.SEVERE, null, ex);
}
}
class StopServerTask extends TimerTask
{
#Override
public void run()
{
stopServer();
}
}
}
Whenever stopServer() in invoked exception is thrown at
UnicastRemoteObject.unexportObject(rmiRegistry, true);
Here is the stack Trace
java.rmi.NoSuchObjectException: object not exported
at sun.rmi.transport.ObjectTable.unexportObject(ObjectTable.java:153)
at java.rmi.server.UnicastRemoteObject.unexportObject(UnicastRemoteObject.java:297)
at rmi.HelloServer.stopServer(HelloServer.java:84)
Things are same even when I clean the service object by using
UnicastRemoteObject.unexportObject(myService, true);
Could someone suggest a clean way to stop the server which also releases the port for reuse.
You need to store the result of LocateRegistry.createRegistry(), and unexport that. At present you're trying to unexport a stub.
I implemented a shutdown-service in my rmi-server. If I want to shut it down, I call it with a password. Simple Example:
public interface ShutdownInterface extends Remote {
public void shutdownService(String password) throws RemoteException;
}
The serverside implementation can look something like:
public class ShutdownService extends UnicastRemoteObject implements ShutdownInterface {
private static final long serialVersionUID = 1L;
private boolean doShutdown = false;
public ShutdownService() throws RemoteException {
super();
}
#Override
public void shutdownService(String password) throws RemoteException {
if ("abcde12345".equals(password)) {
System.out.println("shutdown requested.");
this.doShutdown = true;
} else {
System.out.println("wrong pwd for shutdown");
}
}
public boolean isDoShutdown() {
return this.doShutdown;
}
}
Now the server itself keeps a reference to this:
public class BackendServer {
public final static int RMI_PORT = 1974;
private Registry registry = null;
private ShutdownService shutdownService = null;
public BackendServer() throws RemoteException {
registry = LocateRegistry.createRegistry(RMI_PORT);
this.shutdownService = new ShutdownService();
}
public void initialize() throws AccessException, RemoteException, AlreadyBoundException {
shutdownService = new ShutdownService();
registry.bind("ShutdownService", shutdownService);
registry.bind("MyDataService", new MyDataService());
}
public void stop() throws NoSuchObjectException {
System.out.println("stopping rmi server.");
UnicastRemoteObject.unexportObject(registry, true);
System.exit(0);
}
public boolean shouldStop() {
return this.shutdownService.isDoShutdown();
}
public static void main(String args[]) {
try {
BackendServer bs = new BackendServer();
bs.initialize();
System.out.println("Server ready.");
while (!bs.shouldStop()) {
Thread.sleep(1000);
}
bs.stop();
} catch (Exception e) {
System.err.println("Server exception: " + e.toString());
e.printStackTrace();
}
}
}
Of course, this can be realized in a more beautiful way, but this should give you an idea of how to easily implement a shutdown yourself. You can call it from the main client or from a small commandline-tool you code for your server.
I have a server that contains an ArrayList in " ServerInfo " and when I try to take from ClientRMI an element of the ArrayList(in ServerInfo) for example adf.getSGM ( 0 ).incrementCount( ) ;
"count" does not increase it's as if every time I call it instantiates a new class SGM
in a few words I want to interact from ClientRMI with ArrayList that is on ServerInfo (SORRY FOR ENGLISH)
Hear are the classes :
SERVER
public class ServerRMI {
public static void main(String[] args) {
Registry registry = null;
String name = "ServerInfo";
try {
System.out.println("Init RMI");
ServerInfoInterface sir = ServerInfo.getInstance();
ServerInfoInterface stub = (ServerInfoInterface) UnicastRemoteObject.exportObject(sir, 0);
registry = LocateRegistry.createRegistry(9000);
registry.bind(name, stub);
System.out.println("RMI OK");
System.out.println("Init SGM...");
for(int i=0;i<3;i++){
ServerInfo.getInstance().addSGM(new SGM());
}
System.out.println("Init SGM OK");
} catch (Exception e) {
System.out.println("RMI Error"+e.toString());
registry = null;
}
}
}
public class ServerInfo implements ServerInfoInterface{
private ArrayList<SGM> sgmHandler = new ArrayList<SGM>();
// Singleton pattern
private static ServerInfo instance;
// Singleton pattern
public static ServerInfo getInstance() {
if (instance == null){
System.out.println("ServerInfo new instance");
instance = new ServerInfo();
}
return instance;
}
#Override
public synchronized void addSGM(SGM sgm) throws RemoteException {
sgmHandler.add(sgm);
}
#Override
public synchronized SGM getSGM(int i) throws RemoteException {
return sgmHandler.get(i);
}
}
public interface ServerInfoInterface extends Remote{
public void addSGM(SGM sgm) throws RemoteException;
public SGM getSGM(int i) throws RemoteException;
}
public class SGM implements Serializable{
/**
*
*/
private static final long serialVersionUID = -4756606091542270097L;
private int count=0;
public void incrementCount(){
count++;
}
public void decrementCount(){
count--;
}
public int getCount(){
return count;
}
}
CLIENT
public class ClientRMI {
private ServerInfoInterface sgmInterface;
public void startServer() {
String name = "ServerInfo";
Registry registry;
try {
registry = LocateRegistry.getRegistry(9000);
try {
sgmInterface = (ServerInfoInterface) registry.lookup(name);
sgmInterface.getSGM(0).incrementCount();
System.out.println(sgmInterface.getSGM(0).getCount()); // always 0
} catch (AccessException e) {
System.out.println("RIM AccessException"+ e.toString());
} catch (RemoteException e) {
System.out.println("RIM RemoteException"+ e.toString());
} catch (NotBoundException e) {
System.out.println("RIM NotBoundException"+ e.toString());
}
} catch (RemoteException e) {
System.out.println("RIM RemoteException registry"+ e.toString());
}
}
}
You're creating an SGM at the server, passing it via Serialization to the client, incrementing its count at the client, and then expecting that count to be magically increased at the server.
It can't work.
You will have to make SGM a remote object, with its own remote interface, or else provide a remote method in the original remote interface to increment the count of a GSM, specified by index.
when I tried to connect nodejs from java I am getting this error :
engine intercepting request for path "/socket.io/" +3m
engine handling "GET" http request "/socket.io/1/" +0ms
engine unknown transport "undefined" +0ms
Here is my sample :
package basic;
public class BasicExample implements IOCallback {
public static void main(String[] args) {
try {
new BasicExample();
} catch (Exception e) {
e.printStackTrace();
}
}
public BasicExample() throws Exception {
SocketIO socket = new SocketIO("my nodejs path");
socket.connect(new IOCallback() {
public void onMessage(JSONObject json, IOAcknowledge ack) {
}
public void onMessage(String data, IOAcknowledge ack) {
}
public void onError(SocketIOException socketIOException) {
}
public void onDisconnect() {
}
public void onConnect() {
}
public void on(String event, IOAcknowledge ack, Object... args) {
}
});
socket.send("Hello Server");
socket.send(new JSONObject().put("key", "value").put("key2",
"another value"));
socket.emit("event", "argument1", "argument2", 13.37);
}
public void onMessage(JSONObject json, IOAcknowledge ack) {
try {
System.out.println("Server said:" + json.toString(2));
} catch (JSONException e) {
e.printStackTrace();
}
}
public void onMessage(String data, IOAcknowledge ack) {
System.out.println("Server said: " + data);
}
public void onError(SocketIOException socketIOException) {
System.out.println("an Error occured");
socketIOException.printStackTrace();
}
public void onDisconnect() {
System.out.println("Connection terminated.");
}
public void onConnect() {
System.out.println("Connection established");
}
public void on(String event, IOAcknowledge ack, Object... args) {
System.out.println("Server triggered event '" + event + "'");
}
}
A RMI server which works fine without the stopServer functionality.
public class HelloServer extends UnicastRemoteObject implements HelloInterface
{
private final static int PORT=1102;
private final String serverName="server";
private Timer timer;
public HelloServer() throws RemoteException
{
timer = new Timer(); //At this line a new Thread will be created
timer.schedule(new StopServerTask(), 5000);
}
#Override
public String serverResponse(String request) throws RemoteException
{
return "Hello"+request;
}
public static void main(String[] args)
{
try
{
HelloServer skeleton=new HelloServer();
System.out.println("Starting server");
skeleton.startServer();
System.out.println("Server started");
}
catch (RemoteException ex)
{
ex.printStackTrace();
}
}
public void startServer()
{
try {
HelloServer skeleton=new HelloServer();
Registry reg=LocateRegistry.createRegistry(PORT);
reg.rebind(serverName, skeleton);
System.out.println("Server is ready");
} catch (RemoteException ex)
{
Logger.getLogger(HelloInterface.class.getName()).log(Level.SEVERE, null, ex);
}
}
public void stopServer()
{
System.out.println("Stopping server");
try {
Registry rmiRegistry = LocateRegistry.getRegistry(PORT);
HelloInterface myService = (HelloInterface) rmiRegistry.lookup(serverName);
rmiRegistry.unbind(serverName);
UnicastRemoteObject.unexportObject(rmiRegistry, true);
} catch (NoSuchObjectException e)
{
e.printStackTrace();
} catch (NotBoundException e)
{
e.printStackTrace();
} catch (RemoteException ex) {
Logger.getLogger(HelloServer.class.getName()).log(Level.SEVERE, null, ex);
}
}
class StopServerTask extends TimerTask
{
#Override
public void run()
{
stopServer();
}
}
}
Whenever stopServer() in invoked exception is thrown at
UnicastRemoteObject.unexportObject(rmiRegistry, true);
Here is the stack Trace
java.rmi.NoSuchObjectException: object not exported
at sun.rmi.transport.ObjectTable.unexportObject(ObjectTable.java:153)
at java.rmi.server.UnicastRemoteObject.unexportObject(UnicastRemoteObject.java:297)
at rmi.HelloServer.stopServer(HelloServer.java:84)
Things are same even when I clean the service object by using
UnicastRemoteObject.unexportObject(myService, true);
Could someone suggest a clean way to stop the server which also releases the port for reuse.
You need to store the result of LocateRegistry.createRegistry(), and unexport that. At present you're trying to unexport a stub.
I implemented a shutdown-service in my rmi-server. If I want to shut it down, I call it with a password. Simple Example:
public interface ShutdownInterface extends Remote {
public void shutdownService(String password) throws RemoteException;
}
The serverside implementation can look something like:
public class ShutdownService extends UnicastRemoteObject implements ShutdownInterface {
private static final long serialVersionUID = 1L;
private boolean doShutdown = false;
public ShutdownService() throws RemoteException {
super();
}
#Override
public void shutdownService(String password) throws RemoteException {
if ("abcde12345".equals(password)) {
System.out.println("shutdown requested.");
this.doShutdown = true;
} else {
System.out.println("wrong pwd for shutdown");
}
}
public boolean isDoShutdown() {
return this.doShutdown;
}
}
Now the server itself keeps a reference to this:
public class BackendServer {
public final static int RMI_PORT = 1974;
private Registry registry = null;
private ShutdownService shutdownService = null;
public BackendServer() throws RemoteException {
registry = LocateRegistry.createRegistry(RMI_PORT);
this.shutdownService = new ShutdownService();
}
public void initialize() throws AccessException, RemoteException, AlreadyBoundException {
shutdownService = new ShutdownService();
registry.bind("ShutdownService", shutdownService);
registry.bind("MyDataService", new MyDataService());
}
public void stop() throws NoSuchObjectException {
System.out.println("stopping rmi server.");
UnicastRemoteObject.unexportObject(registry, true);
System.exit(0);
}
public boolean shouldStop() {
return this.shutdownService.isDoShutdown();
}
public static void main(String args[]) {
try {
BackendServer bs = new BackendServer();
bs.initialize();
System.out.println("Server ready.");
while (!bs.shouldStop()) {
Thread.sleep(1000);
}
bs.stop();
} catch (Exception e) {
System.err.println("Server exception: " + e.toString());
e.printStackTrace();
}
}
}
Of course, this can be realized in a more beautiful way, but this should give you an idea of how to easily implement a shutdown yourself. You can call it from the main client or from a small commandline-tool you code for your server.
I'm trying to listen for connections on two different ports
I start 2 Thread in a java main method,every Thread bind a port with netty4,but can't
listener success!
this is my code,the port 3333 is ok,but 1234 is not ok,it looks like 3333 is blocking!
public class ObjectServer
{
private static final Logger logger = LoggerFactory.getLogger(ObjectServer.class);
private String ip;
private int port;
public ObjectServer(int port)
{
this.port = port;
}
public void run(final ChannelInboundHandlerAdapter handler) throws Exception
{
EventLoopGroup bossGroup = new NioEventLoopGroup();
EventLoopGroup workerGroup = new NioEventLoopGroup();
try
{
ServerBootstrap server = new ServerBootstrap();
server.group(bossGroup, workerGroup).channel(NioServerSocketChannel.class).childHandler(new ChannelInitializer<SocketChannel>()
{
#Override
public void initChannel(SocketChannel ch) throws Exception
{
ch.pipeline().addLast(new ObjectEncoder(), new ObjectDecoder(ClassResolvers.cacheDisabled(null)), handler);
}
});
server.bind(port).sync().channel().closeFuture().sync();
}
catch (Exception e)
{
logger.error("开启监听失败！端口[" + port + "]", e);
throw e;
}
finally
{
bossGroup.shutdownGracefully();
workerGroup.shutdownGracefully();
}
}
}
public class SocketServer
{
private static final Logger logger = LoggerFactory.getLogger(SocketServer.class);
private static final StringDecoder DECODER = new StringDecoder();
private static final StringEncoder ENCODER = new StringEncoder();
private int port;
public SocketServer(int port)
{
this.port = port;
}
public void run(final ChannelInboundHandlerAdapter handler) throws Exception
{
EventLoopGroup bossGroup = new NioEventLoopGroup();
EventLoopGroup workerGroup = new NioEventLoopGroup();
try
{
ServerBootstrap b = new ServerBootstrap();
b.group(bossGroup, workerGroup).channel(NioServerSocketChannel.class).childHandler(new ChannelInitializer<SocketChannel>()
{
#Override
public void initChannel(SocketChannel ch) throws Exception
{
ChannelPipeline pipeline = ch.pipeline();
// Add the text line codec combination first,
pipeline.addLast("framer", new DelimiterBasedFrameDecoder(8192, Delimiters.lineDelimiter()));
// the encoder and decoder are static as these are
// sharable
pipeline.addLast("encoder", ENCODER);
pipeline.addLast("decoder", DECODER);
// and then business logic.
pipeline.addLast("handler", handler);
}
});
b.bind(port).sync().channel().closeFuture().sync();
}
catch (Exception e)
{
logger.error("开启监听失败！端口[" + port + "]", e);
throw e;
}
finally
{
bossGroup.shutdownGracefully();
workerGroup.shutdownGracefully();
}
}
}
public class Test
{
public static void main(String[] args) throws Exception
{
Thread1 thread1 = new Thread1();
Thread2 thread2 = new Thread2();
thread2.start();
thread1.start();
new SocketClient("192.168.16.52", 3333).run(new TestHandler4("test4"));
new ObjectClient("192.168.16.52", 1234).run(new TestHandler3("test3"));
}
#Sharable
static class TestHandler1 extends ChannelInboundHandlerAdapter
{
#Override
public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception
{
System.out.println("1234" + msg);
}
}
static class Thread1 extends Thread
{
#Override
public void run()
{
try
{
new ObjectServer(1234).run(new TestHandler1());
}
catch (Exception e)
{
e.printStackTrace();
}
}
}
static class Thread2 extends Thread
{
#Override
public void run()
{
try
{
new SocketServer(3333).run(new TestHandler2());
}
catch (Exception e)
{
e.printStackTrace();
}
}
}
#Sharable
static class TestHandler2 extends SimpleChannelInboundHandler<String>
{
#Override
public void channelRead0(ChannelHandlerContext ctx, String msg) throws Exception
{
System.out.println("3333" + msg);
}
#Override
public void channelActive(ChannelHandlerContext ctx) throws Exception
{
System.out.println("sssssssssssssssss");
}
}
#Sharable
static class TestHandler3 extends ChannelInboundHandlerAdapter
{
private String msg;
public TestHandler3(String msg)
{
this.msg = msg;
}
#Override
public void channelActive(ChannelHandlerContext ctx) throws Exception
{
ctx.writeAndFlush(msg);
}
}
#Sharable
static class TestHandler4 extends SimpleChannelInboundHandler<String>
{
private String msg;
public TestHandler4(String msg)
{
this.msg = msg;
}
#Override
public void channelActive(ChannelHandlerContext ctx) throws Exception
{
ctx.writeAndFlush(msg);
}
#Override
protected void channelRead0(ChannelHandlerContext arg0, String arg1)throws Exception
{
}
}
}
In your run() implementation, you do this:
server.bind(port).sync().channel().closeFuture().sync();
.. which will block until the server socket is closed. Because you do not close the server socket, it will never return. Therefore, only the first server socket will be bound.
What you probably want is just bind and return rather than waiting for the server sockets closed.