We Keep Coding

Apache kafka connect - custom websocket connector

The goal is :develop a custom Kafka connector that read ,messages from the websocket in a loop method. I try to give you an example on what I've realized:
I create an interface IWebsocketClientEndpoint
public interface IWebsocketClientEndpoint {
IWebsocketClientEndpoint Connect() ;
void Disconnect() throws IOException;
IWebsocketClientEndpoint addMessageHandler(IMessageHandler msgHandler);
void SendMessage(String message) throws Exception;
void SendMessage(ByteBuffer message) throws Exception;
void SendMessage(Object message) throws Exception;
boolean isOpen();
void Dispose()throws IOException;
}
and a class that implement above interface:
#ClientEndpoint
public class WebsocketClientEndpoint implements IWebsocketClientEndpoint {
private WebSocketContainer _container;
private Session _userSession = null;
private IMessageHandler _messageHandler;
private URI _endpointURI;
private WebsocketClientEndpoint(URI endpointURI) {
try {
_endpointURI = endpointURI;
_container = ContainerProvider.getWebSocketContainer();
} catch (Exception e) {
throw new RuntimeException(e);
}
}
private WebsocketClientEndpoint(URI endpointURI, int bufferSize) {
try {
_endpointURI = endpointURI;
_container = ContainerProvider.getWebSocketContainer();
_container.setDefaultMaxBinaryMessageBufferSize(bufferSize);
_container.setDefaultMaxTextMessageBufferSize(bufferSize);
} catch (Exception e) {
throw new RuntimeException(e);
}
}
public static IWebsocketClientEndpoint Create(URI endpointURI){
return new WebsocketClientEndpoint(endpointURI);
}
public static IWebsocketClientEndpoint Create(URI endpointURI,int bufferSize){
return new WebsocketClientEndpoint(endpointURI,bufferSize);
}
public IWebsocketClientEndpoint Connect() {
try {
_container.connectToServer(this, _endpointURI);
return this;
} catch (Exception e) {
throw new RuntimeException(e);
}
}
#OnOpen
public void onOpen(Session userSession) {
this._userSession = userSession;
if (this._messageHandler != null) {
this._messageHandler.handleOpen("Web socket "+ _endpointURI +" opened");}
}
#OnClose
public void onClose(Session userSession, CloseReason reason) {
this._userSession = null;
if (this._messageHandler != null) {
this._messageHandler.handleClose("Web socket "+ _endpointURI +" closed. Reason: " + reason.getReasonPhrase());}
}
public void Disconnect() throws IOException {
CloseReason reason = new CloseReason(CloseReason.CloseCodes.NORMAL_CLOSURE,"Web socket closed by user");
this._userSession.close(reason);
this._userSession = null;
//close notification to caller
if (this._messageHandler != null) {
this._messageHandler.handleClose("Web socket "+ _endpointURI +" closed. Reason: " + reason.getReasonPhrase());}
}
#Override
public IWebsocketClientEndpoint addMessageHandler(IMessageHandler msgHandler) {
this._messageHandler = msgHandler;
return this;
}
#OnMessage
public void onMessage(String message) {
if (this._messageHandler != null) {
this._messageHandler.handleMessage(message);
}
}
#OnMessage
public void onMessage(ByteBuffer bytes) {
if (this._messageHandler != null) {
this._messageHandler.handleMessage(bytes);
}
}
public void SendMessage(String message) throws Exception {
try{
this._userSession.getAsyncRemote().sendText(message);
}catch (Exception ex){
throw ex;
}
}
public void SendMessage(ByteBuffer message) throws Exception {
try{
this._userSession.getAsyncRemote().sendBinary(message);
}catch (Exception ex){
throw ex;
}
}
public void SendMessage(Object message) throws Exception {
this._userSession.getAsyncRemote().sendObject(message);
}catch (Exception ex){
throw ex;
}
}
#Override
public boolean isOpen() {
if (this._userSession != null){
return this._userSession.isOpen();
}
return false;
}
}
The class WebsocketClientEndpoint is dedicated to the creation of websocket and manage of connection, disconnection, send and receive message.
The goal is: how can I adapt the my websocket structure in the kafka connect structure? I could queue the message received ("public void handleMessage(String s)) from the socket in a ConcurrentLinkedQueue, and then, in the kafka connect loop method, unqueue them. But is it the best solution?
Below, the implementation of my Kafka custom connector
My kafka Connector
public class MySourceTask extends SourceTask {
IWebsocketClientEndpoint _clientEndPoint;
#Override
public void start(Map<String, String> props) {
_clientEndPoint = WebsocketClientEndpoint
.Create(new URI(socket))
.Connect();
_clientEndPoint.addMessageHandler(new IMessageHandler() {
#Override
public void handleMessage(String s) {
}
#Override
public void handleMessage(ByteBuffer byteBuffer) {
}
#Override
public void handleClose(String s) {
}
#Override
public void handleOpen(String s) {
}
});
}
#Override
public List<SourceRecord> poll() throws InterruptedException {
return null;
}
#Override
public void stop() {
_clientEndPoint.Dispose();
}
}
Thanks in advance to anyone

I'd suggest adding the interface to the class
extends SourceTask implements IMessageHandler
Then
_clientEndPoint.addMessageHandler(this);
And when you implement handleMessage, add those strings to some queue. Inside the poll method, you would pop data off that queue to create SourceRecord objects to return.
Inside of stop, call this.handleClose and clean up other resources.

SignalR java client can't invoke method and send data

I created a basic selfhosted SignalR server with the following code:
class Program
{
static void Main(string[] args)
{
// This will *ONLY* bind to localhost, if you want to bind to all addresses
// use http://*:8080 to bind to all addresses.
// See http://msdn.microsoft.com/en-us/library/system.net.httplistener.aspx
// for more information.
string url = "http://localhost:8080";
using (WebApp.Start(url))
{
Console.WriteLine("Server running on {0}", url);
Console.ReadLine();
}
}
}
class Startup
{
public void Configuration(IAppBuilder app)
{
app.UseCors(CorsOptions.AllowAll);
app.MapSignalR();
}
}
public class MyHub : Hub
{
public void Send(string name, string message)
{
Clients.All.addMessage(name, message);
}
}
Which is taken from: https://learn.microsoft.com/en-us/aspnet/signalr/overview/deployment/tutorial-signalr-self-host and works with the Javascript client.
I am now trying to create a Java client and got the following code that is simply supposed to send a message to the server:
String host = "http://localhost:8080";
HubConnection connection = new HubConnection(host);
HubProxy proxy = connection.createHubProxy("MyHub");
connection.start();
try {
System.out.println("Sendng message...");
proxy.invoke( "Send", "Client", "Hello world!" ).get();
System.out.println("Message sent!");
} catch (InterruptedException e) {
System.out.println("err1");
// Handle ...
} catch (ExecutionException e) {
System.out.println("err2");
// Handle ...
}
The problem that im having is that the message is not received by the server, it seems like the code is stuck at the invoke call and doesn't print the Hello world! message. Does someone know what im doing wrong?

hubProxy.invoke("sendMessageByUser", Message, WebApiToken).done(new Action<Void>() {
#Override
public void run(Void aVoid) {
if (aVoid != null)
handler.post(new Runnable() {
#Override
public void run() {
Toast.makeText(MyApplicationService.this, "Mesaj gönderildi", Toast.LENGTH_SHORT).show();
}
});
}
}).onError(new ErrorCallback() {
#Override
public void onError(final Throwable error) {
handler.post(new Runnable() {
#Override
public void run() {
Toast.makeText(MyApplicationService.this.getApplicationContext(), "Bir hata oluştu" + error.toString(), Toast.LENGTH_SHORT).show();
}
});
}
});

How to design a class that can pass data from server when data is finish downloading

I'm new on Android development and I'm learning how to use MVP pattern correctly in recently.
Now I'm facing a tricky problem, hope can get some helpful suggestion or solution from here.
First, here is my presenter
public class MVPPresenter {
private MVPView mvpView;
public MVPPresenter(MVPView mvpView) {
this.mvpView = mvpView;
}
public void loadData() {
mvpView.startLoading();
final List<MVPModel> list = new ArrayList<>();
//the part that I trying to extract starts here.
Call call = DataRetriever.getDataByGet(URLCombiner.GET_FRONT_PAGE_ITEMS);
call.enqueue(new Callback() {
#Override
public void onFailure(Call call, IOException e) {
mvpView.errorLoading();
}
#Override
public void onResponse(Call call, Response response) throws IOException {
if (response.isSuccessful()) {
try {
JSONObject result = new JSONObject(response.body().string());
int errorCode = result.getInt("ErrorCode");
if (errorCode == 0) {
JSONArray value = result.getJSONObject("Value").getJSONArray("hot");
for (int i = 0; i < value.length(); i++) {
MVPModel mvpModel = new MVPModel();
String name = null;
String image = null;
try {
name = value.getJSONObject(i).getString("title");
image = URLCombiner.IP + value.getJSONObject(i).getString("pic");
} catch (JSONException e) {
e.printStackTrace();
}
mvpModel.setName(name);
mvpModel.setImage(image);
list.add(mvpModel);
}
if (list.size() > 0) {
mvpView.successLoading(list);
mvpView.finishLoading();
} else {
mvpView.errorLoading();
}
} else {
mvpView.errorLoading();
}
} catch (JSONException e) {
e.printStackTrace();
}
} else {
mvpView.errorLoading();
}
}
});
//the part that I trying to extract ends here.
}
}
As you can see, I'm trying to extract the part which is using OKHttp libs into a class (I call it data manager) which I hope it can handle every task between server and client. But here's the thing, when I trying to pass the result from the data manager to presenter, I got NullPointException because of the async mechanism.
I would like to know how to passing the data, which is come from server in async, to the presenter when the data has finish downloading.
And here's my ideal data manager, I know this might looks stupid but I think this can make my problem more clearly.
public class LoadServerData {
private static JSONArray arrayData = new JSONArray();
public static JSONArray getServerData() {
Call call = DataRetriever.getDataByGet(URLCombiner.GET_FRONT_PAGE_ITEMS);
call.enqueue(new Callback() {
#Override
public void onFailure(Call call, IOException e) {
}
#Override
public void onResponse(Call call, Response response) throws IOException {
if (response.isSuccessful()) {
try {
JSONObject result = new JSONObject(response.body().string());
int errorCode = result.getInt("ErrorCode");
if (errorCode == 0) {
arrayData = result.getJSONObject("Value").getJSONArray("hot"); //the data I would like to return.
}
} catch (JSONException e) {
e.printStackTrace();
}
}
}
});
return arrayData; //this is gonna be an empty data.
}
}
I've reading some article that might can solve my problem, but still not getting any fine answer. Perhaps I've using wrong keyword I think. Hopes you guys can give me some ideas or solutions to help me or inspire me.
P.S. version of OKhttp libs is 3.7.0

Create a simple Listener so it can be called whenever the server call finishes:
public class LoadServerData {
public static interface Listener {
public void onSuccess(JSONArray data);
public void onError(Exception error);
}
public static void getServerData(Listener listener) {
Call call = DataRetriever.getDataByGet(URLCombiner.GET_FRONT_PAGE_ITEMS);
call.enqueue(new Callback() {
#Override
public void onFailure(Call call, IOException e) {
listener.onError(e);
}
#Override
public void onResponse(Call call, Response response) throws IOException {
if (response.isSuccessful()) {
try {
JSONObject result = new JSONObject(response.body().string());
int errorCode = result.getInt("ErrorCode");
if (errorCode == 0) {
JSONArray arrayData = result.getJSONObject("Value").getJSONArray("hot"); //the data I would like to return.
listener.onSuccess(arrayData);
} else {
listener.onError(...);
}
} catch (JSONException e) {
e.printStackTrace();
listener.onError(e);
}
} else {
listener.onError(...);
}
}
});
}
}

java cometD + selenium. Client communication not stable

My scheme: ajax long polling to Tomcat. Tomcat executes selenium "operations".
Im trying to execute selenium scenario from tomcat.
Its working ok but i need to get logs back to js client.
Javascript client partially receives messages from server when selenium working.
Inside some selenium operations im using Thread.sleep(). Maybe mistakes because of this ?
Please tell me why messages partially received (i think) by client
Here is ServerInfoLogger. BaseLogger outputs messages to console and file
public class ServerInfoLogger extends BaseLogger {
public ServerSession clientServerSession;
protected void logToClient(String message) {
super.log(message);
sendMessage(message);
}
// Server.serverSession and Server.clientServerSession not null but messages partially not received by client
private void sendMessage(String message) {
// send message to client
if (Server.serverSession!=null && Server.clientServerSession!=null) {
Server.clientServerSession.deliver(Server.serverSession, "/message", message);
} else {
System.err.println("Server error. Server.serverSession=" + Server.serverSession + " clientServerSession=" + clientServerSession);
}
}
}
Here is selenium scenario
public class Task extends ServerInfoLogger {
public static boolean datesAvailable = false;
private TaskParser parser = new TaskParser();
protected ArrayList<Step> steps = new ArrayList<>();
private WebDriver webDriver;
protected int currentStepIndex = 0;
protected Step currentStep;
private WebDriverFactory webDriverFactory = new WebDriverFactory();
private int currentTryout = 1;
private int maxTryouts = 40;
public Task(ServerSession clientServerSession, String taskData) {
this.clientServerSession = clientServerSession;
logToClient("new task"); // client always receives this message
steps = parser.parse(taskData);
logToClient("steps parsed. total: "+steps.size()); // client always receives this message
start();
}
protected void start() {
createWebDriver();
startStep();
}
protected void startStep() {
currentStep = steps.get(currentStepIndex);
currentStep.setWebDriver(webDriver);
currentStep.clientServerSession = clientServerSession;
boolean stepComplete = false;
try {
stepComplete = currentStep.start();
} catch (StepException e) {
logToClient(e.getMessage()+" step id: "+e.getStepId());
e.printStackTrace();
}
log("step complete = "+stepComplete);
if (stepComplete) {
onStepComplete();
} else {
onStepError();
}
}
private void onStepError() {
currentTryout++;
if (currentTryout > maxTryouts) {
destroyWebDriver();
logToClient("Max tryouts reached"); // client never receives this message
} else {
logToClient("Step not complete. Restarting task. currentTryout=" + currentTryout); // client partially receives this message
restart();
}
}
private void onStepComplete() {
currentStepIndex++;
if (currentStepIndex < steps.size()) {
startStep();
} else {
destroyWebDriver();
taskComplete();
}
}
private void destroyWebDriver() {
webDriver.quit();
webDriver = null;
}
private void taskComplete() {
logToClient("Task complete !!!"); // client **never** receives this message
TaskEvent taskEvent = new TaskEvent(TaskEvent.TASK_COMPLETE);
EventDispatcher.getInstance().dispatchEvent(taskEvent);
}
public void restart() {
logToClient("Task restart");
try {
setTimeout(Application.baseOperationWaitSecondsUntil);
new SwitchToDefaultContentOperation().execute();
} catch(OperationException exception) {
logToClient("Cannot get default content"); // client partially receives this message
}
currentStepIndex = 0;
startStep();
}
private void setTimeout(int seconds) {
webDriver.manage().timeouts().implicitlyWait(seconds, TimeUnit.SECONDS);
}
private void createWebDriver() {
webDriver = webDriverFactory.getDriver(BrowserType.CHROME).getDriver();
}
public int getCurrentStepIndex() {
return currentStepIndex;
}
}
Here is creation of logger
private void createLogger() {
Date currentDate = new Date();
String logFilePostfix = currentDate.getMonth()+"_"+currentDate.getDate()+"-"+currentDate.getHours()+"_"+currentDate.getMinutes()+"_"+currentDate.getSeconds();
logger = Logger.getLogger(appName);
logger.setUseParentHandlers(false);
FileHandler fh;
SimplestFormatter formatter = new SimplestFormatter();
try {
fh = new FileHandler("C:\\consultant\\logs\\log_"+logFilePostfix+".txt");
logger.addHandler(fh);
fh.setFormatter(formatter);
} catch (SecurityException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
Selenium operation with Thread.sleep()
public class SwitchToMainFrameOperation extends BaseOperation {
private WebElement mainIframe;
private WebDriverWait wait;
#Override
public boolean execute() throws OperationException {
switchToRoot();
sleepThread();
log("switchToMainIFrame by xPath "+Page.getMainIframeXPath()); // log to console and file
wait = new WebDriverWait(webDriver, Application.baseOperationWaitSecondsUntil);
try {
mainIframe = wait.until(ExpectedConditions.presenceOfElementLocated(By.xpath(Page.getMainIframeXPath())));
webDriver.switchTo().frame(mainIframe);
log("main frame switch OK"); // log to console and file
} catch(StaleElementReferenceException exception) {
log("main frame switch error. StaleElementReferenceException - trying again"); // log to console and file
wait = null;
sleepThread();
execute();
}
return true;
}
private void switchToRoot() {
log("switch to root");
webDriver.switchTo().defaultContent();
}
private void sleepThread() {
try {
Thread.sleep(500);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}

Getting error while implement OutboundMessageListener in blackberry?

Getting error while using OutboundMessageListener and MessageListener by using this code:
public class MainClass extends UiApplication implements OutboundMessageListener,MessageListener
{
public static void main(String[] args)
{
MainClass mainClass = new MainClass();
mainClass.enterEventDispatcher();
}
public MainClass()
{
try
{
MessageConnection _mc = (MessageConnection)Connector.open("sms://");
_mc.setMessageListener(this);
}
catch (IOException e)
{
}
UiApplication.getUiApplication().pushScreen(new SmsCountScreen());
}
public void notifyIncomingMessage(MessageConnection conn)
{
UiApplication.getUiApplication().invokeAndWait(new Runnable()
{
public void run()
{
Dialog dialog = new Dialog(Dialog.D_OK, "Message Received!", 0, null, Dialog.FIELD_HCENTER);
Ui.getUiEngine().pushGlobalScreen(dialog, 1, UiEngine.GLOBAL_MODAL);
}
});
}
public void notifyOutgoingMessage(Message message)
{
UiApplication.getUiApplication().invokeAndWait(new Runnable()
{
public void run()
{
Dialog dialog = new Dialog(Dialog.D_OK, "Message Sent!", 0, null, Dialog.FIELD_HCENTER);
Ui.getUiEngine().pushGlobalScreen(dialog, 1, UiEngine.GLOBAL_MODAL);
}
});
}
}
using this code and getting error
IOException: operation not permitted on a client connection
Please help to solve this?

Looking at this example on the BlackBerry support forums, they use this code:
public class MyMessageListener implements OutboundMessageListener
{
public void notifyOutgoingMessage(javax.wireless.messaging.Message m)
{
try {
String msg = null;
msg = getMessage(m); // my call to convert Message to String
//... process msg
}
catch(Exception ex) {
// handle exception
}
}
public void notifyIncomingMessage(MessageConnection conn)
{
// handle received sms here
}
}
to register the listener
MyMessageListener ml = new MyMessageListener();
MessageConnection mc;
try {
mc = (MessageConnection)Connector.open("sms://:0");
mc.setMessageListener(el);
} catch (Exception e) {
// handle exception
}
Note that the port is specified in the Connection.open() URL. I'd also recommend testing this on a real device, not the simulators.