I am building an Android application that requires me to subscribe to multiple channels. I am using Jedis 2.4.2 for connecting to a Redis server for the same. I am using a separate client for every channel. The problem is that since Jedis subscription is not thread-safe, I am having trouble unsubscribing. As a workaround, I was thinking of a server-side program that keeps listening for messages from different clients on a dedicated channel and kills all subscriptions on receiving a request from them. For this, I need to identify the publisher of the request. Is there any way to accomplish this or a simpler way to execute the unsubscription task?
Related
I need to record from an unreliable web socket connection and stream into Kafka.
Our Kafka cluster is pretty reliable and we can make it highly available.
What is the best approach to make the web socket connection as reliable as possible? I would like to minimize data loss.
One solution would be to have multiple processes or web socket clients listening and streaming to multiple Kafka topics. Then do a filter with Kafka streams. This only works if each message that I obtain has a unique id, which is not always the case.
Another solution would be to monitor the web socket connection and restart or reset it. But then I might have data loss. Or rely on web socket heartbeats?
Or code my own error handlers?
What frameworks / libraries from the Java are in this space to do the best job? Currently, I use the web socket client from org.java-websocket.
I think this is a pretty standard use case in web socket and also in Kafka producer development, so I am sure I do not need to reinvent the wheel.
I'm using Paho to communicate with an MQTT broker and all the example I found (like this) do these 3 steps when performing an action (publish or subscribe):
connect to the broker
do action
disconnect
My question is: are there any drawbacks holding a connection for the whole life of the application instead of opening/closing it for each action? Isn't it a faster solution removing the time for opening the connection?
No, holding a connection open for the lifetime of the application is a fully expected usecase, it's the only real way you'd be able to subscribe to a topic and receive messages when they are published.
The protocol has built in ping messages to ensure the broker knows the client is still connected.
The examples tend to be relatively trivial but want to show the full life cycle of the client which is why they connect, do something, disconnect
I understand JMS as depicted by the following diagram:
(source: techhive.com)
Is there any way for me to access the underlying database using JMS or some other thing? Further, the JDBC connections that the JMS server maintains, can I add new connections in it so as to access other databases also and do CRUD operations on them? If yes, how?
Where did you get this from?
Normally JMS is used to send messages to queue (or topics). You have message producers that push messages in the queue and message consumers consume them and process it.
In your exemple it seems that you have multiple queues. One for the messages that need to be processed, and one for each client to retrieve the result the processing of its messages.
With JMS Server you don't necessarily have a database behind. Everything can stay in memory, or can be written to files. You will need database server behind only if you configure your JMS server to be persistent (and to assure that even if server/application crash your messages won't be lost). But in that case you will never have to interact with the database. Only the JMS server will and you will interact with the JMS server sending and consuming messages.
I am designing a Real Time backend chat application for mobile devices and to do this I am building everything over Java (to deal with incoming HTTP requests ) and Redis (Pub/Sub). Now I am looking for a Worker and already took a look at tools like Resque, Python-RQ and even Celery (also offers Redis integration), but maybe things grow and remain difficult to manage. I want to keep things as simple as possible. Has anyone tried to use Jedis (redis java client) to listening messages from a Redis channel and started a new Thread to each message received? Was the performace bad? what if a had hundreds of requests per second? It seems a poor solution (simple Thread as a Worker)
The flow is (for android example):
Android client send a message to chat
My Rest webservice (tomcat) receives the message and publish (jedis) a message to a Redis channel [quite simple]
The Worker (?) process the message and deliver it to all subscribers over Google Cloud Message (simple http request)
So, any suggestions or experiences about Redis workers implementations or Jedis library? What do you recommend? Thanks.
For those who wants a suggestion:
I opted for Python-RQ because of its simplicity. Too simple, well documented and solved my problem.
Regards.
I am looking to build an instant messenger in Java.
Clients will connect to the server to log in.
They will start a conversation with one or more other clients.
They will then post messages to the server that will relay the messages to all the clients.
The client needs to be continually updated when users post messages or log in.
so the way I see it, the client needs to run a server itself in a separate thread so that the main server can send stuff to it. Otherwise the client will have to the poll the main server every xyz seconds to get the latest updates. And that would need a separate thread anayway, as that would be purely for getting updates whereas the 'main' thread would be used for when the client initiates actions such as posting messages/inviting others to conversations etc...
So anyone recommendations on how to write this instant messenger? Does it sound like a good idea to make the connection a 'two-way' connection where both the client and server act as servers? Or is polling a better option? Anyone know how the IRC protocol does this?
There's no real advantage of having 2 connections unless they can be handled independently (for example receiving / sending a file usually done in a separate connection). A connection itself is already a two-way communication channel so it can be used to both send and receive messages, events etc. You don't need to poll server since client is able to maintain persistent connection and just wait for data to appear (optionally sending periodic PING-like message to ensure connection is alive).
IRC uses a single connection to server to exchange text commands. For example one of the main commands:
PRIVMSG <msgtarget> <message>
This command can be originated either by client or by server. Client sends PRIVMSG to notify that it wants to deliver message to one or more destination (in IRC this either user(s) or channel(s)). Server's task here is to properly broadcast this message to appropriate clients.
If you're using raw InputOutput streams then yes this is a good way of doing it. You create one thread on the clientside that acts in a similar fashion as the server thread - waits for any incoming updates and when it does it updates the client. I wouldn't call it a server though. So you'd ideally have 2 TCP/UDP connections one for requests made by the client and one to notify the client of server changes.
This solution in an enterprise environment would probably be done through some kind of messaging framework such as Spring Integration but dig deep enough and it will essentially be a similar way to how you mentioned.
Do you need a fully custom protocol or would it be sufficient to use the XMPP? There are several open source libraries implementing XMPP.
http://xmpp.org/xmpp-software/libraries/
e.g. http://www.igniterealtime.org/projects/smack/
For me, to develop instant messaging service, I will use websocket protocol instead of normal java socket because the normal socket can not work well with HTTP protocol and moreover some network providers and firewalls banned custom ports. If you develop it in normal socket, your service could not be accessed by web clients.
Did you plan to develop the instant messaging service yourself? How about using other protocols such as Jabber?