I am designing a Real Time backend chat application for mobile devices and to do this I am building everything over Java (to deal with incoming HTTP requests ) and Redis (Pub/Sub). Now I am looking for a Worker and already took a look at tools like Resque, Python-RQ and even Celery (also offers Redis integration), but maybe things grow and remain difficult to manage. I want to keep things as simple as possible. Has anyone tried to use Jedis (redis java client) to listening messages from a Redis channel and started a new Thread to each message received? Was the performace bad? what if a had hundreds of requests per second? It seems a poor solution (simple Thread as a Worker)
The flow is (for android example):
Android client send a message to chat
My Rest webservice (tomcat) receives the message and publish (jedis) a message to a Redis channel [quite simple]
The Worker (?) process the message and deliver it to all subscribers over Google Cloud Message (simple http request)
So, any suggestions or experiences about Redis workers implementations or Jedis library? What do you recommend? Thanks.
For those who wants a suggestion:
I opted for Python-RQ because of its simplicity. Too simple, well documented and solved my problem.
Regards.
Related
I want to run a Java based message broker that will route messages to web clients. Web client connections are handled on our server using our custom Java websocket code, which authenticates users against the user database.
I think my server side websocket handler code would connect to ActiveMQ and perform subscription management via AQMP.
I have a specific requirement however:
route messages for a topic specifically to one or more web clients
Note that I don't need to retain messages if a client is not connected. Messages are being used to inform the web client applications of actions they need to take.
I'm considering ActiveMQ but I was hoping people with experience of the product could clarify if it supports this requirement?
If ActiveMQ isn't the best option, could you recommend something else?
Thanks
Yes, ActiveMQ is a great choice for this.
As far as specific approach goes, it depends on your data model and message flow.
You have several options, including:
Produce and consume to a topic-per-client
a. Messages for Client ABC go to topic://CLIENTS.ABC, for Client XYZ go to topic://CLIENTS.XYZ, and the subscribers connect accordingly.
Produce a message with a header and use a consumer-side selector (aka 'filters' in AMQP) to filter messages on a per-client basis. (abc client subscribes to-- ClientId = ABC, xyz client subscribe to-- ClientId = XYZ)
When using WebSockets, you might also look to STOMP which is text-based protocol. (Just depends on your programming language and available libraries that you had in mind)
I am building an Android application that requires me to subscribe to multiple channels. I am using Jedis 2.4.2 for connecting to a Redis server for the same. I am using a separate client for every channel. The problem is that since Jedis subscription is not thread-safe, I am having trouble unsubscribing. As a workaround, I was thinking of a server-side program that keeps listening for messages from different clients on a dedicated channel and kills all subscriptions on receiving a request from them. For this, I need to identify the publisher of the request. Is there any way to accomplish this or a simpler way to execute the unsubscription task?
I need to develop a platform in Java to download tweets from Twitter (that was obvious). The idea is to have various computers downloading from the streaming API and a main controller to send tasks (keywords to download and other data) to each fetcher. My problem is related with the connection between this programs. Which is the best way to do this? Actually I'm using RMI to send commands like "stop", "start", "setTask" from the Controller (client) to each fetcher (servers) and a SSLSocket to make a quick validation, but I'm not sure if this is a good idea. I could use TCP sockets but maybe it's not a good idea to have permanent connections. What do you think? Is it a good idea to keep using RMI or should I take another point of view?
Thank you ;)
I propose you to use queue (and any queue protocol).
ActiveMQ, RabbitMQ, QPID, or one of many other tool.
I use ActiveMQ in prod and fine with it, but for very highroad RabbitMQ will be better.
you receive easy scaling for any count of workers and easiest way to share/split tasks between workers.
Also please look on ActiveMQ or RabbitMQ or ZeroMQ or
I am looking to build an instant messenger in Java.
Clients will connect to the server to log in.
They will start a conversation with one or more other clients.
They will then post messages to the server that will relay the messages to all the clients.
The client needs to be continually updated when users post messages or log in.
so the way I see it, the client needs to run a server itself in a separate thread so that the main server can send stuff to it. Otherwise the client will have to the poll the main server every xyz seconds to get the latest updates. And that would need a separate thread anayway, as that would be purely for getting updates whereas the 'main' thread would be used for when the client initiates actions such as posting messages/inviting others to conversations etc...
So anyone recommendations on how to write this instant messenger? Does it sound like a good idea to make the connection a 'two-way' connection where both the client and server act as servers? Or is polling a better option? Anyone know how the IRC protocol does this?
There's no real advantage of having 2 connections unless they can be handled independently (for example receiving / sending a file usually done in a separate connection). A connection itself is already a two-way communication channel so it can be used to both send and receive messages, events etc. You don't need to poll server since client is able to maintain persistent connection and just wait for data to appear (optionally sending periodic PING-like message to ensure connection is alive).
IRC uses a single connection to server to exchange text commands. For example one of the main commands:
PRIVMSG <msgtarget> <message>
This command can be originated either by client or by server. Client sends PRIVMSG to notify that it wants to deliver message to one or more destination (in IRC this either user(s) or channel(s)). Server's task here is to properly broadcast this message to appropriate clients.
If you're using raw InputOutput streams then yes this is a good way of doing it. You create one thread on the clientside that acts in a similar fashion as the server thread - waits for any incoming updates and when it does it updates the client. I wouldn't call it a server though. So you'd ideally have 2 TCP/UDP connections one for requests made by the client and one to notify the client of server changes.
This solution in an enterprise environment would probably be done through some kind of messaging framework such as Spring Integration but dig deep enough and it will essentially be a similar way to how you mentioned.
Do you need a fully custom protocol or would it be sufficient to use the XMPP? There are several open source libraries implementing XMPP.
http://xmpp.org/xmpp-software/libraries/
e.g. http://www.igniterealtime.org/projects/smack/
For me, to develop instant messaging service, I will use websocket protocol instead of normal java socket because the normal socket can not work well with HTTP protocol and moreover some network providers and firewalls banned custom ports. If you develop it in normal socket, your service could not be accessed by web clients.
Did you plan to develop the instant messaging service yourself? How about using other protocols such as Jabber?
I'm writing a multiplayer/multiroom game (Hearts) in java, using RMI and a centralized server.
But there's a problem: RMI Callbacks will not work beacause clients are Natted and Firewalled. I basically need the server to push data updates to clients, possibly without using polling and without using sockets (I would code at an higher level)
In your opinion, what's the best solution for realizing this kind of architecture? Is an ajax application the only solution?
You say that you don't want polling, but AJAX is exactly that. You can look at Comet but it's hard to escape polling anyway (e.g. Comet itself uses polling underneath).
You could use a peer to peer framework such as JXTA.
I can suggest two main techniques.
The server has a method getUpdates, callable by clients. The method returns the control to the client when there is an update to show.
When Clients perform the registration, they give the server a callback remote object
Since this object is not registered in any rmi registry, there should no be any issue with natted clients.
I'm not sure how(if) ajax works for a non-browser-based app. You could just maintain your own pool of SocketConnections open for the duration of the application with a Thread per connection.
If you need to scale to a lot of concurrent connections, look to a non-blocking I/O framework like Apache Mina or Netty (related SO post: Netty vs Apache MINA).