I'm developing an Android app that requires 2 (or more) devices to communicate with each other.
I tried using Google Cloud Messaging but I was disappointed to find out that the GCM max capacity is 100 messages, so it is broken and does not fit my requirements.
I was thinking about java sockets. Every device will open a new socket (or keep its socket open) and communicate with a group of sockets (devices).
In order to communicate this way I need a server-side app that can send messages to the client (android device). So I figured out that HTTP or web-service won't help me. Am I right?
What is the best way for me to implement such a server-side app?
You can refer to this question I previously asked and implemented. It was for implementing my own Notification mechanism but it equally (or even more) applies to chatting applications since message queues perfectly fit that usecase.
Building an Android notification server
I ended up not doing it and using GCM at the end but I did have a fully working solution using ActiveMQ and Paho. You can research them both and understand their inner workings. It's easy in principle and definitely possible but the problem is, you may not be able to do this for iOS or WP as it requires running a service in the background (in case your app is not open and you want to make sure the messages are at least sent in a notification).
The possible solution to that problem would be to use both the notification service (GCM or equivalent) for background notifications and then using your MQ for actual communication but I decided that was too much for my project.
If you look at Paho, it will have a fully working MQTT solution that will work even if the phone is not "online" (sleeping or otherwise) and there are plenty of samples for ActiveMQ and drivers for multiple programming languages.
I think this solution is much better than having open sockets between two apps, at least because they allow you to persist messages and guarantee delivery which is an important aspect for a chatting application.
As it is said by kha, choose one of the message queue protocols is the best solution. 3 reasons in brief,
Delivery guaranteed regardless of temporary offline or long latency.
As simple as subscribe / publish, no worry about transport layer any more.
Broker available online. You save time and money for setting up your own.
For mobile devices like in your case, I'd prioritize MQTT too. It's lightweight and stable. If you are totally new to message queue or MQTT, refer to this documentaion and example code
I am interested in putting a chatting functionality as part of an application.
What I am interested in:
I would like to keep my application instances acting as peers, i.e.
I would prefer not to write also some server module to handle
message communications
I would like it to be able to use it with exising IM accounts. E.g.
someone can use it using his MSN account or any other client
account he may have, same way he would use MSN Messenger or Tor client etc
I googled and found that there are some Java MSN libraries available and also some other libraries that support IM e.g. SMACK for JABBER etc (not sure what that is) but I am not sure if the latter could be used for option 2 I mention.
My preference on 2 is because I assume that this way a user could do chat no matter where he is while in other solution I assume that some network infrastructure e.g. with routable IPs etc would be required. Am I wrong here?
Does anyone have expererience with Java IM libraries? Are for example any issues e.g. with different MSN versions or something?(Don't know if the protocol has been changing often to matter for me).
What would be the best path/option for my requirements?
I would go with a Jabber based approach. Jabber (also called XMPP) is an open protocol with lots of implementations and supports connecting to other IM services via transports. That way you would not have to deal with changes to the Windows Live protocols. You can rely on the open source community to provide that functionality for you.
Edit: It seems, that Windows Live even allows native XMPP access.
If you like, you can always set up your own Jabber server to provide a tighter integration with existing user accounts. In that case you wouldn't need to write the whole server.
I am currently trying to transfer a file from a Android device to a Java TCP Server, but I am unable to find a good example which explains the structure I would need to implement this. There are many Java Client&Server examples there which explain file transfer but I want to make sure if this will still work once one throws an Android Device in there.
My question is how do I implement this sort of structure? And if it doesn't work, would I be better sending the file over an HTTP connection to a PHP server? I see a lot of examples and documentation online for the later method so I presume it is more reliable. I would however prefer to use a Java server.
The file consists of a large set of coordinates recorded by the Android device which will then be sent to the server. I have not yet established how I will store this data yet but I was originally going to store them in a primitive text file.
Design
The first thing you need is something to allow you to run Java code on your server.
There are a number of options. Two of the most popular technologies are Glassfish and Apache Tomcat.
Crudely speaking Apache Tomcat is sufficient for simple client-server communication and Glassfish is used if you need to do more complex stuff. Both allow Servlets (which are essentially self contained server classes written in Java) to run on the server-side.
They handle communication with the client by launching a JVM (Java Virtual Machine) each time they receive a request. The Java servlet can run inside the JVM and respond do some processing if required before sending a response back to the client.Each new request is run in a new instance of a servlet. This makes dealing with multiple concurrent requests simpler (no need for more complex threading).
Networking (sending data to and from the server)
In networking situations the client can be a PC, an Android phone, or any other device capable of connecting to the internet. As far as the server is concerned, if the client can communicate using HTTP (a standard protocol which it understands) the it doesn't care what sort of device it is. This means that solutions for PC desktop client-server applications are similar to one for a phone.
You can use library such as Apache HTTP Components to make it easier to handle HTTP requests and responses between the device and the server. Of course you could write your own classes to do this using Sockets but this would be very time consuming, particularly if you have never done it before.
Storage of Data
If you have time I would recommend implementing some sort of database to store the information.
They have a number of benefits to such as data recovery mechanisms, indexing for fast searching of data, ensure data integrity, better structuring of data and so on.
If you decide to use a database I recommend MySQL. It is a free and more importantly - well documented.
Aside: JDBC can be used to communicate with the database with Java.
Sorry about the in-line hyperlinks - apparently my repuation isn't high enough to post more than two!
Source: Personal experience from implementing a similar design.
Wikipedia defines XMPP as:
...an open-standard communications protocol for message-oriented middleware based on XML.
xmpp.org defines XMPP as:
The Extensible Messaging and Presence Protocol (XMPP) is an open XML technology for real-time communication, which powers a wide range of applications.
Although I'm sure both these definitions are very accurate, they don't tell me a thing about what I - a Java developer - can actually do with XMPP!
For instance, I've heard XMPP can be used with message-oriented middleare (MOM). How so? Can XMPP somehow integrate with my Apache Camel routes, my ESB or some SOA implementation to deliver a better/faster/more robust business tier? If so, how?!?!
A good, King's-English explanation of XMPP, along with some practical examples (preferable MOM-centric) would be greatly appreciated. Thanks in advance!
XMPP can be used for a wide range of messaging based applications. Basically, it provides core services which can be used to build XML based messaging applications. Its based on a decentralized client-server architecture and utilizes long-lived TCP connections for communicating...
core services include...
channel encryption, authentication, presence, contact lists, one-to-one messaging, multi-party messaging, notifications
service discovery, capabilities advertisement, structured data formats, workflow management, peer-to-peer media sessions
textbook use cases...
instant messaging (using presence, contact lists, one-to-one messaging)
group chat, gaming, systems control, geolocation, middleware/cloud computing, data syndication
bots (weather, database interface, system monitoring)
messaging modes/patterns...
point-to-point messaging is used to send to a specific receiver
multi-user messaging is used to message to a group of receivers
publish/subscribe support is used when there are large volume of events and systems are interested in differing subsets of events. Publishers put events into topics and subscribers indicate which topics they are interested in. This decouples the publisher/subscriber and allows for scalable real-time messaging. For more information, see this article: http://www.isode.com/whitepapers/xmpp-pubsub.html
deployment methods...
XMPP user - connects as a normal user and responds to requests addressed to the user
XMPP Server plugins - deployed as part of the server plugin architecture
XMPP Components - service external to an XMPP server that connects and behaves like a plugin
Java Integration
Smack API - A pure Java library, it can be embedded into your applications to create anything from a full XMPP client to simple XMPP integrations such as sending notification messages and presence-enabling devices.
Camel XMPP - A Camel component that allows integration with Smack API in Camel routes
To your specific question "can it be used in SOA/middleware?"....
yes, it can be used to wire together applications via XML messaging and XMPP APIs
whether its the best technology choice depends heavily upon requirements
one good use case, interactive system monitoring/management...here are some other examples
Also, XMPP integration with Camel is trivial. See this camel-xmpp unit test for a basic example of interfacing with a Google Talk server. Also, Camel's framework allows you to build an application and easily swap out different messaging technologies (JMS, STOMP, mina, etc).
I can start combining information from al kinds of sources found on the internet using Google (keywords: XMPP Java MoM examples), rewrite (or even copy) the definition of XMPP, but of course I will not do so. There is just too much. I also do not have examples available for you.
Below I will list the links I found most interesting, so you can start reading and get more knowledge on the subject.
1) http://www.xmpp.org/
This is probably the best starting point. Browse through the menu left to right and top to bottom. That is what I did. The site lists servers, clients and libraries, so you should be able to find the desired examples this way.
2) http://www.ibm.com/developerworks/webservices/library/x-xmppintro/index.html
Clear article, which also mentions MoM. No Java examples, but Ruby.
3) http://fyi.oreilly.com/2009/05/what-can-you-do-with-xmpp.html
Maybe you should just get the book? No examples in the article.
4) http://kirkwylie.blogspot.com/2008/07/real-mom-is-hard-lets-use-xmpp.html
Interesting article where the last line basically says: Why use XMPP if you can use AMQP or JMS?
I hope this helps you I finding what you need.
XMPP is an open and extensible standard for real time communications.
XMPP comes with a core that is defined in its rfc, which describes the basic protocol for doing instant messaging and exchanging presence information. However where XMPP really shines is in its extensibility: XMPP defines the building blocks (presence, message and iq stanzas) to create protocols of communication. These typically come as extensions. A list of currently available standard extensions can be found here. The most important of these are typically available for all the popular XMPP servers.
It is exactly this extensibility that makes XMPP appropriate as message-oriented middleware.
Let me take as an example Publish-Subscribe which is a typical pattern for middleware and becomes a necessity as soon as you depart from the scenario with a few entities where simple messaging is adequate. PubSub is used in situations where entities, or producers, produce information that is to be consumed by other entities, the consumers. Typically, the information is written to nodes, to which consumers are subscribed. Being subscribed they receive notifications when an item is added/updated/deleted. An incredible amount of use-cases can be elegantly covered by PubSub, from queuing long-running jobs and having workers handle them, to micro-blogging. XMPP has a very robust and widely available extension to handle PubSub in a standard way, described in XEP-0060 and providing out of the box a workflow for handling publishing, subscriptions, notifications and security. Having a look at the use-cases in the XEP will give you an idea for the simplicity of the whole thing.
Now, while most use-cases are covered by using (or abusing) existing standard extensions, eventually you might need the little extra custom protocol that is not covered elsewhere. Using your language of choice you can write an XMPP component defining your own protocol. You then connect the component to the XMPP server you are running and by using simple namespacing let the server know what kind of messages you can handle and let the server advertise your protocol capabilities to clients connecting to it. There is no end to how simple or complex you can make this. Lack of better example but maybe good enough for illustration, here is a component I wrote that leverages XMPP to do real-time collaborative editing in the Plone CMS (similar to Google docs). While the details can get complicated I think having a look at the "Protocol Specication" on that page will give you an idea.
Finally, concerning Java specific libraries as #boday mentions, there are libraries around that make it easy to start with as well as Apache Camel integration (although it only does simple messaging as far as I can see). Keep in mind though that the investment in understanding how XMPP works and being able to go beyond using existing libraries is really worth it and can lead to extremely powerful and yet simple integration.
Let me just give you a good overview of what XMPP is ?
XMPP ( Extensible Messaging and Presence Protocol )
It is a Real time communication protocol.
The first IM service based on XMPP was Jabber.org
One XMPP binding is BOSH. Others include TCP and WebSocket.
BOSH is "Bidirectional-streams Over Synchronous HTTP", a technology for two-way communication over the Hypertext Transfer Protocol (HTTP).
BOSH emulates many of the transport primitives that are familiar from the Transmission Control Protocol (TCP). For applications that
require both "push" and "pull" communications, BOSH is significantly
more bandwidth-efficient and responsive than most other bidirectional
HTTP-based transport protocols and the techniques known as AJAX.
BOSH achieves this efficiency and low latency by long polling.
How does XMPP works?
XMPP achieves low latency by implementing Long polling.
What is Normal Polling and Long Polling ?
NORMAL POLLING:
Consider the old chat apps that used normal polling. Here the top of the graph indicates client side. Bottom indicates server side. Let the timeout be 5 min.
Client asks the server : " Is there any new message for me ?"
Server Responds : "No ! "
After 5 min
Client asks the server : " Is there any new message for me ?"
Server Responds : "No !"
After 1 min
Client Receives a message
After 4 min
Client asks the server : " Is there any new message for me ?"
Server Responds : "Yes! " + message.
LONG POLLING
The top part of the graph is client . Bottom part is server.
Client asks the server : " Is there any new message for me ?"
Server Answers : "No ! But you may soon receive a new message, so let me just hold your client state for 5 min in the server."
After 5 min
Server Responds : "No !"
Client asks the server : " Is there any new message for me ?"
Server Answers : "No ! But you may soon receive a new message, so let me just hold your client state for 5 min in the server."
After 1 min
Client Receives a message
Server Responds : "Yes! " + message.
You can see clearly, how instantaneously the communication happens.
You can read more about XMPP HERE
In case, if you are curious to set up your own XMPP server, read this.
XMPP is fundamentally a protocol for chat room clients to talk to a chat server. Camel does allow you to integrate with XMPP so that you can consume messages from or produce to such a mechanism http://camel.apache.org/xmpp.html
When you start talking business tier, ESB, SOA etc. and MOMs you're probably looking for a messaging mechanism that supports point to point and publish subscribe messaging. You're probably also thinking about things such as guaranteed messaging, high availability, and fine-grained security. You won't get these from a mechanism that reads and writes text messages into a chat server. A messaging platform such as ActiveMQ is likely to be a far better fit.
There are very few instances where I would contemplate using XMPP with integration mechanisms, maybe as a notification mechanism to a dev chat room, or as a duct-tape mechanism for controlling servers that monitor a chat server for instructions.
I'm currently working on a project that is building a java-based desktop application to interface with a website. We want to incorporate IM capabilities, so we decided to use XMPP.
The problem is our application has other features, and anyone using another client to connect to our XMPP server will cause problems with our website (e.g. our client will be able to send our messages with a certain message type that the user won't be able to use, but with another client they could send those message types).
Is there anyone to either allow only our client to access the XMPP server or prevent other clients from using certain features? I know this is against the idea of open standards, but we don't want to build a proprietary IM solution from scratch.
You are building a proprietary solution, it just might not be completely from scratch, and that's not necessarily a bad thing. But please don't call it a XMPP service unless you are going to support XMPP clients. You will get the same reaction as you get with a "web site" that requires your proprietary browser.
For features that can be negotiated, look at Feature Negotiation and you might be able to get away with saying your server doesn't have a specific feature to other clients, but secretly supporting it in your own. That won't actually block something from being attempted, so it's pretty poor solution.
You can get instant messaging capabilities without building a desktop application (with all of the platform support headaches that entails). Consider Orbited which can give you the instant messaging interactivity your looking for and would make it much easier to integrate on the server.
And just as a side point, there shouldn't be any messages that cause problems with your web site, any more than there could be a URL that causes it problems, or a query combination, etc.
Not sure of all your requirements, but it sounds like you could probably use the pubsub or pep features of XMPP. These are extensions to XMPP that allow you to create specialized payloads that can be accessed on a user to user level (Personal Eventing Protocol XEP-0163). If it is simply a general notification to everyone who is interested, then PubSub (XEP-0060) may be what you want.
These protocols allow for securing access to the pubsub nodes and will not get affected by the standard chat messages, as they are a different protocol.