Does anybody know how to receive (how to know that you received) out-of-band data with Java sockets? In particular I've read the documentation for sendUrgentData and setOOBInline which states that:
Note, only limited support is provided for handling incoming urgent data. In particular, no notification of incoming urgent data is provided and there is no capability to distinguish between normal data and urgent data unless provided by a higher level protocol.
But I'm still hoping that somebody has a solution :-)
More details:
I'm sending serialized objects trough the socket and I'm not really seeing how I can find a "random" OOB byte inserted in the stream (other than the fact that Java will give me an exception when I'll try to deserialize the object).
Tomcat has a maintained implementation of JNI socket which can read urgent packets in a OOB manner.
It looks like Java, up to Java 6 (I didn't investigate Java 7 since I don't know if they've decided for sure what will go in or not) and it looks like Java does not provide any support at all for reading urgent TCP information in an out-of-band manner. And it looks like this is a deliberate choice, thus far.
However, I did find one user who implemented his own solution, using JNI, to successfully read urgent data out of stream. This hack is very old and may or may not work on recent JDK's. I have not tried this myself! Your milage may vary. :) But hopefully this will help you.
Related
I have a project that requires me to create a Java server which will be connected to a MySQL database. This server is going to handle requests from clients, to send them data from the database.
The request from the clients will be:
check if a User is registered in the database
add User to the database
get a list of Users and which of them are on-line(this is where I use the HashMap)
After some searching I've concluded in using NIO, so I won't get too many threads to handle multiple Client requests. My problem is that I can't understand how you can retrieve data from the channel when you want to send, for example, a List or a HashMap. I mean, I've seen how the read(buffer) method works. I just can't understand -for example- how do you get th HashMap object back from the buffer, or how you retrieve any kind of "structured" data for that matter. If someone could explain(maybe with an example), that would be fantastic.
Maybe there is another way to communicate the data that I need, that would be easier for me to understand. I don't know. Your insight is greatly appreciated.
P.S. : My problem isn't that I don't get it because of the NIO, I have the same problem with the typical Input/Output Streams.
I should mention that the actual project is to create a Java server and the clients will be android devices. But since I'm a bit of a newbie, I thought I'd start off by testing the communication between two desktop, Java, applications before going for the android.
I mention this because I've seen something about Java RMI that allows you to use methods of your server remotely, but I think that you can't use it in Android.
You can read and write objects using the serialisation mechanism. The classes that are involved are ObjectOutputStream and ObjectInputStream. They are stream based though, so they don't fit well in the nio model. They are covered in the official tutorial: http://docs.oracle.com/javase/tutorial/essential/io/objectstreams.html
An alternative is using Google protocol buffers.
I have to say that my question might sound very shallow vague, I can't explain it well enough I think. But I will try!
I have a hardware box running on Linux and using C implementation to handle the functionality of the box. I can connect the box to my Laptop via Ethernet, on the Laptop I create higher Layer input for the box.
How can I put this input through to the box?
I might need some light-weight implementation of a proxy, right?
What would be a fast and easy way to do that?
additional information:
On Java side I create ASN1 structures and encode them. These structures I want to pass via Ethernet into my little box, which will send out the data via wlan. Oh man, I realise that my knowledge maybe isn't enough to explain that problem. Ok... Hmm, I am done with encoding the ASN1 structures, but from that point I have no idea how to go on. Somehow there must be a way to indicate to the box that new data is passed through and the box should send it out. maybe like a remote proxy, calling methods in Java but they are executed as C-methods inside the box. Basically a transfer system is needed to pass information out of a Java environment into a C environment over Ethernet. Wooo, possible?
Thanks in advance.
nyyrikki
Since the input will go through Ethernet, I assume data will be encapsulated within packets.
I have no idea what your input data looks like, but I suggest using the Jpcap (available here : http://netresearch.ics.uci.edu/kfujii/Jpcap/doc/).
Once you've messed around a bit with the API, it is really easy to learn and use.
I've personnaly used it a couple of times, and I have created a "Router" which acted as an ARP proxy. This lets me think you could use this library to achieve what you've explained.
Hope this helps!
Long time no see :)
I finally ended up using sockets, works well.
Thanks anyways, nyyrikki
My situation is: A c++ program needs to talk with a Java program using FIX protocol.
My solution:
- Messaging: C++ program publishes a text in FIX format which Java progrma can consume and parse with quickfix/j.
- Socket: Setup a FIX server in Java program, then C++ program as a client can connect to this socket and write byte stream into it using quickfix. Java program uses quickfix/j to parse the byte stream.
My questions:
1. Is there any compatiblity problem for socket solution, i.e. ,the byte stream coded with quickfix can be fully decoded by quickfix/j?
2. Which one is better? Cons and pros.
Thanks in advance.
FIX messaging would be an easier solution, rather than implementing sockets. There are socket communications already embedded in the quickfix libraries. It is no use to reimplement then unless you are doing something very different. The engine is meant to decipher FIX messages. And if you want to modify any of the socket communications for the libraries, you can change the libraries itself. You have the source code anyways.
If you try implementing sockets you may have to write wrappers around the sockets to parse messages from C++ along to Java and vice versa.
You have the C++ version of quickfix library. Use that as a initiator to send FIX messages across to the Java acceptor. You probably wouldn't have to worry about writing a C++ server to send FIX messages in a bytestream. Let the underlying library do the work of doing the communication rather than yourself.
FIX might be easier. But if you choose sockets, make sure to convert data send/recv from/on the C++ program to/from network byte order. (See reference for: htons(), htonl() ntohs(), and ntohl()). Java always uses network byte order so you don't have to do any conversion there.
FIX is a text based protocol, i.e. you don't have to worry about byte order. At the wire level, all you're doing is sending buffers of characters. So if you're writing in a C++ program to a java (quickfixj) based client/server, as long as you adhere to the FIX protocol, you'll have no issues.
Then again, as DumbCoder pointed out above, if you're not overly concerned about performance, you could use quickfix (the C++ version!)
Fix engines are written in a way they can communicate with other party fix engine.
Language or platform of the two parties does not matter.
You caan simply use JAVA version for one and C++ version for the other party.
i have set up a basic client and a basic server using java sockets. it can successfully send strings between them.
now i want to design some basic messages.
could you give me any recommendations on how to lay them out?
should i use java's serialzation to send classes?
or should i just encode the information i need in a custom string and decode on the other side?
what about recognizing the type of messages? is there some convention for this? like the first 4 characters of each message are a identifier for the message?
thanks!
I would recommend you not to reinvent the wheel. If java serialization suits you, just use it.
Also take into account that there are some nice serialization frameworks around:
thrift, from facebook, and protocol buffers from Google.
Thrift also is a RPC mechanism, so you could also use it instead of opening / reading raw sockets, but this, of course, depends on your problem domain.
Edit: And answering your question about the message formatting. Definitely if you want to implement your own protocol and if you have more than one type of messages you should implement a header yes. But I warn you that implementing a protocol is hard and very error prone. Just create an object containing the different inner objects + methods you need, if you want add it a version field and make it implement the java.io.Serializable interface.
Maybe JMS would help you, it's hard to say without knowing the details. But JMS is standard, well thought out and versatile, and there are an impressive number of implementations available, open source and commercial. We use Sun's OpenMQ implementation and we're quite happy with it. It's fast enough for our needs, very mature and reliable.
Mind you, JMS is not a lightweight affair by any standard so it may very well be overkill for your needs.
If you're going to deploy this in a production environment, I'd advice you to look at either RMI or XML web services. (Google's Protocol Buffers are interesting too, but do not include a standard protocol for message transport, although 3rd party implementations exist.)
If you're doing this for the pleasure of learning, there are tons of ways to go about this. In general, a message in a generic messaging system will have some kind of "envelope format" which contains not only the message body, but also metadata about the message. A bare minimum for the header is something that identifies the intended receiver - either an integer identifier, a string representing a method name or a file, or something like it.
A simple example is HTTP, a plain-text format where the envelope and the is made up of all the lines until the first blank line. The first line identifies the protocol version and the intended receiver (≈the file requested), the following lines are metadata about the request, and the message body follows the first blank line.
In general, XML is a common format for distributed services (mostly because of its good schema capabilities and cross-platform support), although some schemes use other formats for simplicity and/or performance. RMI uses standard Java object serialization, for example.
What you choose to use is ultimately based on your needs. If you want to make it easy to interact with your system from a large amount of platforms, use XML web services (or REST). For communication between distributed Java subsystems, use RMI. If your system is extremely transaction intensive, maybe a custom binary format is best for faster processing and smaller messages - but before doing this "optimization", remember that it requires a lot more work to get it working properly and that most apps won't benefit a lot from it.
What would be the easiest way to be able to send and receive raw network packets. Do I have to write my own JNI wrapping of some c API, and in that case what API am I looking for?
EDIT: I want to be able to do what wireshark does, i.e. record all incomming packets on an interface, and in addition be able to send back my own created packets. And I want to do it on a mac.
If you start with the idea that you need something like a packet sniffer, you'll want to look at http://netresearch.ics.uci.edu/kfujii/jpcap/doc/.
My best bet so far seems to be the BPF api and to write a thin JNI wrapper
Raw Socket for Java is a request for JDK for a looong long time. See the request here. There's a long discussion there where you can look for workarounds and solutions. I once needed this for a simple PING operation, but I can't remember how I resolved this. Sorry :)
You can't access raw sockets from pure Java, so you will need some sort of layer between your Java code and the network interfaces.
Also note that access to raw sockets is normally only available to "root" processes, since otherwise any user could both a) sniff all traffic, and b) generate spoofed packets.
Rather than write your whole program so that it needs to run as "root", you might consider having the packet capture and generation done in a standalone program with some sort of IPC (RMI, named pipe, TCP socket, etc) to exchange the data with your Java app.
TINI is a java ethernet controller, which may have libraries and classes for directly accessing data from ethernet frames to TCP streams. You may be able to find something in there that implements your needed classes. If not, there should be pointers or user groups that will give you a head start.