My situation is: A c++ program needs to talk with a Java program using FIX protocol.
My solution:
- Messaging: C++ program publishes a text in FIX format which Java progrma can consume and parse with quickfix/j.
- Socket: Setup a FIX server in Java program, then C++ program as a client can connect to this socket and write byte stream into it using quickfix. Java program uses quickfix/j to parse the byte stream.
My questions:
1. Is there any compatiblity problem for socket solution, i.e. ,the byte stream coded with quickfix can be fully decoded by quickfix/j?
2. Which one is better? Cons and pros.
Thanks in advance.
FIX messaging would be an easier solution, rather than implementing sockets. There are socket communications already embedded in the quickfix libraries. It is no use to reimplement then unless you are doing something very different. The engine is meant to decipher FIX messages. And if you want to modify any of the socket communications for the libraries, you can change the libraries itself. You have the source code anyways.
If you try implementing sockets you may have to write wrappers around the sockets to parse messages from C++ along to Java and vice versa.
You have the C++ version of quickfix library. Use that as a initiator to send FIX messages across to the Java acceptor. You probably wouldn't have to worry about writing a C++ server to send FIX messages in a bytestream. Let the underlying library do the work of doing the communication rather than yourself.
FIX might be easier. But if you choose sockets, make sure to convert data send/recv from/on the C++ program to/from network byte order. (See reference for: htons(), htonl() ntohs(), and ntohl()). Java always uses network byte order so you don't have to do any conversion there.
FIX is a text based protocol, i.e. you don't have to worry about byte order. At the wire level, all you're doing is sending buffers of characters. So if you're writing in a C++ program to a java (quickfixj) based client/server, as long as you adhere to the FIX protocol, you'll have no issues.
Then again, as DumbCoder pointed out above, if you're not overly concerned about performance, you could use quickfix (the C++ version!)
Fix engines are written in a way they can communicate with other party fix engine.
Language or platform of the two parties does not matter.
You caan simply use JAVA version for one and C++ version for the other party.
Related
Is there a way (in Java) to read IP-layer headers (I'm interested in src & dst IPs), using TCP or UDP socket?
I know it's possible using pcap (packet capture), but I didn't find a way to do it using sockets.
What's the reason behind that?
At the operating system level, you need raw sockets to access IP-layer headers. But fortunately raw sockets don't exist in pure Java so you need native code to make the relevant syscalls.
There are Java libraries that use a JNI component for the native calls. One example (which I used successfully in the past) is Rocksaw.
You can just open /dev/socket as a InputStream and read data coming into to socket.
I'd like to somehow get control over active sockets on my computer with java/scala. For example, if a program has stablished a tcp connection, i want to be able to retrieve and listen/write onto this socket.
So, the first question is: is that possible?
And the second: how?
You can do this by writing a custom Socket factory. This is fairly complicated and I don't suggest you do this unless you are trying a to hack a program you have no control over (again a very bad idea)
Instead I suggest you monitor your own usage. When you read/write from a Socket you also keep any information you need. i.e Its your program so change it to do what you need.
I found a java wrapper of lipcap http://jnetpcap.com/ that let me do the sort of things i was after. From the web:
jNetPcap is an open-source java library. It contains:
A Java wrapper for nearly all libpcap library native calls
Decodes captured packets in real-time
Provides a large library of network protocols (core protocols)
Users can easily add their own protocol definitions using java SDK
jNetPcap uses a mixture of native and java implementation for optimum packet decoding performance
I need to transfer files fast over the Internet from a Java server to C++ clients, where often many clients would need the same files. I was looking at say transferTo() in Java which sounds like it would be a decently optimized function to send files. However, I'm not sure when I use transferTo() how to best receive that in C++ (i.e. is it just a raw data transfer, how do I determine when the file is over on the client side, etc.). I need this to work on both Windows and Linux. Also, other than transferTo(), would there be some way to be more efficient, especially by taking advantage of the fact that many clients will usually need the same files? I'm not sure how to do say multicast etc. Also, I'm using application-level security rather than a VPN, and on the Java server, encrypting with AES and using MAC digital signing, so I'm also looking for a cross-platform library recommendation to deal with the crypto on the C++ side with minimal pain.
I'm very proficient in C++ but have no previous experience with network programming, so please consider than in any suggestions.
Thanks.
An embedded webserver? http-transfers are efficient enough for you?
The simplest embeddable Java webserver I remember seeing is http://acme.com/java/software/Acme.Serve.Serve.html. We use embedded Jetty 6 in production at work, but that takes more elbow grease.
If your clients doesn't know where to find your webserver in the first place, consider announcing using Zeroconf. http://jmdns.sourceforge.net/
For scalability reasons, Thorbjørns suggestion of using http seems like a very good idea as it would allow you to easily set up http proxies for caching, use standard load balancing tools and so forth.
If you are looking to transfer more than just a blob of data, you might want to have a look at googles protocol buffers. They allow for very easy and fast encoding/decoding on the java and c++ end.
Consider chunking the file and sending via UDP datagram. C++ can re-compile as it receives it. Have you considered implementing/embedding an existing P2P protocol implementation?
If you need effecient transfer to many clients then your bottleneck is the server.
For this please look at the bit-torrent protocol as it distributes the transfer between the clients.
Does anybody know how to receive (how to know that you received) out-of-band data with Java sockets? In particular I've read the documentation for sendUrgentData and setOOBInline which states that:
Note, only limited support is provided for handling incoming urgent data. In particular, no notification of incoming urgent data is provided and there is no capability to distinguish between normal data and urgent data unless provided by a higher level protocol.
But I'm still hoping that somebody has a solution :-)
More details:
I'm sending serialized objects trough the socket and I'm not really seeing how I can find a "random" OOB byte inserted in the stream (other than the fact that Java will give me an exception when I'll try to deserialize the object).
Tomcat has a maintained implementation of JNI socket which can read urgent packets in a OOB manner.
It looks like Java, up to Java 6 (I didn't investigate Java 7 since I don't know if they've decided for sure what will go in or not) and it looks like Java does not provide any support at all for reading urgent TCP information in an out-of-band manner. And it looks like this is a deliberate choice, thus far.
However, I did find one user who implemented his own solution, using JNI, to successfully read urgent data out of stream. This hack is very old and may or may not work on recent JDK's. I have not tried this myself! Your milage may vary. :) But hopefully this will help you.
What would be the easiest way to be able to send and receive raw network packets. Do I have to write my own JNI wrapping of some c API, and in that case what API am I looking for?
EDIT: I want to be able to do what wireshark does, i.e. record all incomming packets on an interface, and in addition be able to send back my own created packets. And I want to do it on a mac.
If you start with the idea that you need something like a packet sniffer, you'll want to look at http://netresearch.ics.uci.edu/kfujii/jpcap/doc/.
My best bet so far seems to be the BPF api and to write a thin JNI wrapper
Raw Socket for Java is a request for JDK for a looong long time. See the request here. There's a long discussion there where you can look for workarounds and solutions. I once needed this for a simple PING operation, but I can't remember how I resolved this. Sorry :)
You can't access raw sockets from pure Java, so you will need some sort of layer between your Java code and the network interfaces.
Also note that access to raw sockets is normally only available to "root" processes, since otherwise any user could both a) sniff all traffic, and b) generate spoofed packets.
Rather than write your whole program so that it needs to run as "root", you might consider having the packet capture and generation done in a standalone program with some sort of IPC (RMI, named pipe, TCP socket, etc) to exchange the data with your Java app.
TINI is a java ethernet controller, which may have libraries and classes for directly accessing data from ethernet frames to TCP streams. You may be able to find something in there that implements your needed classes. If not, there should be pointers or user groups that will give you a head start.