As we know the Sockets Direct Protocol (SDP) provides access to high performance network connections, such as that provided by InfiniBand
can any one please give me any good example of how can i use Sockets Direct Protocol (SDP) of java ?
If you are running a recent version of Solaris or OpenSolaris I think you are all set to try stuff out.
Linux requires use of the libsdp library and external kernel modules available from OpenFabrics (OFED), there seems to be strong desire to get the modules into Linux mainline but OFED guys seem to be rubbing the kernel developers up the wrong way over some of the issues involved.
There's a useful setup guide for Infiniband under Debian Linux which gives details on how to set up and compile the OFED kernel modules and the libsdp library etc. Once you have that done you can follow the SDP section of that how-to in order to get SDP working and tested.
After that I think it is as simple as getting a recent JDK7 snapshot, currently at M12, and trying stuff out. Set up your config and create connections between addresses specified in your SDP config, all done via the same APIs as network sockets, i.e. Socket for 'old' IO, and SocketChannel for NIO (or even AIO in JDK7 :). As far as I can see, once you have followed the set up guides and configured the addresses that will use SDP the JVM will use SDP based Sockets/SocketChannels whenever you provide addresses within the SDP mapped range. So your existing networking code that uses sockets should be able to use the SDP connections transparently, just at blazingly fast speeds and tiny latencies.
Windows users are out of luck for the time being, as far as I can tell.
Also:
Alan Bateman's SDP blog entry (Oracle JDK SDP lead)
Related
I am working in a desktop application, where application is deployed in both windows and mac platforms. As part of the application, it should communicate with native layer. Currently the communication between native layer and Java layer is done using sockets. Recently some one in the team suggested to use zeroMQ. Can any one of you guys please clarify my doubts.
How zeroMQ better than sockets
Is it possible to install zeroMQ library as part the Desktop client installation
I gone through the link 'https://github.com/zeromq/clrzmq4', it given libraries specific to amd64 and i386 processor family. Do I need to build it separately from the source code for different processors?
Do I still require .dll files to use zeroMQ in Java?
Do I require Visual studio to build zeroMQ libraries in windows (Since my native layer written in C#, my C# application communicate with zeroMQ socket socket written java)?
How zeroMQ better than sockets
http://zeromq.org/topics:omq-is-just-sockets
Is it possible to install zeroMQ library as part the Desktop client installation?
Yes, you need to build the libraries depends on the processor and embed them in your application.
Do I need to build it separately from the source code for different processors?
Yes, you need to build the libraries from source. zeroMQ is processor centric.
Do I still require .dll files to use zeroMQ in Java?
Yes, Following link may help you
Exception in thread "main" java.lang.UnsatisfiedLinkError: ... \jzmq.dll: Can't find dependent libraries
Do I require Visual studio to build zeroMQ libraries in windows?
Yes
This link may help you to get basic examples.
Regarding ZeroMQ in a desktop application on Windows talking to another process on the same machine, bear in mind that zmq_ipc is not supported (see zmq_ipc(7)). Or at least, that's the last I heard. This is because it's fundamentally impossible to implement anything like quite like select() or epoll() for named pipes in Windows. Just use zmq_tcp instead.
The same basic problem plagued the development of the select() implementation in Cygwin and its derivatives. They got round the problem by starting a thread for every non-socket file descriptor (i.e. named pipes, serial ports, etc) being selected, with each thread polling the HANDLE for whether any data had arrived (or whatever events were being set in select()). Not very efficient. Yeurk.
Proactor vs Reactor
Windows is proactor (which can do only proactor), everything else (*nix, VxWorks) is reactor (which can also be used to implement a proactor). The development of the boost.asio library for C++ was influenced by this, and is a proactor design as a result so that it could be run on Windows. RabbitMQ is proactor too.
ZeroMQ with zmq_poll() is reactor.
Proactor - you pro-actively start up an asynchronous routine to handle whatever turns up in the future.
Reactor - you react to the arrival of whatever has turned up by starting a synchronous call to whatever routine you wish to handle it knowing that it will complete very quickly because the data is already there.
The difference is key. In a proactor design, once you have started up that asynchronous routine to read a message, you cannot (easily) stop it or change it until it has done its thing. That is very annoying if you change your mind, for example as a result of reading some message from somewhere else.
Small caveat - Windows does support select() for network sockets (thus reactor programming is possible with network sockets, but only network sockets), and is the only reason why ZMQ is supported to any extent whatsoever on Windows.
Mixing ZMQ with Desktop Application Event Callbacks
Anyway, proactor means that Windows and C# fundamentally expects everything to be served by callbacks. This basically means you won't be using the zmq_poll() call to tell you if new messages have arrived if you also have callbacks handling GUI events. Instead you'd most likely be making asynchronous calls to zmq_revcmsg(). Trying to mix zmq_poll() in with callbacks is asking for trouble (you'd be blending proactor and reactor).
Message Formats
ZeroMQ and sockets both transfer bytes (as discrete messages with ZeroMQ, as a byte stream with sockets). One still has the challenge of deciding what the bytes mean to applications.
I can recommend using something like Google Protocol Buffers to serialise messages for transport by ZeroMQ. It's available for both C# and Java, but it doesn't demarcate message boundaries. Fortunately, ZeroMQ does. [Using GPB over a socket stream can be painful, you have to demarcate the message boundaries oneself]. So you can serialise a message to a buffer, hand the buffer over to ZeroMQ as a message, the recipient receives the message and knows for absolute certain that there is one single solitary GPB within. If you like you can use GPB's "oneof" to smuggle arbitrary message types across, which can be very liberating. You can accomplish the same with other serialisation technologies too of course, my personal favourite being ASN.1.
I have to develop a small web application which would be running on a embedded device.
There are many light weight servers like thttpd, lighttpd. The functionality I want is something similar to router/modem configuration page. I am using mpc5200B (http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC5200B). It'll have a Linux kernel on it.
I am not able to decide on the server and programming language to use. Should I just run an embedded apache server and use JSP/servlets for programming as I believe the chip has enough resources.
Any hints/help/insights would be helpful. Thanks!
I've never seen routers with JVM inside.
From my own experience - every router I've configured had a PHP inside web-admin application.
For smaller embedded systems, based on single-chip microcontrollers, you might want to consider the lwIP TCP/IP stack. There is a YouTube video that shows an embedded web server running on a small embedded ARM Cortex-M3 board with SSI (server-side includes) and CGI (common gateway interface) capabilities. This means that you can generate HTML dynamically (SSI) and you can accept commands from standard HTML forms (CGI). All this without a need for running Java in your browser.
I would suggest Apache + PHP. Even you have enough resource, whether the technology is stable or not still one of the consideration.
Apache + PHP is a very stable combination.
I am not sure, seems the device come with it's own linux.
The micromonitor (uMon... http://www.umonfw.com) package has an LWIP application that includes the HTTP server that started from LWIP contributions; however I modified it quite a bit so that it would serve files out of TFS (basic FFS that comes with uMon). Plus I added some basic hooks to support SSI-like stuff and some form-to-target interaction. Works ok and doesn't need an RTOS; but for really small systems it still needs to be "tuned". So...
I am now working on a condensed version of that package (called uMon-M) for use on devices with very small footprints like the Cortex-M3/M4 stuff. As of this writing, I have it ported to the "Simple-cortex" (http://www.brc-electronics.nl) based on an NXP-LPC1769 (Cortex-M3). The boot monitor itself is a major subset of uMon, but still has all the same fundamental capabilities (FFS, scripting, UDP/IP, etc..). Most of my effort at this point is in the HTTP server as a user interface for an application. It currently works I'm just trying to make it smaller; but still have some reasonable ability to serve dynamic web pages. I've got it hooking to Web Sockets and a very basic SSI-like facility so that the html pages can be more dynamic. Try it, it works as is, and will hopefully just get smaller. It would be great to get some folks looking at this with me for collaboration.
In case of 8- and 16-bit microcontrollers you can use micro IP (uIP). It implements TCP/IP protocol and it is written in C and requires few kilobytes of RAM ( good striped version requires only 200 bytes of RAM). Moreover, uIP is open source TCP/IP stack.
Have you considered Snorkel or Appweb?
Or Raspberry PI. Cost is less than $50, and you can run Linux with a LAMP stack, iptables, etc.
After using RXTX for a number of different projects, I've come across many annoying discrepancies and issues that can only sensibly be put down to bugs in the library - deadlocks, race hazards, and monitor threads deep in the RXTX library being left open preventing the program from closing (even though all ports I was using have been closed!) Running the latest "unstable" version has helped a bit, but it's still far from where I'd call it reliable, and activity on the project seems rather low at present.
However, searching for free cross-platform alternatives to RXTX doesn't seem to come up with much else.
Is there another library that anyone's tried with more luck? I don't need parallel port support, just serial, and it doesn't necessarily have to be compatible with RXTX or the javax.comm API.
We have moved to java-simple-serial-connector which seems to be very stable and comfortable at the current release 0.9.0. For now, it works on windows and linux systems. We use it for modem interactions.
There's jSerialComm too.
Serial communication in Java is a mess. I've had trouble with jSSC with a virtual usb serial on linux, but with serial-comm it connected.
RXTX is a mess.
At least I think that now this SO question has all of the worth mentioning serial communication java libraries.
There's a few of them out there now, now that Arduino is pretty popular.
There's Serialio
and PureJavaComm, but it's pretty early on in development from what I can see.
it really depends on what your needs are. RXTX has been ported pretty much everywhere, so it is widely available.
One of the issues of RXTX I faced was the delay closing the port. When checking with setserial the port configuration I discovered a parameter closing_delay=3000. This means that the port closes 3 sec after the close() method was executed. Resetting the closing_delay toany value is ignored by RXTX. RXTX insists in 3sec delay.
the JSSC library performed to our satisfaction
Serialio lib is $49/platform/developer seat which gets you unlimited distribution in unlimited apps so not too pricey. The Mac library has been stable and reliable for us in an undemanding application and availability/compatibility has been good throughout for the various Mac platforms and OS/X releases.
I use socat configured to serve a socket for a serial port.
The downside is that baud rate etc. is given to the socat invocation, it is not something you can readily alter from your program.
On the upside is I can connect across a network without anything more than changing the IP address I connect to.
jSerialCom is good !
'https://github.com/Fazecast/jSerialComm'
And here is its' wiki pageļ¼ https://github.com/Fazecast/jSerialComm
I am planning on implementing a NIDS (Net Intrusion Detection System) in the Java programming language.
After searching, I found two libraries for this.
1) Jpcap
2) jNetPcap
Which one should I use and why?
Which is more preferable to use?
It appears development has stopped on Jpcap. Their last release was 2007. In contrast, jNetPcap has had very recent releases. The underlying libpcap C library that they both wrap has continued to evolve, so I would go with jNetPcap for that reason.
On the other hand, libpcap is a fairly simple API. If you are comfortable with C, then you may gain performance advantages by using the library directly instead of going through a Java wrapper. Something to consider, anyway.
One simple idea I've had for network intrusion was to listen for messages on the braodcast IP address. New comers to the network may likely need an IP address from the DHCP server. If this is so, then they will have to send out a broadcast that a DHCP server will respond to . You could write a program that sits on client A and displays popups whenever it hears something on the broadcast address.
If it's still an issue, use .NET version jnetpcap. I built a NIDS for a master's term project and I tried to use Java and get jpcap and similar working but it was near impossible. So I went for the most viable option c# and it worked alot easier, even thought I didn't know it then.
Also use ikvm since you will need to use weka.jar from .NET as well.
I need to establish a P2P UDP and TCP Connection between two Users. Both of them are behind a NAT.
A little research leads me to STUN, TURN and ICE.
Is there any Java solution (library) except jSTUN which seems to work only on UDP.
And TURN, ICE is much better for the symmetric NAT Problem.
Not to toot my own horn too much, but might I recommend trying IcedJava. (It's a part of the SipTools project) It's actively being developed, has a stable release 0.9 version, and supports use cases like RTP streaming out of the box.
As far as TCP support, that's slated for a later release, but you're going to have a hard time supporting TCP over symmetric nat due to the three way handshake being blocked. Even the TCP extensions to the ICE specification note that TCP connections are much less likely to succeed than UDP connections between natted peers.
That said, you can workaround this by routing TCP packets over a UDP bridge, including all of the error correction and flow control features. (I forget the RFC for this, but it is a final specification) Still, all of these are relatively new specifications, and the ICE-TCP spec is not yet final. Even in active development, these will take time to implement.
There is also ice4j, which acording to the developers:
A Java implementation of the STUN and TURN and ICE protocols. This project is a continuation of the Stun4J initiative. The goal of the project is to provide VoIP Java developers with a means of handling NATs and firewalls.
it's been developed by the sipcommunicator devs.
You could try jstunt - no longer maintained, but source seems to be available.
They provide a jar with the STUNT library and example java server/client applications.
From my reading, it's not native java, so this may only work on certain architectures.