I'd like to obtain the host's full network usage, that is, the current upload and download rates. How can I do that in Java? If it makes that easier, assume the platform is Linux.
How can I do that in Java?
Doing this in Java is a bad idea ... unless there is a strong requirement to have the information in a larger Java application.
You can't do it in pure Java. There are no standard Java APIs for accessing this kind of information.
I Googled for something Java-specific and free that matched your requirements, but didn't spot anything. (YMMV ... please feel free to repeat the search for yourself!).
There are OS specific tools for looking at network stats, etcetera, and one could attempt to run these tools as external processes from Java and "scrape" the output. But that would necessarily be OS / tool specific.
You could attempt to replicate what these tools are doing in Java, but this may involve native library calls, and would certainly be OS specific. (On Linux, you may be able to get the information you need by reading from the /proc file system ... but there could also be access control issues.)
Nethogs: http://nethogs.sourceforge.net/ is probably the tool you're looking for.
Most people use prefer this as it splits bandwith usage up by process.
Related
I come from a C/Linux background and don't have much background in Java. I generally develop system administrator utilities like :
disk cleanup
retrieve lost data / files
repairing file systems
disk de-fragmentation
I also develop Network monitoring security applications which help admins monitor :
- their networks,
- scan incoming & outgoing data packets,
- remotely block ports / USBs
- monitor emails with attachments etc
Right now we write code in C for Linux which has to be ported to windows but such a problem will not exist in Java.
My questions are :
Is Java the right language for writing these applications & utilities (as mentioned above)?
I understand Java will provide Libraries and classes to access system resources / network / sockets but will Java abstraction be a hindrance at some point (which would restrict the flexibility which C/C++ provide )?
If for example I want to write a utility to repair a file system / or retrieve data for Windows & Unix ...will I be using same API for both OS or there are different API for different OS?
I am not concerned about the speed / execution trade off since none of my applications have to make real time decisions as in the gaming industry.
Java is the right language if you want portability. You can do almost everything you can do with C/C++ and you can utilize patterns and libraries that help you create great maintainable designs. In case there is something very low level you cannot do with Java, you always can create your own native code that is loaded with Java Native Interface. Thus the only non-portable code you will have will be these native-code libraries.
Right now we write code in C for Linux which has to be ported to
windows but such a problem will not exist in Java.
Java can abstract away only so much since in the end, low level stuff always boils down to making system calls, which are different between OSes.
As long as you're working with pure java logic, or simple operating system utilities, you'll be golden. You want to open a TCP socket and connect to google.com? No problem. You want to open a file in a known location, read some lines, process them, and write the results to a different file? No problem, Java has you covered.
But, if you want to do more low-level stuff with Java, you'll run into trouble pretty soon. You want to open a raw socket and send a TCP packet? You can't, windows doesn't allow that. You want to get a file's creation time on Linux? You can't, Linux doesn't keep that information. Java's BasicFileAttributes.creationTime() will return a file's modification time on Linux. You want to get a nanosecond resolution timestamp? Well, you can, but only on some OSes. Or say you want to get the computer's hostname without resorting to a network lookup (which depends on a network being actually available), well, get ready for some hacking (this is my own answer by the way).
Now, to your more specific questions:
Is Java the right language for writing these applications & utilities (as mentioned above)?
I frankly don't know. I never tried defragmenting or restoring a file programmatically from Java. But since it involves some very low level filesystem operations, I suggest you do some serious reading before moving to Java. Check whether the APIs you need exist in the language itself or in some good libraries.
I understand Java will provide Libraries and classes to access system
resources / network / sockets but will Java abstraction be a hindrance
at some point (which would restrict the flexibility which C/C++
provide )?
Yes. For instance, it's impossible to open a raw socket using pure Java. And if I recall correctly, it's also impossible to set some socket options.
If for example I want to write a utility to repair a file system / or
retrieve data for Windows & Unix ...will I be using same API for both
OS or there are different API for different OS?
I never tried repairing a file system in Java, so I can't tell you about the APIs involved. But I find it hard to believe you'll find a pure Java api for doing low level stuff with the file system. You'll probably have to write your own native code (and run it through JNI) or use someone else's library (which probably uses JNI, like the raw socket library I mentioned earlier).
This question already has answers here:
Get OS-level system information
(17 answers)
Closed 8 years ago.
I would like using Java to get data from Windows about CPU, RAM, HDD, Network and list of processes(services). How I can get these system values values without using third-party libraries or JNI?
Is there something similar to /proc in Windows where I can get all necessary values and use them?
You can use native API's using JNI.
You can also use SIGAR which is Apache licensed library which does much more then what you are asking for.It has wrapped the native calls in Java API's so that you dont have to know the inner workings of JNI.
You can also spawn processes from inside the code to run OS specific commands so that you can collect the system stats e.g. systeminfo |find "Available Physical Memory" or wmic command on windows or Linux cat /proc/meminfo . This can be achieved via java.lang.ProcessBuilder.
I dont think you can get everything you want from a single location in the standard java API, but you could use the following classes for some of your requirements:
For the number of processors / processor stats / memory, etc:
http://docs.oracle.com/javase/7/docs/api/java/lang/Runtime.html\
For information about physical disks and whatnot:
http://docs.oracle.com/javase/7/docs/api/java/io/File.html
Another, really, really ugly option would be to make platform specific invokations of utilities via a Process object, and parse their output. EG invoke the top command via a process, read and parse its output.
There is Windows Management Instrumentation API (WMI) for this kind of stuff. See e.g. .NET bindings here: MSDN - System.Management Namespace
Older Stack Overflow answer to similar question is available here: Recommended libraries/howtos for using WMI with java?
Look for newer Java bindings. There used to be J# as one of the .NET languages that was source-level compatible with Java. So in theory there should not be a big problem to find suitable Java→.NET→WMI bridge
This API is very powerful and besides some basic stuff like processes, sound devices, disk drives etc. you can access approximately same set of information that you can get by running msinfo32.exe.
So this is not Java-specific answer this is Windows-specific answer. All Windows versions support .NET and .NET can access that information through WMI easily.
For instance list of processes can be accessed easily through MSDN - System.Diagnostics.Process.GetProcesses
There used to be special Win32 APIs for native binaries and though still supported they are the history now. Piping output from one command line app to input of another is not the usual way to connect processes. In case this is really needed there is Microsoft Technet - Scripting with Windows PowerShell
I hope this will help you find some solution of your problem
There are several options for getting system information in Java (the Runtime class, the java.lang.management package, etc.).
However, this is usually very limited information, and tends not to cover the things you seem to require.
You should consider using a 3rd party solution, such as the following:
SIGAR - http://www.hyperic.com/products/sigar (requires native DLL to be installed)
OSHI - https://github.com/oshi/oshi
Well, I would suggest you to try JavaSysMon:
https://github.com/jezhumble/javasysmon
Manage OS processes and get cpu and memory stats cross-platform in Java. So the good thing about it is , its cross-platform.
Currently it supports Mac OS X, Linux, Windows, and Solaris.
So that is a positive. If this can do your work, you can actually use the cross-platform feature for your application.
Maybe its not as good as Sigar, but give it a try.
Suppose i want to search some text in a file. I want to know when we should use system utilities/programs like grep and when we should use Java API's like reading a line, and then search the text in that line or use java Scanner class.
I want to understand the trade-offs between the two approaches. I mean, suppose if we use grep, then will there be communication overhead between JVM and the grep process? Is creation of a new OS process for grep an overhead?
Does grep performs better than normal java file search?
Please help...
Yes, there will be an overhead. Starting an external process and communicating with it is costly. And moreover, many systems don't have a grep command. If you want to make your Java code portable, don't rely on OS-specific commands.
Another problem is that OS commands will be able to search (for example) in files, but not in your in-memory data structures.
You're basically trading off system independence for the perceived gain of the tool, in some cases this can't be avoid.
Not every system will have the tools you want installed, in the locations you think they should be or the version you need.
Even if you can deploy the tools with your application, you will need to provide an implementation for each of your targeted platforms.
Sure, it's easy to say "it will never be run on X", but can never come around real quick ;)
There is also the added over head of executing and managing the IO of the external applications, while not difficult, it's much more complex that a well written Java API.
As I said, sometimes, you simply don't have a choice (I have some media inspection tools that I use on Windows and Mac that I'm not about to try and implement in Java, not because it can't be, but because it's complex and time consuming and somebody has already done it (with a native program)).
You need to balance the choice over what the benefits are of the external command weight against the issues of using it. You should also investigate if a API has already begin developed that might solve the problem at hand.
IMHO
How does one secure the Java environment when running on a machine you don't control? What is to stop someone from creating a java agent or native JVMTI agent and dumping bytecode or re-writing classes to bypass licensing and/or other security checks? Is there any way to detect if any agents are running from Java code? From JNI? From a JVMTI agent?
If you don't control the environment, then I'm sorry - you're really stuck. Yes, you could look for trivial JVMTI agents via some sort of cmdline sniffing, but that's the least of your worries. Think about java/lang/Classloader.defineClass() being compromised directly. That's easy to do if you own the box - just replace the .class file in rt.jar. In fact, until JVMTI came around, that was a typical way that profilers and monitoring tools instrumented Java code.
Going back to JVMTI - the "Late attach" feature also allows for JVMTI agents to be loaded on the fly. That might not have happened when you scanned the first time around.
Bottom line - if someone can change the bytes of the JRE on disk, they can do anything they want. Is it ethical, no? Can they get caught? Possibly, but you'll never win the war.
It looks like I can go with a combination of checks inside some custom JNI native code.
1.) cmd line sniffing to search for agents.
2.) Ensure that the cmd-line parameter -XX:+DisableAttachMechanism exists. (this will prevent people from attaching to my running VM)
I remember I once made almost a silent Java Agent. I guess you better look for port scanners or something around that.
Java 2 security, signing of jars etc, gives some level of control over what gets loaded into your application.
However in the end if a malicious person has access to a machine such that they can write to disk then in all probability they have plenty of capacity to do harm without resorting to clever Java hacks.
Turn this round, in any language what can you do to detect Trojans?
Careful access control to the machines you care about is non-trivial but essential if you are serious about such issues. Security specialists may seem paranoid, but that often means that they really understand the risks.
If you can't control the platform, you can't control the software upon it.
Even if you could shut down all the avenues of inspection you've listed, Java is open source. They could just take the source code and recompile it with the necessary changes built-in.
Also, try to remember that while it is your code, it's their machine. They have a right to inspect your code to verify that running it on their machine does what they expect it to do, and doesn't perform "extra" actions which they might find undesirable. Less trustworthy companies in the past have scanned for non-relevant files, copied sensitive information back to their home servers, etc.
I would look at the command line and see, if there are any "-agent" parameters. All profilers, debuggers and other code modificators use this for introspection. You could also check for unusual jars on the bootclasspath, since those might also provide a threat (but be aware that you then also must deliver a custom JVM, since some software like Quicktime adds itself to the bootclasspath of ALL java apps running... (I couldn't belive my eyes when I saw that...))
Basically this is a loosing battle.
Have a look at how visualvm in the Sun JDK works and how it can attach to a running process and redefine whatever it pleases. It is extremely hard to detect that in a portable way, and unless you can do so, you might as well give up on this approach.
The question is, what is it you want to avoid?
Some modern network cards support Direct Memory Access for improved performance. How can I utilize this feature from Java?
Does the JVM provide this automatically, or do I need to do an allocateDirect on the ByteBuffers that I am using to talk to that NIC?
Does anyone have documentation that discusses this?
It is the operating systems task to use the DMA feature of the network card. The JVM does not really care how the OS does it, and simply uses the operating system's functions for talking to "network interfaces".
You cannot do this from inside Java in the typical desktop/server JVMs, as this is operating system area which requires you to reach out into C code. Go have a look on JNI or JNA to see how to do this. Please note that this may make your application brittle if you do not get this exactly right.
Yeah - ankon's answer is right. Java operates in a sandbox - a virtual machine (hence the, "VM" in JVM; Sun actually built ONE physical version -- it's on display somewhere).
Java was never designed (intentionally) to reach outside the sandbox, unlike ActiveX, which can go just about anywhere on a PC.
Just think of all the bad things ActiveX has done over the years via a browser. You wouldn't want that to happen with Java, would you?
Although...
you might be able to instantiate an object in Java that does have access to the hardware (like one of those ActiveX controls, or some DLL, for example - which you'd have to write, too).
The problem I see is the throughput. With 100MB or 1000MB cards, would a JVM (remember, this is a VM running on an OS, so you're a couple of layers removed from the hardware) have the speed to handle what's coming in under load? Would you want a Java program holding up data in your NIC while it tinkered with it (think of the impact to the rest of the system)?
At this point, you're probably better off writing the hard-working guts of your solution in C. And, if you still need Java to play with that data, put it in a place where Java can get to it.
If you're not getting the network throughput you need in java, then you're going to need to write a C wrapper in order to access it.
Have you benchmarked your code to find where your performance issues really are? If you let us know that we can likely help you out without resorting to JNI.