We Keep Coding

Developing Networking Security Applications & System utilities in Java

I come from a C/Linux background and don't have much background in Java. I generally develop system administrator utilities like :
disk cleanup
retrieve lost data / files
repairing file systems
disk de-fragmentation
I also develop Network monitoring security applications which help admins monitor :
- their networks,
- scan incoming & outgoing data packets,
- remotely block ports / USBs
- monitor emails with attachments etc
Right now we write code in C for Linux which has to be ported to windows but such a problem will not exist in Java.
My questions are :
Is Java the right language for writing these applications & utilities (as mentioned above)?
I understand Java will provide Libraries and classes to access system resources / network / sockets but will Java abstraction be a hindrance at some point (which would restrict the flexibility which C/C++ provide )?
If for example I want to write a utility to repair a file system / or retrieve data for Windows & Unix ...will I be using same API for both OS or there are different API for different OS?
I am not concerned about the speed / execution trade off since none of my applications have to make real time decisions as in the gaming industry.

Java is the right language if you want portability. You can do almost everything you can do with C/C++ and you can utilize patterns and libraries that help you create great maintainable designs. In case there is something very low level you cannot do with Java, you always can create your own native code that is loaded with Java Native Interface. Thus the only non-portable code you will have will be these native-code libraries.

Right now we write code in C for Linux which has to be ported to
windows but such a problem will not exist in Java.
Java can abstract away only so much since in the end, low level stuff always boils down to making system calls, which are different between OSes.
As long as you're working with pure java logic, or simple operating system utilities, you'll be golden. You want to open a TCP socket and connect to google.com? No problem. You want to open a file in a known location, read some lines, process them, and write the results to a different file? No problem, Java has you covered.
But, if you want to do more low-level stuff with Java, you'll run into trouble pretty soon. You want to open a raw socket and send a TCP packet? You can't, windows doesn't allow that. You want to get a file's creation time on Linux? You can't, Linux doesn't keep that information. Java's BasicFileAttributes.creationTime() will return a file's modification time on Linux. You want to get a nanosecond resolution timestamp? Well, you can, but only on some OSes. Or say you want to get the computer's hostname without resorting to a network lookup (which depends on a network being actually available), well, get ready for some hacking (this is my own answer by the way).
Now, to your more specific questions:
Is Java the right language for writing these applications & utilities (as mentioned above)?
I frankly don't know. I never tried defragmenting or restoring a file programmatically from Java. But since it involves some very low level filesystem operations, I suggest you do some serious reading before moving to Java. Check whether the APIs you need exist in the language itself or in some good libraries.
I understand Java will provide Libraries and classes to access system
resources / network / sockets but will Java abstraction be a hindrance
at some point (which would restrict the flexibility which C/C++
provide )?
Yes. For instance, it's impossible to open a raw socket using pure Java. And if I recall correctly, it's also impossible to set some socket options.
If for example I want to write a utility to repair a file system / or
retrieve data for Windows & Unix ...will I be using same API for both
OS or there are different API for different OS?
I never tried repairing a file system in Java, so I can't tell you about the APIs involved. But I find it hard to believe you'll find a pure Java api for doing low level stuff with the file system. You'll probably have to write your own native code (and run it through JNI) or use someone else's library (which probably uses JNI, like the raw socket library I mentioned earlier).

Questions about JNI on Android

I am planning to create a software that will be compatible with both Windows CE and Android devices.
Nothing has been decided yet, but so far I've imagined that I could write most parts of the program using C++ code that could be reused on these two platforms, except for system-dependant things like threads. C++ is highly recommended for performance in my case.
In the case of system-dependent things I would create interfaces that would be implemented in two different ways, one using the win32 API, and the other one using the linux equivalent. The other parts of the code the logic) would be independent and reused on both platforms.
The only part of the application that would not be written in C++ would be the user interface. Using the Android API on Android platforms, and something else on Windows CE (C#, Java, don't know yet).
I've read that Android is not like other linux distributions because many linux features are not available from native code on it.
So my question is : is it possible to natively create and use sockets, threads, critical sections (and any other system-dependant things) from a native linux api using JNI (i.e the equivalent of the win32 api but for linux) or do i always have to create them into the Java layer and pass them down to the native code ?
I've not yet decided how II am going to build this, i'm just informing myself on the different possibilities.
Thank you.

NDK implemented POSIX (include pthread, mutex) and BSD socket, so you don't necessary create them with java objects.
However, STL support is still crappy IMO, which you may need to pay attention with your own code or porting any dependency libraries.

It is possible to do all of this with JNI, but I would think twice before doing so. Using JNI has its share of liabilities; it will make development and debugging considerably more difficult. You will also end up with a lot of callbacks to Java code to communicate with the GUI. Think of features like visual progress indicators for ongoing operations. And don't forget that in the end, your app's performance may suffer because of the extra indirection required by each JNI call. Be sure to measure performance in either case. JNI, or C++ for that matter, does not automatically make anything faster.
Also, it may be harder than what you think to just "mirror" your app on another operating system. Android, as an operating system, behaves differently not only on the GUI side (it is, for example, nonsense to provide a back button within in an Android app), but also on the inside, with its typical app architecture broken down into Services, Activities and Receivers. It's not just "a different GUI", not at all.
If you have backend functionality that really can and should behave identically on both platforms, then go on, write it in C++ and reuse it on Android via JNI. But it may be easier, and it may be equally or more performant, to just implement it or parts of it from scratch on Android, using Java. Without knowing any details about your project, it is impossible to say.
Why don't you try and implement a simple test app on Android using JNI? Try to use the NDK to send an HTTP request in a background thread and write the respone back to Java as a String. You'll eventually see that it works, but you will also have a better understanding of the difficulties involved.

Java clipboard invoke paste operation

I would like to invoke a paste operation with my java application. Is this possible without using Robot?
For example, the application would invoke a paste operation every so often, so when I am writing in notepad, I would see the contents of my clipboard.
JAVA APP Notepad
clipboard.paste() --------> clipboardContents

It sounds as if you're trying to use Java to interact or partially drive another application (such as Windows NotePad) and paste to that application, and if so there are several possible solutions, one being use of Robot, another having Java make operating system calls, though this can't be done directly with just core Java and would require use of either JNI, JNA or other platform-specific non-core utility programs such as AutoIt (if this is for Windows).
Why are you dead set on not using Robot? Can you explain your needs in greater detail?
Edit
regarding your comment:
I want to explore alternatives to Robot, as my client irrationally rejects Robot.
Since this appears to be for a Windows platform, you might consider exploring the Windows API and the API for whatever non-Java program you're trying to drive (if one exists), and then using JNA to interact with it. The Windows User32.dll would allow you to get the Window handle (hWnd) of the application that you're trying to drive, which may be necessary for this to work.
It's hard to give more specific advice without more specific information from you about your problem though.
Edit 2
regarding your comments:
I would like the app to be platform independent.
Well, Robot comes to mind then. You might want to have a sit-down with your client to find out what they dislike so much about Robot, and then gently explain that it might offer the best path towards a platform-independent solution.
Are there examples for JNA and/or JNI? I'm not familiar with either.
Yes there are lots of examples on this and other sites, and Google will help you find out more. JNA is a bit easier to work with as it doesn't require you to create a C bridge program, but it can be a little slower than JNI, and doesn't work directly with C++ code (as far as I know).
Edit 3
regarding your comments:
I have a serial port listener (java app). I need to provide its contents onto a web browser. Clipboard seemed to be a way to do it.
And this is why it's so important for you to provide the context of your problem rather than what you think your code solution should be.
Communicating between applications is not an easy thing to do, and often Java is not the best tool for this since as it is designed to be as platform agnostic as possible, it does not provide tools that allow for easy integration with low-level OS functions. I don't know the best way to solve your problem, but my intuition tells me that using clipboard may not be the way to go. Much may depend on which web browser you're talking about, whether it has some sort of API that allows for interface with other programs, things I know little about. Also where is your program sitting? On the user's computer? Have you considered using a Java web browser library of some type, creating your own specialized web browser program, and obtaining the data directly from your serial port listener (again, I have not done this myself, but have seen it described on SO)?
I don't understand how bridging to a C program will help me.
I'm not suggesting this. This would only be needed if you used JNI, something I avoid since JNA is much easier (at least for me).

Determining if a Java app is malware

I am curious about what automatic methods may be used to determine if a Java app running on a Windows or PC is malware. (I don't really even know what exploits are available to such an app. Is there someplace I can learn about the risks?) If I have the source code, are there specific packages or classes that could be used more harmfully than others? Perhaps they could suggest malware?
Update: Thanks for the replies. I was interested in knowing if this would be possible, and it basically sounds totally infeasible. Good to know.

If it's not even possible to automatically determine whether a program terminates, I don't think you'll get much leverage in automatically determining whether an app does "naughty stuff".
Part of the problem of course is defining what constitutes malware, but the majority is simply that deducing proofs about the behaviour of other programs is surprisingly difficult/impossible. You may have some luck spotting particular patterns, but on the whole you can't be confident (and I suspect it's provably impossible) that you've caught all possible attack vectors.
And in the general sphere, catching 95% of vectors isn't really worthwhile when the attackers simply concentrate on the remaining 5%.

Well, there's always the fundamental philosophical question: what is a malware? It's code that was intended to do damage, or at least code that doesn't do what it claims to. How do you plan to judge intent based on libraries it uses?
Having said that, if you at least roughly know what the program is supposed to do, you can indeed find suspicious packages, things the program wouldn't normally need to access. Like network connections when the program is meant to run as a desktop app. But then the network connection could just be part of an autoupdate feature. (Is autoupdate itself a malware? Sometimes it feels like it is.)
Another indicator is if a program that ostensibly doesn't need any special privileges, refuses to run in a sandbox. And the biggest threat is if it tries to load a native library when it shouldn't need one.
But all these only make sense if you know what the code is supposed to do. An antivirus package might use very similar techniques to viruses, the only difference is what's on the label.

Here is a general outline for how you can bound the possible actions your java application can take. Basically you are testing to see if the java application is 'inert' (can't take harmful actions) and thus it probably not mallware.
This won't necessarily tell you mallware or not, as others have pointed out. The app could still do annoying things like pop-up windows. Perhaps the best indication, is to see if the application is digitally signed by an author you trust; if not -- be afraid.
You can disassemble the class files to determine which Java APIs the application uses; you are looking for points where the java app uses the OS. Since java uses a virtual machine, there are well defined points where a java application could take potentially harmful actions -- these are the 'gateways' to various OS calls (for example opening a socket or reading a file).
Its difficult to enumerate all the APIs, different functions which execute the same OS action should require the same Permission. But java's docs don't provide an exhaustive list.
Does the java app use any native libraries -- if so its a big red flag.
The JVM does not offer the ability to run arbitrary code, or use native system APIs; in particular it does not offer the ability to modify the registry (a typical action of PC mallware). The only way a java application can do this is via native libraries. Typically there is no need for a normal application written in java to use native code (unless it needs to use devices).
Check for System.loadLibrary() or System.load() or Runtime.loadLibrary() or Runtime.load(). This is how the VM loads native libraries.
Does it use the network or file system?
Look for use of java.io, java.net.
Does it make system calls (via Runtime.exec())
You can check for the use of java.lang.Runtime.exec() or ProcessBuilder.exec().
Does it try to control the keyboard / mouse?
You could also run the application in a restricted policy JVM (the instructions/tools for doing this are not as simple as they should be) and see what fails (see Oracle's security tutorial) -- note that disassembly is the only way to be sure, just because the app doesn't do anything harmful once, doesn't mean it won't in the future.
This definitely is not easy, and I was surprised to find how many places one needs to look at (for example several java functions load native libraries, not just one).

How to detect Java agents, JVMTI, etc

How does one secure the Java environment when running on a machine you don't control? What is to stop someone from creating a java agent or native JVMTI agent and dumping bytecode or re-writing classes to bypass licensing and/or other security checks? Is there any way to detect if any agents are running from Java code? From JNI? From a JVMTI agent?

If you don't control the environment, then I'm sorry - you're really stuck. Yes, you could look for trivial JVMTI agents via some sort of cmdline sniffing, but that's the least of your worries. Think about java/lang/Classloader.defineClass() being compromised directly. That's easy to do if you own the box - just replace the .class file in rt.jar. In fact, until JVMTI came around, that was a typical way that profilers and monitoring tools instrumented Java code.
Going back to JVMTI - the "Late attach" feature also allows for JVMTI agents to be loaded on the fly. That might not have happened when you scanned the first time around.
Bottom line - if someone can change the bytes of the JRE on disk, they can do anything they want. Is it ethical, no? Can they get caught? Possibly, but you'll never win the war.

It looks like I can go with a combination of checks inside some custom JNI native code.
1.) cmd line sniffing to search for agents.
2.) Ensure that the cmd-line parameter -XX:+DisableAttachMechanism exists. (this will prevent people from attaching to my running VM)

I remember I once made almost a silent Java Agent. I guess you better look for port scanners or something around that.

Java 2 security, signing of jars etc, gives some level of control over what gets loaded into your application.
However in the end if a malicious person has access to a machine such that they can write to disk then in all probability they have plenty of capacity to do harm without resorting to clever Java hacks.
Turn this round, in any language what can you do to detect Trojans?
Careful access control to the machines you care about is non-trivial but essential if you are serious about such issues. Security specialists may seem paranoid, but that often means that they really understand the risks.

If you can't control the platform, you can't control the software upon it.
Even if you could shut down all the avenues of inspection you've listed, Java is open source. They could just take the source code and recompile it with the necessary changes built-in.
Also, try to remember that while it is your code, it's their machine. They have a right to inspect your code to verify that running it on their machine does what they expect it to do, and doesn't perform "extra" actions which they might find undesirable. Less trustworthy companies in the past have scanned for non-relevant files, copied sensitive information back to their home servers, etc.

I would look at the command line and see, if there are any "-agent" parameters. All profilers, debuggers and other code modificators use this for introspection. You could also check for unusual jars on the bootclasspath, since those might also provide a threat (but be aware that you then also must deliver a custom JVM, since some software like Quicktime adds itself to the bootclasspath of ALL java apps running... (I couldn't belive my eyes when I saw that...))

Basically this is a loosing battle.
Have a look at how visualvm in the Sun JDK works and how it can attach to a running process and redefine whatever it pleases. It is extremely hard to detect that in a portable way, and unless you can do so, you might as well give up on this approach.
The question is, what is it you want to avoid?