Restrict native code functionality from Java - java

I have a java application which uses JNI in some parts to do some work. It follows the usual loading of DLL and then calling native methods of DLL. Is there any way we can restrict what native methods can do from the java application? For example, can we restrict DLLs not to open any files or not to open any sockets even if it has the code to do it? It can just forbid DLLs it loads for doing certain things, may be by loggin something or throwing an exception.

No you can't. The DLL gets loaded as a whole and then the Java side has no control on what the native code is doing.
One solution might be kind of man in the middle approach. This would involve coding a "shell" DLL that has the same interface as the original DLL. You tell Java to load a "shell" DLL for instance by putting it in a specific location and using the java.library.path property. Then the role of the "shell" DLL is to load the "true" DLL by sandboxing it and redirecting standard functions. This sounds like a lot of pain and this something that would happen in the native side on things, not from Java.

Edit 2021: today it's also relevant to point out that the sandbox to run Java in would likely be a virtual machine, in the cloud, Docker or what have you, in a locked down configuration.
I liked Gregory Pakosz' answer a lot. However, what you could do is sandbox the Java instance itself. Start the Java application itself in a restricted context.
In Windows or Unix you can create a user which is limited to a certain directory and only has access to some DLLs. Thus the DLL called from JNI can do whatever it wants, but it will not get very far, because the user the Java runs as can not do very much.
If your Java program needs to do privileged things, the Java side of it will have to talk to another program (Java or not) to do its' privileged things for it.
Just keep in mind, that if you can not trust the DLL, you can no longer trust the Java code either, since the DLL might have "hacked" the Java machine. On the other hand, no nasty stuff should be able to break out of the limits of the user they run as. (Barring misconfiguration or a bug in the OS.)

Normally you would run your application under the Java security Manager but I don't believe it has any effect on code running through the JNI.

You could implement some kind of setting that your JNI code could get. For example, on an UNIX system, you could create groups for special types of privileges, and check if the current user has the required privileges, else just return 0 or something.

Related

Mixed Code (native, managed): how does it (technically) interoperate?

I basically understand the idea of managed and native code and their difference. But how is it technically possible for them to communicate with each other? Imagine the following example:
I got some static or dynamic c++ library which is compiled for a specific platform. Now I write a Java Programm. Inside this code I call the library functions with the 'native' keyword. I build a jar file with the bytecode and the c++ library files will stay separate. The result will no longer be platform-independent.
But how does the java programm know if the called native methods exists?
How is the whole programmcode executed during runtime? I know that the bytecode will be interpreted or compiled with JIT.
How does this all fit in the sandboxing paradigm? Is the native code also executed inside the sandbox?
Does it work because both (java and c++) code is machine code in the end?
Maybe this is a dumb question. But I was always wondering...
EDIT: I got 3 good answers. really can't decide which helped me the most. But i will mark this question as answered to close this topic from my side.
It doesn't know until you call the method. The native code resides in a .DLL or .so; the java runtime looks for specific entry points that correspond to the native methods you created (if you're using JNI, there's a tool that can parse the methods and create function stubs that'll result in those entry points when compiled). If the wanted entry point is not there, an exception will be thrown.
The code generated by the JIT is not entirely self-suficient; it has to call external native code (both for low-level runtime routines or OS services) from time to time. The same mechanism is used to invoke the code for your native methods.
No. You can do everything you'd do in a pure C/C++ program there. The only things that'll stop it from doing any damage are external security measures you have (login privilege restrictions, other OS protections, security software, etc.) But the VM won't protect you.
No, JNI existed even before JIT appeared. The mechanism is the same, if the bytecode is being run by an interpreter, and you want this interpreter to invoke native code, you just need some logic in it to determine that a given method is "external" and should be called as native code. This information is contained in the compiled .class file, and when the interpreter or JIT loads it, it creates a memory representation that makes easy to direct the call upon a method lookup.
The JVM will check the libraries you defined and see if the method is there
Bytecode will be interpreted or JITted and a call to native code is added. This may include boxing/deboxing values and other things needed to convert the data into suitable format. The libraries have a certain interface which is explained to the Java compiler and it will produce the required interface logic.
Depends on the sandbox. By default native code is native code. It doesn't call Java APIs so the JVM cannot govern it in any way. But there may be other limitations, for example the JVM could run the native code with libraries that provide sandboxing, or the operating system might have a way of sandboxing.
It depends on what you mean. In the end anything the computer does is machine code, but it doesn't really matter in this case. What matters is the translation and execution part. That is the glue that makes everything work.
Think of the system as people. Person A only speaks Japanese, but wants to reserve a hotel in Paris. The receptionist B only speaks French. Person A can get a translator that will translate their commands to French, command receptionist B and in return translate what B produced into a form person A understands. This is the JNI part.
It depends on the platform. On Linux, Solaris, etc., the JRE uses dlopen. On Windows, it uses LoadLibraryEx and GetProcAddress. If the JRE is running in interpreted mode, it calls that function; in compiled mode, it compiles Java bytecode into native code that calls that function.
On all JREs I'm familiar with, you can't call a native function in a static library directly; only one in a dynamic library.
Native code doesn't have to be limited to a single platform; if it's standard C, you can probably compile it with a cross-compiler for every platform on which a JRE is available.

What are the colplications with using jni in a java applet?

This might be a silly question. I am in the process of writing a website. I have already weighed the benifits of using C/C++ in this java program, though every thing i am going to do using C can be implemented in java(Not easily). This code would be written as an applet to be run on the website. I would like to know if anyone could tell me if jni works across different os'es in an applet, and what complications could using jni in an applet pose?
IMO, this is a bad idea. (And this applies to using all forms of native code library in applets, not just JNI libraries.)
First complication is that this can only possibly work in a trusted applet. And (frankly) if a user says "OK" to dialog that asks if your applet should be trusted / run, they are probably making a big mistake. (Potentially nasty applets + potentially nasty native libraries == BIG RISK!)
Second complication is that you need to write, build, test, etcetera a different version of the native library for each and every combination of hardware AND OS platform you want your website to support. If you don't, your website won't work for some of your users.
If you have a corporate user base with a mandated COE and established trust relationships (e.g. preinstalled corporate certificates) these problems are more manageable, but there is still an issue in both cases.
Related:
Dynamically loading a native library inside Java applet

Java only Operating System

Are there any operating systems that when fully loaded only provide a java environment for console application execution? Ideally one which will automatically start programs when its loaded up
Here is a such thing with name: JavaOS
From my point of view better approach is to install a Linux distribution pack or small Linux distribution pack with java support and put your java code to the OS startup. To run it when OS has been started.
There's also the Phantom OS, being worked on by some Russian programmers. The system has no concept of files. Everything is an object living in one large memory space.
Even if it did exist, it would not by itself provide any security benefits.
A java application can still exercise almost all user-ring authority made available by the OS. If it can write somewhere on the library load path or start another java application with a specified library load path then it can:
Write out a native library to the file-system.
Define a class with native methods backed by that library.
Call System.loadLibrary to load that library.
Invoke those native method to execute arbitrary user-ring code.
If you want an operating system that allows you to effectively control the authority available via the shell, then read up on
"secure operating systems" like KeyKOS,
sandboxing processes using "virtual machines", and
providing a smaller shell interface using "restricted shells".

Determining if a Java app is malware

I am curious about what automatic methods may be used to determine if a Java app running on a Windows or PC is malware. (I don't really even know what exploits are available to such an app. Is there someplace I can learn about the risks?) If I have the source code, are there specific packages or classes that could be used more harmfully than others? Perhaps they could suggest malware?
Update: Thanks for the replies. I was interested in knowing if this would be possible, and it basically sounds totally infeasible. Good to know.
If it's not even possible to automatically determine whether a program terminates, I don't think you'll get much leverage in automatically determining whether an app does "naughty stuff".
Part of the problem of course is defining what constitutes malware, but the majority is simply that deducing proofs about the behaviour of other programs is surprisingly difficult/impossible. You may have some luck spotting particular patterns, but on the whole you can't be confident (and I suspect it's provably impossible) that you've caught all possible attack vectors.
And in the general sphere, catching 95% of vectors isn't really worthwhile when the attackers simply concentrate on the remaining 5%.
Well, there's always the fundamental philosophical question: what is a malware? It's code that was intended to do damage, or at least code that doesn't do what it claims to. How do you plan to judge intent based on libraries it uses?
Having said that, if you at least roughly know what the program is supposed to do, you can indeed find suspicious packages, things the program wouldn't normally need to access. Like network connections when the program is meant to run as a desktop app. But then the network connection could just be part of an autoupdate feature. (Is autoupdate itself a malware? Sometimes it feels like it is.)
Another indicator is if a program that ostensibly doesn't need any special privileges, refuses to run in a sandbox. And the biggest threat is if it tries to load a native library when it shouldn't need one.
But all these only make sense if you know what the code is supposed to do. An antivirus package might use very similar techniques to viruses, the only difference is what's on the label.
Here is a general outline for how you can bound the possible actions your java application can take. Basically you are testing to see if the java application is 'inert' (can't take harmful actions) and thus it probably not mallware.
This won't necessarily tell you mallware or not, as others have pointed out. The app could still do annoying things like pop-up windows. Perhaps the best indication, is to see if the application is digitally signed by an author you trust; if not -- be afraid.
You can disassemble the class files to determine which Java APIs the application uses; you are looking for points where the java app uses the OS. Since java uses a virtual machine, there are well defined points where a java application could take potentially harmful actions -- these are the 'gateways' to various OS calls (for example opening a socket or reading a file).
Its difficult to enumerate all the APIs, different functions which execute the same OS action should require the same Permission. But java's docs don't provide an exhaustive list.
Does the java app use any native libraries -- if so its a big red flag.
The JVM does not offer the ability to run arbitrary code, or use native system APIs; in particular it does not offer the ability to modify the registry (a typical action of PC mallware). The only way a java application can do this is via native libraries. Typically there is no need for a normal application written in java to use native code (unless it needs to use devices).
Check for System.loadLibrary() or System.load() or Runtime.loadLibrary() or Runtime.load(). This is how the VM loads native libraries.
Does it use the network or file system?
Look for use of java.io, java.net.
Does it make system calls (via Runtime.exec())
You can check for the use of java.lang.Runtime.exec() or ProcessBuilder.exec().
Does it try to control the keyboard / mouse?
You could also run the application in a restricted policy JVM (the instructions/tools for doing this are not as simple as they should be) and see what fails (see Oracle's security tutorial) -- note that disassembly is the only way to be sure, just because the app doesn't do anything harmful once, doesn't mean it won't in the future.
This definitely is not easy, and I was surprised to find how many places one needs to look at (for example several java functions load native libraries, not just one).

How to detect Java agents, JVMTI, etc

How does one secure the Java environment when running on a machine you don't control? What is to stop someone from creating a java agent or native JVMTI agent and dumping bytecode or re-writing classes to bypass licensing and/or other security checks? Is there any way to detect if any agents are running from Java code? From JNI? From a JVMTI agent?
If you don't control the environment, then I'm sorry - you're really stuck. Yes, you could look for trivial JVMTI agents via some sort of cmdline sniffing, but that's the least of your worries. Think about java/lang/Classloader.defineClass() being compromised directly. That's easy to do if you own the box - just replace the .class file in rt.jar. In fact, until JVMTI came around, that was a typical way that profilers and monitoring tools instrumented Java code.
Going back to JVMTI - the "Late attach" feature also allows for JVMTI agents to be loaded on the fly. That might not have happened when you scanned the first time around.
Bottom line - if someone can change the bytes of the JRE on disk, they can do anything they want. Is it ethical, no? Can they get caught? Possibly, but you'll never win the war.
It looks like I can go with a combination of checks inside some custom JNI native code.
1.) cmd line sniffing to search for agents.
2.) Ensure that the cmd-line parameter -XX:+DisableAttachMechanism exists. (this will prevent people from attaching to my running VM)
I remember I once made almost a silent Java Agent. I guess you better look for port scanners or something around that.
Java 2 security, signing of jars etc, gives some level of control over what gets loaded into your application.
However in the end if a malicious person has access to a machine such that they can write to disk then in all probability they have plenty of capacity to do harm without resorting to clever Java hacks.
Turn this round, in any language what can you do to detect Trojans?
Careful access control to the machines you care about is non-trivial but essential if you are serious about such issues. Security specialists may seem paranoid, but that often means that they really understand the risks.
If you can't control the platform, you can't control the software upon it.
Even if you could shut down all the avenues of inspection you've listed, Java is open source. They could just take the source code and recompile it with the necessary changes built-in.
Also, try to remember that while it is your code, it's their machine. They have a right to inspect your code to verify that running it on their machine does what they expect it to do, and doesn't perform "extra" actions which they might find undesirable. Less trustworthy companies in the past have scanned for non-relevant files, copied sensitive information back to their home servers, etc.
I would look at the command line and see, if there are any "-agent" parameters. All profilers, debuggers and other code modificators use this for introspection. You could also check for unusual jars on the bootclasspath, since those might also provide a threat (but be aware that you then also must deliver a custom JVM, since some software like Quicktime adds itself to the bootclasspath of ALL java apps running... (I couldn't belive my eyes when I saw that...))
Basically this is a loosing battle.
Have a look at how visualvm in the Sun JDK works and how it can attach to a running process and redefine whatever it pleases. It is extremely hard to detect that in a portable way, and unless you can do so, you might as well give up on this approach.
The question is, what is it you want to avoid?

Categories