We Keep Coding

How to create and add a Java class in an APK using Soot?

I want to instrument an APK to count the number of executions of its callback methods (similar to this SO post).
To do so, I want to create a static class from scratch (similar to this link) which keeps the counting numbers. Then add this class to the APK, and finally instrument the beginning of callback methods to call a method of this static class (similar to this link).
I was able to do a similar thing with a Java application but I couldn't do it for an APK. Here is the part of the code that generates a new SootClass and adds it to Scene:
SootClass staticCls = generateClass("StaticCounter");
Scene.v().addClass(staticCls);
Scene.v().addBasicClass("StaticCounter", SootClass.SIGNATURES);
Scene.v().loadNecessaryClasses();
generateClass is adapted from here. But when I examined the modified APK, there was no sign of StaticCounter class. Moreover, I installed and ran the modified APK and got this error:
08-21 14:15:45.936 19917 19917 E AndroidRuntime:
Caused by: java.lang.ClassNotFoundException:
Didn't find class "StaticCounter" on path:
DexPathList[[zip file "/system/framework/org.apache.http.legacy.boot.jar", zip file "/data/app/APKNAME-RdhHiJzxKyyUHh_KCi6RUA==/base.apk"],nativeLibraryDirectories=[/data/app/APKNAME-RdhHiJzxKyyUHh_KCi6RUA==/lib/x86, /system/lib]]
Which simply shows the generated class is not inside the APK.
Any solution to this problem?

You might need to provide the dependency it demands:
android {
useLibrary "org.apache.http.legacy"
}
The reason why it won't build might be, that StaticCounter is unknown at compile time - because it likely should not be referenced from within the application package, which does not know it. Check out this blog post, apparently from the Soot author... the driver class might be key to success:
//prefer Android APK files// -src-prec apk
Options.v().set_src_prec(Options.src_prec_apk);
//output as APK, too//-f J
Options.v().set_output_format(Options.output_format_dex);
// resolve the PrintStream and System soot-classes
Scene.v().addBasicClass("java.io.PrintStream", SootClass.SIGNATURES);
Scene.v().addBasicClass("java.lang.System", SootClass.SIGNATURES);
here's the whole example: AndroidInstrument.java

Soot will not emit the class unless you have flagged it as application class. Can you try that?

I guess I found why the newly generated class was unknown to the app thanks to Martin Zitler's hint. The generated class must be in the package of the APK. Soot handles the package of a class by looking at the signatures of the class. As a result, the StaticCounter class must have a signature like
com.example.app.StaticCounter
which indicates that StaticCounter is in the package com.example.app.
It is possible to find the package name using ProcessManifest class which can be found in FlowDroid:
new ProcessManifest(apkPath).getPackageName();
I simplified the working code and put it here. It creates a new class from scratch and adds a static integer field and a method that the field is the times that the static method is called, and the method prints this value. Then using a transformation pack, every method in the app will call this static method.

Difficulty with Octave's "javaMethod"

In this question, I was trying to import java classes into Octave. In my particular example, I was (and am) working with javaplex, a set of java tools with code for implementation in Matlab. The answer to the question shows that, whereas in Matlab you would do the following:
import edu.stanford.math.plex4.*;
api.Plex4.createExplicitSimplexStream();
The answer provided in the question showed that the way to do this in Octave is
javaMethod( 'createExplicitSimplexStream', 'edu.stanford.math.plex4.api.Plex4')
This was working excellently, but then I ran into a strange problem. There is another method called createVietorisRipsStream. In Matlab, I would run this with a line such as the following:
api.Plex4.createVietorisRipsStream(parameters);
So I would think that the equivalent command in Octave would be
javaMethod( 'createVietorisRipsStream', 'edu.stanford.math.plex4.api.Plex4')
However, when I do this, I get the following error:
error: [java] java.lang.NoSuchMethodException: createVietorisRipsStream
I'm not sure why this error is coming up, and both are in the same JAVA file ('Plex4'). I did take a look at the Plex4 file, and there are two differences between createExplicitSimplexStream and createVietorisRipsStream that I noticed:
There are two instances of createExplicitSimplexStream and six instances of createVietorisRipsStream
There is bit that says <double[]>. I don't know if that is relevant however (I haven't read or wrote much java, up to this point, I've been able to use the tutorial they provided to only use Matlab and not have to look under the hood).
Here is one example of the code from the Plex4 file for a createExplicitSimplexStream:
public static ExplicitSimplexStream createExplicitSimplexStream(double maxFiltrationValue) {
return new ExplicitSimplexStream(maxFiltrationValue);
}
Here is one example of the code from the Plex4 file for a createVietorisRipsStream:
public static VietorisRipsStream<double[]> createVietorisRipsStream(double[][] points, int maxDimension, double maxFiltrationValue, int numDivisions) {
return FilteredStreamInterface.createPlex4VietorisRipsStream(points, maxDimension, maxFiltrationValue, numDivisions);
}
Any idea of why I'm getting the error I'm getting?

Read the octave documentation for the Java section properly, it's only 4 pages, and it explains this well!
As I mentioned in the comments in the previous question, the way to call a java method with arguments is:
javamethod(
name of method as a string,
name of class fully qualified with packages as a string,
method's first argument,
method's second argument,
... etc
)
This is the only way to call 'static' methods; with normal 'instance' methods, you can either use javaMethod and replace the name of the class by the java object itself, or simply use it as you would in java, i.e. objectname.methodname(arg1, arg2, ... etc)
I have implemented here the tutorial for you to have a look at (page 14 in the pdf). (don't forget to run the modified 'load_javaplex' script first).
octave:2> max_dimension = 3;
octave:3> max_filtration_value = 4;
octave:4> num_divisions = 1000;
octave:5> point_cloud = javaMethod( 'getHouseExample', 'edu.stanford.math.plex4.examples.PointCloudExamples')
point_cloud =
<Java object: double[][]>
octave:6> stream = javaMethod( 'createVietorisRipsStream', 'edu.stanford.math.plex4.api.Plex4', point_cloud, max_dimension, max_filtration_value, num_divisions)
stream =
<Java object: edu.stanford.math.plex4.streams.impl.VietorisRipsStream>
octave:7> persistence = javaMethod( 'getModularSimplicialAlgorithm', 'edu.stanford.math.plex4.api.Plex4', max_dimension, 2)
persistence =
<Java object: edu.stanford.math.plex4.autogen.homology.IntAbsoluteHomology>
octave:8> intervals = persistence.computeIntervals(stream)
intervals =
<Java object: edu.stanford.math.plex4.homology.barcodes.BarcodeCollection>
(I have not gone further because plot_barcodes needs to be modified a bit too; it's only a couple of lines but it would be too much to post here, the reasoning is the same though).
Also, if you're not sure what is meant by class constructors, class methods, and static vs instance-specific methods, unfortunately this is more to do with java, although it should be pretty introductory stuff. It is well worth reading up a bit about it first.
Good luck!

Java code in different build environments [duplicate]

My project requires Java 1.6 for compilation and running. Now I have a requirement to make it working with Java 1.5 (from the marketing side). I want to replace method body (return type and arguments remain the same) to make it compiling with Java 1.5 without errors.
Details: I have an utility class called OS which encapsulates all OS-specific things. It has a method
public static void openFile(java.io.File file) throws java.io.IOException {
// open the file using java.awt.Desktop
...
}
to open files like with double-click (start Windows command or open Mac OS X command equivalent). Since it cannot be compiled with Java 1.5, I want to exclude it during compilation and replace by another method which calls run32dll for Windows or open for Mac OS X using Runtime.exec.
Question: How can I do that? Can annotations help here?
Note: I use ant, and I can make two java files OS4J5.java and OS4J6.java which will contain the OS class with the desired code for Java 1.5 and 1.6 and copy one of them to OS.java before compiling (or an ugly way - replace the content of OS.java conditionally depending on java version) but I don't want to do that, if there is another way.
Elaborating more: in C I could use ifdef, ifndef, in Python there is no compilation and I could check a feature using hasattr or something else, in Common Lisp I could use #+feature. Is there something similar for Java?
Found this post but it doesn't seem to be helpful.
Any help is greatly appreciated. kh.

Nope there isn't any support for conditional compilation in Java.
The usual plan is to hide the OS specific bits of your app behind an Interface and then detect the OS type at runtime and load the implementation using Class.forName(String).
In your case there no reason why you can't compile the both OS* (and infact your whole app) using Java 1.6 with -source 1.5 -target 1.5 then in a the factory method for getting hold of OS classes (which would now be an interface) detect that java.awt.Desktop
class is available and load the correct version.
Something like:
public interface OS {
void openFile(java.io.File file) throws java.io.IOException;
}
public class OSFactory {
public static OS create(){
try{
Class.forName("java.awt.Desktop");
return new OSJ6();
}catch(Exception e){
//fall back
return new OSJ5();
}
}
}

Hiding two implementation classes behind an interface like Gareth proposed is probably the best way to go.
That said, you can introduce a kind of conditional compilation using the replace task in ant build scripts. The trick is to use comments in your code which are opened/closed by a textual replacement just before compiling the source, like:
/*{{ Block visible when compiling for Java 6: IFDEF6
public static void openFile(java.io.File file) throws java.io.IOException {
// open the file using java.awt.Desktop
...
/*}} end of Java 6 code. */
/*{{ Block visible when compiling for Java 5: IFDEF5
// open the file using alternative methods
...
/*}} end of Java 5 code. */
now in ant, when you compile for Java 6, replace "IFDEF6" with "*/", giving:
/*{{ Block visible when compiling for Java 6: */
public static void openFile(java.io.File file) throws java.io.IOException {
// open the file using java.awt.Desktop
...
/*}} end of Java 6 code. */
/*{{ Block visible when compiling for Java 5, IFDEF5
public static void openFile(java.io.File file) throws java.io.IOException {
// open the file using alternative methods
...
/*}} end of Java 5 code. */
and when compiling for Java 5, replace "IFDEF5". Note that you need to be careful to use // comments inside the /*{{, /*}} blocks.

You can make the calls using reflection and compile the code with Java 5.
e.g.
Class clazz = Class.forName("java.package.ClassNotFoundInJavav5");
Method method = clazz.getMethod("methodNotFoundInJava5", Class1.class);
method.invoke(args1);
You can catch any exceptions and fall back to something which works on Java 5.

The Ant script introduced below gives nice and clean trick.
link: https://weblogs.java.net/blog/schaefa/archive/2005/01/how_to_do_condi.html
in example,
//[ifdef]
public byte[] getBytes(String parameterName)
throws SQLException {
...
}
//[enddef]
with Ant script
<filterset begintoken="//[" endtoken="]">
<filter token="ifdef" value="${ifdef.token}"/>
<filter token="enddef" value="${enddef.token}"/>
</filterset>
please go to link above for more detail.

In java 9 it's possible to create multi-release jar files. Essentially it means that you make multiple versions of the same java file.
When you compile them, you compile each version of the java file with the required jdk version. Next you need to pack them in a structure that looks like this:
+ com
+ mypackage
+ Main.class
+ Utils.class
+ META-INF
+ versions
+ 9
+ com
+ mypackage
+ Utils.class
In the example above, the main part of the code is compiled in java 8, but for java 9 there is an additional (but different) version of the Utils class.
When you run this code on the java 8 JVM it won't even check for classes in the META-INF folder. But in java 9 it will, and will find and use the more recent version of the class.

I'm not such a great Java expert, but it seems that conditional compilation in Java is supported and easy to do. Please read:
http://www.javapractices.com/topic/TopicAction.do?Id=64
Quoting the gist:
The conditional compilation practice is used to optionally remove chunks of code from the compiled version of a class. It uses the fact that compilers will ignore any unreachable branches of code.
To implement conditional compilation,
define a static final boolean value as a non-private member of some class
place code which is to be conditionally compiled in an if block which evaluates the boolean
set the value of the boolean to false to cause the compiler to ignore the if block; otherwise, keep its value as true
Of course this lets us to "compile out" chunks of code inside any method. To remove class members, methods or even entire classes (maybe leaving only a stub) you would still need a pre-processor.

if you don't want conditionally enabled code blocks in your application then a preprocessor is only way, you could take a look at java-comment-preprocessor which can be used for both maven and ant projects
p.s.
also I have made some example how to use preprocessing with Maven to build JEP-238 multi-version JAR without duplication of sources

Java Primitive Specializations Generator supports conditional compilation:
/* if Windows compilingFor */
start();
/* elif Mac compilingFor */
open();
/* endif */
This tool has Maven and Gradle plugins.

hi I have got similar problem when I have shared library between Java SDK abd Android and in both environments are used the graphics so basically my code must to work with both
java.awt.Graphics and android.graphics.Canvas,
but I don't want to duplicate almost any code.
My solution is to use wrapper, so I access to graphisc API indirectl way, and
I can change a couple of imports, to import the wrapper I want to compile the projects.
The projects have some cone shaded and some are separate, but there is no duplicating anything except of couple of wrappers etc.
I think it is the best what I can do.

How to call a user defined Matlab from Java using matlabcontrol.jar

I am trying to call a user defined Matlab Function(M file) which takes 3 arguments(Java Strings) from my Java application which is developed in Eclipse. At the moment I am able to call proxy.eval and proxy.feval methods with the functions/commands like disp or sqr. But when i try to invoke a user-defined function it says on the matlab console that there is no such function defined like that and on the Java console MatlabInvocationException occurs.
Then I tried with a simple user-defined function which takes no arguments and just has single line disp('Hello') but still the result is same. So I think rather than a type conversion problem there is something wrong with how user-defined functions are getting invoked.
Please can anyone help me soon? I am meeting the deadline very soon for this project. I would be so thankful if someone can come up with a solution. (Mr Joshuwa Kaplan, is there any guide on solving an issue like this in your posts? I tried but found nothing)
Thanks in advance

You must have any user-defined m-files on the MATLAB search path, just as if you were working normally inside MATLAB.
I tested with the following example:
C:\some\path\myfunc.m
function myfunc()
disp('hello from MYFUNC')
end
HelloWorld.java
import matlabcontrol.*;
public class HelloWorld
{
public static void main(String[] args)
throws MatlabConnectionException, MatlabInvocationException
{
// create proxy
MatlabProxyFactoryOptions options =
new MatlabProxyFactoryOptions.Builder()
.setUsePreviouslyControlledSession(true)
.build();
MatlabProxyFactory factory = new MatlabProxyFactory(options);
MatlabProxy proxy = factory.getProxy();
// call builtin function
proxy.eval("disp('hello world')");
// call user-defined function (must be on the path)
proxy.eval("addpath('C:\\some\\path')");
proxy.feval("myfunc");
proxy.eval("rmpath('C:\\some\\path')");
// close connection
proxy.disconnect();
}
}
We compile and run the Java program:
javac -cp matlabcontrol-4.0.0.jar HelloWorld.java
java -cp ".;matlabcontrol-4.0.0.jar" HelloWorld
a MATLAB session will open up, and display the output:
hello world
hello from MYFUNC
You could also add your folder to the path once, then persist it using SAVEPATH. That way you won't have to do it each time.

Determine location of a java class loaded by Matlab

When using java from Matlab, is there some way to figure out from where in matlab's java class path is a class being loaded? I'm trying to diagnose a error caused by conflicting versions of the same class being used simultaneously.
Specifically, the class I'm looking for is org.apache.lucene.store.FSDirectory. It seems to be used by one of the matlab toolboxes, but I don't know which one.

From http://www.exampledepot.com/egs/java.lang/ClassOrigin.html
// Get the location of this class
Class cls = this.getClass();
ProtectionDomain pDomain = cls.getProtectionDomain();
CodeSource cSource = pDomain.getCodeSource();
URL loc = cSource.getLocation(); // file:/c:/almanac14/examples/

Assuming that an URLClassLoader is being used, you can get the file: URL of the class file like this:
ProblemClass.class.getResource("ProblemClass.class")

Per Thorbjørn Ravn Andersen, if j reference to a java object in Matlab, its location can be retrieved with the following line of matlab code:
j.getClass().getProtectionDomain().getCodeSource().getLocation().getPath()

Use the inmem function as follows:
[M,X,J] = inmem
This function returns the list of Java classes in the output argument J. (It also returns the names of all currently loaded M-files in M, and the names of all currently loaded MEX-files in X.)
Here's a sample of output from the inmem function:
[m,x,j] = inmem;
MATLAB displays:
j =
'java.util.Date'
'com.mathworks.ide.desktop.MLDesktop'

Since 1.5 using:
java -verbose:class
Prints where each class was loaded from.