I mean i need to pass the source code as input to a threads and let the thread compile and run the source code while it's already running ?
how could i pass a source code to map function in the mapperWordCount class , while the code of whole project is running , my program needs to take mapper and reducer code from scanner , and the code must be placed inside the map function , and reduce function, you can take a look for the code
by thread i mean not the main thread of java program .
class mapper extends Thread{
// some code
#Override
public void run() {
mapper.map(bucket,reader);
}
}
class mapperWordCount implement Mapper{
public void map ( Bucket bucket , Reader reader){
// code for word count mapper , this code must be entered by scanner as input , and compiled and run while the project is running
}
}
class reducer extends Thread{
// some code
#Override
public void run() {
Reducer.reduce(bucket,reader);
}
}
class reducerWordCount implement Reducer {
public void reduce ( Bucket bucket , MapOfKeysAndLists keyListOfKeysAndValues){
// code for reducer , this code must be entered by scanner as input , and compiled and run while the project is running
}
}
A big warning first: you must be sure of where the code you want to compile comes from, as that creates a huge potential for, well, code injection.
Another warning: if you compile classes, you can't reuse the same class name over and over so your ClassLoader will eventually eat up all of your RAM before OOME! That reason alone should make you find another way of doing that! Edit: since Java 8, there is no more PermGen space (where class metadata were stored) but a Metaspace. One of the difference being Metaspace is garbage-collected, when PermGen wasn't; so it should reduce the potential for OOME.
If you're just looking for a way to add a more dynamic configuration to your program, I would strongly suggest you look at ScriptEngine, which supports ECMAScript (very close to JavaScript) out-of-the-box (if you use Oracle's HotSpot JVM at least). It will save you the overhead of writing your own ClassLoader and compiling code.
That said, if you still want to continue in that direction, I'm just guessing here to give to a headstart, but you'll probably have to compile a file through a JavaCompiler object and inject the content of the compilation result into a ClassLoader.
If I had a computer to test, I would try something like:
Files[] files = ... ; // input for first compilation task
JavaCompiler compiler = ToolProvider.getSystemJavaCompiler();
StandardJavaFileManager fileManager = compiler.getStandardFileManager(null, null, null);
Iterable<? extends JavaFileObject> compilationUnits = fileManager.getJavaFileObjectsFromFiles(Arrays.asList(files));
compiler.getTask(null, fileManager, null, null, null, compilationUnits).call().get(); // Compile the class and wait for it to finish
Class<?> cls = null;
for (JavaFileObject compiledClass : compilationUnits) {
if (compiledClass.getKind() != Kind.CLASS)
continue;
int n;
byte[] classData;
try (InputStream is = compiledClass.openInputStream()) {
classData = new byte[1024]; // DO A PROPER FULL READ HERE!
n = is.read(classData);
} // catch Exceptions here
cls = myClassLoader.defineClass(className, classData, 0, n);
break;
}
if (cls != null) {
// Now, cls.newInstance() etc.
}
Apart from that being a complete try in the dark, you'll also have to define your own ClassLoader, as the defineClass() method is protected.
Alternatively, you could also spawn a call to javac using Runtime.exec() and reading bytecode bytes directly from the generated .class, or even maybe generate it directly into the classpath.
Related
UPDATE: After getting an unexpected-in-a-good-way answer, I've added some context to the bottom of this question, stating exactly how I'll be using these string-function-calls.
I need to translate a string such as
my.package.ClassName#functionName(1, "a string value", true)
into a reflective call to that function. Getting the package, class, and function name is not a problem. I have started rolling my own solution for parsing the parameter list, and determining the type of each and returning an appropriate object.
(I'm limiting the universe of types to the eight primitives, plus string. null would be considered a string, and commas and double-quotes must be strictly escaped with some simple marker, such as __DBL_QT__, to avoid complications with unescaping and splitting on the comma.)
I am not asking how to do this via string-parsing, as I understand how. It's just a lot of work and I'm hoping there's a solution already out there. Unfortunately it's such generic terminology, I'm getting nowhere with searching.
I understand asking for an external existing library is off topic for SO. I'm just hoping to get some feedback before it's shutdown, or even a suggestion on better search terms. Or perhaps, there is a completely different approach that might be suggested...
Thank you.
Context:
Each function call is found within a function's JavaDoc block, and represents a piece of example code--either its source code or its System.out output--which will be displayed in that spot.
The parameters are for customizing its display, such as
indentation,
eliminating irrelevant parts (like the license-block), and
for JavaDoc-linking the most important functions.
This customization is mostly for the source-code presentation, but may also be applied to its output.
(The first parameter is always an Appendable, which will do the actual outputting.)
The user needs to be be able to call any function, which in many cases will be a private-static function located directly below the JavaDoc-ed function itself.
The application I'm writing will read in the source-code file (the one containing the JavaDoc blocks, in which these string-function-calls exist), and create a duplicate of the *.java file, which will subsequently processed by javadoc.
So for every piece of example code, there will be likely two, and possibly more of these string-function-calls. There may be more, because I may want to show different slices of the same example, in different contexts--perhaps the whole example in the overall class JavaDoc block, and snippets from it in the relevant functions in that class.
I have already written the process that parses the source code (the source code containing the JavaDoc blocks, which is separate from the one that reads the example-code), and re-outputs its source-code blindly with insert example-code here and insert example-code-output here markers.
I'm now at the point where I have this string-function-call in an InsertExampleCode object, in a string-field. Now I need to do as described at the top of this question. Figure out which function they want to invoke, and do so.
Change the # to a dot (.), write a class definition around it so that you have a valid Java source file, include tools.jar in your classpath and invoke com.sun.tools.javac.Main.
Create your own instance of a ClassLoader to load the compiled class, and run it (make it implement a useful interface, such as java.util.concurrent.Callable so that you can get the result of the invocation easily)
That should do the trick.
The class I created for this, called com.github.aliteralmind.codelet.simplesig.SimpleMethodSignature, is a significant piece of Codelet, used to translate the "customizer" portion of each taglet, which is a function that customizes the taglet's output.
(Installation instructions. The only jars that must be in your classpath are codelet and xbnjava.)
Example string signatures, in taglets:
{#.codelet.and.out com.github.aliteralmind.codelet.examples.adder.AdderDemo%eliminateCommentBlocksAndPackageDecl()}
The customizer portion is everything following the percent sign (%). This customizer contains only the function name and empty parameters. This implies that the function must exist in one of a few, strictly-specified, set of classes.
{#.codelet.and.out com.github.aliteralmind.codelet.examples.adder.AdderDemo%lineRange(1, false, "Adder adder", 2, false, "println(adder.getSum())", "^ ")}
This specifies parameters as well, which are, by design, "simple"--either non-null strings, or a primitive type.
{#.codelet.and.out com.github.aliteralmind.codelet.examples.adder.AdderDemo%com.github.aliteralmind.codelet.examples.LineRangeWithLinksCompact#adderDemo_lineSnippetWithLinks()}
Specifies the explicit package and class in which the function exists.
Because of the nature of these taglets and how the string-signatures are implemented, I decided to stick with direct string parsing instead of dynamic compilation.
Two example uses of SimpleMethodSignature:
In this first example, the full signature (the package, class, and function name, including all its parameters) are specified in the string.
import com.github.aliteralmind.codelet.simplesig.SimpleMethodSignature;
import com.github.xbn.lang.reflect.InvokeMethodWithRtx;
import java.lang.reflect.Method;
public class SimpleMethodSigNoDefaults {
public static final void main(String[] ignored) {
String strSig = "com.github.aliteralmind.codelet.examples.simplesig." +
"SimpleMethodSigNoDefaults#getStringForBoolInt(false, 3)";
SimpleMethodSignature simpleSig = null;
try {
simpleSig = SimpleMethodSignature.newFromStringAndDefaults(
String.class, strSig, null, null,
null); //debug (on=System.out, off=null)
} catch(ClassNotFoundException cnfx) {
throw new RuntimeException(cnfx);
}
Method m = null;
try {
m = simpleSig.getMethod();
} catch(NoSuchMethodException nsmx) {
throw new RuntimeException(nsmx);
}
m.setAccessible(true);
Object returnValue = new InvokeMethodWithRtx(m).sstatic().
parameters(simpleSig.getParamValueObjectList().toArray()).invokeGetReturnValue();
System.out.println(returnValue);
}
public static final String getStringForBoolInt(Boolean b, Integer i) {
return "b=" + b + ", i=" + i;
}
}
Output:
b=false, i=3
This second example demonstrates a string signature in which the (package and) class name are not specified. The potential classes, one in which the function must exist, are provided directly.
import com.github.aliteralmind.codelet.simplesig.SimpleMethodSignature;
import com.github.xbn.lang.reflect.InvokeMethodWithRtx;
import java.lang.reflect.Method;
public class SimpleMethodSigWithClassDefaults {
public static final void main(String[] ignored) {
String strSig = "getStringForBoolInt(false, 3)";
SimpleMethodSignature simpleSig = null;
try {
simpleSig = SimpleMethodSignature.newFromStringAndDefaults(
String.class, strSig, null,
new Class[]{Object.class, SimpleMethodSigWithClassDefaults.class, SimpleMethodSignature.class},
null); //debug (on=System.out, off=null)
} catch(ClassNotFoundException cnfx) {
throw new RuntimeException(cnfx);
}
Method m = null;
try {
m = simpleSig.getMethod();
} catch(NoSuchMethodException nsmx) {
throw new RuntimeException(nsmx);
}
m.setAccessible(true);
Object returnValue = new InvokeMethodWithRtx(m).sstatic().
parameters(simpleSig.getParamValueObjectList().toArray()).invokeGetReturnValue();
System.out.println(returnValue);
}
public static final String getStringForBoolInt(Boolean b, Integer i) {
return "b=" + b + ", i=" + i;
}
}
Output:
b=false, i=3
I have this constructor:
public Revaluator(File model,PrintStream ps) {
modelFile=model;
rsession=Rsession.newInstanceTry(ps, null);
rsession.eval("library(e1071)");
rsession.load(modelFile);
}
i want to load a model and predict with it.
the problem that Rsession.newInstanceTry(ps, null); is always the same session, so if i load another model, like:
Revaluator re1=new Revaluator(new File("model1.RData"),System.out);
Revaluator re2=new Revaluator(new File("model2.RData"),System.out);
Both re1 and re2 using the same model, since the var name is model, so only the last one loaded.
the evaluate function:
public REXP evaluate(Object[] arr){
String expression=String.format("predict(model, c(%s))", J2Rarray(arr));
REXP ans=rsession.eval(expression);
return ans;
}
//J2Rarray just creates a string from the array like "1,2,true,'hello',false"
i need to load about 250 predictors, is there a way to get every instance of Rsession as a new separated R Session?
You haven't pasted all of your code in your question, so before trying the (complicated) way below, please rule out the simple causes and make sure that your fields modelFile and rsession are not declared static :-)
If they are not:
It seems that the way R sessions are created is OS dependent.
On Unix it relies on the multi-session ability of R itself, on Windows it starts with Port 6311 and checks if it is still free. If it's not, then the port is incremented and it checks again, if it's free and so on.
Maybe something goes wrong with checking free ports (which OS are you working on?).
You could try to configure the ports manually and explicitly start different local R servers like this:
Logger simpleLogger = new Logger() {
public void println(String string, Level level) {
if (level == Level.WARNING) {
p.print("! ");
} else if (level == Level.ERROR) {
p.print("!! ");
}
p.println(string);
}
public void close() {
p.close();
}
};
RserverConf serverConf = new RserverConf(null, staticPortCounter++, null, null, null);
Rdaemon server = new Rdaemon(serverConf, this);
server.start(null);
rsession = Rsession.newInstanceTry(serverConf);
If that does not work, please show more code of your Revaluator class and give details about which OS you are running on. Also, there should be several log outputs (at least if the log level is configured accordingly). Please paste the logged messages as well.
Maybe it could also help to get the source code of rsession from Google Code and use a debugger to set a breakpoint in Rsession.begin(). Maybe this can help figuring out what goes wrong.
I am new to java and I am following this tutorial as it is very informative and explains everything in great detail. At the bottom of the tutorial it explains how a JavaFileManager can be used to compile multiple java files and gives a few examples of this but i still cant get it to compile multiple files myself
Another problem is that in the example it only ever shows how to compile one java file (which i can already get working) but it is the multiple files that i am having a problem with as i want to be able to compile projects made up of multiple java classes in my own system
This is what i have to the moment:
public static void main(String[] args) throws Exception {
JavaCompiler compiler = ToolProvider.getSystemJavaCompiler();
// Line 1.
MyDiagnosticListener listener = new MyDiagnosticListener(); // Line 2.
StandardJavaFileManager fileManager = compiler.getStandardFileManager(
listener, null, null); // Line 3.
String fileToCompile = "test" + File.separator + "ManyErrors.java";
// Line 4
Iterable fileObjects = fileManager.getJavaFileObjectsFromStrings(Arrays
.asList(fileToCompile)); // Line 5
CompilationTask task = compiler.getTask(null, fileManager, listener,
null, null, fileObjects); // Line 6
Boolean result = task.call(); // Line 7
if (result == true) {
System.out.println("Compilation has succeeded");
}
}
class MyDiagnosticListener implements DiagnosticListener {
public void report(Diagnostic diagnostic) {
System.out.println("Code->" + diagnostic.getCode());
System.out.println("Column Number->" + diagnostic.getColumnNumber());
System.out.println("End Position->" + diagnostic.getEndPosition());
System.out.println("Kind->" + diagnostic.getKind());
System.out.println("Line Number->" + diagnostic.getLineNumber());
System.out.println("Message->" + diagnostic.getMessage(Locale.ENGLISH));
System.out.println("Position->" + diagnostic.getPosition());
System.out.println("Source" + diagnostic.getSource());
System.out.println("Start Position->" + diagnostic.getStartPosition());
System.out.println("\n");
}
getJavaFileObjectsFromStrings from StandardJavaFileManager takes an Iterable<String>.
This means you can just pass it any iterable collection of strings in order to get an Iterable<? extends JavaFileObject>, which in turn is passed to the getTask method of any class that implements the JavaCompiler interface.
This is not related to the answer, but I'd like to add that you are probably going down the wrong path, if your goal is to familiarize yourself with Java. Procedural compilation of Java classes is a rather advanced topic, and it doesn't seem you understood the code you posted fully, because the answer to your question is right within it: the Arrays.asList(fileToCompile) call creates an array of strings with exactly one string in it; even without documentation, it should be easy to deduce getJavaFileObjectsFromStrings takes an array of strings corresponding to filenames. So I really wouldn't try going down that road, but rather, I'd familiarize myself with Java documentation and simpler concepts first. Especially if you are not familiar with OO concepts.
I am trying to edit a class file and make it available to the JVM at runtime.
Is that possible, how can I do it?
For example:
main() {
for (int i=0;i<10;i++) {
NewClass.method();
//ask user if to continue..
}
}
public class NewClass() {
static void method() {
sysout("hi");
}
}
When this loop is executing, I want to change the file NewClass() and load it in the JVM, so that it prints "bye".
Here is the complete code:
try {
for (int iCount = 0; iCount < 10; iCount++) {
BufferedReader br = new BufferedReader(new InputStreamReader(
System.in));
System.out.print("Enter which method:");
int i = Integer.parseInt(br.readLine());
System.out.println(i);
if (i == 1) {
Called.calledMethod1();
} else {
Called.calledMethod2();
}
}
} catch (NumberFormatException nfe) {
System.err.println("Invalid Format!");
}
I want to run the main method of one class, and while it is running I want to refer to another class, edit it, and refer the 2nd class's same method and get different output.
I don't want to:
stop the jvm
edit the code
run the code again.
I want to
edit the code at run time, and get the changes reflected immediately.
You have to use Java reflection. Using reflection,
you load dynamically a class at runtime using Class forName method.
Using invoke method on the instance of Class you got,you can call any static method you want giving it's name.
If your class is fixed at design time, you just skeep the first point. Some code:
...
BufferedReader br = new BufferedReader(new InputStreamReader(System.in));
System.out.print("Enter which class:");
JavaCompiler compiler = ToolProvider.getSystemJavaCompiler();
className=br.readLine()
int results = compiler.run(null, null, null, className+".java");
if(results == 0){
Class clazz = Class.forName(className+".class");
System.out.print("Compiled successfully.Enter which method:");
Object returnValue=clazz.getMethod(br.readLine()).invoke(null);
}
...
But beware of security issues: the code above let anyone to execute each static method of each class accessible at runtime. If security is one of your concern, it's better if you first validate the input received from console and test that they match one of the class methods you whant to make available via console.
EDIT:
I better understand your question after I read your comment. In order to compile a class at runtime if you are targeting Java 6, you could use Java Compiler object. I have edit the code to include JavaCompiler usage.
This is a second part to my question here.
I now have a process but I want to know how to get the output from the process?
String filename = matlab.getfileName();
Process p = Runtime.getRuntime().exec("java -cp mediaProperty.java " + filename);
My mediaProperty.java:
public class mediaProperty {
public static Object main(String[] args) {
Object[] mediaProp = null;
java.util.List lstMedia = new ArrayList();
Media media = null;
try {
media = new Media();
lstMedia.add(args);
mediaProp = media.media(3, lstMedia);
} catch (Exception p) {
System.out.println("Exception: " + p.toString());
} finally {
MWArray.disposeArray(mediaProp);
if (media != null) {
media.dispose();
}
}
return mediaProp;
}
}
The mediaProperty.java will return an Object. Inside this is actually String array. How do I get the array? And is the way I'm calling exec() correct?
use public static void main (not Object as return type)
Serialize the object using ObjectOutputStream (all necessary examples are in the javadoc)
The only thing different from the example is the construction - construct it like
ObjectOutputStream oos = new ObjectOutputStream(System.out);
in the program calling exec(), get the output with process.getOutputStream()
Read in an ObjectInputStream based on the already retreived OutputStream (check this)
Deserialize the object (see the javadoc of ObjectInputStream)
Now, this is a weird way to do it, but as I don't know exactly what you are trying to achieve, it sounds reasonable.
You could do System.setOut(new PrintStream(p.getOutputStream())) if you'd like to have the process print its results directly to standard output. Of course, this will override the old standard output. But you could also do other things with the process's output stream, like have a thread that reads from it.
A problem with your code is that the main function of a class must be of type void, and will return nothing. You will not be able to pass Java objects between processes, as they are running in different JVMs. If you must do this you could serialize the object to disk, but I imagine you don't even need to run this in a separate process.
mediaProp is a local variable in your main() method. It's not accessible from the outside.
You'll have to redesign your mediaProperty class a bit.
First, you should use:
"java -cp . mediaProperty " + filename
for calling the java process. The "-cp ." defines the classpath and I have made the assumption that the java file is compiled and the generated class file is at the same path as the executing process.
Then, you need to print the result at the standard output and not just return it. Finally, read this article for reading the output.
Tip 1: Rename the class to MediaProperty
Tip 2: Why you don't call the MediaProperty class directly from your code? Is it necessary to start a new process?
There are a few gotcha's.
In exec you assume that java is on the path, and the filename should be fully qualified or you should know that the current working dir of the java process is OK.
main() should return void (nothing). If you want to pass the results out of your program use something like:
for (Object o : mediaProp) {
System.out.println(o);
}
and parse it again on the input stream (the calling software).
Better yet, include the MediaProperty class in the java path and call main(...) directly in stead of calling a separate java process.