We Keep Coding

when invoking a java method by method.invoke() its not returning any value instead print in the console log

Hello please help me to find a solution for this problem, I'm creating an web compiler for java programs I have implemented the java inbuilt compiler class to compile the code and its also giving the output but it's giving output in the console and I want that output in the return object or some string variable so I can display the output in front. I'm attaching full code.
package online_compiler.web;
import java.io.BufferedReader;
import java.io.Console;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.io.StringWriter;
import java.lang.reflect.InvocationTargetException;
import java.lang.reflect.Method;
import java.net.MalformedURLException;
import java.net.URI;
import java.net.URL;
import java.net.URLClassLoader;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Iterator;
import java.util.NoSuchElementException;
import javax.tools.JavaCompiler;
import javax.tools.JavaFileObject;
import javax.tools.SimpleJavaFileObject;
import javax.tools.ToolProvider;
public class compiler {
public static void myMethod(String name) throws ClassNotFoundException, NoSuchMethodException, SecurityException, IllegalAccessException, IllegalArgumentException, InvocationTargetException
{
int result;
JavaCompiler compiler = ToolProvider.getSystemJavaCompiler();
String program = "public class Main{" + " public static void main (String [] args){"
+ " System.out.println (\"Hello, World\");"
+ " }" + "}";
JavaSourceFromString jsfs = new JavaSourceFromString("Main", program);
Iterable<? extends JavaFileObject> fileObjects = Arrays.asList( jsfs);
java.util.List<String> options = new ArrayList<String>();
options.add("-d");
options.add("C:\\Program Files\\Java\\jdk-12.0.2\\bin");
options.add( "-classpath");
URLClassLoader urlClassLoader =
(URLClassLoader)Thread.currentThread().getContextClassLoader();
StringBuilder sb = new StringBuilder();
for (URL url : urlClassLoader.getURLs()) {
sb.append(url.getFile()).append(java.io.File.pathSeparator);
}
sb.append("C:\\\\Program Files\\\\Java\\\\jdk-12.0.2\\\\bin");
options.add(sb.toString());
StringWriter output = new StringWriter();
boolean success = compiler.getTask( output, null, null, options, null, fileObjects).call();
if(success) {
name = output.toString();
System.out.print(output);
java.io.File root = new java.io.File("C:\\\\Program Files\\\\Java\\\\jdk-12.0.2\\\\bin");
URLClassLoader classLoader;
try {
classLoader = URLClassLoader.newInstance(new URL[] { root.toURI().toURL() });
Object obj = null;
Class<?> cls = Class.forName("Main", true, classLoader);
try {
obj = cls.newInstance();
} catch (InstantiationException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
Method m = cls.getMethod("main", new Class[] { String[].class });
Object[] _args = new Object[] { new String[0] };
Object retobj = m.invoke(obj, _args);
System.out.println(retobj);
} catch (MalformedURLException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
//logger.info( LOG_PREFIX + "Class has been successfully compiled");
} else {
//throw new Exception( "Compilation failed :" + output);
}
}
static Iterable<JavaSourceFromString> getJavaSourceFromString(String code) {
final JavaSourceFromString jsfs;
jsfs = new JavaSourceFromString("code", code);
return new Iterable<JavaSourceFromString>() {
public Iterator<JavaSourceFromString> iterator() {
return new Iterator<JavaSourceFromString>() {
boolean isNext = true;
public boolean hasNext() {
return isNext;
}
public JavaSourceFromString next() {
if (!isNext)
throw new NoSuchElementException();
isNext = false;
return jsfs;
}
public void remove() {
throw new UnsupportedOperationException();
}
};
}
};
}
}
class JavaSourceFromString extends SimpleJavaFileObject {
final String code;
JavaSourceFromString(String name, String code) {
super(URI.create("string:///" + name.replace('.', '/') + Kind.SOURCE.extension), Kind.SOURCE);
this.code = code;
}
public CharSequence getCharContent(boolean ignoreEncodingErrors) {
return code;
}
}
The main driver code is this
enter image description here
output in the console log
I have tried many things searching on internet but nothing worked. I want that output in an object or a string variable so I can use that output or that string to front end.

The method you call is static void main. It doesn't return any value.
If you create a method that actually returns something, it should work better

Zeppelin: How to create DataFrame from within custom interpreter?

I am developing a custom interpreter for a domain specific language. Based on the example given in the Apache Zeppelin documentation (https://zeppelin.incubator.apache.org/docs/latest/development/writingzeppelininterpreter.html), the interpreter works pretty well. Now I want to store some results in a new DataFrame.
I found code to create DataFrames (http://spark.apache.org/docs/latest/sql-programming-guide.html), but I cant use this in my interpreter because I basically dont find a way to access a valid runtime SparkContext (often called "sc") from within my custom interpreter.
I tried (static) SparkContext.getOrCreate() but this even led to a ClassNotFoundException. Then I added the whole zeppelin-spark-dependencies...jar to my interpreter folder, which solved the class loading issue but now I am getting a SparkException ("master url must be set...").
Any idea how I could get access to my Notebook's SparkContext from within the custom interpreter? Thanks a lot!
UPDATE
Thanks to Kangrok Lee's comment below, my code now looks as follows: see below. It runs and seems to create a DataFrame (at least it doesnt throw any Exception any more). But I can not consume the created DataFrame in a subsequent SQL paragraph (the first paragraph uses my "%opl" interpreter, as given below, that should create the "result" DataFrame):
%opl
1 2 3
> 1
> 2
> 3
%sql
select * from result
> Table not found: result; line 1 pos 14
So probably there is still something wrong with my way of dealing with the SparkContext. Any ideas? Thanks a lot!
package opl;
import java.io.ByteArrayOutputStream;
import java.io.PrintStream;
import java.util.ArrayList;
import java.util.List;
import java.util.Properties;
import org.apache.spark.SparkContext;
import org.apache.spark.sql.DataFrame;
import org.apache.spark.sql.Row;
import org.apache.spark.sql.RowFactory;
import org.apache.spark.sql.SQLContext;
import org.apache.spark.sql.types.DataTypes;
import org.apache.spark.sql.types.StructType;
import org.apache.zeppelin.interpreter.Interpreter;
import org.apache.zeppelin.interpreter.InterpreterContext;
import org.apache.zeppelin.interpreter.InterpreterPropertyBuilder;
import org.apache.zeppelin.interpreter.InterpreterResult;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
public class OplInterpreter2 extends Interpreter {
static {
Interpreter.register("opl","opl",OplInterpreter2.class.getName(),
new InterpreterPropertyBuilder()
.add("spark.master", "local[4]", "spark.master")
.add("spark.app.name", "Opl Interpreter", "spark.app.name")
.add("spark.serializer", "org.apache.spark.serializer.KryoSerializer", "spark.serializer")
.build());
}
private Logger logger = LoggerFactory.getLogger(OplInterpreter2.class);
private void log(Object o) {
if (logger != null)
logger.warn("OplInterpreter2 "+o);
}
public OplInterpreter2(Properties properties) {
super(properties);
log("CONSTRUCTOR");
}
#Override
public void open() {
log("open()");
}
#Override
public void cancel(InterpreterContext arg0) {
log("cancel()");
}
#Override
public void close() {
log("close()");
}
#Override
public List<String> completion(String arg0, int arg1) {
log("completion()");
return new ArrayList<String>();
}
#Override
public FormType getFormType() {
log("getFormType()");
return FormType.SIMPLE;
}
#Override
public int getProgress(InterpreterContext arg0) {
log("getProgress()");
return 100;
}
#Override
public InterpreterResult interpret(String string, InterpreterContext context) {
log("interpret() "+string);
PrintStream oldSys = System.out;
try {
ByteArrayOutputStream baos = new ByteArrayOutputStream();
PrintStream ps = new PrintStream(baos);
System.setOut(ps);
execute(string);
System.out.flush();
System.setOut(oldSys);
return new InterpreterResult(
InterpreterResult.Code.SUCCESS,
InterpreterResult.Type.TEXT,
baos.toString());
} catch (Exception ex) {
System.out.flush();
System.setOut(oldSys);
return new InterpreterResult(
InterpreterResult.Code.ERROR,
InterpreterResult.Type.TEXT,
ex.toString());
}
}
private void execute(String code) throws Exception {
SparkContext sc = SparkContext.getOrCreate();
SQLContext sqlc = SQLContext.getOrCreate(sc);
StructType structType = new StructType().add("value",DataTypes.IntegerType);
ArrayList<Row> list = new ArrayList<Row>();
for (String s : code.trim().split("\\s+")) {
int value = Integer.parseInt(s);
System.out.println(value);
list.add(RowFactory.create(value));
}
DataFrame df = sqlc.createDataFrame(list,structType);
df.registerTempTable("result");
}
}

Finally I found a solution although I don't think this is a very nice one. In the code below, I am using a function getSparkInterpreter() that I found in org.apache.zeppelin.spark.PySparkInterpreter.java.
This requires that I put my packaged code (jar) into the Spark interpreter folder, instead of its own interpreter folder, which I believe should be the preferred way (according to https://zeppelin.incubator.apache.org/docs/latest/development/writingzeppelininterpreter.html). Also, my interpreter does not show up in Zeppelin's interpreter configuration page as an interpreter of its own. But it can be used in a Zeppelin paragraph nevertheless.
And: In the code I can create a DataFrame and this is also consumable outside my paragraph -- which is what I wanted to achieve.
package opl;
import java.io.ByteArrayOutputStream;
import java.io.PrintStream;
import java.util.ArrayList;
import java.util.List;
import java.util.Properties;
import org.apache.spark.sql.DataFrame;
import org.apache.spark.sql.Row;
import org.apache.spark.sql.RowFactory;
import org.apache.spark.sql.SQLContext;
import org.apache.spark.sql.types.DataTypes;
import org.apache.spark.sql.types.StructType;
import org.apache.zeppelin.interpreter.Interpreter;
import org.apache.zeppelin.interpreter.InterpreterContext;
import org.apache.zeppelin.interpreter.InterpreterPropertyBuilder;
import org.apache.zeppelin.interpreter.InterpreterResult;
import org.apache.zeppelin.interpreter.LazyOpenInterpreter;
import org.apache.zeppelin.interpreter.WrappedInterpreter;
import org.apache.zeppelin.spark.SparkInterpreter;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
public class OplInterpreter2 extends Interpreter {
static {
Interpreter.register(
"opl",
"spark",//"opl",
OplInterpreter2.class.getName(),
new InterpreterPropertyBuilder()
.add("sth", "defaultSth", "some thing")
.build());
}
private Logger logger = LoggerFactory.getLogger(OplInterpreter2.class);
private void log(Object o) {
if (logger != null)
logger.warn("OplInterpreter2 "+o);
}
public OplInterpreter2(Properties properties) {
super(properties);
log("CONSTRUCTOR");
}
#Override
public void open() {
log("open()");
}
#Override
public void cancel(InterpreterContext arg0) {
log("cancel()");
}
#Override
public void close() {
log("close()");
}
#Override
public List<String> completion(String arg0, int arg1) {
log("completion()");
return new ArrayList<String>();
}
#Override
public FormType getFormType() {
log("getFormType()");
return FormType.SIMPLE;
}
#Override
public int getProgress(InterpreterContext arg0) {
log("getProgress()");
return 100;
}
#Override
public InterpreterResult interpret(String string, InterpreterContext context) {
log("interpret() "+string);
PrintStream oldSys = System.out;
try {
ByteArrayOutputStream baos = new ByteArrayOutputStream();
PrintStream ps = new PrintStream(baos);
System.setOut(ps);
execute(string);
System.out.flush();
System.setOut(oldSys);
return new InterpreterResult(
InterpreterResult.Code.SUCCESS,
InterpreterResult.Type.TEXT,
baos.toString());
} catch (Exception ex) {
System.out.flush();
System.setOut(oldSys);
return new InterpreterResult(
InterpreterResult.Code.ERROR,
InterpreterResult.Type.TEXT,
ex.toString());
}
}
private void execute(String code) throws Exception {
SparkInterpreter sintp = getSparkInterpreter();
SQLContext sqlc = sintp.getSQLContext();
StructType structType = new StructType().add("value",DataTypes.IntegerType);
ArrayList<Row> list = new ArrayList<Row>();
for (String s : code.trim().split("\\s+")) {
int value = Integer.parseInt(s);
System.out.println(value);
list.add(RowFactory.create(value));
}
DataFrame df = sqlc.createDataFrame(list,structType);
df.registerTempTable("result");
}
private SparkInterpreter getSparkInterpreter() {
LazyOpenInterpreter lazy = null;
SparkInterpreter spark = null;
Interpreter p = getInterpreterInTheSameSessionByClassName(SparkInterpreter.class.getName());
while (p instanceof WrappedInterpreter) {
if (p instanceof LazyOpenInterpreter) {
lazy = (LazyOpenInterpreter) p;
}
p = ((WrappedInterpreter) p).getInnerInterpreter();
}
spark = (SparkInterpreter) p;
if (lazy != null) {
lazy.open();
}
return spark;
}
}

I think that you should configure spark cluster such as the below statement.
spark.master = "local[4]"
spark.app.name = "My Spark App"
spark.serializer = "org.apache.spark.serializer.KryoSerializer"
Using SparkContext.getOrCreate() looks good to me.
Thanks,
Kangrok Lee

Read a .java file from another class and call its method

Is it possible that to read a .java file as a file (using file path)from another class and call its method in this class ?
Lets takes a java class as
public Mylogic {
public static void test()
{
 //some logic
}
}
Is there a way another java class can read Mylogic.java file as a file and execute test() method
?
Why I want this?
Once source code goes into application server , then if I have to add another class , I have to wait for complete deployment which takes time.If I am able to do this, I can keep a utility class ready in source code to read .java file from dir and execute it without any deployment, thus saving time.
This is for higher environment (production mode) so no exploded mode.

It is possible, if you have JDK instead of JRE when running the code, using javax.tools.JavaCompiler and a customized class loader:
import java.io.BufferedWriter;
import java.io.ByteArrayOutputStream;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStreamWriter;
import java.lang.reflect.InvocationTargetException;
import java.lang.reflect.Method;
import javax.tools.JavaCompiler;
import javax.tools.StandardJavaFileManager;
import javax.tools.ToolProvider;
public class LoadAndRun extends ClassLoader {
public Class findClass(String name) {
ByteArrayOutputStream out = new ByteArrayOutputStream();
try {
InputStream in = new FileInputStream("./" + name + ".class");
byte [] buff = new byte[4096];
while (true) {
int c = in.read(buff);
if (c == -1) break;
out.write(buff, 0, c);
}
return defineClass(name, out.toByteArray(), 0, out.toByteArray().length);
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
return null;
}
public static void main(String [] args) throws ClassNotFoundException, NoSuchMethodException, InvocationTargetException, IllegalAccessException, IOException {
// Try to get the system compiler
JavaCompiler compiler = ToolProvider.getSystemJavaCompiler();
if (compiler == null) {
System.err.println("System java compiler not found.");
return;
}
// Prepare the java file to be compiled
String classname = "LoadAndRun" + System.currentTimeMillis();
File javaFile = new File(classname + ".java");
BufferedWriter writer = new BufferedWriter(new OutputStreamWriter(new FileOutputStream(javaFile)));
writer.write("public class " + classname + "{\n");
writer.write("public static void test() {System.out.println(\"this is the test class:\"+"+classname+".class);}");
writer.write("}\n");
writer.close();
// Compile it!
StandardJavaFileManager fileManager = compiler.getStandardFileManager(null, null, null);
compiler.getTask(null, fileManager, null, null, null, fileManager.getJavaFileObjects(javaFile)).call();
// Load and invoke static method
ClassLoader classLoader = new LoadAndRun();
Class cls = classLoader.loadClass(classname);
Method m = cls.getMethod("test");
m.invoke(null);
// Clean up
m = null;
cls = null;
classLoader = null;
System.gc();
javaFile.delete();
File classFile = new File(classname + ".class");
classFile.delete();
}
}

Compile and execute java source code without knowing where the Main method is?

I am facing the following problem:
I will receive one or more .java files. The idea is to automate the compile and execute process.
I have not written nor viewed .java source files i receive.
There may be 1 file or multiple files in multiple directories.
All of this is done under linux ( Debian / CentOS ).
Here is an example case:
2 files are received:
SomeFile.java and SomeOtherFile.Java ( This one has the static public void main(String args[]){} method but i do NOT know that !)
A process picks up the files and compiles them in this way:
javac -encoding UTF-8 -sourcepath . -d . *.java
So my problem is: I do not know which package(if any) contains the Main method so i do not know what do execute ?
java packageName.SomeOtherFile

Plenty of ways:
Get a java source code parser, parse the source code, find the method
Compile everything, go over the resulting *.class files using reflection, find the method.
For a small enough number of files, just try them all

I have written something like this before:
package vonbulow.nicki;
import java.io.ByteArrayInputStream;
import java.io.DataInputStream;
import java.io.File;
import java.io.FilenameFilter;
import java.io.IOException;
import java.io.InputStream;
import java.lang.reflect.InvocationTargetException;
import java.lang.reflect.Method;
import java.util.ArrayList;
import java.util.Enumeration;
import java.util.HashMap;
import java.util.Iterator;
import java.util.List;
import java.util.jar.Manifest;
import java.util.logging.Level;
import java.util.logging.Logger;
import java.util.zip.ZipEntry;
import java.util.zip.ZipFile;
/**
*
* #author Nicki
*/
public class AppLoader extends ClassLoader {
private static final String userdir = System.getenv("USERPROFILE");
private static final AppLoader instance = new AppLoader();
private static HashMap<String, Class> loaded = new HashMap<String, Class>();
public static void loadapp(final String name) {
if(loaded.containsKey(name)) {
Thread d = new Thread(new Runnable(){
public void run(){
try {
Class c = loaded.get(name);
Method m = c.getDeclaredMethod("main", String[].class);
m.invoke(null, (Object[])new String[]{null});
} catch (NoSuchMethodException | SecurityException | IllegalAccessException | IllegalArgumentException | InvocationTargetException ex) {
}
}
});
d.start();
return;
}
File ud = new File(userdir+"\\nvbapp");
ud.mkdir();
File[] fa = ud.listFiles(new FilenameFilter() {
public boolean accept(File dir, String name) {
return name.toLowerCase().endsWith(".appn");
}
});
for(final File f:fa) {
if(f.getName().split("\\.")[0].equalsIgnoreCase(name)) {
Thread t = new Thread(new Runnable() {
public void run() {
runapp(f, name);
}
});
t.start();
}
}
}
private static void runapp(File f, String nam) {
List<Class> classes = new ArrayList<Class>();
ZipFile jf;
String name = "";
try {
jf = new ZipFile(f);
Enumeration<? extends ZipEntry> eze = jf.entries();
while(eze.hasMoreElements()){
ZipEntry ze = eze.nextElement();
if(ze.getName().endsWith(".class")&&!ze.isDirectory()){
InputStream fis = jf.getInputStream(ze);
byte[] bytes = new byte[(int)ze.getSize()];
fis.read(bytes);
classes.add(instance.defineClass(getClassName(bytes), bytes, 0, bytes.length));
}
if(ze.getName().equalsIgnoreCase("META-INF/MANIFEST.MF")) {
Manifest manifest = new Manifest(jf.getInputStream(ze));
name = manifest.getMainAttributes().getValue("Main-Class");
}
}
Iterator<Class> classit = classes.iterator();
while(classit.hasNext()) {
Class c = classit.next();
if(c.getName().equals(name)) {
try {
loaded.put(nam, c);
Method m = c.getDeclaredMethod("main", String[].class);
m.invoke(null, (Object[]) new String[]{null});
} catch (NoSuchMethodException | SecurityException | IllegalAccessException | IllegalArgumentException | InvocationTargetException ex) {
ex.printStackTrace();
}
}
}
} catch (IOException ex) {
Logger.getLogger(AppLoader.class.getName()).log(Level.SEVERE, null, ex);
}
}
private static String getClassName(byte[] is) {
try {
DataInputStream dis = new DataInputStream(new ByteArrayInputStream(is));
dis.readLong(); // skip header and class version
int cpcnt = (dis.readShort()&0xffff)-1;
int[] classes = new int[cpcnt];
String[] strings = new String[cpcnt];
for(int i=0; i<cpcnt; i++) {
int t = dis.read();
if(t==7) classes[i] = dis.readShort()&0xffff;
else if(t==1) strings[i] = dis.readUTF();
else if(t==5 || t==6) { dis.readLong(); i++; }
else if(t==8) dis.readShort();
else dis.readInt();
}
dis.readShort(); // skip access flags
return strings[classes[(dis.readShort()&0xffff)-1]-1].replace("/", ".");
} catch (IOException ex) {
return null;
}
}
}
I have not edited it; you will need to edit it so it loads your own classes.
You also need to compile the files first with the JavaCompiler class. This also assumes that the classes are in the zip file.

java urlclassloader that calls a class that has an import dependency

After some help in an other thread on urlclassloaders - understanding urlclassloader, how to access a loaded jar's classes
I have a follow on question as I don't think I am approaching the problem correctly.
myPackageA.start has a urlclassloader calling myPackageB.comms
myPackageB.comms has an dependency to import org.jgroups.JChannel
form /home/myJars/jgroups-3.4.2.Final.jar with the following code
package myPackageB;
import org.jgroups.JChannel;
public class SimpleChat {
JChannel channel;
String user_name=System.getProperty("user.name", "n/a");
private void start() throws Exception {
channel=new JChannel();
channel.connect("ChatCluster");
channel.getState(null, 10000);
channel.close();
}
public static void main(String[] args) throws Exception {
new SimpleChat().start();
}
}
normally I would call the above code with java -cp /home/myJars/jgroups-3.4.2.Final.jar:myPackageB myPackageB.SimpleChat and runs as expected.
My question is howit possible to set the -cp within the script so the import works when using the below code to call myPackageB.SimpleChat from java -cp myPackageA.jar myPackageA.start
package myPackageA;
import java.lang.reflect.Field;
import java.lang.reflect.Method;
import java.net.URL;
import java.net.URLClassLoader;
public class start
{
Class<?> clazz;
private void start() throws Exception
{
if (this.clazz == null)
throw new Exception("The class was not loaded properly");
Object mySc = this.clazz.newInstance();
Method sC = this.clazz.getDeclaredMethod("main", String[].class);
String[] params = null;
sC.invoke(mySc, (Object) params);
}
public void loadSc() throws Exception
{
URL classUrl;
classUrl = new URL("file:///home/myJars/myPackageB.jar");
URL[] classUrls = { classUrl };
URLClassLoader ucl = new URLClassLoader(classUrls);
Class<?> c = ucl.loadClass("myPackageB.SimpleChat");
this.clazz = c;
}
public static void main(String[] args) throws Exception
{
start startnow = new start();
startnow.loadSc();
startnow.start();
}
}
thanks
Art

Just add the URL for jgroups-3.4.2.Final.jar to the URLClassLoader's array of URLs.