Java App to run a Kettle Job and receive the result - java

It is possible to run a Kettle Job/Transformation from a Java application, and then get the result (a variable for example), in the same Java App?

Although command line execution from Java likely isn't ideal, below would work. Just replace the cmd line with the appropriate paths and read in the output file of the job.
public static void main(String[] args) {
try {
String cmd = "\"c:\\Program Files\\Pentaho\\pdi-ce-5.0.1.A-stable\\data-integration\\kitchen.bat\" -file=\"c:\\users\\exampleuser\\desktop\\examplejob.kjb\"";
System.out.println(cmd);
Process process = Runtime.getRuntime().exec(cmd);
process.waitFor();
} catch (Exception e) {
e.printStackTrace(System.err);
}
/*READ OUTPUT FILE OF KJB IN TO OBTAIN VALUES*/
}
*http://forums.pentaho.com/showthread.php?81151-Tutorial-Using-command-line-arguments-with-Kettle-and-scheduling
*http://docs.oracle.com/javase/tutorial/java/data/strings.html
*http://wiki.pentaho.com/display/EAI/.01+Introduction+to+Spoon
*http://javarevisited.blogspot.com/2011/02/how-to-execute-native-shell-commands.html
*http://www.mkyong.com/java/how-to-execute-shell-command-from-java/

I'd try this to execute the transformation this way. Kettle's Java API is quite readable.
The way described above does not cover retrieving the results. You'd need to add some custom logic to read the output data of the job. I'm not sure if there is a generic way to do that with ETL.
You can try grabbing the output into a HSQLDB or another in-memory DB, as described here, and extract the output manually.

Related

Java console log simultaneously to readLine without collisions

I'm currently dealing with the following problem:
I try to make a console input for a java application
that works with multiple threads. So while running the
software it happens sometimes, that a new line of log is
appearing while I'm writing to the readLine with a promt..
When that happens it looks like the following:
Image of the Console
so it does stack the messages like in the image.. so here is the question:
How can I keep the line and text I am writing to and log the text above it like in the example below?
Gif of the input
(Sorry for low quallity but you can guess what I mean)
As you can see my input stays at the bottom, is still editable and the lines do not stack
Thank you for some help, I am struggeling so much after trying Log4j, System.console, BufferedReaders and Scanner
Solved. It was horrible complicated..
But here for the future:
class ConsoleThread implements Runnable {
private ConsoleReader reader;
private OutputStream output;
public ConsoleThread(OutputStream output, ConsoleReader reader) {
this.output = output;
this.reader = reader;
}
#Override
public void run() {
try {
String message;
while (true) {
message = LoggingQueue.getNextLogEvent();
if(message == null) continue;
reader.print(Ansi.ansi().eraseLine(Ansi.Erase.ALL).toString() + ConsoleReader.RESET_LINE);
reader.flush();
output.write((message + System.lineSeparator()).getBytes());
output.flush();
try {
reader.drawLine();
} catch (Throwable ex) {
reader.getCursorBuffer().clear();
}
reader.flush();
}
} catch (IOException e) {
Controller.handleException(Thread.currentThread(), e);
}
}
Using the ConsoleReader of jLine2 and jAnsi. The output stream is just System.out.
You just need a second thread which reads and you are done :)
You need to handle the threads competing for stdin/stdout.
In theory, that would imply some kind of mutex but, since you're using external libraries, it seems like too much trouble...
From your images, it seems that you're running a sort of server application that takes commands.
If that's the case, I recommend re-architecting to use two separate processes: one for the server part and one for the command prompt.
The two processes then communicate through a socket.
This allows you to make the command prompt single threaded or, at least, behave like a single threaded application, since it is only reacting to user commands.
This is what lots of applications, like Docker, Kubernetes or MySQL do.
In the case of Docker and Kubernetes, they expose full REST APIs on that socket so you can leverage libraries for that.

How to execute a linux terminal command from LUAJ?

I want to simply execute a linux terminal command like ls from LuaJ and the result that it will return or anything that returns i want to receive it and will show the names in the Java Gui. I searched but found this but not one with LuaJ.
Is there any function to execute the terminal command from LuaJ ??
There are multiple ways to do this, for one, you can implement it yourself in Java then link it to LuaJ.
LuaFunction command = new OneArgFunction()
{
public LuaValue call(LuaValue cmd)
{
Process p = Runtime.getRuntime().exec("/bin/sh", "-c", cmd.checkstring());
BufferedReader br = new BufferedReader(new InputStreamReader(p.getInputStream()));
int returnCode = p.waitFor();
return LuaValue.valueOf(returnCode);
}
}
globals.set("command", command);
Then in Lua:
local code = command("ls");
The problem with actually getting the output of a command is that you can't just have a fixall solution. For all the system knows you could be calling a program which runs for 2 hours generating constant output, which could be an issue, not to mention if the program requires input. If you know you're only going to use certain functions you can make a dirty version of above function to capture the output from the stream and return it all instead of the exit code, just don't use it on other processes that don't return quickly. The other alternative is to create a class that wraps the input and output streams from the process and return a coerced version of that class, and manage the input and output from lua.
Lua does have a function that's part of the OsLib called execute(), if execute doesn't exist in your current environment then in Java call:
globals.load(new OsLib());
Before loading the lua code. the os.execute() function returns the status code, and doesn't return the streams, so no way to get the output there. To get around this you can modify the command to pipe the output to a temp file and open it with the io library (new IoLib() if doesn't exist in current environment).
The other option is to use io.openProcess, which also executes the command and returns a file to read the output from.
Resources:
http://luaj.org/luaj/3.0/api/org/luaj/vm2/lib/OsLib.html
http://luaj.org/luaj/3.0/api/org/luaj/vm2/lib/IoLib.html

git java wrapper - git pull never ends

I'm creating a simple Java wrapper for git executable, that I want to use in my app.
A small code example:
public static void main(String[] args) {
String gitpath = "C:/eclipse/git/bin/git.exe";
File folder = new File("C:/eclipse/teste/ssadasd");
try {
folder.mkdirs();
Runtime.getRuntime().exec(
gitpath + " clone git#192.168.2.15:test.git", null,
folder);
} catch (IOException e) {
e.printStackTrace();
}
}
The code simply never ends the execution.. seems like it has caught inside exec.
If I run the git clone via command line, it work as expected.
If I try another repository, from github, e.g., it works too.
Someone have a ide for what is going on here?
Thanks in advance
This isn't a direct answer to your question, but you may want to take a look at JGit, which is direct Java implementation of Git operations (no wrapping of commandline git). JGit gets a lot of use and stabilization work as it is the foundation for EGit (Eclipse Git integration).
Runtime.getRuntime().exec returns a Process object that you can use to interact with the process and see what's going on. My suspicion is that you just need to do something like this:
Process p = Runtime.getRuntime().exec(
gitpath + " clone git#192.168.2.15:test.git", null,
folder);
p.waitFor();
If not, you can also do getErrorStream() or getOutputStream() on the process to see what it's writing out; that might be helpful in debugging.
Runtime.exec() can cause hanging under various circumstances - see this article which quotes the Javadoc, which says (in JDK 7):
Because some native platforms only provide limited buffer size for standard input and output streams, failure to promptly write the input
stream or read the output stream of the subprocess may cause the
subprocess to block, and even deadlock.
The article gives some example solutions, which consume the output and error streams, although I think the ProcessBuilder class was introduced after the article was written, so may be more satisfactory: the newer Javadoc adds:
Where desired, subprocess I/O can also be redirected using methods of the ProcessBuilder class.

linux shell and java

I want to send a command to linux shell and get it's response with java.How can i do this?
Have a look at ProcessBuilder - example here.
You should look at the Runtime class, and its exec() family of methods.
It's probably best to explicitly specify that you want to run the command through a shell, i.e. create a command line like "bash -c 'my command'".
Execute a process like this
Runtime.getRuntime().exec("ls");
...then you could get the process input stream and read it with a Reader to get the response
See the Runtime class and the exec() method.
Note that you need to consume the process's stdout/sterr concurrently, otheriwse you'll get peculiar blocking behaviour. See this answer for more information.
I wrote a little class to do this in a very similar question a couple of weeks ago:
java shell for executing/coordinating processes?
The class basically let's you do:
ShellExecutor excutor = new ShellExecutor("/bin/bash", "-s");
try {
System.out.println(excutor.execute("ls / | sort -r"));
} catch (IOException e) {
e.printStackTrace();
}

Eclipse executing java application in a loop

Does enyone know if it is possible to launch a java application in Eclipse but in a loop.
I need to execute a application all the night.
If I try tu run it in win console, it is complicated, I have ti specify a bunch a parameters.
Wrap your main in a call to Runtime.exec and put that in a loop.
public static void main(String args[]){
while(true){
Process proc = Runtime.getRuntime().exec("java yourclasshere");
try{
proc.waitFor();
} catch (InterruptedException e) {}
}
}
Just run the way you would run a regular Java program, in a loop:
public class MyProgram {
public static void main(String[] args) {
// your infite, of finite loop goes here
}
}
And then, just run it, and if there are not unhandled exceptions, it will run all night indeed.
Eclipse is a development environment, not an application manager. It's not well suited for what you're asking to do. It might be best to learn what those parameters are and why you need them, and write a batch file to handle them for you. There might be a better answer if you provide more details.

Categories