We Keep Coding

7zip cmd extracting using ProcessBuilder in java

I have a problem extracting an archive to the desired category using Java 10 ProcessBuilder and 7z.exe (18.05) with command line. The exact same command works as intended when I use Windows CMD, but no longer functions when issued by my JavaFX application using ProcessBuilder:
public static void decompress7ZipEmbedded(File source, File destination) throws IOException, InterruptedException {
ProcessBuilder pb = new ProcessBuilder(
getSevenZipExecutablePath(),
EXTRACT_WITH_FULL_PATHS_COMMAND,
quotifyPath(source.getAbsolutePath()),
OUTPUT_DIRECTORY_SWITCH + quotifyPath(destination.getAbsolutePath())
);
processWithSevenZipEmbedded(pb);
}
private static void processWithSevenZipEmbedded(ProcessBuilder pb) throws IOException, InterruptedException {
LOG.info("7-zip command issued: " + String.join(" ", pb.command()));
Process p = pb.start();
new Thread(new InputConsumer(p.getInputStream())).start();
System.out.println("Exited with: " + p.waitFor());
}
public static class InputConsumer implements Runnable {
private InputStream is;
InputConsumer(InputStream is) {
this.is = is;
}
#Override
public void run() {
try {
int value = -1;
while ((value = is.read()) != -1) {
System.out.print((char) value);
}
} catch (IOException exp) {
exp.printStackTrace();
}
LOG.debug("Output stream completed");
}
}
public static String getSevenZipExecutablePath() {
return FileUtil.quotifyPath(getDirectory() + "7z" + "/" + "7z");
}
public static String quotifyPath(String path) {
return '"' + path + '"';
}
public class Commands {
public static final String EXTRACT_COMMAND = "e";
public static final String EXTRACT_WITH_FULL_PATHS_COMMAND = "x";
public static final String PACK_COMMAND = "a";
public static final String DELETE_COMMAND = "d";
public static final String BENCHMARK_COMMAND = "b";
public static final String LIST_COMMAND = "l";
}
public class Switches {
public static final String OUTPUT_DIRECTORY_SWITCH = "-o";
public static final String RECURSIVE_SWITCH = "-r";
public static final String ASSUME_YES = "y";
}
The command looks like this:
"C:/Users/blood/java_projects/AppRack/target/classes/7z/7z" x "D:\Pulpit\AppRack Sandbox\test\something\Something 2\Something2.7z" -o"D:\Pulpit\AppRack Sandbox\Something2"
And the output from ProcessBuilder:
7-Zip 18.05 (x64) : Copyright (c) 1999-2018 Igor Pavlov : 2018-04-30
Scanning the drive for archives:
1 file, 59177077 bytes (57 MiB)
Extracting archive: D:\Pulpit\AppRack Sandbox\test\Something\Something 2\Something2.7z
--
Path = D:\Pulpit\AppRack Sandbox\test\Something\Something 2\Something2.7z
Type = 7z
Physical Size = 5917Exited with: 0
7077
Headers Size = 373
Method = LZMA2:26 LZMA:20 BCJ2
Solid = +
Blocks = 2
No files to process
Everything is Ok
Files: 0
Size: 0
Compressed: 59177077
It doesn't do ANYTHING. Doesn't create a desired folder, nothing. Using CMD it works like a charm (here log from Windows 10 CMD using the same command):
7-Zip 18.05 (x64) : Copyright (c) 1999-2018 Igor Pavlov : 2018-04-30
Scanning the drive for archives:
1 file, 59177077 bytes (57 MiB)
Extracting archive: D:\Pulpit\AppRack Sandbox\test\Something\Something 2\Something2.7z
--
Path = D:\Pulpit\AppRack Sandbox\test\Something\Something 2\Something2.7z
Type = 7z
Physical Size = 59177077
Headers Size = 373
Method = LZMA2:26 LZMA:20 BCJ2
Solid = +
Blocks = 2
Everything is Ok
Folders: 1
Files: 5
Size: 64838062
Compressed: 59177077
Do you have any idea what causes a difference here and why it says "No files to process, everything is ok" without doing anything? I've tried already to create a folder first using File class but it doesn't seem to be an issue because the results are the same whether the destination folder exists prior to extracting or not.
I've already tried everything that has come to my mind and I run out of ideas at the moment. Please share with me any suggestions that you may have regarding this issue. Thanks a lot.

Thank you very much for your help.
Don’t quote your arguments. Quotes are for the command shell’s benefit. ProcessBuilder is not a command shell; it executes a command directly, so any quotes are seen as part of the argument itself (that is, the file name). Also, pb.inheritIO(); is a better way to see the output of the child process than manually consuming process streams.
Thank you #VGR it seemed to be the issue - after I remove the method to quote paths in the mentioned command it works like a charm and extracting archive without any problem! So the conclusion is I shouldn't have used quotes in paths while using Java ProcessBuilder.
I've also used pb.inheritIO() and you are right it is much better and easier to manage it this way.
public static void decompress7ZipEmbedded(File source, File destination) throws IOException {
ProcessBuilder pb = new ProcessBuilder().inheritIO().command(
getSevenZipExecutablePath(),
EXTRACT_WITH_FULL_PATHS_COMMAND,
source.getAbsolutePath(),
OUTPUT_DIRECTORY_SWITCH + destination.getAbsolutePath(),
OVERWRITE_WITHOUT_PROMPT
);
processWithSevenZipEmbedded(pb);
}
private static void processWithSevenZipEmbedded(ProcessBuilder pb) throws IOException {
LOG.info("7-zip command issued: " + String.join(" ", pb.command()));
pb.start();
}
public class Commands {
public static final String EXTRACT_WITH_FULL_PATHS_COMMAND = "x";
}
public class Switches {
public static final String OUTPUT_DIRECTORY_SWITCH = "-o";
public static final String OVERWRITE_WITHOUT_PROMPT = "-aoa";
}
Double click on file 7zip.chm or start 7-Zip and open the Help and read the help page Command Line Version - Syntax with first line 7z [...] [...]. There is clearly explained that first the command x must be specified, next should be the switches like -o with best last switch being --, then the archive file name and last further arguments like names of files/folders to extract. Switches can be also specified after archive file name, but that is not recommended although examples on help page for -o are also with -o at end.
Thank you #Mofi for the tip. I used -aoa switch instead of -y and it finally started to work as I wanted - to overwrite files without any prompt. I left the rest of the command the way it was as it works as intended, so it finally looks like this:
C:/Users/blood/java_projects/AppRack/target/classes/7z/7z" x D:\Pulpit\AppRack Sandbox\test\Test\Test 2\Test.7z -oD:\Desktop\AppRack Sandbox\Test 2 -aoa
Thanks a lot for help once again!

Using displayLogs() in WLSTInterpreter NameError: displayLogs

I am attempting to get a log output from the displayLogs() command and I have trying to do this within the WLST Interpreter. I get the following error which is "NameError: displayLogs" I am able to perform other commands such as domainRuntime() and a number of others, but this one seems to be outside the realm. Do I need to run it with some sort of classes in the classpath when running it? Any help would be appreciated.
The source code that I am using below:
package wlst;
import weblogic.management.scripting.utils.WLSTInterpreter;
import org.python.util.InteractiveInterpreter;
import org.python.core.PyObject;
public class EmbeddedWLST
{
static InteractiveInterpreter interpreter = null;
EmbeddedWLST() {
interpreter = new WLSTInterpreter();
}
private static void connect() {
StringBuffer buffer = new StringBuffer();
buffer.append("connect('USERNAME','PASSWORD','t3://HOSTANAME:PORT')");
interpreter.exec(buffer.toString());
}
public static void main(String[] args) {
new EmbeddedWLST();
connect();
PyObject cmo = interpreter.get("cmo");
String command = getLogs();
System.out.println("Executing Get Logs");
interpreter.exec(command);
System.out.println("Getting Output Object");
PyObject output = interpreter.get("output");
System.out.println(output.getClass());
System.out.println(output);
}
private static String getLogs() {
StringBuffer buf = new StringBuffer();
buf.append( "output = displayLogs(returnData=1)\n" );
return buf.toString();
}
}

UPDATE
Everything you are looking for lives in:
<install dir>/oracle_common/common/wlst
A simple grep -R displayLogs * returned the python module you need:
<install dir>/oracle_common/common/wlst/oracle-logging.py
You will need to include the jars that script needs on your classpath, specifically the logging jar ojdl.jar found under <install dir>/oracle_common/modules/oracle.odl
The above information was found by comparing the scripts below (I am using 10.3.6):
This script <install dir>/wlserver_10.3/common/bin/wlst.sh fails with:
wls:/offline> listLogs()
Traceback (innermost last):
File "<console>", line 1, in ?
NameError: listLogs
This script <install dir>/oracle_common/common/bin/wlst.sh succeeds (and has many more options than the script above) :
wls:/offline> listLogs()
Not connected to a Weblogic server. Connect to a Weblogic server first.
Make sure you have all the same jars and properties set as the second script does.

Java reading file and storing text as array

I wrote a program someone told me to do for reading file and storing the text as an array, but when I run the program, I get an error that looks like this:
run:
Exception in thread "main" java.io.FileNotFoundException: KeyWestTemp.txt (No such file or directory)
at java.io.FileInputStream.open(Native Method)
at java.io.FileInputStream.<init>(FileInputStream.java:146)
at java.util.Scanner.<init>(Scanner.java:656)
at HeatIndex.main(HeatIndex.java:32)
Java Result: 1
BUILD SUCCESSFUL (total time: 0 seconds)
I want to have my program display the array to make sure it works correctly. Any help would be greatly appreciated.
Below is the program code:
import java.util.Scanner;
import java.io.File;
import java.io.IOException;
import java.util.LinkedList;
import java.util.List;
public class HeatIndex {
/**
* #param args the command line arguments
* #throws java.io.IOException
*/
public static void main(String[] args) throws IOException{
// TODO code application logic here
// // read KeyWestTemp.txt
// create token1
String token1 = "";
// create Scanner inFile1
Scanner inFile1 = new Scanner(new File("KeyWestTemp.txt")).useDelimiter(",\\s*");
// create List
List<String> temps = new LinkedList<String>();
// while loop
while(inFile1.hasNext()){
// find next line
token1 = inFile1.next();
// initialize temps
temps.add(token1);
}
// close inFile1
inFile1.close();
// create array
String[] tempsArray = temps.toArray(new String[0]);
// for-each loop
for(String s : tempsArray){
// display s to make sure program works correctly
System.out.println(s);
}
}
}

Your program is unable to locate your file. Make sure the file is in your working directory, or change new File("KeyWestTemp.txt") to provide a complete path (e.g. C:/Users/...../KeyWestTemp.txt).

Apparently, the file KeyWestTemp.txt is not found by the Java program. The most likely problem is that you haven't placed it in the right place.
You have 2 options:
Place the file in the working directory, from where you run the java command
When initializing the Scanner, set the full path: new Scanner(new File("/file/path/to/KeyWestTemp.txt")) to the file

Two ways to fix it:
Put your txt file in the same dir of your Java program.
Write the full path of the txt file (c:/files...). Remember to use File.separator instead of the "/" or "\" to make the program platform independent.

Using Hadoop to find files that contain a particular string

I have around 1000 files and each file is of the size of 1GB. And I need to find a String in all these 1000 files and also which files contains that particular String. I am working with Hadoop File System and all those 1000 files are in Hadoop File System.
All the 1000 files are under real folder, so If I do like this below, I will be getting all the 1000 files. And I need to find which files contains a particular String hello under real folder.
bash-3.00$ hadoop fs -ls /technology/dps/real
And this is my data structure in hdfs-
row format delimited
fields terminated by '\29'
collection items terminated by ','
map keys terminated by ':'
stored as textfile
How I can write MapReduce jobs to do this particular problem so that I can find which files contains a particular string? Any simple example will be of great help to me.
Update:-
With the use of grep in Unix I can solve the above problem scenario, but it is very very slow and it takes lot of time to get the actual output-
hadoop fs -ls /technology/dps/real | awk '{print $8}' | while read f; do hadoop fs -cat $f | grep cec7051a1380a47a4497a107fecb84c1 >/dev/null && echo $f; done
So that is the reason I was looking for some MapReduce jobs to do this kind of problem...

It sounds like you're looking for a grep-like program, which is easy to implement using Hadoop Streaming (the Hadoop Java API would work too):
First, write a mapper that outputs the name of the file being processed if the line being processed contains your search string. I used Python, but any language would work:
#!/usr/bin/env python
import os
import sys
SEARCH_STRING = os.environ["SEARCH_STRING"]
for line in sys.stdin:
if SEARCH_STRING in line.split():
print os.environ["map_input_file"]
This code reads the search string from the SEARCH_STRING environmental variable. Here, I split the input line and check whether the search string matches any of the splits; you could change this to perform a substring search or use regular expressions to check for matches.
Next, run a Hadoop streaming job using this mapper and no reducers:
$ bin/hadoop jar contrib/streaming/hadoop-streaming-*.jar \
-D mapred.reduce.tasks=0
-input hdfs:///data \
-mapper search.py \
-file search.py \
-output /search_results \
-cmdenv SEARCH_STRING="Apache"
The output will be written in several parts; to obtain a list of matches, you can simply cat the files (provided they aren't too big):
$ bin/hadoop fs -cat /search_results/part-*
hdfs://localhost/data/CHANGES.txt
hdfs://localhost/data/CHANGES.txt
hdfs://localhost/data/ivy.xml
hdfs://localhost/data/README.txt
...

To get the filename you are currently processing, do:
((FileSplit) context.getInputSplit()).getPath().getName()
When you are searching your file record by record, when you see hello, emit the above path (and maybe the line or anything else).
Set the number of reducers to 0, they aren't doing anything here.
Does 'row format delimited' mean that lines are delimited by a newline? in which case TextInputFormat and LineRecordReader work fine here.

You can try something like this, though I'm not sure if it's an efficient way to go about it. Let me know if it works - I haven't tested it or anything.
You can use it like this: java SearchFiles /technology/dps/real hello making sure you run it from the appropriate directory of course.
import java.io.BufferedReader;
import java.io.FileReader;
import java.io.File;
import java.io.IOException;
import java.util.ArrayList;
import java.util.Scanner;
public class SearchFiles {
public static void main(String[] args) throws IOException {
if (args.length < 2) {
System.err.println("Usage: [search-dir] [search-string]");
return;
}
File searchDir = new File(args[0]);
String searchString = args[1];
ArrayList<File> matches = checkFiles(searchDir.listFiles(), searchString, new ArrayList<File>());
System.out.println("These files contain '" + searchString + "':");
for (File file : matches) {
System.out.println(file.getPath());
}
}
private static ArrayList<File> checkFiles(File[] files, String search, ArrayList<File> acc) throws IOException {
for (File file : files) {
if (file.isDirectory()) {
checkFiles(file.listFiles(), search, acc);
} else {
if (fileContainsString(file, search)) {
acc.add(file);
}
}
}
return acc;
}
private static boolean fileContainsString(File file, String search) throws IOException {
BufferedReader in = new BufferedReader(new FileReader(file));
String line;
while ((line = in.readLine()) != null) {
if (line.contains(search)) {
in.close();
return true;
}
}
in.close();
return false;
}
}

Execute a Java program from our Java program

I used
Runtime.getRuntime().exec("_____")
but it throws a IOException as below:
java.io.IOException: CreateProcess: c:/ error=5
at java.lang.Win32Process.create(Native Method)
at java.lang.Win32Process.<init>(Win32Process.java:63)
at java.lang.Runtime.execInternal(Native Method
I don't know whether I have the problem with specifying the path or something else. Can anyone please help me with the code.

You're trying to execute "C:/". You'll want to execute something like:
"javaw.exe d:\\somejavaprogram\\program.jar"
Notice the path separators.
I'm assuming this is for an ad-hoc project, rather than something large. However, for best practice running external programs from code:
Don't hardcode the executable location, unless you're certain it will never change
Look up directories like %windir% using System.getenv
Don't assume programs like javaw.exe are in the search path: check them first, or allow the user to specify a location
Make sure you're taking spaces into account: "cmd /c start " + myProg will not work if myProg is "my program.jar".

You can either launch another JVM (as described in detail in other answers).
But that is not a solution i would prefer.
Reasons are:
calling a native program from java is "dirty" (and sometimes crashes your own VM)
you need to know the path to the external JVM (modern JVMs don't set JAVA_HOME anymore)
you have no control on the other program
Main reason to do it anyway is, that the other application has no control over your part of the program either. And more importantly there's no trouble with unresponsive system threads like the AWT-Thread if the other application doesn't know its threading 101.
But! You can achieve more control and similar behaviour by using an elementary plugin technique. I.e. just call "a known interface method" the other application has to implement. (in this case the "main" method).
Only it's not quite as easy as it sounds to pull this off.
you have to dynamically include required jars at runtime (or include them in the classpath for your application)
you have to put the plugin in a sandbox that prevents compromising critical classes to the other application
And this calls for a customized classloader. But be warned - there are some well hidden pitfalls in implementing that. On the other hand it's a great exercise.
So, take your pick: either quick and dirty or hard but rewarding.

java.io.IOException: CreateProcess: c:/ error=5
at java.lang.Win32Process.create(Native Method)
at java.lang.Win32Process.<init>(Win32Process.java:63)
at java.lang.Runtime.execInternal(Native Method)
If I recall correctly, error code 5 means access denied. This could be because your path is incorrect (trying to execute "c:/") or you are bumping against your OS security (in which case, look at the permissions).
If you are having trouble locating the Java executable, you can usually find it using system properties:
public class LaunchJre {
private static boolean isWindows() {
String os = System.getProperty("os.name");
if (os == null) {
throw new IllegalStateException("os.name");
}
os = os.toLowerCase();
return os.startsWith("windows");
}
public static File getJreExecutable() throws FileNotFoundException {
String jreDirectory = System.getProperty("java.home");
if (jreDirectory == null) {
throw new IllegalStateException("java.home");
}
File exe;
if (isWindows()) {
exe = new File(jreDirectory, "bin/java.exe");
} else {
exe = new File(jreDirectory, "bin/java");
}
if (!exe.isFile()) {
throw new FileNotFoundException(exe.toString());
}
return exe;
}
public static int launch(List<String> cmdarray) throws IOException,
InterruptedException {
byte[] buffer = new byte[1024];
ProcessBuilder processBuilder = new ProcessBuilder(cmdarray);
processBuilder.redirectErrorStream(true);
Process process = processBuilder.start();
InputStream in = process.getInputStream();
while (true) {
int r = in.read(buffer);
if (r <= 0) {
break;
}
System.out.write(buffer, 0, r);
}
return process.waitFor();
}
public static void main(String[] args) {
try {
Runtime.getRuntime().exec("c:/");
List<String> cmdarray = new ArrayList<String>();
cmdarray.add(getJreExecutable().toString());
cmdarray.add("-version");
int retValue = launch(cmdarray);
if (retValue != 0) {
System.err.println("Error code " + retValue);
}
System.out.println("OK");
} catch (IOException e) {
e.printStackTrace();
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
(Tested Windows XP, Sun JRE 1.6; Ubuntu 8.04, OpenJDK JRE 1.6)
This is the equivalent of running:
java -version
You may also want to look at the "java.library.path" system property (and "path.separator") when trying to locate the executable.

How about just calling the main from your java program?
Test.main(null);
This worked fine for me

Is there any reason you can't just call it directly in your Java code?
If there is a reason I've not tried it for executing a Java Program but you could try Jakarta Commons Exec works well for executing most programs.

I had to do this recently.
Here is how I did it, picking up only the relevant parts:
private static final String[] straJavaArgs =
{
"?i/j2re/bin/java",
"-ms64m",
"-mx64m",
"-Djava.ext.dirs=?i/lib;?i/jar/lib;?i/jar"
};
// ...
// AppDesc appToRun;
List<String> params = new ArrayList<String>();
// Java exe and parameters
params.addAll(ExpandStrings(straJavaArgs));
// Common VM arguments
params.addAll(Arrays.asList(AppDesc.GetCommonVMArgs()));
// Specific VM arguments
params.addAll(ExpandStrings(appToRun.GetVMArgs()));
// The program to run
params.add(appToRun.GetClass());
// Its arguments
params.addAll(ExpandStrings(appToRun.GetProgramArgs()));
// The common arguments
params.addAll(ExpandStrings(AppDesc.GetCommonProgramArgs()));
ProcessBuilder processBuilder = new ProcessBuilder(params);
process = processBuilder.start();
return CaptureProcessOutput(); // Uses a StreamGobbler class
protected ArrayList<String> ExpandStrings(String[] stra)
{
ArrayList<String> alResult = new ArrayList<String>();
for (int i = 0; i < stra.length; i++)
{
// Super flexible, eh? Ad hoc for the current task, at least...
alResult.add(stra[i]
.replaceAll("\\?i", strInstallDir)
.replaceAll("\\?c", strConfigDir)
);
}
return alResult;
}
public enum AppDesc
{
// Enumerate the applications to run, with their parameters
}
Incomplete, if you need more details, just ask.

public class Test {
public static void main(String[] args) throws Exception {
Process p = Runtime.getRuntime().exec("\"c:/program files/windows/notepad.exe\"");
p.waitFor();
}
}
The above works quite well, instead of passing \"c:/program files/windows/notepad.exe\" as the arguments for the executable, use the path to your program, I'm not sure if this solution is JVM version dependent, or if it can use relative paths.

You must pass the path of your executable at the exec method. Are you really trying to execute the "-" process?
Also, have a look at this for some useful tips.

Put ant lib in you classpath ( project lib ) and run this code :
import org.apache.tools.ant.taskdefs.Execute;
Execute exe = new Execute();
exe.setCommandline(new String[]{"java", "-version"});
exe.execute();

I can't remember the exact code that I used to get this to work, but you have to pass "java.exe" (or the equivalent) as the executable, and then the class or jar to run as the parameter, with the correct working directory. So it's not as simple as just calling one method.

I had a similiar problem. I needed to run a section of Java code in a seperate VM as it invoked native code via JNI that occasionally blew up taking out the entire VM.
I cheated a little though. I initially used Runtime to invoke a simple batch command file and put the work-in-progress java command in there. This enabled me to tweak it as needed and to run the command in a DOS prompt for easy testing. Once it was finished I simply copied the result into the Runtime invocation.

First you compile the prog-A code and convert to jar file(ie:In NetBeans Shift-F11)and the path is of netbeans(NetBeansProjects/prog-A/dist/prog-A.jar)
public class ProgA {
/**
* #param args the command line arguments
*/
public static void main(String[] args) {
System.out.println("Helllo print thr ProgA");
}
}
}
Second open the new project in prog-B and add the libraries, and select the jar and give to the prog-A.jar file and write the two line in your program
public class ProgB {
public static void main(String[] args) {
ProgA progA = new ProgA();
String arg[] = null;
progA.main(arg);
}
}

I agree with Ushsa Varghese, if you just want to run your jar file instead of compiling the .java file that is in the same directory you are executing your application from try the code below. This is the same as executing your java application from the command line so you have to invoke the jvm in order to run your application. Also make sure you have the complete path to your jar file the example below assumes that the jar file is in the same directory as the application that is executing the code below. keep in mind this is system dependent code.
try {
Runtime runTime = Runtime.getRuntime();
Process process = runTime.exec("java -jar deleteDriveC.jar");
} catch (IOException ex) {
//jar file doesnt exist
//Logger.getLogger(this.class.getName()).log(Level.SEVERE, null, ex);
}

The answer is simple all you have to do is put the code -
$ process p = Runtime.getRuntime().exec("javac factorial.java"); in the try catch block
The code would look like this -
try
{
process p = Runtime.getRuntime().exec("javac factorial.java");
}
catch(IOException e)
{
e.printStackTrace();
}
Hey I think this should work. Atleast for me it did work