We Keep Coding

dlsym ends in infinite loop

I am trying to build a Linux library(*.so) to use it in a Java application. This library itself loads an dll-file with native functions.
This is my C++ code:
__delspec(dllexport) void __cdecl GetDllVersion(void){
typedef int(*GetDllVersion)(int*,int*,int*,int*);
void* lib = dlopen("~/lib.dll",RTLD_NOW);
cout << "Loading Symbol..." << endl;
GetDllVersion getVer= (GetDllVersion) dlsym(lib,"GetDllVersion");
dlclose(lib);
}
The code was compiled with wineg++ -shared lib.cpp -o libexports.so -Wl,--no-as-needed -ldl without errors.
The Java application prints out "Loading Symbol..." in a loop and then terinates without any message. I could determine that this has to do something with dlsym().
With nm -D lib.so I could look inside the lib.so. The function GetDllVersion() is indeed a symbol inside this library.
Can somebody tell me why there is a infinite loop and wyh the Java VM is terminating?
Regards Wurmi

This line:
void* lib = dlopen("~/lib.dll",RTLD_NOW);
will always fail, because dlopen does not do tilde-expansion (in general, only shell does). You really should check dlopen return value.
This line:
GetDllVersion getVer= (GetDllVersion) dlsym(lib,"GetDllVersion");
is equivalent to dlsym(RTLD_DEFAULT, ...) (because RTLD_DEFAULT == 0 and lib == NULL) and as such returns you a pointer to the function you are already in, resulting in infinite recursion, and eventual crash due to stack exhaustion.

How can I override java -Xmx option in application?

I am running a script in PowerShell, that calls a java application, but this application fails due to Out.of.memory.error: Heap Size. I created environment variable, but it didn't work. From the dump txt file I found out, that the java_command_line contains -Xmx512m option:
IBM_JAVA_COMMAND_LINE=C:\Program Files\ (x86)\IBM\Java70\jre\bin\java.exe -Xmx512m -classpath C:\Users\IBM_ADMIN\Desktop\Nessus_Scripts\nessusconverter.exe;...
From what I learned, this overrides my created environment variable and lowers the max heap size for this application. I guess it is coded in the called application. My problem is, that I don't know the author of the application and need it for my work. I am also time-limited, so some deep debugging or improving of this app is off limits.
Is there a way to override the option set in the application (without interfering in the app)?
I have Windows 7 and IBM SDK Java version 7.0.
Script calling the java app:
Function CallNessusConverter ( [Array]$input_files
[String]$output_filename, [String]$exceptions_filename,
[String]$hosts_filename, [String]$output_directory ) {
[String]$output_XML_file = $output_directory + "\" + $output_filename + ".xml"
[String]$output_Issue_file = $output_directory + "\" + $output_filename + "-Issues.csv"
[String]$output_Missed_file = $output_directory + "\" + $output_filename + "-NotScanned.csv"
[String]$input_XML_files = ""
foreach ($file in $input_files){
$input_XML_files += " " + $file
}
Write-Host "**** Calling nessusconverter ****"
[string]$CLI_nessusconverter = ".\nessusconverter -inputXMLfiles $input_files -exceptions $exceptions_filename -hosts $hosts_filename -outputXML $output_XML_file -outputIssue $output_Issue_file -rmunknown -outputMissed $output_Missed_file"
iex $CLI_nessusconverter
...
CallNessusConverter $input_files $output_filename $exceptions_filename $hosts_filename $output_directory`

As some kind of workaround, if there no other solution, you could try to replace the java.exe with a file that calls the real java with the desired parameters... Of course this is not a good idea if other applications use this java.exe as well.

Multiple stdin/stdout actions during one process call

I use Google Closure Compiler to compile automatically javascript using PHP (is needed to do it that way - in PHP, hovewer no security limitations on Windows machine). I wrote simple PHP script which calls process, pass .js content to stdin and receive recompiled .js via stdout. It works fine, problem is, when I compiling for example 40 .js files, it takes on strong machine almost 2 minutes. However, mayor delay is because java starts new instance of .jar app for every script. Is there any way how to modify script below to create process only one and send/receive .js content multiple times before process ends?
function compileJScript($s) {
$process = proc_open('java.exe -jar compiler.jar', array(
0 => array("pipe", "r"), 1 => array("pipe", "w")), $pipes);
if (is_resource($process)) {
fwrite($pipes[0], $s);
fclose($pipes[0]);
$output = stream_get_contents($pipes[1]);
fclose($pipes[1]);
if (proc_close($process) == 0) // If fails, keep $s intact
$s = $output;
}
return $s;
}
I can see several options, but don't know if it is possible and how to do it:
Create process once and recreate only pipes for every file
Force java to keep JIT-ed .jar in memory for much faster re-executing
If PHP can't do it, is possible to use bridge (another .exe file which will start fast every time, transfer stdin/out and redirects it to running compiler; if something like this even exists)

This is really a matter of coordination between the two process.
Here I wrote a quick 10-minutes script (just for the fun) that launches a JVM and sends an integer value, which java parses and returns incremented.. which PHP will just send it back ad-infinitum..
PHP.php
<?php
echo 'Compiling..', PHP_EOL;
system('javac Java.java');
echo 'Starting JVM..', PHP_EOL;
$pipes = null;
$process = proc_open('java Java', [0 => ['pipe', 'r'],
1 => ['pipe', 'w']], $pipes);
if (!is_resource($process)) {
exit('ERR: Cannot create java process');
}
list($javaIn, $javaOut) = $pipes;
$i = 1;
while (true) {
fwrite($javaIn, $i); // <-- send the number
fwrite($javaIn, PHP_EOL);
fflush($javaIn);
$reply = fgetss($javaOut); // <-- blocking read
$i = intval($reply);
echo $i, PHP_EOL;
sleep(1); // <-- wait 1 second
}
Java.java
import java.util.Scanner;
class Java {
public static void main(String[] args) {
Scanner s = new Scanner(System.in);
while (s.hasNextInt()) { // <-- blocking read
int i = s.nextInt();
System.out.print(i + 1); // <-- send it back
System.out.print('\n');
System.out.flush();
}
}
}
To run the script simply put those files in the same folder and do
$ php PHP.php
you should start seeing the numbers being printed like:
1
2
3
.
.
.
Note that while those numbers are printed by PHP, they are actually generated by Java

I don't think #1 from your list is possible because compiler.jar would need to have native support for keeping the process alive, which it doesn't (and if you consider that a compression algorithm needs the entire input before it can start processing data, it makes sense that the process doesn't stay alive).
According to Anyway to Boost java JVM Startup Speed? some people have been able to reduce their jvm startup times with nailgun
Nailgun is a client, protocol, and server for running Java programs
from the command line without incurring the JVM startup overhead.
Programs run in the server (which is implemented in Java), and are
triggered by the client (written in C), which handles all I/O.

Compiling a class using Java code using process

I have this piece of code that compiles a class called tspClassName, when I compile using this code:
Process compileProc = null;
try {
compileProc = Runtime.getRuntime().exec("javac -classpath ."
+ File.separator + "src" + File.separator
+ File.separator + "generated." + tspClassName + ".java -d ." + File.separator + "bin");
// catch exception
if (compileProc.exitValue() != 0)
{
System.out.println("Compile exit status: "
+ compileProc.exitValue());
System.err.println("Compile error:" +
compileProc.getErrorStream());
it outputs this:
"Compile exit status: 2
Compile error:java.io.FileInputStream#17182c1"
The class tspClassName.java compiles without errors otherwise, so I am guessing it has to do with the path,and in my eclipse project, the tspClassName.java resides in package homework4.generated inside src, is there something wrong with the path that I use in the code?
thanks

Your Java code runs a command that looks something like this:
javac -classpath ./src//generated.ClassName.java -d ./bin
I don't think that's what you want. I think you need to change your Java code so it maybe generates something like:
javac -classpath . src/generated/ClassName.java -d ./bin
^
Note the space after the classpath (".").

You can use the javax.tools.JavaCompiler or JCI that wrap this functionality.

I recommend doing something like this:
String command = String.format(
"javac -classpath . src%1$sgenerated%1$s%2$s.java -d .%1$sbin",
File.separator,
tspClassName
);
LOG("Executing " + command);
//... exec(command) etc
... where LOG is whatever your logging framework uses to log the command to be executed. This will help debugging immensely, since it was pointed out that the command you built is ill-constructed.
Alternately you can also build the string using replace
String command =
"javac -classpath . src/generated/ClassName.java -d ./bin"
.replace("/", File.separator)
.replace("ClassName", tspClassName);
This is perhaps more readable.
On draining Process streams
OP's comment suggests that waitFor() never returns. This is likely caused by compilation errors/warnings in javac process.
From the API:
Because some native platforms only provide limited buffer size for standard input and output streams, failure to promptly write the input stream or read the output stream of the subprocess may cause the subprocess to block, and even deadlock.
You need to continuously drain the Process.getOutputStream() et.al.
See also
Java Puzzlers, Puzzle 82: Beer Blast
Related questions
Draining standard error in Java

I think the proper way to do this kind of work is programatically using the javax.tools API, not an external process:
JavaCompiler compiler = ToolProvider.getSystemJavaCompiler()
Reference:
ToolProvider.getSystemJavaCompiler()

The problem could be with the file location instead of using single value parameter for exec()
try 3 parameter method which has the command, environment and location as parameters which helps us to move to the specified location and execute the command
check 6 and 8 methods for reference
http://docs.oracle.com/javase/7/docs/api/java/lang/Runtime.html
Process compile = Runtime.getRuntime().exec("javac "+fname,null,new File(dir));

firstly you should use apache exec library if you want to use processes like that. Apache exec library makes things very easy.
Secondly you should print your std output and std error streams of your process which you are executing. Without them its no way to know whats being executed and what's it doing.
Thirdly, try to print the full cmd line which the process is executing. Copy that cmd line and try to run it manually. Most of the time you would find your issues this way.
And finally if your aim is just to compile a class / generate or modify a class file at runtime give this a good read and try. It has examples too. You could also try code generation / class manipulation libraries like BCEL, JavaAssist etc.
Best of luck.

How to call java from C++

I need to run this line from my c++ program:
java -jar test.jar text1 text2
the java app will give a float value and give it to the c++ program.
How can I do this? I never call a java stuff before from my ms visual studio C++ file.

If you want strong coupling use JNI wrapper.

When I run the java command directly on my command prompt, it works. but when I run the command from the c++ file, the error says "The system cannot execute the specified program" .
here's my code, im using ms visual studio 2005 :
#include "stdafx.h"
#include <conio.h>
int _tmain(int argc, _TCHAR* argv[])
{
float value;
FILE *child = _popen("java -jar c:\simmetrics_jar_v1_6_2_d07_02_07.jar text1 ssdyr445", "r");
if (fscanf(child, "%f", &value) == 1)
{
fprintf(stdout,"Got Value from simmetrics: %f\n", value);
}
else
{
fprintf(stdout,"ERROR\n");
}
fclose(child);
return 0;
}

A simple solution is to use popen() and pclose().
The function popen(), forks() and execs() a sub processes. But it attaches the sub-processes standard-in and standard-out the stream object created by popen. Thus writting anything to the stream in the parent sends it to the standard-in of the sub-processes while anything the sub-processes writes to standard-out can be read from the stream by the parent:
double value;
FILE* child = popen("java -jar test.jar text1 text2", "r");
if (fscanf(child, "%f", &value) == 1)
{
fprintf(stdout,"Got Value: %f\n", value);
}
else
{
fprintf(stdout,"ERROR\n");
}
fclose(child);

The easiest if You can modify your java code:
write the result to environment variable (pseudo code below):
solution 1.
(Write directly to env. in java app.)
java:
...
setenv('ret', somefloatvalue);
...
exit..
c++:
system("java -jar test.jar text1 text2")
...
getenv("ret")
(I haven't test it, but important here is the context, does system creates another shell (console),
if yes, you'll not see those envs, therefore some other spawn method is necessary)
CreateProcess() on windows
fork() on linux.
There are also more complex solutions,
send some JASON's through the sockets.... etc.
Write to text file in java, read in
c++.
MPI...
extreme in this case CORBA ;)