We Keep Coding

Using rsync4j library, a Java implementation of Linux rsync tool, how to sync data from a remote directory with a local directory

I need the functionality like that of the rsync linux tool in my Java program. For that, I chose the rsync4j library.
Using their documentation, I wrote the following program:
import com.github.fracpete.processoutput4j.output.ConsoleOutputProcessOutput;
import com.github.fracpete.rsync4j.RSync;
public class MainClass {
public static void main(String [] args) {
System.out.println("Started");//check
RSync rsync = new RSync()
.source("/home/arth/DataSourceFolder/a.txt")
.destination("/home/arth/DataDestinationFolder/")
.recursive(true);
// or if you prefer using commandline options:
// rsync.setOptions(new String[]{"-r", "/one/place/", "/other/place/"});
CollectingProcessOutput output = null;
try {
System.out.println("Inside try");
output = rsync.execute();
System.out.println("End of try");
} catch (Exception e) {
e.printStackTrace();
}
System.out.println(output.getStdOut());
System.out.println("Exit code: " + output.getExitCode());
if (output.getExitCode() > 0)
System.err.println(output.getStdErr());
}
}
In the snippet, in out local machine, a file a.txt is copied from one location to another. This works perfectly. The file is successfully copied when I run it and here is the output:
Started
Inside try
End of try
Exit code: 0
But my need is to sync a local directory with a directory lying at a remote host/machine. When I tried to do it using a simple rsync command from a terminal using the following command
rsync remoteUserName#23.24.25.244:/home/beth/remoteFolder/a.png /home/arth/DataSourceFolder
it works like a charm. a.png IS copied to local machine at path specified, although a password of remote machine is asked first.
But the problem when I use the above Java program to do the same operation, by replacing line # 11 and 12 by:
.source("remoteUserName#23.24.25.244:/home/beth/remoteFolder/a.png")
.destination("/home/arth/DataDestinationFolder/")
the program gets stuck after printing Started in the console. Neither an exception is thrown nor does the program proceed.
The question is that how do I fix this problem?

(old post, I know, but here it goes...) The rsync4j library does not allow interaction. In your case, the underlying rysnc binary prompts for a password in the process that the Java library created, but never receives one.
Starting with release 3.2.3-7, you can supply an instance of the sshpass wrapper to feed in the password (see this comment for an example).

Manually loading native libraries to circumvent a restrictive environment

I'm maintaining a Java Swing application that requires a connection to an instance of Microsoft SQL Server. For various reasons, I opted to replace the native SQL Server driver being used with jTDS (the aforementioned Microsoft drivers were not working at the time and have apparently failed in the field as well). When I try to run the executable .jar outside of the IDE, I run into issues because I'm missing the appropriate ntlmauth.dll dependency.
Before proceeding, it's important to note that this application is being developed and used in an extremely restrictive (Windows-only) environment:
I cannot install any software that requires Windows UAC authentication
My users cannot install or run any software that requires UAC authentication
This currently means I cannot write files to System32 or JAVA_HOME, and cannot use any sort of ProcessBuilder tomfoolery to start another JVM with whatever command line arguments I need
I cannot use executable wrappers/installers that would only require the UAC permission for the first time installation/setup
The solution I'm trying is a combination of this one and this one to check it--essentially packaging the .dll inside of the .jar, then extracting it and loading it if necessary--as most of the other solutions I've found have been incompatible with the above restrictions; however, I'm running into an issue where even after the native library is ostensibly "loaded," I get an exception saying it isn't.
My pre-startup code:
private static final String LIB_BIN = "/lib-bin/";
private static final String JTDS_AUTH = "ntlmauth";
// load required JTDS binaries
static {
logger.info("Attempting to load library {}.dll", JTDS_AUTH);
try {
System.loadLibrary(JTDS_AUTH);
} catch (UnsatisfiedLinkError e) {
loadFromJar();
}
try {
// do some quick checks to make sure that went ok
NativeLibraries nl = new NativeLibraries();
logger.debug("Loaded libraries: {}", nl.getLoadedLibraries().toString());
} catch (NoSuchFieldException ex) {
logger.info("Native library checker load failed", ex);
}
}
/**
* When packaged into JAR extracts DLLs, places these into
*/
private static void loadFromJar() {
// we need to put DLL in temp dir
String path = ***;
loadLib(path, JTDS_AUTH);
}
/**
* Puts library to temp dir and loads to memory
*/
private static void loadLib(String path, String name) {
name = name + ".dll";
try {
// have to use a stream
InputStream in = net.sourceforge.jtds.jdbc.JtdsConnection.class.getResourceAsStream(LIB_BIN + name);
// always write to different location
File fileOut = new File(System.getProperty("java.io.tmpdir") + "/" + path + LIB_BIN + name);
logger.info("Writing dll to: " + fileOut.getAbsolutePath());
OutputStream out = FileUtils.openOutputStream(fileOut);
IOUtils.copy(in, out);
in.close();
out.close();
System.load(fileOut.toString());
} catch (Exception e) {
logger.error("Exception with native library loader", e);
JOptionPane.showMessageDialog(null, "Exception loading native libraries: " + e.getLocalizedMessage(), "Exception", JOptionPane.ERROR_MESSAGE);
}
}
As you can see, I basically copied the solution from the first link verbatim, with a few minor modifications just to try and get the application running. I also copied the class from the second link and named it NativeLibraries, the invocation of that method is fairly irrelevant but it shows up in the logs.
Anyway here are the relevant bits of the log output on starting up the application:
2015-07-20 12:32:33 INFO - Attempting to load library ntlmauth.dll
2015-07-20 12:32:33 INFO - Writing dll to: C:\Users\***\lib-bin\ntlmauth.dll
2015-07-20 12:32:33 DEBUG - Loaded libraries: [C:\Program Files\Java\jre1.8.0_45\bin\zip.dll, C:\Program Files\Java\jre1.8.0_45\bin\prism_d3d.dll, C:\Program Files\Java\jre1.8.0_45\bin\prism_sw.dll, C:\Program Files\Java\jre1.8.0_45\bin\msvcr100.dll, C:\Program Files\Java\jre1.8.0_45\bin\glass.dll, C:\Program Files\Java\jre1.8.0_45\bin\net.dll, C:\Users\***\lib-bin\ntlmauth.dll]
2015-07-20 12:32:33 INFO - Application startup
***
2015-07-20 12:32:36 ERROR - Database exception
java.sql.SQLException: I/O Error: SSO Failed: Native SSPI library not loaded. Check the java.library.path system property.
at net.sourceforge.jtds.jdbc.TdsCore.login(TdsCore.java:654) ~[jtds-1.3.1.jar:1.3.1]
at net.sourceforge.jtds.jdbc.JtdsConnection.<init>(JtdsConnection.java:371) ~[jtds-1.3.1.jar:1.3.1]
at net.sourceforge.jtds.jdbc.Driver.connect(Driver.java:184) ~[jtds-1.3.1.jar:1.3.1]
at java.sql.DriverManager.getConnection(Unknown Source) ~[na:1.8.0_45]
at java.sql.DriverManager.getConnection(Unknown Source) ~[na:1.8.0_45]
One can see that the library was, indeed, "loaded," from the third line in the log (it's the last entry, if you don't feel like scrolling). However, I simply used the class that I felt like was probably using the native libraries (I also tried the TdsCore class to no avail), as the example that showed how to do this was just using a random class from the package the library was needed in.
Is there something I'm missing here? I'm not very experienced with the JNI or the inner workings of ClassLoaders, so I might just be loading it wrong. Any advice or suggestions would be greatly appreciated!

Welp I figured out a workaround: I ended up using JarClassLoader. This basically entailed copying all my dependencies, both Java and native, into a "libraries" folder within my main .jar, and disabling .jar signing in the IDE. The application is then run by a new class that simply creates a new JarClassLoader object and running the "invokeMain" method--an example is on the website. The whole thing took about three minutes, after several days of banging my head against a wall.
Hope this helps someone someday!

UnsatsifiedLinkError on JSSC Serial Interface (Processing)

I'm using Proclipsing (processing in Eclipse) but am getting an error when I try and open a port (printing the serial list works fine). I have a feeling some sort of native library is not connected but I'm baffled as to how to do so in Eclipse (and where it link to).
Here's my code:
import processing.core.PApplet;
import processing.serial.Serial;
public class visualization extends PApplet {
public Serial usb = null;
public void setup() {
println(Serial.list());
println(Serial.list()[5]);
usb = new Serial(this, Serial.list()[5], 115200);
}
public void draw() {
}
}
and the error it throws is:
Exception in thread "Animation Thread" java.lang.UnsatisfiedLinkError: jssc.SerialNativeInterface.openPort(Ljava/lang/String;Z)J
at jssc.SerialNativeInterface.openPort(Native Method)
at jssc.SerialPort.openPort(SerialPort.java:158)
at processing.serial.Serial.<init>(Unknown Source)
at processing.serial.Serial.<init>(Unknown Source)
at bioauthvisualization3.BioauthVisualization3.setup(BioauthVisualization3.java:15)
at processing.core.PApplet.handleDraw(PApplet.java:2361)
at processing.core.PGraphicsJava2D.requestDraw(PGraphicsJava2D.java:240)
at processing.core.PApplet.run(PApplet.java:2256)
at java.lang.Thread.run(Thread.java:745)

I ran into the same issue while setting up IntelliJ IDEA for development with Processing. But I imagine you can apply the same method in Proclipsing. The solution that worked for me was threefold.
I created a global library that contained the Processing and Serial libraries. In OS X the required jar files can be found in the directory /Applications/Processing.app/Contents/Java/ and /Applications/Processing.app/Contents/Java/modes/libraries/serial/library/. I added the following libraries: core.jar, serial.jarand jssc.jar.
Next I had to add the global library to my module dependencies.
The last step was to add the path to the native libraries to the VM options for the appletviewer: -Djava.library.path="/Applications/Processing.app/Contents/Java/modes/java/libraries/serial/library/macosx"
My guess is that Proclipsing does the first two steps for you but you have to add the native libraries to the vm options manually since these depend on the system you're working on. Hope that helps.
I derived the solution for step 3 from this forum entry.

Java: CaptureDeviceManager#getDeviceList() is empty?

I am trying to print out all of the capture devices that are supported using the #getDeviceList() method in the CaptureDeviceManager class and the returned Vector has a size of 0.
Why is that? I have a webcam that works - so there should be at least one. I am running Mac OS X Lion - using JMF 2.1.1e.
Thanks!

CaptureDeviceManager.getDeviceList(Format format) does not detect devices. Instead it reads from the JMF registry which is the jmf.properties file. It searches for the jmf.properties file in the classpath.
If your JMF install has succeeded, then the classpath would have been configured to include all the relevant JMF jars and directories. The JMF install comes with a jmf.properties file included in the 'lib' folder under the JMF installation directory. This means the jmf.properties would be located by JMStudio and you would usually see the JMStudio application executing correctly. (If your JMF install is under 'C:\Program Files', then run as administrator to get around UAC)
When you create your own application to detect the devices, the problem you described above might occur. I have seen a few questions related to the same problem. This is because your application's classpath might be different and might not include the environment classpath. Check out your IDE's properties here. The problem is that CaptureDeviceManager cannot find the jmf.properties file because it is not there.
As you have found out correctly, you can copy the jmf.properties file from the JMF installation folder. It would contain the correct device list since JMF detects it during the install (Check it out just to make sure anyway).
If you want do device detection yourself, then create an empty jmf.properties file and put it somewhere in your classpath (it might throw a java.io.EOFException initially during execution but that's properly handled by the JMF classes). Then use the following code for detecting webcams...
import javax.media.*;
import java.util.*;
public static void main(String[] args) {
VFWAuto vfwObj = new VFWAuto();
Vector devices = CaptureDeviceManager.getDeviceList(null);
Enumeration deviceEnum = devices.elements();
System.out.println("Device count : " + devices.size());
while (deviceEnum.hasMoreElements()) {
CaptureDeviceInfo cdi = (CaptureDeviceInfo) deviceEnum.nextElement();
System.out.println("Device : " + cdi.getName());
}
}
The code for the VFWAuto class is given below. This is part of the JMStudio source code. You can get a good idea on how the devices are detected and recorded in the registry. Put both classes in the same package when you test. Disregard the main method in the VFWAuto class.
import com.sun.media.protocol.vfw.VFWCapture;
import java.util.*;
import javax.media.*;
public class VFWAuto {
public VFWAuto() {
Vector devices = (Vector) CaptureDeviceManager.getDeviceList(null).clone();
Enumeration enum = devices.elements();
while (enum.hasMoreElements()) {
CaptureDeviceInfo cdi = (CaptureDeviceInfo) enum.nextElement();
String name = cdi.getName();
if (name.startsWith("vfw:"))
CaptureDeviceManager.removeDevice(cdi);
}
int nDevices = 0;
for (int i = 0; i < 10; i++) {
String name = VFWCapture.capGetDriverDescriptionName(i);
if (name != null && name.length() > 1) {
System.err.println("Found device " + name);
System.err.println("Querying device. Please wait...");
com.sun.media.protocol.vfw.VFWSourceStream.autoDetect(i);
nDevices++;
}
}
}
public static void main(String [] args) {
VFWAuto a = new VFWAuto();
System.exit(0);
}
}
Assuming you are on a Windows platform and you have a working web-cam, then this code should detect the device and populate the jmf.properties file. On the next run you can also comment out the VFWAuto section and it's object references and you can see that CaptureDeviceManager reads from the jmf.properties file.
The VFWAuto class is part of jmf.jar. You can also see the DirectSoundAuto and JavaSoundAuto classes for detecting audio devices in the JMStudio sample source code. Try it out the same way as you did for VFWAuto.
My configuration was Windows 7 64 bit + JMF 2.1.1e windows performance pack + a web-cam.

I had the same issue and I solved by invoking flush() on my ObjectInputStream object.
According to the API documentation for ObjectInputStream's constructor:
The stream header containing the magic number and version number are read from the stream and verified. This method will block until the corresponding ObjectOutputStream has written and flushed the header.
This is a very important point to be aware of when trying to send objects in both directions over a socket because opening the streams in the wrong order will cause deadlock.
Consider for example what would happen if both client and server tried to construct an ObjectInputStream from a socket's input stream, prior to either constructing the corresponding ObjectOutputStream. The ObjectInputStream constructor on the client would block, waiting for the magic number and version number to arrive over the connection, while at the same time the ObjectInputStream constructor on the server side would also block for the same reason. Hence, deadlock.
Because of this, you should always make it a practice in your code to open the ObjectOutputStream and flush it first, before you open the ObjectInputStream. The ObjectOutputStream constructor will not block, and invoking flush() will force the magic number and version number to travel over the wire. If you follow this practice in both your client and server, you shouldn't have a problem with deadlock.
Credit goes to Tim Rohaly and his explanation here.

Before calling CaptureDeviceManager.getDeviceList(), the available devices must be loaded into the memory first.
You can do it manually by running JMFRegistry after installing JMF.
or do it programmatically with the help of the extension library FMJ (Free Media in Java). Here is the code:
import java.lang.reflect.Field;
import java.util.Vector;
import javax.media.*;
import javax.media.format.RGBFormat;
import net.sf.fmj.media.cdp.GlobalCaptureDevicePlugger;
public class FMJSandbox {
static {
System.setProperty("java.library.path", "D:/fmj-sf/native/win32-x86/");
try {
final Field sysPathsField = ClassLoader.class.getDeclaredField("sys_paths");
sysPathsField.setAccessible(true);
sysPathsField.set(null, null);
} catch (Exception e) {
e.printStackTrace();
}
}
public static void main(String args[]) {
GlobalCaptureDevicePlugger.addCaptureDevices();
Vector deviceInfo = CaptureDeviceManager.getDeviceList(new RGBFormat());
System.out.println(deviceInfo.size());
for (Object obj : deviceInfo ) {
System.out.println(obj);
}
}
}
Here is the output:
USB2.0 Camera : civil:\\?\usb#vid_5986&pid_02d3&mi_00#7&584a19f&0&0000#{65e8773d-8f56-11d0-a3b9-00a0c9223196}\global
RGB, -1-bit, Masks=-1:-1:-1, PixelStride=-1, LineStride=-1

Office 2007 is unable to open files when called through JACOB from a service

I'm using JACOB to do COM calls to PowerPoint and other Office applications from Java. On a particular Windows 7 box I'm getting the following message quite often, but not always:
Source: Microsoft Office PowerPoint 2007
Description: PowerPoint could not open the file.
From excel I get:
ERROR - Invoke of: Open
Source: Microsoft Office Excel
Description: Microsoft Office Excel cannot access the file 'c:\marchena\marchena10\work\marchena\batch_58288\input\content_1.xlsx'. There are several possible reasons:
? The file name or path does not exist.
? The file is being used by another program.
? The workbook you are trying to save has the same name as a currently open workbook.
The Word error is just:
VariantChangeType failed
The following is what I'm running, the error comes from the last line.
ComThread.InitSTA();
slideApp = new ActiveXComponent("PowerPoint.Application");
Dispatch presentations = slideApp.getProperty("Presentations").toDispatch();
Dispatch presentation = Dispatch.call(presentations, "Open", inputFile.getAbsolutePath(),
MsoTriState.msoTrue.getInteger(), // ReadOnly
MsoTriState.msoFalse.getInteger(), // Untitled The Untitled parameter is used to create a copy of the presentation.
MsoTriState.msoFalse.getInteger() // WithWindow
).toDispatch();
I've tried putting a breakpoint just before doing the Open call and the file is there, and I can actually open it with PowerPoint in the GUI but when I step the exception is thrown.
The annoying thing about this issue is that it seems to happen continuously to begin with, but after poking at it for a while (rerunning the same code), it eventually completes successfully, and after that never reoccurs.
Further research I've found this only happens with to .ppt, .doc and .xls files, not .pptx, .docx and .xlsx. And as far as I can tell it's not file system related (I've swapped out the mechanism that copies the files and tried putting the files on a different file system).
I've just noticed that this only happens when the Java application is running as a service, not when I run catalina.bat start from command line.

I had the same problem (jacob in service not working), and this helpful posting (changing the dcomcnfg) did the trick:
http://bytes.com/topic/c-sharp/answers/819740-c-service-excel-application-workbooks-open-fails-when-called-service#post3466746

Does this work for you?
import com.jacob.activeX.ActiveXComponent;
import com.jacob.com.ComThread;
import com.jacob.com.Dispatch;
import com.jacob.com.Variant;
public class PPT {
private static final String inputFile = "c:\\learning.ppt";
public static void main(String[] args) {
ActiveXComponent slideApp = new ActiveXComponent("PowerPoint.Application");
slideApp.setProperty("Visible", new Variant(true));
ActiveXComponent presentations = slideApp.getPropertyAsComponent("Presentations");
ActiveXComponent presentation = presentations.invokeGetComponent("Open",new Variant(inputFile), new Variant(true));
ComThread.Release();
}
}
Update: If this is working the client and it's just automation that is causing the issues, you can view http://support.microsoft.com/kb/257757 to look at possible issues. There error codes are obviously different, but it may help you troubleshoot all the same.