We Keep Coding

Writing to $HOME from a jar file

I'm trying to write to a file located in my $HOME directory. The code to write to that file has been packaged into a jar file. When I run the unit tests to package the jar file, everything works as expected - namely the file is populated and can be read from again.
When I try to run this code from another application where the jar file is contained the lib directory it fails. The file is created - but the file is never written to. When the app goes to read the file it fails parsing it because it is empty.
Here is the code that writes to the file:
logger.warn("TestNet wallet does not exist creating one now in the directory: " + walletPath)
testNetFileName.createNewFile()
logger.warn("Wallet file name: " + testNetFileName.getAbsolutePath)
logger.warn("Can write: "+ testNetFileName.canWrite())
logger.warn("Can read: " + testNetFileName.canRead)
val w = Wallet.fromWatchingKey(TestNet3Params.get(), testNetSeed)
w.autosaveToFile(testNetFileName, savingInterval, TimeUnit.MILLISECONDS, null)
w
}
here is the log form the above method that is relevant:
2015-12-30 15:11:46,416 - [WARN] - from class com.suredbits.core.wallet.ColdStorageWallet$ in play-akka.actor.default-dispatcher-9
TestNet wallet exists, reading in the one from disk
2015-12-30 15:11:46,416 - [WARN] - from class com.suredbits.core.wallet.ColdStorageWallet$ in play-akka.actor.default-dispatcher-9
Wallet file name: /home/chris/testnet-cold-storage.wallet
then it bombs.
Here is the definition for autoSaveToFile
public WalletFiles autosaveToFile(File f, long delayTime, TimeUnit timeUnit,
#Nullable WalletFiles.Listener eventListener) {
lock.lock();
try {
checkState(vFileManager == null, "Already auto saving this wallet.");
WalletFiles manager = new WalletFiles(this, f, delayTime, timeUnit);
if (eventListener != null)
manager.setListener(eventListener);
vFileManager = manager;
return manager;
} finally {
lock.unlock();
}
}
and the definition for WalletFiles
https://github.com/bitcoinj/bitcoinj/blob/master/core/src/main/java/org/bitcoinj/wallet/WalletFiles.java#L68
public WalletFiles(final Wallet wallet, File file, long delay, TimeUnit delayTimeUnit) {
// An executor that starts up threads when needed and shuts them down later.
this.executor = new ScheduledThreadPoolExecutor(1, new ContextPropagatingThreadFactory("Wallet autosave thread", Thread.MIN_PRIORITY));
this.executor.setKeepAliveTime(5, TimeUnit.SECONDS);
this.executor.allowCoreThreadTimeOut(true);
this.executor.setExecuteExistingDelayedTasksAfterShutdownPolicy(false);
this.wallet = checkNotNull(wallet);
// File must only be accessed from the auto-save executor from now on, to avoid simultaneous access.
this.file = checkNotNull(file);
this.savePending = new AtomicBoolean();
this.delay = delay;
this.delayTimeUnit = checkNotNull(delayTimeUnit);
this.saver = new Callable<Void>() {
#Override public Void call() throws Exception {
// Runs in an auto save thread.
if (!savePending.getAndSet(false)) {
// Some other scheduled request already beat us to it.
return null;
}
log.info("Background saving wallet, last seen block is {}/{}", wallet.getLastBlockSeenHeight(), wallet.getLastBlockSeenHash());
saveNowInternal();
return null;
}
};
}
I'm guessing it is some sort of permissions issue but I cannot seem to figure this out.
EDIT: This is all being run on the exact same Ubuntu 14.04 machine - no added complexity of different operating systems.

You cannot generally depend on the existence or writability of $HOME. There are really only two portable ways to identify (i.e. provide a path to) an external file.
Provide an explicit path using a property set on the invocation command line or provided in the environment, or
Provide the path in a configuration properties file whose location is itself provided as a property on the command line or in the environment.
The problem with using $HOME is that you cannot know what userID the application is running under. The user may or may not even have a home directory, and even if the user does, the directory may or may not be writable. In your specific case, your process may have the ability to create a file (write access on the directory itself) but write access to a file may be restricted by the umask and/or ACLs (on Windows) or selinux (on Linux).
Put another way, the installer/user of the library must explicitly provide a known writable path for your application to use.
Yet another way to think about it is that you are writing library code that may be used in completely unknown environments. You cannot assume ANYTHING about the external environment except what is in the explicit contract between you and the user. You can declare in your interface specification that $HOME must be writable, but that may be highly inconvenient for some users whose environment doesn't have $HOME writable.
A much better and portable solution is to say
specify -Dcom.xyz.workdir=[path] on the command line to indicate the work path to be used
or
The xyz library will look for its work directory in the path specified by the XYZ_WORK environment variable
Ideally, you do BOTH of these to give the user some flexibility.

savePending is always false. In the beginning of call you check that it is false, and return null. The actual save code is never executed. I am guessing you meant to check if it was true there, and also set it to true, not false. You then also need to reset it back to false in the end.
Now, why this works in your unit test is a different story. The test must be executing different code.

Detecting whether an application was launched from a read-only file system on OS X

I want to know whether the user launched our Java-based application from a read-only file system like from a .dmg, so functions like auto-update will be able to show meaningful information instead of aborting with an error. I first thought checking the .app's path would be sufficient (when launched from a .dmg it is something like /Volumes/MyApp 1.2.3/MyApp.app, but this won't work, because the user might have installed the application on a different partition. What other things may I check?

You can use -[NSURL getResourceValue:forKey:error:] with the key NSURLVolumeIsReadOnlyKey. You would apply this to the app bundle URL as returned by [[NSBundle mainBundle] bundleURL]. So:
NSBundle* bundle = [NSBundle mainBundle];
NSURL* bundleURL = bundle.bundleURL;
NSNumber* readOnly;
NSError* error;
if ([bundleURL getResourceValue:&readOnly forKey:NSURLVolumeIsReadOnlyKey error:&error])
{
BOOL isReadOnly = [readOnly boolValue];
// act on isReadOnly value
}
else
{
// Handle error
}

If OSX is POSIX compliant, to determine if filesystem is mounted R/O, You can use statvfs() or fstatvfs(), returned struct statvfs field f_flag should have ST_RDONLY bit set for R/O filesystem.
As it was pointed in comments, check if this information is correctly provided by OS.
JNA and this question may be usefull for Java.
A few more ideas, which may be usefull here (access(), open(), utime() ).
OS X specific statfs() may be used too, but this function is not portable (Linux and *BSD have slightly different statfs() functions).

You can also check directly from Java whether a certain path points to something within a read-only directory by querying the FileStore associated with your path:
File classpathRoot = new File(MyClass.class.getClassLoader().getResource("").getPath());
/* getPath() actually returns a String instead of a Path object,
* so we need to take this little detour */
Path yourAppPath = classpathRoot.toPath();
boolean isReadOnly = Files.getFileStore(yourAppPath).isReadOnly();

Simplicity for executing Java from C++

Background info: I am used to program in Java and I know how to use Eclipse and Visual Studio.
Final objective: to create a GUI, preferably in Visual Studio, which executes Java functions.
What I wish to accomplish from this question: a button in C++ which, on click, executes a Java function and returns the results to C++. (probably by invoking a JVM)
I've currently considered the following datastructures:
Sharing data through 'common' files such as .txt files (but then how do I start the Java functions?)
Opening a socket (seems too complicated for this problem)
Connecting through a server (too complicated)
Invoking a JVM from C++ which then executes the Java file (I think this is the most reasonable way but this needs a lot of code)
Now I know about the existence of Jace, JNI and SWIG but I think they are very handy for making complicated programs, not easy interfaces. I don't want to make a complicated program hence I feel that learning all their commands is quite bothersome.
I have also read up on a lot of Stack Exchange questions asking the exact same thing but all of them seem to give very complicated answers.
So here is my question:
What is the absolute simplest way to execute a (if necessary: precompiled) Java function from C++ where the C++ code passes some arguments to this Java function
Thanks in advance.

Invoking a JVM from C++ which then executes the Java file (I think this is the most reasonable way but this needs a lot of code)
Yes, it definitely is the most reasonable way. And with JNI and the invocation API it's not even that much code.
Finding the jvm.dll
You could try things like hardcoding the path to the Oracle JVM's jvm.dll or searching for a file called jvm.dll in the programs folder, but all that is obviously extremely hacky. However, there is apparently a pretty easy solution: The registry. The key HKEY_LOCAL_MACHINE\SOFTWARE\JavaSoft\Java Runtime Environment contains a REG_SZ called CurrentVersion. You can read the value of this key (currently it's 1.7) and open a child key with that name (HKEY_LOCAL_MACHINE\SOFTWARE\JavaSoft\Java Runtime Environment\1.7 in this example). That key will then contain a REG_SZ called RuntimeLib which is the path to your jvm.dll. Don't worry about Program files vs Program files (x86). WOW64 will automatically redirect your registry query to HKLM\SOFTWARE\Wow6432Node if you're a 32bit process on a 64bit windows and that key contains the path to the 32 bit jvm.dll. Code:
#include <Windows.h>
#include <jni.h> // C:\Program Files\Java\jdk1.7.0_10\include\jni.h
// ...
DWORD retval;
// fetch jvm.dll path from registry
HKEY jKey;
if (retval = RegOpenKeyEx(HKEY_LOCAL_MACHINE, TEXT("SOFTWARE\\JavaSoft\\Java Runtime Environment"), 0, KEY_READ, &jKey))
{
RegCloseKey(jKey);
// assuming you're using C++/CLI
throw gcnew System::ComponentModel::Win32Exception(retval);
}
TCHAR versionString[16]; // version numbers shouldn't be longer than 16 chars
DWORD bufsize = 16 * sizeof(TCHAR);
if (retval = RegGetValue(jKey, NULL, TEXT("CurrentVersion"), RRF_RT_REG_SZ, NULL, versionString, &bufsize))
{
RegCloseKey(jKey);
// assuming you're using C++/CLI
throw gcnew System::ComponentModel::Win32Exception(retval);
}
TCHAR* dllpath = new TCHAR[512];
bufsize = 512 * sizeof(TCHAR);
retval = RegGetValue(jKey, versionString, TEXT("RuntimeLib"), RRF_RT_REG_SZ, NULL, dllpath, &bufsize)
RegCloseKey(jKey);
if (retval)
{
delete[] dllpath;
// assuming you're using C++/CLI
throw gcnew System::ComponentModel::Win32Exception(retval);
}
Loading the jvm.dll and getting the CreateJavaVM function
This part is pretty straightforward, you just use LoadLibrary and GetProcAddress:
HMODULE jniModule = LoadLibrary(dllpath);
delete[] dllpath;
if (jniModule == NULL)
throw gcnew System::ComponentModel::Win32Exception();
typedef int (JNICALL * JNI_CreateJavaVM)(JavaVM** jvm, JNIEnv** env, JavaVMInitArgs* initargs);
JNI_CreateJavaVM createJavaVM = (JNI_CreateJavaVM)GetProcAddress(jniModule, "JNI_CreateJavaVM");
Creating the JVM
Now you can invoke that function:
JavaVMInitArgs initArgs;
initArgs.version = JNI_VERSION_1_6;
initArgs.nOptions = 0;
JavaVM* jvm;
JNIEnv* env;
if ((retval = createJavaVM(&jvm, &env, &initArgs)) != JNI_OK)
throw gcnew System::Exception(); // beyond the scope of this answer
Congratulations! There's now a JVM running right inside your process! You would probably launch the JVM at the startup of your application. Unless you are 100% sure that you will only ever invoke Java code from the thread that just created the JVM, you can throw away the env pointer, but you have to keep the jvm pointer.
Getting the JNI environment (optional)
So now you created the JVM and your application is up and running and then somebody clicks that button. Now you want to invoke Java code. If you are 100% sure that you are right now on the thread that created the JVM in the previous step and you still have the env pointer, then you can skip this. Otherwise, perform a quick check if the current thread is attached to the JVM and attach it if it isn't:
JNIEnv* env;
bool mustDetach = false;
jint retval = jvm->GetEnv((void**)&env, JNI_VERSION_1_6);
if (retval == JNI_EDETACHED)
{
JavaVMAttachArgs args;
args.version = JNI_VERSION_1_6;
args.name = NULL;
args.group = NULL;
retval = jvm->AttachCurrentThread(&env, &args);
mustDetach = true; // to clean up afterwards
}
if (retval != JNI_OK)
throw gcnew System::Exception(); // should never happen
invokeJavaCode(env); // next step
if (mustDetach)
jvm->DetachCurrentThread();
Invoking Java code
Now you are right there, you want to invoke that Java code and you even have the env pointer. You want the easiest solution, so this is how you call a static method:
jclass clazz = env->FindClass("com/myself/MyClass");
if (clazz == NULL)
throw gcnew System::Exception();
jmethodID mid = env->GetStaticMethodID(clazz, "myStaticMethod", "<signature>");
if (mid == NULL)
throw gcnew System::Exception();
<type> returnedValue = env->CallStatic<type>Method(clazz, mid, <arguments>);
You can use javap -s (command line tool) to determine a method's signature. <type> can be any primitive type (it must match the return type of the Java method). The arguments can be of any primitive type, as long as they match the arguments of the Java method.
The end
And there you have it: The easiest way to invoke Java code from C++ on Windows (actually only the first two parts are windows-specific...). Oh, and also the most efficient one. Screw databases and files. Using 127.0.0.1 sockets would be an option but that's significantly less efficient and probably not less work than this. Wow, this answer is a bit longer than I expected. Hopefully it helps.

Any sure fire way to check file existence on Linux NFS? [duplicate]

This question already has answers here:
Alternative to File.exists() in Java
(6 answers)
Closed 2 years ago.
I am working on a Java program that requires to check the existence of files.
Well, simple enough, the code make use calls to File.exists() for checking file existence. And the problem I have is, it reports false positive. That means the file does not actually exist but exists() method returns true. No exception was captured (at least no exception like "Stale NFS handle"). The program even managed to read the file through InputStream, getting 0 bytes as expected and yet no exception. The target directory is a Linux NFS. And I am 100% sure that the file being looked for never exists.
I know there are known bugs (kind of API limitation) exist for java.io.File.exists(). So I've then added another way round by checking file existence using Linux command ls. Instead of making call to File.exists() the Java code now runs a Linux command to ls the target file. If exit code is 0, file exists. Otherwise, file does not exist.
The number of times the issue is hit seems to be reduced with the introduction of the trick, but still pops. Again, no error was captured anywhere (stdout this time). That means the problem is so serious that even native Linux command won't fix for 100% of the time.
So there are couple of questions around:
I believe Java's well known issue on File.exists() is about reporting false negative. Where file was reported to not exist but in fact does exist. As the API does not throws IOException for File.exists(), it choose to swallow the Exception in the case calls to OS's underlying native functions failed e.g. NFS timeout. But then this does not explain the false positive case I am having, given that the file never exist. Any throw on this one?
My understanding on Linux ls exit code is, 0 means okay, equivalent to file exists. Is this understanding wrong? The man page of ls is not so clear on explaining the meaning of exit code: Exit status is 0 if OK, 1 if minor problems, 2 if serious trouble.
All right, back to subject. Any surefire way to check File existence with Java on Linux? Before we see JDK7 with NIO2 officially released.

Here is a JUnit test that shows the problem and some Java Code that actually tries to read the file.
The problem happens e.g. using Samba on OSX Mavericks. A possible reason
is explaned by the statement in:
http://appleinsider.com/articles/13/06/11/apple-shifts-from-afp-file-sharing-to-smb2-in-os-x-109-mavericks
It aggressively caches file and folder properties and uses opportunistic locking to enable better caching of data.
Please find below a checkFile that will actually attempt to read a few bytes and forcing a true file access to avoid the caching misbehaviour ...
JUnit test:
/**
* test file exists function on Network drive replace the testfile name and ssh computer
* with your actual environment
* #throws Exception
*/
#Test
public void testFileExistsOnNetworkDrive() throws Exception {
String testFileName="/Volumes/bitplan/tmp/testFileExists.txt";
File testFile=new File(testFileName);
testFile.delete();
for (int i=0;i<10;i++) {
Thread.sleep(50);
System.out.println(""+i+":"+OCRJob.checkExists(testFile));
switch (i) {
case 3:
// FileUtils.writeStringToFile(testFile, "here we go");
Runtime.getRuntime().exec("/usr/bin/ssh phobos /usr/bin/touch "+testFileName);
break;
}
}
}
checkExists source code:
/**
* check if the given file exists
* #param f
* #return true if file exists
*/
public static boolean checkExists(File f) {
try {
byte[] buffer = new byte[4];
InputStream is = new FileInputStream(f);
if (is.read(buffer) != buffer.length) {
// do something
}
is.close();
return true;
} catch (java.io.IOException fnfe) {
}
return false;
}

JDK7 was released a few months ago. There are exists and notExists methods in the Files class but they return a boolean rather than throwing an exception. If you really want an exception then use FileSystems.getDefault().provider().checkAccess(path) and it will throw an exception if the file does not exist.

If you need to be robust, try to read the file - and fail gracefully if the file is not there (or there is a permission or other problem). This applies to any other language than Java as well.
The only safe way to tell if the file exist and you can read from it is to actually read a data from the file. Regardless of a file system - local, or remote. The reason is a race condition which can occur right after you get success from checkAccess(path): check, then open file, and you find it suddenly does not exist. Some other thread (or another remote client) may have removed it, or has acquired an exclusive lock. So don't bother checking access, but rather try to read the file. Spending time in running ls just makes race condition window easier to fit.

Pinning a Java application to the Windows 7 taskbar

I use Launch4j as a wrapper for my Java application under Windows 7, which, to my understanding, in essence forks an instance of javaw.exe that in turn interprets the Java code. As a result, when attempting to pin my application to the task bar, Windows instead pins javaw.exe. Without the required command line, my application will then not run.
As you can see, Windows also does not realize that Java is the host application: the application itself is described as "Java(TM) Platform SE binary".
I have tried altering the registry key HKEY_CLASSES_ROOT\Applications\javaw.exe to add the value IsHostApp. This alters the behavior by disabling pinning of my application altogether; clearly not what I want.
After reading about how Windows interprets instances of a single application (and a phenomenon discussed in this question), I became interested in embedding a Application User Model ID (AppUserModelID) into my Java application.
I believe that I can resolve this by passing a unique AppUserModelID to Windows. There is a shell32 method for this, SetCurrentProcessExplicitAppUserModelID. Following Gregory Pakosz suggestion, I implemented it in an attempt to have my application recognized as a separate instance of javaw.exe:
NativeLibrary lib;
try {
lib = NativeLibrary.getInstance("shell32");
} catch (Error e) {
Logger.out.error("Could not load Shell32 library.");
return;
}
Object[] args = { "Vendor.MyJavaApplication" };
String functionName = "SetCurrentProcessExplicitAppUserModelID";
try {
Function function = lib.getFunction(functionName);
int ret = function.invokeInt(args);
if (ret != 0) {
Logger.out.error(function.getName() + " returned error code "
+ ret + ".");
}
} catch (UnsatisfiedLinkError e) {
Logger.out.error(functionName + " was not found in "
+ lib.getFile().getName() + ".");
// Function not supported
}
This appears to have no effect, but the function returns without error. Diagnosing why is something of a mystery to me. Any suggestions?
Working implementation
The final implementation that worked is the answer to my follow-up question concerning how to pass the AppID using JNA.
I had awarded the bounty to Gregory Pakosz' brilliant answer for JNI that set me on the right track.
For reference, I believe using this technique opens the possibility of using any of the APIs discussed in this article in a Java application.

I don't have Windows 7 but here is something that might get you started:
On the Java side:
package com.stackoverflow.homework;
public class MyApplication
{
static native boolean setAppUserModelID();
static
{
System.loadLibrary("MyApplicationJNI");
setAppUserModelID();
}
}
And on the native side, in the source code of the `MyApplicationJNI.dll library:
JNIEXPORT jboolean JNICALL Java_com_stackoverflow_homework_MyApplication_setAppUserModelID(JNIEnv* env)
{
LPCWSTR id = L"com.stackoverflow.homework.MyApplication";
HRESULT hr = SetCurrentProcessExplicitAppUserModelID(id);
return hr == S_OK;
}
Your question explicitly asked for a JNI solution. However, since your application doesn't need any other native method, jna is another solution which will save you from writing native code just for the sake of forwarding to the windows api. If you decide to go jna, pay attention to the fact that SetCurrentProcessExplicitAppUserModelID() is expecting a UTF-16 string.
When it works in your sandbox, the next step is to add operating system detection in your application as SetCurrentProcessExplicitAppUserModelID() is obviously only available in Windows 7:
you may do that from the Java side by checking that System.getProperty("os.name"); returns "Windows 7".
if you build from the little JNI snippet I gave, you can enhance it by dynamically loading the shell32.dll library using LoadLibrary then getting back the SetCurrentProcessExplicitAppUserModelID function pointer using GetProcAddress. If GetProcAddress returns NULL, it means the symbol is not present in shell32 hence it's not Windows 7.
EDIT: JNA Solution.
References:
The JNI book for more JNI examples
Java Native Access (JNA)

There is a Java library providing the new Windows 7 features for Java. It's called J7Goodies by Strix Code. Applications using it can be properly pinned to the Windows 7 taskbar. You can also create your own jump lists, etc.

I have implemented access to the SetCurrentProcessExplicitAppUserModelID method using JNA and it works quite well when used as the MSDN documentation suggests. I've never used the JNA api in the way you have in your code snippet. My implementation follows the typical JNA usage instead.
First the Shell32 interface definition:
interface Shell32 extends StdCallLibrary {
int SetCurrentProcessExplicitAppUserModelID( WString appID );
}
Then using JNA to load Shell32 and call the function:
final Map<String, Object> WIN32API_OPTIONS = new HashMap<String, Object>() {
{
put(Library.OPTION_FUNCTION_MAPPER, W32APIFunctionMapper.UNICODE);
put(Library.OPTION_TYPE_MAPPER, W32APITypeMapper.UNICODE);
}
};
Shell32 shell32 = (Shell32) Native.loadLibrary("shell32", Shell32.class,
WIN32API_OPTIONS);
WString wAppId = new WString( "Vendor.MyJavaApplication" );
shell32.SetCurrentProcessExplicitAppUserModelID( wAppId );
Many of the API's in the last article you mentioned make use of Windows COM which is quite difficult to use directly with JNA. I have had some success creating a custom DLL to call these API's (eg. using the SHGetPropertyStoreForWindow to set a different app ID for a submodule window) which I then use JNA to access at runtime.

Try to use JSmooth. I use always this one. In JSmooth is there an option under Skeleton by Windowed Wrapper called
Lauch java app in exe process
See on this image.
(source: andrels.com)
Also command line arguments can be passed.
I think this can be a solution for you.
Martijn

SetCurrentProcessExplicitAppUserModelID (or SetAppID()) would in fact do what you're trying to do. However, it might be easier to modify your installer to set the AppUserModel.ID property on your shortcut - quoting from the Application User Model ID document mentioned above:
In the System.AppUserModel.ID property of the application's shortcut file. A shortcut (as an IShellLink, CLSID_ShellLink, or a .lnk file) supports properties through IPropertyStore and other property-setting mechanisms used throughout the Shell. This allows the taskbar to identify the proper shortcut to pin and ensures that windows belonging to the process are appropriately associated with that taskbar button.
Note: The System.AppUserModel.ID property should be applied to a shortcut when that shortcut is created. When using the Microsoft Windows Installer (MSI) to install the application, the MsiShortcutProperty table allows the AppUserModelID to be applied to the shortcut when it is created during installation.

The latest jna-platform library now includes JNA bindings for SetCurrentProcessExplicitAppUserModelID:
https://github.com/java-native-access/jna/pull/680

I fixed mine without any ID settings.
There is an option in Launch4J if you are using it and you say you do then...
You can change the header to JNI Gui and then wrap it around the jar with the JRE.
The good thing is that it runs .exe in the process now instead on running javaw.exe with your jar. It probably does it under the hood (not sure).
Also I have noticed also that it takes around 40-50% less CPU resource which is even better!
And the pinning works fine and all that window features are enabled.
I hope it helps to someone as I spent nearly 2 days trying to solve that issue with my undecorated javafx app.