I am currently trying to have Logstash work on Solaris with the File Input method. But I am encountering some bugs (see LOGSTASH-665). After digging a lot, it appears that native support for File.stat on my system (SunOS 5.10, JDK 1.6.0_21, 32 bit) is totally deficient, so I am looking for a way to properly handle it.
Specifically I want to access the inode information. Based on the metadata I can gather about the file (like its path and whatever is available on solaris), I want to calculate a number which is unique for that file, and which changes when the file is replaced by another file which has the same name. At first I thought about simply using a hash of the file path and used this function as a replacement, but indeed, when the file is rolled over the number does not change, so I need to also access the ctime information...
..Or make a system call to get the ls -li result to get the real inode number by another way.
Problem is that I never used ruby before, I am more used to java, so I am struggling to find a solution. Every suggestion will be appreciated.
The best solution I know of is to wrap the native call using JNI or JNA.
There do appear to be a couple of projects that have done this, although I haven't used either of them. See this question: Is there a Java library of Unix functions?
Related
How is the java utility that begins the process of launching a class told to "spill its guts" on what it's doing as it tries to load classes?
In particular, what file paths is it TRYING to access, only to perhaps discover whatever it's looking for is not there, at least as it interprets the specification given? There was a way to get that information, but I can't find it now.
Note that this is Java version "1.8.0_333" on Windows 10.
I've tried every flag known to me, via the -h and -X flags, and I strongly suspect what I'm looking for is (was) an X flag that's been removed, just as the -X help output warns. And so, there must be an OS way to figure this out, I sure hope!
You might ask why? Whatever for? What are you trying to do? Well, that's the bulk of this question's text. To wit:
As one of the very early users of Java (I started with 1.1) way back in the '90s, I had an issue moving an application suite I'd written for my company on Linux to MS Windows and I got it working by using Cygwin. Along the way, this same sort of issue came up and I quite vividly recall having found a mechanism for getting the Java launcher to articulate just what file specifications - paths - it was actually using in searching for the appropriate class. And through using this, I found that the CLASSPATH was being specified incorrectly, and with some experimentation, I got it working reliably. Now I need to do that again!
This flag I'd used was immensely helpful in figuring out just what the file specification format CLASSPATH needed to be (we're not talking semicolons here) this combination of OS, Java, and Cygwin. After some hours of what I hope was reasonable hunting, I'm wondering if this capability has been removed at some point? Either that or "I'm looking for the wrong thing." Heck, since the source is available (I think!), maybe some brave soul has hacked the java utility to do such a thing?
It may help to understand that for this application I wrote for my company, it was a major goal to have the source work pretty much the same on all Windows and Linux / Unix systems (and at the time, macOS), and just use a configuration file to tell the code what's different. And that wasn't easy to figure out, but with this flag, it wasn't that hard, either.
But, unfortunately, I haven't needed this knowledge since I figured it all out all those years ago, and apparently, this little kernel of knowledge is very hard to find today. Or, it's no longer pertinent to the modern version(s).
I don't think this has anything much to do with the actual problem, but it may help in people's thinking if they understood the scenario: The current situation is that I have a fully functional installation of this software on Windows 7 to use as a comparison for how to configure things on Windows 10 (and hopefully younger). The Windows 7 is running a pretty modern Cygwin installation and very nearly the most modern Java - just a sub-version away from the new installation from last week on a Windows 10 box. (Everything's bright and shiny new on the new box.)
The required format for CLASSPATH on the nearly identical but fully functional Windows 7 system is:
CLASSPATH="C:/opt/OurInstallationDir/lib"
And that's it.
This value is picked up in several places as the code later needs to launch Java itself to do some unusual things. However, the java command that gets it all going is launched from a C program - not that that matters for this problem - but the C program (compiled under Cygwin, but perfectly runnable from any Windows environment) helps ensure that the Java environment is secured (policy file contents and so forth) before getting into Java, else it refuses. And this program on Windows 11 launches Java just fine, it's just giving it a CLASSPATH that isn't useful, apparently, even though the files are there where they should be, etc.
Configuring things as before just doesn't work, even from the command line. No version of specifying CLASSPATH seems to work if it's more than a dot; the only thing that works, is being in the /lib directory when starting and using "-cp ." ... But that's just not going to fly for so many reasons! To be a little more clear, I've tried reversing the slashes, using /cygdrive/c/, and whatever else I could think of. But, at least we know that if you're in the directory and use -cp, it will find and launch the program. So, there's nothing wrong with the Java, just pointing the java utility at it.
Again: How is the java utility that begins the process of launching a class told to "spill its guts" on what it's doing as it's trying to load classes?
You use this construct on the JVM:
java -XX:+TraceClassPaths -cp "C:\opt\SomeDirectory\lib" myClass
I was able to get confirmation of what Java was using, not only for my CLASSPATH, but "internally" by using the above.
The fact that it echoed back both what I was doing and what it was doing somehow gave me the insight to check everything about it. Java itself doesn't work (at all) if it's installed in a location that it thinks has a link in it, and it's own fetches go right back to the system disk specification.
From that I found that Java on Windows won't take a CLASSPATH that has a link in it!
Simply ensuring that the whole tree was specified "from the top" of the drive it's on works. If it's not, it won't.
It's now working happily using the syntax noted above.
This is quite different from every other application I've seen on Windows. But, well, it's Java!
This really came from a pointer from Mark Rotteveel who commented above about this article: How to track when class is loaded and destroyed in jvm? And therein I learned how to get the list of all the options the presently in-use JVM supports. All Java developers should be aware of this in my opinion, so thanks to Mark for that.
I need to read data from the mic input in realtime in my Java program. I experimented with TaroDSP a bit and while it does provide a fairly straightforward API for this, its latency is too high for my purposes.
I was thinking that maybe I can implement this in Max/DSP, write a program that would simply read the mic input levels, export it as a jar and include it in my java program as a library and simply call it from within my java program to query the mic input levels, something like:
ExportedMaxObject o = new ExportedMaxObject();
int currentMicLevel = o.getMicInputLevel();
I know Max/Msp makes somehow possible to use mxj and mxj~ Java objects to access data to and from Max but it is not clear to me how they are used. Do I need to have Max running or installed in my computer in order to use it? I am looking for a solution where I can embed this into my Java program and use my own UI so I can distribute it without any dependencies to Max/Msp.
Thanks in advance for any help.
I would say it is very unlikely that you will find a way to load a Max patch in Java.
The other way around might be possible: building an app in Max that incorporates your Java work, however it's important to note that Java support in Max has not received a lot of attention since Apple dropped it.
Currently I am using file.delete() but it is showing a security risk for this as files deleted like this can be recovered by different means. So please provide me a correct way to delete a file. The security risk depicted here is provided by a testing tool called Quixxi and it checks for any vulnerability in app.
The reason a "deleted" file is recoverable is because a delete operation simply unlinks the file in the filesystem, so the directory no longer considers that file part of it. The contents on disk (or whatever storage) still exist on that device.
If you want to guarantee the contents can never be recovered, you have to overwrite the contents first. There are no built-in functions to do this - you'd have to find a library or write the code yourself. Typically you'd write something like all 0s over the file (make sure to flush to media), write all 1s, write a pattern of 01 repeating, 10 repeating, something like that. After you've written with garbage patterns to media (flush) a few times, then you issue the delete.
Not possible in JRE, unfortunately. The JVM is not designed for that, and you need OS-dependent utilities.
The answer by user1676075 contains a mistake. Let's go by steps.
As pointed out already, Java's File.delete method only unlinks the file leaving its contents on disk. It actually invokes the underlying OS APIs to perform this unlink operation.
The problem occurs when you want to overwrite contents in Java.
Java can open a file for overwrite, but will leverage OS utils to do so. And the OS will likely:
Unlink the allocated space on disk
Link the file to a new free area of disk
The result is that you are now writing tons of zeroes... somewhere else!!!
And even if you managed to write zeroes on the same sectors used by the original file, Gutmann method exists for a reason. Gutmann utilities require root/Administrator (Super User) permissions and direct DMA access to precisely control where the writes have to occur.
And with SSDs, things changes. Actually, it might get easier! At this point, I should provide source for SSDs having a CLEAR instructions to replace a sector with zeroes and that privacy-savy disk controllers do that. But maybe pretend you have read nothing.
This will be a sufficient answer for now, because we have demonstrated that there is no out-of-the-box and straightforward way to securely clear a file in Java.
What Java allows, and is called Java Native Interfaces (please also see Java Native Access), is to call native code from Java. So, you got your Gutmann tool in C++ ready? Are you running root? You can write code to invoke Gutmann-ish erasure from Java, but that's a whole other point.
Never tried, but surely feasible
For whatever reason, I sometimes need to find the current user's My Documents folder on Windows in a Java program to read some files. But as far as I can tell, there is no way to do it that isn't severely flawed.
The first wrong way: System.getProperty("user.home");
Why it won't work:
It only returns the \username\ folder; I'd need to add "\Documents\" on to the end to get the Documents folder... and that only works in English.
Sun bugs 6519127 and 4787931. Java finds the user home folder on Windows by reading a deprecated registry key* to find the Desktop then taking the parent; this method has multiple known problems that will easily cause a completely wrong folder to be returned. The bugs are 3.75 years and 8 years old with no fix.
The second wrong way: Using a registry-reading program to get the Personal folder of the user, which is My Documents (but i18n'd).
Why it won't work:
While it fixes the English-only problem, it's still using the same deprecated registry area, so the bugs still apply to it.
The deprecated registry key says to use a native call (SHGetKnownFolderPath) which I obviously can't do from Java.
The third wrong way:
JFileChooser fr = new JFileChooser();
FileSystemView fw = fr.getFileSystemView();
File documents = fw.getDefaultDirectory();
Why it won't work: It works great!
Except when it doesn't. While I had a program that used this open and running in the background, I opened a DirectX game (Fallout: New Vegas). The Java program immediately terminated with no stack trace. Always reproducible (for me on that game, and who knows what else). Couldn't find a Sun bug#.
So is there any method to find a user's Documents folder, on Windows, from Java, that doesn't have known problems?
(This is a nice big question.)
*(The key is "HKCU\Software\Microsoft\Windows\CurrentVersion\Explorer\Shell Folders")
There's no pure java way to do it, but you could use the JNA wrapper over JNI to do it without having to write any native code yourself. There's a good example of how to get the Documents folder on Windows halfway down the responses at:
What is the best way to find the users home directory in Java?
A time consuming, but reliable way of finding the 'Documents' folder of a windows user: Make your java app execute a bat script that uses Reg.exe (a windows system file) to find the value of the reg key which has the path in it. Then use a pipeline in the same bat file to send that data to the 'findstr' function which windows command prompt has. Use another pipeline to output the returned value to a text file. Then, simply make your java app read that text file, and delete it once its done :) Worked well enough for me.
Code for the bat file:
# echo off
Title Find Documents Folder
Reg Query "HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Explorer\Shell Folders" |findstr "Personal">>DocPath.dat
exit
There is a custom Java API that someone built (their website no longer works), but there code remains on Google Code:
http://winfoldersjava.googlecode.com/files/WinFoldersJava_1.1.zip
There are two DLL's that need to be referenced, one for each architecture(x86 and x64).
user.home is not "my documents", but users home folder, like on Unix ~/.
To get to "My documents" you can use System.getProperty("user.home")+"\Documents"; irrespective of the language system. Try it.
In my Scala (runs on top of Java) Application I would like to get a list of all drives that contain DVD media, e.g. something like this:
/dev/scd0 Star Trek DS9 DVD1
/dev/scd0 The 4400 DVD1
Not sure if it's possible to get the name of the disc, but the path is the important thing for me anyway.
I would prefer a pure Java / Scala solution (using file.io stuff). If that's not possible, accessing the right Linux files is fine, too (like /proc/something).
Thanks in advance!
I think you're out of luck with java.io.* but if you don't mind making calls out to Linux commands, you could assemble the data by:
Calling "mount" and capturing the first column of output.
Calling "volname" on each value you captured from step 1.
According to the man page for volname, it only returns data for ISO-9660 filesystems (e.g. DVDs), so any device path that returns empty can be ignored.
There is one (untested) possibility to get your drives with pure Java code. At least on Windows.
Its a little bit hacky and doesn't work under linux (because linux gets not as much integration love from sun I believe).
import javax.swing._
import javax.swing.filechooser._
val chooser = new JFileChooser()
val view = chooser.getFileSystemView()
The FileSystemView clas provides a few features such as asking the possible roots if they
are a drive (isDrive()). Swing uses this to present the file chooser with the right icons to
you so it should work under windows because IIRC it shows the correct symbols there. Under
Linux it unfortunately does only show the "/" root.
One of the reasons this doesn't work under linux could be, that the linux developers constantly change their preferred way of presenting such information to the user space. at the moment it is IIRC hal and dbus. Maybe SUN didn't want to publish a new java version each time this changes.
If pure java doesn't cut it maybe you could use a little bit of jni (which is not so hard to use anymore if you're using tools like JNA or such) to access the linux apis directly. I haven't done that but could try if you're interested.