I never thought about it.
But if you read a file you can use for example this code.
FileReader fileReader = new FileReader("c:\\data\\input-text.txt");
int data = fileReader.read();
while(data != -1) {
data = fileReader.read();
}
But how is actually recognised that the file ends. Is this because operating system know size of the file. Or is there a special character . I think java will call some C/C++ function from operating system and this function will return -1 , so java knows end of file is reached. But how does operating system know that file end is reached. Which special character is used for this.
How is actually End of File detected in java?
Java doesn't detect it. The operating system does.
The meaning of end-of-file depends on the nature of the "file" that you are reading.
If the file is a regular file in a file system, then the operating system knows or can find out what the actual file size is. It is part of the file's metadata.
If the file is a Socket stream, then end-of-file means that all available data has been consumed, and the OS knows that there cannot be any more. Typically, the socket has been closed or half closed.
If the file is a Pipe, then end-of-file means that the other end of the Pipe has closed it, and there will be no maore data.
If the file is a Linux/UNIX device file, then the precise end-of-file meaning will be device dependent. For example, if the device is a "tty" device on Linux/UNIX, it could mean:
the modem attached to the serial line has dropped out
the tty was in "cooked" mode and received the character that denotes EOF
and possibly other things.
It is common for a command shell to provide a way to signal an "end of file". Depending on the implementation, it may implement this itself, or it may be implemented at the device driver level. In either case, Java is not involved in the recognition.
I think java will call some C/C++ function from operating system and this function will return -1 , so java knows end of file is reached.
On Linux / UNIX / MacOS, the Java runtime calls the read(fd, buffer, count) native library method. That will return -1 if the fd is at the end-of-file position.
I see the chances of most popular file systems like ext and NTFS using a delimiter/ special char to mark the end of data as very slim. This is because files often have to store binary information too rather than text data and if the delimiter is present within its data, it can easily confuse the OS. In Linux, VFS (Virtual Filesystem Layer) offloads these details to implementations themselves and most of them construct a unique iNode (sort of like metadata) for every file that's resident in the filesystem. iNodes tend to have information on the blocks where the data is stored and also the exact size of the file among other things. Detecting EOF becomes is trivial when you have those.
Related
(This is a hypothetical question since it's very broad, and workarounds exist for specific cases.)
Is it possible to atomically write a byte[] to a file (as FileOutputStream or FileWriter?
If writing fails, then it's unacceptable that part of the array is written. For example, if the array is 1,000,000 bytes and the disk is full after 500,000 bytes, then no bytes should be written to the file, or the changes should somehow be rolled back. This should even be the case if a medium is physically disconnected mid-write.
Assume that the maximum size of the array is known.
Atomic writes to files are not possible. Operating systems don't support it, and since they don't, programming language libraries can't do it either.
The best you are going to get with a files in a conventional file system is atomic file renaming; i.e.
write new file into same file system as the old one
use FileDescriptor.sync() to ensure that new file is written
rename the new file over the old one; e.g. using
java.nio.file.Files.move(Path source, Path target,
CopyOption... options)
with CopyOptions ATOMIC_MOVE. According to the javadocs, this may not be supported, but if it isn't supported you should get an exception.
But note that the atomicity is implemented in the OS, and if the OS cannot give strong enough guarantees, you are out of luck.
(One issue is what might happen in the event of a hard disk error. If the disk dies completely, then atomicity is moot. But if the OS is still able to read data from the disk after the failure, then the outcome may depend on the OS'es ability to repair a possibly inconsistent file system.)
I've got a third party program which puts data into a windows named pipe.
I access the pipe with
String pipename = "\\\\.\\pipe\\the_pipe";
RandomAccessFile pipe = new RandomAccessFile(pipename, "r");
DataInputStream input = new DataInputStream(Channels.newInputStream(pipe.getChannel()));
So sometimes someone gets the 'perfect' idea to close the third party program before my small data converting tool which of course closes the pipe. When that happens, my tool writes down the last message received million times into the resultfile and successfully fills every hdd up to the last byte within hours, because I am not able to check if the pipe has been closed.
Things I've tried:
// checking file descriptor and file channel
if(!(pipe.getFD().valid() && pipe.getChannel().isOpen())) {
// quit
}
But both options do not trigger.
Is there a other way to access named pipes where such an information can be obtained?
Or do I have overlooked something?
When that happens, my tool writes down the last message received million times into the resultfile
Only if your tool ignores EOFExceptions and -1 return values from read().
I would recommend looking at JNA to access the named pipe directly. Yes, you can detect the pipe closure if you use JNA.
I have done so in a project called NuProcess available on github.com. Particularly, look at the com.zaxxer.nuprocess.windows package. Look at the WindowsProcess.createPipes() method for setting up pipes (the code creates both ends, you only need one).
On the read side, NuProcess is using Windows IOCompletionPorts for asynchronous I/O (in ProcessCompletions.java), which may be overkill for what you need (or not). But once you get your feet wet setting up the pipes, you should be able to figure it out from there reading the Microsoft API docs.
When a I do ls for a file in linux and I see: crw-rw---- what does the c imply?
I mean I read that c is for character device but is there any consequence on which Java API I/O methods I will use (e.g. BufferedWriter etc) or must not use?
I think this should give sufficient advice. FileInputStream and FileOutputStream are sufficient to access the device file which in turn talks to the character device.
This wikipedia explains the usage of the c character.
crw-rw-r-- a character special file whose user and group classes have the read and write permissions and whose others class has only the read permission.
This c character will not affect your ability to read this file, however you may want to check the write permissions if you are not in correct group. There are much better ways to check read/write permissions on files rather than reading through ls.
Use File.canWrite() and File.canRead() for checking file permissions. More info on these methods here - File - JavaDoc.
I have an applet that retrieves a byte array from a backend server. This byte array contains a dynamic library (DLL or SO, depending on which OS the applet is running on), that must be written to disk and subsequently loaded from the disk by a call to System.load().
I need to ensure that the file is not tampered with after it's been written to disk and before it's loaded by the OS through the call to System.load(). I obtain an exclusive lock on the file while it's written to disk, but my testing shows that I must release this lock before the call to System.load(), or it'll fail to load the library.
Is there some way I can keep the lock on the file while I load it?
Sample code:
File f = File.createTempFile("tmp", "");
RandomAccessFile raf = new RandomAccessFile(f, "rwd");
FileChannel channel = raf.getChannel();
FileLock lock = channel.lock(0, Long.MAX_VALUE, false);
// This would be where I write the DLL/SO from a byte array...
raf.write((int)65); // 'A'
raf.write((int)66); // 'B'
raf.write((int)67); // 'C'
System.out.println("Wrote dynamic library to file...");
// Close and release lock
raf.close();
System.out.println("File closed. Lock released.");
// This call fails if the file is still locked.
System.load(f.getAbsolutePath());
Any help is greatly appreciated. The solution (if there is any) must not be native to any OS, but work on all platforms supported by Java. It is also a requirement that the solution be compatible with Java 1.4.
In Java 7 you can implement in-memory file system (see java.nio.file.spi.FileSystemProvider), so library content will be completely in memory thus making attacker's life much harder.
Another possible approach is to sign the library and let OS do security checks after reading file from disk; might be not very portable though.
Most important - is it really the biggest security issue you're facing? The time between 2 calls will be some micro- (ok, maybe milli-) seconds - one must hack really deep into filesystem to do bad things. Wouldn't it be easier to alter file while it is transferred over network? Or don't you think an attacker that advanced may... let's say hack JVM and substitute content while library is written to disk? Nothing is bulletproof, maybe this is the risk you can accept?
Just out of interest - what exactly is the error you're getting?
I've to make a code to upload/download a file on remote machine. But when i upload the file new line is not saved as well as it automatically inserts some binary characters. Also I'm not able to save the file in its actual format, I've to save it as "filename.ser". I'm using serialization-deserialization concept of java.
Thanks in advance.
How exactly are you transmitting the files? If you're using implementations of InputStream and OutputStream, they work on a byte-by-byte level so you should end up with a binary-equal output.
If you're using implementations of Reader and Writer, they convert the bytes to characters according to some character mapping, and then perform the reverse process when saving. Depending on the platform encodings of the various machines (and possibly other effects if you're not specifying the charset explicitly), you could well end up with differences in the binary file.
The fact that you mention newlines makes me think that you're using Readers to send strings (and possibly that you're stitching the strings back together yourself by manually adding newlines). If you want the files to be binary equal, then send them as a stream of bytes and store that stream verbatim. If you want them to be equal as strings in a given character set, then use Readers and Writers but specify the character set explicitly. If you want them to be transmitted as strings in the platform default set (not very useful), then accept that they're not going to be binary equal as files.
(Also, your question really doesn't provide much information to solve it. To me, it basically reads "I wrote some code to do X, and it doesn't work. Where did I go wrong?" You seem to assume that your code is correct by not listing it, but at the same time recognise that it's not...)