I am attempting to convert a program that reads a binary file in C++ to java. The file is in little-endian.
fread(n, sizeof (unsigned), 1, inputFile);
The snippet above of c++ reads 1 integer into the integer variable 'n'.
I am currently using this method to accomplish the same thing:
public static int readInt(RandomAccessFile inputStream) throws IOException {
int retVal;
byte[] buffer = new byte[4];
inputStream.readFully(buffer);
ByteBuffer wrapped = ByteBuffer.wrap(buffer);
wrapped.order(ByteOrder.LITTLE_ENDIAN);
retVal = wrapped.getInt();
return retVal;
}
but this method sometimes differs in its result to the c++ example. I haven't been able to determine which parts of the file cause this method to fail, but I know it does. For example, when reading one part of the file my readInt method returns 543974774 but the C++ version returns 1.
Is there a better way to read little endian values in Java? Or is there some obvious flaw in my implementation? Any help understanding where I could be going wrong, or how could I could read these values in a more effective way would be very appreciated.
Update:
I am using RandomAcccessFile because I frequently require fseek functionality which RandomAccessFile provides in java
543974774 is, in hex, 206C6576.
There is no endianness on the planet that turns 206C6576 into '1'. The problem is therefore that you aren't reading what you think you're reading: If the C code is reading 4 bytes (or even a variable, unknown number of bytes) and turns that into '1', then your java code wasn't reading the same bytes - your C code and java code is out of sync: At some point, your C code read, for example, 2 bytes, and your java code read 4 bytes, or vice versa.
The problem isn't in your readInt method - that does the job properly every time.
I would like to know the specific difference between BufferedReader and FileReader.
I do know that BufferedReader is much more efficient as opposed to FileReader, but can someone please explain why (specifically and in detail)? Thanks.
First, You should understand "streaming" in Java because all "Readers" in Java are built upon this concept.
File Streaming
File streaming is carried out by the FileInputStream object in Java.
// it reads a byte at a time and stores into the 'byt' variable
int byt;
while((byt = fileInputStream.read()) != -1) {
fileOutputStream.write(byt);
}
This object reads a byte(8-bits) at a time and writes it to the given file.
A practical useful application of it would be to work with raw binary/data files, such as images or audio files (use AudioInputStream instead of FileInputStream for audio files).
On the other hand, it is very inconvenient and slower for text files, because of looping through a byte at a time, then do some processing and store the processed byte back is tedious and time-consuming.
You also need to provide the character set of the text file, i.e if the characters are in Latin or Chinese, etc. Otherwise, the program would decode and encode 8-bits at a time and you'd see weird chars printed on the screen or written in the output file (if a char is more than 1 byte long, i.e. non-ASCII characters).
File Reading
This is just a fancy way of saying "File streaming" with inclusive charset support (i.e no need to define the charset, like earlier).
The FileReader class is specifically designed to deal with the text files.
As you've seen earlier, the file streaming is best to deal with raw binary data, but for the sake of text, it is not so efficient.
So the Java-dudes added the FileReader class, to deal specifically with the text files. It reads 2 bytes (or 4 bytes, depends on the charset) at a time. A remarkably huge improvement over the preceding FileInputStream!!
so the streaming operation is like this,
int c;
while ( (c = fileReader.read()) != -1) { // some logic }
Please note, Both classes use an integer variable to store the value retrieved from the input file (so every char is converted into an integer while fetching and back to the char while storing).
The only advantage here is that this class deals only with text files, so you don't have to specify the charset and a few other properties. It provides an out-of-the-box solution, for most of the text files processing cases. It also supports internationalization and localization.
But again it's still very slow (Imaging reading 2 bytes at a time and looping through it!).
Buffering streams
To tackle the problem of continuous looping over a byte or 2. The Java-dudes added another spectacular functionality. "To create a buffer of data, before processing."
The concept is pretty much alike when a user streams a video on YouTube. A video is buffered before playing, to provide flawless video watching experience. (Tho, the browser keeps buffering until the whole video is buffered ahead of time.) The same technique is used by the BufferedReader class.
A BufferedReader object takes a FileReader object as an input which contains all the necessary information about the text file that needs to be read. (such as the file path and charset.)
BufferedReader br = new BufferedReader( new FileReader("example.txt") );
When the "read" instruction is given to the BufferedReader object, it uses the FileReader object to read the data from the file. When an instruction is given, the FileReader object reads 2 (or 4) bytes at a time and returns the data to the BufferedReader and the reader keeps doing that until it hits '\n' or '\r\n' (The end of the line symbol).
Once a line is buffered, the reader waits patiently, until the instruction to buffer the next line is given.
Meanwhile, The BufferReader object creates a special memory place (On the RAM), called "Buffer", and stores all the fetched data from the FileReader object.
// this variable points to the buffered line
String line;
// Keep buffering the lines and print it.
while ((line = br.readLine()) != null) {
printWriter.println(line);
}
Now here, instead of reading 2 bytes at a time, a whole line is fetched and stored in the RAM somewhere, and when you are done with processing the data, you can store the whole line back to the hard disk. So it makes the process run way faster than doing 2 bytes a time.
But again, why do we need to pass FileReader object to the BufferReader? Can't we just say "buffer this file" and the BufferReader would take care of the rest? wouldn't that be sweet?
Well, the BufferReader class is created in a way that it only knows how to create a buffer and to store incoming data. It is irrelevant to the object from where the data is coming. So the same object can be used for many other input streams than just text files.
So being said that, When you provide the FileReader object as an input, it buffers the file, the same way if you provide the InputStreamReader as an object, it buffers the Terminal/Console input data until it hits a newline symbol. such as,
// Object that reads console inputs
InputStreamReader console = new InputStreamReader(System.in);
BufferedReader br = new BufferedReader(console);
System.out.println(br.readLine());
This way, you can read (or buffer) multiple streams with the same BufferReader class, such as text files, consoles, printers, networking data etc, and all you have to remember is,
bufferedReader.readLine();
to print whatever you've buffered.
In simple manner:
A FileReader class is a general tool to read in characters from a File. The BufferedReader class can wrap around Readers, like FileReader, to buffer the input and improve efficiency. So you wouldn't use one over the other, but both at the same time by passing the FileReader object to the BufferedReader constructor.
Very Detail
FileReader is used for input of character data from a disk file. The input file can be an ordinary ASCII, one byte per character text file. A Reader stream automatically translates the characters from the disk file format into the internal char format. The characters in the input file might be from other alphabets supported by the UTF format, in which case there will be up to three bytes per character. In this case, also, characters from the file are translated into char format.
As with output, it is good practice to use a buffer to improve efficiency. Use BufferedReader for this. This is the same class we've been using for keyboard input. These lines should look familiar:
BufferedReader stdin =
new BufferedReader(new InputStreamReader( System.in ));
These lines create a BufferedReader, but connect it to an input stream from the keyboard, not to a file.
Source: http://www.oopweb.com/Java/Documents/JavaNotes/Volume/chap84/ch84_3.html
BufferedReader requires a Reader, of which FileReader is one - it descends from InputStreamReader, which descends from Reader.
FileReader - read character files
BufferedReader - "Read text from a character-input stream, buffering characters so as to provide for the efficient reading of characters, arrays, and lines."
http://docs.oracle.com/javase/7/docs/api/java/io/BufferedReader.html
http://docs.oracle.com/javase/7/docs/api/java/io/FileReader.html
Actually BufferedReader makes use of Readers like FileReader.
FileReader class helps in writing on file but its efficency is low since it has yo retrive one character at a time from file but BufferedReader takes chunks of data and store it in buffer so instead of retriving one character at atime from file retrival becomes easy using buffer.
Bufferedreader - method that you can use actually as a substitute for Scanner method, gets file, gets input.
FileReader - as the name suggests.
I am writing to a file for the first time, but the text on the file comes out completely wrong. Instead of numbers (which it is supposed to print), it prints unrecognizable characters. I can't seem to understand why this is happening? (in my code the print statement is inside a for loop, but this is the "shell" around the loop)
Is there a logical explanation for this?
try {
FileWriter outFile = new FileWriter("newFile.txt", true);
outFile.write(number);
} catch (IOException e) {
e.printStackTrace();
}
You're calling Writer.write(int):
Writes a single character. The character to be written is contained in the 16 low-order bits of the given integer value; the 16 high-order bits are ignored.
My guess is that's not what you want to do. If you want to write the text representation of a number, you need to do that explicitly:
outFile.write(String.valueOf(number));
(Personally I'd recommend using OutputStreamWriter wrapped around a FIleOutputStream, as then you can - and should - specify an encoding. FileWriter always uses the platform default encoding.)
I create buffered writer
BufferedWriter errorReport = new BufferedWriter(new FileWriter("ErrorReport.txt"));
Then I wanted to write while converting my integer to a hex.
errorReport.write(Integer.toHexString(version))
This works perfectly, except it omits leading 0's as it writes the minimum possible length. Say 'version' is a byte in length and simply prints 6. Well I know the actual value is actually 06. How do I keep these leading 0's?
I tried errorReport.write(String.format("%03x", Integer.toHexString(version)), but get an error for illegalFormatConversionException x != java.lang.String
The x specifies hexadecimal so format will perform the conversion by passing the integer directly
errorReport.write(String.format("%03x", version));
You're very close. The JVM is complaining that you are trying to hex-format a string. Try errorReport.write(String.format("%03x", version))
I'm trying to convert plain text data to binary format so that it becomes non-redable.
The data needs to be written to a file. The conversion works if I print it in console window, and cannot read original text.
However, when it is written to a file, the same original text appears.
How can I write this binary data to a file, without encrypting it, but making it non-readable?
This file, later needs to be processed by another third party tool which accepts binary data. This is why I cannot encrypt it using my own algo.
This is my code:
import java.io.*;
import java.lang.*;
public class convert{
public static void main( String args[] ){
String s= "This is text";
try{
File file= new File("test.php");
file.createNewFile();
FileWriter fw= new FileWriter( file.getName(), true );
BufferedWriter bw= new BufferedWriter( fw );
byte[] b= s.getBytes();
for(int x=0; x<b.length; x++){
byte c=b[x];
bw.write( c );
System.out.println(c);
}
bw.close();
}catch( Exception e ){ e.printStackTrace(); }
}//main
}//class
Plain text is in itself a binary format, where each byte is interpreted as a character (or other variants, depending on assumed or specified encoding, e.g. UTF-8, UTF-16...).
This means that if you write an ASCII character as a byte to a file, it will look identical and will be readable by anyone. In fact, a lot of binary file formats still save strings as normal bytes, which means they can be read when opened in a hex editor, such as here:
What you will need to do to make it unreadable is to serialize it into some common format that is not readable. Normal serialization in Java is unfortunately readable, but you can check here for advice on how to obscure it. You can also use ZIP or similar compression algorithms as well.
Another, more hacky way, is to shift all your character bytes by some known value. This will result in them becoming a different character and it will be unreadable. This can be seen as a very basic Caesar Chipher.
But in the end, all that matters is which formats your target program is able to read.
Well you can use a RandomAccessFile for this. It becomes easy.
RandomAccessFile raf = new RandomAccessFile(path,permissions); //permissions are r,w,rw. The usual ones.
raf.seek(0); //this will set the pointer to first position.
raf.writeUTF(what ever you want to write to file);
//to read the file use readUTF()
raf.seek(0); //you can read from a part of file using this seek method.
System.out.println(raf.readUTF());