We Keep Coding

Data loss when writing bytes to a file

I'm working on a string compressor for a school assignment,
There's one bug that I can't seem to work out. The compressed data is being written a file using a FileWriter, represented by a byte array. The compression algorithm returns an input stream so the data flows as such:
piped input stream
-> input stream reader
-> data stored in char buffer
-> data written to file with file writer.
Now, the bug is, that with some very specific strings, the second to last byte in the byte array is written wrong. and it's always the same bit values "11111100".
Every time it's this bit values and always the second to last byte.
Here are some samples from the code:
InputStream compress(InputStream){
//...
//...
PipedInputStream pin = new PipedInputStream();
PipedOutputStream pout = new PipedOutputStream(pin);
ObjectOutputStream oos = new ObjectOutputStream(pout);
oos.writeObject(someobject);
oos.flush();
DataOutputStream dos = new DataOutputStream(pout);
dos.writeFloat(//);
dos.writeShort(//);
dos.write(SomeBytes); // ---Here
dos.flush();
dos.close();
return pin;
}
void write(char[] cbuf, int off, int len){
//....
//....
InputStreamReader s = new InputStreamReader(
c.compress(new ByteArrayInputStream(str.getBytes())));
s.read(charbuffer);
out.write(charbuffer);
}
A string which triggers it is "hello and good evenin" for example.
I have tried to iterate over the byte array and write them one by one, it didn't help.
It's also worth noting that when I tried to write to a file using the output stream in the algorithm itself it worked fine. This design was not my choice btw.
So I'm not really sure what i'm doing wrong here.

Considering that you're saying:
Now, the bug is, that with some very specific strings, the second to
last byte in the byte array is written wrong. and it's always the same
bit values "11111100".
You are taking a
binary stream (the compressed data)
-> reading it as chars
-> then writing it as chars.
And your are converting bytes to chars without clearly defining the encoding.
I'd say that the problem is that your InputStreamReader is translating some byte sequences in a way that you're not expecting.
Remember that in encodings like utf-8 two or three bytes may become one single char.
It can't be coincidence that the very byte pattern you pointed out (11111100) Is one of the utf-8 escape codes (1111110x). Check this wikipedia table at and you'll see that uft-8 is destructive since if a byte starts with: 1111110x the next must start with 10xxxxxx.
Meaning that if using utf-8 to convert
bytes1[] -> chars[] -> bytes2[]
in some cases bytes2 will be different from bytes1.
I recommend changing your code to remove those readers. Or specify ASCII encoding to see if that prevent the translations.

I solved this by encoding and decoding the bytes with Base64.

How do buffers work in java IO?

I'm having trouble understanding how do buffers work in Java IO.
Excuse me if I don't express myself as clearly as I would like, I'm not
strong on all these concepts.
As I undestand it, in Java there are readers/writers, for chars (meaning the
possibility of more than one byte per char and encoding options), and streams
for bytes.
Then there are some classes that use buffers.
I believe that a buffer is used mainly so that we can avoid unnecessary system
calls that would involve expensive operations, like accesing a slower device, by
storing more in memory and making the system call useful for more data.
The issue I have is that there seem to be buffering classes for both readers/writers and streams.
I would like to know if buffering characters is enough, or if, by the time those
bytes get to the streaming class, they would be flushed on for example newlines,
as some classes state.
The best I've found about this is this post
How do these different code snippets compare in regard to buffering?
Does autoflush thwart the intent of buffering?
Should there be only one buffer in play, and if so, where (writer or stream)
and why?:
// resolveSocket is some socket
PrintWriter out = new PrintWriter(
resolveSocket.getOutputStream(),
true);
// vs
PrintWriter out = new PrintWriter(
new OutputStreamWriter(
new BufferedOutputStream(resolveSocket.getOutputStream()),
StandardCharsets.UTF_8),
true)
My interest is first and foremost to understand buffering better, and practical only after that.
Thak you all in advance.
EDIT: Found this other stack overflow question interesting and related.
Also this other one talks about BufferedOutputStream.

It may help you to understand the difference between a writer and a stream. An OutputStream is a binary sink. A Writer has a character encoding and understands about newlines. The OutputStreamWriter allows you to send character encoded data, and have it correctly translated to binary for consumption by an OutputStream. What you probably want is:
PrintWriter out = new PrintWriter(new BufferedWriter(new OutputStreamWriter(resolveSocket.getOutputStream())));
This will allow you to use your PrintWriter to output chars, have it buffered by the BufferedWriter, and then translated by the OutputStreamWriter for consumption by the socket output stream.

I'm using a program to analyse my Java code and I don't understand the error or how to fix it

I'm using the FindBug program from Maryland University and it gives me this error.
I've tested my code on numerous platforms and it works, so why is this code bad-practice, and what can I do to improve it?

It's telling you the encoding (how the string is turned into bytes) isn't specified.
If you write a text file in Turkey, and load it up in Uzbekistan then you might get different results. Instead (for example) you could specify the encoding directly by converting the string to bytes yourself using a specified encoding (see String.getBytes for an example).

you need to specify the charset
you can use anOutputStreamWriter
fileWriter = new OutputStreamWriter(new FileOutputStream(file),charset);

See the FileWriter documentation: "The constructors of this class assume that the default character encoding and the default byte-buffer size are acceptable. To specify these values yourself, construct an OutputStreamWriter on a FileOutputStream."
It can be considered bad practice to depend on default character encoding.

Use FileOutputStream, instead of FileWriter. Which can be wrapped using the OutputStreamWriter, which allows you to pass an encoding in the constructor.
Or else, as said by Jeff, the data won't load correctly.
Example
OutputStream fout = new FileOutputStream("test.txt");
OutputStream bout = new BufferedOutputStream(fout);
OutputStreamWriter out = new OutputStreamWriter(bout, "UTF-8");

BufferedWriter is acting strange

I am trying to make a game with a working highscore mechanism and I am using java.io.BufferedWriter to write to a highscore file. I don't have an encryption on the highscore and I am using Slick2D and LWJGL for rendering and user input. The program executes this code:
FileWriter fstream = new FileWriter("res/gabjaphou.txt");
BufferedWriter writer = new BufferedWriter(fstream);
writer.write(score); // score is an int value
writer.close(); // gotta save m'resources! lol
I open the text file generated by this and all it reads is a question mark. I don't know why this happens, and I used other code from another project I was making and I had no problem with that... Does anyone know why? This is really annoying! :C

BufferedWriter.write(int) is meant to write a single charecter, not a integer.
public void write(int c)
throws IOException
Writes a single character.
Overrides: write in class Writer
Parameters: c - int specifying a character to be written
Throws: IOException - If an I/O error occurs
Try
writer.write(String.valueOf(score));

Please use writer.write(String.valueOf(score)); otherwise it writes score as a character.
See the documentation:
Writes a single character. The character to be written is contained in the 16 low-order bits of the given integer value; the 16 high-order bits are ignored.
What you want to use is Writer.write(String); convert score to a String using String.valueOf or Integer.toString.
writer.write(String.valueOf(score));

BufferedWriter is attempting to write a series of bytes to the file, not numbers. A number is still a character.
Consider using FileWriter instead, and something as simple as:
fileWriter.write(Integer.toString(score)) Write takes a string here, but the output should be the same.

Adding characters to beginning and end of InputStream in Java

I have an InputStream which I need to add characters to the beginning and end of, and should end up with another variable of type InputStream. How could I easily do this?

You want a SequenceInputStream and a couple of ByteArrayInputStreams. You can use String.getBytes to make the bytes for the latter. SequenceInputStream is ancient, so it's a little clunky to use:
InputStream middle ;
String beginning = "Once upon a time ...\n";
String end = "\n... and they lived happily ever after.";
List<InputStream> streams = Arrays.asList(
new ByteArrayInputStream(beginning.getBytes()),
middle,
new ByteArrayInputStream(end.getBytes()));
InputStream story = new SequenceInputStream(Collections.enumeration(streams));
If you have a lot of characters to add, and don't want to convert them to bytes en masse, you could put them in a StringReader, then use a ReaderInputStream from Commons IO to read them as bytes. But you would need to add Commons IO to your project to do that. Exact code for that is left as an exercise for the reader.

1 Create a new OutputStream, backed by a byte array as Greg suggested..
2 Write the beginning characters to your new OutputStream.
3 Copy your existing InputStream to your new OutputStream.
4 Write the ending characters to your new OutputStream.
5 Close your new OutputStream, taking care to preserve the backing array.
6 Open the backing arrray as a new InputStream.
Let us know if you have a problem with any of these steps.