We Keep Coding

WriteObject not properly writing a Set?

I hope I didn't just find a bug in Java! I am running JDK 7u11 (mostly because that is the sanctioned JVM allowed by my employer) and I am noticing a very odd issue.
Namely, I am chunking data into a LinkedHashSet and writing it to a file using the ObjectOutputStream daisy changed through the GZIpOutputStream (mentioning this just in case it matters).
Now, when I get to the other side of the program and readObject() I notice that the size always reads 68, which I is the first size. The underlying table can have many more or less than 68, but the .size() method always returns 68. More troubling, when I try to manually iterate the underlying Set, it also stops at 68.
while(...) {
oos.writeInt(p_rid);
oos.writeObject(wptSet);
wptSet.clear();
// wptSet = new LinkedHashSet<>(); // **This somehow causes the heapsize to increase dramatically, but it does solve the problem**
}
And when reading it
Set<Coordinate> coordinates = (Set<Coordinate>) ois.readObject();
the coordinates.size() always returns 68. Now, I could make a workaround by also .writeInt() the size, but I can only iterate through 68 members!
Notice the wptSet = new LinkedHashSet<>() line actually solves the issue. The main problem with that is it makes my heapsize skyrocket when looking at the program in JVisualVM.
Update:
I actually just found a viable workaround that fixes the memory leak of re-instantiating wptSet... System.gc() Calling that after each call to .clear() actually keeps the memory leak away.
Either way, I shouldn't have to do this and shipping out the LinkedHashSet should not exhibit this behavior.

Alright, I think I understand what you are asking.
Here is an example to reproduce...
import java.util.*;
import java.io.*;
class Example {
public static void main(String[] args) throws Exception {
Set<Object> theSet = new LinkedHashSet<>();
final int size = 3;
for(int i = 0; i < size; ++i) {
theSet.add(i);
}
ByteArrayOutputStream bytesOut = new ByteArrayOutputStream();
ObjectOutputStream objectsOut = new ObjectOutputStream(bytesOut);
for(int i = 0; i < size; ++i) {
objectsOut.writeObject(theSet);
theSet.remove(i); // mutate theSet for each write
}
ObjectInputStream objectsIn = new ObjectInputStream(
new ByteArrayInputStream(bytesOut.toByteArray()));
for(;;) {
try {
System.out.println(((Set<?>)objectsIn.readObject()).size());
} catch(EOFException e) {
break;
}
}
}
}
The output is
3
3
3
What is going on here is that ObjectOutputStream detects that you are writing the same object every time. Each time theSet is written, a "shared reference" to the object is written so that the same object is deserialized each time. This is explained in the documentation:
Multiple references to a single object are encoded using a reference sharing mechanism so that graphs of objects can be restored to the same shape as when the original was written.
In this case you should use writeUnshared(Object) which will bypass this mechanism, instead of writeObject(Object).

Picking up from where I left off when reading a file in Java

I am trying to read info from a file and create objects out of that information. Every 6 or so lines of the file is a different unit, meaning that the first set of lines are relevant to object A, the next set to object B, and so on.
I can read from the file and create my object just fine--for the first set. My problem is that I don't know how to get the reader to pick up from the spot it left off at when creating the next object...
(Note: the read() method which creates the file is part of the new object being created, not in a main() or anything like that). Here are the relevant bits of code:
The driver:
public class CSD{
public static void main (String[] argv){
Vector V=new Vector(10);
CoS jon=new CoS();
jon.display();
}//end main
}
which calls CoS, whose constructor is:
public CoS(){
try{
String fileName=getFileName();
FileReader freader=new FileReader(fileName);
BufferedReader inputFile=new BufferedReader(freader);
this.read(inputFile);
setDegree(major);
setStatus(credits);
} catch(FileNotFoundException ex){
}//end catch
}
Which calls both read() and getFileName():
public void read(BufferedReader inputFile){
try{
int n;
super.read(inputFile);
String str=inputFile.readLine();
if (str!=null){
n=Integer.parseInt(str);
setCredits(n);
str=inputFile.readLine();
setMajor(str);
}//end if
}catch(IOException ex){}
}//end method
public String getFileName() {
Scanner scan = new Scanner(System.in);
String filename;
System.out.print("Enter the file name and path ==> ");
filename = scan.nextLine();
System.out.println("");
return filename;
}
Thanks in advance, guys!

Why not use ObjectInputStream and ObjectOutputStream? Or any kind of real serialization?
javadoc: http://docs.oracle.com/javase/6/docs/api/java/io/ObjectOutputStream.html
example code: http://www.javadb.com/writing-objects-to-file-with-objectoutputstream
Basically, since you write your objects to a file and want to take care of the lines where they are located, I'll suggest a few other serialization alternatives.
One is the Object * Stream - you create a ObjectStream on a File and just write objects thru it. Later when you read, you read the objects in the reverse order you wrote them and they will come back just as you wrote them.
Another is to implement Serializable. Remember that transient keyword? Use it on fields you do not want to save to the file.
And then there's the raw "by hand" approach where you save only the things you want to save and reconstruct the objects later by passing these initialization values to their constructor. Kinda like people suggested that you make the file line a argument to the ctor :)
EDIT:
guess writing with Object*Streams requires you to implement Serializable or Externalizable.
but if the example code isn't clear enough, ask :)

Encoding & decoding serialized XML in the class which the methods are in

I can't think of a way of describing it, basically I'm wondering if it is possible to write a method makes the current object become the decoded serialized XML.
I know that the saveTillXML works, as it generates an XML file which matches the state of the current Till object [but really I don't know that it does work, as the decoding fails]. However, I get these exceptions when I call the loadTillXML() method:
java.lang.reflect.InvocationTargetException
java.lang.IllegalStateException
java.lang.ArrayIndexOutOfBoundsException
I call the saveTillXML() method when the window is closed, & the loadTillXML method in the Till's constructor.
I have all the gets & sets & argumentless constructors in the Till, & in all the clases which the Till uses.
public void saveTillXML() throws IOException{
XMLEncoder encoder=new XMLEncoder(new BufferedOutputStream(new FileOutputStream("till.xml")));
encoder.writeObject(this);
encoder.close();
}
public static void loadTillXML() throws IOException{
XMLDecoder decoder=new XMLDecoder(new BufferedInputStream(new FileInputStream("till.xml")));
decoder.readObject();
decoder.close();
}
The only thing I can think of is the fact that decoder.readObject() doesn't go into anything, which is probably should if I want to make the current instance be the XML it's decoding.
Thanks in advance.

Okay, I've managed to solve my problem. The loadTillXML needed a return method - Till - which then prompted me to think that it needs to be called outside Till. So I used it instead of private Till till=new Till(); which was in the window.
The working method:
public static Till loadTillXML() {
try{
XMLDecoder decoder=new XMLDecoder(new BufferedInputStream(new FileInputStream("till.xml")));
return (Till) decoder.readObject();
}
catch(FileNotFoundException fNFE){
System.out.println("Cannot find file");
Till till=new Till();
return till;
}
}

How do you mock an output stream?

By 'output steam' i mean any object which receives a sequence of bytes, or characters or whatever. So, java.io.OutputStream, but also java.io.Writer, javax.xml.stream.XMLStreamWriter's writeCharacters method, and so on.
I'm writing mock-based tests for a class whose main function is to write a stream of data to one of these (the XMLStreamWriter, as it happens).
The problem is that the stream of data is written in a series of calls to the write method, but what matters is not the calls, but the data. For example, given an XMLStreamWriter out, these:
out.writeCharacters("Hello, ");
out.writeCharacters("world!");
Are equivalent to this:
out.writeCharacters("Hello, world!");
It really doesn't matter (for my purposes) which happens. There will be some particular sequence of calls, but i don't care what it is, so i don't want to write expectations for that particular sequence. I just want to expect a certain stream of data to be written any which way.
One option would be to switch to state-based testing. I could accumulate the data in a buffer, and make assertions about it. But because i'm writing XML, that would mean making some fairly complex and ugly assertions. Mocking seems a much better way of dealing with the larger problem of writing XML.
So how do i do this with a mock?
I'm using Moxie for mocking, but i'm interested in hearing about approaches with any mocking library.

A fairly elegant strategy to test output or input streams is to use PipedInputStream and PipedOutputStream classes. You can wire them together in the set up of the test, and then check what has been written after the target method is executed.
You can work the other direction preparing some input and then let the test read this prepared data from the input stream as well.
In your case, you could just mock that "out" variable with a PipedOutputStream, and plug a PipedInputStream to it this way:
private BufferedReader reader;
#Before
public void init() throws IOException {
PipedInputStream pipeInput = new PipedInputStream();
reader = new BufferedReader(
new InputStreamReader(pipeInput));
BufferedOutputStream out = new BufferedOutputStream(
new PipedOutputStream(pipeInput))));
//Here you will have to mock the output somehow inside your
//target object.
targetObject.setOutputStream (out);
}
#Test
public test() {
//Invoke the target method
targetObject.targetMethod();
//Check that the correct data has been written correctly in
//the output stream reading it from the plugged input stream
Assert.assertEquals("something you expects", reader.readLine());
}

I'll admit that I'm probably partial to using a ByteArrayOutputStream as the lowest level OutputStream, fetching the data after execution and peforming whatever assertions that are needed. (perhaps using SAX or other XML parser to read in the data and dive through the structure)
If you want to do this with a mock, I'll admit I'm somewhat partial to Mockito, and I think you could accomplish what you're looking to do with a custom Answer which when the user invokes writeCharacters on your mock, would simply append their argument to a Buffer, and then you can make assertions on it afterwards.
Here's what I have in my head (hand written, and haven't executed so syntax issues are to be expected :) )
public void myTest() {
final XMLStreamWriter mockWriter = Mockito.mock(XMLStreamWriter.class);
final StringBuffer buffer = new StringBuffer();
Mockito.when(mockWriter.writeCharacters(Matchers.anyString())).thenAnswer(
new Answer<Void>() {
Void answer(InvocationOnMock invocation) {
buffer.append((String)invocation.getArguments()[0]);
return null;
}
});
//... Inject the mock and do your test ...
Assert.assertEquals("Hello, world!",buffer.toString());
}

(Disclaimer: I'm the author of Moxie.)
I assume you want to do this using logic embedded in the mock so that calls that violate your expectation fail fast. Yes, this is possible - but not elegant/simple in any mocking library I know of. (In general mock libraries are good at testing the behavior of method calls in isolation/sequence, but poor at testing more complex interactions between calls over the lifecycle of the mock.) In this situation most people would build up a buffer as the other answers suggest - while it doesn't fail fast, the test code is simpler to implement/understand.
In the current version of Moxie, adding custom parameter-matching behavior on a mock means writing your own Hamcrest matcher. (JMock 2 and Mockito also let you use custom Hamcrest matchers; EasyMock lets you specify custom matchers that extend a similar IArgumentMatcher interface.)
You'll want a custom matcher that will verify that the string passed to writeCharacters forms the next part of the sequence of text you expect to be passed into that method over time, and which you can query at the end of the test to make sure it's received all of the expected input. An example test following this approach using Moxie is here:
http://code.google.com/p/moxiemocks/source/browse/trunk/src/test/java/moxietests/StackOverflow6392946Test.java
I've reproduced the code below:
import moxie.Mock;
import moxie.Moxie;
import moxie.MoxieOptions;
import moxie.MoxieRule;
import moxie.MoxieUnexpectedInvocationError;
import org.hamcrest.BaseMatcher;
import org.hamcrest.Description;
import org.junit.Assert;
import org.junit.Rule;
import org.junit.Test;
import javax.xml.stream.XMLStreamException;
import javax.xml.stream.XMLStreamWriter;
// Written in response to... http://stackoverflow.com/questions/6392946/
public class StackOverflow6392946Test {
private static class PiecewiseStringMatcher extends BaseMatcher<String> {
private final String toMatch;
private int pos = 0;
private PiecewiseStringMatcher(String toMatch) {
this.toMatch = toMatch;
}
public boolean matches(Object item) {
String itemAsString = (item == null) ? "" : item.toString();
if (!toMatch.substring(pos).startsWith(itemAsString)) {
return false;
}
pos += itemAsString.length();
return true;
}
public void describeTo(Description description) {
description.appendText("a series of strings which when concatenated form the string \"" + toMatch + '"');
}
public boolean hasMatchedEntirely() {
return pos == toMatch.length();
}
}
#Rule
public MoxieRule moxie = new MoxieRule();
#Mock
public XMLStreamWriter xmlStreamWriter;
// xmlStreamWriter gets invoked with strings which add up to "blah blah", so the test passes.
#Test
public void happyPathTest() throws XMLStreamException{
PiecewiseStringMatcher addsUpToBlahBlah = new PiecewiseStringMatcher("blah blah");
Moxie.expect(xmlStreamWriter).anyTimes().on().writeCharacters(Moxie.argThat(addsUpToBlahBlah));
xmlStreamWriter.writeCharacters("blah ");
xmlStreamWriter.writeCharacters("blah");
Assert.assertTrue(addsUpToBlahBlah.hasMatchedEntirely());
}
// xmlStreamWriter's parameters don't add up to "blah blah", so the test would fail without the catch clause.
// Also note that the final assert is false.
#Test
public void sadPathTest1() throws XMLStreamException{
// We've specified the deprecated IGNORE_BACKGROUND_FAILURES option as otherwise Moxie works very hard
// to ensure that unexpected invocations can't get silently swallowed (so this test will fail).
Moxie.reset(xmlStreamWriter, MoxieOptions.IGNORE_BACKGROUND_FAILURES);
PiecewiseStringMatcher addsUpToBlahBlah = new PiecewiseStringMatcher("blah blah");
Moxie.expect(xmlStreamWriter).anyTimes().on().writeCharacters(Moxie.argThat(addsUpToBlahBlah));
xmlStreamWriter.writeCharacters("blah ");
try {
xmlStreamWriter.writeCharacters("boink");
Assert.fail("above line should have thrown a MoxieUnexpectedInvocationError");
} catch (MoxieUnexpectedInvocationError e) {
// as expected
}
// In a normal test we'd assert true here.
// Here we assert false to verify that the behavior we're looking for has NOT occurred.
Assert.assertFalse(addsUpToBlahBlah.hasMatchedEntirely());
}
// xmlStreamWriter's parameters add up to "blah bl", so the mock itself doesn't fail.
// However the final assertion fails, as the matcher didn't see the entire string "blah blah".
#Test
public void sadPathTest2() throws XMLStreamException{
PiecewiseStringMatcher addsUpToBlahBlah = new PiecewiseStringMatcher("blah blah");
Moxie.expect(xmlStreamWriter).anyTimes().on().writeCharacters(Moxie.argThat(addsUpToBlahBlah));
xmlStreamWriter.writeCharacters("blah ");
xmlStreamWriter.writeCharacters("bl");
// In a normal test we'd assert true here.
// Here we assert false to verify that the behavior we're looking for has NOT occurred.
Assert.assertFalse(addsUpToBlahBlah.hasMatchedEntirely());
}
}

Java InputStream != Readable

I am using java.util.Scanner for things such as nextInt(), and all was working fine as long as I was using a java.lang.Readable (one and only constructor argument). However, when I changed to using an InputStream instead, Scanner.nextInt() never returns. Do you know why?
My implementation of the InputStream looks like this:
private static class ConsoleInputStream extends InputStream {
...
private byte[] buffer;
private int bufferIndex;
public int read() throws IOException {
...
while (...) {
if (buffer != null && bufferIndex < buffer.length) {
return buffer[bufferIndex++]; // THE COMMENT!
}
...
}
...
}
}
When I print the data by THE COMMENT I (correctly) get stuff like '1','2','\n' for "12\n", etc. Is there some Scanner hookup, unbeknown to me, that cause this behavior?

From the javadocs for InputStream's read() method:
"Returns: the next byte of data, or -1 if the end of the stream is reached."
I would guess that you're never returning -1?

I think the problem is with your self-built InputStream. Why did you build your own, rather than simply simply using System.in ?
Update:
Wanted input from a JTextField.
OK, got it. It usually doesn't make sense to use I/O handling to read stuff that's already available, in character form, but I can see how that would make your life easier with Scanner.
Still, you could probably have saved yourself some coding and grief by using a "finished" InputStream. What comes to mind is
InputStream is = new ByteArrayInputStream(myTextField.getText().getBytes());
Java I/O is yucky. Be glad the bright people from Sun have encapsulated most of it away for you.