I hope I didn't just find a bug in Java! I am running JDK 7u11 (mostly because that is the sanctioned JVM allowed by my employer) and I am noticing a very odd issue.
Namely, I am chunking data into a LinkedHashSet and writing it to a file using the ObjectOutputStream daisy changed through the GZIpOutputStream (mentioning this just in case it matters).
Now, when I get to the other side of the program and readObject() I notice that the size always reads 68, which I is the first size. The underlying table can have many more or less than 68, but the .size() method always returns 68. More troubling, when I try to manually iterate the underlying Set, it also stops at 68.
while(...) {
oos.writeInt(p_rid);
oos.writeObject(wptSet);
wptSet.clear();
// wptSet = new LinkedHashSet<>(); // **This somehow causes the heapsize to increase dramatically, but it does solve the problem**
}
And when reading it
Set<Coordinate> coordinates = (Set<Coordinate>) ois.readObject();
the coordinates.size() always returns 68. Now, I could make a workaround by also .writeInt() the size, but I can only iterate through 68 members!
Notice the wptSet = new LinkedHashSet<>() line actually solves the issue. The main problem with that is it makes my heapsize skyrocket when looking at the program in JVisualVM.
Update:
I actually just found a viable workaround that fixes the memory leak of re-instantiating wptSet... System.gc() Calling that after each call to .clear() actually keeps the memory leak away.
Either way, I shouldn't have to do this and shipping out the LinkedHashSet should not exhibit this behavior.
Alright, I think I understand what you are asking.
Here is an example to reproduce...
import java.util.*;
import java.io.*;
class Example {
public static void main(String[] args) throws Exception {
Set<Object> theSet = new LinkedHashSet<>();
final int size = 3;
for(int i = 0; i < size; ++i) {
theSet.add(i);
}
ByteArrayOutputStream bytesOut = new ByteArrayOutputStream();
ObjectOutputStream objectsOut = new ObjectOutputStream(bytesOut);
for(int i = 0; i < size; ++i) {
objectsOut.writeObject(theSet);
theSet.remove(i); // mutate theSet for each write
}
ObjectInputStream objectsIn = new ObjectInputStream(
new ByteArrayInputStream(bytesOut.toByteArray()));
for(;;) {
try {
System.out.println(((Set<?>)objectsIn.readObject()).size());
} catch(EOFException e) {
break;
}
}
}
}
The output is
3
3
3
What is going on here is that ObjectOutputStream detects that you are writing the same object every time. Each time theSet is written, a "shared reference" to the object is written so that the same object is deserialized each time. This is explained in the documentation:
Multiple references to a single object are encoded using a reference sharing mechanism so that graphs of objects can be restored to the same shape as when the original was written.
In this case you should use writeUnshared(Object) which will bypass this mechanism, instead of writeObject(Object).
Related
I'm trying to use EasyMock to test that a method runs a specific number of times but I keep getting an IllegalStateException error and I don't understand why. I'm new to EasyMock and JUnit and not very familiar with how to use them so I'm not sure what I'm doing wrong.
My code is:
FileOutputStream mockWriter;
Numbers mockByte;
#Test
public void testNumbers() throws IOException{
mockWriter = createMock(FileOutputStream.class);
mockByte = new Numbers(mockWriter);
mockByte.initByte();
expect(mockByte.generate()).times(10000);
replay(mockWriter);
}
And these are the methods initByte and generate from my Numbers class:
public void initByte() throws IOException{
File outFile = new File("NumbersOutput.txt");
FileOutputStream f = new FileOutputStream(outFile);
for(int i = 0; i < 10000; i++){
int b = generate();
f.write(b);
}
f.flush();
f.close();
}
public int generate(){
return rand.nextInt(100001);
}
The error you're getting is because nothing's calling anything on your mock.
Contrary to your naming, mockByte doesn't refer to a mock at all, so using it in an expect call like this is not going to help you. You should be expecting calls on mockWriter if anything.
However, it's not clear why you're using a mock for a stream at all, nor what the OutputStream in the Numbers constructor is used for. Your initByte() method doesn't use any state within the object other than rand. Even when that's fixed, it would probably be simplest just to use a ByteArrayOutputStream... make your API talk in terms of OutputStream instead of FileOutputStream, and it'll be much easier to test.
I suspect you should:
Remove the construction of a new FileOutputStream from the initByte method, instead writing to the stream you accept in the Numbers constructor
If your constructor parameter type is FileOutputStream, change it to OutputStream to make it cleaner and easier to test
Create a ByteArrayOutputStream in your test - you don't need mocking at all. You can then get all the bytes that have been written, and check them for whatever you want.
Think carefully about what you expect f.write(b) to do. It's only going to write a single byte, so the top 24 bits of your random number are going to be ignored. At that point, why are you choosing a number in the range [0, 10000] anyway?
Well firstly i am using NetBeans IDE. There is a client that one thread of it receives messages from the server and puts them in a Vector and another thread handles them. MessageListener and MessageHandler are the ones. So the problem is that the first message that it receives it is working good, but for the next message when it calls the method byte[] getFirstMessage() it returns byte with values of 0`s.
In my oppinion the problem is either with the Vector addElement method that it adds the second message to the index 1 and not 0 although it deletes the contents of vector`s first element as soon as it passes that data to the MessageHandler or that i use local variables in some methods. P.S it should have been the message queue.
MessageListener.java
package org.rebirth;
import java.io.*;
import java.util.*;
public class MessageListener implements Runnable{
Vector v;
int size = 0;
Connections con;
byte[] buffer = new byte[100000];
boolean noErrors = true;
public MessageListener(Connections con){
v = new Vector(50,10);
this.con = con;
Thread thr = new Thread(this);
thr.start();
}
public void run(){
while(noErrors){
try{
listenForData();
Thread.sleep(1);
}catch(Exception exc){
exc.printStackTrace();
noErrors = false;
}
}
}
public void listenForData() throws IOException{
con.fill(buffer,(byte)0);
System.out.println("Trying to receive data");
// InputStream
con.in.read(buffer);
System.out.println("Data received id "+con.ReadInt3Bytes(buffer,1));
v.addElement(buffer);
size++;
if(v.isEmpty()){
System.out.println("empty");
}
}
public byte[] getFirstMessage(){
if(v.size()>0){
byte[] data = (byte[]) v.firstElement();
v.removeElementAt(0);
size--;
System.out.println("first byte element "+(int)data[0]);
return data;
}
return null;
}
}
Messagehandler.java
package org.rebirth;
import java.util.*;
import javax.microedition.lcdui.game.*;
public class MessageHandler implements Runnable{
Vector v;
MessageListener lst;
Connections con;
int vienas = 1;
public MessageHandler(Vector v,MessageListener lst){
this.v = v;
this.lst = lst;
this.con = lst.con;
Thread thr = new Thread(this);
thr.start();
}
public void run(){
while(true){
try{
if(!v.isEmpty()){
handleMessages();
}
Thread.sleep(10);
}catch(Exception exc){}
}
}
public void handleMessages(){
// vectordsfds
int id;
byte [] gotByte = lst.getFirstMessage();
id=con.ReadInt3Bytes(gotByte,1);
System.out.println("handler id: "+id);
// call a method to handle received message;
handleMessage(id,gotByte);
}
public void handleMessage(int id,byte[] gotByte){
switch(id){
case 62:
// GameServerList
con.serverNumber = (int)gotByte[4];
System.out.println("Servers "+con.serverNumber);
int nri = 6;
for(int i=0;i<con.serverNumber;i++){
nameLength = (int)gotByte[nri];
nri+=1;
con.serverName[i] = new String(gotByte, nri, nameLength);
nri+=nameLength;
int ipLength = (int)gotByte[nri];
nri+=1;
con.serverIp[i] = new String(gotByte, nri, ipLength);
nri+=ipLength;
con.online[i] = con.ReadInt3Bytes(gotByte,nri);
nri+=3;
con.maxOnline[i] = con.ReadInt3Bytes(gotByte,nri);
System.out.println("Server name " +con.serverName[i]);
System.out.println("ip "+con.serverIp[i]);
System.out.println("online "+con.online[i]);
System.out.println("max online "+con.maxOnline[i]);
nri+=4;
}
break;
case 64:
//GameVersion
int success = (int)gotByte[4];
if(success == 1){
con.version=true;
System.out.println("Version match!");
}else{
System.out.println("version does not match");
System.out.println(success);
}
break;
}
}
}
EDIT 2: I added a statement InputStream class available() method before reading the data.
I am surprised it works even sometimes. If I am reading the sources right, you are using the same buffer instance for all incoming messages. Vector.addElement does not create a copy of the buffer, it just saves the reference passed as param, so listenForData running in while(noErrors) loop will clear the (one and only) buffer with zeros as soon as it runs again. And when it runs again depedsn on how threads are scheduled, so sometimes the message listener gets the buffer with real data and sometimes with zeros.
Also I'm not sure that CLDC Vector is, unlike in J2SE, synchronized. So I think vector access should be synchronized.
And also I'm not sure what type of connections you are using, but reading bytes from stream without checking return value and without knowing how many bytes you should read seems like a bit unreliable piece of code...
EDIT: by "knowing how many bytes you should read" I did not mean using available() function, which is likely to return 0 for many types of streams. When read() returns, it may (and it often does) read less bytes than requested, so you can never be sure if you have enough data in buffer to work with, unless you known how much data you are waiting and you keep reading until you have them. I think you should define a protocol, it can be as simple as fixed-size messages if it fits you needs.
It is not so trivial to understand what happens with your code exactly. However method listenForData() indeed adds byte array filled by zeros to your vector:
con.fill(buffer,(byte)0);
......
v.addElement(buffer);
So, although you are new here and I would like to welcome you and to wish you a lot of success I would like to ask you some questions.
do you know that J2ME is obsolete? Take a look on Android OS if you want to program form mobile devices.
Do you really think that JDK classes (like Vector) contain bugs that you can discover by writing your first program?
Did you hear about debuggers? Try to debug your code to understand why it does not work.
If you want to ask question and get a good answer take a look on SSCCE.
Good luck.
i have a question on lines 26 & 27:
String dumb = input.nextLine();
output.println(dumb.replaceAll(REMOVE, ADD));
i was hoping that i'd be able to shrink this down to a single line and be able to save space, so i did:
output.println(new String(input.nextLine()).replaceAll(REMOVE, ADD));
but now i'm wondering about performance. i understand that this program is quiet basic and doesn't need optimization, but i'd like to learn this.
the way i look at it, in the first scenario i'm creating a string object dumb, but once i leave the loop the object is abandoned and the JVM should clean it up, right? but does the JVM clean up the abandoned object faster than the program goes through the loop? or will there be several string objects waiting for garbage collection once the program is done?
and is my logic correct that in the second scenario the String object is created on the fly and destroyed once the program has passed through that line? and is this in fact a performance gain?
i'd appreciate it if you could clear this up for me.
thank you,
p.s. in case you are wondering about the program (i assumed it was straight forward) it takes in an input file, and output file, and two words, the program takes the input file, replaces the first word with the second and writes it into the second file. if you've actually read this far and would like to suggest ways i could make my code better, PLEASE DO SO. i'd be very grateful.
import java.io.File;
import java.util.Scanner;
import java.io.PrintWriter;
public class RW {
public static void main(String[] args) throws Exception{
String INPUT_FILE = args[0];
String OUTPUT_FILE = args[1];
String REMOVE = args[2];
String ADD = args[3];
File ifile = new File(INPUT_FILE);
File ofile = new File(OUTPUT_FILE);
if (ifile.exists() == false) {
System.out.println("the input file does not exists in the current folder");
System.out.println("please provide the input file");
System.exit(0);
}
Scanner input = new Scanner(ifile);
PrintWriter output = new PrintWriter(ofile);
while(input.hasNextLine()) {
String dumb = input.nextLine();
output.println(dumb.replaceAll(REMOVE, ADD));
}
input.close();
output.close();
}
}
The very, very first thing I'm going to say is this:
Don't worry about optimizing performance prematurely. The Java compiler is smart, it'll optimize a lot of this stuff for you, and even if it didn't you're optimizing out incredibly tiny amounts of time. The stream IO you've got going there is already running for orders of magnitude longer than the amount of time you're talking about.
What is most important is how easy the code is to understand. You've got a nice code style, going from your example, so keep that up. Which of the two code snippets is easier for someone other than you to read? That is the best option. :)
That said, here are some more specific answers to your questions:
Garbage collection will absolutely pick up objects which are instantiated inside the scope of a loop. The fact that it's instantiated inside the loop means that Java will already have marked it for clean up as soon as it fell out of scope. The next time GC runs, it will clean up all of those things which have been marked for clean up.
Creating an object inline will still create an object. The constructor is still called, memory is still allocated... Under the hood, they are really, really similar. It's just that in one case that object has a name, and in the other it doesn't. You're not going to save any real resources by combining two lines of code into one.
"input.nextLine()" already returns a String, so you don't need to wrap it in a new String(). (So yes, removing that actually will result in one less object being instantiated!)
Local Objects are eligible for GC once they go out of scope. That does not mean that GC cleans them that very moment. The eligible objects undergone a lifecycle. GC may or may not collect them immediately.
As far your program is concerned, there is nothing much to optimize except a line or two. Below is a restructured program.
import java.io.File;
import java.util.Scanner;
import java.io.PrintWriter;
public class Test {
public static void main(String[] args) throws Exception {
String INPUT_FILE = args[0];
String OUTPUT_FILE = args[1];
String REMOVE = args[2];
String ADD = args[3];
File ifile = new File(INPUT_FILE);
File ofile = new File(OUTPUT_FILE);
if (ifile.exists() == false) {
System.out.println("the input file does not exists in the current folder\nplease provide the input file");
System.exit(0);
}
Scanner input = null;
PrintWriter output = null;
try {
input = new Scanner(ifile);
output = new PrintWriter(ofile);
while (input.hasNextLine()) {
output.println(input.nextLine().replaceAll(REMOVE, ADD));
}
} finally {
if (input != null)
input.close();
if(output != null)
output.close();
}
}
}
If you arew concerned about obejct creation and performance, use a profiler to mesure your code. And keep in mind that doing new String(input.nextLine()) is totally pointless since input.nextLine() returns an immutable instance of String. So just do output.println(input.nextLine().replaceAll(REMOVE, ADD));.
Say, for example, I have a complex dynamically allocated structure (such as a binary tree) that needs to be written to a file made up of different sections. I would like to first write the size of the structure as a dword followed by the structure itself, however the size of the structure is only known after I have written the structure to the file. It is difficult, in this case, to pre-determine the size of the structure in memory.
Is it best to write the size as 0, then write the structure, then seek back and overwrite the size with the correct value? I don't like that idea, though. Is there a better/proper way to do it?
Just an idea: write the data to a ByteArrayOutputStream, after that, you should be able to call size() to get the actual length in bytes and call toByteArray() to get the byte buffer, that can be written to a file.
Code example
public static void main (String[] args) throws java.lang.Exception {
ArrayList objects = new ArrayList();
objects.add("Hello World");
objects.add(new Double(42.0));
System.out.println(sizeof(objects));
}
public static int sizeof(Serializable object) {
ObjectOutputStream out = null;
ByteArrayOutputStream baos = null;
try {
baos = new ByteArrayOutputStream();
out = new ObjectOutputStream(baos);
out.writeObject(object);
} catch (Exception e) {
e.printStackTrace();
} finally {
if (out != null) {
try {
out.close();
} catch (Exception e) {
e.printStackTrace();
}
}
}
return baos != null? baos.size() : -1;
}
This just demonstrate a sizeof emulator (which is different from the c implementation, because it calculates the size of a serialized object - the implementation for raw bytes would be slightly different).
Did you looked at Random Access Files yet?
Why do you need to write the size at all? Won't the file be the size of the structure after you have written it?
If you have variable components like arrays or lists, you can write the sizes of those as you write the data. However the total length is redundant and not very useful.
If you really have to, you can write the data to a ByteArrayOutputStream first to get the length. (But I seriously doubt it)
Please refer the below url http://www.javapractices.com/topic/TopicAction.do?Id=83 for calculating size of object .This utility seems worthful for your need.
To measure the size of a particular object containing data, measure JVM memory use before and after building the object.
Does Java have a using statement that can be used when opening a session in hibernate?
In C# it is something like:
using (var session = new Session())
{
}
So the object goes out of scope and closes automatically.
Java 7 introduced Automatic Resource Block Management which brings this feature to the Java platform. Prior versions of Java didn't have anything resembling using.
As an example, you can use any variable implementing java.lang.AutoCloseable in the following way:
try(ClassImplementingAutoCloseable obj = new ClassImplementingAutoCloseable())
{
...
}
Java's java.io.Closeable interface, implemented by streams, automagically extends AutoCloseable, so you can already use streams in a try block the same way you would use them in a C# using block. This is equivalent to C#'s using.
As of version 5.0, Hibernate Sessions implement AutoCloseable and can be auto-closed in ARM blocks. In previous versions of Hibernate Session did not implement AutoCloseable. So you'll need to be on Hibernate >= 5.0 in order to use this feature.
Before Java 7, there was no such feature in Java (for Java 7 and up see Asaph's answer regarding ARM).
You needed to do it manually and it was a pain:
AwesomeClass hooray = null;
try {
hooray = new AwesomeClass();
// Great code
} finally {
if (hooray!=null) {
hooray.close();
}
}
And that's just the code when neither // Great code nor hooray.close() can throw any exceptions.
If you really only want to limit the scope of a variable, then a simple code block does the job:
{
AwesomeClass hooray = new AwesomeClass();
// Great code
}
But that's probably not what you meant.
Since Java 7 it does: http://blogs.oracle.com/darcy/entry/project_coin_updated_arm_spec
The syntax for the code in the question would be:
try (Session session = new Session())
{
// do stuff
}
Note that Session needs to implement AutoClosable or one of its (many) sub-interfaces.
Technically:
DisposableObject d = null;
try {
d = new DisposableObject();
}
finally {
if (d != null) {
d.Dispose();
}
}
The closest java equivalent is
AwesomeClass hooray = new AwesomeClass();
try{
// Great code
} finally {
hooray.dispose(); // or .close(), etc.
}
As of now, no.
However there is a proposal of ARM for Java 7.
If you're interested in resource management, Project Lombok offers the #Cleanup annotation. Taken directly from their site:
You can use #Cleanup to ensure a given
resource is automatically cleaned up
before the code execution path exits
your current scope. You do this by
annotating any local variable
declaration with the #Cleanup
annotation like so:
#Cleanup InputStream in = new FileInputStream("some/file");
As a
result, at the end of the scope you're
in, in.close() is called. This call is
guaranteed to run by way of a
try/finally construct. Look at the
example below to see how this works.
If the type of object you'd like to
cleanup does not have a close()
method, but some other no-argument
method, you can specify the name of
this method like so:
#Cleanup("dispose") org.eclipse.swt.widgets.CoolBar bar = new CoolBar(parent, 0);
By default, the cleanup method is presumed to be
close(). A cleanup method that takes
argument cannot be called via
#Cleanup.
Vanilla Java
import java.io.*;
public class CleanupExample {
public static void main(String[] args) throws IOException {
InputStream in = new FileInputStream(args[0]);
try {
OutputStream out = new FileOutputStream(args[1]);
try {
byte[] b = new byte[10000];
while (true) {
int r = in.read(b);
if (r == -1) break;
out.write(b, 0, r);
}
} finally {
out.close();
}
} finally {
in.close();
}
}
}
With Lombok
import lombok.Cleanup;
import java.io.*;
public class CleanupExample {
public static void main(String[] args) throws IOException {
#Cleanup InputStream in = new FileInputStream(args[0]);
#Cleanup OutputStream out = new FileOutputStream(args[1]);
byte[] b = new byte[10000];
while (true) {
int r = in.read(b);
if (r == -1) break;
out.write(b, 0, r);
}
}
}
No, Java has no using statement equivalent.
In java 8 you can use try. Please refer to following page. http://docs.oracle.com/javase/tutorial/essential/exceptions/tryResourceClose.html
Please see this List of Java Keywords.
The using keyword is unfortunately not part of the list.
And there is also no equivalence of the C# using keyword through any other keyword as for now in Java.
To imitate such "using" behaviour, you will have to use a try...catch...finally block, where you would dispose of the resources within finally.
ARM blocks, from project coin will be in Java 7. This is feature is intended to bring similar functionality to Java as the .Net using syntax.
To answer the question regarding limiting scope of a variable, instead of talking about automatically closing/disposing variables.
In Java you can define closed, anonymous scopes using curly brackets. It's extremely simple.
{
AwesomeClass hooray = new AwesomeClass()
// Great code
}
The variable hooray is only available in this scope, and not outside it.
This can be useful if you have repeating variables which are only temporary.
For example, each with index. Just like the item variable is closed over the for loop (i.e., is only available inside it), the index variable is closed over the anonymous scope.
// first loop
{
Integer index = -1;
for (Object item : things) {index += 1;
// ... item, index
}
}
// second loop
{
Integer index = -1;
for (Object item : stuff) {index += 1;
// ... item, index
}
}
I also use this sometimes if you don't have a for loop to provide variable scope, but you want to use generic variable names.
{
User user = new User();
user.setId(0);
user.setName("Andy Green");
user.setEmail("andygreen#gmail.com");
users.add(user);
}
{
User user = new User();
user.setId(1);
user.setName("Rachel Blue");
user.setEmail("rachelblue#gmail.com");
users.add(user);
}