We Keep Coding

Java StackOverflowError at java.io.PrintStream.write(PrintStream.java:480) and no further stack trace

I am running a Java program written by another person on more data than the program was initially designed for, e.g. 10-times longer input files, roughly quadratic runtime. I encountered different problems and now aim to solve them bit by bit.
During execution when a lot of output has already been printed (redirected to a file) I get following output:
Exception in thread "main" java.lang.StackOverflowError
at java.io.PrintStream.write(PrintStream.java:480)
[...]
at java.io.PrintStream.write(PrintStream.java:480)
The stack trace is the first thing that confuses me as it is a long repetition of just the same line again and again. Furthermore, it gives no intention where in the code or the execution the problem occurred.
My thoughts / research
StackOverflowError
may indicate too less memory. With -Xmx110G flag I provided 110 G memory and monitored it while execution, only up to ~32 G were used. So this is probably not the problem here.
may be thrown due to programming mistakes causing an infinite loop. However, I cant really check this since I am not familiar enough with the code and the stack trace does not help me finding the location of the problem in the code.
[hypothesis] may be caused, because the writing of output is slower than execution and new print/write calls. Still, why is there no further stack trace? How could I check and fix this?
PrintStream
only code fragments after searching for "PrintStream"
// reset output stream to suppress the annoying output of the Apache batik library. Gets reset after lib call.
OutputStream tmp=System.out;
System.setOut(new PrintStream(new org.apache.commons.io.output.NullOutputStream()));
drawRes.g2d.stream(new FileWriter(svgFilePath), false);
System.setOut(new PrintStream(tmp));
[hypothesis] the write to void / null does not work
[workaround] if skip the change of output stream and just "live" with the big output created the programm seems to run (into other problems, but this is another case). Any ideas why this is happening?
Call for advice
If you have any advice on what is going one, what the Java code specifically does, please help me understand it. Especially the stack trace frustrates me as it provides no place to begin the fixing. I am also thankful for a general approach on how to tackle this problem, get a stack trace, fix the code to avoid StackOverflow, etc.
Some system environment facts
Linux machine
128 G memory
Java
openjdk version "1.8.0_121"
OpenJDK Runtime Environment (IcedTea 3.3.0) (suse-28.1-x86_64)
OpenJDK 64-Bit Server VM (build 25.121-b13, mixed mode)
Please ask if you need more information!
Notes
I am rather new to Java so I am grateful for every advice (the program is not written by me)
This is my first post on stackoverflow, please inform me where I can improve my style of asking and formatting
I am no native English speaker, so please excuse mistakes and feel free to ask for understanding or correct me
Thanks all for your replies!

These two lines look suspicious:
OutputStream tmp=System.out;
//...
System.setOut(new PrintStream(tmp));
System.out is already a PrintStream, so IMHO the lines should read
PrintStream tmp=System.out;
//...
System.setOut(tmp);
What happens otherwise is that you have a near endless wrapping of PrintStreams within PrintStreams. The nesting of PrintStreams is only limited through Java heap space - but the call level nesting is much lower.
To verify the hypothesis I created a small test program that first wraps System.out 20 times and prints a stacktrace to verify the call chain. After that it wraps System.out 10_000 times and produces a StackOverflowException.
import java.io.OutputStream;
import java.io.PrintStream;
public class CheckPrintStream {
public static void main(String[] args) {
PrintStream originalSystemOut = System.out;
System.setOut(new PrintStream(System.out) {
#Override
public void write(byte buf[], int off, int len) {
originalSystemOut.write(buf, off, len);
if (len > 2) {
new RuntimeException("Testing PrintStream nesting").printStackTrace(originalSystemOut);
}
}
});
for (int i = 0; i < 20; i++) {
wrapSystemOut();
}
System.out.println("Hello World!");
for (int i = 20; i < 10_000; i++) {
wrapSystemOut();
}
System.out.println("crash!");
}
private static void wrapSystemOut() {
OutputStream tmp = System.out;
System.setOut(new PrintStream(System.out));
}
}
A nesting of around 6000 to 7000 PrintWriters is sufficient to produce a stack overflow.

JAVA : Performance and Memory improvement code comparison from codechef

So today I solved a very simple problem from Codechef and I solved the problem using JAVA my answer was accepted. My code was.
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
class INTEST {
public static void main(String args[]) throws IOException {
BufferedReader reader = new BufferedReader(new InputStreamReader(System.in));
String input = reader.readLine();
int n = Integer.parseInt(input.split(" ")[0]);
long k = Long.parseLong(input.split(" ")[1]);
int count = 0;
String element;
for (int i = 0; i < n; i++) {
element = reader.readLine();
if (Long.parseLong(element) % k == 0) {
count++;
}
}
System.out.println(count);
}
}
The onine judge reported
Running Time : 0.58 Second
Memory : 1340.5M
So, I looked into some other solutions for the same problem (sorted the solution by time) and I got another solution by the user indontop.
public class Main{
public static void main(String ...args)throws Exception{
byte b;
byte barr[]=new byte[1028];
int r=0,n=0,k=0;
while((r=System.in.read())!= ' '){
n=n*10+r-'0';
}
//System.out.println(n);
while((r=System.in.read())!='\n'){ //change
k=k*10+r-'0';
}
//System.out.println(k);
//System.in.read(); // remove
n=0;
int count=0;
while((r=System.in.read(barr,0,1028))!=-1){
for(int i=0;i<barr.length;i++){
b=barr[i];
if(b!='\n'){ //change
n=n*10+b-'0';
}
else{
// i++; //remove
if(n%k==0)count++;
n=0;
}
}
}
System.out.println(count);
}
}
the execution time and memory for the above code.
Running Time : 0.13 Second
Memory : OM
I wonder how was the user able to achieve this much performance and Memory gain with this very simple problem.
I dont understand the logic behind this code, can anyone help me by explaining this code, and also please explain what is wrong with my code.
Thank You.

How indontop achieved a better memory footprint
Basically, indontop's program reads bytes directly from the input stream, without going through readers or reading lines. The only structure it allocates is a single array of 1028 bytes, and no other objects are created directly.
Your program, on the other hand, reads lines from a BufferedReader. Each such line is allocated in memory as a string. But your program is rather short, so it's highly likely that the garbage collector doesn't kick in, hence all those lines that were read are not cleared away from memory.
What indontop's program does
It reads the input byte by byte and parses the numbers directly from it, without using Integer.parseInt or similar methods. The characters '0' through '9' can be converted to their respective values (0-9) by subtracting '0' from each of them. The numbers themselves are parsed by noting that a number like '123' can be parsed as 1*10*10 + 2*10 + 3.
The bottom line is that the user is implementing the very basic algorithm for interpreting numbers without ever having the full string in memory.
Is indontop's program better than yours?
My answer to this is no. First, his program is not entirely correct: he is reading an array of bytes and is not checking how many bytes were actually read. The last array read can contain bytes from the previous read, which may give wrong output, and it is by sheer luck that this didn't happen when he ran it.
Now, the rest of this is opinion-based:
Your program is much more readable than his. You have meaningful variable names, he doesn't. You are using well-known methods, he doesn't. Your code is concise, his is verbose and repeats the same code many times.
He is reinventing the wheel - there are good number parsing methods in Java, no need to rewrite them.
Reading data byte-by-byte is inefficient as far as system calls are concerned, and improves efficiency only in artificial environments like CodeChef and like sites.
Runtime efficiency
You really can't tell by looking at one run. Those programs run under a shared server that does lots of other things and there are too many factors that affect performance. Benchmarking is a complicated issue. The numbers you see? Just ignore them.
Premature Optimization
In real world programs, memory is garbage collected when it's needed. Memory efficiency should be improved only if it's something very obvious (don't allocate an array of 1000000 bytes if you only intend to use 1000 of them), or when the program, when running under real conditions, has memory issues.
This is true for the time efficiency as well, but as I said, it's not even clear if his program is more runtime efficient than yours.
Is your program good?
Well, not perfect, you are running the split twice, and it would be better to just do it once and store the result in a two-element array. But other than that, it's a good answer to this question.

String split method java

I'm having a bit of difficulty getting my code to work. One of my assignments requires me to use this data from an external file (basically a passage/poem):
Good morning life and all
Things glad and beautiful
My pockets nothing hold
But he that owns the gold
The sun is my great friend
His spending has no end
Hail to the morning sky
Which bright clouds measure high
Hail to you birds whose throats
Would number leaves by notes
Hail to you shady bowers
And you green fields of flowers
Hail to you women fair
That make a show so rare
In cloth as white as milk
Be it calico or silk
Good morning life and all
Things glad and beautiful
We are trying to find the total number of words, the number of words that have only three letters, and the percentage of occurrence of the three words. I think I can handle the assignment, but something went wrong in my code while I was working it out:
import java.io.*;
import java.util.*;
public class Prog739h
{
public static void main(String[] args) throws IOException
{
Scanner kbReader = new Scanner(new File("C:\\Users\\Guest\\Documents\\Java programs\\Prog739h\\Prog739h.in"));
int totalWords = 0;
while(kbReader.hasNextLine())
{
String data = kbReader.nextLine();
String[] words = data.split(" ");
totalWords+=words.length();
System.out.println(totalWords);
}
}
}
When I tried to compile to test the code at the moment to see if everything I had done was working properly, I was given an error that said it can't find symbol method length(). I checked my line with the "totalWords+=words.length()", but I don't know what I can do to fix the problem. Could someone please explain to me why this happened and provide some direction on how to fix this error? Thanks!

The answer is that the length of an array is given by the length field, not the length method. In other words, change
totalWords+=words.length();
to
totalWords+=words.length;

length is a public field on an Array object, the code is attempting to invoke it as a method using ()
Remove the () after length:
totalWords+=words.length

length is a property of array, access it without ()

please change:
totalWords+=words.length();
to
totalWords+=words.length;

Array properties shouldn't contain parenthesis
totalWords += words.length;
^

Reading large files for a simulation (Java crashes with out of heap space)

For a school assignment, I need to create a Simulation for memory accesses. First I need to read 1 or more trace files. Each contains memory addresses for each access. Example:
0 F001CBAD
2 EEECA89F
0 EBC17910
...
Where the first integer indicates a read/write etc. then the hex memory address follows. With this data, I am supposed to run a simulation. So the idea I had was parse these data into an ArrayList<Trace> (for now I am using Java) with trace being a simple class containing the memory address and the access type (just a String and an integer). After which I plan to loop through these array lists to process them.
The problem is even at parsing, it running out of heap space. Each trace file is ~200MB. I have up to 8. Meaning minimum of ~1.6 GB of data I am trying to "cache"? What baffles me is I am only parsing 1 file and java is using 2GB according to my task manager ...
What is a better way of doing this?
A code snippet can be found at Code Review

The answer I gave on codereview is the same one you should use here .....
But, because duplication appears to be OK, I'll duplicate the answer here.
The issue is almost certainly in the structure of your Trace class, and it's memory efficiency. You should ensure that the instrType and hexAddress are stored as memory efficient structures. The instrType appears to be an int, which is good, but just make sure that it is declared as an int in the Trace class.
The more likely problem is the size of the hexAddress String. You may not realise it but Strings are notorious for 'leaking' memory. In this case, you have a line and you think you are just getting the hexString from it... but in reality, the hexString contains the entire line.... yeah, really. For example, look at the following code:
public class SToken {
public static void main(String[] args) {
StringTokenizer tokenizer = new StringTokenizer("99 bottles of beer");
int instrType = Integer.parseInt(tokenizer.nextToken());
String hexAddr = tokenizer.nextToken();
System.out.println(instrType + hexAddr);
}
}
Now, set a break-point in (I use eclipse) your IDE, and then run it, and you will see that hexAddr contains a char[] array for the entire line, and it has an offset of 3 and a count of 7.
Because of the way that String substring and other constructs work, they can consume huge amounts of memory for short strings... (in theory that memory is shared with other strings though). As a consequence, you are essentially storing the entire file in memory!!!!
At a minimum, you should change your code to:
hexAddr = new String(tokenizer.nextToken().toCharArray());
But even better would be:
long hexAddr = parseHexAddress(tokenizer.nextToken());

Like rolfl I answered your question in the code review. The biggest issue, to me, is the reading everything into memory first and then processing. You need to read a fixed amount, process that, and repeat until finished.

Try use class java.nio.ByteBuffer instead of java.util.ArrayList<Trace>. It should also reduce the memory usage.
class TraceList {
private ByteBuffer buffer;
public TraceList(){
//allocate byte buffer
}
public void put(byte operationType, int addres) {
//put data to byte buffer
}
public Trace get(int index) {
//get data from byte buffer by index
byte type = ...//read type
int addres = ...//read addres
return new Trace(type, addres)
}
}

Java 7 escape analysis is not working

For the below given code , I see lot of GC activity. As per my understanding this is a suitable scenario for EA. Why EA is not effective. DummyObject has nothing allocated inside it. JVM options used : -server , -verbosegc.
static void anayzeEA()
{
for(int i = 0 ; i < 100000000; i++) {
DummyObject obj = new DummyObject();
if(obj.hashCode() == 97787) { //to prevent the obj being optimized
System.out.println(obj.hashCode());
}
}
}

See related Q&A here which suggests you can download a debug JDK and use command line options:
-XX:+UnlockDiagnosticVMOptions -XX:+PrintEscapeAnalysis -XX:+PrintEliminateAllocations
to print out the escape analysis events as they happen.

Some observations
It seems like the obj.hashCode() is a native call and the object may escape . Changing obj.hashCode() to obj.getMyCode() (a method which returns the System.currentTimeMillis()% staticObjCount) made it work . No GC activity observed. However following method never got escape analysis in effect with all the suggestions mentioned
here
public static long test1()
{
long r = 0;
byte[] arr = new byte[(int)System.currentTimeMillis() % 1024];
if(arr.length == 998 ) {
++r;
}
return r;
}
JVM options used
-server
-verbosegc
-XX:CompileThreshold=1
Test1 is called over a number of times . Same old story. Java allocates memory in heap, GC comes and makes everything slow. was too excited about this feature.

Java API says:
As much as is reasonably practical, the hashCode method defined by class Object does return distinct integers for distinct objects. (This is typically implemented by converting the internal address of the object into an integer, but this implementation technique is not required by the JavaTM programming language.)
So you are generating objects, producing non-predictable hashCodes for each and compare them to some value. Additional it's a native method, so the JIT doesn't know what's happening inside.
Escape Analysis may be good, but there's currently no support for crystal balls. ;-)
Try overriding it with your own method, returning 12345.