This question already has answers here:
How to get 0-padded binary representation of an integer in java?
(17 answers)
Closed 5 years ago.
I'm currently working with a Huffman Tree to compress/decompress text files. Currently my problem is that when writing bytes and reading them, I lose any leading 0's in my numbers.
In my OutputStream class, my writeBit()method, I am fed one bit at a time and when my count of bits reaches 8, I write the byte to the file. Currently using a String to build this binary number, although the problem occurs when actually writing the bit.
HuffmanOutputStream.java:
/**
* Created by Sully on 3/20/2017.
*/
import java.io.IOException;
public class HuffmanOutputStream extends BitOutputStream {
private int count = 0;
private String bytes = "";
public HuffmanOutputStream(String filename, String tree, int totalChars) {
super(filename);
try {
d.writeUTF(tree);
d.writeInt(totalChars);
} catch (IOException e) {
}
}
public void writeBit(int bit) {
//PRE bit == 0 || bit == 1
if (count < 8) {
bytes += bit;
count++;
}
try {
if (count == 8) {
d.writeByte(Integer.parseInt(bytes, 2));
count = 0;
bytes = "";
}
} catch (IOException e) {
e.printStackTrace();
}
}
public void close() {
}
}
An example of when things go wrong, for my text file, the first byte that I construct is 01100001, although when I use Integer.parseInt(byte,2), the integer given is 97, which when it is then read as a binary number, only returns 1100001. As Huffman Trees rely on these 0's being included, how can I keep this 0 in place? Also to make sure that it is read correctly with the 0's remaining in place?
HuffmanInputStream.java:
/**
* Created by Sully on 3/20/2017.
*/
import java.io.IOException;
public class HuffmanInputStream extends BitInputStream {
private String tree;
private int totalChars;
private int currentByte;
private int bitCount;
private static final int BYTE_SIZE = 8;
private int[] bufferedBits = new int[BYTE_SIZE];
public HuffmanInputStream(String filename) {
super(filename);
try {
tree = d.readUTF();
totalChars = d.readInt();
currentByte = 0;
bitCount = 8;
} catch (IOException e) {
}
}
public int readBit() {
if (currentByte == -1) {
return -1;
}
if (bitCount == 8) {
try {
currentByte = d.read();
if(currentByte == -1){
return -1;
}
String binary = Integer.toBinaryString(currentByte);
for (int x = 0; x < binary.length(); x++) {
bufferedBits[x] = Character.valueOf(binary.charAt(x));
}
bitCount = 0;
} catch (IOException e) {
e.printStackTrace();
}
}
int val = bufferedBits[bitCount];
bitCount++;
return val % 2;
}
public String getTree() {
return tree;
}
public int totalChars() {
return totalChars;
}
public void close() {
try {
d.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
I know it's a bit lengthy of a question but any help is greatly appreciated!
I assume that you're looking to have enough leading 0s to make the length of the String that is returned from Integer#toBinaryString 8; the following code will achieve this for you:
String binary = String.format("%8s", Integer.toBinaryString(currentByte)).replace(' ', '0');
Related
It's my second time asking here and straight to the point. I can't seem to find a solution and I know it's not impossible. I wrote a java program that can generate a set of combination of any length, when I stop the program I don't want to start from the beginning how can I pick up from where I stopped?
Thanks.
Example (for length 3):
If I start from aaa ==> 9zI and I stop the program here, I don't want to start from aaa all over but start from 9zI and continue to 999. I just want to continue from where I left off.
public class Main {
public static void main(String[] args) {
S_Permutation sp = new S_Permutation();
String text = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789";
FileClass.fileExist("new.txt", true);
System.out.println("");
sp.permutation(text, "", 7, "sha256.txt","Kaaaaaa");
}
}
=====================================================================
public class S_Permutation {
private List<String> permutation;
public S_Permutation() {
permutation = new ArrayList<>();
}
public boolean saveThis(String words, char a, int limit) {
int count = 0;
limit++;
for (char character : words.toCharArray()) {
if (count == limit) {
return false;
}
if (character == a) {
count++;
} else {
count = 0;
}
}
return count < limit;
}
private int counter = 0;
private boolean seen = false;
public void permutation(String str, String prefix, int lengthOfPermutationString, String filename, String startPoint) {
if (prefix.equalsIgnoreCase(startPoint))
{
seen = true;
}
if (counter == 0) {
if (startPoint.length() != lengthOfPermutationString) {
for (int i = startPoint.length(); i < lengthOfPermutationString; i++) {
startPoint += str.charAt(0);
}
}
counter = -45;
}
if (prefix.length() == lengthOfPermutationString) {
boolean savethis = true;
for (int i = 0; i < prefix.length(); i++) {
savethis = this.saveThis(prefix, prefix.charAt(i), 13);
if (!savethis) {
break;
}
}
if (savethis && seen) {
System.out.println(prefix);
//permutation.add(prefix);
}
} else {
for (int i = 0; i < str.length(); i++) {
if (permutation.size() == 1000) {
FileClass.WriteFile("new.txt", permutation);
permutation.clear();
}
permutation(str, prefix + str.charAt(i), lengthOfPermutationString, filename, startPoint);
}
FileClass.WriteFile("new.txt", permutation);
permutation.clear();
}
}
}
=========================================================================
public class FileClass {
public static boolean WriteFile(String filename, List<String> doc) {
try {
if (!filename.contains(".txt")) {
filename += ".txt";
}
RandomAccessFile raf = new RandomAccessFile(filename, "rw");
String writer = "";
writer = doc.stream().map((string) -> string + "\n").reduce(writer, String::concat);
raf.seek(raf.length());
raf.writeBytes(writer);
raf.close();
} catch (Exception e) {
System.out.println(e.getMessage());
System.out.println("Error");
new Scanner(System.in).nextLine();
return false;
}
return true;
}
static RandomAccessFile raf;
public static boolean fileExist(String filename, boolean delete){
File file = new File(filename);
if (file.exists() && delete)
{
return file.delete();
}
return file.exists();
}
public static void WriteFile(String filename, String text) {
try {
if (!filename.contains(".txt")) {
filename += ".txt";
}
raf = new RandomAccessFile(filename, "rw");
long length = raf.length();
raf.setLength(length + 1);
raf.seek(raf.length());
raf.writeBytes(text + "\n");
} catch (Exception e) {
}
}
private static void write(List<String> records, Writer writer) throws IOException {
for (String record : records) {
writer.write(record);
}
writer.flush();
writer.close();
}
public static void stringWriter(List<String> records, String filename) {
try {
File file = new File(filename);
FileWriter writer = new FileWriter(file, true);
write(records, writer);
} catch (Exception ex) {
System.out.println(ex.getMessage());
new Scanner(System.in).nextLine();
}
}
public static boolean CloseFile() {
try {
raf.close();
return true;
} catch (Exception e) {
return false;
}
}
}
In order to add a "Resume" mechanism, you need to make your program idempotent. One way to do it, is instead of saving the permutations - save to file the parameters that are sent to permutation on each iteration:
now each time that the program starts, it will check what were the last parameters that permutation was called with (the last line in the file), and start from there (when the program starts on the first time, nothing will be written in the file - so it will start from the beginning).
After that the recursion finished, we can call another method that will go over the lines of the file, and read only the permutations (ignoring the other parameters) and write them into a cleaner "final_result.txt" file.
Needless to say that this implementation is more costly (all the additional reads and write from disc) but that's the tradeoff for having it support "resume" operation.
To save/restore process in the middle of its work, you need something we can call a "state" and implement generating combinations in iterative way.
In my implementation the "state" is pos object (I assume set and k will not change on "resume").
My implementation of the problem would be following:
public class RepeatComb {
private int[] pos;
private String set;
public RepeatComb(String set, int k) {
this.set = set;
pos = new int[k];
}
public int[] getState() {return Arrays.copyOf(pos, pos.length);}
public void resume(int[] a) {pos = Arrays.copyOf(a,a.length);}
public boolean next() {
int i = pos.length-1;
for (int maxpos = set.length()-1; pos[i] >= maxpos; ) {
if (i==0) return false;
--i;
}
++pos[i];
while (++i < pos.length) pos[i]=0;
return true;
}
public String getCur() {
StringBuilder s = new StringBuilder(pos.length);
for (int i=0; i < pos.length; ++i)
s.append(set.charAt(pos[i]));
return s.toString();
}
public static void main(String[] args) {
int[] state;
String text = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789";
RepeatComb comb = new RepeatComb(text, 3);
int stop = 10; //break after 10
do {
if (stop-- == 0) break;
System.out.println(comb.getCur());
} while (comb.next());
//save state
state = comb.getState();
System.out.println("---------");
//resume (with the same args: text,3)
stop = 10; //break after 10
comb = new RepeatComb(text, 3);
comb.resume(state); // resume here
do {
if (stop-- == 0) break;
System.out.println(comb.getCur());
} while (comb.next());
}
}
Update: I've added functions for getting state and resuming from it
and example of use. state array can be saved in file, then restored.
I have an Arduino program that sends via Bluetooth a byte array to a Java program.
The Java program receives the data on JSSC. The array is 72 bytes long (18 floats a´ 4byte).
How JSSC recognizes the end of transmission, or know that a new byte array becomes available?
In most cases, the transfer is correct. But from time to time waste is received.
I've found this example here.
http://www.javaprogrammingforums.com/java-se-api-tutorials/5603-jssc-library-easy-work-serial-ports.html
This
if(event.getEventValue() == 10)
means an linefeed, but bytearray have no linefeeds.
Arduino:
Serial.write(bytearray, size of bytearray);
Java:
class SerialPortReader implements SerialPortEventListener {
public void serialEvent(SerialPortEvent event) {
int dataNumber = 0;
try {
if(event.isRXCHAR()){
if(event.getEventValue() >= BYTE_NUMBER){
receivedByte = usbPort.readBytes(BYTE_NUMBER);
if(receivedByte != null) {
isRequestOK = true;
byte[] myArray = new byte[FLOATSIZE];
for(int i = 0, y = 0; i < receivedByte.length; i++, y++) { //i < receivedByte.length-1
if(myArray == null) {
y = 0;
myArray = new byte[FLOATSIZE];
}
if((i + 1) % FLOATSIZE != 0) {
myArray[y] = receivedByte[i];
}
else {
myArray[y] = receivedByte[i];
receivedValue[dataNumber] = ByteBuffer.wrap(myArray).order(ByteOrder.LITTLE_ENDIAN).getFloat();
myArray = null;
dataNumber++;
}
}
}
}
}
} catch (SerialPortException e) { } ///InvocationTargetException oder NullPointerException
}//--------------------------- End of serialEvent -------------------------------------------------------------
}//--------------------------- End of SerialPortReader ------------------------------------------------------------
Can anybody help me?
Best regards Willi
P.S. This is my first question in this forum. Hopefully I have followed the rules.
It's difficult to tell what is being asked here. This question is ambiguous, vague, incomplete, overly broad, or rhetorical and cannot be reasonably answered in its current form. For help clarifying this question so that it can be reopened, visit the help center.
Closed 10 years ago.
I'm trying to write a program that gets a .txt file that only has something like 10000010000010000010001
I'm trying to count the number of zeros and output it like 5 5 5 3. I thought if I convert a string into a double or int I could write an if or for loop.
import java.util.Scanner;
public class test1{
public static void main(String[] args) {
java.io.File test2 = new java.io.File("test3.txt");
try
{
Scanner input = new Scanner(test2);
while(input.hasNext())
{
String num = input.nextLine();
System.out.println(num);
double n = Double.parseDouble(num);
System.out.println(n);
}
}
catch (Exception e){
System.out.println("could not find file");
}
}
}
Here you go:
char[] numArray = num.toCharArray();
int counter=0;
for(int i=0;i<numArray.length;i++) {
if(numArray[i]=='0') {
counter++;
}
if((i==numArray.length-1&&counter>0)||(counter>0&&numArray[i]!='0')) {
System.out.println("Number of Zeroes: "+counter);
counter=0;
}
}
Some important points:
1) It's best to use an array of char values here, instead of operating using a double, because a char array can store many more values- the example you posted is too long for a double to handle.
2) Most of this should be self-explanatory (at least, if you study it bit-by-bit), but in case the i==numArray.length-1 part is confusing, this ensures that if the string ends with a 0, the final count of 0's will be printed out as well.
This should work for any string you can throw at it- including values besides 0 and 1, if you need support for it!
where is your effort?
you can simply try (if your string contains only 1s and 0s):
String[] splitArr = num.split("1");
String countStr = "";
for (int i = 0; i < splitArr.length; i++) {
if( ! splitArr[i].isEmpty() )
countStr += splitArr[i].length();
}
System.out.println(countStr);
import java.util.*;
import java.io.*;
public class ZeroCounter {
ArrayList <Integer> listOfNumbers = new ArrayList <Integer> ();
DataInputStream inStream;
long inFileSize;
long outFileSize;
// Track how many bytes we've read. Useful for large files.
int byteCount;
public ZeroCounter() {
}
//read the file and turn it into an array of integers
public void readFile(String fileName) {
try {
// Create a new File object, get size
File inputFile = new File(fileName);
inFileSize = inputFile.length();
// The constructor of DataInputStream requires an InputStream
inStream = new DataInputStream(new FileInputStream(inputFile));
}
// Oops. Errors.
catch (FileNotFoundException e) {
e.printStackTrace();
System.exit(0);
}
// Read the input file
try {
// While there are more bytes available to read...
while (inStream.available() > 0) {
// Read in a single byte and store it in a character
int c = (int)inStream.readByte();
if ((++byteCount)% 1024 == 0)
System.out.println("Read " + byteCount/1024 + " of " + inFileSize/1024 + " KB...");
// Print the integer to see them for debugging purposes
//System.out.print(c);
// Add the integer to an ArrayList
fileArray.add(c);
}
// clean up
inStream.close();
System.out.println("File has been converted into an ArrayList of Integers!");
}
// Oops. Errors.
catch (IOException e) {
e.printStackTrace();
System.exit(0);
}
//Print the ArrayList contents for debugging purposes
//System.out.println(fileArray);
}
public void countZeroes() {
int zeroCounter = 0;
for (int i = 0; i < listOfNumbers.size(); i++) {
if (listOfNumbers.get(i) == 0) {
zeroCounter++;
}
else if (listOfNumbers.get(i) != 0 && zeroCounter > 0) {
//this only prints the number of zeroes if the zero counter isn't zero
System.out.println(zeroCounter + " ");
zeroCounter = 0;
}
else {
//do nothing
}
}
}
public static void main(String[] args) {
ZeroCounter comp = new ZeroCounter();
comp.readFile("test3.txt");
comp.countZeroes();
}
}
It's difficult to tell what is being asked here. This question is ambiguous, vague, incomplete, overly broad, or rhetorical and cannot be reasonably answered in its current form. For help clarifying this question so that it can be reopened, visit the help center.
Closed 10 years ago.
how to write a Java class to sort 10 billion integers, assuming we can only fit a subset of them in memory at once.
I have done sorting but questions is how i would get the 1 billion values ?
How I am gonna sort them if i am going to load a portion of them in memory ?
If you can help me with source code it would be much appreciated.
Thanks in advance.
here is my last code, you can run it and guide me now.
import java.io.BufferedWriter;
import java.io.File;
import java.io.FileNotFoundException;
import java.io.FileWriter;
import java.io.IOException;
import java.io.PrintWriter;
import java.util.ArrayList;
import java.util.Collections;
import java.util.List;
import java.util.Random;
import java.util.Scanner;
/**
* #project jqcontacts
* #date Mar 26, 2012
* #time 11:16:35 AM
*/
public class SortIntegers {
private static String BIG_FILE="g:/bigFile.txt";
private static String SORT_FILE_PREFIX = "g:/sortFile";
/*private static final int SORT_FILE_MAX_SIZE = 1000000;*/
private static final int SORT_FILE_MAX_SIZE = 10;
private static final String MAIN_FILE = "g:/rawfile1.txt";
private static int RAWA_FILE_MAX_SIZE = 100;
// if i keep the size of MERGE_BUFFER_INITIAL_SIZE = SORT_FILE_MAX_SIZE, the end big file is sorted.
private static int MERGE_BUFFER_INITIAL_SIZE=5;
private static int MERGE_BUFFER_SIZE_NEXT = MERGE_BUFFER_INITIAL_SIZE;
private static int MERGE_BUFFER_SIZE_PREVIOUS = 0;
private static int countFile = 0;
public static void readFile(String name) throws FileNotFoundException{
Scanner scanner = new Scanner(new File(name));
List<Integer> intList = new ArrayList<Integer>();
int fileSize = 0 ;
while(scanner.hasNextInt()){
intList.add(scanner.nextInt());
++fileSize;
if(fileSize>=SORT_FILE_MAX_SIZE){
Collections.sort(intList);
/*System.out.println("list size: " + intList.size());*/
String fileName = SORT_FILE_PREFIX + countFile +".txt";
++fileSize;
PrintWriter out = openWriter(fileName);
for(int i:intList){
writeFile(i, out);
}
out.close();
intList.clear();
++countFile;
fileSize = 0;
}
}
System.out.println("done!");
}
public static List<Integer> readSortFile(String name, List<Integer> list) throws FileNotFoundException{
Scanner scanner = new Scanner(new File(name));
int bufferSize = 0;
while(scanner.hasNextInt()){
++bufferSize;
if(bufferSize>=MERGE_BUFFER_SIZE_PREVIOUS && bufferSize<=MERGE_BUFFER_SIZE_NEXT){
list.add(scanner.nextInt());
}
if(bufferSize>=MERGE_BUFFER_SIZE_NEXT){
break;
}
}
Collections.sort(list);
return list;
}
private static PrintWriter openWriter(String name) {
try {
File file = new File(name);
PrintWriter out = new PrintWriter(new BufferedWriter(new FileWriter(file)), true);
return out;
} catch (IOException e) {
//System.out.println("I/O Error");
e.printStackTrace();
System.exit(0);
}
return null;
}
private static void writeFile(int i, PrintWriter out) {
/* String line = "0" + "\t" + Integer.toString(i);*/
String line = Integer.toString(i) + "\t";
out.println(line);
}
/**
* #param args
*/
public static void main(String[] args) {
generateRawIntFile();
try {
readFile(MAIN_FILE);
} catch (FileNotFoundException e) {
e.printStackTrace();
}
System.out.println("countFile: " + countFile);
// merge sort here, merge the sorted files into one
List<Integer> comboList = new ArrayList<Integer>();
boolean isDone = true;
PrintWriter outP = openWriter(BIG_FILE);
while(isDone){
for(int i=0;i<countFile;i++){
try {
//TODO: do we need the return type for readSortFile ????
comboList = readSortFile(SORT_FILE_PREFIX+i+".txt", comboList);
} catch (FileNotFoundException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
// System.out.println("hun writing on big file " + comboList.size());
// add the list into bigfile and clear it for further processing
try{
for(int value:comboList){
writeFile(value, outP);
}
comboList.clear();
MERGE_BUFFER_SIZE_PREVIOUS = MERGE_BUFFER_SIZE_NEXT;
MERGE_BUFFER_SIZE_NEXT += MERGE_BUFFER_INITIAL_SIZE;
System.out.println("MERGE_BUFFER_SIZE_PREVIOUS: " + MERGE_BUFFER_SIZE_PREVIOUS + " MERGE_BUFFER_SIZE_NEXT:" + MERGE_BUFFER_SIZE_NEXT);
if(MERGE_BUFFER_SIZE_PREVIOUS >= RAWA_FILE_MAX_SIZE){
System.out.println("sorting is finished");
isDone = false;
break;
}
}catch (Exception e) {
e.printStackTrace();
}
}
}
/**
*
*/
public static void generateRawIntFile() {
Random randomGenerator = new Random();
PrintWriter out = openWriter(MAIN_FILE);
for (Integer i = 0; i < RAWA_FILE_MAX_SIZE;i++){
Integer value = randomGenerator.nextInt(RAWA_FILE_MAX_SIZE);
writeFile(value, out);
}
out.close();
}
}
There are only 4 billion possible int values so the most efficient way of doing this, is to count the number of occurrences of any value. You can use a memory MappedByteBuffer so you don't have to have 16 GB of memory. Once you have counted all the occurrences the counts will naturally be in order, so no further sorting is required. The time complexity is O(n) instead of O(n * log n) line merge sort or quick sort.
import sun.nio.ch.DirectBuffer;
import java.io.File;
import java.io.IOException;
import java.io.RandomAccessFile;
import java.nio.MappedByteBuffer;
import java.nio.channels.FileChannel;
public class Sort10Billion {
public static void main(String... args) throws IOException {
Runtime runtime = Runtime.getRuntime();
long used1 = runtime.totalMemory() - runtime.freeMemory();
MassiveCounterStore mcs = new MassiveCounterStore();
long start = System.nanoTime();
long count = 10 * 1000 * 1000 * 1000L;
for (long i = count; i > 0; i--)
mcs.incrementIndex((int) (i / 1019));
mcs.iterator(new NumberCountFunction() {
#Override
public void counted(int n, long count) {
// System.out.println(n + ": " + count);
}
});
long time = System.nanoTime() - start;
long used2 = runtime.totalMemory() - runtime.freeMemory();
System.out.printf("Took %.1f seconds to sort %,d numbers, using %.3f MB%n", time / 1e9, count, (used2-used1)/1e6);
mcs.close();
}
}
interface NumberCountFunction {
public void counted(int n, long count);
}
class MassiveCounterStore {
public static final int PARTITION_BITS = 26;
static final int PARTITIONS = (1 << (34 - PARTITION_BITS)); // 32-bit * 4 bytes.
final MappedByteBuffer[] buffers = new MappedByteBuffer[PARTITIONS];
final FileChannel channel;
int smallest = PARTITIONS;
int largest = 0;
public MassiveCounterStore() throws IOException {
File tmpStore = File.createTempFile("counter", "dat");
tmpStore.deleteOnExit();
channel = new RandomAccessFile(tmpStore, "rw").getChannel();
for (int i = 0; i < PARTITIONS; i++)
buffers[i] = channel.map(FileChannel.MapMode.READ_WRITE, (long) i << PARTITION_BITS, 1 << PARTITION_BITS);
}
public void incrementIndex(int n) {
long l = (n + Integer.MIN_VALUE) & 0xFFFFFFFFL;
int partition = (int) (l >> (PARTITION_BITS - 2)); // 4 bytes each.
int index = (int) ((l << 2) & ((1 << PARTITION_BITS) - 1));
MappedByteBuffer buffer = buffers[partition];
int count = buffer.getInt(index);
buffer.putInt(index, count + 1);
if (smallest > partition) smallest = partition;
if (largest < partition) largest = partition;
}
public void iterator(NumberCountFunction nfc) {
int n = (smallest << (PARTITION_BITS -2)) + Integer.MIN_VALUE;
for (int p = smallest; p <= largest; p++) {
MappedByteBuffer buffer = buffers[p];
for (int i = 0; i < 1 << PARTITION_BITS; i += 4) {
int count = buffer.getInt(i);
if (count != 0)
nfc.counted(n, count & 0xFFFFFFFFL);
n++;
}
}
assert n == Integer.MIN_VALUE;
}
public void close() {
try {
channel.close();
} catch (IOException ignored) {
}
for (MappedByteBuffer buffer : buffers) {
((DirectBuffer) buffer).cleaner().clean();
}
}
}
prints when run with -XX:-UseTLAB (which gives you more accurate memory usage)
Took 150.7 seconds to sort 10,000,000,000 numbers, using 0.202 MB
I think using 202 KB is pretty good. ;)
Note: your performance is heavily dependant on the distribution of values as this impacts the efficiency of the cache.
It's difficult to tell what is being asked here. This question is ambiguous, vague, incomplete, overly broad, or rhetorical and cannot be reasonably answered in its current form. For help clarifying this question so that it can be reopened, visit the help center.
Closed 12 years ago.
Is there a way that I can optimize this code as to not run out of memory?
import java.util.HashMap;
import java.util.Map;
import java.util.PriorityQueue;
import java.util.Random;
import java.util.Stack;
public class TilePuzzle {
private final static byte ROWS = 4;
private final static byte COLUMNS = 4;
private static String SOLUTION = "123456789ABCDEF0";
private static byte RADIX = 16;
private char[][] board = new char[ROWS][COLUMNS];
private byte x; // Row of the space ('0')
private byte y; // Column of the space ('0') private String representation;
private boolean change = false; // Has the board changed after the last call to toString?
private TilePuzzle() {
this(SOLUTION);
int times = 1000;
Random rnd = new Random();
while(times-- > 0) {
try {
move((byte)rnd.nextInt(4));
}
catch(RuntimeException e) {
}
}
this.representation = asString();
}
public TilePuzzle(String representation) {
this.representation = representation;
final byte SIZE = (byte)SOLUTION.length();
if (representation.length() != SIZE) {
throw new IllegalArgumentException("The board must have " + SIZE + "numbers.");
}
boolean[] used = new boolean[SIZE];
byte idx = 0;
for (byte i = 0; i < ROWS; ++i) {
for (byte j = 0; j < COLUMNS; ++j) {
char digit = representation.charAt(idx++);
byte number = (byte)Character.digit(digit, RADIX);
if (number < 0 || number >= SIZE) {
throw new IllegalArgumentException("The character " + digit + " is not valid.");
} else if(used[number]) {
throw new IllegalArgumentException("The character " + digit + " is repeated.");
}
used[number] = true;
board[i][j] = digit;
if (digit == '0') {
x = i;
y = j;
}
}
}
}
/**
* Swap position of the space ('0') with the number that's up to it.
*/
public void moveUp() {
try {
move((byte)(x - 1), y);
} catch(IllegalArgumentException e) {
throw new RuntimeException("Move prohibited " + e.getMessage());
}
}
/**
* Swap position of the space ('0') with the number that's down to it.
*/
public void moveDown() {
try {
move((byte)(x + 1), y);
} catch(IllegalArgumentException e) {
throw new RuntimeException("Move prohibited " + e.getMessage());
}
}
/**
* Swap position of the space ('0') with the number that's left to it.
*/
public void moveLeft() {
try {
move(x, (byte)(y - 1));
} catch(IllegalArgumentException e) {
throw new RuntimeException("Move prohibited " + e.getMessage());
}
}
/**
* Swap position of the space ('0') with the number that's right to it.
*/
public void moveRight() {
try {
move(x, (byte)(y + 1));
} catch(IllegalArgumentException e) {
throw new RuntimeException("Move prohibited " + e.getMessage());
}
}
private void move(byte movement) {
switch(movement) {
case 0: moveUp(); break;
case 1: moveRight(); break;
case 2: moveDown(); break;
case 3: moveLeft(); break;
}
}
private boolean areValidCoordinates(byte x, byte y) {
return (x >= 0 && x < ROWS && y >= 0 && y < COLUMNS);
}
private void move(byte nx, byte ny) {
if (!areValidCoordinates(nx, ny)) {
throw new IllegalArgumentException("(" + nx + ", " + ny + ")");
}
board[x][y] = board[nx][ny];
board[nx][ny] = '0';
x = nx;
y = ny;
change = true;
}
public String printableString() {
StringBuilder sb = new StringBuilder();
for (byte i = 0; i < ROWS; ++i) {
for (byte j = 0; j < COLUMNS; ++j) {
sb.append(board[i][j] + " ");
}
sb.append("\r\n");
}
return sb.toString();
}
private String asString() {
StringBuilder sb = new StringBuilder();
for (byte i = 0; i < ROWS; ++i) {
for (byte j = 0; j < COLUMNS; ++j) {
sb.append(board[i][j]);
}
}
return sb.toString();
}
public String toString() {
if (change) {
representation = asString();
}
return representation;
}
private static byte[] whereShouldItBe(char digit) {
byte idx = (byte)SOLUTION.indexOf(digit);
return new byte[] { (byte)(idx / ROWS), (byte)(idx % ROWS) };
}
private static byte manhattanDistance(byte x, byte y, byte x2, byte y2) {
byte dx = (byte)Math.abs(x - x2);
byte dy = (byte)Math.abs(y - y2);
return (byte)(dx + dy);
}
private byte heuristic() {
byte total = 0;
for (byte i = 0; i < ROWS; ++i) {
for (byte j = 0; j < COLUMNS; ++j) {
char digit = board[i][j];
byte[] coordenates = whereShouldItBe(digit);
byte distance = manhattanDistance(i, j, coordenates[0], coordenates[1]);
total += distance;
}
}
return total;
}
private class Node implements Comparable<Node> {
private String puzzle;
private byte moves; // Number of moves from original configuration
private byte value; // The value of the heuristic for this configuration.
public Node(String puzzle, byte moves, byte value) {
this.puzzle = puzzle;
this.moves = moves;
this.value = value;
}
#Override
public int compareTo(Node o) {
return (value + moves) - (o.value + o.moves);
}
}
private void print(Map<String, String> antecessor) {
Stack toPrint = new Stack();
toPrint.add(SOLUTION);
String before = antecessor.get(SOLUTION);
while (!before.equals("")) {
toPrint.add(before);
before = antecessor.get(before);
}
while (!toPrint.isEmpty()) {
System.out.println(new TilePuzzle(toPrint.pop()).printableString());
}
}
private byte solve() {
if(toString().equals(SOLUTION)) {
return 0;
}
PriorityQueue<Node> toProcess = new PriorityQueue();
Node initial = new Node(toString(), (byte)0, heuristic());
toProcess.add(initial);
Map<String, String> antecessor = new HashMap<String, String>();
antecessor.put(toString(), "");
while(!toProcess.isEmpty()) {
Node actual = toProcess.poll();
for (byte i = 0; i < 4; ++i) {
TilePuzzle t = new TilePuzzle(actual.puzzle);
try {
t.move(i);
} catch(RuntimeException e) {
continue;
}
if (t.toString().equals(SOLUTION)) {
antecessor.put(SOLUTION, actual.puzzle);
print(antecessor);
return (byte)(actual.moves + 1);
} else if (!antecessor.containsKey(t.toString())) {
byte v = t.heuristic();
Node neighbor = new Node(t.toString(), (byte)(actual.moves + 1), v);
toProcess.add(neighbor);
antecessor.put(t.toString(), actual.puzzle);
}
}
}
return -1;
}
public static void main(String... args) {
TilePuzzle puzzle = new TilePuzzle();
System.out.println(puzzle.solve());
}
}
The problem
The root cause is the tons of String objects you are creating and storing in the toProcess Queue and the antecessor Map. Why are you doing that?
Look at your algorithm. See if you really need to store >2 million nodes and 5 million strings in each.
The investigation
This was hard to spot because the program is complex. Actually, I didn't even try to understand all of the code. Instead, I used VisualVM – a Java profiler, sampler, and CPU/memory usage monitor.
I launched it:
And took a look at the memory usage. The first thing I noticed was the (obvious) fact that you're creating tons of objects.
This is an screenshot of the app:
As you can see, the amount of memory used is tremendous. In as few as 40 seconds, 2 GB were consumed and the entire heap was filled.
A dead end
I initially thought the problem had something to do with the Node class, because even though it implements Comparable, it doesn't implement equals. So I provided the method:
public boolean equals( Object o ) {
if( o instanceof Node ) {
Node other = ( Node ) o;
return this.value == other.value && this.moves == other.moves;
}
return false;
}
But that was not the problem.
The actual problem turned out to be the one stated at the top.
The workaround
As previously stated, the real solution is to rethink your algorithm. Whatever else can be done, in the meantime, will only delay the problem.
But workarounds can be useful. One is to reuse the strings you're generating. You're very intensively using the TilePuzzle.toString() method; this ends up creating duplicate strings quite often.
Since you're generating string permutations, you may create many 12345ABCD strings in matter of seconds. If they are the same string, there is no point in creating millions of instances with the same value.
The String.intern() method allows strings to be reused. The doc says:
Returns a canonical representation for the string object.
A pool of strings, initially empty, is maintained privately by the class String.
When the intern method is invoked, if the pool already contains a string equal to this String object as determined by the equals() method, then the string from the pool is returned. Otherwise, this String object is added to the pool and a reference to this String object is returned.
For a regular application, using String.intern() could be a bad idea because it doesn't let instances be reclaimed by the GC. But in this case, since you're holding the references in your Map and Queue anyway, it makes sense.
So making this change:
public String toString() {
if (change) {
representation = asString();
}
return representation.intern(); // <-- Use intern
}
Pretty much solves the memory problem.
This is a screenshot after the change:
Now, the heap usage doesn't reach 100 MB even after a couple of minutes.
Extra remarks
Remark #1
You're using an exception to validate if the movement is valid or not, which is okay; but when you catch them, you're just ignoring them:
try {
t.move(i);
} catch(RuntimeException e) {
continue;
}
If you're not using them anyway, you can save a lot of computation by not creating the exceptions in the first place. Otherwise you're creating millions of unused exceptions.
Make this change:
if (!areValidCoordinates(nx, ny)) {
// REMOVE THIS LINE:
// throw new IllegalArgumentException("(" + nx + ", " + ny + ")");
// ADD THIS LINE:
return;
}
And use validation instead:
// REMOVE THESE LINES:
// try {
// t.move(i);
// } catch(RuntimeException e) {
// continue;
// }
// ADD THESE LINES:
if(t.isValidMovement(i)){
t.move(i);
} else {
continue;
}
Remark #2
You're creating a new Random object for every new TilePuzzle instance. It would be better if you used just one for the whole program. After all, you are only using a single thread.
Remark #3
The workaround solved the heap memory problem, but created another one involving PermGen. I simply increased the PermGen size, like this:
java -Xmx1g -Xms1g -XX:MaxPermSize=1g TilePuzzle
Remark #4
The output was sometimes 49 and sometimes 50. The matrices were printed like:
1 2 3 4
5 6 7 8
9 A B C
D E 0 F
1 2 3 4
5 6 7 8
9 A B C
D E F 0
... 50 times