I'm writing a program which creates a large number of large arrays to store data. All of this data has to held in RAM, so I'm avoiding objects and currently using shorts to save space. These shorts serve as ID numbers which can be put into a lookup class to get the corresponding object on demand. I have recently questioned whether I'll need the whole 2 bytes of a short, and so I'm now wondering if there's anyway to define the data type being stored in one place in my code so that I can change it easily without having to hunt down every cast, return type, etc. that is currently set to short.
If I were willing to use objects I could easily just do
class MySmallNumber extends Short{}
and change the parent class if necessary.
If this were C/C++, i could use
#define small short
for the effect I'm looking for.
I'm searching for a way to do something like this in java that won't require storing 64-bit object references in my arrays. Any help is greatly appreciated. Right now I'm looking at a really messy IDE replace all in order to do this.
You can incapsulate you array in some custom class. It shouldn't add considerable space overhead because you work with large arrays.
In all other places in your code you can use long. When you pass these longs to you array custom class you can convert it to the one you use inside it.
Finally you have to make changes in this one class only.
I would suggest factoring out all code that depends on the type of your ID values into a separate class. Let that class handle all the operations (including lookup) that depend on whether the ID values are short, byte, or something else. You can pass individual values in and out as short or even int values, even if internally they are converted to byte. (This is, for instance, how java.io.DataOutputStream.writeByte(int) was written—it takes an int argument and treats it as a byte value.)
not quite sure what you are after here, but this may be of interest:
import java.util.Arrays;
interface Index {
short getIndex(int i);
void setIndex(int i, short value);
int size();
}
class ShortIndexImpl implements Index {
ShortIndexImpl(int n) {
indices = new short[n];
}
#Override public short getIndex(int i) {
return indices[i];
}
#Override public void setIndex(int i, short value) {
indices[i] = value;
}
#Override public int size() {
return indices.length;
}
final short[] indices;
}
class TenBitIndexImpl implements Index {
TenBitIndexImpl(int n) {
indices = new int[(n + 2) / 3];
}
#Override public short getIndex(int i) {
int index = i / 3;
int remainder = i % 3;
int word = indices[index];
return (short) (0x3ff & (word >> shifts[remainder]));
}
#Override public void setIndex(int i, short value) {
int index = i / 3;
int remainder = i % 3;
int word = indices[index] & ~masks[remainder];
int shiftedValue = ((int) value) << shifts[remainder];
word |= shiftedValue;
indices[index] = word;
}
#Override public int size() {
return indices.length;
}
final int masks[] = new int[] { 0x3ff00000, 0xffc00, 0x3ff };
final int shifts[] = new int[] { 20, 10, 0 };
final int[] indices;
}
public class Main {
static void test(Index index) {
for (int i = 0; i < values.length; i++)
index.setIndex(i, values[i]);
for (int i = 0; i < values.length; i++) {
System.out.println(values[i] + " " + index.getIndex(i));
if (index.getIndex(i) != values[i])
System.out.println("expected " + values[i] + " but got " + index.getIndex(i));
}
}
public static void main(String[] args) {
Index index = new ShortIndexImpl(values.length);
test(index);
index = new TenBitIndexImpl(values.length);
test(index);
System.out.println("indices");
for (int i = 0; i < ((TenBitIndexImpl) index).indices.length; i++)
System.out.println(((TenBitIndexImpl) index).indices[i]);
}
static short[] values = new short[] { 1, 2, 3, 4, 5, 6 };
}
Related
I'm trying to write a class named Range, that takes an array of integers (unsorted) of length n, containing only numbers in the range are from 0 to k.
At first, I declare a constructor which will preprocess the array via Counting Sort algorithm.
Then I want to write query() method that takes two integer arguments: a and b, which form a range of numbers from a to b and returns the total frequency of all the elements in the array having the values within the given range.
My code:
import java.util.Arrays;
import java.util.HashMap;
public class Range {
private int[] a;
private int k;
public Range(int[] a, int k) {
int index = 0;
int[] counterArray = new int[k + 1];
for (int i : a)
counterArray[i]++; // initialize counterArray
for (int i = 0; i < counterArray.length; i++)
while (0 < counterArray[i]) {
a[index++] = i;
counterArray[i]--;
} // end while()
this.a = a;
this.k = k;
} // end constructor()
public int query(int a, int b) {
HashMap<Integer, Integer> map = new HashMap<>(a);
} // end query()
#Override
public String toString() {
return Arrays.toString(a);
} // end toString()
}
I chose HashMap data structure because I need query() method to be executed in constant time O(1).
So my question is: Is it possible to implement the method query() via HashMap?
If not, what are the alternatives? (Note: the time complexity should be O(1) for query(), not bothering about the space complexity).
Code in the main() :
int[] a = {13,12,13,1,2,0,0,1,3,4};
Range range = new Range(a, 13);
System.out.print(range); // prints [0,0,1,1,2,3,4,12,13,13] because array has been sorted
System.out.print(range.query(1, 4)); // calculating number of elements in the range [1, 4]
Expected Output:
5 // elements 1,1,2,3,4 are within the range [1, 4]
Explanation: provided arguments of the query() are: a=1 and b=4, hence, values to be tested are 1,2,3,4. The output should be 5 because there are 5 elements: 1,1,2,3,4.
To obtain the number of elements in the given range (from a to b inclusive) in O(1) time after the array has been sorted, you don't need to use HashMap. Instead, you can reuse the countingArray by making it an instance variable.
This approach also requires a slight modification of the sorting in order to retain the values in the countingArray intact. It's done by introducing one additional variable.
Note that it's a good practice to avoid mutating the input, that why in the code I've used Arrays.copyOf() (you can remove it, if you consider it irrelevant for this exercise).
I've extracted the logic responsible for sorting from the constructor into a separate method. And introduced a method which is meant to calculate the cumulative count for every number in the array (i.e. a number of element having values from 0 up to the current number inclusive).
So, after invoking method init() on the instance of Range we would be able to find the number of elements in the range from a to b by looking at the values stored in the countingArray at corresponding indices. And that would have a cost O(1).
public class Range {
private int[] arr;
private int[] counterArray;
private int k;
private Range(int[] arr, int k) { // constructor is not exposed
this.arr = Arrays.copyOf(arr, arr.length);
this.counterArray = new int[k + 1];
this.k = k;
}
public static Range getInstance(int[] arr, int k) {
Range range = new Range(arr, k);
range.init();
return range;
}
private void init() {
sort();
sumUpCount();
}
private void sort() {
for (int i : arr) {
counterArray[i]++;
}
int index = 0;
int copy;
for (int i = 0; i < counterArray.length; i++) {
copy = counterArray[i];
while (0 < counterArray[i]) {
arr[index++] = i;
counterArray[i]--;
}
counterArray[i] = copy;
}
}
private void sumUpCount() {
for (int i = 1; i < counterArray.length; i++) {
counterArray[i] += counterArray[i - 1];
}
}
public int query(int a, int b) {
return a == 0 ? counterArray[b] : counterArray[b] - counterArray[a - 1];
}
}
main()
public static void main(String[] args) {
int[] a = {13,12,13,1,2,0,0,1,3,4};
Range range = Range.getInstance(a, 13);
System.out.print(range.query(1,4));
}
Output:
5
Yes, in order to cache/precompute the return values of query(), you need to create a composite key, that holds both values. The easiest way to do that is to use a string that holds both numbers divided by a separator. Separator is important otherwise composite key(21, 5) = "215" and key(2, 15) = "215". With separator that would be "21;5" and "2;15" respectivly.
private String key(int a, int b) {
return String.format("%d;%d", a, b);
}
Then for each composite key possible you put the value into HashMap. In the query(a, b) method you just get the value from the Map.
public query(int a, int b) {
return map.get(key(a, b));
}
The downside of this approach is that creation of this object is pretty expensive.
i get this error after waiting long time for my code to execute and its pointing me to this method
public Iterable<Board> neighbors() {
Queue<Board> q = new LinkedList<>();
int n = dimension();
int x = 0, y = 0;
outer:
// do some stuff to get the x and y
if (y+1 < n) {
the line where i get the error -> int [][]arr = new int[n][n];
for (int i = 0; i < tiles.length; i++) {
arr[i] = Arrays.copyOf(tiles[i], n);
}
// do some stuff
Board br = new Board(arr);
if(!this.equals(br)) {
q.add(new Board(arr));
}
}
if (y-1 >= 0) {
int [][]arr = new int[n][n];
for (int i = 0; i < tiles.length; i++) {
arr[i] = Arrays.copyOf(tiles[i], n);
}
// do some stuff
Board br = new Board(arr);
if(!this.equals(br)) {
q.add(new Board(arr));
}
}
if (x-1 >= 0) {
int [][]arr = new int[n][n];
for (int i = 0; i < tiles.length; i++) {
arr[i] = Arrays.copyOf(tiles[i], n);
}
// do some stuff
Board br = new Board(arr);
if(!this.equals(br)) {
q.add(new Board(arr));
}
}
if (x+1 < n) {
int [][]arr = new int[n][n];
for (int i = 0; i < tiles.length; i++) {
arr[i] = Arrays.copyOf(tiles[i], n);
}
// do some stuff
Board br = new Board(arr);
if(!this.equals(br)) {
q.add(new Board(arr));
}
}
return q;
}
i basically need to copy tiles array and make changes to the copy "arr" but keep the tiles array without changing to use it later..i really don't like the way i'm doing it copying and pasting code i think its inefficient but no other way comes to my mind so i would like to know why i get this error "i know its because GC taking more time and not doing alot" but i want to know why its happening in this case also if there is better way to copy the array.
also i increased the heap memory to -Xmx1600m
Thanks for your time.
The Problem
It is likely that the problem arises from creating a lot of objects in a short period of time. See this answer for more information.
At the moment, one Board consist of at least four objects:
The Board itself
The array arr inside the board
The three arrays inside arr
Creating Less Objects
Our goal is to create fewer objects (arrays). Since you want to deal with small boards only, we could use one long to store the complete 3×3 board. A long has 64 bit. We use 64 / 9 = 7 bits per field to store the value on that field:
state = ... 0000100 0000011 0000010 0000001 0000000
4th field ↑ 2nd field ↑ 0th field
3rd field 1st field
The following class handles the bit operations.
class Board {
private final static int SIDE_LENGTH = 3;
private final static int FIELDS = SIDE_LENGTH * SIDE_LENGTH;
private final static int BITS_PER_FIELD = 64 / FIELDS;
private final static long FIELD_MASK = (1 << BITS_PER_FIELD) - 1;
private long state;
public Board() {
for (int field = 0; field < FIELDS; ++field) {
set(field, field);
}
}
/** Copy constructor. */
public Board(Board other) {
this.state = other.state;
}
public int get(int x, int y) {
return get(coordinatesToField(x, y));
}
public void set(int x, int y, int value) {
set(coordinatesToField(x, y), value);
}
private int coordinatesToField(int x, int y) {
return SIDE_LENGTH * y + x;
}
private int get(int field) {
return (int) ((state >>> (field * BITS_PER_FIELD)) & FIELD_MASK);
}
private void set(int field, int value) {
int shift = field * BITS_PER_FIELD;
state &= ~(FIELD_MASK << shift);
state |= (long) value << shift;
}
public String toString() {
StringBuilder sb = new StringBuilder();
for (int field = 0; field < FIELDS; ++field) {
sb.append(get(field));
sb.append((field + 1) % SIDE_LENGTH == 0 ? "\n" : "\t");
}
return sb.toString();
}
// TODO implement equals and hashCode
}
When using this class, you don't have to deal with arrays anymore, which saves not only a lot of objects, but also the copy code in your prorgram.
The class also works for 1×1, 2×2, and 4×4 boards, but not for larger ones due to the 64 bit limit.
Usage Examples
public static void main(String[] args) {
// Create and print the initial board
// 0 1 2
// 3 4 5
// 6 7 8
Board b = new Board();
System.out.println(b);
// Copy an existing board
Bord copy = new Board(b);
// Set the upper right field to value 8
copy.set(2, 0, 8);
// Print the center field
// 4
Syste.out.println(copy.get(1, 1));
}
Additional Ideas
You even could avoid creating Board objects at all, and just store the long values. But that doesn't help when you are using generics (such as LinkedList) because of Java's auto boxing.
Also note that LinkedList wraps each entry in an additional node object. Maybe you can use a more efficient DataStructure like a circular buffer.
Depending on what you are doing, you might as well have a look at the Flyweight design pattern.
I'm trying to learn Java and I am currently writing a program that will split an array of integers into two sub-arrays, one containing the positive values and the other the negative values.
Due to the fact that the sizes of the two sub-arrays can not be specified from the start (because this program should work on any array of integers) I wrote two methods which will calculate the sizes of the two sub-arrays (maxNeg and maxPos). Afterwards, the two sub-arrays (arrayNegative and arrayPositive) are initialized having the the two corresponding sizes.
The problem is that, when I try to populate the two arrays using the arraySorter(), the compiler gives me the ArrayIndexOutOfBoundsException: 0 error right at the first iteration inside the arraySorter() method.
Note 1: The issue doesn't occur when values are assigned to the maxNeg and maxPos variables from the start.
Note 2: I know this type of problem is usually solved using ArrayLists for storing the positives and negatives, but my assignment's requirements force me to do this using only arrays.
public class sortMethod {
public int max; // max = main array length
public int maxNeg; // the length of the final array that will store the
// negative integers
public int maxPos; // the length of the final array that will store the
// positive integers
public int[] intArray = new int[max];
public int getMaxNeg() {
for (int i = 0; i < max; i++) {
if (intArray[i] < 0) {
maxNeg++;
}
}
return maxNeg;
}
public int getMaxPos() {
for (int i = 0; i < max; i++) {
if (intArray[i] >= 0) {
maxPos++;
}
}
return maxPos;
}
public int[] negativeArray = new int[maxNeg];
public int[] positiveArray = new int[maxPos];
public int negIndex = 0;
public int posIndex = 0;
public void arraySorter() {
for (int a = 0; a < max; a++) {
if (intArray[a] < 0) {
negativeArray[negIndex] = intArray[a];
negIndex++;
// System.out.println(negativeArray[0]);
} else {
positiveArray[posIndex] = intArray[a];
posIndex++;
}
}
}
public sortMethod(int[] y, int z) {
this.intArray = y;
this.max = z;
}
Can someone explain why I'm keep getting the NullPointerException when using formulas to calculate the sizes of the two sub-arrays and why I don't get the error when values are assigned to 2 variables upon declaring them.
This is the main method where I created the test object:
public class ArraySort {
public static void main(String[] args) {
int[] array = { -12, 23, -22, 0, 43, -545, -4, -55, -43, -12, 0, -999};
int z = array.length;
sortMethod arrayTest = new sortMethod(array, z);
arrayTest.getMaxNeg();
arrayTest.getMaxPos();
arrayTest.arraySorter();
}
Thanks and please excuse me if my question's formating is not compliant with the site's standards, it's my first and I will try to improve in the future.
When this code is executed
public int[] negativeArray = new int[maxNeg];
public int[] positiveArray = new int[maxPos];
maxNeg and maxPos is equal to 0, so it is normal to have an out of bound exception.
To get it to works, you could initialize the arrays directly in the method so your code would become :
public int[] negativeArray;
public int[] positiveArray;
public int negIndex = 0;
public int posIndex = 0;
public void arraySorter() {
negativeArray = new int[maxNeg];
positiveArray = new int[maxPos];
for (int a = 0; a < max; a++) {
if (intArray[a] < 0) {
negativeArray[negIndex] = intArray[a];
negIndex++;
} else {
positiveArray[posIndex] = intArray[a];
posIndex++;
}
}
}
Another part of your code that is wrong is the constructor of sortMethod. You give the array length to the maximum bound, but the maximum bound is actually length - 1. so change this.max = z; to this.max = z - 1;
Also, your algorithm does not sort numbers, it simply separate negatives of positives... If you want to sort an array of number you can simplify it in one line :
Arrays.sort(array);
The order in which you define variables and methods in Java is not important. Your instance variables (e.g. your negativeArray and positiveArray) are created as soon as the class instance is created (and before the constructor is called). You therefore initialize both arrays with a size of 0.
Every source I've looked at I either don't understand, doesn't seem to apply, or uses something like an Array list. I'm not familiar with those. So I'd like to use a basic toString method that prints out the index of the array as well as the number held when compared to the variable 'length' -- num.length could be different as that's the physical size of the underlying array. The for loop at the bottom has the gist of it. I'm trying to print out the index (0-infinite) with int's that are held in the resizeable array. The variable 'length' is not the actual size of the array but a working size that contains 0 until another cell is added. The 'strang' variable is just something I've tried. I don't think it will work, but anything else I doesn't seem to help as I'm stuck.
public class XArray
{
private int[] nums;
private int length;
public XArray()
{
nums = new int[10];
length = 0;
}
public void add(int value)
{
if (length == nums.length)
{
int[] nums2 = new int[(int)(nums.length * 1.2)];
for ( int i = length - 1; i >= 0; i-- )
{
nums2[i] = nums[i];
}
nums = nums2;
}
nums[length] = value;
length++;
}
public void set(int index, int value)
{
if (index < length)
{
nums[index] = value;
}
}
public int get(int index)
{
return nums[index];
}
public int size()
{
return length;
}
public void remove()
{
nums[length - 1] = 0;
length--;
}
public String toString(int[] nums)
{
String strang = "l";
for ( int i = 0 ; i < length; i++ )
{
strang = "Index: " + i + " Number: " + nums[i] + ", ";
}
return strang;
}
}
You need to concatenate the values on each iteration of the loop...something like...
public String toString(int[] nums)
{
StringBuilder strang = new StringBuilder(length);
for ( int i = 0 ; i < length; i++ )
{
strang.append("Index: ").append(i).append(" Number: ").append(nums[i]).append(", ");
}
return strang.toString();
}
Generally speaking, toString should't take parameters, there would a difference between nums and length which could cause issues
#Override
public String toString() {...
This way, you will be printing the contents of the objects num array, which is contextually associated with length
You probably meant to use += instead of = in that method (though many people will tell you to use a StringBuilder because successive concatenations, if not optimized by a compiler` will generate a lot of garbage).
Also, don't pass in nums! You want to use the field nums; passing in an argument will use the argument. The real toString has no parameters (and should have an #Override annotation).
I got this question in an interview the other day and would like to know some best possible answers(I did not answer very well haha):
Scenario: There is a webpage that is monitoring the bytes sent over a some network. Every time a byte is sent the recordByte() function is called passing that byte, this could happen hundred of thousands of times per day. There is a button on this page that when pressed displays the last 100 bytes passed to recordByte() on screen (it does this by calling the print method below).
The following code is what I was given and asked to fill out:
public class networkTraffic {
public void recordByte(Byte b){
}
public String print() {
}
}
What is the best way to store the 100 bytes? A list? Curious how best to do this.
Something like this (circular buffer) :
byte[] buffer = new byte[100];
int index = 0;
public void recordByte(Byte b) {
index = (index + 1) % 100;
buffer[index] = b;
}
public void print() {
for(int i = index; i < index + 100; i++) {
System.out.print(buffer[i % 100]);
}
}
The benefits of using a circular buffer:
You can reserve the space statically. In a real-time network application (VoIP, streaming,..)this is often done because you don't need to store all data of a transmission, but only a window containing the new bytes to be processed.
It's fast: can be implemented with an array with read and write cost of O(1).
I don't know java, but there must be a queue concept whereby you would enqueue bytes until the number of items in the queue reached 100, at which point you would dequeue one byte and then enqueue another.
public void recordByte(Byte b)
{
if (queue.ItemCount >= 100)
{
queue.dequeue();
}
queue.enqueue(b);
}
You could print by peeking at the items:
public String print()
{
foreach (Byte b in queue)
{
print("X", b); // some hexadecimal print function
}
}
Circular Buffer using array:
Array of 100 bytes
Keep track of where the head index is i
For recordByte() put the current byte in A[i] and i = i+1 % 100;
For print(), return subarray(i+1, 100) concatenate with subarray(0, i)
Queue using linked list (or the java Queue):
For recordByte() add new byte to the end
If the new length to be more than 100, remove the first element
For print() simply print the list
Here is my code. It might look a bit obscure, but I am pretty sure this is the fastest way to do it (at least it would be in C++, not so sure about Java):
public class networkTraffic {
public networkTraffic() {
_ary = new byte[100];
_idx = _ary.length;
}
public void recordByte(Byte b){
_ary[--_idx] = b;
if (_idx == 0) {
_idx = _ary.length;
}
}
private int _idx;
private byte[] _ary;
}
Some points to note:
No data is allocated/deallocated when calling recordByte().
I did not use %, because it is slower than a direct comparison and using the if (branch prediction might help here too)
--_idx is faster than _idx-- because no temporary variable is involved.
I count backwards to 0, because then I do not have to get _ary.length each time in the call, but only every 100 times when the first entry is reached. Maybe this is not necessary, the compiler could take care of it.
if there were less than 100 calls to recordByte(), the rest is zeroes.
Easiest thing is to shove it in an array. The max size that the array can accommodate is 100 bytes. Keep adding bytes as they are streaming off the web. After the first 100 bytes are in the array, when the 101st byte comes, remove the byte at the head (i.e. 0th). Keep doing this. This is basically a queue. FIFO concept. Ater the download is done, you are left with the last 100 bytes.
Not just after the download but at any given point in time, this array will have the last 100 bytes.
#Yottagray Not getting where the problem is? There seems to be a number of generic approaches (array, circular array etc) & a number of language specific approaches (byteArray etc). Am I missing something?
Multithreaded solution with non-blocking I/O:
private static final int N = 100;
private volatile byte[] buffer1 = new byte[N];
private volatile byte[] buffer2 = new byte[N];
private volatile int index = -1;
private volatile int tag;
synchronized public void recordByte(byte b) {
index++;
if (index == N * 2) {
//both buffers are full
buffer1 = buffer2;
buffer2 = new byte[N];
index = N;
}
if (index < N) {
buffer1[index] = b;
} else {
buffer2[index - N] = b;
}
}
public void print() {
byte[] localBuffer1, localBuffer2;
int localIndex, localTag;
synchronized (this) {
localBuffer1 = buffer1;
localBuffer2 = buffer2;
localIndex = index;
localTag = tag++;
}
int buffer1Start = localIndex - N >= 0 ? localIndex - N + 1 : 0;
int buffer1End = localIndex < N ? localIndex : N - 1;
printSlice(localBuffer1, buffer1Start, buffer1End, localTag);
if (localIndex >= N) {
printSlice(localBuffer2, 0, localIndex - N, localTag);
}
}
private void printSlice(byte[] buffer, int start, int end, int tag) {
for(int i = start; i <= end; i++) {
System.out.println(tag + ": "+ buffer[i]);
}
}
Just for the heck of it. How about using an ArrayList<Byte>? Say why not?
public class networkTraffic {
static ArrayList<Byte> networkMonitor; // ArrayList<Byte> reference
static { networkMonitor = new ArrayList<Byte>(100); } // Static Initialization Block
public void recordByte(Byte b){
networkMonitor.add(b);
while(networkMonitor.size() > 100){
networkMonitor.remove(0);
}
}
public void print() {
for (int i = 0; i < networkMonitor.size(); i++) {
System.out.println(networkMonitor.get(i));
}
// if(networkMonitor.size() < 100){
// for(int i = networkMonitor.size(); i < 100; i++){
// System.out.println("Emtpy byte");
// }
// }
}
}