We Keep Coding

What is an efficient in place way of sorting an array based on an index array?

In certain machine learning algorithms the columns of the matrix are rotated and sorted based relevance of each column. New data to come should be transformed in the same order. So if my initial sort gives me [0,2,1,3] as an index array, than new data should also be ordered in this way: first, third, second, fourth element. That's why I wanted to create a sorted index array, that could later on be used as a source for reordering new data. I've managed to do that in the implementation below.
My question is about the use of the index array for reoordering new data. In my implementation I first create a clone of the new data array. Than it's easy to just copy elements from my source array to the proper index in the target array. Is this the most efficient way to do it? Or is there a more efficient way, for instance by sorting the data in place?
import java.util.stream.*;
import java.util.*;
public class IndexSorter<T> {
private final int[] indices;
private final int[] reverted;
public IndexSorter(T[] data, Comparator<T> comparator){
// generate index array based on initial data and a comparator:
indices = IntStream.range(0, data.length)
.boxed()
.sorted( (a, b) -> comparator.compare(data[a],data[b]))
.mapToInt(a -> a)
.toArray();
// also create an index array to be able to revert the sort
reverted = new int[indices.length];
for(int i=0;i<indices.length;i++){
reverted[indices[i]] = i;
}
}
// sort new data based on initial array
public T[] sort(T[] data){
return sortUsing(data, indices);
}
// revert sorted data
public T[] revert(T[] data){
return sortUsing(data, reverted);
}
private T[] sortUsing(T[] data, int[] ind){
if(data.length != indices.length){
throw new IllegalArgumentException(
String.format("Data length does not match: (%s, should be: %s) "
, data.length, indices.length));
}
// create a copy of the data (efficively this just creates a new array)
T[] sorted = data.clone();
// fill the copy with the sorted data
IntStream.range(0, ind.length)
.forEach(i -> sorted[i]=data[ind[i]]);
return sorted;
}
}
class App {
public static void main(String args[]){
IndexSorter<String> sorter = new IndexSorter<>(args, String::compareTo);
String[] data = sorter.sort(args);
System.out.println(Arrays.toString(data));
data = sorter.revert(data);
System.out.println(Arrays.toString(data));
data = IntStream.range(0, data.length)
.mapToObj(Integer::toString)
.toArray(String[]::new);
data = sorter.sort(data);
System.out.println(Arrays.toString(data));
data = sorter.revert(data);
System.out.println(Arrays.toString(data));
}
}

I would not recommend copying data. Because this is a memory allocation that can be quite expensive. It is much more efficient to sort data in place with library methods, like as Arrays.sort

I've found a way to sort in place, using a BitSet to keep track of what indexes are having the right element. It is in the method sortUsing. I hope someone will have a use for this algorithm.
You can test it like this:
java App this is just some random test to show the result
Then outcome will first show you the sorted result, than the reverted result.
The same index array is also used for ordering an int array of indexes, and the reverted version:
[is, just, random, result, show, some, test, the, this, to]
[this, is, just, some, random, test, to, show, the, result]
[1, 2, 4, 9, 7, 3, 5, 8, 0, 6]
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
Here is the code:
import java.util.stream.*;
import java.util.*;
public class IndexSorter<T> {
private final int[] indices;
private final int[] reverted;
private final BitSet done;
public IndexSorter(T[] data, Comparator<T> comparator){
// generate index array based on initial data and a comparator:
indices = IntStream.range(0, data.length)
.boxed()
.sorted( (a, b) -> comparator.compare(data[a],data[b]))
.mapToInt(a -> a)
.toArray();
// also create an index array to be able to revert the sort
reverted = new int[indices.length];
for(int i=0;i<indices.length;i++){
reverted[indices[i]] = i;
}
done = new BitSet(data.length);
}
// sort new data based on initial array
public void sort(T[] data){
sortUsing(data, indices);
}
// revert sorted data
public void revert(T[] data){
sortUsing(data, reverted);
}
private void sortUsing(T[] data, int[] ind){
if(data.length != indices.length){
throw new IllegalArgumentException(
String.format("Data length does not match: (%s, should be: %s) "
, data.length, indices.length));
}
int ia=0, ib=0, x = 0;
T a = null, b = null;
for (int i=0; i< data.length && done.cardinality()<data.length; i++){
ia = i;
ib = ind[ia];
if(done.get(ia)){ // index is already done
continue;
}
if(ia==ib){ // element is at the right place
done.set(ia);
continue;
}
x = ia; // start a loop at x = ia
// some next index will be x again eventually
a = data[ia]; // keep element a as the last value after the loop
while(ib!=x && !done.get(ia) ){
b = data[ib]; // element from index b must go to index a
data[ia]=b;
done.set(ia);
ia = ib;
ib = ind[ia]; // get next index
}
data[ia]=a; // set value a to last index
done.set(ia);
}
done.clear();
}
}
class App {
public static void main(String args[]){
IndexSorter<String> sorter = new IndexSorter<>(args, String::compareTo);
sorter.sort(args);
System.out.println(Arrays.toString(args));
sorter.revert(args);
System.out.println(Arrays.toString(args));
String[] data = IntStream.range(0, args.length)
.mapToObj(Integer::toString)
.toArray(String[]::new);
sorter.sort(data);
System.out.println(Arrays.toString(data));
sorter.revert(data);
System.out.println(Arrays.toString(data));
}
}

Initializing large number of arrays in Java?

I am writing an program for an engineering application that needs a large number of 1x3 arrays to store coefficients for a very long rigid body mechanics equation. As such, I have about 90 arrays that need to conform to a very specific and longstanding engineering syntax. Imagine I have the following:
double[] a = {0.0, 0.0, 0.0}
double[] b = {0.0, 0.0, 0.0}
double[] c = {0.0, 0.0, 0.0}
etc.......
double[] zz = {0.0, 0.0, 0.0}
You can see that this is quite unwieldy.
As I understand from reading this discussion on primitive declaration in Java, there is no way to initialize a list of variables to the same value. For instance, in the following code:
double[] a, b, c, ...... zy, zz = new double[3];
However, this only initializes array zz, is there really no better way to initialize a large number of arrays than with the first method in this post? Is there a better overall approach to storing a large number of arrays that need to be descriptively named? I would like to be able to store the data in a two dimensional array but unfortunately these values need to be descriptive.
For reference, I need to initialize these arrays because the data that is being assigned to them originates from a list that is passed to the object instance when it is constructed. I can't assign a value to an uninitialized variable or I get a NullPointerException. For example:
a[index] = listWithDataIWantToAssign.get(listIndex);
returns a NullPointerException when using the second (invalid) method of initialization mentioned above.

If you have to work with a large ammount of arrays it's preferable to use a double array instead of excessive ammount of variables. I recommand you to create a simple double array double[][] myArray = new double[90][3]
Is there a better overall approach to storing a large number of arrays that need to be descriptively named?
if all variables are named it will be difficult to not type all names. But instead of name variables why not name values it by ints using static names ?
you can create a class keeping all names like that :
public static class VariablesNames{
private static int counter = 0;
public static final int NAME_1 = counter++;
public static final int NAME_2 = counter++;
public static final int NAME_3 = counter++;
...
}
Then use these variables in your code
double val = myArray[VariablesNames.NAME_1][0];
is there really no better way to initialize a large number of arrays than with the first method in this post?
If you have a really large ammout of arrays, it's not really usefull to initialise all arrays at the beginning. You can lazy load the array and use getter an setter on it
double[][] myArray = new double[90][];
public void init(int position){
if(myArray[position] == null){
myArray[position] = new double[3];
}
}
public double[] get(int position) {
init(position);
return myArray[position];
}
public void set(int position, List<Double> listWithDataIWantToAssign){
init(position);
for (int i = 0; i < 3; i++) {
myArray[position][i] = listWithDataIWantToAssign.get(i);
}
}
Then use these methods in your code
set(VariablesNames.NAME_2, listWithDataIWantToAssign);
double val = get(VariablesNames.NAME_2)[1];

The following code will return a 2d array of 'double' primitives, with size [N][3]. N is the number of arrays, or 90 in your example, and 3 the three double numbers in each array:
public static double[][] initDoubleArrays (int N) {
double[][] doublesArray = new double[N][3];
// the doubles will be set to 0.0 by default.
return doublesArray;
}
Alternatively, the following code will return a list where each element is a Double[] array with three Double objects
public static List<Double[]> initArrays (int nArrays) {
List<Double[]> arrays = new ArrayList<Double[]>(nArrays);
for (int i = 0; i<nArrays; i++) {
Double[] doubles = { 0.0, 0.0, 0.0 };
arrays.add(doubles);
}
return arrays;
}

Since a List couldn't be the best approach you should consider creating your own Factory, improved a little by maybe playing with the size, and the initial value...
like:
public class MyClass implements IArrayFactory {
public static void main(String[] args) {
MyClass foo = new MyClass();
List<List<Double>> l = new ArrayList<>();
for (int i = 0; i < 10; i++) {
l.add(foo.getListDouble(3, 0.0));
}
System.out.println(l);
}
#Override
public List<Double> getListDouble(int size, double initVal) {
Double[] d = new Double[size];
Arrays.fill(d, initVal);
return Arrays.asList(d);
}
}
interface IArrayFactory {
List<Double> getListDouble(int size, double initVal);
}

Filter TreeMap with integer array key and perform arithmetic operations

I write a complex program in Java where I need to manage many large sparse matrix that contain integer values.
Example:
int A[][][][];
int B[][][];
int C[][];
In order to saving memory, space, etc, I decided to store this data in TreeMap. I know that exist a lot of libraries that do it better but I would try to implement my personal solution.
First of all I created a class Index that identifies the indeces of this array without knowing how many they are.
public class Index implements Comparable<Index> {
private final int[] index;
public Index(int ... index) {
this.index = index;
}
#Override
public int toCompare(Index i) {
...
}
}
Obviously I've used this class to define my three Maps in this way:
Map <Index, Integer> mapA = new TreeMap <> ();
Map <Index, Integer> mapB = new TreeMap <> ();
Map <Index, Integer> mapC = new ...
Now, I need to store the data from the matrix that I previous create to those Maps. The most easy way is use 4/3/2/.. loops, but I accept with pleasure others solutions.
for(int s = 0; s < s_max; s++) {
for(int c = 0; c < c_max; c++) {
for(int o = 0; o < o_max; o++) {
for(int d = 0; d < d_max; d++) {
mapA.put(new Index(s,c,o,d), A[s][c][o][d]);
}
}
}
}
The focus of the problem is when I need to retrieve the data, but let's me explain. The more usually operations between those maps are multiplications and additions considering them as matrix or arrays. I can't make 6(..) nested loops every times that I need to perform operations likes this below (I can't insert images because my reputation is still low):
http://postimg.org/image/7m8v0kiwn/
My questions are:
How can I filter my values from keys?
There is a way (maybe using lambda expression) to perform this type of operations?
Any advices about this problem?

Note that you don’t need to create an Index class; the type you need already exists.
From the documentation of IntBuffer.compareTo:
Two int buffers are compared by comparing their sequences of remaining elements lexicographically, without regard to the starting position of each sequence within its corresponding buffer.
So if you wrap integer array holding the indices and keep the position and limit at their defaults, the buffers will provide the desired behavior.
Performing the array to map conversion in a general way works, if you consider that in Java, each multi-dimensional array is also an instance of Object[], regardless of their base type. I.e., the type int[][] is a subtype of Object[] whose elements are instances of int[] which is a subtype of Object. Further, Object[][] is a subtype of Object[] whose elements are of type Object[] which is a subtype of Object.
Therefore you can process all dimensions the same way as Object[] to traverse it and only have to care about the actual type at the last dimension, which is always int[] for any n-dimensional int array:
/** accepts all int arrays from one to 255 dimensions */
public static TreeMap<IntBuffer,Integer> toMap(Object array) {
int dim=1;
for(Class<?> cl=array.getClass(); ; cl=cl.getComponentType())
if(Object[].class.isAssignableFrom(cl)) dim++;
else if(cl==int[].class) break;
else throw new IllegalArgumentException(array.getClass().getSimpleName());
TreeMap<IntBuffer,Integer> map=new TreeMap<>();
fill(map, new int[dim], 0, array);
return map;
}
private static void fill(TreeMap<IntBuffer, Integer> map, int[] i, int ix, Object array) {
int next=ix+1;
i[ix]=0;
if(next<i.length)
for(Object part: (Object[])array) {
if(part!=null) fill(map, i, next, part);
i[ix]++;
}
else
for(int val: (int[])array) {
if(val!=0) map.put(IntBuffer.wrap(i.clone()), val);
i[ix]++;
}
}
This solution handles the dimensions recursively though a non-recursive one is in principle possible. However, the recursion depth is naturally limited to the number of dimensions of the array/matrix.
It will put only non-zero values into the map and also skip any null sub-array, thus it doesn’t mind if the array representation is already sparse.
An example usage might look like
int[][][][] array=new int[5][5][5][5];
array[1][2][3][4]=42;
array[4][3][2][1]=1234;
TreeMap<IntBuffer, Integer> sparse=toMap(array);
sparse.forEach((index,value)-> {
for(int ix:index.array()) System.out.print("["+ix+']');
System.out.println("="+value);
});
which will print:
[1][2][3][4]=42
[4][3][2][1]=1234
Another example:
int[][] id = {
{1},
{0, 1},
{0, 0, 1},
{0, 0, 0, 1},
{0, 0, 0, 0, 1},
{0, 0, 0, 0, 0, 1},
{0, 0, 0, 0, 0, 0, 1},
};
TreeMap<IntBuffer, Integer> idAsMap = toMap(id);
System.out.println(idAsMap.size()+" non-zero values");
idAsMap.forEach((index,value)-> {
for(int ix:index.array()) System.out.print("["+ix+']');
System.out.println("="+value);
});
will print:
7 non-zero values
[0][0]=1
[1][1]=1
[2][2]=1
[3][3]=1
[4][4]=1
[5][5]=1
[6][6]=1

Index related sorting problem in Java

This is the requirement where I am facing problem finding the solution.
Problem:
I have ArrayList with data 20, 10, 30, 50, 40, 10.
If we sort this in ascending order the result will be 10, 10, 20, 30, 40, 50.
But I need the result as 3, 1, 4, 6, 5, 2.(The index of each element after sorting).
Strictly this should work even if there are repetitive elements in the list.
Please share your idea/approach solving this problem.

Here is my solution. We define a comparator to sort a list of indices based on the corresponding object in the list. That gives us a list of indices which is effectively a map: indices[i] = x means that the element at location x in the original list is at element i in the sorted list. We can then create a reverse mapping easily enough.
Output is the indices starting from 0: [2, 0, 3, 5, 4, 1]
import java.util.ArrayList;
import java.util.Collections;
import java.util.Comparator;
class LookupComparator<T extends Comparable<T>> implements Comparator<Integer> {
private ArrayList<T> _table;
public LookupComparator(ArrayList<T> table) {
_table = table;
}
public int compare(Integer o1, Integer o2) {
return _table.get(o1).compareTo(_table.get(o2));
}
}
public class Test {
public static <T extends Comparable<T>> ArrayList<Integer> indicesIfSorted(ArrayList<T> list) {
ArrayList<Integer> indices = new ArrayList<Integer>();
for (int i = 0; i < list.size(); i++)
indices.add(i);
Collections.sort(indices, new LookupComparator(list));
ArrayList<Integer> finalResult = new ArrayList<Integer>();
for (int i = 0; i < list.size(); i++)
finalResult.add(0);
for (int i = 0; i < list.size(); i++)
finalResult.set(indices.get(i), i);
return finalResult;
}
public static void main(String[] args) {
ArrayList<Integer> list = new ArrayList<Integer>();
list.add(20);
list.add(10);
list.add(30);
list.add(50);
list.add(40);
list.add(10);
ArrayList<Integer> indices = indicesIfSorted(list);
System.out.println(indices);
}
}

My idea is creating 1 more attribute call index beside your value (in each data of aray). It will hold your old index, then u can take it out for using.

Building off what Hury said, I think the easiest way I can see to do this is to make a new data type that looks something like:
public class Foo {
private Integer value;
private int origPosition;
private int sortedPosition;
/*Constructors, getters, setters, etc... */
}
And some psuedo code for what to do with it...
private void printSortIndexes(ArrayList<Integer> integerList) {
// Create an ArrayList<Foo> from integerList - O(n)
// Iterate over the list setting the origPosition on each item - O(n)
// Sort the list based on value
// Iterate over the list setting the sortedPosition on each item - O(n)
// Resort the list based on origPositon
// Iterate over the lsit and print the sortedPositon - O(n)
}
That won't take long to implement, but it is horribly inefficient. You are throwing in an extra 4 O(n) operations, and each time you add or remove anything from your list, all the positions stored in the objects are invalidated - so you'd have to recaculate everything. Also it requires you to sort the list twice.
So if this is a one time little problem with a small-ish data set it will work, but if you trying to make something to use for a long time, you might want to try to think of a more elegant way to do it.

Here is the approach of adding an index to each element, written out in Scala. This approach makes the most sense.
list.zipWithIndex.sortBy{ case (elem, index) => elem }
.map{ case (elem, index) => index }
In Java you would need to create a new object that implements comperable.
class IndexedItem implements Comparable<IndexedItem> {
int index;
int item;
public int compareTo(IndexItem other) {
return this.item - other.item;
}
}
You could then build a list of IndexedItems, sort it with Collection.sort, and then pull out the indices.
You could also use Collections.sort on the original list followed by calls to indexOf.
for (int elem : originalList) {
int index = newList.indexOf(elem);
newList.get(index) = -1; // or some other value that won't be found in the list
indices.add(index);
}
This would be very slow (all the scans of indexOf), but would get the job done if you only need to do it a few times.

A simplistic approach would be to sort the list; then loop on the original list, find the index of the element in the sorted list and insert that into another list.
so a method like
public List<Integer> giveSortIndexes(List<Integer> origList) {
List<Integer> retValue = new ArrayList<Integer>();
List<Integer> originalList = new ArrayList<Integer>(origList);
Collections.sort(origList);
Map<Integer, Integer> duplicates = new HashMap<Integer, Integer>();
for (Integer i : originalList) {
if(!duplicates.containsKey(i)) {
retValue.add(origList.indexOf(i) + 1);
duplicates.put(i, 1);
} else {
Integer currCount = duplicates.get(i);
retValue.add(origList.indexOf(i) + 1 + currCount);
duplicates.put(i, currCount + 1);
}
}
return retValue;
}
I haven't tested the code and it might need some more handling for duplicates.

The best way to transform int[] to List<Integer> in Java? [duplicate]

This question already has answers here:
Closed 11 years ago.
Possible Duplicate:
How to create ArrayList (ArrayList<T>) from array (T[]) in Java
How to implement this method:
List<Integer> toList(int[] integers) {
???
//return Arrays.asList(integers); doesn't work
}

There's probably a built-in method to do it somewhere* (as you note, Arrays.asList won't work as it expects an Integer[] rather than an int[]).
I don't know the Java libraries well enough to tell you where that is. But writing your own is quite simple:
public static List<Integer> createList(int[] array) {
List<Integer> list = new ArrayList<Integer>(array.length);
for (int i = 0; i < array.length; ++i) {
list.add(array[i]);
}
return list;
}
Obviously one downside of this is that you can't do it generically. You'll have to write a separate createList method for each autoboxed primitive type you want.
*And if there isn't, I really wonder why not.

Use commons-lang3 org.apache.commons.lang3.ArrayUtils.toObject(<yout int array>) and then java.util.Arrays.asList(<>)
ArrayUtils.toObject() will copy the array, and Array.asList() will simply create list that is backed by new array.
int[] a = {1, 2, 3};
List<Integer> aI = Arrays.asList(ArrayUtils.toObject(a));
EDIT: This wont work if you want to add() new elements (resize) though the list interface, if you want to be able to add new elements, you can use new ArrayList(), but this will create one more copy.

List<Integer> asList(final int[] integers) {
return new AbstractList<Integer>() {
public Integer get(int index) {
return integers[index];
}
public int size() {
return integers.length;
}
};
}

List<Integer> toList(int[] integers) {
// Initialize result's size to length of the incoming array
// this way it will not require reallocations
ArrayList<Integer> result = new ArrayList<Integer>( integers.length );
for ( int cur: integers )
{
result.add( Integer.valueOf( cur ) );
}
return result;
}

I do not think there is a quick way to do it unfortunately. I believe you will have to iterate the array and add it one by one.

import java.util.Arrays;
import java.util.ArrayList;
import java.util.List;
public class Listing {
public static void main(String[] args) {
int[] integers = {1,2,3,4};
java.util.List<Integer> list = new ArrayList<Integer>();
for (int i=0; i< integers.length; i++)
{
list.add(integers[i]);
}
System.out.println(list);
}
}
Tested and working as expected!