We Keep Coding

I'm trying to fill an array with integers by iterating, but I'm getting an ArrayIndexOutOfBoundsException

I'm trying to fill an array with integers using this code:
int[] steps = new int[1000001];
steps[0] = 0;
steps[1] = 1;
steps[2] = 2;
for(int i = 1001; i < steps.length; i++){
if(steps[i]==0){
steps[i] = steps[i-1]+1;
}
int current = i;
for(int m = current; m > 1; m--){
int mult = current*m;
if(mult<steps.length){
int suma = steps[current]+1;
if(steps[mult]==0){
steps[mult] = suma;
}
if(suma<steps[mult]){
steps[mult] = suma;
}
}
}
}
Scanner scan = new Scanner(System.in);
int n = scan.nextInt();
for(int k = 0; k < n; k++){
int q = scan.nextInt();
System.out.println(steps[q]);
}
scan.close();
And I'm getting this:
Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: -2147479015
at javaapplication6.JavaApplication6.main(JavaApplication6.java:26)
C:\Users\User\AppData\Local\NetBeans\Cache\8.2\executor-snippets\run.xml:53: Java returned: 1
BUILD FAILED (total time: 1 second)
But I can't see why this is happening. I understand that such an exception occurs when you try to access an index that doesn't exist. I've checked many times now my code and I haven't been able to find the issue.
Do you know what does this number mean? Because it is confusing me:
Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: -2147479015
NetBeans says the problem is at:
javaapplication6.JavaApplication6.main(JavaApplication6.java:26)
Line 26 is this one:
if(steps[mult]==0)
I think what that is saying is, that at some point, the program is trying to access a nonexistent index of the array. The thing is that I don't understand how that could happen here.
I put this line before to avoid that, but it seemed not to work:
if(mult<steps.length)
Then I modified that line by changing it to this one and it seems to work. It is not showing the exception anymore:
if(mult<steps.length && mult >=0)
Now the problem is that apparently, it does not get into this part of the code:
Scanner scan = new Scanner(System.in);
int n = scan.nextInt();
for(int k = 0; k < n; k++){
int q = scan.nextInt();
System.out.println(steps[q]);
}
scan.close();
I want to understand first, why that exception is occurring and why it is getting "solved" by adding that other part of the code, and second, why my code seems to get stuck in my first for loop.
I really would appreciate if you could please help me to understand those things.
Thanks in advance.

In your code you have
int mult = current*m;
where current and m is the size of the steps array
so 1000001 * 1000001 is going to exceed the size of the array and also exceed the max int value
I am not sure what logic you are wanting to do by doing
if(steps[mult]==0){
You problem can be shown with this code
int val = 1000000;
for (int i = 0; i < 100; i++) {
val = val * val;
if (val < 0) {
System.out.println(val);
}
}

Loop twice but cost same time in JAVA

I have a loop like this:
double[][] ED = new double[n][m];
for(int k = 0; k < 20000; k++)
for (int i = 0; i < 200; i++)
for (int j = 0; j < 200; j++)
ED[i][j] = dis(qx[k][i], qy[k][i], dx[k][j], dy[k][j]);
"dis" is a function to calculate the distance between (x1,y1) and (x2,y2). Don't mind it. The problem is when I add another boolean assignment in the loop just like this:
double[][] ED = new double[n][m];
boolean[][] bool = new boolean[n][m];
for(int k = 0; k < 20000; k++)
for (int i = 0; i < 200; i++)
for (int j = 0; j < 200; j++)
{
ED[i][j] = dis(qx[k][i], qy[k][i], dx[k][j], dy[k][j]);
bool[i][j] = ED[i][j] > 5000;
}
The new loop cost 1.5 time over the first one. I think it cost too much. For testing, I break 2 assignment into 2 loop.The strange thing happens, two cost of time are same. Sometimes, code 3 cost less time than code 2
double[][] ED = new double[n][m];
boolean[][] bool = new boolean[n][m];
for(int k = 0; k < 20000; k++)
{
for (int i = 0; i < 200; i++)
for (int j = 0; j < 200; j++)
{
ED[i][j] = dis(qx[k][i], qy[k][i], dx[k][j], dy[k][j]);
}
for (int i = 0; i < 200; i++)
for (int j = 0; j < 200; j++)
{
bool[i][j] = ED[i][j] > 5000;
}
}
My aim is use as less time too calculate bool[i][j], how should I do.

Introducing new, big array bool[][] may have more impact than it seems.
When only single arrayED[i][j] is used, you put less stress on L1 processor cache.
With second array, you have twice as much data, therefore cache will be invalidated more often.
Could you try, instead of using two arrays (bool and arrayED) use single array that holds both double and boolean? There will be significant overhead for array of Objects, but (maybe) compiler will be smart enough to destructure the Object.
With single array, you will have better data locality.
Also, as suggested in comments make sure you do your microbenchmarking properly.
Use http://openjdk.java.net/projects/code-tools/jmh/ and read its documentation how to use it correctly.
Another soltution that would help on multicore system is to use parallel processing. You may create ThreadPoolExecutor (with pool size equal to number of cores you have) then submit each opearation as task to the executor. Operation may be inner-most loop (with counter j) or even two inner-most loops (with counters i and j).
There will be some overhead for coordinating the work but execution time should be much faster if you have 4 or 8 cores.
Yet another idea is to change input data structure. Instead of operating on 4 two-dimensional arrays (qx,qy,dx,dy) could you have single array? it may make dis function faster.

Two smallest numbers in a set

I am trying to find the two smallest numbers in a set WITHOUT USING ARRAYS. Here is the code:
Scanner in = new Scanner(System.in);
int N = in.nextInt();
int min = in.nextInt();
for(int i = 1; i < N; i++){
int a = in.nextInt();
if(a < min){
min = a;
}
}
System.out.println(min);
It find the smallest number but there is nothing about the second smallest number.
How do I do that?
Please, note that I am a complete beginner to Java so easy explanation and help will be much appreciated)

It´s very very easy:
Scanner in= new Scanner(System.in);
int N = in.nextInt();
int min,min2 = Integer.MAX_VALUE,Integer.MAX_VALUE;
for(int i = 0; i < N; i++){
int a = in.nextInt();
if( a < min){
min = a;
min2 = min;
}
else if( a < min2){
min2 = a;
}
}
System.out.println(min);
System.out.println(min2);

It is about one condition you have to add:
Scanner in = new Scanner(System.in);
int N = in.nextInt();
int min = Integer.MAX_VALUE;
int secondMin = Integer.MAX_VALUE;
for(int i = 0; i < N; i++){
int a = in.nextInt();
if(a < min){
secondMin = min; // the current minimum must be the second smallest
min = a; // allocates the new minimum
}
else if (a < secondMin) {
secondMin = a; // if we did not get a new minimum, it may still be the second smallest number
}
}
System.out.println(min);
System.out.println(secondMin);

General hint: You should call the close method of your Scanner, preferably in a try-with-ressources block:
try(Scanner in = new Scanner(System.in)) {
// other code here
}
That way the stream gets closed, which you should do, if you open a stream.
Solution 1:
The easiest way, that uses your existing code, would be also tracking the second smallest number:
Scanner in = new Scanner(System.in);
int N = in.nextInt();
int min = in.nextInt();
int sMin = Integer.MAX_VALUE;
for(int i = 1; i < N; i++){
int a = in.nextInt();
if(a < min){
sMin = min;
min = a;
} else if(a < sMin) {
sMin = a;
}
}
System.out.println(min);
System.out.println(sMin);
Explanation 1:
The two cases, that can occure with a new Value are:
The new value is smaller than min and sMin. Then you have to set the value of min into smin and afterwards set min to the new min value.
The new value is larger than min and smaller than sMin. Then you only have to set the value of sMin to the new value.
Both min-values are smaller. Then nothing is to do.
Solution 2:
Another, more generic approach would be using a PriorityQueue:
int N = in.nextInt();
PriorityQueue<Integer> minQueue = new PriorityQueue<>();
for(int i = 0; i < N; i++) {
int value = in.nextInt();
minQueue.add(value);
}
int minValue = minQueue.poll();
int secondMinValue = minQueue.poll();
This way you can get the n smallest numbers given by using a loop in which you call the poll() method. (n may be a number < N).
Explanation 2:
The Priority Queue is a datastructure, that internally orders the given elements by the natural order. In the case of Integers this order is given by <,> and =. So when calling poll() you remove the smallest element, that the PriorityQueue has yet encountered.

Try this:
Scanner in = new Scanner(System.in);
int n = in.nextInt();
int min2 = in.nextInt();
int min1 = min2;
for(int i = 1; i < n; i++){
int a = in.nextInt();
if( a < min2){
if(a < min1){
min2 = min1;
min1 = a;
}
else{
min2 = a;
}
}
}
System.out.println(min1 + " " + min2);

This problem can be solved in multiple ways, and the first step in choosing the right solution is to decide what is the most important for you:
Space efficiency, that is, use the minimum possible amount of storage. ThreeFx does this, (s)he only uses constant additional space for the variables N, a, min, and secondMin. "Constant space" means that the amount of data that you store does not depend on how many numbers you are going to read from the stream. In contrast, Tarlen uses linear space, storing all the numbers read from the stream. This means that the amount of space required is directly proportional to N instead of being constant.
Time efficiency, that is, perform as few computations as possible. I believe that ThreeFx's solution is one of the most efficient from this point of view. Tarlen's solution will be a bit slower, because managing the priority queue might require more comparisons.
Extensibility, that is, how easily you could adapt your code when the requirements change slightly. Say that your boss makes you solve the problem that you just posted, to find the two smallest numbers. The next day, he wants the first three, and so on, until the end of the week. You get tired of changing your code every day, so you write a general solution, that will work for any number of elements that he asks for. This is where Tarlen's solution is better.
Readability, that is, how short and easy to understand your code is. I will introduce my own solution here, which is based on a simple idea: put all the numbers in a list, sort it, and take out the first two numbers. Note that this is quite wasteful when it comes to resources: the space is linear (I am storing N numbers), and the time efficiency is O(N log N) at best. Here is my code:
List<Integer> list = new ArrayList<Integer>();
for (int i = 0; i < N; i++) list.add(in.nextInt());
Collections.sort(list);
System.out.println(list.get(0));
System.out.println(list.get(1));

Floyd Warshall Algorithm for Shortest Path

I was looking through some old contest questions, and I found this one, it looked fun, http://dwite.ca/old/Problem5Jan2006.pdf , I tried using the floyd warshall algorithm to get the shortest path from any node to any other node, can you guys see what I did wrong? it does not give the desired output set out on the contest question page
import java.io.*;
import java.util.*;
public class DistanceBetween {
public static void main(String[] args) throws FileNotFoundException {
Scanner s = new Scanner(new File("DATA5.txt"));
int n = Integer.parseInt(s.nextLine());
int[][] dist = new int[60][60];
for(int y=0;y<60;++y)for(int x=0;x<60;++x)dist[y][x]=10000000;
Map<Character, Integer> map = new TreeMap<Character, Integer>();
for (int i = 0; i < n; ++i) {
String text[] = s.nextLine().split(" ");
int c = 0;
if (!map.containsKey(text[0].charAt(0))) {
map.put(text[0].charAt(0), c);
c++;
}
if (!map.containsKey(text[0].charAt(1))) {
map.put(text[0].charAt(1), c);
c++;
}
dist[map.get(text[0].charAt(0))][map.get(text[0].charAt(1))] = Integer.parseInt(text[1]);
}
for (int k = 0; k < map.size(); ++k) {
for (int i = 0; i < map.size(); ++i) {
for (int j = 0; j < map.size(); ++j) {
dist[i][j] = Math.min(dist[i][j], dist[i][k] + dist[k][j]);
}
}
}
for (int i = 0; i < 5; ++i) {
String text = s.nextLine();
System.out.println(dist[map.get(text.charAt(0))][map.get(text.charAt(1))]);
}
}}

There are several problems in your code:
Overwritten mapping
Your int c is local variable of the for cycle which means the highest used mapping index doesn't survive to the next iteration, so the reading in next iteration overrides the previous one. So the distance matrix is not properly filled after data loading.
Solution: move the int c = 0; outside from the for loop.
Unidirectional roads
The roads are bidirectional in the instructions, but you register them only as unidirectional. As the consequence of that are higher on non-existent connections between towns.
Solution: add dist[map.get(text[0].charAt(1))][map.get(text[0].charAt(0))] = Integer.parseInt(text[1]); right after the similar one.
Besides these hard issues I have also couple hints for you. You do not have follow them but as if you want to improve your programming skills then you should think about them.
Messy code
Your code is hard to read, there are multiple restated information such as indicies, the solving process is in the single method etc. Such code is not only hard to read but also extremely hard to debug and fix. For your own good I recommend you to write it cleaner.
Algorithm efficiency
Floyd-Warshall's algorithm has a O(n^3) complexity. The size of problem (amount of towns) is A-M = 13. In this complexity it makes 13^3 = 2197 iterations. I know, it might not seem to be a lot, but consider the amount of tasks to solve in a given time limit.
I would recommend you to use Dijkstra's algorithm which has complexity O(|E| + |V|log|V|). In this task the worst case with some simplification is |E| = (|V|^2)/2, |V|=13. It means, that the final number of iterations is 5 (|V|^2 / 2 + |V|log|V|) = 5 (13^2 / 2 + 13 * log13) ~ 5 * 132 = 660. If I am not wrong and made any mistake, this is significantly less, especially when we consider the total amount of tasks.
Input reading
I might be wrong but I attended multiple programming contests and competitions and it never forced attendees to work with files. An input was always redirected from files to a standard input. I guess, that the main reason for this is a security, but the simplification is probably also highly beneficial.

Well that question I got, I am starting to do SPOJ now, and I gotta admit it is pretty difficult later on, but I came across the same kind of question http://www.spoj.com/problems/SHPATH/ , I also used Floyd Warshall
import java.util.*;
public class Floydwarshall {
public static void main(String[] args) {
Scanner s = new Scanner(System.in);
String q = s.nextLine();
for(int t=0;t<Integer.parseInt(q);++t){
int n = Integer.parseInt(s.nextLine());
int[][] cost = new int[n][n];
for (int y = 0; y < n; ++y) {
for (int x = 0; x < n; ++x) {
cost[x][y] = 10000;
}
}
Map<String, Integer> map = new TreeMap<String, Integer>();
int c = 0;
for (int i = 0; i < n; ++i) {
String a = s.nextLine();
if (!map.containsKey(a)) {
map.put(a, c);
c++;
}
int f = Integer.parseInt(s.nextLine());
for (int j = 0; j < f; ++j) {
String text[] = s.nextLine().split(" ");
cost[map.get(a)][Integer.parseInt(text[0]) - 1] =
cost[Integer.parseInt(text[0]) - 1][map.get(a)] = Integer.parseInt(text[1]);
}
}
for (int k = 0; k < map.size(); ++k) {
for (int i = 0; i < map.size(); ++i) {
for (int j = 0; j < map.size(); ++j) {
cost[i][j] = Math.min(cost[i][j], cost[i][k] + cost[k][j]);
}
}
}
int num = Integer.parseInt(s.nextLine());
for (int i = 0; i < num; ++i) {
String text[] = s.nextLine().split(" ");
System.out.println(cost[map.get(text[0])][map.get(text[1])]);
}
}
}}
now it runs alright for the sample input, but when I hand it in, it gives me this
NZEC (non-zero exit code) - this message means that the program exited returning a value different from 0 to the shell. For languages such as C, this probably means you forgot to add "return 0" at the end of the program. For interpreted languages (including JAVA) NZEC will usually mean that your program either crashed or raised an uncaught exception.
Problem is I cannot kind where it crashes or raises an uncaught exception since it
works with the sample input

Neural Network Backpropagation does not compute weights correctly

Currently, I am having problems with the Backpropagation algorithm.
I am trying to implement it and use it to recognize the direction of faces (left, right, down, straight).
Basically, I have N images, read the pixels and change its values(0 to 255) to values from 0.0 to 1.0. All images are 32*30.
I have an input layer of 960 neurons, a hidden layer of 3 neurons and an output layer of 4 neurons. For example, the output <0.1,0.9,0.1,0.1> means that the person looks to the right.
I followed the pseudy-code. However, it doesn't work right - it does not compute the correct weights and consequently it can't handle the training and test examples.
Here are parts of the code:
// main function - it runs the algorithm
private void runBackpropagationAlgorithm() {
for (int i = 0; i < 900; ++i) {
for (ImageUnit iu : images) {
double [] error = calcOutputError(iu.getRatioMatrix(), iu.getClassification());
changeHiddenUnitsOutWeights(error);
error = calcHiddenError(error);
changeHiddenUnitsInWeights(error,iu.getRatioMatrix());
}
}
}
// it creates the neural network
private void createNeuroneNetwork() {
Random generator = new Random();
for (int i = 0; i < inHiddenUnitsWeights.length; ++i) {
for (int j = 0; j < hiddenUnits; ++j) {
inHiddenUnitsWeights[i][j] = generator.nextDouble();
}
}
for (int i = 0; i < hiddenUnits; ++i) {
for (int j = 0; j < 4; ++j) {
outHddenUnitsWeights[i][j] = generator.nextDouble();
}
}
}
// Calculates the error in the network. It runs through the whole network.
private double [] calcOutputError(double[][] input, double [] expectedOutput) {
int currentEdge = 0;
Arrays.fill(hiddenUnitNodeValue, 0.0);
for (int i = 0; i < input.length; ++i) {
for (int j = 0; j < input[0].length; ++j) {
for (int k = 0; k < hiddenUnits; ++k) {
hiddenUnitNodeValue[k] += input[i][j] * inHiddenUnitsWeights[currentEdge][k];
}
++currentEdge;
}
}
double[] out = new double[4];
for (int j = 0; j < 4; ++j) {
for (int i = 0; i < hiddenUnits; ++i) {
out[j] += outHddenUnitsWeights[i][j] * hiddenUnitNodeValue[i];
}
}
double [] error = new double [4];
Arrays.fill(error, 4);
for (int i = 0; i < 4; ++i) {
error[i] = ((expectedOutput[i] - out[i])*(1.0-out[i])*out[i]);
//System.out.println((expectedOutput[i] - out[i]) + " " + expectedOutput[i] + " " + out[i]);
}
return error;
}
// Changes the weights of the outgoing edges of the hidden neurons
private void changeHiddenUnitsOutWeights(double [] error) {
for (int i = 0; i < hiddenUnits; ++i) {
for (int j = 0; j < 4; ++j) {
outHddenUnitsWeights[i][j] += learningRate*error[j]*hiddenUnitNodeValue[i];
}
}
}
// goes back to the hidden units to calculate their error.
private double [] calcHiddenError(double [] outputError) {
double [] error = new double[hiddenUnits];
for (int i = 0; i < hiddenUnits; ++i) {
double currentHiddenUnitErrorSum = 0.0;
for (int j = 0; j < 4; ++j) {
currentHiddenUnitErrorSum += outputError[j]*outHddenUnitsWeights[i][j];
}
error[i] = hiddenUnitNodeValue[i] * (1.0 - hiddenUnitNodeValue[i]) * currentHiddenUnitErrorSum;
}
return error;
}
// changes the weights of the incomming edges to the hidden neurons. input is the matrix of ratios
private void changeHiddenUnitsInWeights(double [] error, double[][] input) {
int currentEdge = 0;
for (int i = 0; i < input.length; ++i) {
for (int j = 0; j < input[0].length; ++j) {
for (int k = 0; k < hiddenUnits; ++k) {
inHiddenUnitsWeights[currentEdge][k] += learningRate*error[k]*input[i][j];
}
++currentEdge;
}
}
}
As the algorithm works, it computes bigger and bigger weights, which finally approach infinity (NaN values). I checked the code. Alas, I didn't manage to solve my problem.
I will be firmly grateful to anyone who would try to help me.

I didn't check all of your code. I just want to give you some general advices. I don't know if your goal is (1) to learn the direction of faces or (2) to implement your own neural network.
In case (1) you should consider one of those libraries. They just work and give you much more flexible configuration options. For example, standard backpropagation is one of the worst optimization algorithms for neural networks. The convergence depends on the learning rate. I can't see which value you chose in your implementation, but it could be too high. There are other optimization algorithms that don't require a learning rate or adapt it during training. In addition, 3 neurons in the hidden layer is most likely not enough. Most of the neural networks that have been used for images have hundreds and sometimes even thousands of hidden units. I would suggest you first try to solve your problem with a fully developed library. If it does work, try implementing your own ANN or be happy. :)
In case (2) you should first try to solve a simpler problem. Take a very simple artificial data set, then take a standard benchmark and then try it with your data. A good way to verify that your backpropagation implementation works is a comparison with a numerical differentation method.

Your code is missing the transfer functions. It sounds like you want the logistic function with a softmax output. You need to include the following in calcOutputError
// Logistic transfer function for hidden layer.
for (int k = 0; k < hiddenUnits; ++k) {
hiddenUnitNodeValue[k] = logistic(hiddenUnitNodeValue[k]);
}
and
// Softmax transfer function for output layer.
sum = 0;
for (int j = 0; j < 4; ++j) {
out[j] = logistic(out[j]);
sum += out[j];
}
for (int j = 0; j < 4; ++j) {
out[j] = out[j] / sum;
}
where the logistic function is
public double logistic(double x){
return (1/(1+(Math.exp(-x)));
}
Note that the softmax transfer function gives you outputs that sum to 1, so they can be interpreted as probabilities.
Also, your calculation of the error gradient for the output layer is incorrect. It should simply be
for (int i = 0; i < 4; ++i) {
error[i] = (expectedOutput[i] - out[i]);
}

I haven't tested your code but I am almost certain that you start out with to large weights.
Most of the introductions on the subjects leave it at "init the weights with random values" and leaving out that the algorithm actually diverges (goes to Inf) for some starting values.
Try using smaller starting values, for example between -1/5 and 1/5 and shrink it down.
And additionally do an method for matrix multiplication, you have (only) used that 4 times, much easier to see if there is some problem there.

I had a similar problem with a neural network processing grayscale images. You have 960 input values ranging between 0 and 255. Even with small initial weights, you can end up having inputs to your neurons with a very large magnitude and the backpropagation algorithm gets stuck.
Try dividing each pixel value by 255 before passing it into the neural network. That's what worked for me. Just starting with extremely small initial weights wasn't enough, I believe due to the floating-point precision issue brought up in the comments.
As suggested in another answer, a good way to test your algorithm is to see if your network can learn a simple function like XOR.
And for what it's worth, 3 neurons in the hidden layer was plenty for my purpose (identifying the gender of a facial image)

I wrote an entire new neural-network library and it works. It is sure that in my previous attempt I missed the idea of using transfer functions and their derivatives. Thank you, all!