I'm trying to find all connected components and their sizes in a graph. I don't know why, but the size is always 0. Maybe something is wrong in the method.
This is the problem that I am trying to solve. https://www.codechef.com/LRNDSA08/problems/FIRESC
public class B {
static void dfs(int s, int v, boolean[] visited, ArrayList<ArrayList<Integer>> adj) {
s++;
visited[v] = true;
for (int u : adj.get(v)) {
if (!visited[u]) {
dfs(s, u, visited, adj);
}
}
public static void main(String[] args) {
Scanner sc = new Scanner(System.in);
StringBuilder str = new StringBuilder();
int t = sc.nextInt();
for (int xx = 0; xx < t; xx++) {
int n = sc.nextInt();
int m = sc.nextInt();
ArrayList<ArrayList<Integer>> arr = new ArrayList<>();
for (int i = 0; i < n; i++) {
arr.add(new ArrayList<Integer>());
}
boolean[] visited = new boolean[n];
Arrays.fill(visited, false);
for (int i = 0; i < m; i++) {
int a = sc.nextInt();
int b = sc.nextInt();
a--;
b--;
arr.get(a).add(b);
arr.get(b).add(a);
}
long ways = 1;
int groups = 0;
for (int i = 0; i < n; i++) {
if (visited[i])
continue;
int size = 0;
dfs(size, i, visited, arr);
groups++;
ways *= size;
ways %= 1000000007;
}
System.out.println(groups + " " + ways);
}
}
}
You know size is passed as value and not as reference. So it won't get updated after you return from the call. One thing you could do is define a single element array like
int[] size = new int[1];
and modify your dfs like:
static void dfs(int[] s, int v, boolean[] visited, ArrayList<ArrayList<Integer>> adj) {
s[0]++;
visited[v] = true;
for (int u : adj.get(v)) {
if (!visited[u]) {
dfs(s, u, visited, adj);
}
}
}
Then your result will be in size[0] which you can use to update ways like ways *= size[0]
Or you could modify dfs to return size which is a cleaner way to get the size like below:
static int dfs(int v, boolean[] visited, ArrayList<ArrayList<Integer>> adj) {
visited[v] = true;
int sz = 1;
for (int u : adj.get(v)) {
if (!visited[u]) {
sz += dfs(u, visited, adj);
}
}
return sz;
}
And it seems like you have a misconception on how variables in Java work (see). Incrementing an int variable that resides on one lair of the stack would not affect a variable on another stack lair. That's why the size is always 0.
The following solution passes base test on CodeChef:
public class CountComponents {
public static void main(String[] args) {
Scanner sc = new Scanner(System.in);
int testCases = sc.nextInt();
for (int i = 0; i < testCases; i++) {
EmployeeGraph graph = parseGraph(sc);
graph.countComponentsAndComponentSizes();
}
}
public static EmployeeGraph parseGraph(Scanner sc) {
int employeeCount = sc.nextInt();
int connectionsCount = sc.nextInt();
boolean[][] adjacencyMatrix = new boolean[employeeCount][employeeCount];
for (int i = 0; i < connectionsCount; i++) {
int row = sc.nextInt() - 1;
int col = sc.nextInt() - 1;
adjacencyMatrix[row][col] = true;
adjacencyMatrix[col][row] = true;
}
return new EmployeeGraph(adjacencyMatrix);
}
}
class EmployeeGraph {
public static final int BILLION_SEVEN = 1_000_000_007;
private boolean[][] adjacencyMatrix;
public EmployeeGraph(boolean[][] adjacencyMatrix) {
this.adjacencyMatrix = adjacencyMatrix;
}
public void countComponentsAndComponentSizes() {
boolean[] visited = new boolean[adjacencyMatrix.length];
int componentCount = 0;
int waysToChooseCaptain = 1;
for (int row = 0; row < adjacencyMatrix.length; row++) {
if (!visited[row]) {
componentCount++;
waysToChooseCaptain = (waysToChooseCaptain % BILLION_SEVEN) * dfs(visited, row);
}
}
System.out.println(componentCount + " " + waysToChooseCaptain % BILLION_SEVEN);
}
public int dfs(boolean[] visited, int row) {
visited[row] = true; // marking the current employee as visited
int size = 1; // this component consists at least from 1 employee
for (int col = 0; col < adjacencyMatrix.length; col++) {
if (adjacencyMatrix[row][col] && !visited[col]) {
size += dfs(visited, col);
}
}
return size;
}
}
I have a JAVA program where I am creating graphs and I have a Breadth-First Search but I would like to change it to Depth First Search. What changes should I make in a code? Thanks for help in advance.
public class ConnectedComponents
{
static final int MAXV = 100;
static boolean processed[] = new boolean[MAXV];
static boolean discovered[] = new boolean[MAXV];
static int parent[] = new int[MAXV];
static void bfs(CCGraph g, int start)
{
Queue<Integer> q = new LinkedList<Integer>();
int i, v;
q.offer(start);
discovered[start] = true;
while (!q.isEmpty())
{
v = q.remove();
process_vertex(v);
processed[v] = true;
for (i = g.degree[v] - 1; i >= 0; i--)
{
if (!discovered[g.edges[v][i]])
{
q.offer(g.edges[v][i]);
discovered[g.edges[v][i]] = true;
parent[g.edges[v][i]] = v;
}
}
}
}
I think you should understand the difference between depth first search and breadth first search. The code for depth first search goes as follows:
public class ConnectedComponents
{
static final int MAXV = 100;
static boolean processed[] = new boolean[MAXV];
static boolean discovered[] = new boolean[MAXV];
static int parent[] = new int[MAXV];
static void dfs(CCGraph g, int vertex)
{
discovered[vertex] = true;
for (i = g.degree[vertex] - 1; i >= 0; i--)
{
if (!discovered[g.edges[vertex][i]])
{
parent[g.edges[v][i]]=vertex;
dfs(g.edges[v][i]]);
}
}
}
}
The basic difference is the order by which vertexes are tested. While BFS uses queue (FIFO: First In First Out), DFS use stack (LIFO: Last In First Out).
You could implement stack using LinkedList:
LinkedList<Integer> stack = new LinkedList<Integer>();
stack.pop(); //returns the top of the stack
For more information please post mcve including test data.
Full code of the program. The goal is to change bfs to dfs.
import java.util.LinkedList;
import java.util.Queue;
import java.util.Scanner;
class CCGraph
{
static final int MAXV = 100;
static final int MAXDEGREE = 50;
public int edges[][] = new int[MAXV + 1][MAXDEGREE];
public int degree[] = new int[MAXV + 1];
public int nvertices;
public int nedges;
CCGraph()
{
nvertices = nedges = 0;
for (int i = 1; i <= MAXV; i++)
degree[i] = 0;
}
void read_CCGraph(boolean directed)
{
int x, y;
Scanner sc = new Scanner(System.in);
System.out.println("Enter the number of vertices: ");
nvertices = sc.nextInt();
System.out.println("Enter the number of edges: ");
int m = sc.nextInt();
System.out.println("Enter the edges: <from> <to>");
for (int i = 1; i <= m; i++)
{
x = sc.nextInt();
y = sc.nextInt();
insert_edge(x, y, directed);
}
sc.close();
}
void insert_edge(int x, int y, boolean directed)
{
if (degree[x] > MAXDEGREE)
System.out.printf(
"Warning: insertion (%d, %d) exceeds max degree\n", x, y);
edges[x][degree[x]] = y;
degree[x]++;
if (!directed)
insert_edge(y, x, true);
else
nedges++;
}
void print_CCGraph()
{
for (int i = 1; i <= nvertices; i++)
{
System.out.printf("%d: ", i);
for (int j = degree[i] - 1; j >= 0; j--)
System.out.printf(" %d", edges[i][j]);
System.out.printf("\n");
}
}
}
public class ConnectedComponents
{
static final int MAXV = 100;
static boolean processed[] = new boolean[MAXV];
static boolean discovered[] = new boolean[MAXV];
static int parent[] = new int[MAXV];
static void bfs(CCGraph g, int start)
{
LinkedList<Integer> q = new LinkedList<Integer>();
int i, v;
q.offer(start);
discovered[start] = true;
while (!q.isEmpty())
{
v = q.remove();
process_vertex(v);
processed[v] = true;
for (i = g.degree[v] - 1; i >= 0; i--)
{
if (!discovered[g.edges[v][i]])
{
q.offer(g.edges[v][i]);
discovered[g.edges[v][i]] = true;
parent[g.edges[v][i]] = v;
}
}
}
}
static void initialize_search(CCGraph g)
{
for (int i = 1; i <= g.nvertices; i++)
{
processed[i] = discovered[i] = false;
parent[i] = -1;
}
}
static void process_vertex(int v)
{
System.out.printf(" %d", v);
}
static void connected_components(CCGraph g)
{
int c;
initialize_search(g);
c = 0;
for (int i = 1; i <= g.nvertices; i++)
{
if (!discovered[i])
{
c++;
System.out.printf("Component %d:", c);
bfs(g, i);
System.out.printf("\n");
}
}
}
static public void main(String[] args)
{
CCGraph g = new CCGraph();
g.read_CCGraph(false);
g.print_CCGraph();
connected_components(g);
}
}
So far I have tried to create the method below, but when I run it, the new array leaves zeros for the empty spaces. If a find all method is created to work with this how can it be implemented with a binary search instead of a linear search
package bp;
import java.util.Arrays;
public class SortedList implements IUnsortedList {
/**
* The max size of the List.
*/
public static final int MAX_SIZE = 10000;
/**
* The max value of each occurence.
*/
public static final int MAX_VALUE = 10;
/**
* Flag for the amount of items on the list.
*/
private int sizeOfList = 0;
/**
* Variable to define true or false for duplicates.
*/
private boolean duplicatesAllowed = true;
/**
* Array saves the occurences in the list.
*/
private final int[] listItems = new int[MAX_SIZE];
/**
* Variable for the value to find or delete.
*/
private int searchKey;
/**
* Variable for counter in a loop.
*/
private int f;
#Override
public int getSizeOfList() {
return sizeOfList;
}
#Override
public boolean areDuplicatesAllowed() {
return duplicatesAllowed;
}
#Override
public void setDupliatesAllowed(boolean pDuplicatesAllowed) {
duplicatesAllowed = pDuplicatesAllowed;
}
#Override
public void clear() {
sizeOfList = 0;
}
#Override
public void insert(int pValueToInsert) {
//Loop finds the position of the Item
for (f = 0; f < sizeOfList; f++)
if (listItems[f] > pValueToInsert)
break;
//Loop moves the items after the position up
for (int n = sizeOfList; n > f; n-- )
listItems[n] = listItems[n - 1];
//Insert the Value in the right position
listItems[f] = pValueToInsert;
//Increment List size
sizeOfList++;
}
#Override
public void delete(int pValueToDelete) {
int destroyHAHAHA = find(pValueToDelete);
//If it doesnt find it the item
if (destroyHAHAHA==sizeOfList)
System.out.println("I let you down boss, Can't find "
+ pValueToDelete);
//If it does, kill it with fire
else {
for (int n = destroyHAHAHA; n <sizeOfList; n++)
listItems[n] = listItems[n + 1];
sizeOfList--;
}
}
#Override
public void deleteAll(int pValueToDelete) {
int j = 0;
for(int i = 0; i < listItems.length; i++ )
{
if (listItems[i] != pValueToDelete)
listItems[j++] = listItems[i];
}
int [] newArray = new int[j];
System.arraycopy(listItems, 0, newArray, 0, j );
}
#Override
public void initializeWithRandomData(int pSizeOfList) {
// Loop creates an array with certain number of elements
if (duplicatesAllowed) {
for (int n = 0; n < pSizeOfList; ++n) {
insert(listItems[n] = (int) (Math.random() * MAX_VALUE + 1));
}
} else {
int newvalue=0;
for (int n = 0; n < pSizeOfList; ++n) {
listItems[n] = newvalue++;
++sizeOfList;
}
}
}
#Override
public int find(int pValueToFind) {
searchKey = pValueToFind;
int lowNumber = 0;
int highNumber = sizeOfList - 1;
int result;
while (true) {
result = (lowNumber + highNumber) / 2;
if (listItems[result] == searchKey)
return result;
else if (lowNumber > highNumber)
return sizeOfList;
else {
if (listItems[result] < searchKey)
lowNumber = result + 1;
else
highNumber = result - 1;
}
}
}
#Override
public int[] findAll(int pValueToFind) {
//Array with the location of item
int[] answerArray = new int[sizeOfList];
int searchIndex;
int answerIndex = 0;
for (searchIndex = 0; searchIndex < sizeOfList; searchIndex++) {
if (listItems[searchIndex] == pValueToFind) {
answerArray[answerIndex++] = searchIndex;
}
}
if (answerIndex > 0) {
return Arrays.copyOfRange(answerArray, 0, answerIndex);
} else {
return new int[0];
}
}
#Override
public String toString() {
return Arrays.toString(Arrays.copyOfRange(listItems, 0, sizeOfList));
}
public void bubbleshort(){
int out;
int in;
int middle;
for (out = 0; out < sizeOfList - 1; out++) {
middle = out;
for(in = out +1; in < sizeOfList; in++)
if(listItems[in] < listItems[middle])
middle = in;
selectionSort(out, middle);
}
}
public void selectionSort(int one, int two) {
int temporal = listItems[one];
listItems[one] = listItems[two];
listItems[two] = temporal;
}
}
You can use Common langs ArrayUtils.removeElement() or ArrayUtils.removeAll() method to remove all the elements from the array.
Set contains no duplicates. You can use a Set.
Set<T> mySet = new HashSet<T>(Arrays.asList(someArray));
or
Set<T> mySet = new HashSet<T>();
Collections.addAll(mySet, myArray);
import java.util.Random;
public class TestBed {
public static void main(String a[]) {
// creating the array
int[] array = new int[100];
Random random = new Random();
// changing the variable of my clone array for reference
int[] arr = cloneArray(array);
for (int i1 = 0; i1 < 100; i1++)
array[i1] = random.nextInt(100) + 1;
// print out of bubble sort before and after the sort
System.out
.println("***********************Bubble Sort ****************************");
arr = cloneArray(array);
System.out.println("Values Before the sort:\n");
printArray(arr);
System.out.println();
bubble_srt(arr);
System.out.print("Values after the sort:\n");
printArray(arr);
// print out of selection sort before and after the sort
System.out.println();
System.out
.println("********************Selection Sort*****************************");
System.out.println(" Selection Sort\n\n");
arr = cloneArray(array);
System.out.println("Values Before the sort:\n");
printArray(arr);
System.out.println();
selection_srt(arr);
System.out.print("Values after the sort:\n");
printArray(arr);
System.out.println();
Stack stack = new Stack();
}
public static int[] buildArray(int bound) {
return null;
}
// clone array as a data type
public static int[] cloneArray(int[] data) {
return (int[]) data.clone();
}
// print array as a data type
public static void printArray(int[] data) {
// while there are still numbers left in the array print out the next
// value
for (int i = 0; i < data.length; i++)
System.out.print(data[i] + " ");
}
// bubble syntax
public static void bubble_srt(int a[]) {
int t = 0;
for (int i = 0; i < a.length; i++) {
for (int j = 1; j < (a.length - i); j++) {
if (a[j - 1] > a[j]) {
t = a[j - 1];
a[j - 1] = a[j];
a[j] = t;
}
}
}
}
// selection syntax
public static void selection_srt(int array[]) {
for (int x = 0; x < array.length; x++) {
int index_of_min = x;
for (int y = x; y < array.length; y++) {
if (array[index_of_min] > array[y]) {
index_of_min = y;
}
}
int temp = array[x];
array[x] = array[index_of_min];
array[index_of_min] = temp;
}
}
}
From here i have to link my stack class but i don't know how to get the 2 classes together correctly im new to programming and my school threw me into a java class that was to high for me and now im stuck 5 weeks in.
public class Stack {
Node top;
int size;
public Stack() {
top = null;
size = 0;
}
public int pop() {
if (top != null) {
int item = top.data;
top = top.next;
size--;
return item;
}
return -1;
}
public void push(int data) {
Node t = new Node(data);
t.next = this.top;
this.top = t;
size++;
}
public boolean isEmpty() {
return size <= 0;
}
public int getSize() {
return size;
}
public int peek() {
return top.data;
}
public void printStack() {
Node n = this.top;
int pos = this.getSize();
while (pos > 0) {
System.out.println("Position: " + pos + " Element: " + n.data);
if (pos > 0) {
n = n.next;
}
pos--;
}
}
}
class Node {
public int data;
public Node next;
Node(int d) {
data = d;
next = null;
}
public int getData() {
return data;`enter code here`
}`enter code here`
{
Stack s = new Stack();
s.push(9);
s.push(2);
s.push(7);
s.push(3);
s.push(6);
s.push(4);
s.push(5);
System.out.println("Size is: " + s.getSize());
// s.printStack();
int size = s.getSize();
for (int i = 0; i < size; i++) {
System.out.print(s.pop() + " ");
}
}
}
At the beginning of Stack class file, create a package name, eg:
package com.example.testcode;
Then in your TestBed class file, import your Stack class from the package, eg:
import com.example.testcode.Stack;
I'm trying to implement a feed-forward neural network in Java.
I've created three classes NNeuron, NLayer and NNetwork. The "simple" calculations seem fine (I get correct sums/activations/outputs), but when it comes to the training process, I don't seem to get correct results. Can anyone, please tell what I'm doing wrong ?
The whole code for the NNetwork class is quite long, so I'm posting the part that is causing the problem:
[EDIT]: this is actually pretty much all of the NNetwork class
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
public class NNetwork
{
public static final double defaultLearningRate = 0.4;
public static final double defaultMomentum = 0.8;
private NLayer inputLayer;
private ArrayList<NLayer> hiddenLayers;
private NLayer outputLayer;
private ArrayList<NLayer> layers;
private double momentum = NNetwork1.defaultMomentum; // alpha: momentum, default! 0.3
private ArrayList<Double> learningRates;
public NNetwork (int nInputs, int nOutputs, Integer... neuronsPerHiddenLayer)
{
this(nInputs, nOutputs, Arrays.asList(neuronsPerHiddenLayer));
}
public NNetwork (int nInputs, int nOutputs, List<Integer> neuronsPerHiddenLayer)
{
// the number of neurons on the last layer build so far (i.e. the number of inputs for each neuron of the next layer)
int prvOuts = 1;
this.layers = new ArrayList<>();
// input layer
this.inputLayer = new NLayer(nInputs, prvOuts, this);
this.inputLayer.setAllWeightsTo(1.0);
this.inputLayer.setAllBiasesTo(0.0);
this.inputLayer.useSigmaForOutput(false);
prvOuts = nInputs;
this.layers.add(this.inputLayer);
// hidden layers
this.hiddenLayers = new ArrayList<>();
for (int i=0 ; i<neuronsPerHiddenLayer.size() ; i++)
{
this.hiddenLayers.add(new NLayer(neuronsPerHiddenLayer.get(i), prvOuts, this));
prvOuts = neuronsPerHiddenLayer.get(i);
}
this.layers.addAll(this.hiddenLayers);
// output layer
this.outputLayer = new NLayer(nOutputs, prvOuts, this);
this.layers.add(this.outputLayer);
this.initCoeffs();
}
private void initCoeffs ()
{
this.learningRates = new ArrayList<>();
// learning rates of the hidden layers
for (int i=0 ; i<this.hiddenLayers.size(); i++)
this.learningRates.add(NNetwork1.defaultLearningRate);
// learning rate of the output layer
this.learningRates.add(NNetwork1.defaultLearningRate);
}
public double getLearningRate (int layerIndex)
{
if (layerIndex > 0 && layerIndex <= this.hiddenLayers.size()+1)
{
return this.learningRates.get(layerIndex-1);
}
else
{
return 0;
}
}
public ArrayList<Double> getLearningRates ()
{
return this.learningRates;
}
public void setLearningRate (int layerIndex, double newLearningRate)
{
if (layerIndex > 0 && layerIndex <= this.hiddenLayers.size()+1)
{
this.learningRates.set(
layerIndex-1,
newLearningRate);
}
}
public void setLearningRates (Double... newLearningRates)
{
this.setLearningRates(Arrays.asList(newLearningRates));
}
public void setLearningRates (List<Double> newLearningRates)
{
int len = (this.learningRates.size() <= newLearningRates.size())
? this.learningRates.size()
: newLearningRates.size();
for (int i=0; i<len; i++)
this.learningRates
.set(i,
newLearningRates.get(i));
}
public double getMomentum ()
{
return this.momentum;
}
public void setMomentum (double momentum)
{
this.momentum = momentum;
}
public NNeuron getNeuron (int layerIndex, int neuronIndex)
{
if (layerIndex == 0)
return this.inputLayer.getNeurons().get(neuronIndex);
else if (layerIndex == this.hiddenLayers.size()+1)
return this.outputLayer.getNeurons().get(neuronIndex);
else
return this.hiddenLayers.get(layerIndex-1).getNeurons().get(neuronIndex);
}
public ArrayList<Double> getOutput (ArrayList<Double> inputs)
{
ArrayList<Double> lastOuts = inputs; // the last computed outputs of the last 'called' layer so far
// input layer
//lastOuts = this.inputLayer.getOutput(lastOuts);
lastOuts = this.getInputLayerOutputs(lastOuts);
// hidden layers
for (NLayer layer : this.hiddenLayers)
lastOuts = layer.getOutput(lastOuts);
// output layer
lastOuts = this.outputLayer.getOutput(lastOuts);
return lastOuts;
}
public ArrayList<ArrayList<Double>> getAllOutputs (ArrayList<Double> inputs)
{
ArrayList<ArrayList<Double>> outs = new ArrayList<>();
// input layer
outs.add(this.getInputLayerOutputs(inputs));
// hidden layers
for (NLayer layer : this.hiddenLayers)
outs.add(layer.getOutput(outs.get(outs.size()-1)));
// output layer
outs.add(this.outputLayer.getOutput(outs.get(outs.size()-1)));
return outs;
}
public ArrayList<ArrayList<Double>> getAllSums (ArrayList<Double> inputs)
{
//*
ArrayList<ArrayList<Double>> sums = new ArrayList<>();
ArrayList<Double> lastOut;
// input layer
sums.add(inputs);
lastOut = this.getInputLayerOutputs(inputs);
// hidden nodes
for (NLayer layer : this.hiddenLayers)
{
sums.add(layer.getSums(lastOut));
lastOut = layer.getOutput(lastOut);
}
// output layer
sums.add(this.outputLayer.getSums(lastOut));
return sums;
}
public ArrayList<Double> getInputLayerOutputs (ArrayList<Double> inputs)
{
ArrayList<Double> outs = new ArrayList<>();
for (int i=0 ; i<this.inputLayer.getNeurons().size() ; i++)
outs.add(this
.inputLayer
.getNeuron(i)
.getOutput(inputs.get(i)));
return outs;
}
public void changeWeights (
ArrayList<ArrayList<Double>> deltaW,
ArrayList<ArrayList<Double>> inputSet,
ArrayList<ArrayList<Double>> targetSet,
boolean checkError)
{
for (int i=0 ; i<deltaW.size()-1 ; i++)
this.hiddenLayers.get(i).changeWeights(deltaW.get(i), inputSet, targetSet, checkError);
this.outputLayer.changeWeights(deltaW.get(deltaW.size()-1), inputSet, targetSet, checkError);
}
public int train2 (
ArrayList<ArrayList<Double>> inputSet,
ArrayList<ArrayList<Double>> targetSet,
double maxError,
int maxIterations)
{
ArrayList<Double>
input,
target;
ArrayList<ArrayList<ArrayList<Double>>> prvNetworkDeltaW = null;
double error;
int i = 0, j = 0, traininSetLength = inputSet.size();
do // during each itreration...
{
error = 0.0;
for (j = 0; j < traininSetLength; j++) // ... for each training element...
{
input = inputSet.get(j);
target = targetSet.get(j);
prvNetworkDeltaW = this.train2_bp(input, target, prvNetworkDeltaW); // ... do backpropagation, and return the new weight deltas
error += this.getInputMeanSquareError(input, target);
}
i++;
} while (error > maxError && i < maxIterations); // iterate as much as necessary/possible
return i;
}
public ArrayList<ArrayList<ArrayList<Double>>> train2_bp (
ArrayList<Double> input,
ArrayList<Double> target,
ArrayList<ArrayList<ArrayList<Double>>> prvNetworkDeltaW)
{
ArrayList<ArrayList<Double>> layerSums = this.getAllSums(input); // the sums for each layer
ArrayList<ArrayList<Double>> layerOutputs = this.getAllOutputs(input); // the outputs of each layer
// get the layer deltas (inc the input layer that is null)
ArrayList<ArrayList<Double>> layerDeltas = this.train2_getLayerDeltas(layerSums, layerOutputs, target);
// get the weight deltas
ArrayList<ArrayList<ArrayList<Double>>> networkDeltaW = this.train2_getWeightDeltas(layerOutputs, layerDeltas, prvNetworkDeltaW);
// change the weights
this.train2_updateWeights(networkDeltaW);
return networkDeltaW;
}
public void train2_updateWeights (ArrayList<ArrayList<ArrayList<Double>>> networkDeltaW)
{
for (int i=1; i<this.layers.size(); i++)
this.layers.get(i).train2_updateWeights(networkDeltaW.get(i));
}
public ArrayList<ArrayList<ArrayList<Double>>> train2_getWeightDeltas (
ArrayList<ArrayList<Double>> layerOutputs,
ArrayList<ArrayList<Double>> layerDeltas,
ArrayList<ArrayList<ArrayList<Double>>> prvNetworkDeltaW)
{
ArrayList<ArrayList<ArrayList<Double>>> networkDeltaW = new ArrayList<>(this.layers.size());
ArrayList<ArrayList<Double>> layerDeltaW;
ArrayList<Double> neuronDeltaW;
for (int i=0; i<this.layers.size(); i++)
networkDeltaW.add(new ArrayList<ArrayList<Double>>());
double
deltaW, x, learningRate, prvDeltaW, d;
int i, j, k;
for (i=this.layers.size()-1; i>0; i--) // for each layer
{
learningRate = this.getLearningRate(i);
layerDeltaW = new ArrayList<>();
networkDeltaW.set(i, layerDeltaW);
for (j=0; j<this.layers.get(i).getNeurons().size(); j++) // for each neuron of this layer
{
neuronDeltaW = new ArrayList<>();
layerDeltaW.add(neuronDeltaW);
for (k=0; k<this.layers.get(i-1).getNeurons().size(); k++) // for each weight (i.e. each neuron of the previous layer)
{
d = layerDeltas.get(i).get(j);
x = layerOutputs.get(i-1).get(k);
prvDeltaW = (prvNetworkDeltaW != null)
? prvNetworkDeltaW.get(i).get(j).get(k)
: 0.0;
deltaW = -learningRate * d * x + this.momentum * prvDeltaW;
neuronDeltaW.add(deltaW);
}
// the bias !!
d = layerDeltas.get(i).get(j);
x = 1;
prvDeltaW = (prvNetworkDeltaW != null)
? prvNetworkDeltaW.get(i).get(j).get(prvNetworkDeltaW.get(i).get(j).size()-1)
: 0.0;
deltaW = -learningRate * d * x + this.momentum * prvDeltaW;
neuronDeltaW.add(deltaW);
}
}
return networkDeltaW;
}
ArrayList<ArrayList<Double>> train2_getLayerDeltas (
ArrayList<ArrayList<Double>> layerSums,
ArrayList<ArrayList<Double>> layerOutputs,
ArrayList<Double> target)
{
// get ouput deltas
ArrayList<Double> outputDeltas = new ArrayList<>(); // the output layer deltas
double
oErr, // output error given a target
s, // sum
o, // output
d; // delta
int
nOutputs = target.size(), // #TODO ?== this.outputLayer.size()
nLayers = this.hiddenLayers.size()+2; // #TODO ?== layerOutputs.size()
for (int i=0; i<nOutputs; i++) // for each neuron...
{
s = layerSums.get(nLayers-1).get(i);
o = layerOutputs.get(nLayers-1).get(i);
oErr = (target.get(i) - o);
d = -oErr * this.getNeuron(nLayers-1, i).sigmaPrime(s); // #TODO "s" or "o" ??
outputDeltas.add(d);
}
// get hidden deltas
ArrayList<ArrayList<Double>> hiddenDeltas = new ArrayList<>();
for (int i=0; i<this.hiddenLayers.size(); i++)
hiddenDeltas.add(new ArrayList<Double>());
NLayer nextLayer = this.outputLayer;
ArrayList<Double> nextDeltas = outputDeltas;
int
h, k,
nHidden = this.hiddenLayers.size(),
nNeurons = this.hiddenLayers.get(nHidden-1).getNeurons().size();
double
wdSum = 0.0;
for (int i=nHidden-1; i>=0; i--) // for each hidden layer
{
hiddenDeltas.set(i, new ArrayList<Double>());
for (h=0; h<nNeurons; h++)
{
wdSum = 0.0;
for (k=0; k<nextLayer.getNeurons().size(); k++)
{
wdSum += nextLayer.getNeuron(k).getWeight(h) * nextDeltas.get(k);
}
s = layerSums.get(i+1).get(h);
d = this.getNeuron(i+1, h).sigmaPrime(s) * wdSum;
hiddenDeltas.get(i).add(d);
}
nextLayer = this.hiddenLayers.get(i);
nextDeltas = hiddenDeltas.get(i);
}
ArrayList<ArrayList<Double>> deltas = new ArrayList<>();
// input layer deltas: void
deltas.add(null);
// hidden layers deltas
deltas.addAll(hiddenDeltas);
// output layer deltas
deltas.add(outputDeltas);
return deltas;
}
public double getInputMeanSquareError (ArrayList<Double> input, ArrayList<Double> target)
{
double diff, mse=0.0;
ArrayList<Double> output = this.getOutput(input);
for (int i=0; i<target.size(); i++)
{
diff = target.get(i) - output.get(i);
mse += (diff * diff);
}
mse /= 2.0;
return mse;
}
}
Some methods' names (with their return values/types) are quite self-explanatory, like "this.getAllSums" that returns the sums (sum(x_i*w_i) for each neuron) of each layer, "this.getAllOutputs" that return the outputs (sigmoid(sum) for each neuron) of each layer and "this.getNeuron(i,j)" that returns the j'th neuron of the i'th layer.
Thank you in advance for your help :)
Here is a very simple java implementation with tests in the main method :
import java.util.Arrays;
import java.util.Random;
public class MLP {
public static class MLPLayer {
float[] output;
float[] input;
float[] weights;
float[] dweights;
boolean isSigmoid = true;
public MLPLayer(int inputSize, int outputSize, Random r) {
output = new float[outputSize];
input = new float[inputSize + 1];
weights = new float[(1 + inputSize) * outputSize];
dweights = new float[weights.length];
initWeights(r);
}
public void setIsSigmoid(boolean isSigmoid) {
this.isSigmoid = isSigmoid;
}
public void initWeights(Random r) {
for (int i = 0; i < weights.length; i++) {
weights[i] = (r.nextFloat() - 0.5f) * 4f;
}
}
public float[] run(float[] in) {
System.arraycopy(in, 0, input, 0, in.length);
input[input.length - 1] = 1;
int offs = 0;
Arrays.fill(output, 0);
for (int i = 0; i < output.length; i++) {
for (int j = 0; j < input.length; j++) {
output[i] += weights[offs + j] * input[j];
}
if (isSigmoid) {
output[i] = (float) (1 / (1 + Math.exp(-output[i])));
}
offs += input.length;
}
return Arrays.copyOf(output, output.length);
}
public float[] train(float[] error, float learningRate, float momentum) {
int offs = 0;
float[] nextError = new float[input.length];
for (int i = 0; i < output.length; i++) {
float d = error[i];
if (isSigmoid) {
d *= output[i] * (1 - output[i]);
}
for (int j = 0; j < input.length; j++) {
int idx = offs + j;
nextError[j] += weights[idx] * d;
float dw = input[j] * d * learningRate;
weights[idx] += dweights[idx] * momentum + dw;
dweights[idx] = dw;
}
offs += input.length;
}
return nextError;
}
}
MLPLayer[] layers;
public MLP(int inputSize, int[] layersSize) {
layers = new MLPLayer[layersSize.length];
Random r = new Random(1234);
for (int i = 0; i < layersSize.length; i++) {
int inSize = i == 0 ? inputSize : layersSize[i - 1];
layers[i] = new MLPLayer(inSize, layersSize[i], r);
}
}
public MLPLayer getLayer(int idx) {
return layers[idx];
}
public float[] run(float[] input) {
float[] actIn = input;
for (int i = 0; i < layers.length; i++) {
actIn = layers[i].run(actIn);
}
return actIn;
}
public void train(float[] input, float[] targetOutput, float learningRate, float momentum) {
float[] calcOut = run(input);
float[] error = new float[calcOut.length];
for (int i = 0; i < error.length; i++) {
error[i] = targetOutput[i] - calcOut[i]; // negative error
}
for (int i = layers.length - 1; i >= 0; i--) {
error = layers[i].train(error, learningRate, momentum);
}
}
public static void main(String[] args) throws Exception {
float[][] train = new float[][]{new float[]{0, 0}, new float[]{0, 1}, new float[]{1, 0}, new float[]{1, 1}};
float[][] res = new float[][]{new float[]{0}, new float[]{1}, new float[]{1}, new float[]{0}};
MLP mlp = new MLP(2, new int[]{2, 1});
mlp.getLayer(1).setIsSigmoid(false);
Random r = new Random();
int en = 500;
for (int e = 0; e < en; e++) {
for (int i = 0; i < res.length; i++) {
int idx = r.nextInt(res.length);
mlp.train(train[idx], res[idx], 0.3f, 0.6f);
}
if ((e + 1) % 100 == 0) {
System.out.println();
for (int i = 0; i < res.length; i++) {
float[] t = train[i];
System.out.printf("%d epoch\n", e + 1);
System.out.printf("%.1f, %.1f --> %.3f\n", t[0], t[1], mlp.run(t)[0]);
}
}
}
}
}
I tried going over your code, but as you stated, it was pretty long.
Here's what I suggest:
To verify that your network is learning properly, try to train a simple network, like a network that recognizes the XOR operator. This shouldn't take all that long.
Use the simplest back-propagation algorithm. Stochastic backpropagation (where the weights are updated after the presentation of each training input) is the easiest. Implement the algorithm without the momentum term initially, and with a constant learning rate (i.e., don't start with adaptive learning-rates). Once you're satisfied that the algorithm is working, you can introduce the momentum term. Doing too many things at the same time increases the chances that more than one thing can go wrong. This makes it harder for you to see where you went wrong.
If you want to go over some code, you can check out some code that I wrote; you want to look at Backpropagator.java. I've basically implemented the stochastic backpropagation algorithm with a momentum term. I also have a video where I provide a quick explanation of my implementation of the backpropagation algorithm.
Hopefully this is of some help!