I've been trying so hard to train a network but I cannot do it. Neuroph Studio doesn't help at all, it always return null when training.
Then I tried this code in a Java app :
// create new perceptron network
NeuralNetwork neuralNetwork = new Perceptron(2, 1);
// create training set
DataSet trainingSet = new DataSet(2, 1);
// add training data to training set (logical OR function)
trainingSet.addRow(new DataSetRow(new double[]{0, 0}, new double[]{0.5d}));
trainingSet.addRow(new DataSetRow(new double[]{0, 1}, new double[]{1}));
trainingSet.addRow(new DataSetRow(new double[]{1, 0}, new double[]{1}));
trainingSet.addRow(new DataSetRow(new double[]{1, 1}, new double[]{1}));
// learn the training set
neuralNetwork.learn(trainingSet);
// save the trained network into file
neuralNetwork.save("or_perceptron.nnet");
// load the saved network
neuralNetwork = NeuralNetwork.createFromFile("or_perceptron.nnet");
// set network input
neuralNetwork.setInput(1, 1);
// calculate network
neuralNetwork.calculate();
// get network output
double[] networkOutput = neuralNetwork.getOutput();
for (double res : networkOutput) {
System.out.println(res);
}
This works, but I want to train something like this:
Input: 0.3 , 0.5
Output : 0.2
It keeps training forever, what is wrong with neuroph, or it doesn't work at all ?
At the end the only thing that worked was to load the training set from external files. Maybe there is another solution, but that was the only thing that worked for me at the end.
Related
I have been trying to build a CNN model using dl4j, but it is giving me an error. The code:
RecordReader rr;
rr = new CSVRecordReader();
rr.initialize(new FileSplit(new File(dataLocalPath, nameTrain)));
DataSetIterator trainIter = new RecordReaderDataSetIterator(rr, batchSize, 0, 2);
MultiLayerConfiguration conf = new NeuralNetConfiguration.Builder().seed(123).updater(new Adam(0.1))
.list()
.layer(0,new Convolution1DLayer.Builder().kernelSize(3).padding(1).nIn(371).nOut(64).build())
.layer(1,new Subsampling1DLayer.Builder().kernelSize(3).padding(1).build())
.layer(2,new Convolution1DLayer.Builder().kernelSize(3).activation(Activation.RELU).padding(1).nIn(64)
.nOut(32).build())
.layer(3,new Subsampling1DLayer.Builder().kernelSize(3).padding(1).build())
.layer(4,new DenseLayer.Builder().activation(Activation.RELU).nIn(32).nOut(16).build())
.layer(5,new OutputLayer.Builder(LossFunction.RECONSTRUCTION_CROSSENTROPY).activation(Activation.SIGMOID)
.nIn(16).nOut(2).build())
.build();
MultiLayerNetwork model = new MultiLayerNetwork(conf);
model.init();
model.setListeners(new ScoreIterationListener(10));
final DataSet trainData = trainIter.next();
INDArray a = trainData.getFeatures();
final INDArray b = trainData.getLabels();
a = a.reshape(new int[] { (int) a.size(0), (int) a.size(1), 1 });
model.fit(a, b);
Added the error below,
Exception in thread "main" org.deeplearning4j.exception.DL4JInvalidInputException: Input that is not a matrix; expected matrix (rank 2), got rank 3 array with shape [128, 32, 1]. Missing preprocessor or wrong input type? (layer name: layer4, layer index: 4, layer type: DenseLayer)
at org.deeplearning4j.nn.layers.BaseLayer.preOutputWithPreNorm(BaseLayer.java:306)
at org.deeplearning4j.nn.layers.BaseLayer.preOutput(BaseLayer.java:289)
at org.deeplearning4j.nn.layers.BaseLayer.activate(BaseLayer.java:337)
at org.deeplearning4j.nn.layers.AbstractLayer.activate(AbstractLayer.java:257)
at org.deeplearning4j.nn.multilayer.MultiLayerNetwork.ffToLayerActivationsInWs(MultiLayerNetwork.java:1129)
at org.deeplearning4j.nn.multilayer.MultiLayerNetwork.computeGradientAndScore(MultiLayerNetwork.java:2741)
at org.deeplearning4j.nn.multilayer.MultiLayerNetwork.computeGradientAndScore(MultiLayerNetwork.java:2699)
at org.deeplearning4j.optimize.solvers.BaseOptimizer.gradientAndScore(BaseOptimizer.java:170)
at org.deeplearning4j.optimize.solvers.StochasticGradientDescent.optimize(StochasticGradientDescent.java:63)
at org.deeplearning4j.optimize.Solver.optimize(Solver.java:52)
at org.deeplearning4j.nn.multilayer.MultiLayerNetwork.fitHelper(MultiLayerNetwork.java:2303)
at org.deeplearning4j.nn.multilayer.MultiLayerNetwork.fit(MultiLayerNetwork.java:2261)
at org.deeplearning4j.nn.multilayer.MultiLayerNetwork.fit(MultiLayerNetwork.java:2248)
at com.rssoftware.efrm.AnnModelFromKeras.trainModel(AnnModelFromKeras.java:73)
at com.rssoftware.efrm.AnnModelFromKeras.main(AnnModelFromKeras.java:89)
I have tried using input pre-processor, CNN to feed forward pre processor, but it is not working.
The expected input shape into a conv1d layer is [minibatchSize, convNIn, length] or [minibatchSize, featuresSize, sequenceLength] in terms of a time series. The reshape in your code sets your length to 1. Maybe you intended to set featuresize/convNIn to 1?
When I run below code:
model.output(samples).getDouble(0);
I receive error:
org.deeplearning4j.exception.DL4JInvalidInputException:
Cannot do forward pass in Convolution layer (layer name = conv1d_1, layer index = 0): input array channels does not match CNN layer configuration
(data input channels = 80, [minibatch,inputDepth,height,width]=[1, 80, 3, 1]; expected input channels = 3)
(layer name: conv1d_1, layer index: 0, layer type: Convolution1DLayer)
I created the data as float[] array, with length = 240.
Creation of INDArray:
INDArray features = Nd4j.create(data, new int[]{1, 240}, 'c');
Here is my keras model:
model = Sequential()
model.add(Reshape((const.PERIOD, const.N_FEATURES), input_shape=(240,)))
model.add(Conv1D(100, 10, activation='relu', input_shape=(const.PERIOD, const.N_FEATURES)))
model.add(Conv1D(100, 10, activation='relu'))
model.add(MaxPooling1D(const.N_FEATURES))
model.add(Conv1D(160, 10, activation='relu'))
model.add(Conv1D(160, 10, activation='relu'))
model.add(Flatten())
model.add(Dropout(0.5))
model.add(Dense(7, activation='softmax'))
model.summary()
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
Where PERIOD = 80, N_FEATURES = 3
If I set shape as:
INDArray features = Nd4j.create(data, new int[]{240, 1});
Then error is:
IllegalStateException: Input shape [240, 1] and output shape[240, 1] do not match
at org.deeplearning4j.nn.modelimport.keras.preprocessors.ReshapePreprocessor.preProcess(ReshapePreprocessor.java:103)
at org.deeplearning4j.nn.multilayer.MultiLayerNetwork.outputOfLayerDetached(MultiLayerNetwork.java:1256)
at org.deeplearning4j.nn.multilayer.MultiLayerNetwork.output(MultiLayerNetwork.java:2340)
at org.deeplearning4j.nn.multilayer.MultiLayerNetwork.output(MultiLayerNetwork.java:2303)
at org.deeplearning4j.nn.multilayer.MultiLayerNetwork.output(MultiLayerNetwork.java:2294)
at org.deeplearning4j.nn.multilayer.MultiLayerNetwork.output(MultiLayerNetwork.java:2281)
at org.deeplearning4j.nn.multilayer.MultiLayerNetwork.output(MultiLayerNetwork.java:2377)
Can you file an issue? This looks like a bug. Thanks.
https://github.com/eclipse/deeplearning4j/issues
I am trying to run this simple example of RCaller on my windows 7 machine.
RCaller caller = new RCaller();
RCode code = new RCode();
caller.setRscriptExecutable("C:\\Program Files\\R\\R-3.2.1\\bin\\Rscript.exe");
double[] numbers = new double[]{1, 4, 3, 5, 6, 10};
code.addDoubleArray("x", numbers);
code.addRCode("my.mean<-mean(x)");
code.addRCode("my.var<-var(x)");
code.addRCode("my.all<-list(mean=my.mean, variance=my.var)");
caller.setRCode(code);
caller.runAndReturnResult("my.all");
double[] results = caller.getParser().getAsDoubleArray("mean");`
System.out.println(results[0]);
This is the error message:
cat(makexml(obj=my.all, name="my.all"), file="C:/Users/Bob Smith/AppData/Local/Temp/Routput8089051805366000971")
rcaller.exception.ParseException: Can not handle R results due to : rcaller.exception.ParseException: Can not parse output: The generated file C:\Users\Bob Smith\AppData\Local\Temp\Routput8089051805366000971 is empty
I've tried several versions of RCaller (this example is 2.2.0) and have received similar errors. Does anyone know how to fix this?
I researched a lot of questions and examples but I can't seem to find out what's wrong with my RPROP NN. It's also the first time I use Encog so I'm wondering if it's something I'm doing wrong.
I am trying to train the network to recognize a cat by feeding it images (50x50), then converting it to grayscale and feeding the network an input double[][] along with a target double[][]. I noticed that the error is constantly at 4.0, so I performed a dumpWeights() with every training iteration to see what's going on. I noticed that the weights were constantly zero. I then went back to the basics to see if I'm doing things right so I modified it for an XOR problem:
//////////First created the network:
BasicNetwork network = new BasicNetwork();
network.addLayer(new BasicLayer(null, true, 2));
network.addLayer(new BasicLayer(new ActivationBiPolar(), true, 2));
network.addLayer(new BasicLayer(new ActivationBiPolar(), false, 1));
network.getStructure().finalizeStructure();
network.reset();
//////Then created my data set and target vector (ideal vector) and fed it to a new RPROP training class:
final double targetVector[][] = { { -1 }, { 1.0 }, { 1.0 }, { -1 } };
final double inputData[][] = { { -1, -1 }, { 1.0, -1 },{ -1, 1.0 }, { 1.0, 1.0 } };
MLDataSet trainingSet = new BasicMLDataSet(inputData, targetVector);
final ResilientPropagation train = new ResilientPropagation(network, trainingSet);
///////train network
int epoch = 1;
do{
train.iteration();
System.out.println("Epoch #" + epoch + " Error : " + train.getError()) ;
epoch++;
System.out.println(network.dumpWeights());
}while(train.getError() > 0.01) ;
train.finishTraining();
System.out.println("End of training");
I get the following output, notice the lines of 0.0 as a result of the network.dumpWeights() method:
Epoch #132636 Error : 2.0
0,0,0,0,0,0,0,0,0
Epoch #132637 Error : 2.0
0,0,0,0,0,0,0,0,0
Epoch #132638 Error : 2.0
0,0,0,0,0,0,0,0,0
Epoch #132639 Error : 2.0
0,0,0,0,0,0,0,0,0
Epoch #132640 Error : 2.0
... and so on.
Anything obvious you can see that I'm doing wrong here? I also tried a 2-3-1 architecture as the XORHelloWorld.java example implemented.
Any help would be greatly appreciated.
Try switching your hidden layer to a TANH activation function, such as this:
network.addLayer(new BasicLayer(null, true, 2));
network.addLayer(new BasicLayer(new ActivationTANH(), true, 2));
network.addLayer(new BasicLayer(new ActivationBiPolar(), false, 1));
With this change, I can get your example above to converge. I think it will work better than Sigmoid, if you are using -1 to 1 as the input. It is okay to is a linear activation function (i.e. ActivationBiPolar as the output activation function) but you need something such as sigmoid/tanh as the hidden. Something that does not just return 1.0 as the derivative, like the linear functions do.
I am beginner in java and Weka tool, I want to use Logitboost algorithm with DecisionStump as weak learner in my java code, but I don't know how do this work. I create a vector with six feature(without label feature) and I want feed it into logitboost for labeling and probability of its assignment. Labels are 1 or -1 and train/test data is in an arff file.This is my code, but algorithm always return 0 !
Thanks
double candidate_similarity(ha_nodes ha , WeightMatrix[][] wm , LogitBoost lgb ,ArrayList<Attribute> atts){
LogitBoost lgb = new LogitBoost();
lgb.buildClassifier(newdata);//newdata is an arff file with some labeled data
Evaluation eval = new Evaluation(newdata);
eval.crossValidateModel(lgb, newdata, 10, new Random(1));
try {
feature_vector[0] = IP_sim(Main.a_new.dip, ha.candidate.dip_cand);
feature_vector[1] = IP_sim(Main.a_new.sip, ha.candidate.sip_cand);
feature_vector[2] = IP_s_d_sim(Main.a_new.sip, ha);
feature_vector[3] = Dport_sim(Main.a_new.dport, ha);
freq_weight(Main.a_new.Atype, ha, freq_avg, weight_avg , wm);
feature_vector[4] = weight_avg;
feature_vector[5] = freq_avg;
double[] values = new double[]{feature_vector[0],feature_vector[1],feature_vector[2],feature_vector[3],feature_vector[4],feature_vector[5]};
DenseInstance newInst = new DenseInstance(1.0,values);
Instances dataUnlabeled = new Instances("TestInstances", atts, 0);
dataUnlabeled.add(newInst);
dataUnlabeled.setClassIndex(dataUnlabeled.numAttributes() - 1);
double clslable = lgb.classifyInstance(inst);
} catch (Exception ex) {
//Logger.getLogger(Module2.class.getName()).log(Level.SEVERE, null, ex);
}
return clslable;}
Where did this newdata come from? you need to load the file properly to get a correct classification, use this class to load features from the file:
http://weka.sourceforge.net/doc/weka/core/converters/ArffLoader.html
I'm not posting an example code because I use weka with MATLAB, so I dont have examples in Java.