As I described in the title, I tried to access a picture file several times using an InputStream, then save it to blob type database but the result is null for the second, third, and so on. Here is the snippet of the code:
File image1 = new File("src/template1.png").getAbsoluteFile();
File image2 = new File("src/template2.png").getAbsoluteFile();
InputStream in1 = new FileInputStream(image1);
InputStream in2 = new FileInputStream(image2);
long length1 = image1.length();
long length2 = image2.length();
pstmt.setString(1, jTextField18.getText().toUpperCase());
pstmt.setBlob( 2, in1, length1 );
pstmt.setBlob( 3, in1, length1 );
pstmt.setBlob( 4, in1, length1 );
pstmt.setBlob( 5, in1, length1 );
pstmt.setBlob( 6, in1, length1 );
pstmt.setBlob( 7, in2, length2 );
Pstmt.setBlob( 8, in2, length2 );
then the result is:
YR/21
[BLOB - 5.6 KiB]
[BLOB - 0 B]
[BLOB - 0 B]
[BLOB - 0 B]
[BLOB - 0 B]
[BLOB - 5.5 KiB]
[BLOB - 0 B]
I tried to use new InputStream like this code and another error occured. It says:
Row size too large(>8126). Changing some columns to TEXT or BLOB or using ROW_FORMAT=DINAMIC or ROW_FORMAT=COMPRESSED may help. In current row format, BLOB prefix of 768 bytes is stored inline.
pstmt.setBlob( 2, in1, length1 );
image1 = new File("src/template1.png").getAbsoluteFile();
in1 = new FileInputStream(image1);
pstmt.setBlob( 3, in1, length1 );
image1 = new File("src/template1.png").getAbsoluteFile();
in1 = new FileInputStream(image1);
pstmt.setBlob( 4, in1, length1 );
I think the problem is answered and change into another problem. Thanks for the answer
The first call consumes the stream, and the subsequent calls get no data. You need a separate InputStream for each call.
"Is there any way to save a picture to multiple blob?"
Yes, but your code will only work for small images. Instead, have a parent reference for the image and a children for the actual image, which will have the blobs. Make the children small and save only a chunk of the image to every one of them.
This will help you manage the space and it will be easier to back up.
For a reference on how to read a file in chunks, take a look here: http://www.javapractices.com/topic/TopicAction.do?Id=245
Save the chunk of the image to the database using the while-loop that reads the file.
It could be a lengthy operation to save the file, so you could have a flag on the parent defining if the children were all loaded or not. Thus preventing other threads from reading incomplete images.
If you are using MySQL, there's this tutorial on how to read and write images: http://zetcode.com/db/mysqljava/
Have fun.
Related
I have been trying to build a CNN model using dl4j, but it is giving me an error. The code:
RecordReader rr;
rr = new CSVRecordReader();
rr.initialize(new FileSplit(new File(dataLocalPath, nameTrain)));
DataSetIterator trainIter = new RecordReaderDataSetIterator(rr, batchSize, 0, 2);
MultiLayerConfiguration conf = new NeuralNetConfiguration.Builder().seed(123).updater(new Adam(0.1))
.list()
.layer(0,new Convolution1DLayer.Builder().kernelSize(3).padding(1).nIn(371).nOut(64).build())
.layer(1,new Subsampling1DLayer.Builder().kernelSize(3).padding(1).build())
.layer(2,new Convolution1DLayer.Builder().kernelSize(3).activation(Activation.RELU).padding(1).nIn(64)
.nOut(32).build())
.layer(3,new Subsampling1DLayer.Builder().kernelSize(3).padding(1).build())
.layer(4,new DenseLayer.Builder().activation(Activation.RELU).nIn(32).nOut(16).build())
.layer(5,new OutputLayer.Builder(LossFunction.RECONSTRUCTION_CROSSENTROPY).activation(Activation.SIGMOID)
.nIn(16).nOut(2).build())
.build();
MultiLayerNetwork model = new MultiLayerNetwork(conf);
model.init();
model.setListeners(new ScoreIterationListener(10));
final DataSet trainData = trainIter.next();
INDArray a = trainData.getFeatures();
final INDArray b = trainData.getLabels();
a = a.reshape(new int[] { (int) a.size(0), (int) a.size(1), 1 });
model.fit(a, b);
Added the error below,
Exception in thread "main" org.deeplearning4j.exception.DL4JInvalidInputException: Input that is not a matrix; expected matrix (rank 2), got rank 3 array with shape [128, 32, 1]. Missing preprocessor or wrong input type? (layer name: layer4, layer index: 4, layer type: DenseLayer)
at org.deeplearning4j.nn.layers.BaseLayer.preOutputWithPreNorm(BaseLayer.java:306)
at org.deeplearning4j.nn.layers.BaseLayer.preOutput(BaseLayer.java:289)
at org.deeplearning4j.nn.layers.BaseLayer.activate(BaseLayer.java:337)
at org.deeplearning4j.nn.layers.AbstractLayer.activate(AbstractLayer.java:257)
at org.deeplearning4j.nn.multilayer.MultiLayerNetwork.ffToLayerActivationsInWs(MultiLayerNetwork.java:1129)
at org.deeplearning4j.nn.multilayer.MultiLayerNetwork.computeGradientAndScore(MultiLayerNetwork.java:2741)
at org.deeplearning4j.nn.multilayer.MultiLayerNetwork.computeGradientAndScore(MultiLayerNetwork.java:2699)
at org.deeplearning4j.optimize.solvers.BaseOptimizer.gradientAndScore(BaseOptimizer.java:170)
at org.deeplearning4j.optimize.solvers.StochasticGradientDescent.optimize(StochasticGradientDescent.java:63)
at org.deeplearning4j.optimize.Solver.optimize(Solver.java:52)
at org.deeplearning4j.nn.multilayer.MultiLayerNetwork.fitHelper(MultiLayerNetwork.java:2303)
at org.deeplearning4j.nn.multilayer.MultiLayerNetwork.fit(MultiLayerNetwork.java:2261)
at org.deeplearning4j.nn.multilayer.MultiLayerNetwork.fit(MultiLayerNetwork.java:2248)
at com.rssoftware.efrm.AnnModelFromKeras.trainModel(AnnModelFromKeras.java:73)
at com.rssoftware.efrm.AnnModelFromKeras.main(AnnModelFromKeras.java:89)
I have tried using input pre-processor, CNN to feed forward pre processor, but it is not working.
The expected input shape into a conv1d layer is [minibatchSize, convNIn, length] or [minibatchSize, featuresSize, sequenceLength] in terms of a time series. The reshape in your code sets your length to 1. Maybe you intended to set featuresize/convNIn to 1?
When I run below code:
model.output(samples).getDouble(0);
I receive error:
org.deeplearning4j.exception.DL4JInvalidInputException:
Cannot do forward pass in Convolution layer (layer name = conv1d_1, layer index = 0): input array channels does not match CNN layer configuration
(data input channels = 80, [minibatch,inputDepth,height,width]=[1, 80, 3, 1]; expected input channels = 3)
(layer name: conv1d_1, layer index: 0, layer type: Convolution1DLayer)
I created the data as float[] array, with length = 240.
Creation of INDArray:
INDArray features = Nd4j.create(data, new int[]{1, 240}, 'c');
Here is my keras model:
model = Sequential()
model.add(Reshape((const.PERIOD, const.N_FEATURES), input_shape=(240,)))
model.add(Conv1D(100, 10, activation='relu', input_shape=(const.PERIOD, const.N_FEATURES)))
model.add(Conv1D(100, 10, activation='relu'))
model.add(MaxPooling1D(const.N_FEATURES))
model.add(Conv1D(160, 10, activation='relu'))
model.add(Conv1D(160, 10, activation='relu'))
model.add(Flatten())
model.add(Dropout(0.5))
model.add(Dense(7, activation='softmax'))
model.summary()
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
Where PERIOD = 80, N_FEATURES = 3
If I set shape as:
INDArray features = Nd4j.create(data, new int[]{240, 1});
Then error is:
IllegalStateException: Input shape [240, 1] and output shape[240, 1] do not match
at org.deeplearning4j.nn.modelimport.keras.preprocessors.ReshapePreprocessor.preProcess(ReshapePreprocessor.java:103)
at org.deeplearning4j.nn.multilayer.MultiLayerNetwork.outputOfLayerDetached(MultiLayerNetwork.java:1256)
at org.deeplearning4j.nn.multilayer.MultiLayerNetwork.output(MultiLayerNetwork.java:2340)
at org.deeplearning4j.nn.multilayer.MultiLayerNetwork.output(MultiLayerNetwork.java:2303)
at org.deeplearning4j.nn.multilayer.MultiLayerNetwork.output(MultiLayerNetwork.java:2294)
at org.deeplearning4j.nn.multilayer.MultiLayerNetwork.output(MultiLayerNetwork.java:2281)
at org.deeplearning4j.nn.multilayer.MultiLayerNetwork.output(MultiLayerNetwork.java:2377)
Can you file an issue? This looks like a bug. Thanks.
https://github.com/eclipse/deeplearning4j/issues
I've been trying so hard to train a network but I cannot do it. Neuroph Studio doesn't help at all, it always return null when training.
Then I tried this code in a Java app :
// create new perceptron network
NeuralNetwork neuralNetwork = new Perceptron(2, 1);
// create training set
DataSet trainingSet = new DataSet(2, 1);
// add training data to training set (logical OR function)
trainingSet.addRow(new DataSetRow(new double[]{0, 0}, new double[]{0.5d}));
trainingSet.addRow(new DataSetRow(new double[]{0, 1}, new double[]{1}));
trainingSet.addRow(new DataSetRow(new double[]{1, 0}, new double[]{1}));
trainingSet.addRow(new DataSetRow(new double[]{1, 1}, new double[]{1}));
// learn the training set
neuralNetwork.learn(trainingSet);
// save the trained network into file
neuralNetwork.save("or_perceptron.nnet");
// load the saved network
neuralNetwork = NeuralNetwork.createFromFile("or_perceptron.nnet");
// set network input
neuralNetwork.setInput(1, 1);
// calculate network
neuralNetwork.calculate();
// get network output
double[] networkOutput = neuralNetwork.getOutput();
for (double res : networkOutput) {
System.out.println(res);
}
This works, but I want to train something like this:
Input: 0.3 , 0.5
Output : 0.2
It keeps training forever, what is wrong with neuroph, or it doesn't work at all ?
At the end the only thing that worked was to load the training set from external files. Maybe there is another solution, but that was the only thing that worked for me at the end.
I am trying to run this simple example of RCaller on my windows 7 machine.
RCaller caller = new RCaller();
RCode code = new RCode();
caller.setRscriptExecutable("C:\\Program Files\\R\\R-3.2.1\\bin\\Rscript.exe");
double[] numbers = new double[]{1, 4, 3, 5, 6, 10};
code.addDoubleArray("x", numbers);
code.addRCode("my.mean<-mean(x)");
code.addRCode("my.var<-var(x)");
code.addRCode("my.all<-list(mean=my.mean, variance=my.var)");
caller.setRCode(code);
caller.runAndReturnResult("my.all");
double[] results = caller.getParser().getAsDoubleArray("mean");`
System.out.println(results[0]);
This is the error message:
cat(makexml(obj=my.all, name="my.all"), file="C:/Users/Bob Smith/AppData/Local/Temp/Routput8089051805366000971")
rcaller.exception.ParseException: Can not handle R results due to : rcaller.exception.ParseException: Can not parse output: The generated file C:\Users\Bob Smith\AppData\Local\Temp\Routput8089051805366000971 is empty
I've tried several versions of RCaller (this example is 2.2.0) and have received similar errors. Does anyone know how to fix this?
I want to generate a R boxplot by using rcaller with java.
My code is :
try {
RCaller caller = new RCaller();
caller.setRExecutable("/usr/bin/R");
caller.setGraphicsTheme(new DefaultTheme());
RCode code = new RCode();
code.clear();
File file = code.startPlot();
code.addRCode("boxplot((1:10),main=\"1-10\")");
System.out.println(code.toString());
code.endPlot();
caller.setRCode(code);
caller.runAndReturnResultOnline("boxplot(1:10),main=\"1-10\"");
code.showPlot(file);
But it does not keep run on codecaller.runAndReturnResultOnline("boxplot(1:10),main=\"1-10\"");
i try to use code below ,that can plot a R plot.what differences between them ?
try {
RCaller caller = new RCaller();
caller.setRExecutable("/usr/bin/R");
caller.setGraphicsTheme(new DefaultTheme());
RCode code = new RCode();
code.clear();
double[] numbers = new double[] { 1, 4, 4, 5, 6, 10 };
code.addDoubleArray("x", numbers);
File file = code.startPlot();
System.out.println(file.toString());
code.addRCode("plot.ts(x)");
System.out.println(code.toString());
code.endPlot();
caller.setRCode(code);
caller.runAndReturnResultOnline("plot.ts(x)");
code.showPlot(file);
Replace your line :
code.addRCode("boxplot((1:10),main=\"1-10\")");
By this :
code.addRCode("boxplot(c(1:10),main='1-10')");
Or as in the second example (the working one), you can give your x vector from java and replace this line:
caller.runAndReturnResultOnline("plot.ts(x)");
by
caller.runAndReturnResultOnline("boxplot(x)");
PS: I dont' have java to test.