We Keep Coding

Need help converting DCGAN to Java for Tensorflow for Java

I am trying to get DCGAN ( Deep Convolutional Generative Adversarial Networks) to work with tensorflow for Java.
I have added the necessary code to DCGAN’s model.py as below to output a graph to be later used in tensorflow for Java.
//at the beginning to define where the model will be saved
#
self.load_dir = load_dir
self.models_dir = models_dir
graph = tf.Graph()
self.graph = graph
self.graph.as_default()
#
//near the end where the session is ran in order to build and save the model to be used in tensorflow for java. A model is saved every 200 samples as defined by DCGAN’s default settings.
#
steps = "training_steps-" + "{:08d}".format(step)
set_models_dir = os.path.join(self.models_dir, steps)
builder = tf.saved_model.builder.SavedModelBuilder(set_models_dir)
self.builder = builder
self.builder.add_meta_graph_and_variables(self.sess, [tf.saved_model.tag_constants.SERVING])
self.builder.save()
#
The above codes output a graph that is loaded by the following Java code
package Main;
import java.awt.image.BufferedImage;
import java.io.File;
import java.util.Random;
import javax.imageio.ImageIO;
import org.tensorflow.Tensor;
public class DCGAN {
public static void main(String[] args) throws Exception {
String model_dir = "E:\\AgentWeb\\mnist-steps\\training_steps-00050000";
//SavedModelBundle model = SavedModelBundle.load(model_dir , "serve");
//Session sess = model.session();
Random rand = new Random();
int sample_num = 64;
int z_dim = 100;
float [][] gen_random = new float [64][100];
for(int i = 0 ; i < sample_num ; i++) {
for(int j = 0 ; j < z_dim ; j++) {
gen_random[i][j] = (float)rand.nextGaussian();
}
}
Tensor <Float> sample_z = Tensor.<Float>create(gen_random, Float.class);
Tensor <Float> sample_inputs = Tensor.<Float>create(placeholder, Float.class);
// placeholder is the tensor which I want to create after solving the problem below.
//Tensor result = sess.runner().fetch("t_vars").feed("z", sample_z).feed("inputs", sample_inputs).run().get(3);
}
}
(I have left some comments as I used them for debugging)
With this method I am stuck at a certain portion of translating the python code to Java for use in tensorflow for Java. In DCGAN’s model.py where the images are processed there’s the following code.
get_image(sample_file,
input_height=self.input_height,
input_width=self.input_width,
resize_height=self.output_height,
resize_width=self.output_width,
crop=self.crop,
grayscale=self.grayscale) for sample_file in sample_files]
which calls get_iamge in saved_utils.py as follows
def get_image(image_path, input_height, input_width,
resize_height=64, resize_width=64,
crop=True, grayscale=False):
image = imread(image_path, grayscale)
return transform(image, input_height, input_width,
resize_height, resize_width, crop)
which then calls a method called imread as follows
def imread(path, grayscale = False):
if (grayscale):
return scipy.misc.imread(path, flatten = True).astype(np.float)
else:
# Reference: https://github.com/carpedm20/DCGAN-tensorflow/issues/162#issuecomment-315519747
img_bgr = cv2.imread(path)
# Reference: https://stackoverflow.com/a/15074748/
img_rgb = img_bgr[..., ::-1]
return img_rgb.astype(np.float)
My question is that I am unsure what the img_rgb = img_bgr[..., ::-1]
part does and how do I translate it for use in my Java file in tensorflow.java.
I am familiar with the way python slices arrays but I am unfamiliar with the three dots used in there.
I did read about the reference to the stackoverflow questions there and it mentions that it is similar to img[:, :, ::-1]. But I am not really sure about what it is exactly doing.
Any help is appreciated and thank you for taking your time to read this long post.

What basically do the imread and get_image is
1) reads an image
2) convert it from BGR to RGB
3) convert it to floats
4) rescale the image
You can do this in Java either by using an imaging library, such as JMagick or AWT, or by using TensorFlow.
If you use TensorFlow, it is possible to run this preprocessing in eager mode or by building and running a small graph. For example, given tf an instance of org.tensorflow.op.Ops:
tf.image.decode* can read content of an image (you know to know the type of your image though to pick the right operation).
tf.reverse can reverse the value in your channel dimension (RGB to BGR)
tf.dtypes.cast can convert the image to floats
tf.image.resizeBilinear can rescale your image

How to feed boolean placeholder by means of TensorFlowInferenceInterface.java?

I'm trying to launch trained in Keras Tensorflow graph by means of Java Tensorflow API.
Aside from standard input image placeholder, this graph contains 'keras_learning_phase' placeholder that is needed to be fed with a boolean value.
The thing is, there is no method in TensorFlowInferenceInterface for boolean values - you can only feed it with float, double, int or byte values.
Obviously, when I try to pass int to this tensor by means of this code:
inferenceInterface.fillNodeInt("keras_learning_phase",
new int[]{1}, new int[]{0});
I get
tensorflow_inference_jni.cc:207 Error during inference: Internal:
Output 0 of type int32 does not match declared output type bool for
node _recv_keras_learning_phase_0 = _Recvclient_terminated=true,
recv_device="/job:localhost/replica:0/task:0/cpu:0",
send_device="/job:localhost/replica:0/task:0/cpu:0",
send_device_incarnation=4742451733276497694,
tensor_name="keras_learning_phase", tensor_type=DT_BOOL,
_device="/job:localhost/replica:0/task:0/cpu:0"
Is there a way to circumvent it?
Maybe it is possible somehow to explicitly convert Placeholder node in graph to Constant?
Or maybe it is possible to initially avoid creation of this Placeholder in the graph?

The TensorFlowInferenceInterface class essentially is a convenience wrapper over the full TensorFlow Java API, which does support boolean values.
You could perhaps add a method to TensorFlowInferenceInterface to do what you want. Similar to fillNodeInt, you could add the following (note the caveat that booleans in TensorFlow are represented as one byte):
public void fillNodeBool(String inputName, int[] dims, bool[] src) {
byte[] b = new byte[src.length];
for (int i = 0; i < src.length; ++i) {
b[i] = src[i] ? 1 : 0;
}
addFeed(inputName, Tensor.create(DatType.BOOL, mkDims(dims), ByteBuffer.wrap(b)));
}
Hope that helps. If it works, I'd encourage you to contribute back to the TensorFlow codebase.

This is in addition to the answer by ash, as the Tensorflow API has changed a little. Using this worked for me:
public void feed(String inputName, boolean[] src, long... dims) {
byte[] b = new byte[src.length];
for (int i = 0; i < src.length; i++) {
b[i] = src[i] ? (byte) 1 : (byte) 0;
}
addFeed(inputName, Tensor.create(Boolean.class, dims, ByteBuffer.wrap(b)));
}

How do I know that my neural network is being trained correctly

I've written an Adaline Neural Network. Everything that I have compiles, so I know that there isn't a problem with what I've written, but how do I know that I have to algorithm correct? When I try training the network, my computer just says the application is running and it just goes. After about 2 minutes I just stopped it.
Does training normally take this long (I have 10 parameters and 669 observations)?
Do I just need to let it run longer?
Hear is my train method
public void trainNetwork()
{
int good = 0;
//train until all patterns are good.
while(good < trainingData.size())
{
for(int i=0; i< trainingData.size(); i++)
{
this.setInputNodeValues(trainingData.get(i));
adalineNode.run();
if(nodeList.get(nodeList.size()-1).getValue(Constants.NODE_VALUE) != adalineNode.getValue(Constants.NODE_VALUE))
{
adalineNode.learn();
}
else
{
good++;
}
}
}
}
And here is my learn method
public void learn()
{
Double nodeValue = value.get(Constants.NODE_VALUE);
double nodeError = nodeValue * -2.0;
error.put(Constants.NODE_ERROR, nodeError);
BaseLink link;
int count = inLinks.size();
double delta;
for(int i = 0; i < count; i++)
{
link = inLinks.get(i);
Double learningRate = value.get(Constants.LEARNING_RATE);
Double value = inLinks.get(i).getInValue(Constants.NODE_VALUE);
delta = learningRate * value * nodeError;
inLinks.get(i).updateWeight(delta);
}
}
And here is my run method
public void run()
{
double total = 0;
//find out how many input links there are
int count = inLinks.size();
for(int i = 0; i< count-1; i++)
{
//grab a specific link in sequence
BaseLink specificInLink = inLinks.get(i);
Double weightedValue = specificInLink.weightedInValue(Constants.NODE_VALUE);
total += weightedValue;
}
this.setValue(Constants.NODE_VALUE, this.transferFunction(total));
}
These functions are part of a library that I'm writing. I have the entire thing on Github here. Now that everything is written, I just don't know how I should go about actually testing to make sure that I have the training method written correctly.
I asked a similar question a few months ago.

Ten parameters with 669 observations is not a large data set. So there is probably an issue with your algorithm. There are two things you can do that will make debugging your algorithm much easier:
Print the sum of squared errors at the end of each iteration. This will help you determine if the algorithm is converging (at all), stuck at a local minimum, or just very slowly converging.
Test your code on a simple data set. Pick something easy like a two-dimensional input that you know is linearly separable. Will your algorithm learn a simple AND function of two inputs? If so, will it lean an XOR function (2 inputs, 2 hidden nodes, 2 outputs)?

You should be adding debug/test mode messages to watch if the weights are getting saturated and more converged. It is likely that good < trainingData.size() is not happening.
Based on Double nodeValue = value.get(Constants.NODE_VALUE); I assume NODE_VALUE is of type Double ? If that's the case then this line nodeList.get(nodeList.size()-1).getValue(Constants.NODE_VALUE) != adalineNode.getValue(Constants.NODE_VALUE) may not really converge exactly as it is of type double with lot of other parameters involved in obtaining its value and your convergence relies on it. Typically while training a neural network you stop when the convergence is within an acceptable error limit (not a strict equality like you are trying to check).
Hope this helps

Using CameraDevice and CameraSettings javacv

I don't know how this classes work. I am trying to know what cameras are available. Also, if possible, I would like to change settings of the camera (this I think I can figure it out by myself, but an example would be great).
Thanks.

Get the camera list you can simply use
int n = com.googlecode.javacv.cpp.videoInputLib.videoInput.listDevices();
for (int i = 0; i < n; i++) {
String info = com.googlecode.javacv.cpp.videoInputLib.videoInput.getDeviceName(i);
System.out.println(info);
}
you can get the image resolution by calling
grabber.getImageWidth();
grabber.getImageHeight();
also for set can use setImageWidth(), setImageHeight()

What to do with orb feature matches?

I am working in logo detection application in OpenCV on Android. I have a lot of searches and find that for this purpose most of the time feature detection is used.
So I searched and tried different detectors and matchers, and finally I have written a code that works well with ORBFeatureDetector and BruteForce matcher :
private DescriptorMatcher BruteMatcher = DescriptorMatcher.create(DescriptorMatcher.BRUTEFORCE_HAMMING);
I found the number of matches and minimum distance and number of good matches like this :
List<DMatch> matches = mat_of_matches.toList();
double max_dist =0, min_dist= 100;
int row_count = matches.size();
for(int i=0;i<row_count;i++)
{
double dist = matches.get(i).distance;
//System.out.println("dist="+dist);
if(dist<min_dist)min_dist = dist;
if(dist>max_dist)max_dist = dist;
}
// Log.e("Max_dist,Min_dist", "Max="+max_dist+", Min="+min_dist);
List<DMatch> good_matches = new ArrayList<DMatch>();
double good_dist = 3*min_dist;
for(int i =0;i<row_count; i++)
{
if(matches.get(i).distance<good_dist)
{
good_matches.add(matches.get(i));
//Log.e("good_matches", "good_match_id="+matches.get(i).trainIdx);
}
}
, and finally I created a threshold like this :
if(row_count>490&&good_matches.size()<60&&min_dist<12)logo_detected=true;
else logo_detected=false;
The problem is that for many other things that are also in the threshold accessed and the application keeps saying the logo is detected .
I want to know what should I do with detected matched features? Is it the right thing to do (thresholding)? Or do I need to do something else to detect the logo ?
Please help, thanks.