I'm on OpenCV for java (but that's not relevant I guess). I'm using the BackgroundSubtractorMOG2 class which is (poorly) referenced here. I have read and understood the Zivkovic paper about the algorithm which you can find here.
BackgroundSubtractorMOG2 takes in its constructor a parameter called history. What is it, and how does it influence the result? Could you point me to its reference inside the paper, for example?
From the class source code, line 106, it is said that alpha = 1/history. That would mean that history is namely the T parameter inside the paper, i.e. (more or less) the number of frames that constitute the training set.
However it doesn't seem so. Changing the value in the constructor, from 10 to 500 or beyond, has no effect on the final result. This is what I'm calling:
Mat result = new Mat();
int history = 10; //or 50, or 500, or whatever
BackgroundSubtractorMOG2 sub = new BackgroundSubtractorMOG2(history, 16, false);
for (....) {
sub.apply(frame[i], result);
}
imshow(result); //let's see last frame
It doesn't matter what history I set, be it 5, 10, 500, 1000 - the result is always the same. Whereas, if I change the alpha value (the learning rate) through apply(), I can see its real influence:
Mat result = new Mat();
float alpha = 0.1; //learning rate, 1/T (1/history?)
BackgroundSubtractorMOG2 sub = new BackgroundSubtractorMOG2(whatever, 16, false);
for (...) {
sub.apply(frame[i], result, alpha);
}
imshow(result);
If I change alpha here, result changes a lot, which is understandable. So, two conjectures:
history is not really 1/alpha as the source code states. But then: what is it? how does it affect the algorithm?
history is really 1/alpha, but there's a bug in the java wrapper that makes the history value you set in the constructor useless.
Could you help me?
(Tagging c++ also as this is mainly a question about an OpenCV class and the whole OpenCV java framework is just a wrapper around c++).
It seems clear that alpha = 1 / history (except for some transitory instants). In void BackgroundSubtractorMOG2Impl::apply method:
learningRate = learningRate >= 0 && nframes > 1 ? learningRate : 1./std::min( 2*nframes, history );
You can test if the BackgroundSubtractorMOG2 object is using the history value that you pass in the constructor using the getHistory() method.
Related
I'm trying to do a simple prediction in DL4j (going to use it later for a large dataset with n features) but no matter what I do my network just doesn't want to learn and behaves very weird. Of course I studied all the tutorials and did the same steps shown in dl4j repo, but it doesn't work for me somehow.
For dummy features data I use:
*double[val][x] features; where val = linspace(-10,10)...; and x= Math.sqrt(Math.abs(val)) * val;
my y is : double[y] labels; where y = Math.sin(val) / val
DataSetIterator dataset_train_iter = getTrainingData(x_features, y_outputs_train, batchSize, rnd);
DataSetIterator dataset_test_iter = getTrainingData(x_features_test, y_outputs_test, batchSize, rnd);
// Normalize data, including labels (fitLabel=true)
NormalizerMinMaxScaler normalizer = new NormalizerMinMaxScaler(0, 1);
normalizer.fitLabel(false);
normalizer.fit(dataset_train_iter);
normalizer.fit(dataset_test_iter);
// Use the .transform function only if you are working with a small dataset and no iterator
normalizer.transform(dataset_train_iter.next());
normalizer.transform(dataset_test_iter.next());
dataset_train_iter.setPreProcessor(normalizer);
dataset_test_iter.setPreProcessor(normalizer);
//DataSet setNormal = dataset.next();
//Create the network
MultiLayerConfiguration conf = new NeuralNetConfiguration.Builder()
.seed(seed)
.optimizationAlgo(OptimizationAlgorithm.STOCHASTIC_GRADIENT_DESCENT)
.weightInit(WeightInit.XAVIER)
//.miniBatch(true)
//.l2(1e-4)
//.activation(Activation.TANH)
.updater(new Nesterovs(0.1,0.3))
.list()
.layer(new DenseLayer.Builder().nIn(numInputs).nOut(20).activation(Activation.TANH)
.build())
.layer(new DenseLayer.Builder().nIn(20).nOut(10).activation(Activation.TANH)
.build())
.layer( new DenseLayer.Builder().nIn(10).nOut(6).activation(Activation.TANH)
.build())
.layer(new OutputLayer.Builder(LossFunctions.LossFunction.MSE)
.activation(Activation.IDENTITY)
.nIn(6).nOut(1).build())
.build();
//Train and fit network
final MultiLayerNetwork net = new MultiLayerNetwork(conf);
net.init();
net.setListeners(new ScoreIterationListener(100));
//Train the network on the full data set, and evaluate in periodically
final INDArray[] networkPredictions = new INDArray[nEpochs / plotFrequency];
for (int i = 0; i < nEpochs; i++) {
//in fit we have already Backpropagation. See Release deeplearning
// https://deeplearning4j.konduit.ai/release-notes/1.0.0-beta3
net.fit(dataset_train_iter);
dataset_train_iter.reset();
if((i+1) % plotFrequency == 0) networkPredictions[i/ plotFrequency] = net.output(x_features, false);
}
// evaluate and plot
dataset_test_iter.reset();
dataset_train_iter.reset();
INDArray predicted = net.output(dataset_test_iter, false);
System.out.println("PREDICTED ARRAY " + predicted);
INDArray output_train = net.output(dataset_train_iter, false);
//Revert data back to original values for plotting
// normalizer.revertLabels(predicted);
normalizer.revertLabels(output_train);
normalizer.revertLabels(predicted);
PlotUtil.plot(om, y_outputs_train, networkPredictions);
My output seems then very weird (see picture below), even when I use miniBatch (1, 20,100 Samples/Batch) change number of epochs or add hidden nodes and hidden Layers (tryed to add 1000 Nodes and 5 Layers). The network either outputs very stochastic values or the one constant y. I just can't recognize, what is going wrong here. Why the network even doesn't approach the train function.
Another question: what doesn iter.reset() do exactly. Does the Iterator turn the pointer back to 0-Batch in the DataSetIterator?
A pretty common problem is people doing toy problems like this is dl4j's assumption of minibatches (which 99% of problems tend to be). You aren't actually doing minibatch learning (which actually defeats the point of actually using an iterator, which is meant to iterate through slices of a dataset, not an in memory small dataset) - a small recommendation is to just use the normal dataset api (which is what's returned from dataset.next())
Ensure you turn off the minibatch penalty dl4j assigns to all losses with:
.minibatch(false) - you can see that configuration here:
https://github.com/eclipse/deeplearning4j/blob/master/deeplearning4j/deeplearning4j-nn/src/main/java/org/deeplearning4j/nn/conf/NeuralNetConfiguration.java#L434
A unit test testing this behavior can be found here:
https://github.com/eclipse/deeplearning4j/blob/b4047006ac8175df295c2f3c008e7601437ea4dc/deeplearning4j/deeplearning4j-core/src/test/java/org/deeplearning4j/gradientcheck/GradientCheckTests.java#L94
For posterity, here is the relevant configuration:
MultiLayerConfiguration conf = new NeuralNetConfiguration.Builder().miniBatch(false)
.dataType(DataType.DOUBLE)
.optimizationAlgo(OptimizationAlgorithm.STOCHASTIC_GRADIENT_DESCENT).updater(new NoOp())
.list()
.layer(0,
new DenseLayer.Builder().nIn(4).nOut(3)
.dist(new NormalDistribution(0, 1))
.activation(Activation.TANH)
.build())
.layer(1, new OutputLayer.Builder(LossFunction.MCXENT)
.activation(Activation.SOFTMAX).nIn(3).nOut(3).build())
.build();
You'll notice 2 things: 1 is minibatch is false and 2 is the configuration for data type double. You are also welcome to try that for your problem.
Dl4j to save memory tends to also assume float for the default data type.
This is a reasonable assumption when working on larger problems, but may not work well for toy problems.
For reference, you can find the application of the minibatch math here:
https://github.com/eclipse/deeplearning4j/blob/fc735d30023981ebbb0fafa55ea9520ec44292e0/deeplearning4j/deeplearning4j-nn/src/main/java/org/deeplearning4j/nn/updater/BaseMultiLayerUpdater.java#L332
This affects the gradient updates.
The score penalty can be found in the output layer:
https://github.com/eclipse/deeplearning4j/blob/master/deeplearning4j/deeplearning4j-nn/src/main/java/org/deeplearning4j/nn/layers/BaseOutputLayer.java#L84
Essentially, both of these automatically penalize the loss update for your dataset reflected in both the loss and the gradient updates.
I'm making a basic image editor to improve my image process skills. I have 12 filters(for now).
All filters have a clickable JLabel that has image
I update images of all of them when all filters apply with this function:
public static void buttonImagesUpdater(){
for(int i = 0; i < effects.size(); i++){
effects.get(i).getButton().setImage(new ImageIcon(effects.get(i).process(image)));
}
}
All filters have a process function like that:
public BufferedImage process(BufferedImage base) {
BufferedImage product = new BufferedImage(base.getWidth(), base.getHeight(), base.getType());
for(int indisY = 0; indisY < base.getHeight(); indisY++){
for(int indisX = 0; indisX < base.getWidth(); indisX++){
Color currentColor = new Color(base.getRGB(indisX, indisY));
int greyTone = 0;
greyTone = (int) (currentColor.getRed()*0.315) +
(int) (currentColor.getGreen()*0.215)
+ (int) (currentColor.getBlue()*0.111);
product.setRGB(indisX, indisY, new Color(greyTone,greyTone,greyTone).getRGB());
}
}
return product;
}
Program works so slowly. When I click an effect's button it done 45 second later when I use 5000x3000 image. How can I fix this performance problem?
You have got to remember that 3000 * 5000 is 15,000,000 so you're creating 15,000,000 Color objects, you're calling setRGB 15,000,000 times. If I were you, I would look into potentially using ForkJoinPool for this.
I agree with #Jason - the problem is that you're creating (and destroying) 15 million Color objects.
However, I don't think that just using multiple threads is going to get you enough of a performance increase, because you are still going to be putting a lot of pressure on memory and the garbage collector, since you'll still be creating and destroying 15 million objects, you'll just be doing several in parallel.
I think that you can both stay away from creating Color objects entirely, and make fewer loops, by using the result of the BufferedImage class' getRGB() method directly, instead of creating a Color object. Further, you can use the overload of getRGB() that returns an array of ints, to get, say, a row of pixels (or more) at a time, to reduce the number of calls that you have to make inside the loop. You can similarly use the version of setRGB() that takes an array of pixel values.
The trick is to be able to convert the int color value to a gray value (or whatever else you need to do) without separating the R, G, and B values, or finding an efficient way to separate R, G, and B - more efficient than creating, using, and destroying a Color object.
For a lead on getting R, G, and B values from the int returned by getRGB(), note that the documentation for Color.getRGB() says,
"Returns the RGB value representing the color in the default sRGB
ColorModel. (Bits 24-31 are alpha, 16-23 are red, 8-15 are green, 0-7
are blue)."
Once you have that working, you can think about parallelizing it.
You might try this to see if things speed up a little.
This uses a DataBuffer from an Image Raster.
And uses a map to retain previous converted colors. It may help over a period of time depending on the type of image.
And works with doubles since the data buffer supports various types.
I also multiplied your values by powers of 2 to shift them in the proper position. The get* methods of Color return values between 0 and 255 inclusive. The RGB occupy lower 24 bits on an int (Alpha is in the left most byte).
All I see is a dark image but I tested this with other parameters and I know it works. The long pole in the tent seems to be reading and writing the images. I was using a 6637 3787 image and could read, alter, and write it in 12 seconds. For more advanced processing you may want to check on AffineTransformOp.
static Map<Color, Double> colorMap = new HashMap<>();
public BufferedImage process(BufferedImage base) {
DataBuffer db = base.getRaster().getDataBuffer();
for (int i = 0; i < db.getSize(); i++) {
Color currentColor = new Color(db.getElem(i));
double greyTone = colorMap.computeIfAbsent(currentColor, v->
currentColor.getRed() * .315 *256*256
+ currentColor.getGreen() *.215 * 256
+ currentColor.getBlue()*.115);
db.setElemDouble(i, greyTone);
}
return base;
}
Disclaimer I'm new to Java still learning. I have created a class called CharacterProfiles as part of my textgame to store information about a certain characters. I have created two characters so far (skeleton and zombie) and am trying to put them into an array list so that a random generator can pick either skeleton or zombie everytime the game runs. I'm having trouble getting the computer to pick one of the two characters from inside the Arraylist and print it.
My code so far.
public static void levelOne {
CharacterProfiles skeleton = new CharacterProfiles("Sword", 50, 100, 2, 100);
CharacterProfiles zombie = new CharacterProfiles("Sword", 100, 100, 2, 120);
List<CharacterProfiles> enemies = new ArrayList<>();
enemies.add(skeleton);
enemies.add(zombie);
String randomEnemyGenerator = enemies.get(new Random().nextInt(enemies.size()));
System.out.println(randomEnemyGenerator);
}
The: String randomEnemyGenerator = enemies.get(new Random().nextInt(enemies.size())); I have just gotten from trying to look up solutions on stackOverflow (I tried some other ones I found as well) but nothing seems to work. The error code I get is: Incompatible types Required java.lang.String found Characterprofiles.
Any solutions would be appreciated, thanks!
I'm trying to use JFugue 5.0.9 in my Java project to create *.midi files with it. While implementing JFugue's midi capabilities into my project that uses frequencies of a 24 tune makam piano, I realized it has some tune issues.
For example this code:
ChordProgression cp = new ChordProgression("I-III-IV-iv").setKey("E");
System.out.println(cp);
Player player = new Player();
player.play(cp);
should print
E4MAJ G#4MAJ A4MAJ A4MIN
on console as it's said here.
But prints
E4MAJ E4MAJ E4MAJ A4MIN
in my project. Yes, first three chords are same for some reason. They sound same also, of course.
Also, while using microtones like "m390", it doesn't sound exactly in that frequency.
In another file, in the main method I wrote this:
Player player = new Player();
player.play("A4 m440 m400 m390 m380 m370 m360");
I know that A4 and m440 are the same but as it's said here, A5 and m440 should sound same. But in my project A4 and m440 sound same but it's not exactly 440Hz. When I realized something goes wrong, I decided to use a tuning app and here are the frequencies it calculated respectively:
221.5 221.5 197.5 186.6 186.6 186.6 176.2
As you can see, it plays something very near A3 instead of a clear A4. But this is not all. It also sounds exactly same for m390, m380 and m370.
What exactly is wrong here? Any help would be appreciated.
A little note: I'm pretty sure it has nothing to do with my project. I tryed running the codes above in a brand new project, the same problems occured. And there's no problem with my system because my main project and any other softwares like SunVox, they all sound very good actually.
OK, I had a look at the source codes of JFugue 5.0.9 and here is what I got in ChordProgression.java:
/**
* Only converts Roman numerals I through VII, because that's all we need in music theory...
* VIII would be the octave and equal I!
*/
private int romanNumeralToIndex(String romanNumeral) {
String s = romanNumeral.toLowerCase();
if (s.startsWith("vii")) { return 6; }
else if (s.startsWith("vi")) { return 5; }
else if (s.startsWith("v")) { return 4; }
else if (s.startsWith("iv")) { return 3; }
else if (s.startsWith("iii")) { return 2; }
else if (s.startsWith("ii")) { return 1; }
else if (s.startsWith("i")) { return 0; }
else { return 0; }
}
A little laziness... :D One of the problems lies right here. If you use toLowerCase() method without specifying a locale, it causes some problems in runtime. In Turkish alphabet which I'm using right now lower case of I is "ı", not "i". So, the program converts my chords from "I-II-III" to "i-i-i" because as you can see there is no if statement for lower case I of Turkish alphabet (ı) and this leads it to return 0 as in the case of "i".
To solve this issue, we have to either delete lower case conversion and write all if statements for also upper case I's or set default locale to "en" to make sure it converts (upper case)I's to (lower case)i's. So, this should be used in the source:
Locale.setDefault(new Locale("en"));
Fortunately, JFugue is an open source software published under Apache 2.0.
This still doesn't explain why tunes sound wrong but now at least we know that these are totally different problems. Or maybe it seems so... I don't know. I will edit this answer if I ever find out an explanation or a solution for the rest.
Edit
Finally I figured out the problem with microtones by accident. I decided to look at microtone calculation functions in the source one more time.
In MicrotonePreprocessor class (which is in org.staccato package) there is a function named convertFrequencyToStaccato(). What this function does is convert frequency to midi note number and pitch bend value. At the 107th line, that code rounds semitone, octave and pitch values if calculated pitch value is very close to the next note:
// If we're close enough to the next note, just use the next note.
if (pitches >= 16380)
{
pitches = 0;
semitone += 1;
if (semitone == 12)
{
octave += 1;
semitone = 0;
}
}
The line where pitch is reset should be changed as:
pitches = 8192;
Because, you know, neutral pitch value is 8192. 0 (zero) is minimum pitch value and 16384 is maximum pitch value. At first, I thought the same way as the developer: "After 16384, it should be 0. That is okay. No problem here.". Then I said "What if I change pitch-reset value from 0 to 8192?". It worked. This was a beautiful perception error we both had. :D I'm really having a lot of laugh now.
This fixes the microtone issue. I can hear perfect intervals now! I feel happy and satisfied.
Edit2
I just wanted to share my further changes which results in better microtonal tunning:
if (pitches >= 12288)
{
int diff = 16384-pitches;
int applieddiff = 8192-diff;
pitches = applieddiff;
semitone += 1;
if (semitone == 12)
{
octave += 1;
semitone = 0;
}
}
This also helps use semitone (black) keys on a midi-board instead of only tone (white) keys with high pitch values. So, if you send the data to another software, it will detect all keys seperately. For example, there were no G and G# before, only G and G-with-high-pitch. These caused keys to stop eachother when note-on messages are sent simultaneously.
I am generating my world (random, infinite and 2d) in sections that are x by y, when I reach the end of x a new section is formed. If in section one I have hills, how can I make it so that in section two those hills will continue? Is there some kind of way that I could make this happen?
So it would look something like this
1221
1 = generated land
2 = non generated land that will fill in the two ones
I get this now:
Is there any way to make this flow better?
This seems like just an algorithm issue. Your generation mechanism needs a start point. On the initial call it would be say 0, on subsequent calls it would be the finishing position of the previous "chunk".
If I was doing this, I'd probably make the height of the next point plus of minus say 0-3 from the previous, using some sort of distribution - e.g. 10% of the time it's +/1 3, 25% of the time it is +/- 2, 25% of the time it is 0 and 40% of the time it is +/- 1.
If I understood your problem correctly, here is a solution:
If you generated the delta (difference) between the hills and capped at a fixed value (so changes are never too big), then you can carry over the value of the last hill from the previous section when generating the new one and apply the first randomly genenarted delta (of the new section) to the carried-over hill size.
If you're generating these "hills" sequentially, I would create an accessor method that provides the continuation of said hill with a value to begin the next section. It seems that you are creating a random height for the hill to be constrained by some value already when drawing a hill in a single section. Extend that functionality with this new accessor method.
My take on a possible implementation of this.
public class DrawHillSection {
private int index;
private int x[50];
public void drawHillSection() {
for( int i = 0; i < 50; i++) {
if (i == 0) {
getPreviousHillSectionHeight(index - 1)
}
else {
...
// Your current implementation to create random
// height with some delta-y limit.
...
}
}
}
public void getPreviousHillSectionHeight(int index)
{
return (x[49].height);
}
}