We Keep Coding

Neuroph API Custom Network Not Configured Correctly (Outputs 0)

I'm trying to build a custom neural network the Neuroph API. In the constructor, I've added all my layers and connected them but, even before I train it, it constantly outputs 0 no matter what the inputs are.
The class is below. Am I missing any steps to prepare the network? Am I correctly calculating the output? If it helps, the learning process doesn't do anything either (doesn't change output or error) so I think it's just that I setup the network wrong.
FYI, I would like to avoid out-of-the-box options if I can because this is part of a research project and I would like to have full autonomy over the characteristics of the network.
Also, Neuroph seems to be a fairly lightweight solution and I don't mind it being slow, but if anyone has any other simple java NN solutions, I would happily take suggestions.
imports ...
public class ScoringNetwork extends NeuralNetwork<LMS> implements Serializable {
private static final long serialVersionUID = 2L;
private static final transient int seed = 123456;
private final transient Lock lock = new ReentrantLock(); // For concurrency
private final transient Random random = new Random(seed);
public ScoringNetwork() {
// create input layer
NeuronProperties inputNeuronProperties = new NeuronProperties(InputNeuron.class, Linear.class);
Layer inputLayer = LayerFactory.createLayer(8, inputNeuronProperties);
inputLayer.addNeuron(new BiasNeuron());
this.addLayer(inputLayer);
NeuronProperties hiddenNeuronProperties = new NeuronProperties(TransferFunctionType.RECTIFIED, true);
Layer hiddenLayer = LayerFactory.createLayer(50, hiddenNeuronProperties);
hiddenLayer.addNeuron(new BiasNeuron());
this.addLayer(hiddenLayer);
ConnectionFactory.fullConnect(inputLayer, hiddenLayer);
// Create output layer
NeuronProperties outputNeuronProperties = new NeuronProperties(TransferFunctionType.RECTIFIED, false);
Layer outputLayer = LayerFactory.createLayer(1, outputNeuronProperties);
this.addLayer(outputLayer);
ConnectionFactory.fullConnect(hiddenLayer, outputLayer);
NeuralNetworkFactory.setDefaultIO(this);
this.setLearningRule(new LMS());
this.getLearningRule().setLearningRate(0.1);
this.getLearningRule().setMaxError(0.1);
this.getLearningRule().setErrorFunction(new MeanSquaredError());
}
// My method to set the inputs, calculate the (single) output, then return the output
public double calculateOutputs(/* Custom Input Parameters */) {
lock.lock();
this.setInput(/* Custom Input Parameters into 8 network input parameters of type double */);
this.calculate();
double output = this.getOutput()[0];
System.out.println("output: " + output);
lock.unlock();
return output;
}
}

Fixed by using a Leaky ReLu function to solve the Dying ReLu problem.
More info
I found my problem: I was using a ReLu function everywhere but gave negative weights for half of the random values, resulting in deactivated nodes everywhere. I believe that if the nodes weren't already deactivated, they would get there within one iteration.
I tried fixing my problem by bringing all my weights to positive values but I found that my error value still was stuck at basically a constant value (after the first iteration, which did show a change in error). So that was a bust!
Finally, I fixed the actual problem by switching to the Leaky ReLu function and returned to a distribution of random weights including negatives. It seems that my network fell prey to the Dying ReLu Problem. Now my network runs like a charm!

Getting previous window data in DataFlow

Trying to create some mechanism of alerting system, I am looking to find a drop in an average between two windows.
I was happy to find TrafficRoutes example, specifically when I saw it says:
A 'slowdown' occurs if a supermajority of speeds in a sliding window
are less than the reading of the previous window.
I looked in the code, but failed to understand why this means we get the previous value from the previous window. Since I had no experience with sliding windows till now, I thought I might missing something.
Implementing this kind of mechanism, with or without sliding windows - does not get data from previous windows, as I suspected.
Any idea what do I miss ?
Is there a certain way to get values from previous window ?
I am executing on GCP Dataflow, with SDK 1.9.0.
Please advise,
Shushu

My assumptions:
Your alerting system has data partitioned into "metrics" identified by "metric ids".
The value of a metric at a given time is Double.
You are receiving the metric data as a PCollection<KV<String, Double>> where the String is metric id, the Double is the metric value, and each element has the appropriate implicit timestamp (if it doesn't, you can assign one using the WithTimestamps transform).
You want to compute sliding averages of each metric for each 5-minute interval starting at every 1 minute, and want to do something in case the average for interval starting at T+1min is smaller than average for interval starting at T
You can accomplish it like this:
PCollection<KV<String, Double>> metricValues = ...;
// Collection of (metric, timestamped 5-minute average)
// windowed into the same 5-minute windows as the input,
// where timestamp is assigned as the beginning of the window.
PCollection<KV<String, TimestampedValue<Double>>>
metricSlidingAverages = metricValues
.apply(Window.<KV<String, Double>>into(
SlidingWindows.of(Duration.standardMinutes(5))
.every(Duration.standardMinutes(1))))
.apply(Mean.<String, Double>perKey())
.apply(ParDo.of(new ReifyWindowFn()));
// Rewindow the previous collection into global window so we can
// do cross-window comparisons.
// For each metric, an unsorted list of (timestamp, average) pairs.
PCollection<KV<String, Iterable<TimestampedValue<Double>>>
metricAverageSequences = metricSlidingAverages
.apply(Window.<KV<String, TimestampedValue<Double>>>into(
new GlobalWindows()))
// We need to group the data by key again since the grouping key
// has changed (remember, GBK implicitly groups by key and window)
.apply(GroupByKey.<String, TimestampedValue<Double>>create())
metricAverageSequences.apply(new DetectAnomaliesFn());
...
class ReifyWindowFn extends DoFn<
KV<String, Double>, KV<String, TimestampedValue<Double>>> {
#ProcessElement
public void process(ProcessContext c, BoundedWindow w) {
// This DoFn makes the implicit window of the element be explicit
// and extracts the starting timestamp of the window.
c.output(KV.of(
c.element().getKey(),
TimestampedValue.of(c.element.getValue(), w.minTimestamp())));
}
}
class DetectAnomaliesFn extends DoFn<
KV<String, Iterable<TimestampedValue<Double>>>, Void> {
#ProcessElement
public void process(ProcessContext c) {
String metricId = c.element().getKey();
// Sort the (timestamp, average) pairs by timestamp.
List<TimestampedValue<Double>> averages = Ordering.natural()
.onResultOf(TimestampedValue::getTimestamp)
.sortedCopy(c.element().getValue());
// Scan for anomalies.
for (int i = 1; i < averages.size(); ++i) {
if (averages.get(i).getValue() < averages.get(i-1).getValue()) {
// Detected anomaly! Could do something with it,
// e.g. publish to a third-party system or emit into
// a PCollection.
}
}
}
}
Note that I did not test this code, but it should provide enough conceptual guidance for you to accomplish the task.

Return debug information from Hadoop

I'm writing a Java application to run a MapReduce job on Hadoop. I've set up some local variables in my mapper/reducer classes but I'm not able to return the information to the main Java application. For example, if I set up a variable inside my Mapper class:
private static int nErrors = 0;
Each time I process a line from the input file, I increment the error count if the data is not formatted correctly. Finally, I define a get function for the errors and call this after my job is complete:
public static int GetErrors()
{
return nErrors;
}
But when I print out the errors at the end:
System.out.println("Errors = " + UPMapper.GetErrors());
This always returns "0" no matter what I do! If I start with nErrors = 12;, then the final value is 12. Is it possible to get information from the MapReduce functions like this?
UPDATE
Based on the suggestion from Binary Nerd, I implemented some Hadoop counters:
// Define this enumeration in your main class
public static enum MyStats
{
MAP_GOOD_RECORD,
MAP_BAD_RECORD
}
Then inside the mapper:
if (SomeCheckOnTheInputLine())
{
// This record is good
context.getCounter(MyStats.MAP_GOOD_RECORD).increment(1);
}
else
{
// This record has failed in some way...
context.getCounter(MyStats.MAP_BAD_RECORD).increment(1);
}
Then in the output stream from Hadoop I see:
MAP_BAD_RECORD=11557
MAP_GOOD_RECORD=8676
Great! But the question still stands, how do I get those counter values back into the main Java application?

BigDecimal.valueOf caching mechanism

I heard that the BigDecimal.valueOf() method is better than calling new BigDecimal() because it caches common values. I wanted to know how the caching mechanism of valueOf works.

Looking at the JDK 1.8 sources, it looks like it's just a static array which is initialized as part of class initialization - it only caches the values 0 to 10 inclusive, but that's an implementation detail. For example, given dasblinkenlight's post, it looks like earlier versions only cached 0 and 1.
For more detail - and to make sure you're getting information about the JDK which you're actually running - look at the source of the JDK you're using for yourself - most IDEs will open the relevant source code automatically, if they detect the source archive has been included in your JDK installation. Of course, if you're using a different JRE at execution time, you'd need to validate that too.
It's easy to tell whether or not a value has been cached, based on reference equality. Here's a short but complete program which finds the first non-negative value which isn't cached:
import java.math.BigDecimal;
public class Test {
public static void main(String[] args) {
for (long x = 0; x < Long.MAX_VALUE; x++) {
if (BigDecimal.valueOf(x) != BigDecimal.valueOf(x)) {
System.out.println("Value for " + x + " wasn't cached");
break;
}
}
}
}
On my machine with Java 8, the output is:
Value for 11 wasn't cached
Of course, an implementation could always cache the most recently requested value, in which case the above code would run for a very long time and then finish with no output...

If I m not wrong it caches only zero,one, two and ten, Thats why we have only
public static final BigInteger ZERO = new BigInteger(new int[0], 0);
public static final BigInteger ONE = valueOf(1);
private static final BigInteger TWO = valueOf(2);
public static final BigInteger TEN = valueOf(10);
That also calls valueOf(x) method.

How to use Caliper benchmark beta snapshot without maven?

I have been asked to use Google's Caliper project to create a few microbenchmarks. I would very much like to use the annotation features of the newest beta snapshot, but aside from a few small examples I am having trouble finding good documentation on how to actually run the thing... There is a video tutorial up which instructs users on the new maven integration feature, which I was also asked NOT to use.
Right now I just have a small example stripped from one of theirs, modified with some other information I gleaned from another SO question:
public class Benchmarks {
public class Test {
#Param int size; // set automatically by framework
private int[] array; // set by us, in setUp()
#BeforeExperiment void setUp() {
// #Param values are guaranteed to have been injected by now
array = new int[size];
}
#Benchmark int timeArrayIteration(int reps) {
int dummy = 0;
for (int i = 0; i < reps; i++) {
for (int doNotIgnoreMe : array) {
dummy += doNotIgnoreMe;
}
}
return dummy;
}
}
//(Questionable practice here?)
public static void main (String args[]) {
CaliperMain.main(Test.class, args);
}
}
Running it gives me the message that I did not set a default value for size. I am having trouble tracking down where I should be putting it.
Removing "size" entirely by commenting out the #Param line and giving a hard value to the array declaration in setUp just leads to it deciding that there are "No Experiments to be done," which makes sense, I suppose.
If there are any up-to-date resources or tutorials that could point out what I am doing wrong (probably a whole lot, honestly) I would be very appreciative.
EDIT:
I have updated to this as per some advice:
public class Benchmarks {
#Param({"1", "10", "1000"}) int size; // set automatically by framework
private int[] array; // set by us, in setUp()
#BeforeExperiment void setUp() {
// #Param values are guaranteed to have been injected by now
array = new int[size];
}
#Benchmark int timeArrayIteration(int reps) {
int dummy = 0;
for (int i = 0; i < reps; i++) {
for (int doNotIgnoreMe : array) {
dummy += doNotIgnoreMe;
}
}
return dummy;
}
}
I am running through the beta snapshot and passing in the Benchmarks class as an argument. I receive the following:
Experiment selection:
Instruments: []
User parameters: {size=[1, 10, 1000]}
Virtual machines: [default]
Selection type: Full cartesian product
There were no experiments to be performed for the class Benchmarks using the instruments [allocation, runtime]
It doesn't seem to be detecting any Instruments. I am not passing any in, as it's mentioned in the documentation that it simply uses default allocation, runtime (which is fine for my purposes).
DOUBLE EDIT: Found that problem, stupid mistake. Will do a quick write-up once I confirm it.

Running it gives me the message that I did not set a default value for size.
Parameters are set either from default values:
#Param({"1", "10", "1000"}) int size;
Or by passing values via the the -D flag. E.g.: -Dsize=1,10,1000. Enums and booleans get special treatment in that it uses all possible values without having to list them in the annotation.
Removing "size" entirely by commenting out the #Param line and giving a hard value to the array declaration in setUp just leads to it deciding that there are "No Experiments to be done," which makes sense, I suppose.
The issue is likely that that your benchmark is an inner class and needs a reference to the enclosing class (though this should have been an error). Either make your benchmark class a top-level class (recommended) or make it static.
Also, there is no particular need to include the main method. Invoking CaliperMain with your benchmark class as the first parameter is equivalent.

Running it gives me the message that I did not set a default value for size.
That's very simple:
#Param({"1", "10", "1000"}) int size;
Removing "size" entirely by commenting out the #Param line and giving a hard value to the array declaration in setUp just leads to it deciding that there are "No Experiments to be done," which makes sense, I suppose.
No, it doesn't. Without any params, each benchmark method is to be run exactly once. See the other answer for the solution.
There's a quite some Javadoc, e.g., on #Param. Actually, not so much has changed. Annotations have replaced conventions (now you don't need the time prefix), params stayed the same, setup uses an annotation instead of inheritance.