We Keep Coding

Sigmoid function return NaN in Java

I am trying to create a logistic regression algorithm in java but when I calculate the logarithm of the likelihood it is always returning NaN. My method which calculates the logarithm looks like this :
//Calculate log likelihood on given data
private double getLogLikelihood(double cat, double[] x) {
return cat * Math.log(findProbability(x))
+ (1 - cat) * Math.log(1 - findProbability(x));
}
And the findProbability method is just take an instance from the dataset and returning the sigmoid funcion result which is between 0 and 1.
//Calculate the sum of w * x for each weight and attribute
//call the sigmoid function with that s
public double findProbability(double[] x){
double s = 0;
for(int i = 0; i < this.weights.length; i++){
if(i >= x.length) break;
s += this.weights[i] * x[i];
}
return sigmoid(s);
}
private double sigmoid(double s){
return 1 / (1 + Math.exp(-s));
}
Moreover, my starting weights are :
[-0.2982955509135178, -0.4984900460081106, -1.816880187922516, -2.7325608512266073, 0.12542715714800834, 0.1516078084483485, 0.27631147403449774, 0.1371611094778011, 0.16029832096058613, 0.3117065974657231, 0.04262385176091778, 0.1948263133838624, 0.10788353525185314, 0.770608588466501, 0.2697281907888033, 0.09920694325563077, 0.003224073601703939, 0.021573742410541247, 0.21528348692817675, 0.3275511757298476, -0.1500597314893408, -0.7221692528386277, -2.062544912370121, 1.4315146889363015, 0.2522133355419722, 0.23919315019065995, 0.3200037377021523, 0.059466770771758076, 0.04012493980772944, 0.2553236501265919]
Finally, an instance from my dataset is :[M,17.99,10.38,122.8,1001,0.1184,0.2776,0.3001,0.1471,0.2419,0.07871,1.095,0.9053,8.589,153.4,0.006399,0.04904,0.05373,0.01587,0.03003,0.006193,25.38,17.33,184.6,2019,0.1622,0.6656,0.7119,0.2654,0.4601,0.1189]
I tried to initialize the starting weightss with different random numbers but thats didnt solve the problem.

The arithematic is causing a rounding error leaving you with 1.
double b = 1 + Math.exp(-3522);
b will be equal to 1, because otherwise you will need too many sig figs. You'll have to approximate the value to keep the precision. 1/(1+s) ~= 1 - s; Which means you need to calculate log(1) and log(s).
edit: sorry, I made a mistake, it appears Math.exp(-3522) is evaluated as 0 after rounding. Ill leave this answer because Math.exp(-x) might be too small to add to 1, or it might just be zero.

NaN is a result of dividing by zero or calling Math.log on a non-positive number, so u should try and find where exactly this happens. I suggest debugging or adding code to print the values of which u take the logarithm/dividy by.
EDIT: it seems it is a rounding error: exp(-s) will return a result so small that added with 1 it will still remain 1. This causes the logarithm to return -Inf. I'd suggest u try and find a mathematical way to solve this by trying to perhaps to approximate the log of the exponential.

I found a solution to my problem so I post it here:
I added an overflow check:
private double sigmoid(double s){
if(s>20){
s=20;
}else if(s<-20){
s=-20;
}
double exp = Math.exp(s);
return exp/(1+exp);
}
Also changing 1/(1+Math.exp(s) to exp/(1+exp) proved to be more stable in small disturbances of inputs.

Computing Pi in Java

This weeks assignment in programming is to compute Pi in java using this as the basis for the assignment:
(for 80% of the marks):
USING A WHILE OR A DO-WHILE LOOP write a program to compute PI using the following equation:
PI = 3 + 4/(2*3*4) - 4/(4*5*6) + 4/(6*7*8) - 4/(8*9*10) + ...
Allow the user to specify the number of terms (5 terms are shown) to use in the computation.
Each time around the loop only one extra term should be added to the estimate for PI.
(for 20% of the marks):
Alter your solution from part one so that the user is
allowed to specify the precision required between 1 and 8 digits
(i.e. the number of digits which are correct; e.g. to 5 digits PI is 3.14159),
rather than the number of terms. The condition on the loop should be altered so that it
continues until the required precision is obtained. Note that you need only submit this
second version of the program (assuming you have it working).
I'm only able to use the above method to compute Pi, as thats what the lecturer wants. Ive got this so far, although the code keeps giving me the same wrong answer for every even number and a different wrong answer for each odd number. Im still on part one as i havent got the right answer yet to be able to progress onto part 2.
All help would be great, as the program needs to be submitted by tueday.
Thanks in advance!
import java.util.Scanner;
public class ComputePI {
public static void main(String[] args) {
System.out.print( "Please enter the amount of decimal "
+ "digits of PI, you would like to set it too");
Scanner termScan = new Scanner( System.in );
int term = termScan.nextInt();
termScan.close();
double pi = 3.0;
int loopCount = 2;
int number = 2;
while ( loopCount <= term )
{
if (loopCount % 2 == 0)
{
pi = pi + ( 4.0/ ((number) * (number+1) * (number+2)) );
}
else
{
pi = pi - ( 4.0 / ((number) * (number+1) * (number+2)) );
}
number = number + 2;
loopCount++;
}
System.out.print( "The Value of Pi in " + term +
" terms is equal to " + pi);
}
}

I am not going to give you code (you can figure it out for yourself, I'm certain), but I'll give you the location for where to look for the problem.
In the negative terms, you are adding 2 to each number multiplied together. However, you are adding 2 to each number in every iteration of the loop: the numberXXX + 2 part should probably just be numberXXX.
You are now also incrementing the numberXXX variables when loopCount is 1. In fact, the if (loopCount == 1) part is unnecessary, since you already initialize pi. You should just remove the if block there and switch the loopCount % 2 == X blocks around.
I'll also give you general advice about things you might want to consider in your code.
You don't need constants like 4.0 to be in a variable. Just replace fourConstant with 4.0.
You don't need to use an else if for the third block: if loopCount % 2 is not 0 it is definitely 1.
loopCount can only get integer values, so it should probably be an int. A double just consumes extra memory (this is not too problematic here, but may be in large programs) and can in some cases lead to errors (too large numbers may cause rounding errors).
You don't need three variables for numberOne, numberTwo and numberThree; they can always be represented as numberOne, numberOne + 1 and numberOne + 2.

You are incrementing the variables numerOne,numberTwo,numberThree in case the loopCount = 1. In this case you should just continue the loop without incrementing this variables. So change this:
if (loopCount == 1 )
{
pi = 3.0;
}
in:
if (loopCount == 1 )
{
pi = 3.0;
loopCount++;
continue;
}
And change this:
pi = pi - ( fourConstant / ((numberOne+2)*(numberTwo+2)*(numberThree+2)));
into:
pi = pi - ( fourConstant / ((numberOne)*(numberTwo)*(numberThree)));
Or you could just initialize loop count to 2 and remove the first if.
Additionally it would be better is loopCount and term were integer variables instead of Double since they are going to hold only integer values.

Can you replicate the Floor function found in Excel in Java?

I have searched the internet but have not found any solutions for my question.
I would like to be able to use the same/replicate the type of FLOOR function found in Excel in Java. In particular I would like to be able to provide a value (double or preferably BigDecimal) and round down to the nearest multiple of a significance I provide.
Examples 1:
Value = 24,519.30235
Significance = 0.01
Returned Value = 24,519.30
Example 2:
Value = 76.81485697
Significance = 1
Returned Value = 76
Example 3:
Value = 12,457,854
Significance = 100
Returned Value = 12,457,800
I am pretty new to java and was wondering if someone knew if an API already includes the function or if they would be kind enough to give me a solution to the above. I am aware of BigDecimal but I might have missed the correct function.
Many thanks

Yes you can.
Lets say given numbers are
76.21445
and
0.01
what you can do is multiply 76.21445 by 100 (or divide per 0.01)
round the result to nearest or lower integer (depending which one you want)
and than multiply it by the number again.
Note that it may not exactly print what you want if you will not go for the numbers with decimal precision. (The problem of numbers which in the binary format are not finite in extansion). Also in Math you have the round function taking doing pretty much what you want.
http://docs.oracle.com/javase/7/docs/api/java/lang/Math.html you use it like this
round(200.3456, 2);
one Example Code could be
public static void main(String[] args) {
BigDecimal value = new BigDecimal("2.0");
BigDecimal significance = new BigDecimal("0.5");
for (int i = 1; i <= 10; i++) {
System.out.println(value + " --> " + floor(value, significance));
value = value.add(new BigDecimal("0.1"));
}
}
private static double floor(BigDecimal value, BigDecimal significance) {
double result = 0;
if (value != null) {
result = value.divide(significance).doubleValue();
result = Math.floor(result) * significance.doubleValue();
}
return result;
}

To round a BigDecimal, you can use setScale(). In your case, you want RoundingMode.FLOOR.
Now you need to determine the number of digits from the "significance". Use Math.log10(significance) for that. You'll probably have to round the result up.
If the result is negative, then you have a significance < 1. In this case, use setScale(-result, RoundingMode.FLOOR) to round to N digits.
If it's > 1, then use this code:
value
.divide(significance)
.setScale(0, RoundingMode.FLOOR)
.multiply(significance);
i.e. 1024 and 100 gives 10.24 -> 10 -> 1000.

Issue with implementation of Fermat's little therorm

Here's my implementation of Fermat's little theorem. Does anyone know why it's not working?
Here are the rules I'm following:
Let n be the number to test for primality.
Pick any integer a between 2 and n-1.
compute a^n mod n.
check whether a^n = a mod n.
myCode:
int low = 2;
int high = n -1;
Random rand = new Random();
//Pick any integer a between 2 and n-1.
Double a = (double) (rand.nextInt(high-low) + low);
//compute:a^n = a mod n
Double val = Math.pow(a,n) % n;
//check whether a^n = a mod n
if(a.equals(val)){
return "True";
}else{
return "False";
}
This is a list of primes less than 100000. Whenever I input in any of these numbers, instead of getting 'true', I get 'false'.
The First 100,008 Primes
This is the reason why I believe the code isn't working.

In java, a double only has a limited precision of about 15 to 17 digits. This means that while you can compute the value of Math.pow(a,n), for very large numbers, you have no guarantee you'll get an exact result once the value has more than 15 digits.
With large values of a or n, your computation will exceed that limit. For example
Math.pow(3, 67) will have a value of 9.270946314789783e31 which means that any digit after the last 3 is lost. For this reason, after applying the modulo operation, you have no guarantee to get the right result (example).
This means that your code does not actually test what you think it does. This is inherent to the way floating point numbers work and you must change the way you hold your values to solve this problem. You could use long but then you would have problems with overflows (a long cannot hold a value greater than 2^64 - 1 so again, in the case of 3^67 you'd have another problem.
One solution is to use a class designed to hold arbitrary large numbers such as BigInteger which is part of the Java SE API.

As the others have noted, taking the power will quickly overflow. For example, if you are picking a number n to test for primality as small as say, 30, and the random number a is 20, 20^30 = about 10^39 which is something >> 2^90. (I took the ln of 10^39).
You want to use BigInteger, which even has the exact method you want:
public BigInteger modPow(BigInteger exponent, BigInteger m)
"Returns a BigInteger whose value is (this^exponent mod m)"
Also, I don't think that testing a single random number between 2 and n-1 will "prove" anything. You have to loop through all the integers between 2 and n-1.

#evthim Even if you have used the modPow function of the BigInteger class, you cannot get all the prime numbers in the range you selected correctly. To clarify the issue further, you will get all the prime numbers in the range, but some numbers you have are not prime. If you rearrange this code using the BigInteger class. When you try all 64-bit numbers, some non-prime numbers will also write. These numbers are as follows;
341, 561, 645, 1105, 1387, 1729, 1905, 2047, 2465, 2701, 2821, 3277, 4033, 4369, 4371, 4681, 5461, 6601, 7957, 8321, 8481, 8911, 10261, 10585, 11305, 12801, 13741, 13747, 13981, 14491, 15709, 15841, 16705, 18705, 18721, 19951, 23001, 23377, 25761, 29341, ...
https://oeis.org/a001567
161038, 215326, 2568226, 3020626, 7866046, 9115426, 49699666, 143742226, 161292286, 196116194, 209665666, 213388066, 293974066, 336408382, 376366, 666, 566, 566, 666 2001038066, 2138882626, 2952654706, 3220041826, ...
https://oeis.org/a006935
As a solution, make sure that the number you tested is not in this list by getting a list of these numbers from the link below.
http://www.cecm.sfu.ca/Pseudoprimes/index-2-to-64.html
The solution for C # is as follows.
public static bool IsPrime(ulong number)
{
return number == 2
? true
: (BigInterger.ModPow(2, number, number) == 2
? (number & 1 != 0 && BinarySearchInA001567(number) == false)
: false)
}
public static bool BinarySearchInA001567(ulong number)
{
// Is number in list?
// todo: Binary Search in A001567 (https://oeis.org/A001567) below 2 ^ 64
// Only 2.35 Gigabytes as a text file http://www.cecm.sfu.ca/Pseudoprimes/index-2-to-64.html
}

Dealing with overflow in Java without using BigInteger

Suppose I have a method to calculate combinations of r items from n items:
public static long combi(int n, int r) {
if ( r == n) return 1;
long numr = 1;
for(int i=n; i > (n-r); i--) {
numr *=i;
}
return numr/fact(r);
}
public static long fact(int n) {
long rs = 1;
if(n <2) return 1;
for (int i=2; i<=n; i++) {
rs *=i;
}
return rs;
}
As you can see it involves factorial which can easily overflow the result. For example if I have fact(200) for the foctorial method I get zero. The question is why do I get zero?
Secondly how do I deal with overflow in above context? The method should return largest possible number to fit in long if the result is too big instead of returning wrong answer.
One approach (but this could be wrong) is that if the result exceed some large number for example 1,400,000,000 then return remainder of result modulo
1,400,000,001. Can you explain what this means and how can I do that in Java?
Note that I do not guarantee that above methods are accurate for calculating factorial and combinations. Extra bonus if you can find errors and correct them.
Note that I can only use int or long and if it is unavoidable, can also use double. Other data types are not allowed.
I am not sure who marked this question as homework. This is NOT homework. I wish it was homework and i was back to future, young student at university. But I am old with more than 10 years working as programmer. I just want to practice developing highly optimized solutions in Java. In our times at university, Internet did not even exist. Today's students are lucky that they can even post their homework on site like SO.

Use the multiplicative formula, instead of the factorial formula.

Since its homework, I won't want to just give you a solution. However a hint I will give is that instead of calculating two large numbers and dividing the result, try calculating both together. e.g. calculate the numerator until its about to over flow, then calculate the denominator. In this last step you can chose the divide the numerator instead of multiplying the denominator. This stops both values from getting really large when the ratio of the two is relatively small.
I got this result before an overflow was detected.
combi(61,30) = 232714176627630544 which is 2.52% of Long.MAX_VALUE
The only "bug" I found in your code is not having any overflow detection, since you know its likely to be a problem. ;)

To answer your first question (why did you get zero), the values of fact() as computed by modular arithmetic were such that you hit a result with all 64 bits zero! Change your fact code to this:
public static long fact(int n) {
long rs = 1;
if( n <2) return 1;
for (int i=2; i<=n; i++) {
rs *=i;
System.out.println(rs);
}
return rs;
}
Take a look at the outputs! They are very interesting.
Now onto the second question....
It looks like you want to give exact integer (er, long) answers for values of n and r that fit, and throw an exception if they do not. This is a fair exercise.
To do this properly you should not use factorial at all. The trick is to recognize that C(n,r) can be computed incrementally by adding terms. This can be done using recursion with memoization, or by the multiplicative formula mentioned by Stefan Kendall.
As you accumulate the results into a long variable that you will use for your answer, check the value after each addition to see if it goes negative. When it does, throw an exception. If it stays positive, you can safely return your accumulated result as your answer.
To see why this works consider Pascal's triangle
1
1 1
1 2 1
1 3 3 1
1 4 6 4 1
1 5 10 10 5 1
1 6 15 20 15 6 1
which is generated like so:
C(0,0) = 1 (base case)
C(1,0) = 1 (base case)
C(1,1) = 1 (base case)
C(2,0) = 1 (base case)
C(2,1) = C(1,0) + C(1,1) = 2
C(2,2) = 1 (base case)
C(3,0) = 1 (base case)
C(3,1) = C(2,0) + C(2,1) = 3
C(3,2) = C(2,1) + C(2,2) = 3
...
When computing the value of C(n,r) using memoization, store the results of recursive invocations as you encounter them in a suitable structure such as an array or hashmap. Each value is the sum of two smaller numbers. The numbers start small and are always positive. Whenever you compute a new value (let's call it a subterm) you are adding smaller positive numbers. Recall from your computer organization class that whenever you add two modular positive numbers, there is an overflow if and only if the sum is negative. It only takes one overflow in the whole process for you to know that the C(n,r) you are looking for is too large.
This line of argument could be turned into a nice inductive proof, but that might be for another assignment, and perhaps another StackExchange site.
ADDENDUM
Here is a complete application you can run. (I haven't figured out how to get Java to run on codepad and ideone).
/**
* A demo showing how to do combinations using recursion and memoization, while detecting
* results that cannot fit in 64 bits.
*/
public class CombinationExample {
/**
* Returns the number of combinatios of r things out of n total.
*/
public static long combi(int n, int r) {
long[][] cache = new long[n + 1][n + 1];
if (n < 0 || r > n) {
throw new IllegalArgumentException("Nonsense args");
}
return c(n, r, cache);
}
/**
* Recursive helper for combi.
*/
private static long c(int n, int r, long[][] cache) {
if (r == 0 || r == n) {
return cache[n][r] = 1;
} else if (cache[n][r] != 0) {
return cache[n][r];
} else {
cache[n][r] = c(n-1, r-1, cache) + c(n-1, r, cache);
if (cache[n][r] < 0) {
throw new RuntimeException("Woops too big");
}
return cache[n][r];
}
}
/**
* Prints out a few example invocations.
*/
public static void main(String[] args) {
String[] data = ("0,0,3,1,4,4,5,2,10,0,10,10,10,4,9,7,70,8,295,100," +
"34,88,-2,7,9,-1,90,0,90,1,90,2,90,3,90,8,90,24").split(",");
for (int i = 0; i < data.length; i += 2) {
int n = Integer.valueOf(data[i]);
int r = Integer.valueOf(data[i + 1]);
System.out.printf("C(%d,%d) = ", n, r);
try {
System.out.println(combi(n, r));
} catch (Exception e) {
System.out.println(e.getMessage());
}
}
}
}
Hope it is useful. It's just a quick hack so you might want to clean it up a little.... Also note that a good solution would use proper unit testing, although this code does give nice output.

You can use the java.math.BigInteger class to deal with arbitrarily large numbers.

If you make the return type double, it can handle up to fact(170), but you'll lose some precision because of the nature of double (I don't know why you'd need exact precision for such huge numbers).
For input over 170, the result is infinity

Note that java.lang.Long includes constants for the min and max values for a long.
When you add together two signed 2s-complement positive values of a given size, and the result overflows, the result will be negative. Bit-wise, it will be the same bits you would have gotten with a larger representation, only the high-order bit will be truncated away.
Multiplying is a bit more complicated, unfortunately, since you can overflow by more than one bit.
But you can multiply in parts. Basically you break the to multipliers into low and high halves (or more than that, if you already have an "overflowed" value), perform the four possible multiplications between the four halves, then recombine the results. (It's really just like doing decimal multiplication by hand, but each "digit" is, say, 32 bits.)

You can copy the code from java.math.BigInteger to deal with arbitrarily large numbers. Go ahead and plagiarize.